On Wed, Sep 1, 2021 at 3:15 AM Tetsuo Handa penguin-kernel@i-love.sakura.ne.jp wrote:
syzbot is reporting page fault at vga16fb_fillrect() [1], for vga16fb_check_var() is failing to detect multiplication overflow.
if (vxres * vyres > maxmem) { vyres = maxmem / vxres; if (vyres < yres) return -ENOMEM; }
Since no module would accept too huge resolutions where multiplication overflow happens, let's reject in the common path.
Link: https://syzkaller.appspot.com/bug?extid=04168c8063cfdde1db5e [1] Reported-by: syzbot syzbot+04168c8063cfdde1db5e@syzkaller.appspotmail.com Debugged-by: Randy Dunlap rdunlap@infradead.org Signed-off-by: Tetsuo Handa penguin-kernel@I-love.SAKURA.ne.jp
Reviewed-by: Geert Uytterhoeven geert+renesas@glider.be
Gr{oetje,eeting}s,
Geert