vmw_fb_kms_framebuffer() declares a drm_mode_fb_cmd2 struct on the stack without zero-ing it and then continues with initializing only some fields.
This leads to drm_mode_fb_cmd2.modifiers[0] containing garbage, which eventually gets used by drm_helper_mode_fill_fb_struct() to set fb->modifier when leads to the following atomic-check failure:
vmwgfx 0000:00:02.0: [drm:drm_atomic_check_only] [PLANE:34:plane-0] invalid pixel format XR24 little-endian (0x34325258), modifier 0xffff94d64719e000 fbcon_init: detected unhandled fb_set_par error, error code -22
Which causes the fbdev emulation and thus also fbcon to not work.
Initialize the struct with all zeros to fix this.
Fixes: dabdcdc9822a ("drm/vmwgfx: Switch to mode_cmd2") BugLink: https://bugzilla.redhat.com/show_bug.cgi?id=2072556 Signed-off-by: Hans de Goede hdegoede@redhat.com --- While working on this I noticed that at least the vmwgfx_fb.c code has a tendency to declare structs on the stack without zero-ing them, instead relying on manually initializing all the fields. Specifically struct drm_mode_set is used multiple times like this and there also is an occurence of struct drm_clip being used like this, but that one is less worrying.
This is a bad idea because some of these structs may get new fields added later. Someone really should do an audit of the entire vmwgfx code for this pattern and make sure all structs declared on the stack are zero-ed before use. --- drivers/gpu/drm/vmwgfx/vmwgfx_fb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c index 8ee34576c7d0..7509013818ba 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fb.c @@ -483,7 +483,7 @@ static int vmw_fb_kms_detach(struct vmw_fb_par *par,
static int vmw_fb_kms_framebuffer(struct fb_info *info) { - struct drm_mode_fb_cmd2 mode_cmd; + struct drm_mode_fb_cmd2 mode_cmd = {}; struct vmw_fb_par *par = info->par; struct fb_var_screeninfo *var = &info->var; struct drm_framebuffer *cur_fb;