The commit c0e09200dc08 ("drm: reorganise drm tree to be more future
proof.") changed the behavior from only compiling drm/ if CONFIG_DRM=y
to always compiling drm/. This restores the behavior, so people don't
need to waste time compiling stuff they don't need.
Fixes: c0e09200dc08 ("drm: reorganise drm tree to be more future proof.")