With many more non-desktop form factor devices landing in the kernel, we're starting to run up against some limitations. Notably devices with display notches, cutouts and rounded corners.
Given that the DRI subsystem already deals with physical display properties like panel orientation which is fixed in device tree, it would make sense to also define other properties like parts of the display which are obscured there too. This can then be read by desktop environments and UI elements can be suitably adjusted.
Android already deals with non-square displays, however they also keep the device configuration in userspace: https://developer.android.com/guide/topics/display-cutout
A solution to make this configuration generic and exposed by the kernel would standardise this across Linux and potentially Android too which would be extremely beneficial to the mobile Linux community at large.
Does this seem like a feasible solution to solving the notch problem and avoiding a situation where every compositor have their own method of describing cutouts and rounded corners?
Regards,
Caleb
A solution to make this configuration generic and exposed by the kernel would standardise this across Linux
Having a KMS property for this makes sense to me.
Chatting with Jani on IRC, it doesn't seem like there's any EDID or DisplayID block for this.
Note, Android exposes a data structure [1] with:
- Margin of the cut-out for each edge of the screen - One rectangle per edge describing the cut-out region - Size of the curved area for each edge of a waterfall display
I haven't found anything describing the rounded corners of the display.
[1]: https://developer.android.com/reference/android/view/DisplayCutout
On Wed, 28 Apr 2021 07:21:28 +0000 Simon Ser contact@emersion.fr wrote:
A solution to make this configuration generic and exposed by the kernel would standardise this across Linux
Having a KMS property for this makes sense to me.
Chatting with Jani on IRC, it doesn't seem like there's any EDID or DisplayID block for this.
Note, Android exposes a data structure [1] with:
- Margin of the cut-out for each edge of the screen
- One rectangle per edge describing the cut-out region
- Size of the curved area for each edge of a waterfall display
I haven't found anything describing the rounded corners of the display.
Hi,
I'm kind of worried whether you can design a description structure that would be good for a long time. That list already looks quite complicated. Add also watch-like devices with circular displays.
Would the kernel itself use this information at all?
If not, is there not a policy that DT is not a userspace configuration store?
You mentioned the panel orientation property, but that is used by the kernel for fbcon or something, is it not? Maybe as the default value for the CRTC rotation property which actually turns the image?
Assuming that you succeed in describing these non-usable, funny (waterfall edge), funny2 (e.g. behind a shade or filter so visible but not normal), funny3 (e.g. phone button area with maybe tactile markings), and normal areas, how would userspace handle this information?
Funny2 and funny3 are hypothetical but maybe not too far-fetched.
Is there any provision for generic userspace to handle this generically?
This seems more like a job for the hypothetical liboutput, just like recognising HMDs (yes, I know, kernel does that already, but there is a point that kernel may not want to put fbcon on a HMD).
Thanks, pq
On Wednesday, April 28th, 2021 at 9:44 AM, Pekka Paalanen ppaalanen@gmail.com wrote:
I'm kind of worried whether you can design a description structure that would be good for a long time. That list already looks quite complicated. Add also watch-like devices with circular displays.
Would the kernel itself use this information at all?
fbcon might want to letter-box its output to make sure it's not obscured behind a cut-out area.
If not, is there not a policy that DT is not a userspace configuration store?
You mentioned the panel orientation property, but that is used by the kernel for fbcon or something, is it not? Maybe as the default value for the CRTC rotation property which actually turns the image?
I wonder if fbcon uses it at all. In general CRTC rotation is not well-supported by HW drivers, at least for linear buffers. CRTC rotation is just an optimization.
Assuming that you succeed in describing these non-usable, funny (waterfall edge), funny2 (e.g. behind a shade or filter so visible but not normal), funny3 (e.g. phone button area with maybe tactile markings), and normal areas, how would userspace handle this information?
Funny2 and funny3 are hypothetical but maybe not too far-fetched.
Is there any provision for generic userspace to handle this generically?
I think the main use-case here is make sure there's nothing important being cut out on screen. I agree we still don't know how the hw will evolve and might design an API which is too restricted. But building something that ends up too complicated and too generic wouldn't be great either.
On Wed, Apr 28, 2021 at 10:44:03AM +0300, Pekka Paalanen wrote:
On Wed, 28 Apr 2021 07:21:28 +0000 Simon Ser contact@emersion.fr wrote:
A solution to make this configuration generic and exposed by the kernel would standardise this across Linux
Having a KMS property for this makes sense to me.
Chatting with Jani on IRC, it doesn't seem like there's any EDID or DisplayID block for this.
Note, Android exposes a data structure [1] with:
- Margin of the cut-out for each edge of the screen
- One rectangle per edge describing the cut-out region
- Size of the curved area for each edge of a waterfall display
I haven't found anything describing the rounded corners of the display.
Hi,
I'm kind of worried whether you can design a description structure that would be good for a long time. That list already looks quite complicated. Add also watch-like devices with circular displays.
Would the kernel itself use this information at all?
If not, is there not a policy that DT is not a userspace configuration store?
If someone is sufficiently bored it would make sense to teach fbcon (but not fbdev I guess for full sized boot splash) to avoid the edges/corners for output.
But also fbcon/fbdev is I think finally dead on Android, so the intersection of people who care about cut-outs and fbcon is likely 0.
Otherwise I can't think of anything.
You mentioned the panel orientation property, but that is used by the kernel for fbcon or something, is it not? Maybe as the default value for the CRTC rotation property which actually turns the image?
Assuming that you succeed in describing these non-usable, funny (waterfall edge), funny2 (e.g. behind a shade or filter so visible but not normal), funny3 (e.g. phone button area with maybe tactile markings), and normal areas, how would userspace handle this information?
Funny2 and funny3 are hypothetical but maybe not too far-fetched.
Is there any provision for generic userspace to handle this generically?
This seems more like a job for the hypothetical liboutput, just like recognising HMDs (yes, I know, kernel does that already, but there is a point that kernel may not want to put fbcon on a HMD).
I think the desktop linux solution would be hwdb entries, except we've never done this for anything display related. So yeah liboutput sounds about right for this :-)
Btw on fbcon on HMD, I thought we're already taking care of that? -Daniel
On Wed, 28 Apr 2021, Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Apr 28, 2021 at 10:44:03AM +0300, Pekka Paalanen wrote:
This seems more like a job for the hypothetical liboutput, just like recognising HMDs (yes, I know, kernel does that already, but there is a point that kernel may not want to put fbcon on a HMD).
I think the desktop linux solution would be hwdb entries, except we've never done this for anything display related. So yeah liboutput sounds about right for this :-)
Btw on fbcon on HMD, I thought we're already taking care of that?
This is a bit off-topic, but DisplayID 2.0 defines primary use cases for head mounted VR and AR, so we wouldn't have to quirk them.
BR, Jani.
On 28/04/2021 8:21 am, Simon Ser wrote:
A solution to make this configuration generic and exposed by the kernel would standardise this across Linux
Having a KMS property for this makes sense to me.
Chatting with Jani on IRC, it doesn't seem like there's any EDID or DisplayID block for this.
Note, Android exposes a data structure [1] with:
- Margin of the cut-out for each edge of the screen
- One rectangle per edge describing the cut-out region
- Size of the curved area for each edge of a waterfall display
I haven't found anything describing the rounded corners of the display.
It looks like rounded corners only get a developer facing API in Android 12 [1].
However from a vendor perspective it's possible to set padding for rounded corners [2], although this seems to mostly be a hack.
It seems like it would make sense to deal with cutouts and rounded corners separately, cutouts are always convex where rounded corners are always concave, they have different implications on how content should be adjusted.
[1]: https://developer.android.com/about/versions/12/features#rounded_corner_apis
[2]: https://github.com/LineageOS/android_device_oneplus_enchilada/commit/923c86a...
dri-devel@lists.freedesktop.org
-
Caleb Connolly -
Daniel Vetter -
Jani Nikula -
Pekka Paalanen -
Simon Ser