During XDC this year, we heard a few presentations and had a lot of hallway talk about sharing code for driving DRM/KMS for display.
I think the general consensus is that there is enough shared functionality between all of the various DRM/KMS clients that we can start thinking about building a library that deals with the complexity of DRM/KMS and can help take full advantage of the hardware.
I've started writing up some ideas, mostly just goals, but also some random thoughts about how this might work. This is all in the 'liboutput' project on freedesktop.org:
https://gitlab.freedesktop.org/xorg/liboutput
I picked a spot in the 'xorg' namespace, not to confine this to X systems, but to reflect the goal that this be a library shared across all x.org-related projects, including Wayland compositors, EGL and Vulkan direct applications and the X server.
All there is at this point is a README.md file; I'd like to generate some consensus on a direction for the project before we start writing any actual design documents.
Please submit MRs to that repo, or respond to this email as you prefer.
Hey Keith,
Thanks for setting this up, and picking a descriptive project name :)
How is liboutput going to relate to the libliftoff[1] project?
[1] https://github.com/emersion/libliftoff
Rob.
On 05.10.19 18:09, Keith Packard wrote:
During XDC this year, we heard a few presentations and had a lot of hallway talk about sharing code for driving DRM/KMS for display.
I think the general consensus is that there is enough shared functionality between all of the various DRM/KMS clients that we can start thinking about building a library that deals with the complexity of DRM/KMS and can help take full advantage of the hardware.
I've started writing up some ideas, mostly just goals, but also some random thoughts about how this might work. This is all in the 'liboutput' project on freedesktop.org:
https://gitlab.freedesktop.org/xorg/liboutputI picked a spot in the 'xorg' namespace, not to confine this to X systems, but to reflect the goal that this be a library shared across all x.org-related projects, including Wayland compositors, EGL and Vulkan direct applications and the X server.
All there is at this point is a README.md file; I'd like to generate some consensus on a direction for the project before we start writing any actual design documents.
Please submit MRs to that repo, or respond to this email as you prefer.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Robert Foss robert.foss@collabora.com writes:
Hey Keith,
Thanks for setting this up, and picking a descriptive project name :)
How is liboutput going to relate to the libliftoff[1] project?
We heard about libliftoff at XDC. It sounds like it does the plane selection stuff, including searching for configurations that support 'most' of what the user asked for while passing back a list of 'uh, these didn't fit'. That sounds like a good foundation for liboutput.
I think we want a bit more, including things like render-based operations for compositing fallback, format conversion, transforms, PRIME support etc. I think we might also want to be able to create pseudo outputs (to memory only) for things like virtualization and testing.
Mostly, I heard lots of ideas at XDC about more we might do to share code between DRM/KMS users, especially compositors.
I'd love to keep the conversation going and see if we can generate a solid set of shared goals, then put together some kind of architecture and bits of code to see what might work.
Hi Keith,
On Sat, 5 Oct 2019 at 17:16, Keith Packard keithp@keithp.com wrote:
During XDC this year, we heard a few presentations and had a lot of hallway talk about sharing code for driving DRM/KMS for display.
Definitely. That would be great.
I think the general consensus is that there is enough shared functionality between all of the various DRM/KMS clients that we can start thinking about building a library that deals with the complexity of DRM/KMS and can help take full advantage of the hardware.
Yeeeessss. Ish.
There are definitely a few annoying problems which we should have common resolution for. I'm thinking of: - libliftoff's mandate of solving the views -> planes conundrum; it's a ton of work to do properly, is extremely easy to get wrong, and will ultimately require platform-specific tweaks as well - similarly, working backwards from 'I want to light these connectors up' to determine the plane -> CRTC -> connector routing, which also requires a mixture of brute force and platform-specific tweaks - currently this is mostly just to handle asymmetric CRTC capabilities (e.g. max clock), but is being complicated by the requirement for merged CRTCs - EDID/CEA/etc parsing, which is not only tedious and sucks, but requires a quirks database - sharing a mode-selection description language and semantics might also be nice - parsing DRM properties - particularly enums - is painful enough that Weston wrote its own internal wrapper around it - TTY handling :(
Having those as helper libraries would solve a _lot_ of issues Weston has today. But pushing much more than that - thinking of rendering and transforms in particular - is going to be actively harmful to us. Given that we want to work on not just KMS, but nested-Wayland, nested-X11, remote protocols like RDP/Waltham, and even mixed local/remote setups where some outputs are KMS and others are at the end of an RTSP stream, I'm going to need to implement most of this stuff anyway.
Delegating everything to a big does-everything-KMS library means that I'll just have to do it twice, and hope there isn't too much impedance mismatch between the results. I also worry about both colour management and timing: the users are going to have radically different wants and requirements for those, and a one-size-fits-all API seems like it would be tough to deal with.
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above. Most of them have good prior art: libliftoff is working from Weston's plane-assignment code, Mutter has pretty good CRTC <-> connector routing IIRC, EDID parsing is best in the X server, DRM properties are probably best done in either Weston or kmscube (depending on the enum vs. string efficiency tradeoff), and TTY handling is probably most robust in the X server.
If I was starting a compositor from scratch then I'd probably think pretty hard about a does-everything-for-you library, but with libweston already existing, it's a much harder sell tbh.
Cheers, Daniel
Hi,
On Mon, 7 Oct 2019 at 18:35, Daniel Stone daniel@fooishbar.org wrote:
There are definitely a few annoying problems which we should have common resolution for. I'm thinking of:
- [...]
Oh, and add backlight handling to that list.
Cheers, Daniel
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
I definitely like the idea of stealing the best parts of all existing systems and trying to make them work together.
How many libraries we end up with isn't nearly as important to me as making sure they work well together; common data types, similar style, etc.
On Mon, 7 Oct 2019 at 19:16, Keith Packard keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
I definitely like the idea of stealing the best parts of all existing systems and trying to make them work together.
How many libraries we end up with isn't nearly as important to me as making sure they work well together; common data types, similar style, etc.
Yeah, totally AOL.
Cheers, Daniel
On Mon, 07 Oct 2019 14:16:32 -0400 "Keith Packard" keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
I definitely like the idea of stealing the best parts of all existing systems and trying to make them work together.
How many libraries we end up with isn't nearly as important to me as making sure they work well together; common data types, similar style, etc.
Hi,
a bunch of helpers is the more attractive idea to me too. We just have to pay attention to API/ABI instability vs. usage: we should be able to both redesign the API at will to go forward, but people will not be happy using a library that breaks ABI all the time.
Libweston worked around that issue by making all major release versions parallel-installable. That means literally all files or directories have the major number in their base name. Another option would be to have a static library only. I'm not sure which one would be more attractive. If going with static-library-only, then we probably one day want to convert into a dynamic library, and then the question of whether the ABI is stable enough will arise again.
Using Meson should make the static-lib-only approach very easy, downstream projects could include liboutput as a git sub-module or such. That should eliminate the version dependency issues that builds might otherwise have.
Thanks, pq
Hi,
On Mon, 7 Oct 2019 at 19:16, Keith Packard keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
To elaborate a little bit, one of the reasons I'm loath to hide complexity like transforms, colour management, and timing away in an encapsulated lower layer, is because I have to expose all those details anyway. Ultimately to make those work properly, we'll require awareness not just in the compositor itself, but pushed through to clients.
Wayland already has facility for informing clients about output transforms so they can render pre-rotated and avoid the compositor-side transform; in order to make HDR and other colour management (e.g. just simple calibration) properly we need to have full plumbing back through to clients; doing timing properly, particularly for multiple simultaneous clients, also requires a fair bit of mechanics and back-and-forth.
There's a lot that we could usefully share between all the users, and having a shared library to help with that would be great. But the thought of tucking it all away in an opaque layer which (*waves hands*) just does it, gives me cold EGLStreams sweats.
Maybe a good place to start is if we all listed the bits of code which we'd be delighted to jettison?
Cheers, Daniel
On Tue, Oct 8, 2019 at 5:32 AM Daniel Stone daniel@fooishbar.org wrote:
Hi,
On Mon, 7 Oct 2019 at 19:16, Keith Packard keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
To elaborate a little bit, one of the reasons I'm loath to hide complexity like transforms, colour management, and timing away in an encapsulated lower layer, is because I have to expose all those details anyway. Ultimately to make those work properly, we'll require awareness not just in the compositor itself, but pushed through to clients.
Wayland already has facility for informing clients about output transforms so they can render pre-rotated and avoid the compositor-side transform; in order to make HDR and other colour management (e.g. just simple calibration) properly we need to have full plumbing back through to clients; doing timing properly, particularly for multiple simultaneous clients, also requires a fair bit of mechanics and back-and-forth.
There's a lot that we could usefully share between all the users, and having a shared library to help with that would be great. But the thought of tucking it all away in an opaque layer which (*waves hands*) just does it, gives me cold EGLStreams sweats.
Maybe a good place to start is if we all listed the bits of code which we'd be delighted to jettison?
On the flipside, it would be nice to have one set of code to do modesets from userspace. Certainly simplifies QA since in theory we should be seeing the same sequences from all apps using the helpers rather than every app rolling their own and getting it subtly different enough that strange things happen with different compositors.
While we are on the topic, it would be nice to have a central place for robustness/context lost handling for compositors so we can properly handle GPU resets. Not sure that is really possible though.
Alex
On Tue, Oct 8, 2019 at 4:36 PM Alex Deucher alexdeucher@gmail.com wrote:
On Tue, Oct 8, 2019 at 5:32 AM Daniel Stone daniel@fooishbar.org wrote:
Hi,
On Mon, 7 Oct 2019 at 19:16, Keith Packard keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
To elaborate a little bit, one of the reasons I'm loath to hide complexity like transforms, colour management, and timing away in an encapsulated lower layer, is because I have to expose all those details anyway. Ultimately to make those work properly, we'll require awareness not just in the compositor itself, but pushed through to clients.
Wayland already has facility for informing clients about output transforms so they can render pre-rotated and avoid the compositor-side transform; in order to make HDR and other colour management (e.g. just simple calibration) properly we need to have full plumbing back through to clients; doing timing properly, particularly for multiple simultaneous clients, also requires a fair bit of mechanics and back-and-forth.
There's a lot that we could usefully share between all the users, and having a shared library to help with that would be great. But the thought of tucking it all away in an opaque layer which (*waves hands*) just does it, gives me cold EGLStreams sweats.
Maybe a good place to start is if we all listed the bits of code which we'd be delighted to jettison?
On the flipside, it would be nice to have one set of code to do modesets from userspace. Certainly simplifies QA since in theory we should be seeing the same sequences from all apps using the helpers rather than every app rolling their own and getting it subtly different enough that strange things happen with different compositors.
For atomic drivers, and atomic userspace this really doesn't matter anymore. Because userspace just assembles an overall update, and the kernel applies that update no matter in which order userspace listed the property/value/object triples.
It does somewhat matter for legacy kms (but not from a driver validation pov, anything legacy can do can also be done as a legit request through atomic), but I really can't care about that one much. Imo better to just cut everyone over to atomic. The only thing that might make sense is the fake atomic support, where userspace tries to not be too dumb about a transition on drivers without atomic (i.e. shut down crtcs first, then light up the new ones, to avoid in-between state that oversubscribes available hw resources). But that's really a niche thing.
While we are on the topic, it would be nice to have a central place for robustness/context lost handling for compositors so we can properly handle GPU resets. Not sure that is really possible though.
I think this blows up the scope too much and will break the project. In the past there's been attempts at a libcompositor (and you need that much to recover from context loss), and that never got anywhere at all. I think the modular helpers approach has a much bigger chance at success. -Daniel
On Mon, 7 Oct 2019 at 20:16, Keith Packard keithp@keithp.com wrote:
Daniel Stone daniel@fooishbar.org writes:
I think there would be a load of value in starting with simple helpers which can be used independently of any larger scheme, tackling that list above.
Yeah, a helper library that didn't enforce at tonne of policy and just let the user glue things together on their own is probably going to be more generally usable by existing and new systems.
I really like that idea. For product development where the compositor itself might not be open-source, it would also really help companies behind these products to better work with the open-source community, i.e. being able to contribute to a set of shared helper libraries.
- Heinrich
Hi,
On 05/10/2019 18:09, Keith Packard wrote:
During XDC this year, we heard a few presentations and had a lot of hallway talk about sharing code for driving DRM/KMS for display.
I think the general consensus is that there is enough shared functionality between all of the various DRM/KMS clients that we can start thinking about building a library that deals with the complexity of DRM/KMS and can help take full advantage of the hardware.
Seeing the description, it seems to be a libdrm with steroids, why libdrm doesn't handle all this already ?
Is there a plan to maybe use it as a foundation for projects like wlroots or drm_hwcomposer for example ?
Neil
I've started writing up some ideas, mostly just goals, but also some random thoughts about how this might work. This is all in the 'liboutput' project on freedesktop.org:
https://gitlab.freedesktop.org/xorg/liboutputI picked a spot in the 'xorg' namespace, not to confine this to X systems, but to reflect the goal that this be a library shared across all x.org-related projects, including Wayland compositors, EGL and Vulkan direct applications and the X server.
All there is at this point is a README.md file; I'd like to generate some consensus on a direction for the project before we start writing any actual design documents.
Please submit MRs to that repo, or respond to this email as you prefer.
dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel
Neil Armstrong narmstrong@baylibre.com writes:
Seeing the description, it seems to be a libdrm with steroids, why libdrm doesn't handle all this already ?
That'd be a lot of steroids; we're talking about creating helper functions all the way up to rendering images into scanout buffers (presumably using Vulkan?) for format conversion or flattening.
Is there a plan to maybe use it as a foundation for projects like wlroots or drm_hwcomposer for example ?
Yes, the goal is to start to share code across a wide range of DRM users, instead of having everyone roll their own.
dri-devel@lists.freedesktop.org
-
Alex Deucher -
Daniel Stone -
Daniel Vetter -
Heinrich Fink -
Keith Packard -
keithp＠keithp.com
-
Neil Armstrong -
Pekka Paalanen -
Robert Foss