Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
I am a bit disappointed with this being considered a desirable way of handling that transfer of control over a shared DRM device as it shows a lack of ambition. Sure, it's probably easy to implement, but it will also greatly limit how such transitions can be presented to the user. In practice it would mean plymouthd closing the DRM device and exiting so that systemd can start the display manager which then starts an X server to present the login screen. If for that several shared libraries have to first be loaded and relocated while the system is under heavy load then there will be a noticeable delay manifesting as a frozen screen. After that the best you can hope for is blending the still-frame over into the login screen (or whatever comes then). The VT-API-based switching mechanism currently en vogue suffers from similar limitations.
If the approach to transferring control were to be changed to a scheme that involves both donor and recipient process connecting to each other on a unix socket and actively coordinating the transfer (i.e. the calls to drmSetMaster and drmDropMaster) then this would open the door to a host of possibilities. Not only could the transition be kept infinitesimally short since both processes are already up, but it could also involve e.g. the recipient continuing an animation the donor had going reusing state that is transferred as a memfd. This way there wouldn't be any noticeable freezes on the display making for a far more polished, and thus impressive experience. It would be a feat a program alone cannot achieve on its own. Another option made possible would be implementing a watchdog. If the recipient transfers e.g. file descriptors for a pipe and a pidfd of itself, then the donor could monitor those for a heartbeat/process termination and take back control over the device if something goes awry (deadlock/crash) and initiate a recovery mechanism. With the other approach implementing such features is simply not possible.
Making processes talk to each other and work together like this would also be a far more accurate software representation of what is actually going on: different subsystems passing control over a shared device around to work towards the common goal of a good user experience.
A bit of context: The idea underlying this came from my experience with accessibility technology under Linux where uncoordinated fighting over the audio device among all kinds of processes led to countless ways in which things would break with no hope of ever fixing anything. It instilled in me the conviction that user-facing programs are broken if they are not written to talk to each other to coordinate access to shared resources for the goal of rendering a good user experience, but instead leave it to the distro maintainer/user to set things up into a static, brittle working order. Seeing a much-needed cultural shift begin somewhere would be nice. The Plymouth->X transition would lend itself well as a starting point since many building blocks are already there.
Regards
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
- Xorg doing something (e.g. resetting instead of inheriting KMS state)
- Something missed in the hand-off sequence which allows fbcon to momentarily take over between Plymouth and Xorg. This would need to be fixed between Plymouth and Xorg.
- Maybe systemd-logind does something odd to the KMS device? It has pretty wild code there. Or maybe it causes fbcon to take over.
What is the new ioctl you referred to?
Being able to be started with an open DRM file descriptor is not necessary for a smooth hand-over as far as I know. There are tons of other details that are, though.
I am a bit disappointed with this being considered a desirable way of handling that transfer of control over a shared DRM device as it shows a lack of ambition. Sure, it's probably easy to implement, but it
Or more likely bigger fires and lack of time, like with everything.
will also greatly limit how such transitions can be presented to the user. In practice it would mean plymouthd closing the DRM device and exiting so that systemd can start the display manager which then starts an X server to present the login screen. If for that several shared libraries have to first be loaded and relocated while the system is under heavy load then there will be a noticeable delay manifesting as a frozen screen. After that the best you can hope for is blending the still-frame over into the login screen (or whatever comes then). The VT-API-based switching mechanism currently en vogue suffers from similar limitations.
All that is already solvable purely in userspace in my opinion, today. It's just a big project over several independent userspace software projects.
If the approach to transferring control were to be changed to a scheme that involves both donor and recipient process connecting to each other on a unix socket and actively coordinating the transfer (i.e. the calls to drmSetMaster and drmDropMaster) then this would open the door to a host of possibilities. Not only could the transition be kept infinitesimally short since both processes are already up, but it could also involve e.g. the recipient continuing an animation the donor had going reusing state that is transferred as a memfd. This way there wouldn't be any noticeable freezes on the display making for a far more polished, and thus impressive experience. It would be a feat a program alone cannot achieve on its own. Another option made possible would be implementing a watchdog. If the recipient transfers e.g. file descriptors for a pipe and a pidfd of itself, then the donor could monitor those for a heartbeat/process termination and take back control over the device if something goes awry (deadlock/crash) and initiate a recovery mechanism. With the other approach implementing such features is simply not possible.
Nothing in the kernel stops userspace developers from doing exactly that. Seems like you would be working on Plymouth and a display server of your choice. Don't forget to count in systemd-logind as well, since that is a popular component for managing sessions and is involved with e.g. drmSetMaster.
It's a good goal.
It's also more or less necessary for a smooth hand-over of a KMS device between any two processes also for other reasons I've discussed in the past with DRM developers. This is the topic about any KMS client (a program using KMS) needing to reach a guaranteed "clean" KMS state to display correctly.
The kernel DRM subsystem never resets KMS state in any way, apart from driver initialisation.
This means that when a new KMS client takes over, the KMS state could be anything, whatever the previous KMS client left in. This is a problem, because the KMS client may not know how to reset all the KMS properties to clean, sane defaults. Currently there is also no reset ioctl in the kernel either, and no userspace space solution for storing a sane default state. The problem arises from KMS properties: each KMS client may not know how to program all the KMS properties the kernel supports on the device. For example, if one KMS client leaves the output in HDR mode, and the next KMS client does not understand the HDR property, then quite likely the latter KMS client will display an awful image without knowing it.
There is also the convention of a KMS client not restoring the inherited KMS state on exit or switch-out, because that could cause unnecessary flicker on screen and delays. This amplifies the above problem.
The only time the KMS state is at "sane" defaults is right after driver initialisation. Presumably. So only the first KMS client after a reboot can expect a sane KMS state.
Mind, also fbcon is a KMS client, it's just a kernel-internal one. Letting fbcon take over momentarily can reset some KMS properties to nice defaults, but not all, because when new KMS properties are added, no-one usually remembers to patch fbcon/fbdev/whatever to reset it when fbcon takes over. Switching temporarily to fbcon also causes flicker and delays.
The above problems could perhaps be solved if there was a generic KMS hand-over protocol in userspace. The two KMS clients could agree on which KMS properties should be reset and by whom. Who understands and programs which KMS properties. And perhaps, some system component in early'ish boot could save the driver-initial KMS state which is presumably the good default for any use case, on the properties any particular KMS client does not know or bother to program by itself.
So yes, userspace protocol for KMS hand-off would be very welcome. But who would have the time to develop it, when so far we can just limp forward with the current undocumented conventions.
Such protocol, if widely used, might also make it unnecessary for KMS clients to save and restore KMS properties they do not understand when they switch out and later back in. When a KMS client has released DRM master on the device, some other KMS client could have "messed up" the KMS state, so restoring to what you used before is necessary. I don't think anyone actually implements this save/restore yet for unknown KMS properties, and it would be much easier to implement than the hand-off protocol. Maybe switching is either not done, or it is always done to/from a display manager process which sanitises the KMS state or enforces that KMS clients do not leave random state behind. Or maybe most KMS clients are just really good at agreeing on which KMS properties everyone will use. If some stale KMS property causes a problem on some KMS client (display server), it is pretty easy to just add support for programming that property in the KMS client. Problem solved and no hand-off protocol needed. Then the next KMS client hits and does the same.
Making processes talk to each other and work together like this would also be a far more accurate software representation of what is actually going on: different subsystems passing control over a shared device around to work towards the common goal of a good user experience.
A bit of context: The idea underlying this came from my experience with accessibility technology under Linux where uncoordinated fighting over the audio device among all kinds of processes led to countless ways in which things would break with no hope of ever fixing anything. It instilled in me the conviction that user-facing programs are broken if they are not written to talk to each other to coordinate access to shared resources for the goal of rendering a good user experience, but instead leave it to the distro maintainer/user to set things up into a static, brittle working order. Seeing a much-needed cultural shift begin somewhere would be nice. The Plymouth->X transition would lend itself well as a starting point since many building blocks are already there.
I might recommend picking a Wayland display server instead of Xorg first. The thing is, Xorg is only a middle-man and it is some X11 client that decides what and how will be displayed. Therefore with X11 architecture, it's not just two processes that need to communicate the hand-off, it's three: Plymouth, Xorg, and the X11 client that will actually draw stuff.
In Wayland architecture, you only need to communicate between Plymouth and the Wayland compositor you picked. How that Wayland compositor draws anything is an internal detail to it, so you can solve that in-project.
You could also think that how Xorg gets the content is an internal detail to the X11 desktop, but that might lead to needing some new X11 protocol extension to be able to control Xorg's actions on the KMS device sufficiently. It may also be hard to get any new feature code into Xorg and released.
Thanks, pq
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser contact@emersion.fr wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
Userspace protocol is also useful for starting the next KMS client first and handing off only later once it's actually running. I'm not sure if that is already possible with the session switching stuff, but I have a feeling it might be fragile or miss pieces like the next KMS client signalling ready before actually switching to it.
Thanks, pq
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser <contact at emersion.fr> wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
The state reset already is an implicit protocol. Another IPC mechanism however could extend it to work the other way around: instead of inheriting all the state and trying to transition from that to the second client's desired state the second client would send its own desired state back to the first (instead of applying it immediately) which would then try to transition from its own state to the second state (and if it can't you fall back to the implicit inherited state protocol). However, this is only an improvement if the first client knows how to do the transition and the second does not. All in all I doubt that you can convince most people to add this kind of complexity just for slightly higher chances of a good transition.
The reset state protocol on the other hand solves real problems and gives you a good transition as long as the second client knows about the same properties as the previous one which usually is the case for the typical bootsplash->login manager->compositor chain.
Maybe I'm completely missing how such a protocol would work though.
Userspace protocol is also useful for starting the next KMS client first and handing off only later once it's actually running. I'm not sure if that is already possible with the session switching stuff, but I have a feeling it might be fragile or miss pieces like the next KMS client signalling ready before actually switching to it.
Thanks, pq
On Tue, Sep 07, 2021 at 05:52:41PM +0200, Sebastian Wick wrote:
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser <contact at emersion.fr> wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
The state reset already is an implicit protocol. Another IPC mechanism however could extend it to work the other way around: instead of inheriting all the state and trying to transition from that to the second client's desired state the second client would send its own desired state back to the first (instead of applying it immediately) which would then try to transition from its own state to the second state (and if it can't you fall back to the implicit inherited state protocol). However, this is only an improvement if the first client knows how to do the transition and the second does not. All in all I doubt that you can convince most people to add this kind of complexity just for slightly higher chances of a good transition.
The reset state protocol on the other hand solves real problems and gives you a good transition as long as the second client knows about the same properties as the previous one which usually is the case for the typical bootsplash->login manager->compositor chain.
Maybe I'm completely missing how such a protocol would work though.
The idea was that since you would have to have some IPC mechanism in user space anyway to quickly effect a flicker-free transition from Plymouth to the display manager (since, as de Goede reiterates in the other message, both processes must have the device already open and call drmSetMaster/drmDropMaster coordinatedly) you might just as well look for ways how it could be designed for the benefit of everyone. Using "implicit protocols" for things like this is usually the go-to way, not because it's good design, but because it is easy to implement. But these "implicit protocols" have a tendency to greatly limit what can be done and to not be easily adaptable once the use cases become more complicated or refined, and thus they force contortions on everyone eventually.
How such a protocol could look? I don't know. Maybe some DBus interface for a broker/multiplexer for shared devices that would keep track of the current DRM master and tell any process interested in obtaining it what process to talk to. It could then contact it either via DBus or over a separate socket, communicate its capabilities, negotiate the modalities for the transition and acquire the necessary resources in the form of file descriptors passed over DBus/the socket. Then both processes could set themselves up for the transition and effect it, which could involve e.g. unlocking a locked mutex/semaphore in shared memory. Alternatively, the donor could refuse the handover, e.g. if a screen locker is configured to prohibit release of the device. Complexitywise the sky would be the limit, of course, but it needn't be this complicated from the beginning. An initial version of such a protocol could be held just as simple as the status quo.
As for the point raised by Paalanen that implementing something like this would require a lot of effort I must state that, while certainly true, many of the things I mentioned here are already implemented somehow somewhere. Plymouth has a control socket and protocol with which the state of the splash screen can be controlled from the outside to make the transition to gdm smoother. The xlease project apparently was designed with the intent that DRM devices should be leased (and subleased) out to processes, and cross-process coordination would be governed this way. The kmscon project also had to come up with something to govern device access since it could no longer piggy-back on the VT-API. systemd-logind also draws up a framework for governance over a shared device and how to tie them to sessions/seats (with such peculiarities that you cannot auto-spawn a getty on tty1 since that is "reserved" for Wayland). Then there is the VT console, and probably lots of other little things I don't even know about.
All this effort is already being expended, and IMO it proves the need for some way to gracefully coordinate access to shared devices that offers more than what can be done with current "implicit protocols". The community should consider acknowledging this need and trying to answer the question what such a subsystem should and should not look like. Once the nature of the problem is understood better implementational questions will become easier.
Regards.
On Wed, 8 Sep 2021 18:21:12 +0200 Dennis Filder d.filder@web.de wrote:
The idea was that since you would have to have some IPC mechanism in user space anyway to quickly effect a flicker-free transition from Plymouth to the display manager (since, as de Goede reiterates in the other message, both processes must have the device already open and call drmSetMaster/drmDropMaster coordinatedly) you might just as well look for ways how it could be designed for the benefit of everyone. Using "implicit protocols" for things like this is usually the go-to way, not because it's good design, but because it is easy to implement. But these "implicit protocols" have a tendency to greatly limit what can be done and to not be easily adaptable once the use cases become more complicated or refined, and thus they force contortions on everyone eventually.
How such a protocol could look? I don't know. Maybe some DBus interface for a broker/multiplexer for shared devices that would keep track of the current DRM master and tell any process interested in obtaining it what process to talk to.
Hi,
such broker daemon exists already. It is called systemd-logind, and maybe some of the logind replacements as well. Currently it handles session switching, the D-Bus API replacing the legacy VT switching API and drmSetMaster/drmDropMaster. It's also independent of the VT subsystem, so it can be used on all physical seats.
The existing D-Bus API there is probably not flexible enough for the best hand-over experience though, as Simon said.
Since session switching is what the logind API does, hand-over protocols would fit there IMO.
It could then contact it either via DBus or over a separate socket, communicate its capabilities, negotiate the modalities for the transition and acquire the necessary resources in the form of file descriptors passed over DBus/the socket. Then both processes could set themselves up for the transition and effect it, which could involve e.g. unlocking a locked mutex/semaphore in shared memory. Alternatively, the donor could refuse the handover, e.g. if a screen locker is configured to prohibit release of the device. Complexitywise the sky would be the limit, of course, but it needn't be this complicated from the beginning. An initial version of such a protocol could be held just as simple as the status quo.
As for the point raised by Paalanen that implementing something like this would require a lot of effort I must state that, while certainly true, many of the things I mentioned here are already implemented somehow somewhere. Plymouth has a control socket and protocol with which the state of the splash screen can be controlled from the outside to make the transition to gdm smoother. The xlease project apparently was designed with the intent that DRM devices should be leased (and subleased) out to processes, and cross-process coordination would be governed this way.
DRM leasing was not designed for permanent hand-over. Instead, it was designed for temporarily leasing some KMS resources to another process, where the lessor can forcibly revoke them at any time.
That said, I'm not sure what happens to the lessee if the lessor quits. I would expect the lease to get revoked, since how else would the next display server instance gain control if the lessor e.g. crashed.
I'd keep the DRM leasing out of the permanent whole-device hand-off design, it seems like a detour to me.
The kmscon project also had to come up with something to govern device access since it could no longer piggy-back on the VT-API. systemd-logind also draws up a framework for governance over a shared device and how to tie them to sessions/seats (with such peculiarities that you cannot auto-spawn a getty on tty1 since that is "reserved" for Wayland). Then there is the VT console, and probably lots of other little things I don't even know about.
All this effort is already being expended, and IMO it proves the need for some way to gracefully coordinate access to shared devices that offers more than what can be done with current "implicit protocols". The community should consider acknowledging this need and trying to answer the question what such a subsystem should and should not look like. Once the nature of the problem is understood better implementational questions will become easier.
Your initial email came out a bit ranty rather than a technical question. I think it would help if you had a tangible goal, some specific behaviour you want to implement, and write that down in detail. Then it would be easier to discuss how to achieve that while not making the solution excessively single-purpose or hacky. Imaginary use cases are both fairly abstract so hard to discuss, and uncertain if they would ever actually be done - putting the effort needed at risk of being wasted, reducing interest.
I acknowledge a lot of needs all over, but there are only so many hours in a day. My wish for a "KMS state reset" feature is in the same position. First I am working towards userspace features (color management and HDR) that I expect to highlight the need for the feature, and then see if the wish gets more traction - unless one day I would be able to work on it myself. It is also perfectly possible that KMS state reset feature will never happen, since it is too easy to fix KMS clients to just program more KMS properties.
Thanks, pq
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser contact@emersion.fr wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
The KMS state can be limited to single FB on primary plane covering the whole CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.
Is it useful to make the previous client perform the animation? I don't really understand the use-case here.
Userspace protocol is also useful for starting the next KMS client first and handing off only later once it's actually running. I'm not sure if that is already possible with the session switching stuff, but I have a feeling it might be fragile or miss pieces like the next KMS client signalling ready before actually switching to it.
Hm, right. I'm not 100% clear if it's possible for the next client to set everything up while the VT is not active.
It would help to make logind/seatd give a non-master DRM FD when VT-switched away. Not sure they do it atm.
On Wed, 08 Sep 2021 09:51:54 +0000 Simon Ser contact@emersion.fr wrote:
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser contact@emersion.fr wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
The KMS state can be limited to single FB on primary plane covering the whole CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.
Is it useful to make the previous client perform the animation? I don't really understand the use-case here.
I guess the use cases are more or less imaginary for now.
Imagine one HDR-capable display server handing off to another HDR-capable display server. If the releasing display server does not know the receiving display server understands HDR, the releasing display server might run an animation to turn HDR off - fade to black, for instance, so that the impact from changing from HDR to SDR is minimized. Then the receiving display server sees KMS is in SDR mode, and maybe sets up a black image and then switches back to HDR.
If you're happy with fade-to-black on switch, then no problem. However, the only way to not fade-to-black or even come cross-fade is some negotiation to see that both sides understand HDR.
If the previous FB was rendered for HDR display, you will need to know a lot from it if you want to do a cross-fade that doesn't glitch.
Also, while I don't see why changing between SDR and HDR would require a modeset in KMS, I suppose it might take a moment for the monitor to adapt. It might cause glitches similar to changing video modes.
Userspace protocol is also useful for starting the next KMS client first and handing off only later once it's actually running. I'm not sure if that is already possible with the session switching stuff, but I have a feeling it might be fragile or miss pieces like the next KMS client signalling ready before actually switching to it.
Hm, right. I'm not 100% clear if it's possible for the next client to set everything up while the VT is not active.
It would help to make logind/seatd give a non-master DRM FD when VT-switched away. Not sure they do it atm.
Oh yeah, that may be an obvious gap I missed.
Thanks, pq
On Wed, Sep 08, 2021 at 09:51:54AM +0000, Simon Ser wrote:
On Tue, 07 Sep 2021 10:19:03 +0000 Simon Ser contact@emersion.fr wrote:
FWIW, I've just hit a case where a compositor leaves a "rotation" KMS prop set behind, then Xorg tries to startup and fails because it doesn't reset this prop. So none of this is theoretical.
I still think a "reset all KMS props to an arbitrary default value" flag in drmModeAtomicCommit is the best way forward. I'm not sure a user-space protocol would help too much.
Hi Simon,
for the "reset KMS state" problem, sure. Thanks for confirming the problem, too.
The hand-off problem does need userspace protocol though, so that the two parties can negotiate what part of KMS state can be inherited by the receiver and who will do the animation from the first to the second state in case you want to avoid abrupt changes. It would also be useful for a cross-fade as a perhaps more flexible way than the current "leak an FB, let the next KMS client scrape it via ioctls and copy it so it can be textured from".
The KMS state can be limited to single FB on primary plane covering the whole CRTC, no scaling, no other property set than FB_ID/CRTC_*/SRC_*.
Is it useful to make the previous client perform the animation? I don't really understand the use-case here.
The use case for the animation is e.g. the transition from Plymouth to the display server. Currently it is done as a still frame transition, maybe with a blend-over effect. But with the current design it is not possible to blend Plymouth's animation over into another animation in the display server because the second client lacks the knowledge how to keep it going for a little bit.
Another use case is switching between sessions which currently also is only possible as a still frame transition. However, if you wanted to present the session switching by doing e.g. a shaking screen animation and blending the old display content over into the new content then the first client would have to render the first half of the animation, and the second client would have to render the second half during which it would then blend away the content of the first screen while blending in its own content and also slowing the shaking to a stop. For that to work the second client would need all the information necessary to render that animation, and also a way to perform the frame-perfect change-over.
Granted, that is a very complicated, eye-candy-oriented use case, but it would serve to show-case the potential of the design.
Regards.
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
Xorg doing something (e.g. resetting instead of inheriting KMS state)
Something missed in the hand-off sequence which allows fbcon to momentarily take over between Plymouth and Xorg. This would need to be fixed between Plymouth and Xorg.
Maybe systemd-logind does something odd to the KMS device? It has pretty wild code there. Or maybe it causes fbcon to take over.
What is the new ioctl you referred to?
It is an ioctl to mark a FB to not have it auto-removed on device-close, instead leaving it in place until some some kernel/userspace client actively installs another FB. This was proposed by Rob Clark quite a while ago, but it never got anywhere because of lack of userspace actually interested in using it.
I've been thinking about reviving Rob's patch, since at least for plymouth this would be pretty useful to have.
Regards,
Hans
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
Thanks, pq
Xorg doing something (e.g. resetting instead of inheriting KMS state)
Something missed in the hand-off sequence which allows fbcon to momentarily take over between Plymouth and Xorg. This would need to be fixed between Plymouth and Xorg.
Maybe systemd-logind does something odd to the KMS device? It has pretty wild code there. Or maybe it causes fbcon to take over.
What is the new ioctl you referred to?
It is an ioctl to mark a FB to not have it auto-removed on device-close, instead leaving it in place until some some kernel/userspace client actively installs another FB. This was proposed by Rob Clark quite a while ago, but it never got anywhere because of lack of userspace actually interested in using it.
I've been thinking about reviving Rob's patch, since at least for plymouth this would be pretty useful to have.
Regards,
Hans
Hi,
On 9/8/21 9:36 AM, Pekka Paalanen wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
I'm pretty sure that that is what is happening though.
But hopefully someone else can either confirm or deny this :)
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
plymouth is actually keeping the device open longer for exactly this reason, the following happens:
1. plmouth starts 2. gdm starts and tells plymouth to "deactivate" which will stop it from making drm ioctls and drop its drm master rights, while keeping the fb around 3. gdm waits for the greeter process to tell it that it has successfully taken over the screen 4. gdm tells plymouth to quit
And something similar is happening on gdm greeter -> gnome user session handover.
But we need the new ioctl at least on shutdown / reboot to avoid the "fallback fb" (typically the EFI/BIOS setup fb which i915 inherited at boot) showing for a brief moment when plymouth quits at shutdown / reboot and there is nothing to hand-over the fb to in that case.
And the new ioctl would also make the above handover a lot simpler.
And we currently also have a flicker when going from user-session to gdm on logout or from gdm/user-session to plymout on shutdown/reboot.
Basically we have quite a few transitions and currently only the boot + login path is smooth and the rest needs more work, which either requires a standardized handover method (instead of the current hardcoded plymouth -> gdm stuff), or just allowing the FB to sit around until the next drm-client installs its FB, which would be much more KISS, so that has my preference.
And this KISS method will also work with transitions to a new console-owner process which is not aware of any handover protocols, as long as the old process uses the ioctl the transition will be smooth. So e.g. gdm -> i3 on Xorg session will be smooth (1)
Regards,
Hans
1) I think this actually already is smooth because in this case gdm just sleeps for 5 seconds before killing the greeter I believe, but with the ioctl we could remove this hack
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor: - RMFB instead of CLOSEFB semantics, especially when closing the drmfd. This is uapi, so anything we change needs to be opt-in - Forced fbdev restore on final close of all drm fd. This is only prevented if there's a drm master left around (systemd-logind can keep that instead of forcing the compositor to survive until the other has taken over, which it needs to do anyway to prevent the drm master handover from going sideways). This can be fixed by simply disabling fbdev completely, which you really want to do anyway. Again it's uabi, people will complain if we break this I think.
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
Yeah, that stops both forced restore/disable from kicking in.
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
With closefb semantics and fbdev disabled plymouth could quit early, and things still work. -Daniel
Thanks, pq
Xorg doing something (e.g. resetting instead of inheriting KMS state)
Something missed in the hand-off sequence which allows fbcon to momentarily take over between Plymouth and Xorg. This would need to be fixed between Plymouth and Xorg.
Maybe systemd-logind does something odd to the KMS device? It has pretty wild code there. Or maybe it causes fbcon to take over.
What is the new ioctl you referred to?
It is an ioctl to mark a FB to not have it auto-removed on device-close, instead leaving it in place until some some kernel/userspace client actively installs another FB. This was proposed by Rob Clark quite a while ago, but it never got anywhere because of lack of userspace actually interested in using it.
I've been thinking about reviving Rob's patch, since at least for plymouth this would be pretty useful to have.
Regards,
Hans
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
- Forced fbdev restore on final close of all drm fd. This is only
prevented if there's a drm master left around (systemd-logind can keep that instead of forcing the compositor to survive until the other has taken over, which it needs to do anyway to prevent the drm master handover from going sideways). This can be fixed by simply disabling fbdev completely, which you really want to do anyway. Again it's uabi, people will complain if we break this I think.
Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode to stop fbcon/fbdev from taking over?
Is it really fbdev on its own rather than fbcon (poking at fbdev) that will change the KMS state?
That is, it's not enough to disable fbcon?
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
Yeah, that stops both forced restore/disable from kicking in.
Which "that"?
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
With closefb semantics and fbdev disabled plymouth could quit early, and things still work.
What is "closefb semantics"?
Thanks, pq
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
Hans de Goede asked me to take a topic from a private discussion here. I must also preface that I'm not a graphics person and my knowledge of DRI/DRM is cursory at best.
I initiated the conversation with de Goede after learning that the X server now supports being started with an open DRM file descriptor (this was added for Keith Packard's xlease project). I wondered if that could be used to smoothen the Plymouth->X transition somehow and asked de Goede if there were any such plans. He denied, but mentioned that a new ioctl is in the works to prevent the kernel from wiping the contents of a frame buffer after a device is closed, and that this would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
- Forced fbdev restore on final close of all drm fd. This is only
prevented if there's a drm master left around (systemd-logind can keep that instead of forcing the compositor to survive until the other has taken over, which it needs to do anyway to prevent the drm master handover from going sideways). This can be fixed by simply disabling fbdev completely, which you really want to do anyway. Again it's uabi, people will complain if we break this I think.
Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode to stop fbcon/fbdev from taking over?
Nope. You need an open drm master.
This is because we do actually support /dev/fb clients rendering in KD_GRAPHICS mode for backwards compat with the fbdev subsystem.
Is it really fbdev on its own rather than fbcon (poking at fbdev) that will change the KMS state?
That is, it's not enough to disable fbcon?
fbcon doesn't disable fbdev, and the only way to block fbdev is to have a drm master around.
I guess we could try and make this smarter by creating some kind of weak master status for fbdev, but only when either fbcon or fbdev is opened. Maybe this would help?
fbdev already keep track of this open count, so wouldn't be too onerous to wire that up into drm_client.
The problem there is then that not yet all drivers use the drm_client stuff for fbdev emulation, so you'd need to either convert more, or hack up a few more things to make this consistent.
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
Yeah, that stops both forced restore/disable from kicking in.
Which "that"?
that = open drm master. Open drm master alwasy wins agains fbdev/fbcon, and with latest patches it's guaranteed to be race free.
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
With closefb semantics and fbdev disabled plymouth could quit early, and things still work.
What is "closefb semantics"?
closefb semantics = no forced plane/crtc disable, active plane keeps a drm_fb reference
rmfb semantics = forced plane/crtc disable, the drm_fb is guaranteed to be forcefully removed from the system
Cheers, Daniel
On Tue, 14 Sep 2021 15:45:21 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote:
On Fri, 3 Sep 2021 21:08:21 +0200 Dennis Filder d.filder@web.de wrote:
> Hans de Goede asked me to take a topic from a private discussion here. > I must also preface that I'm not a graphics person and my knowledge of > DRI/DRM is cursory at best. > > I initiated the conversation with de Goede after learning that the X > server now supports being started with an open DRM file descriptor > (this was added for Keith Packard's xlease project). I wondered if > that could be used to smoothen the Plymouth->X transition somehow and > asked de Goede if there were any such plans. He denied, but mentioned > that a new ioctl is in the works to prevent the kernel from wiping the > contents of a frame buffer after a device is closed, and that this > would help to keep transitions smooth.
Hi,
I believe the kernel is not wiping anything on device close. If something in the KMS state is wiped, it originates in userspace:
- Plymouth doing something (e.g. RmFB on an in-use FB will turn the output off, you need to be careful to "leak" your FB if you want a smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
Hi Daniel,
such flag would make sense to me.
- Forced fbdev restore on final close of all drm fd. This is only
prevented if there's a drm master left around (systemd-logind can keep that instead of forcing the compositor to survive until the other has taken over, which it needs to do anyway to prevent the drm master handover from going sideways). This can be fixed by simply disabling fbdev completely, which you really want to do anyway. Again it's uabi, people will complain if we break this I think.
Do you mean that it is not enough to leave the tty in KD_GRAPHICS mode to stop fbcon/fbdev from taking over?
Nope. You need an open drm master.
This is because we do actually support /dev/fb clients rendering in KD_GRAPHICS mode for backwards compat with the fbdev subsystem.
Is it really fbdev on its own rather than fbcon (poking at fbdev) that will change the KMS state?
That is, it's not enough to disable fbcon?
fbcon doesn't disable fbdev, and the only way to block fbdev is to have a drm master around.
Thanks for pointing this out.
I guess we could try and make this smarter by creating some kind of weak master status for fbdev, but only when either fbcon or fbdev is opened. Maybe this would help?
It would make sense to me if we wanted a bandaid, but do we? I don't know.
fbdev already keep track of this open count, so wouldn't be too onerous to wire that up into drm_client.
The problem there is then that not yet all drivers use the drm_client stuff for fbdev emulation, so you'd need to either convert more, or hack up a few more things to make this consistent.
Or am I confused because display servers do not tend to close the DRM device fd on switch-out but Plymouth does (too early)?
Yeah, that stops both forced restore/disable from kicking in.
Which "that"?
that = open drm master. Open drm master alwasy wins agains fbdev/fbcon, and with latest patches it's guaranteed to be race free.
If so, why can't Plymouth keep the device open longer and quit only when the hand-off is complete? Not quitting too early would be a prerequisite for any explicit hand-off protocol as well.
With closefb semantics and fbdev disabled plymouth could quit early, and things still work.
What is "closefb semantics"?
closefb semantics = no forced plane/crtc disable, active plane keeps a drm_fb reference
rmfb semantics = forced plane/crtc disable, the drm_fb is guaranteed to be forcefully removed from the system
Very good.
Thanks, pq
Hi,
On 9/22/21 10:56 AM, Pekka Paalanen wrote:
On Tue, 14 Sep 2021 15:45:21 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/7/21 12:07 PM, Pekka Paalanen wrote: > On Fri, 3 Sep 2021 21:08:21 +0200 > Dennis Filder d.filder@web.de wrote: > >> Hans de Goede asked me to take a topic from a private discussion here. >> I must also preface that I'm not a graphics person and my knowledge of >> DRI/DRM is cursory at best. >> >> I initiated the conversation with de Goede after learning that the X >> server now supports being started with an open DRM file descriptor >> (this was added for Keith Packard's xlease project). I wondered if >> that could be used to smoothen the Plymouth->X transition somehow and >> asked de Goede if there were any such plans. He denied, but mentioned >> that a new ioctl is in the works to prevent the kernel from wiping the >> contents of a frame buffer after a device is closed, and that this >> would help to keep transitions smooth. > > Hi, > > I believe the kernel is not wiping anything on device close. If > something in the KMS state is wiped, it originates in userspace: > > - Plymouth doing something (e.g. RmFB on an in-use FB will turn the > output off, you need to be careful to "leak" your FB if you want a > smooth hand-over)
The "kernel is not wiping anything on device close" is not true, when closing /dev/dri/card# any remaining FBs from the app closing it will be dealt with as if they were RmFB-ed, causing the screen to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
Hi Daniel,
such flag would make sense to me.
Hmm, I was thinking having a separate call to mark a FB to switch to closefb semantics. But both plymouth (because of end of animation) and GNOME (because a mostly empty gnome-shell needs to be rendered to avoid leaking privacy sensitive info) will need to prepare a special FB on exit anyways, so then an ADDFB2 flag would work fine.
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Regards,
Hans
On Wed, 22 Sep 2021 11:21:16 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/22/21 10:56 AM, Pekka Paalanen wrote:
On Tue, 14 Sep 2021 15:45:21 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote:
On Tue, 7 Sep 2021 14:42:56 +0200 Hans de Goede hdegoede@redhat.com wrote:
> Hi, > > On 9/7/21 12:07 PM, Pekka Paalanen wrote: >> On Fri, 3 Sep 2021 21:08:21 +0200 >> Dennis Filder d.filder@web.de wrote: >> >>> Hans de Goede asked me to take a topic from a private discussion here. >>> I must also preface that I'm not a graphics person and my knowledge of >>> DRI/DRM is cursory at best. >>> >>> I initiated the conversation with de Goede after learning that the X >>> server now supports being started with an open DRM file descriptor >>> (this was added for Keith Packard's xlease project). I wondered if >>> that could be used to smoothen the Plymouth->X transition somehow and >>> asked de Goede if there were any such plans. He denied, but mentioned >>> that a new ioctl is in the works to prevent the kernel from wiping the >>> contents of a frame buffer after a device is closed, and that this >>> would help to keep transitions smooth. >> >> Hi, >> >> I believe the kernel is not wiping anything on device close. If >> something in the KMS state is wiped, it originates in userspace: >> >> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the >> output off, you need to be careful to "leak" your FB if you want a >> smooth hand-over) > > The "kernel is not wiping anything on device close" is not true, > when closing /dev/dri/card# any remaining FBs from the app closing > it will be dealt with as if they were RmFB-ed, causing the screen > to show what I call "the fallback fb", at least with the i915 driver.
No, that's not what should happen AFAIK.
True, all FBs that are not referenced by active CRTCs or planes will get freed, since their refcount drops to zero, but those CRTCs and planes that are active will remain active and therefore keep their reference to the respective FBs and so the FBs remain until replaced or turned off explicitly (by e.g. fbcon if you switch to that rather than another userspace KMS client). I believe that is the whole reason why e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client would not have anything to scrape.
danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
Hi Daniel,
such flag would make sense to me.
Hmm, I was thinking having a separate call to mark a FB to switch to closefb semantics. But both plymouth (because of end of animation) and GNOME (because a mostly empty gnome-shell needs to be rendered to avoid leaking privacy sensitive info) will need to prepare a special FB on exit anyways, so then an ADDFB2 flag would work fine.
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Right, but I'm thinking this from the other side: why would anyone deliberately *want* RmFB semantics on device close?
I can't think of any, and hence I would be inclined to assume that userspace would just switch to using closefb semantics for everything all the time.
Legacy userspace is one thing, but userspace that is updated to set closefb semantics will also be aware of what closefb means: it leaves the FBs up and CRTCs and planes enabled, if you leave them like that. So if they don't want that, they know they should not do that.
Asking in another way: why would the same program sometimes use RmFB semantics and sometimes closefb semantics? Even more so, why would one switch an FB from one to the other?
Hmm... to prevent leaking sensitive FBs on crash, perhaps? But you can do that decision at AddFB2 time, right? Maybe not, as you can't really force EGL to allocate a new buffer at will. Oh, but when EGL gives me a buffer that I know is safe to leave up, I also know that it is not up on any KMS plane (no front buffer rendering), so I can just RmFB and AddFB2 again. That's a bit of a detour though.
At least a separate ioctl on an FB would be more flexible than a flag at AddFB2.
Btw. what happens if I try to AddFB2 the same buffer twice?
Thanks, pq
Hi,
On 9/23/21 10:23 AM, Pekka Paalanen wrote:
On Wed, 22 Sep 2021 11:21:16 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/22/21 10:56 AM, Pekka Paalanen wrote:
On Tue, 14 Sep 2021 15:45:21 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote: > > On Tue, 7 Sep 2021 14:42:56 +0200 > Hans de Goede hdegoede@redhat.com wrote: > >> Hi, >> >> On 9/7/21 12:07 PM, Pekka Paalanen wrote: >>> On Fri, 3 Sep 2021 21:08:21 +0200 >>> Dennis Filder d.filder@web.de wrote: >>> >>>> Hans de Goede asked me to take a topic from a private discussion here. >>>> I must also preface that I'm not a graphics person and my knowledge of >>>> DRI/DRM is cursory at best. >>>> >>>> I initiated the conversation with de Goede after learning that the X >>>> server now supports being started with an open DRM file descriptor >>>> (this was added for Keith Packard's xlease project). I wondered if >>>> that could be used to smoothen the Plymouth->X transition somehow and >>>> asked de Goede if there were any such plans. He denied, but mentioned >>>> that a new ioctl is in the works to prevent the kernel from wiping the >>>> contents of a frame buffer after a device is closed, and that this >>>> would help to keep transitions smooth. >>> >>> Hi, >>> >>> I believe the kernel is not wiping anything on device close. If >>> something in the KMS state is wiped, it originates in userspace: >>> >>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the >>> output off, you need to be careful to "leak" your FB if you want a >>> smooth hand-over) >> >> The "kernel is not wiping anything on device close" is not true, >> when closing /dev/dri/card# any remaining FBs from the app closing >> it will be dealt with as if they were RmFB-ed, causing the screen >> to show what I call "the fallback fb", at least with the i915 driver. > > No, that's not what should happen AFAIK. > > True, all FBs that are not referenced by active CRTCs or planes will > get freed, since their refcount drops to zero, but those CRTCs and > planes that are active will remain active and therefore keep their > reference to the respective FBs and so the FBs remain until replaced or > turned off explicitly (by e.g. fbcon if you switch to that rather than > another userspace KMS client). I believe that is the whole reason why > e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client > would not have anything to scrape. > > danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
Hi Daniel,
such flag would make sense to me.
Hmm, I was thinking having a separate call to mark a FB to switch to closefb semantics. But both plymouth (because of end of animation) and GNOME (because a mostly empty gnome-shell needs to be rendered to avoid leaking privacy sensitive info) will need to prepare a special FB on exit anyways, so then an ADDFB2 flag would work fine.
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Right, but I'm thinking this from the other side: why would anyone deliberately *want* RmFB semantics on device close?
Device-close also happens on application crashes. So basically any time where the app (typically a display-server) may display privacy sensitive contents which we don't want to leak to other apps, we want RmFB semantics on close.
ATM e.g. GNOME just leaves everything on the screen in place at logout / shutdown. So "apps" are definitely going to need to have some work done to them to make sure that from a privacy pov it is safe to leave the FB in place after they exit.
I can't think of any, and hence I would be inclined to assume that userspace would just switch to using closefb semantics for everything all the time.
Legacy userspace is one thing, but userspace that is updated to set closefb semantics will also be aware of what closefb means: it leaves the FBs up and CRTCs and planes enabled, if you leave them like that. So if they don't want that, they know they should not do that.
Asking in another way: why would the same program sometimes use RmFB semantics and sometimes closefb semantics? Even more so, why would one switch an FB from one to the other?
Hmm... to prevent leaking sensitive FBs on crash, perhaps?
Right, I should have read your whole email before responding.
But you can do that decision at AddFB2 time, right?
Yes I believe so.
Maybe not, as you can't really force EGL to allocate a new buffer at will.
It does not need to be a new buffer as long as we are sure it has safe contents in it, there will need to be a special render path for rendering a "safe" desktop view on logout and once that rendering has completed to a buffer, then that buffer can be considered safe I guess (note I don't have a whole lot of experience with EGL).
Oh, but when EGL gives me a buffer that I know is safe to leave up, I also know that it is not up on any KMS plane (no front buffer rendering), so I can just RmFB and AddFB2 again. That's a bit of a detour though.
At least a separate ioctl on an FB would be more flexible than a flag at AddFB2.
Btw. what happens if I try to AddFB2 the same buffer twice?
These questions are best answered by someone who knows this stuff better then me...
Regards,
Hans
On Thu, Sep 23, 2021 at 11:23:00AM +0300, Pekka Paalanen wrote:
On Wed, 22 Sep 2021 11:21:16 +0200 Hans de Goede hdegoede@redhat.com wrote:
Hi,
On 9/22/21 10:56 AM, Pekka Paalanen wrote:
On Tue, 14 Sep 2021 15:45:21 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Thu, Sep 09, 2021 at 10:37:03AM +0300, Pekka Paalanen wrote:
On Wed, 8 Sep 2021 18:27:09 +0200 Daniel Vetter daniel@ffwll.ch wrote:
On Wed, Sep 8, 2021 at 9:36 AM Pekka Paalanen ppaalanen@gmail.com wrote: > > On Tue, 7 Sep 2021 14:42:56 +0200 > Hans de Goede hdegoede@redhat.com wrote: > >> Hi, >> >> On 9/7/21 12:07 PM, Pekka Paalanen wrote: >>> On Fri, 3 Sep 2021 21:08:21 +0200 >>> Dennis Filder d.filder@web.de wrote: >>> >>>> Hans de Goede asked me to take a topic from a private discussion here. >>>> I must also preface that I'm not a graphics person and my knowledge of >>>> DRI/DRM is cursory at best. >>>> >>>> I initiated the conversation with de Goede after learning that the X >>>> server now supports being started with an open DRM file descriptor >>>> (this was added for Keith Packard's xlease project). I wondered if >>>> that could be used to smoothen the Plymouth->X transition somehow and >>>> asked de Goede if there were any such plans. He denied, but mentioned >>>> that a new ioctl is in the works to prevent the kernel from wiping the >>>> contents of a frame buffer after a device is closed, and that this >>>> would help to keep transitions smooth. >>> >>> Hi, >>> >>> I believe the kernel is not wiping anything on device close. If >>> something in the KMS state is wiped, it originates in userspace: >>> >>> - Plymouth doing something (e.g. RmFB on an in-use FB will turn the >>> output off, you need to be careful to "leak" your FB if you want a >>> smooth hand-over) >> >> The "kernel is not wiping anything on device close" is not true, >> when closing /dev/dri/card# any remaining FBs from the app closing >> it will be dealt with as if they were RmFB-ed, causing the screen >> to show what I call "the fallback fb", at least with the i915 driver. > > No, that's not what should happen AFAIK. > > True, all FBs that are not referenced by active CRTCs or planes will > get freed, since their refcount drops to zero, but those CRTCs and > planes that are active will remain active and therefore keep their > reference to the respective FBs and so the FBs remain until replaced or > turned off explicitly (by e.g. fbcon if you switch to that rather than > another userspace KMS client). I believe that is the whole reason why > e.g. DRM_IOCTL_MODE_GETFB2 can be useful, otherwise the next KMS client > would not have anything to scrape. > > danvet, what is the DRM core intention?
Historical accidents mostly. There's two things that foil easy handover to the next compositor:
- RMFB instead of CLOSEFB semantics, especially when closing the
drmfd. This is uapi, so anything we change needs to be opt-in
What does this mean and refer to?
Are you trying to say, that closing the DRM device fd (freeing the file description) causes an implicit RmFB on all the FBs tied to that DRM device file description?
I never realised that before.
Yes, final close does iterate over fb and do an RMFB. Which is why we've had this discussion whether closefb semantics should be an ADDFB2 flag at creation time instead.
Hi Daniel,
such flag would make sense to me.
Hmm, I was thinking having a separate call to mark a FB to switch to closefb semantics. But both plymouth (because of end of animation) and GNOME (because a mostly empty gnome-shell needs to be rendered to avoid leaking privacy sensitive info) will need to prepare a special FB on exit anyways, so then an ADDFB2 flag would work fine.
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Right, but I'm thinking this from the other side: why would anyone deliberately *want* RmFB semantics on device close?
I can't think of any, and hence I would be inclined to assume that userspace would just switch to using closefb semantics for everything all the time.
Legacy userspace is one thing, but userspace that is updated to set closefb semantics will also be aware of what closefb means: it leaves the FBs up and CRTCs and planes enabled, if you leave them like that. So if they don't want that, they know they should not do that.
Asking in another way: why would the same program sometimes use RmFB semantics and sometimes closefb semantics? Even more so, why would one switch an FB from one to the other?
Hmm... to prevent leaking sensitive FBs on crash, perhaps? But you can do that decision at AddFB2 time, right? Maybe not, as you can't really force EGL to allocate a new buffer at will. Oh, but when EGL gives me a buffer that I know is safe to leave up, I also know that it is not up on any KMS plane (no front buffer rendering), so I can just RmFB and AddFB2 again. That's a bit of a detour though.
At least a separate ioctl on an FB would be more flexible than a flag at AddFB2.
Btw. what happens if I try to AddFB2 the same buffer twice?
drm_fb are just refcounted metadata containers. So you could have the same underlying gem bo wrapped in 2 addfb, one with rmfb and one with closefb semantics. Depending which one you're using on which crtc, the crtc might be shut off or not when you close the drmfd.
It's a bit silly, but no problem for the kernel, so I think this is all fine.
Real use-case of multiple drm_fb for the same underlying object is stuff like XRGB vs ARGB or different modifiers (like with compression enabled or compression metadata not up to date). -Daniel
On Wednesday, September 22nd, 2021 at 11:21, Hans de Goede hdegoede@redhat.com wrote:
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Do you have plans to work on the kernel side part of this? If so, feel free to CC me for a review. If not, let me know if you'd like to switch roles, I could work on a patch.
Simon
Hi,
On 10/1/21 6:33 PM, Simon Ser wrote:
On Wednesday, September 22nd, 2021 at 11:21, Hans de Goede hdegoede@redhat.com wrote:
I would be happy to work on the plymouth side of this, so that we have at least one consumer of such a flag lined up for merging.
Do you have plans to work on the kernel side part of this?
No I don't have any plans to work on the kernel side of this.
If so, feel free to CC me for a review. If not, let me know if you'd like to switch roles, I could work on a patch.
I would be happy to review any kernel patches for this (together with adding support to plymouth for this).
Regards,
Hans
dri-devel@lists.freedesktop.org
-
Daniel Vetter -
Dennis Filder -
Hans de Goede -
Pekka Paalanen -
Sebastian Wick -
Simon Ser