So I've been experimenting with support for Dave Airlie's new RandR 1.4 provider object interface, so that Optimus-based laptops can use our driver to drive the discrete GPU and display on the integrated GPU. The good news is that I've got a proof of concept working.
During a review of the current code, we came up with a few concerns:
1. The output source is responsible for allocating the shared memory
Right now, the X server calls CreatePixmap on the output source screen and then expects the output sink screen to be able to display from whatever memory the source allocates. Right now, the source has no mechanism for asking the sink what its requirements are for the surface. I'm using our own internal pitch alignment requirements and that seems to be good enough for the Intel device to scan out, but that could be pure luck.
Does it make sense to add a mechanism for drivers to negotiate this with each other, or is it sufficient to just define a lowest common denominator format and if your hardware can't deal with that format, you just don't get to share buffers?
One of my coworkers brought to my attention the fact that Tegra requires a specific pitch alignment, and cannot accommodate larger pitches. If other SoC designs have similar restrictions, we might need to add a handshake mechanism.
2. There's no fallback mechanism if sharing can't be negotiated
If RandR fails to share a pixmap with the output sink screen, the whole modeset fails. This means you'll end up not seeing anything on the screen and you'll probably think your computer locked up. Should there be some sort of software copy fallback to ensure that something at least shows up on the display?
3. How should the memory be allocated?
In the prototype I threw together, I'm allocating the shared memory using shm_open and then exporting that as a dma-buf file descriptor using an ioctl I added to the kernel, and then importing that memory back into our driver through dma_buf_attach & dma_buf_map_attachment. Does it make sense for user-space programs to be able to export shmfs files like that? Should that interface go in DRM / GEM / PRIME instead? Something else? I'm pretty unfamiliar with this kernel code so any suggestions would be appreciated.
-- Aaron
P.S. for those unfamiliar with PRIME: Dave Airlie added new support to the X Resize and Rotate extension version 1.4 to support offloading display and rendering to different drivers. PRIME is the DRM implementation in the kernel, layered on top of DMA-BUF, that implements the actual sharing of buffers between drivers.
http://cgit.freedesktop.org/xorg/proto/randrproto/tree/randrproto.txt?id=ran... http://airlied.livejournal.com/75555.html - update on hotplug server http://airlied.livejournal.com/76078.html - randr 1.5 demo videos
----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -----------------------------------------------------------------------------------
On 08/30/2012 10:31 AM, Aaron Plattner wrote:
So I've been experimenting with support for Dave Airlie's new RandR 1.4 provider object interface, so that Optimus-based laptops can use our driver to drive the discrete GPU and display on the integrated GPU. The good news is that I've got a proof of concept working.
During a review of the current code, we came up with a few concerns:
- The output source is responsible for allocating the shared memory
Right now, the X server calls CreatePixmap on the output source screen and then expects the output sink screen to be able to display from whatever memory the source allocates. Right now, the source has no mechanism for asking the sink what its requirements are for the surface. I'm using our own internal pitch alignment requirements and that seems to be good enough for the Intel device to scan out, but that could be pure luck.
Does it make sense to add a mechanism for drivers to negotiate this with each other, or is it sufficient to just define a lowest common denominator format and if your hardware can't deal with that format, you just don't get to share buffers?
One of my coworkers brought to my attention the fact that Tegra requires a specific pitch alignment, and cannot accommodate larger pitches. If other SoC designs have similar restrictions, we might need to add a handshake mechanism.
- There's no fallback mechanism if sharing can't be negotiated
If RandR fails to share a pixmap with the output sink screen, the whole modeset fails. This means you'll end up not seeing anything on the screen and you'll probably think your computer locked up. Should there be some sort of software copy fallback to ensure that something at least shows up on the display?
- How should the memory be allocated?
In the prototype I threw together, I'm allocating the shared memory using shm_open and then exporting that as a dma-buf file descriptor using an ioctl I added to the kernel, and then importing that memory back into our driver through dma_buf_attach & dma_buf_map_attachment. Does it make sense for user-space programs to be able to export shmfs files like that? Should that interface go in DRM / GEM / PRIME instead? Something else? I'm pretty unfamiliar with this kernel code so any suggestions would be appreciated.
There's also a #4 that didn't seem relevant to cross-post to linaro-mm-sig:
4. There's no mechanism for double buffering the output sink
RandR allocates one pixmap on the output source screen and sets up tracking so the source driver can copy the screen into the shared pixmap. However, the sink driver scans out from the shared pixmap directly. There's no mechanism to prevent tearing on the sink side of the pipeline.
It seems like it would be nice if the source could trigger the sink device to flip between front and back buffers when the copy is finished, and get back a fence to indicate when the flip has occurred and the source can start the next copy.
-- Aaron
On Thu, Aug 30, 2012 at 10:34:23AM -0700, Aaron Plattner wrote:
- There's no mechanism for double buffering the output sink
RandR allocates one pixmap on the output source screen and sets up tracking so the source driver can copy the screen into the shared pixmap. However, the sink driver scans out from the shared pixmap directly. There's no mechanism to prevent tearing on the sink side of the pipeline.
It seems like it would be nice if the source could trigger the sink device to flip between front and back buffers when the copy is finished, and get back a fence to indicate when the flip has occurred and the source can start the next copy.
Dave already answered your other questions, I'll chip in with some more details on the in-kernel fencing that Maarten Lankhorst is working on. Current wip code is available at
http://cgit.freedesktop.org/~mlankhorst/linux/
The v10-wip branch is the latest, with some experimental code to port ttm over to the new core reservation code. The fencing itself would be all kernel-internal with no explicit fence objects exposed to userspace: I.e. the driver doing the pageflip would just wait for any write-fences (= exclusive fences) before doing the pageflip and then attach a read (= shared) fence on the to-be-swapped out fb that signals once the pageflip completed (to avoid other gpus rendering into the new backbuffer while it's still being displayed).
People involved in the discussion of this are mostly Maarten, Rob Clark (from TI) and me, and most of the discussion happens on #dri-devel, but at least Maarten&me should be at XDC in Nürnberg. Comments/discussion of this code, especially whether the fence/reservation code would suit you for shared buffers highly welcome. I'll definitely annoy any nvidia guy who shows up at xdc about this ;-)
The longer-term idea is to port all drm/* drivers over to the new reservation framework (hence the experimental ttm port), which would allow us to dynamically evict even shared buffers (which should be rather useful on highly constrained SoC iommus shared among a few graphics IP blocks).
Cheers, Daniel
object interface, so that Optimus-based laptops can use our driver to drive the discrete GPU and display on the integrated GPU. The good news is that I've got a proof of concept working.
Don't suppose you'll be interested in adding the other method at some point as well? since saving power is probably important to a lot of people :-)
During a review of the current code, we came up with a few concerns:
- The output source is responsible for allocating the shared memory
Right now, the X server calls CreatePixmap on the output source screen and then expects the output sink screen to be able to display from whatever memory the source allocates. Right now, the source has no mechanism for asking the sink what its requirements are for the surface. I'm using our own internal pitch alignment requirements and that seems to be good enough for the Intel device to scan out, but that could be pure luck.
Well in theory it might be nice but it would have been premature since so far the only interactions for prime are combination of intel, nvidia and AMD, and I think everyone has fairly similar pitch alignment requirements, I'd be interested in adding such an interface but I don't think its some I personally would be working on.
other, or is it sufficient to just define a lowest common denominator format and if your hardware can't deal with that format, you just don't get to share buffers?
At the moment I'm happy to just go with linear, minimum pitch alignment 64 or something as a base standard, but yeah I'm happy for it to work either way, just don't have enough evidence it's worth it yet. I've not looked at ARM stuff, so patches welcome if people consider they need to use this stuff for SoC devices.
- There's no fallback mechanism if sharing can't be negotiated
If RandR fails to share a pixmap with the output sink screen, the whole modeset fails. This means you'll end up not seeing anything on the screen and you'll probably think your computer locked up. Should there be some sort of software copy fallback to ensure that something at least shows up on the display?
Uggh, it would be fairly slow and unuseable, I'd rather they saw nothing, but again open to suggestions on how to make this work, since it might fail for other reasons and in that case there is still nothing a sw copy can do. What happens if the slave intel device just fails to allocate a pixmap, but yeah I'm willing to think about it a bit more when we have some reference implementations.
- How should the memory be allocated?
In the prototype I threw together, I'm allocating the shared memory using shm_open and then exporting that as a dma-buf file descriptor using an ioctl I added to the kernel, and then importing that memory back into our driver through dma_buf_attach & dma_buf_map_attachment. Does it make sense for user-space programs to be able to export shmfs files like that? Should that interface go in DRM / GEM / PRIME instead? Something else? I'm pretty unfamiliar with this kernel code so any suggestions would be appreciated.
Your kernel driver should in theory be doing it all, if you allocate shared pixmaps in GTT accessible memory, then you need an ioctl to tell your kernel driver to export the dma buf to the fd handle. (assuming we get rid of the _GPL, which people have mentioned they are open to doing). We have handle->fd and fd->handle interfaces on DRM, you'd need something similiar on the nvidia kernel driver interface.
Yes for 4 some sort of fencing is being worked on by Maarten for other stuff but would be a pre-req for doing this, and also some devices don't want fullscreen updates, like USB, so doing flipped updates would have to be optional or negoitated. It makes sense for us as well since things like gnome-shell can do full screen pageflips and we have to do full screen dirty updates.
Dave.
On 08/31/2012 08:00 PM, Dave Airlie wrote:
object interface, so that Optimus-based laptops can use our driver to drive the discrete GPU and display on the integrated GPU. The good news is that I've got a proof of concept working.
Don't suppose you'll be interested in adding the other method at some point as well? since saving power is probably important to a lot of people
That's milestone 2. I'm focusing on display offload to start because it's easier to implement and lays the groundwork for the kernel pieces. I have to emphasize that I'm just doing a feasibility study right now and I can't promise that we're going to officially support this stuff.
During a review of the current code, we came up with a few concerns:
- The output source is responsible for allocating the shared memory
Right now, the X server calls CreatePixmap on the output source screen and then expects the output sink screen to be able to display from whatever memory the source allocates. Right now, the source has no mechanism for asking the sink what its requirements are for the surface. I'm using our own internal pitch alignment requirements and that seems to be good enough for the Intel device to scan out, but that could be pure luck.
Well in theory it might be nice but it would have been premature since so far the only interactions for prime are combination of intel, nvidia and AMD, and I think everyone has fairly similar pitch alignment requirements, I'd be interested in adding such an interface but I don't think its some I personally would be working on.
Okay. Hopefully that won't be too painful to add if we ever need it in the future.
other, or is it sufficient to just define a lowest common denominator format and if your hardware can't deal with that format, you just don't get to share buffers?
At the moment I'm happy to just go with linear, minimum pitch alignment 64 or
256, for us.
something as a base standard, but yeah I'm happy for it to work either way, just don't have enough evidence it's worth it yet. I've not looked at ARM stuff, so patches welcome if people consider they need to use this stuff for SoC devices.
We can always hack it to whatever is necessary if we see that the sink side driver is Tegra, but I was hoping for something more general.
- There's no fallback mechanism if sharing can't be negotiated
If RandR fails to share a pixmap with the output sink screen, the whole modeset fails. This means you'll end up not seeing anything on the screen and you'll probably think your computer locked up. Should there be some sort of software copy fallback to ensure that something at least shows up on the display?
Uggh, it would be fairly slow and unuseable, I'd rather they saw nothing, but again open to suggestions on how to make this work, since it might fail for other reasons and in that case there is still nothing a sw copy can do. What happens if the slave intel device just fails to allocate a pixmap, but yeah I'm willing to think about it a bit more when we have some reference implementations.
Just rolling back the modeset operation to whatever was working before would be a good start.
It's worse than that on my current laptop, though, since our driver sees a phantom CRT output and we happily start driving pixels to it that end up going nowhere. I'll need to think about what the right behavior is there since I don't know if we want to rely on an X client to make that configuration work.
- How should the memory be allocated?
In the prototype I threw together, I'm allocating the shared memory using shm_open and then exporting that as a dma-buf file descriptor using an ioctl I added to the kernel, and then importing that memory back into our driver through dma_buf_attach & dma_buf_map_attachment. Does it make sense for user-space programs to be able to export shmfs files like that? Should that interface go in DRM / GEM / PRIME instead? Something else? I'm pretty unfamiliar with this kernel code so any suggestions would be appreciated.
Your kernel driver should in theory be doing it all, if you allocate shared pixmaps in GTT accessible memory, then you need an ioctl to tell your kernel driver to export the dma buf to the fd handle. (assuming we get rid of the _GPL, which people have mentioned they are open to doing). We have handle->fd and fd->handle interfaces on DRM, you'd need something similiar on the nvidia kernel driver interface.
Okay, I can do that. We already have a mechanism for importing buffers allocated elsewhere so reusing that for shmfs and/or dma-buf seemed like a natural extension. I don't think adding a separate ioctl for exporting our own allocations will add too much extra code.
Yes for 4 some sort of fencing is being worked on by Maarten for other stuff but would be a pre-req for doing this, and also some devices don't want fullscreen updates, like USB, so doing flipped updates would have to be optional or negoitated. It makes sense for us as well since things like gnome-shell can do full screen pageflips and we have to do full screen dirty updates.
Right now my implementation has two sources of tearing:
1. The dGPU reads the vidmem primary surface asynchronously from its own rendering to it.
2. The iGPU fetches the shared surface for display asynchronously from the dGPU writing into it.
#1 I can fix within our driver. For #2, I don't want to rely on the dGPU being able to push complete frames over the bus during vblank in response to an iGPU fence trigger so I was thinking we would want double-buffering all the time. Also, I was hoping to set up a proper flip chain between the dGPU, the dGPU's DMA engine, and the Intel display engine so that for full-screen applications, glXSwapBuffers is stalled properly without relying on the CPU to schedule things. Maybe that's overly ambitious for now?
-- Aaron
On Tue, Sep 04, 2012 at 01:57:32PM -0700, Aaron Plattner wrote:
On 08/31/2012 08:00 PM, Dave Airlie wrote:
Yes for 4 some sort of fencing is being worked on by Maarten for other stuff but would be a pre-req for doing this, and also some devices don't want fullscreen updates, like USB, so doing flipped updates would have to be optional or negoitated. It makes sense for us as well since things like gnome-shell can do full screen pageflips and we have to do full screen dirty updates.
Right now my implementation has two sources of tearing:
The dGPU reads the vidmem primary surface asynchronously from its own rendering to it.
The iGPU fetches the shared surface for display asynchronously from the dGPU writing into it.
#1 I can fix within our driver. For #2, I don't want to rely on the dGPU being able to push complete frames over the bus during vblank in response to an iGPU fence trigger so I was thinking we would want double-buffering all the time. Also, I was hoping to set up a proper flip chain between the dGPU, the dGPU's DMA engine, and the Intel display engine so that for full-screen applications, glXSwapBuffers is stalled properly without relying on the CPU to schedule things. Maybe that's overly ambitious for now?
For the frontbuffer tearing Chris Wilson added a special mode to the SNA intel driver that uses pageflips for all buffer updates (like windowized Xv or dri2copybuffers), mostly because vsync'ed blits are busted on snb (and not yet proved to be fixed on ivb). So we could use that mode for an optimus platform.
Wrt the full flip-chain, that's what Maarten Lankhorst has running in his proof-of-concept (but only for a second or so, since nouveau is totally bust on his machine). The only place he wakes up the cpu is to sync from nv to intel, but even there we can kick of the intel gpu directly from the nv irq handler (with a simple register write). intel -> nv sync uses memory based sequence numbers. Only proof of concept for rendering though, iirc the fence support isn't wired up with the pageflipping on the intel side yet. -Daniel
dri-devel@lists.freedesktop.org
-
Aaron Plattner -
Daniel Vetter -
Dave Airlie