Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this: 1) Two independent DRM drivers, i.e. modesetting and render node drivers 2) A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
- Existing user space is not fully prepared to handle render nodes.
- There is concern that any IMG DRM render node driver will need knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Thanks Frank
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need
knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the
component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Rule of thumb is that if it's an externally licensed IP block it should be a separate driver. Which is the case here. The idea is that the mostly generic IMG driver could be reused on other platforms that ship the same IP-block, while linking up with the respective display controller driver. The end result is 2 drm drivers: - Display block drm driver which expose KMS objects for modesetting, but only very basic gem (just enough to allocate dumb framebuffers and import/export dma-bufs). - Full-blown gem driver for the img render IP block.
For an example look at the tegra/nouveau combo which can run on TK1.
Plugggin in an IMG driver into each display block like it's currently done with all the armsoc stuff on android is imo completely no-go.
Note that the component interface is completely irrelevant wrt the interface you expose to userspace. It's just an driver-internal helper library useful in certain situation. Not even the drm core really cares whether you use component helpers or not.
Thanks, Daniel
+Puneet,+Zel
We will get back to you guys with answers soon. On Dec 1, 2014 9:28 AM, "Daniel Vetter" daniel@ffwll.ch wrote:
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need
knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the
component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Rule of thumb is that if it's an externally licensed IP block it should be a separate driver. Which is the case here. The idea is that the mostly generic IMG driver could be reused on other platforms that ship the same IP-block, while linking up with the respective display controller driver. The end result is 2 drm drivers:
- Display block drm driver which expose KMS objects for modesetting, but only very basic gem (just enough to allocate dumb framebuffers and import/export dma-bufs).
- Full-blown gem driver for the img render IP block.
For an example look at the tegra/nouveau combo which can run on TK1.
Plugggin in an IMG driver into each display block like it's currently done with all the armsoc stuff on android is imo completely no-go.
Note that the component interface is completely irrelevant wrt the interface you expose to userspace. It's just an driver-internal helper library useful in certain situation. Not even the drm core really cares whether you use component helpers or not.
Thanks, Daniel
Daniel Vetter Software Engineer, Intel Corporation +41 (0) 79 365 57 48 - http://blog.ffwll.ch
On 01/12/14 15:28, Daniel Vetter wrote:
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need
knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the
component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Rule of thumb is that if it's an externally licensed IP block it should be a separate driver. Which is the case here. The idea is that the mostly generic IMG driver could be reused on other platforms that ship the same IP-block, while linking up with the respective display controller driver. The end result is 2 drm drivers:
- Display block drm driver which expose KMS objects for modesetting, but only very basic gem (just enough to allocate dumb framebuffers and import/export dma-bufs).
- Full-blown gem driver for the img render IP block.
For an example look at the tegra/nouveau combo which can run on TK1.
Plugggin in an IMG driver into each display block like it's currently done with all the armsoc stuff on android is imo completely no-go.
Note that the component interface is completely irrelevant wrt the interface you expose to userspace. It's just an driver-internal helper library useful in certain situation. Not even the drm core really cares whether you use component helpers or not.
Thanks, Daniel
OK, so it seems the consensus is that IMG should provide a separate render-node only DRM driver.
Having not worked directly on the core DRM code I'm not completely familiar with it but it seems to me that the DRIVER_MODESET flag has a dual meaning. Firstly it means that the driver supports KMS and secondly it means that a lot of the legacy stuff isn't supported. It also changes the way in which driver initialisation is performed. Would it make sense for the DRIVER_RENDER flag to have a similar effect? In other words, should it turn off legacy stuff and use the newer method of driver initialisation?
Thanks Frank
On Tue, Dec 02, 2014 at 09:21:51AM +0000, Frank Binns wrote:
On 01/12/14 15:28, Daniel Vetter wrote:
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need
knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the
component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Rule of thumb is that if it's an externally licensed IP block it should be a separate driver. Which is the case here. The idea is that the mostly generic IMG driver could be reused on other platforms that ship the same IP-block, while linking up with the respective display controller driver. The end result is 2 drm drivers:
- Display block drm driver which expose KMS objects for modesetting, but only very basic gem (just enough to allocate dumb framebuffers and import/export dma-bufs).
- Full-blown gem driver for the img render IP block.
For an example look at the tegra/nouveau combo which can run on TK1.
Plugggin in an IMG driver into each display block like it's currently done with all the armsoc stuff on android is imo completely no-go.
Note that the component interface is completely irrelevant wrt the interface you expose to userspace. It's just an driver-internal helper library useful in certain situation. Not even the drm core really cares whether you use component helpers or not.
Thanks, Daniel
OK, so it seems the consensus is that IMG should provide a separate render-node only DRM driver.
Having not worked directly on the core DRM code I'm not completely familiar with it but it seems to me that the DRIVER_MODESET flag has a dual meaning. Firstly it means that the driver supports KMS and secondly it means that a lot of the legacy stuff isn't supported. It also changes the way in which driver initialisation is performed. Would it make sense for the DRIVER_RENDER flag to have a similar effect? In other words, should it turn off legacy stuff and use the newer method of driver initialisation?
DRIVER_MODESET means you have a modern driver which binds to the device. We should probably RENAME it to DRIVER_LEGACY and invert it's sense, but no one has stepped up to the taks.
DRIVER_RENDER just means you support render nodes. And new drm driver which exposes driver-specific ioctls should set this flag. Again we should probably rename this to DRIVER_IOCTLS_BROKEN_CANT_DO_RENDER and invert the sense ;-) Ofc with a better neame.
So for the IMG render-only driver you'd need to set both. Both flags just hide legacy behaviour if not set, but that's the way you need to do it without changing every driver. -Daniel
On Tue, Dec 02, 2014 at 04:10:15PM +0100, Daniel Vetter wrote:
On Tue, Dec 02, 2014 at 09:21:51AM +0000, Frank Binns wrote:
On 01/12/14 15:28, Daniel Vetter wrote:
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need
knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
- There is a trend, at least for DRM SoC drivers, towards using the
component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Rule of thumb is that if it's an externally licensed IP block it should be a separate driver. Which is the case here. The idea is that the mostly generic IMG driver could be reused on other platforms that ship the same IP-block, while linking up with the respective display controller driver. The end result is 2 drm drivers:
- Display block drm driver which expose KMS objects for modesetting, but only very basic gem (just enough to allocate dumb framebuffers and import/export dma-bufs).
- Full-blown gem driver for the img render IP block.
For an example look at the tegra/nouveau combo which can run on TK1.
Plugggin in an IMG driver into each display block like it's currently done with all the armsoc stuff on android is imo completely no-go.
Note that the component interface is completely irrelevant wrt the interface you expose to userspace. It's just an driver-internal helper library useful in certain situation. Not even the drm core really cares whether you use component helpers or not.
Thanks, Daniel
OK, so it seems the consensus is that IMG should provide a separate render-node only DRM driver.
Having not worked directly on the core DRM code I'm not completely familiar with it but it seems to me that the DRIVER_MODESET flag has a dual meaning. Firstly it means that the driver supports KMS and secondly it means that a lot of the legacy stuff isn't supported. It also changes the way in which driver initialisation is performed. Would it make sense for the DRIVER_RENDER flag to have a similar effect? In other words, should it turn off legacy stuff and use the newer method of driver initialisation?
DRIVER_MODESET means you have a modern driver which binds to the device. We should probably RENAME it to DRIVER_LEGACY and invert it's sense, but no one has stepped up to the taks.
Actually I did[0] a while ago. This is the second time that this has come up within a month, so perhaps I should revive the series.
Thierry
On Mon, Dec 01, 2014 at 10:01:37AM +0000, Frank Binns wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
There is a trend, at least for DRM SoC drivers, towards using the component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Writting a standalone KMS only drm driver for the whole modesetting side is fine. So is using DMA buffer to export/import buffer btw your KMS driver and your accel providing driver (two separate standalone driver). As long as you do not do anything as ugly as IMG have done in the past in which they hook up inside each drm driver and add there own set of ioctl on top.
A side note is that any IMG driver can not be accepted without open source user space and also a proper design. AFAICT current IMG driver design is unsuited for upstream given how unsecure it is. My understanding of it is that it's very easy for userspace to use IMG to read/write to any system memory.
So frankly i do not expect anything on the IMG side to be worth troubling us with as long as there is no open source userspace and proper secure driver design.
Cheers, Jérôme
Thanks Frank
On Mon, Dec 1, 2014 at 5:01 AM, Frank Binns frank.binns@imgtec.com wrote:
Hi,
We are currently in negotiations with one of our customers (Mediatek) on a strategy that will allow them to push a DRM modesetting driver into the upstream kernel. We are writing to get people's opinions and feedback on our proposed approach.
Currently, our driver is structured in such a way that the display driver is more tightly integrated with the GPU driver than we would like. Although our kernel driver has been shipped with a GPL license for a long time, it is not in a form that would be considered acceptable upstream. Unfortunately, it is going to be a long process to get this part of the driver into a reasonable state. However, in the meantime, we don't want to prevent customer portions of the driver from being upstreamed. With the work done on recent kernels, and with a willing partner in Mediatek, we now think that we can restructure our driver in such a way as to allow this to happen.
We see two basic approaches to achieving this:
- Two independent DRM drivers, i.e. modesetting and render node drivers
- A single componentised DRM driver
As others have said, a render-node kms driver is the way to go, so that the various KMS drivers it works with can be themselves upstreamed.
In the past, I had gone (more or less) the second approach w/ omapdrm but that required carrying some downstream omapdrm patches for the integration with gpu. This was, because at the time there was really no alternative. But now that we have dmabuf/prime and fence stuff in place, there is no more a need to do that. So the best approach is one that allows the display/kms drivers to be used without downstream patches.
Ofc, it would be nice to see an upstream-worthy driver for the gpu side of things, but as others have noted, upstream kernel drivers require *some* form of open source userspace so that we actually have a proper way to review and test the code.
BR, -R
Our (IMG's) preferred approach is to have a single componentised DRM driver. This is due to the following reasons:
Existing user space is not fully prepared to handle render nodes.
There is concern that any IMG DRM render node driver will need knowledge about multiple SoCs, each one being from a different vendor. Would this be deemed acceptable?
There is a trend, at least for DRM SoC drivers, towards using the component interface. Although there appears to be very few (one?) examples of GPU component drivers.
To give some high level details on how we expect the componentised DRM driver model to work, each vendor (in this case Mediatek) will write their own DRM driver (supporting modesetting, dumb buffers, GEM, prime, etc) and IMG will provide an almost entirely independent component driver that adds in GPU support. Until our GPU driver is in a suitable state this will most likely necessitate a small kernel patch to wire up support, e.g. GPU specific ioctls.
Cross-device and cross-process memory allocations will be made using the DRM driver. In order for this memory to be shared with the GPU component driver it will be necessary, at least for the time being, to export it via prime and import it via a GPU ioctl. Synchronisation between the display and GPU will be performed via reservation objects.
Does this sound like a sane approach? Questions and/or feedback is very welcome.
Thanks Frank
dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel
dri-devel@lists.freedesktop.org
-
Alberto Martin -
Daniel Vetter -
Frank Binns -
Jerome Glisse -
Rob Clark -
Thierry Reding