Hi Greg, at FOSDEM we had a session on CDF (Common Display Framework). You can read more details about this in the posts from Laurent Pinchart on dridevel: http://lists.freedesktop.org/archives/dri-devel/2012-November/030888.html http://lists.freedesktop.org/archives/dri-devel/2013-February/034576.html Adding Arnd in CC since he has guided me in the device driver model before.
One design issue came up and Rob/Daniel proposed asking you for guidance. Here is some background to the issue.
In CDF we are trying to get some common code between all the embedded panels (and maybe also some eDP/DVI/HDMI encoders). Target user is KMS encoder/connector "drivers" in DRM on ARM, Intel, SuperH (or others). CDF consist of two major APIs (dev ops). One API for display controller to use for controlling the display and one API for display driver to use to access the data/video/control bus master/host. Much like any device hooked into some kernel framework. Here is a small overview of the parts and APIs.
KMS API: display controller drv access ---> CDF API: Display/Encoder/Panel drv access ---> Data/Control bus API: bus drv (often the same drv as display controller - panel bus host)
The issue is related to the latter bus API. The bus is a combination of I2C/SPI/DBI/DSI/DPI busses. I2C and SPI I guess you know. DBI and a command version of DSI are MIPI standard command mode interfaces (DBI - Display Bus Interface, DSI - Display Serial Interface), which means they are combined control and data busses (much like I2C). Then there is DPI (Display Parallel Interface) and DSI video mode version which are MIPI standard video mode data busses. They are used to send pixel data to the display. DPI is a pure one directional data bus, but DSI can be used to send commands as well so it can be used as a control interface, even during video data streaming (it is a packet protocol).
Now we have some different types of panels which are attached on a combination of busses:
a) control:DBI, data:DBI b) control:I2C/SPI, data:I2C/SPI c) control:static setup, data:DPI d) control:I2C/SPI, data:DPI e) control:DSI, data:DSI f) control:I2C, data:DSI g) control:DSI+I2C, data:DSI
As you could guess, g) is our issue. We have a device family that has two control bus attachments and one data bus. The kernel device/driver model only supports a single parent device which is normally the bus device.
We will write drivers for all device types a-g implementing the CDF API. Those with only I2C/SPI bus attachemnt will of course be I2C drivers registered to CDF, same probably goes for DBI and DSI panels if we create a new bus type for them (if not - platform devices). But the CDF drivers still need some way to access the bus/host operations. So we will have to create an API for the exposed operations possible for the MIPI type busses (DBI/DSI/DPI), some control and some data bus related. For this problem we have discussed a few solutions, which is where we need your guidance:
1) Due to the fact there is no support for multiple parents in the device driver model and the fact that there are no DSI/DBI busses in the kernel, Tomi has come up with a sort of logical parent device for displays (see video source section, top section is "CDF API"): http://gitorious.org/linux-omap-dss2/linux/blobs/work/dss-dev-model-cdf/incl...
Pros: Simple, easy to implement, merging all bus types into one logical bus (simplicity) Cons: Diverging from device driver model, merging all bus types into one logical bus (scalability, maintainability), has to make a copy of many things already in device driver model (pm, enumeration, registration, relations, ...), solution only really needed for one special type (g)
2) Another solution discussed at FOSDEM is to make full use of the device driver model. This would include adding the MIPI DBI/DSI bus types to the kernel. Add a full featured DSI/DBI API for devices attached on those busses (functionally the same APIs as for video source, but split into the actual bus types). Have display controller/host implement DBI/DSI bus ops. And finally register display as DBI/DSI device on the respective bus. For displays of type g) this could mean having two devices, one I2C device and one DSI device (this is the major objection to using the device driver model).
Pros: Device driver model is well understood which will make it easier to understand for new CDF developers. Only g) type of panels will diverge from driver model (multi parents) Cons: Two struct device for g) type displays (one per bus), slightly more code to create the busses than a simple video source
3) Daniel V hinted that multiple parents (or multiple busses) is something that has been discussed as a limitation of device driver model before. And that maybe now was the time to fix that or at least sort out how to handle it.
Pros: If possible to solve, probably the way to go, since it will eliminate the need of 1) and 2) that where created to work around the driver model limitation with multi parent Cons: Out of scope for CDF project, needs device driver model resources to fix
4) A combination of 1) and 2). Use video source solution for type g) devices and device driver model for the rest.
Pros: See above, each CDF driver can choose the model that fits that driver the best Cons: See above, similar displays might have CDF drivers that look very different, still forces hosts to implement video source API
So, what option do you prefer if any? Any comment on 3) and what has been said so far on multiple bus HW designs?
/BR /Marcus
Hi Marcus,
On 2013-02-12 17:04, Marcus Lorentzon wrote:
Now we have some different types of panels which are attached on a combination of busses:
a) control:DBI, data:DBI b) control:I2C/SPI, data:I2C/SPI c) control:static setup, data:DPI d) control:I2C/SPI, data:DPI e) control:DSI, data:DSI f) control:I2C, data:DSI g) control:DSI+I2C, data:DSI
As you could guess, g) is our issue. We have a device family that has two control bus attachments and one data bus. The kernel device/driver model only supports a single parent device which is normally the bus device.
We will write drivers for all device types a-g implementing the CDF API. Those with only I2C/SPI bus attachemnt will of course be I2C drivers registered to CDF, same probably goes for DBI and DSI panels if we create a new bus type for them (if not - platform devices). But the CDF drivers still need some way to access the bus/host operations. So we will have to create an API for the exposed operations possible for the MIPI type busses (DBI/DSI/DPI), some control and some data bus related. For this problem we have discussed a few solutions, which is where we need your guidance:
- Due to the fact there is no support for multiple parents in the
device driver model and the fact that there are no DSI/DBI busses in the kernel, Tomi has come up with a sort of logical parent device for displays (see video source section, top section is "CDF API"): http://gitorious.org/linux-omap-dss2/linux/blobs/work/dss-dev-model-cdf/incl...
When I made that, I didn't even have in mind the case g).
I made it because I think we have issues with case f) also (and, well, in some sense we have issues with all cases. see below). If we have a full linux bus for DSI and DBI, I don't quite understand how we should manage f), because we have both I2C and DSI busses to which the display device should belong.
I also had these points in my mind why I chose the video_source approach in my version:
- The display busses are very specialized, point-to-point busses, so a real linux bus doesn't give very much, I think.
- You never have a video bus used only for control, for example, a panel controlled via DSI but video data sent via DPI. Yes, possible in theory, but that would be rather insane.
- We anyway need some kind of non-bus approach for simple video data busses like DPI. And if we have that, the slightly more complex video busses like DSI fit quite easily in.
- We need something to represent all the data busses (see below).
Pros: Simple, easy to implement, merging all bus types into one logical bus (simplicity) Cons: Diverging from device driver model, merging all bus types into one logical bus (scalability, maintainability), has to make a copy of many things already in device driver model (pm, enumeration, registration, relations, ...), solution only really needed for one special type (g)
It's not only for g). We need something similar for all cases. We need to represent the chain of display devices with something, which is based on the data busses. The control bus plays no role in this chain (except when the data and control busses happen to be the same).
My video_source really represents the data bus, but offers some extended features so that it also offers control bus operations for those video busses that have control and data in the same bus.
If we go for a full DSI/DBI linux bus, we still need something to represent the video bus. Then we'll have two separate entities for DSI control (the real bus) and for DSI data (video_source or similar), which in the end go via the same physical DSI bus.
Tomi
On 02/12/2013 04:53 PM, Tomi Valkeinen wrote:
On 2013-02-12 17:04, Marcus Lorentzon wrote:
Now we have some different types of panels which are attached on a combination of busses:
a) control:DBI, data:DBI b) control:I2C/SPI, data:I2C/SPI c) control:static setup, data:DPI d) control:I2C/SPI, data:DPI e) control:DSI, data:DSI f) control:I2C, data:DSI g) control:DSI+I2C, data:DSI
As you could guess, g) is our issue. We have a device family that has two control bus attachments and one data bus. The kernel device/driver model only supports a single parent device which is normally the bus device.
We will write drivers for all device types a-g implementing the CDF API. Those with only I2C/SPI bus attachemnt will of course be I2C drivers registered to CDF, same probably goes for DBI and DSI panels if we create a new bus type for them (if not - platform devices). But the CDF drivers still need some way to access the bus/host operations. So we will have to create an API for the exposed operations possible for the MIPI type busses (DBI/DSI/DPI), some control and some data bus related. For this problem we have discussed a few solutions, which is where we need your guidance:
- Due to the fact there is no support for multiple parents in the
device driver model and the fact that there are no DSI/DBI busses in the kernel, Tomi has come up with a sort of logical parent device for displays (see video source section, top section is "CDF API"): http://gitorious.org/linux-omap-dss2/linux/blobs/work/dss-dev-model-cdf/incl...
When I made that, I didn't even have in mind the case g).
I made it because I think we have issues with case f) also (and, well, in some sense we have issues with all cases. see below). If we have a full linux bus for DSI and DBI, I don't quite understand how we should manage f), because we have both I2C and DSI busses to which the display device should belong.
You are right that any display where control bus is different from data bus have the same issue of two parents/busses.
I also had these points in my mind why I chose the video_source approach in my version:
- The display busses are very specialized, point-to-point busses, so a
real linux bus doesn't give very much, I think.
Maybe in most cases, like a modern mobile phone. But for DBI and DSI you can have many devices or endpoints on the same bus. On DSI you can even probe for devices. So I don't think they are that far off Linux busses. Agreed that DPI is not one of those, but even that can be used for multiple panels with a scanline buffered panel and scanline interleaving (think interlaced). But the latter DPI is not a use case I think we should target, it could be simulated as two point to point connections. In all cases I would like to use normal pm, enumeration etc, but then we need a way to describe multiple parents so I2C is not turned off before DPI etc. Not all platforms have dedicated I2C master under graphics sub system control. And even point-to-point will have a parent. A DPI bus could be extremely simple, the bus type struct, the dispc as bus device (display parent), and some DPI ops for the device, not the parent device. I assume video sources is a point to point reference (maybe video pipe is more correct), that is, it encapsulates both the host and the client in one object, since there is no reference to the actual device you operate on in the video source ops.
- You never have a video bus used only for control, for example, a panel
controlled via DSI but video data sent via DPI. Yes, possible in theory, but that would be rather insane.
Agreed, insane :)
- We anyway need some kind of non-bus approach for simple video data
busses like DPI. And if we have that, the slightly more complex video busses like DSI fit quite easily in.
Agreed, but that doesn't mean we need to bake them all into one. There is very little overlap between the different bus APIs.
- We need something to represent all the data busses (see below).
Pros: Simple, easy to implement, merging all bus types into one logical bus (simplicity) Cons: Diverging from device driver model, merging all bus types into one logical bus (scalability, maintainability), has to make a copy of many things already in device driver model (pm, enumeration, registration, relations, ...), solution only really needed for one special type (g)
It's not only for g). We need something similar for all cases. We need to represent the chain of display devices with something, which is based on the data busses. The control bus plays no role in this chain (except when the data and control busses happen to be the same).
The idea is to keep things simple. If you have video bridge chips, you can create a "bridge-CDF" driver that aggregate the next device in the chain. That way we never expose the complexity in those special cases with chains of encoders. This would also be easy to describe using the dev->parent relationship.
My video_source really represents the data bus, but offers some extended features so that it also offers control bus operations for those video busses that have control and data in the same bus.
For MIPI type interfaces yes. But not for I2C+DSI for example. I think it is this idea to handle some interfaces differently from others that my pattern matching brain don't accept ;).
If we go for a full DSI/DBI linux bus, we still need something to represent the video bus. Then we'll have two separate entities for DSI control (the real bus) and for DSI data (video_source or similar), which in the end go via the same physical DSI bus.
That depends on what ops we put in the bus. If we put both data and control ops as bus ops, we don't need that. The only thing I don't like with the full bus solutions is the idea of having two struct device for one physical device. But that is only for displays with multiple busses, which are kind of rare compared to the main target embedded panels which normally only use DSI/DBI bus for both data and control. Of course there are some DPI panels which would need dual busses. But most DPI panels I have used in products (if not all) never use I2C at all, only power on/off regulator and nothing more.
BTW. It should be noted that my own SoC driver use a design similar to "video source" currently :). Except it is called port in driver and formatter in dispc HW.
/BR /Marcus
On 2013-02-12 19:06, Marcus Lorentzon wrote:
Maybe in most cases, like a modern mobile phone. But for DBI and DSI you can have many devices or endpoints on the same bus. On DSI you can even
DBI supports two devices and DSI supports four, if I recall right. But for DSI you need an external hub anyway, so one could argue that it's still a point to point bus.
Anyway, I have not seen a board that uses those features. Do you have those?
probe for devices. So I don't think they are that far off Linux busses.
Hmm, you can probe similarly with DBI also, can't you? In neither case the bus itself supports probing, but you just have to send read-id messages, and handle the possible errors. This is what you mean, wasn't it?
I'm not sure if DSI probing gives meaningful values for all cases, though, at least as a generic probing (versus board specific probing of a few pre-defined devices). Some devices respond to multiple DSI-IDs, each of which handle different parts of the device (like, one goes to framebuffer management, other core control, etc.).
Probing over those busses is also something I have not seen used. Have you?
Agreed that DPI is not one of those, but even that can be used for multiple panels with a scanline buffered panel and scanline interleaving (think interlaced). But the latter DPI is not a use case I think we should target, it could be simulated as two point to point connections.
Well, I guess you can turn any one-to-one bus to a one-to-many if you hack enough =).
Anyway, DPI is not a control bus (ok, you _could_ send control messages embedded in the video data, but again, hacking), so there shouldn't be a DPI bus in any of the approaches we're thinking about. Or are you suggesting having linux busses for plain one way data busses also?
In all cases I would like to use normal pm, enumeration etc, but then we need a way to describe multiple parents so I2C is not turned off before DPI etc. Not all platforms have dedicated I2C master under graphics sub
Yes, the PM is an issue. But isn't it an issue with multiple busses also? Who says the DPI would be turned off first?
My view to this has been that the control bus side is in control (yay!). So, for example, DPI never turns off except if turned off explicitly by the display device driver, most likely via some kind of disable call from the higher level driver which manages the whole display chain.
In many cases we need to do the disabling of the device in a controlled manner, using both control and video busses. Say, first send a message via control bus to blank the panel. Then turn off the video bus. Then turn off the control bus.
But yes, there's still the issue to get the order right, so PM needs thinking.
system control. And even point-to-point will have a parent. A DPI bus could be extremely simple, the bus type struct, the dispc as bus device (display parent), and some DPI ops for the device, not the parent
Ok, I see you're thinking DPI as a linux bus also. Having linux busses for plain video data busses also would make things coherent, but it would mean that almost all display devices would need to be on multiple busses, right?
device. I assume video sources is a point to point reference (maybe video pipe is more correct), that is, it encapsulates both the host and the client in one object, since there is no reference to the actual device you operate on in the video source ops.
In my approach the video source (the host you mention above, I guess) doesn't need to know anything about the video sink (client, I guess). So I didn't need to store any references on the video source.
In the future we possibly do need that, so I imagine there would be something like video_pipe, which has references to both ends.
The idea is to keep things simple. If you have video bridge chips, you can create a "bridge-CDF" driver that aggregate the next device in the chain. That way we never expose the complexity in those special cases with chains of encoders. This would also be easy to describe using the dev->parent relationship.
So when you add a new board, which uses display devices that already have ready drivers in the kernel, you still need to create a new, board specific bridge driver that describes how the sub-devices are used and connected? I'm not very fond of that idea.
My video_source really represents the data bus, but offers some extended features so that it also offers control bus operations for those video busses that have control and data in the same bus.
For MIPI type interfaces yes. But not for I2C+DSI for example. I think it is this idea to handle some interfaces differently from others that my pattern matching brain don't accept ;).
Do you mean I2C+DSI as a control bus? Or I2C as control and DSI as data?
I wonder if the dual control bus case is something we should even target at? It sounds to me rather case specific optimization, not something that's really needed (at least for the v1). I have boards that have devices that can be controlled via I2C or DSI, but I've just used either one.
If we go for a full DSI/DBI linux bus, we still need something to represent the video bus. Then we'll have two separate entities for DSI control (the real bus) and for DSI data (video_source or similar), which in the end go via the same physical DSI bus.
That depends on what ops we put in the bus. If we put both data and control ops as bus ops, we don't need that. The only thing I don't like with the full bus solutions is the idea of having two struct device for one physical device. But that is only for displays with multiple busses, which are kind of rare compared to the main target embedded panels which normally only use DSI/DBI bus for both data and control. Of course there are some DPI panels which would need dual busses. But most DPI panels I have used in products (if not all) never use I2C at all, only power on/off regulator and nothing more.
I think I2C or SPI is quite common way to control DPI panels. I've encountered many.
BTW. It should be noted that my own SoC driver use a design similar to "video source" currently :). Except it is called port in driver and formatter in dispc HW.
Ok, interesting =).
Anyway, note that I'm not trying to say that the full-bus approach doesn't work. My "brain just doesn't accept it" as you said above, which could be because I'm so accustomed to a certain way of seeing this whole display bus thing. So please continue with this approach, and prove my worries wrong =).
Tomi
On Tue, Feb 12, 2013 at 04:04:53PM +0100, Marcus Lorentzon wrote:
- Daniel V hinted that multiple parents (or multiple busses) is
something that has been discussed as a limitation of device driver model before. And that maybe now was the time to fix that or at least sort out how to handle it.
No, it's the other way around, we have discussed ways about having multiple drivers control a single device at the same time. The "multiple parents" issue has come up a number of times with the power management people, but they solved this by keeping a separate tree of how to properly control and walk things to handle power domains and the like.
sorry,
greg k-h
Den 12 feb 2013 23:02 skrev "Greg KH" gregkh@linuxfoundation.org:
On Tue, Feb 12, 2013 at 04:04:53PM +0100, Marcus Lorentzon wrote:
- Daniel V hinted that multiple parents (or multiple busses) is
something that has been discussed as a limitation of device driver model before. And that maybe now was the time to fix that or at least sort out how to handle it.
No, it's the other way around, we have discussed ways about having multiple drivers control a single device at the same time. The "multiple parents" issue has come up a number of times with the power management people, but they solved this by keeping a separate tree of how to properly control and walk things to handle power domains and the like.
Thanks, does this mean there are no other devices in the kernel that sit on multiple busses that can be use as inspiration? Was the discussion about handling one device from multiple drivers concluded/implemented? It sounds very much what we are trying to do? Or this was just some f2f discussion?
BR Marcus Mobile
On Tue, Feb 12, 2013 at 11:20:04PM +0100, Marcus Lorentzon wrote:
Den 12 feb 2013 23:02 skrev "Greg KH" gregkh@linuxfoundation.org:
On Tue, Feb 12, 2013 at 04:04:53PM +0100, Marcus Lorentzon wrote:
- Daniel V hinted that multiple parents (or multiple busses) is
something that has been discussed as a limitation of device driver model before. And that maybe now was the time to fix that or at least sort out how to handle it.
No, it's the other way around, we have discussed ways about having multiple drivers control a single device at the same time. The "multiple parents" issue has come up a number of times with the power management people, but they solved this by keeping a separate tree of how to properly control and walk things to handle power domains and the like.
Thanks, does this mean there are no other devices in the kernel that sit on multiple busses that can be use as inspiration?
Lots of devices do this, but they all have an individual 'struct device' for the part that sits on each bus, with a single driver controlling both of them
Was the discussion about handling one device from multiple drivers concluded/ implemented? It sounds very much what we are trying to do? Or this was just some f2f discussion?
We've been talking about it off and on for many years now, and I think the framework in the driver core is almost there, but in reality, the different "types" of devices usually solve this problem, so we've never spent the time to do anything else here.
sorry,
greg k-h
On Tuesday 12 February 2013, Greg KH wrote:
On Tue, Feb 12, 2013 at 11:20:04PM +0100, Marcus Lorentzon wrote:
Den 12 feb 2013 23:02 skrev "Greg KH" gregkh@linuxfoundation.org:
On Tue, Feb 12, 2013 at 04:04:53PM +0100, Marcus Lorentzon wrote:
- Daniel V hinted that multiple parents (or multiple busses) is
something that has been discussed as a limitation of device driver model before. And that maybe now was the time to fix that or at least sort out how to handle it.
No, it's the other way around, we have discussed ways about having multiple drivers control a single device at the same time. The "multiple parents" issue has come up a number of times with the power management people, but they solved this by keeping a separate tree of how to properly control and walk things to handle power domains and the like.
Thanks, does this mean there are no other devices in the kernel that sit on multiple busses that can be use as inspiration?
Lots of devices do this, but they all have an individual 'struct device' for the part that sits on each bus, with a single driver controlling both of them
A typical example of this is an ethernet device where the MAC and PHY are on different buses. While we have some drivers that just bind to both devices, another solution is to have separate device drivers for each portion, and an interface to communicate between them.
It depends a bit on what the two bus interfaces are used for.
Arnd
dri-devel@lists.freedesktop.org
-
Arnd Bergmann -
Greg KH -
Marcus Lorentzon -
Marcus Lorentzon -
Tomi Valkeinen