Hi,

On 2012-11-22 23:45, Laurent Pinchart wrote:

From: Laurent Pinchart laurent.pinchart+renesas@ideasonboard.com

Signed-off-by: Laurent Pinchart laurent.pinchart@ideasonboard.com

drivers/video/display/Kconfig | 4 + drivers/video/display/Makefile | 1 + drivers/video/display/mipi-dbi-bus.c | 228 ++++++++++++++++++++++++++++++++++ include/video/display.h | 5 + include/video/mipi-dbi-bus.h | 125 +++++++++++++++++++ 5 files changed, 363 insertions(+), 0 deletions(-) create mode 100644 drivers/video/display/mipi-dbi-bus.c create mode 100644 include/video/mipi-dbi-bus.h

I've been doing some omapdss testing with CDF and DSI, and I have some thoughts about the bus stuff. I already told these to Laurent, but I'll write them to the mailing list also for discussion.

So with the current CDF model we have separate control and video buses. The control bus is represented as proper Linux bus, and video bus is represented via custom display_entity. This sounds good on paper, and I also agreed to this approach when we were planning CDF.

However, now I doubt that approach.

First, I want to list some examples of devices with different bus configurations:

1) Panel without any control, only video bus 2) Panel with separate control and video buses, e.g. i2c for control, DPI for video 3) Panel with the same control and video buses, like DSI or DBI.

The first one is simple, it's just a platform device. No questions there.

The second one can be a bit tricky. Say, if we have a panel controlled via i2c, and DSI/DBI used for video. The problem here is that with the current model, DSI/DBI would be represented as a real bus, for control. But in this case there's only the video path.

So if all the DSI/DBI bus configuration is handled on the DSI/DBI control bus side, how can it be handled with only the video bus? And if we add the same bus configuration to the video bus side as we have on control bus side, then we have duplicated the API, and it's also somewhat confusing. I don't have any good suggestion for this.

Third one is kinda clear, but I feel slightly uneasy about it. In theory we can have separate control and video buses, which use the same HW transport. However, I feel that we'll have some trouble with the implementation, as we'll then have two more or less independent users for the HW transport. I can't really point out why this would not be possible to implement, but I have a gut feeling that it will be difficult, at least for DSI.

So I think my question is: what does it give us to have separate control and video buses, and what does the Linux bus give us with the control bus?

I don't see us ever having a case where a device would use one of the display buses only for control. So either the display bus is only used for video, or it's used for both control and video. And the display bus is always 1-to-1, so we're talking about really simple bus here.

I believe things would be much simpler if we just have one entity for the display buses, which support both video and (when available) control. What would be the downsides of this approach versus the current CDF proposal?

Tomi

Hi Tomi,

I finally have time to work on a v3 :-)

On Friday 23 November 2012 16:51:37 Tomi Valkeinen wrote:

On 2012-11-22 23:45, Laurent Pinchart wrote:

...

From: Laurent Pinchart laurent.pinchart+renesas@ideasonboard.com

Hi everybody,

Here's the second RFC of what was previously known as the Generic Panel Framework.

Nice work! Thanks for working on this.

I was doing some testing with the code, seeing how to use it in omapdss. Here are some thoughts:

In your model the DSS gets the panel devices connected to it from platform data. After the DSS and the panel drivers are loaded, DSS gets a notification and connects DSS and the panel.

I think it's a bit limited way. First of all, it'll make the DT data a bit more complex (although this is not a major problem). With your model, you'll need something like:

soc-base.dtsi:

dss { dpi0: dpi { }; };

board.dts:

&dpi0 { panel = &dpi-panel; };

/ { dpi-panel: dpi-panel { ...panel data...; }; };

Second, it'll prevent hotplug, and even if real hotplug would not be supported, it'll prevent cases where the connected panel must be found dynamically (like reading ID from eeprom).

Hotplug definitely needs to be supported, as the common display framework also targets HDMI and DP. The notification mechanism was actually designed to support hotplug.

How do you see the proposal preventing hotplug ?

Third, it kinda creates a cyclical dependency: the DSS needs to know about the panel and calls ops in the panel, and the panel calls ops in the DSS. I'm not sure if this is an actual problem, but I usually find it simpler if calls are done only in one direction.

I don't see any way around that. The panel is not a standalone entity that can only receive calls (as it needs to control video streams, per your request :-)) or only emit calls (as something needs to control it, userspace doesn't control the panel directly).

What I suggest is take a simpler approach, something alike to how regulators or gpios are used, even if slightly more complex than those: the entity that has a video output (SoC's DSS, external chips) offers that video output as resource. It doesn't know or care who uses it. The user of the video output (panel, external chips) will find the video output (to which it is connected in the HW) by some means, and will use different operations on that output to operate the device.

This would give us something like the following DT data:

soc-base.dtsi:

dss { dpi0: dpi { }; };

board.dts:

/ { dpi-panel: dpi-panel { source = <&dpi0>; ...panel data...; }; };

The panel driver would do something like this in its probe:

int dpi_panel_probe() { // Find the video source, increase ref src = get_video_source_from_of("source");

// Reserve the video source for us. others can still get and // observe it, but cannot use it as video data source. // I think this should cascade upstream, so that after this call // each video entity from the panel to the SoC's CRTC is // reserved and locked for this video pipeline. reserve_video_source(src);

// set DPI HW configuration, like DPI data lines. The // configuration would come from panel's platform data set_dpi_config(src, config);

// register this panel as a display. register_display(this); }

The DSS's dpi driver would do something like:

int dss_dpi_probe() { // register as a DPI video source register_video_source(this); }

A DSI-2-DPI chip would do something like:

int dsi2dpi_probe() { // get, reserve and config the DSI bus from SoC src = get_video_source_from_of("source"); reserve_video_source(src); set_dsi_config(src, config);

// register as a DPI video source register_video_source(this); }

Here we wouldn't have similar display_entity as you have, but video sources and displays. Video sources are elements in the video pipeline, and a video source is used only by the next downstream element. The last element in the pipeline would not be a video source, but a display, which would be used by the upper layer.

I don't think we should handle pure sources, pure sinks (displays) and mixed entities (transceivers) differently. I prefer having abstract entities that can have a source and a sink, and expose the corresponding operations. That would make pipeline handling much easier, as the code will only need to deal with a single type of object. Implementing support for entities with multiple sinks and/or sources would also be possible.

Video source's ops would deal with things related to the video bus in question, like configuring data lanes, sending DSI packets, etc. The display ops would be more high level things, like enable, update, etc. Actually, I guess you could consider the display to represent and deal with the whole pipeline, while video source deals with the bus between two display entities.

What is missing in your proposal is an explanation of how the panel is controlled. What does your register_display() function register the display with, and what then calls the display operations ?

Hi Tomi,

On Monday 17 December 2012 17:29:15 Tomi Valkeinen wrote:

On 2012-12-17 16:36, Laurent Pinchart wrote:

...

On Friday 23 November 2012 16:51:37 Tomi Valkeinen wrote:

...

On 2012-11-22 23:45, Laurent Pinchart wrote:

...

From: Laurent Pinchart laurent.pinchart+renesas@ideasonboard.com

Hi everybody,

Here's the second RFC of what was previously known as the Generic Panel Framework.

Nice work! Thanks for working on this.

I was doing some testing with the code, seeing how to use it in omapdss. Here are some thoughts:

In your model the DSS gets the panel devices connected to it from platform data. After the DSS and the panel drivers are loaded, DSS gets a notification and connects DSS and the panel.

I think it's a bit limited way. First of all, it'll make the DT data a bit more complex (although this is not a major problem). With your model, you'll need something like:

soc-base.dtsi:

dss { dpi0: dpi { }; };

board.dts:

&dpi0 { panel = &dpi-panel; };

/ { dpi-panel: dpi-panel { ...panel data...; }; };

Second, it'll prevent hotplug, and even if real hotplug would not be supported, it'll prevent cases where the connected panel must be found dynamically (like reading ID from eeprom).

Hotplug definitely needs to be supported, as the common display framework also targets HDMI and DP. The notification mechanism was actually designed to support hotplug.

HDMI or DP hotplug may or may not be a different thing than what I talk about here. We may have two kinds of hotplug: real linux device hotplug, i.e. a linux device appears or is removed during runtime, or just a cable hotplug, handled inside a driver, which doesn't have any effect on the linux devices.

If we do implement HDMI and DP monitors with real linux drivers, then yes, we could use real hotplug. But we could as well have the monitor driver always registered, and just have a driver internal cable-hotplug system.

To be honest, I'm not sure if implementing real hotplug is easily possible, as we don't have real, probable (probe-able =) busses. So even if we'd get a hotplug event of a new display device, what kind of device would the bus master register? It has no way to know that.

I get your point.

My design goal is to handle both HDMI/DP and panels through a single hotplug interface. I believe it would be simpler for display controller drivers to handle all display entities with a common API instead of implementing support for HDMI/DP and panels separately. This would require real HDMI and DP monitor drivers. I share your concern, I don't know whether this can work in the end, the only way to find out will be to try it.

...

How do you see the proposal preventing hotplug ?

Well, probably it doesn't prevent. But it doesn't feel right to me.

Say, if we have a DPI panel, controlled via foo-bus, which has a probing mechanism. When the foo-bus master detects a new hardware device, it'll create linux device for it. The driver for this device will then be probed.

That's correct. That's how Linux handles devices, and I don't think we should diverge from that model without a very good reason to do so. In my understanding you agree with me here, could you please confirm that ?

In the probe function it should somehow register itself to the cdf, or perhaps the previous entity in the chain.

The panel driver would register the panel device to CDF in its probe function.

From a panel point of view I think we agree that two sets of operations exist.

- The panel control operations are called by an upper layer component (let's call it A) to control the panel (retrieve the list of modes, enable the panel, ...). That upper layer component will usually call the panel in response to a userspace request (that can go through several layers in the kernel before reaching the panel), but can also call it in response to a hotplug event, without userspace being involved.

- The panel calls video operations of the entity that provides it with a video stream (the video source entity, let's call it B) to configure and control the video bus.

A and B could be implemented in the same driver or in two separate drivers, but at the end of the day I don't think that matters much. A needs a reference to the panel, and the panel needs a reference to B, that's all we need to provide, regardless of whether A and B come from the same kernel module or not.

This sounds to me that the link is from the panel to the previous entity, not the other way around as you describe, and also the previous entity doesn't know of the panel entities.

The data flows from the video source to the panel (I'm 100% confident that we agree on that :-)), and the video source is controlled by the panel as per your request. The link is thus from the video source to the panel, but is controlled by the sink, not the source.

...

...

Third, it kinda creates a cyclical dependency: the DSS needs to know about the panel and calls ops in the panel, and the panel calls ops in the DSS. I'm not sure if this is an actual problem, but I usually find it simpler if calls are done only in one direction.

I don't see any way around that. The panel is not a standalone entity that can only receive calls (as it needs to control video streams, per your request :-)) or only emit calls (as something needs to control it, userspace doesn't control the panel directly).

Right, but as I see it, the destination of the panel's calls, and the source of the calls to panel are different things. The destination is the bus layer, dealing with the video signal being transferred. The source is a bit higher level thing, something that's controlling the display in general.

That's correct. They can both be implemented in the same driver, but they're different logical entities. (I actually think they should be implemented in the same driver, but that's not very relevant here.)

...

...

Here we wouldn't have similar display_entity as you have, but video sources and displays. Video sources are elements in the video pipeline, and a video source is used only by the next downstream element. The last element in the pipeline would not be a video source, but a display, which would be used by the upper layer.

I don't think we should handle pure sources, pure sinks (displays) and mixed entities (transceivers) differently. I prefer having abstract entities that can have a source and a sink, and expose the corresponding operations. That would make pipeline handling much easier, as the code will only need to deal with a single type of object. Implementing support for entities with multiple sinks and/or sources would also be possible.

Ok. I think having pure sources is simpler model, but it's true that if we need to iterate and study the pipeline during runtime, it's probably better to have single entities with multiple sources/sinks.

A pure source is an entity with a source pad only that only exposes source pad operations, I think the complexity to handle them from the panel point of view would roughly be the same (there might be an extra argument to a couple of functions with a pad number, but that's more or less it).

...

...

Video source's ops would deal with things related to the video bus in question, like configuring data lanes, sending DSI packets, etc. The display ops would be more high level things, like enable, update, etc. Actually, I guess you could consider the display to represent and deal with the whole pipeline, while video source deals with the bus between two display entities.

What is missing in your proposal is an explanation of how the panel is controlled. What does your register_display() function register the display with, and what then calls the display operations ?

In my particular case, the omapfb calls the display operations, which is the higher level "manager" for the whole display. So omapfb does calls both to the DSS side and to the panel side of the pipeline.

I agree that making calls to both ends is a bit silly, but then again, I think it also happens in your model, it's just hidden there.

That's probably the biggest difference between our models. Let's discuss it face to face tomorrow and hopefully come up with an agreement.

Hi Jani,

On Monday 17 December 2012 18:53:37 Jani Nikula wrote:

On Mon, 17 Dec 2012, Laurent Pinchart wrote:

...

On Friday 23 November 2012 16:51:37 Tomi Valkeinen wrote:

...

On 2012-11-22 23:45, Laurent Pinchart wrote:

...

From: Laurent Pinchart laurent.pinchart+renesas@ideasonboard.com

Hi everybody,

Here's the second RFC of what was previously known as the Generic Panel Framework.

Nice work! Thanks for working on this.

I was doing some testing with the code, seeing how to use it in omapdss. Here are some thoughts:

In your model the DSS gets the panel devices connected to it from platform data. After the DSS and the panel drivers are loaded, DSS gets a notification and connects DSS and the panel.

I think it's a bit limited way. First of all, it'll make the DT data a bit more complex (although this is not a major problem). With your model, you'll need something like:

soc-base.dtsi:

dss { dpi0: dpi { }; };

board.dts:

&dpi0 { panel = &dpi-panel; };

/ { dpi-panel: dpi-panel { ...panel data...; }; };

Second, it'll prevent hotplug, and even if real hotplug would not be supported, it'll prevent cases where the connected panel must be found dynamically (like reading ID from eeprom).

Hotplug definitely needs to be supported, as the common display framework also targets HDMI and DP. The notification mechanism was actually designed to support hotplug.

I can see the need for a framework for DSI panels and such (in fact Tomi and I have talked about it like 2-3 years ago already!) but what is the story for HDMI and DP? In particular, what's the relationship between DRM and CDF here? Is there a world domination plan to switch the DRM drivers to use this framework too? ;) Do you have some rough plans how DRM and CDF should work together in general?

There's always a world domination plan, isn't there ? :-)

I certainly want CDF to be used by DRM (or more accurately KMS). That's what the C stands for, common refers to sharing panel and other display entity drivers between FBDEV, KMS and V4L2.

I currently have no plan to expose CDF internals to userspace through the KMS API. We might have to do so later if the hardware complexity grows in such a way that finer control than what KMS provides needs to be exposed to userspace, but I don't think we're there yet. The CDF API will thus only be used internally in the kernel by display controller drivers. The KMS core might get functions to handle common display entity operations, but the bulk of the work will be in the display controller drivers to start with. We will then see what can be abstracted in KMS helper functions.

Regarding HDMI and DP, I imagine HDMI and DP drivers that would use the CDF API. That's just a thought for now, I haven't tried to implement them, but it would be nice to handle HDMI screens and DPI/DBI/DSI panels in a generic way.

Do you have thoughts to share on this topic ?

On 2012-12-19 16:57, Jani Nikula wrote:

It just seems to me that, at least from a DRM/KMS perspective, adding another layer (=CDF) for HDMI or DP (or legacy outputs) would be overengineering it. They are pretty well standardized, and I don't see there would be a need to write multiple display drivers for them. Each display controller has one, and can easily handle any chip specific requirements right there. It's my gut feeling that an additional framework would just get in the way. Perhaps there could be more common HDMI/DP helper style code in DRM to reduce overlap across KMS drivers, but that's another thing.

So is the HDMI/DP drivers using CDF a more interesting idea from a non-DRM perspective? Or, put another way, is it more of an alternative to using DRM? Please enlighten me if there's some real benefit here that I fail to see!

The use of CDF is an option, not something that has to be done. A DRM driver developer may use it if it gives benefit for him for that particular driver.

I don't know much about desktop display hardware, but I guess that using CDF would not really give much there. In some cases it could, if the IPs used on the graphics card are something that are used elsewhere also (sounds quite unlikely, though). In that case there could be separate drivers for the IPs.

And note that CDF is not really about the dispc side, i.e. the part that creates the video stream from pixels in the memory. It's more about the components after that, and how to connect those components.

For DSI panels (or DSI-to-whatever bridges) it's of course another story. You typically need a panel specific driver. And here I see the main point of the whole CDF: decoupling display controllers and the panel drivers, and sharing panel (and converter chip) specific drivers across display controllers. Making it easy to write new drivers, as there would be a model to follow. I'm definitely in favour of coming up with some framework that would tackle that.

Right. But if you implement drivers for DSI panels with CDF for, say, OMAP, I think it's simpler to use CDF also for HDMI/DP on OMAP. Otherwise it'll be a mishmash with two different models.

Tomi

On Wed, Dec 19, 2012 at 8:57 AM, Jani Nikula jani.nikula@linux.intel.com wrote:

Hi Laurent -

On Tue, 18 Dec 2012, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

...

Hi Jani,

On Monday 17 December 2012 18:53:37 Jani Nikula wrote:

...

I can see the need for a framework for DSI panels and such (in fact Tomi and I have talked about it like 2-3 years ago already!) but what is the story for HDMI and DP? In particular, what's the relationship between DRM and CDF here? Is there a world domination plan to switch the DRM drivers to use this framework too? ;) Do you have some rough plans how DRM and CDF should work together in general?

There's always a world domination plan, isn't there ? :-)

I certainly want CDF to be used by DRM (or more accurately KMS). That's what the C stands for, common refers to sharing panel and other display entity drivers between FBDEV, KMS and V4L2.

I currently have no plan to expose CDF internals to userspace through the KMS API. We might have to do so later if the hardware complexity grows in such a way that finer control than what KMS provides needs to be exposed to userspace, but I don't think we're there yet. The CDF API will thus only be used internally in the kernel by display controller drivers. The KMS core might get functions to handle common display entity operations, but the bulk of the work will be in the display controller drivers to start with. We will then see what can be abstracted in KMS helper functions.

Regarding HDMI and DP, I imagine HDMI and DP drivers that would use the CDF API. That's just a thought for now, I haven't tried to implement them, but it would be nice to handle HDMI screens and DPI/DBI/DSI panels in a generic way.

Do you have thoughts to share on this topic ?

It just seems to me that, at least from a DRM/KMS perspective, adding another layer (=CDF) for HDMI or DP (or legacy outputs) would be overengineering it. They are pretty well standardized, and I don't see there would be a need to write multiple display drivers for them. Each display controller has one, and can easily handle any chip specific requirements right there. It's my gut feeling that an additional framework would just get in the way. Perhaps there could be more common HDMI/DP helper style code in DRM to reduce overlap across KMS drivers, but that's another thing.

So is the HDMI/DP drivers using CDF a more interesting idea from a non-DRM perspective? Or, put another way, is it more of an alternative to using DRM? Please enlighten me if there's some real benefit here that I fail to see!

fwiw, I think there are at least a couple cases where multiple SoC's have the same HDMI IP block.

And, there are also external HDMI encoders (for example connected over i2c) that can also be shared between boards. So I think there will be a number of cases where CDF is appropriate for HDMI drivers. Although trying to keep this all independent of DRM (as opposed to just something similar to what drivers/gpu/i2c is today) seems a bit overkill for me. Being able to use the helpers in drm and avoiding an extra layer of translation seems like the better option to me. So my vote would be drivers/gpu/cdf.

BR, -R

For DSI panels (or DSI-to-whatever bridges) it's of course another story. You typically need a panel specific driver. And here I see the main point of the whole CDF: decoupling display controllers and the panel drivers, and sharing panel (and converter chip) specific drivers across display controllers. Making it easy to write new drivers, as there would be a model to follow. I'm definitely in favour of coming up with some framework that would tackle that.

BR, Jani. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

On Wed, Dec 19, 2012 at 9:37 AM, Tomi Valkeinen tomi.valkeinen@ti.com wrote:

On 2012-12-19 17:26, Rob Clark wrote:

...

And, there are also external HDMI encoders (for example connected over i2c) that can also be shared between boards. So I think there will be a number of cases where CDF is appropriate for HDMI drivers. Although trying to keep this all independent of DRM (as opposed to just something similar to what drivers/gpu/i2c is today) seems a bit overkill for me. Being able to use the helpers in drm and avoiding an extra layer of translation seems like the better option to me. So my vote would be drivers/gpu/cdf.

Well, we need to think about that. I would like to keep CDF independent of DRM. I don't like tying different components/frameworks together if there's no real need for that.

Also, something that Laurent mentioned in our face-to-face discussions: Some IPs/chips can be used for other purposes than with DRM.

He had an example of a board, that (if I understood right) gets video signal from somewhere outside the board, processes the signal with some IPs/chips, and then outputs the signal. So there's no framebuffer, and the image is not stored anywhere. I think the framework used in these cases is always v4l2.

The IPs/chips in the above model may be the exact same IPs/chips that are used with "normal" display. If the CDF was tied to DRM, using the same drivers for normal and these streaming cases would probably not be possible.

Well, maybe there is a way, but it really seems to be over-complicating things unnecessarily to keep CDF independent of DRM.. there will be a lot more traditional uses of CDF compared to one crazy use-case. So I don't really fancy making it more difficult than in needs to be for everyone.

Probably the thing to do is take a step back and reconsider that one crazy use-case. For example, KMS doesn't enforce that the buffer handled passed when you create a drm framebuffer object to scan out is a GEM buffer. So on that one crazy platform, maybe it makes sense to have a DRM/KMS display driver that takes a handle to identify which video stream coming from the capture end of the pipeline. Anyways, that is just an off-the-top-of-my-head idea, probably there are other options too.

BR, -R

Tomi

Hi Rob,

On Wednesday 19 December 2012 09:26:40 Rob Clark wrote:

On Wed, Dec 19, 2012 at 8:57 AM, Jani Nikula wrote:

...

On Tue, 18 Dec 2012, Laurent Pinchart wrote:

...

On Monday 17 December 2012 18:53:37 Jani Nikula wrote:

...

I can see the need for a framework for DSI panels and such (in fact Tomi and I have talked about it like 2-3 years ago already!) but what is the story for HDMI and DP? In particular, what's the relationship between DRM and CDF here? Is there a world domination plan to switch the DRM drivers to use this framework too? ;) Do you have some rough plans how DRM and CDF should work together in general?

There's always a world domination plan, isn't there ? :-)

I certainly want CDF to be used by DRM (or more accurately KMS). That's what the C stands for, common refers to sharing panel and other display entity drivers between FBDEV, KMS and V4L2.

I currently have no plan to expose CDF internals to userspace through the KMS API. We might have to do so later if the hardware complexity grows in such a way that finer control than what KMS provides needs to be exposed to userspace, but I don't think we're there yet. The CDF API will thus only be used internally in the kernel by display controller drivers. The KMS core might get functions to handle common display entity operations, but the bulk of the work will be in the display controller drivers to start with. We will then see what can be abstracted in KMS helper functions.

Regarding HDMI and DP, I imagine HDMI and DP drivers that would use the CDF API. That's just a thought for now, I haven't tried to implement them, but it would be nice to handle HDMI screens and DPI/DBI/DSI panels in a generic way.

Do you have thoughts to share on this topic ?

It just seems to me that, at least from a DRM/KMS perspective, adding another layer (=CDF) for HDMI or DP (or legacy outputs) would be overengineering it. They are pretty well standardized, and I don't see there would be a need to write multiple display drivers for them. Each display controller has one, and can easily handle any chip specific requirements right there. It's my gut feeling that an additional framework would just get in the way. Perhaps there could be more common HDMI/DP helper style code in DRM to reduce overlap across KMS drivers, but that's another thing.

So is the HDMI/DP drivers using CDF a more interesting idea from a non-DRM perspective? Or, put another way, is it more of an alternative to using DRM? Please enlighten me if there's some real benefit here that I fail to see!

fwiw, I think there are at least a couple cases where multiple SoC's have the same HDMI IP block.

And, there are also external HDMI encoders (for example connected over i2c) that can also be shared between boards. So I think there will be a number of cases where CDF is appropriate for HDMI drivers. Although trying to keep this all independent of DRM (as opposed to just something similar to what drivers/gpu/i2c is today) seems a bit overkill for me. Being able to use the helpers in drm and avoiding an extra layer of translation seems like the better option to me. So my vote would be drivers/gpu/cdf.

I don't think there will be any need for translation (except perhaps between the DRM mode structures and the common video mode structure that is being discussed). Add a drm_ prefix to the existing CDF functions and structures, and there you go :-)

The reason why I'd like to keep CDF separate from DRM (or at least not requiring a drm_device) is that HDMI/DP encoders can be used by pure V4L2 drivers.

...

For DSI panels (or DSI-to-whatever bridges) it's of course another story. You typically need a panel specific driver. And here I see the main point of the whole CDF: decoupling display controllers and the panel drivers, and sharing panel (and converter chip) specific drivers across display controllers. Making it easy to write new drivers, as there would be a model to follow. I'm definitely in favour of coming up with some framework that would tackle that.

Hi Jani,

On Wednesday 19 December 2012 16:57:56 Jani Nikula wrote:

On Tue, 18 Dec 2012, Laurent Pinchart wrote:

...

On Monday 17 December 2012 18:53:37 Jani Nikula wrote:

...

I can see the need for a framework for DSI panels and such (in fact Tomi and I have talked about it like 2-3 years ago already!) but what is the story for HDMI and DP? In particular, what's the relationship between DRM and CDF here? Is there a world domination plan to switch the DRM drivers to use this framework too? ;) Do you have some rough plans how DRM and CDF should work together in general?

There's always a world domination plan, isn't there ? :-)

I certainly want CDF to be used by DRM (or more accurately KMS). That's what the C stands for, common refers to sharing panel and other display entity drivers between FBDEV, KMS and V4L2.

I currently have no plan to expose CDF internals to userspace through the KMS API. We might have to do so later if the hardware complexity grows in such a way that finer control than what KMS provides needs to be exposed to userspace, but I don't think we're there yet. The CDF API will thus only be used internally in the kernel by display controller drivers. The KMS core might get functions to handle common display entity operations, but the bulk of the work will be in the display controller drivers to start with. We will then see what can be abstracted in KMS helper functions.

Regarding HDMI and DP, I imagine HDMI and DP drivers that would use the CDF API. That's just a thought for now, I haven't tried to implement them, but it would be nice to handle HDMI screens and DPI/DBI/DSI panels in a generic way.

Do you have thoughts to share on this topic ?

It just seems to me that, at least from a DRM/KMS perspective, adding another layer (=CDF) for HDMI or DP (or legacy outputs) would be overengineering it. They are pretty well standardized, and I don't see there would be a need to write multiple display drivers for them. Each display controller has one, and can easily handle any chip specific requirements right there. It's my gut feeling that an additional framework would just get in the way. Perhaps there could be more common HDMI/DP helper style code in DRM to reduce overlap across KMS drivers, but that's another thing.

So is the HDMI/DP drivers using CDF a more interesting idea from a non-DRM perspective? Or, put another way, is it more of an alternative to using DRM? Please enlighten me if there's some real benefit here that I fail to see!

As Rob pointed out, you can have external HDMI/DP encoders, and even internal HDMI/DP encoder IPs can be shared between SoCs and SoC vendors. CDF aims at sharing a single driver between SoCs and boards for a given HDMI/DP encoder.

CDF isn't an alternative to DRM/KMS. It should be seen as a framework that helps DRM/KMS drivers (as well as V4L2 drivers, and possibly FBDEV drivers, although those should be ported to DRM/KMS) sharing encoder and connector code.

For DSI panels (or DSI-to-whatever bridges) it's of course another story. You typically need a panel specific driver. And here I see the main point of the whole CDF: decoupling display controllers and the panel drivers, and sharing panel (and converter chip) specific drivers across display controllers. Making it easy to write new drivers, as there would be a model to follow. I'm definitely in favour of coming up with some framework that would tackle that.

That's the main (and original) goal of CDF (originally called Generic Panel Framwork, and renamed to CDF to support encoder drivers as explained above). I'm glad to know that you're in favour of it :-)

On Thu, Dec 27, 2012 at 10:04:22AM -0600, Rob Clark wrote:

On Mon, Dec 24, 2012 at 11:27 AM, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

...

On Wednesday 19 December 2012 16:57:56 Jani Nikula wrote:

...

It just seems to me that, at least from a DRM/KMS perspective, adding another layer (=CDF) for HDMI or DP (or legacy outputs) would be overengineering it. They are pretty well standardized, and I don't see there would be a need to write multiple display drivers for them. Each display controller has one, and can easily handle any chip specific requirements right there. It's my gut feeling that an additional framework would just get in the way. Perhaps there could be more common HDMI/DP helper style code in DRM to reduce overlap across KMS drivers, but that's another thing.

So is the HDMI/DP drivers using CDF a more interesting idea from a non-DRM perspective? Or, put another way, is it more of an alternative to using DRM? Please enlighten me if there's some real benefit here that I fail to see!

As Rob pointed out, you can have external HDMI/DP encoders, and even internal HDMI/DP encoder IPs can be shared between SoCs and SoC vendors. CDF aims at sharing a single driver between SoCs and boards for a given HDMI/DP encoder.

just fwiw, drm already has something a bit like this.. the i2c encoder-slave. With support for a couple external i2c encoders which could in theory be shared between devices.

The problem with this code is that it only works when the i2c device is registered by a master driver. Once the i2c device comes from the devicetree there is no possibility to find it.

Sascha

On 2012-11-23 21:56, Thierry Reding wrote:

On Thu, Nov 22, 2012 at 10:45:31PM +0100, Laurent Pinchart wrote: [...]

...

Display entities are accessed by driver using notifiers. Any driver can register a display entity notifier with the CDF, which then calls the notifier when a matching display entity is registered. The reason for this asynchronous mode of operation, compared to how drivers acquire regulator or clock resources, is that the display entities can use resources provided by the display driver. For instance a panel can be a child of the DBI or DSI bus controlled by the display device, or use a clock provided by that device. We can't defer the display device probe until the panel is registered and also defer the panel device probe until the display is registered. As most display drivers need to handle output devices hotplug (HDMI monitors for instance), handling other display entities through a notification system seemed to be the easiest solution.

Note that this brings a different issue after registration, as display controller and display entity drivers would take a reference to each other. Those circular references would make driver unloading impossible. One possible solution to this problem would be to simulate an unplug event for the display entity, to force the display driver to release the dislay entities it uses. We would need a userspace API for that though. Better solutions would of course be welcome.

Maybe I don't understand all of the underlying issues correctly, but a parent/child model would seem like a better solution to me. We discussed this back when designing the DT bindings for Tegra DRM and came to the conclusion that the output resource of the display controller (RGB, HDMI, DSI or TVO) was the most suitable candidate to be the parent of the panel or display attached to it. The reason for that decision was that it keeps the flow of data or addressing of nodes consistent. So the chain would look something like this (on Tegra):

CPU +-host1x +-dc +-rgb | +-panel +-hdmi +-monitor

In a natural way this makes the output resource the master of the panel or display. From a programming point of view this becomes quite easy to implement and is very similar to how other busses like I2C or SPI are modelled. In device tree these would be represented as subnodes, while with platform data some kind of lookup could be done like for regulators or alternatively a board setup registration mechanism like what's in place for I2C or SPI.

You didn't explicitly say it, but I presume you are talking about the device model for panels, not just how to refer to the outputs.

How would you deal with a, say, DPI panel that is controlled via I2C or SPI? You can have the panel device be both a panel device, child of a RGB output, and an i2c device.

The model you propose is currently used in omapdss, and while it seems simple and logical, it's not that simple with panels/chips with separate control and data busses.

I think it makes more sense to consider the device as a child of the control bus. So a DPI panel controlled via I2C is an I2C device, and it just happens to use a DPI video output as a resource (like it could use a regulator, gpio, etc).

Tomi

Hi Alan,

On Monday 26 November 2012 14:47:08 Alan Cox wrote:

On Sat, 24 Nov 2012 09:15:51 +0200 Tomi Valkeinen wrote:

...

On 2012-11-23 21:56, Thierry Reding wrote:

...

On Thu, Nov 22, 2012 at 10:45:31PM +0100, Laurent Pinchart wrote: [...]

...

Display entities are accessed by driver using notifiers. Any driver can register a display entity notifier with the CDF, which then calls the notifier when a matching display entity is registered.

The framebuffer layer has some similar 'anyone can' type notifier behaviour and its not a good thing. That kind of "any one can" behaviour leads to some really horrible messes unless the connections and the locking are well defined IMHO.

I agree with you. I dislike the FBDEV notifier model, and I definitely don't intend to duplicate it in the common display framework.

In the CDF model, when the display device driver registers a notifier, it tells the core which device it wants to receive events for. This currently takes the form of a struct device pointer, and the API will also support device nodes in a future version (this is still work in progress). The goal is to implement panel discovery in a way that is compatible with (and very similar to) hotpluggable display discovery.

Thinking about it now, the API could be cleaner and less subject to abuse if the notifier was registered for a given video port instead of a given connected device. I'll add that to my TODO list.

Hi Thierry,

On Friday 23 November 2012 20:56:07 Thierry Reding wrote:

On Thu, Nov 22, 2012 at 10:45:31PM +0100, Laurent Pinchart wrote: [...]

...

Display entities are accessed by driver using notifiers. Any driver can register a display entity notifier with the CDF, which then calls the notifier when a matching display entity is registered. The reason for this asynchronous mode of operation, compared to how drivers acquire regulator or clock resources, is that the display entities can use resources provided by the display driver. For instance a panel can be a child of the DBI or DSI bus controlled by the display device, or use a clock provided by that device. We can't defer the display device probe until the panel is registered and also defer the panel device probe until the display is registered. As most display drivers need to handle output devices hotplug (HDMI monitors for instance), handling other display entities through a notification system seemed to be the easiest solution.

Note that this brings a different issue after registration, as display controller and display entity drivers would take a reference to each other. Those circular references would make driver unloading impossible. One possible solution to this problem would be to simulate an unplug event for the display entity, to force the display driver to release the dislay entities it uses. We would need a userspace API for that though. Better solutions would of course be welcome.

Maybe I don't understand all of the underlying issues correctly, but a parent/child model would seem like a better solution to me. We discussed this back when designing the DT bindings for Tegra DRM and came to the conclusion that the output resource of the display controller (RGB, HDMI, DSI or TVO) was the most suitable candidate to be the parent of the panel or display attached to it. The reason for that decision was that it keeps the flow of data or addressing of nodes consistent. So the chain would look something like this (on Tegra):

CPU +-host1x +-dc +-rgb | +-panel +-hdmi +-monitor

In a natural way this makes the output resource the master of the panel or display. From a programming point of view this becomes quite easy to implement and is very similar to how other busses like I2C or SPI are modelled. In device tree these would be represented as subnodes, while with platform data some kind of lookup could be done like for regulators or alternatively a board setup registration mechanism like what's in place for I2C or SPI.

That works well for panels that have a shared control and video bus (DBI, DSI) or only a video bus (DPI), but breaks when you need to support panels with separate control and video busses, such as panels with a parallel data bus and an I2C or SPI control bus.

Both Linux and DT have a tree-based device model. Devices can have a single parent, so you can't represent your panel as a child of both the video source and the control bus master. We have the exact same problem in V4L2 with I2C camera sensors that output video data on a separate parallel or serial bus, and we decided to handle the device as a child of its control bus master. This model makes usage of the Linux power management model easier (but not straightforward when power management dependencies exist across video busses, outside of the kernel device tree).

As the common display framework should handle both panels with common control and video busses and panels with separate busses in a similar fashion, DT bindings needs to reference the panel through a phandle, even though in some cases they could technically just be children of the display controller.

Hi Sascha,

On Friday 23 November 2012 22:41:58 Sascha Hauer wrote:

On Thu, Nov 22, 2012 at 10:45:31PM +0100, Laurent Pinchart wrote:

...

From: Laurent Pinchart laurent.pinchart+renesas@ideasonboard.com

The CDF models this using a Russian doll's model. From the display controller point of view only the first external entity (LVDS to DSI converter) is visible. The display controller thus calls the control operations implemented by the LVDS to DSI transmitter driver (left-most green arrow). The driver is aware of the next entity in the chain,

I can't find this in the code. I can see the video operations propagating upstream using the source field of struct display_entity, but how do the control operations propagate downstream? Am I missing something?

There's no downstream propagation yet, as there's no display entity driver that requires it at the moment. Propagation would be implemented in transceiver drivers for instance. I'll have to find one with public documentation (and hopefully an existing mainline driver) on one of the boards I own.

On Mon, Dec 17, 2012 at 11:04 PM, Dave Airlie airlied@gmail.com wrote:

...

Many developers showed interest in the first RFC, and I've had the opportunity to discuss it with most of them. I would like to thank (in no particular order) Tomi Valkeinen for all the time he spend helping me to draft v2, Marcus Lorentzon for his useful input during Linaro Connect Q4 2012, and Linaro for inviting me to Connect and providing a venue to discuss this topic.

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

I tend to think that sub-devices are useful just to have a way to probe hw which may or may not be there, since on ARM we often don't have any alternative.. but beyond that, suspend/resume, and other life-cycle aspects, they should really be treated as all one device. Especially to avoid undefined suspend/resume ordering.

CDF or some sort of mechanism to share panel drivers between drivers is useful. Keeping it within drm, is probably a good idea, if nothing else to simplify re-use of helper fxns (like avi-infoframe stuff, for example) and avoid dealing with merging changes across multiple trees. Treating them more like shared libraries and less like sub-devices which can be dynamically loaded/unloaded (ie. they should be not built as separate modules or suspend/resumed or probed/removed independently of the master driver) is a really good idea to avoid uncovering nasty synchronization issues later (remove vs modeset or pageflip) or surprising userspace in bad ways.

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

yeah, let's not have multiple frameworks to do the same thing.. For fbdev, it is pretty clear that it is a dead end. For v4l2 (subdev+mcf), it is perhaps bit more flexible when it comes to random arbitrary hw pipelines than kms. But to take advantage of that, your userspace isn't going to be portable anyways, so you might as well use driver specific properties/ioctls. But I tend to think that is more useful for cameras. And from userspace perspective, kms planes are less painful to use for output than v4l2, so lets stick to drm/kms for output (and not try to add camera/capture support to kms).. k, thx

BR, -R

Dave. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel

2012/12/18 Daniel Vetter daniel@ffwll.ch

On Tue, Dec 18, 2012 at 7:21 AM, Rob Clark rob.clark@linaro.org wrote:

...

...

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

yeah, let's not have multiple frameworks to do the same thing.. For fbdev, it is pretty clear that it is a dead end. For v4l2 (subdev+mcf), it is perhaps bit more flexible when it comes to random arbitrary hw pipelines than kms. But to take advantage of that, your userspace isn't going to be portable anyways, so you might as well use driver specific properties/ioctls. But I tend to think that is more useful for cameras. And from userspace perspective, kms planes are less painful to use for output than v4l2, so lets stick to drm/kms for output (and not try to add camera/capture support to kms).. k, thx

Yeah, I guess having a v4l device also exported by the same driver that exports the drm interface might make sense in some cases. But in many cases I think the video part is just an independent IP block and shuffling data around with dma-buf is all we really need. So yeah, I guess sharing display resources between v4l and drm kms driver should be a last resort option, since coordination (especially if it's supposed to be somewhat dynamic) will be extremely hairy.

I think the one reason that the CDF was appeared is to avoid duplicating codes. For example, we should duplicate mipi-dsi or dbi drivers into drm to avoid ordering issue. And for this, those should be re-implemented in based on drm framework so that those could be treated as all one device. Actually, in case of Exynos, some guys tried to duplicate eDP driver into exynos drm framework in same issue. So I think the best way is to avoid duplicating codes and resolve ordering issue such as s/r operations between all the various components.

And the below is my opinion,

----------------------------------------------------- Display Controller-------------CDF---------------|MIPI-DSI/DBI---------------LCD Panel|

-----------------------------------------------------

1. to access MIPI-DSI/DBI and LCD Panel drivers. - Display Controller is controlled by linux framebuffer or drm kms based specific drivers like now. And each driver calls some interfaces of CDF.

2. to control the power of these devices. - drm kms based specific driver calls dpms operation and next the dpms operation calls fb blank operation of linux framebuffer. But for this, we need some interfaces that it can connect between drm and linux framebuffer framework and you can refer to the below link.

http://lists.freedesktop.org/archives/dri-devel/2011-July/013242.html - linux framebuffer based driver calls fb blank operation.

fb blank(fb)------------------pm runtime(fb)--------------------fb_blank----------mipi and lcd dpms(drm kms)------------pm runtime(drm kms)----------fb_blank----------mipi and lcd

3. suspend/resume - pm suspend/resume are implemented only in linux framebuffer or drm kms based specific drivers. - MIPI-DSI/DBI and LCD Panel drivers are controlled only by fb blank interfaces.

s/r(fb)-----------------------------------------------pm runtime(fb)----------------fb blank-------mipi and lcd s/r(drm kms)--------dpms(drm kms)-------pm runtime(drm kms)------fb_blank------mipi and lcd

We could resolve ordering issue to suspend/resume simply duplicating relevant drivers but couldn't avoid duplicating codes. So I think we could avoid the ordering issue using fb blank interface of linux framebuffer and also duplicating codes.

Thanks, Inki Dae

-Daniel

Daniel Vetter Software Engineer, Intel Corporation

+41 (0) 79 365 57 48 - http://blog.ffwll.ch

To unsubscribe from this list: send the line "unsubscribe linux-fbdev" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Hi Inki,

On Tuesday 18 December 2012 18:38:31 Inki Dae wrote:

2012/12/18 Daniel Vetter daniel@ffwll.ch

...

On Tue, Dec 18, 2012 at 7:21 AM, Rob Clark rob.clark@linaro.org wrote:

...

...

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

yeah, let's not have multiple frameworks to do the same thing.. For fbdev, it is pretty clear that it is a dead end. For v4l2 (subdev+mcf), it is perhaps bit more flexible when it comes to random arbitrary hw pipelines than kms. But to take advantage of that, your userspace isn't going to be portable anyways, so you might as well use driver specific properties/ioctls. But I tend to think that is more useful for cameras. And from userspace perspective, kms planes are less painful to use for output than v4l2, so lets stick to drm/kms for output (and not try to add camera/capture support to kms).. k, thx

Yeah, I guess having a v4l device also exported by the same driver that exports the drm interface might make sense in some cases. But in many cases I think the video part is just an independent IP block and shuffling data around with dma-buf is all we really need. So yeah, I guess sharing display resources between v4l and drm kms driver should be a last resort option, since coordination (especially if it's supposed to be somewhat dynamic) will be extremely hairy.

I think the one reason that the CDF was appeared is to avoid duplicating codes. For example, we should duplicate mipi-dsi or dbi drivers into drm to avoid ordering issue. And for this, those should be re-implemented in based on drm framework so that those could be treated as all one device. Actually, in case of Exynos, some guys tried to duplicate eDP driver into exynos drm framework in same issue. So I think the best way is to avoid duplicating codes and resolve ordering issue such as s/r operations between all the various components.

And the below is my opinion,

                                      +--------------------------------+

Display Controller -------- CDF --------- |MIPI-DSI/DBI-----------LCD Panel| +--------------------------------+

1. to access MIPI-DSI/DBI and LCD Panel drivers.
   • Display Controller is controlled by linux framebuffer or drm kms

based specific drivers like now. And each driver calls some interfaces of CDF.

2. to control the power of these devices.
   • drm kms based specific driver calls dpms operation and next the dpms

operation calls fb blank operation of linux framebuffer. But for this, we need some interfaces that it can connect between drm and linux framebuffer framework and you can refer to the below link.

http://lists.freedesktop.org/archives/dri-devel/2011-July/013242.html

(Just FYI, I plan to clean up the backlight framework when I'll be done with CDF, to remove the FBDEV dependency)

- linux framebuffer based driver calls fb blank operation.

fb blank(fb)---------pm runtime(fb)-----------fb_blank----------mipi and lcd dpms(drm kms)--------pm runtime(drm kms)------fb_blank----------mipi and lcd

3. suspend/resume
   • pm suspend/resume are implemented only in linux framebuffer or drm

kms based specific drivers. - MIPI-DSI/DBI and LCD Panel drivers are controlled only by fb blank interfaces.

s/r(fb)------------------------pm runtime(fb)--------fb blank---mipi and lcd s/r(drm kms)---dpms(drm kms)---pm runtime(drm kms)---fb_blank---mipi and lcd

We could resolve ordering issue to suspend/resume simply duplicating relevant drivers but couldn't avoid duplicating codes. So I think we could avoid the ordering issue using fb blank interface of linux framebuffer and also duplicating codes.

As I mentioned before, we have multiple ordering issues related to suspend and resume. Panels and display controllers will likely want to enforce a S/R order on the video bus, and control busses will also require a specific S/R order. My plan is to use early suspend/late resume in the display controller driver to control the video busses, and let the PM core handle control bus ordering issues. This will of course need to be prototyped and tested.

On 12/18/2012 07:21 AM, Rob Clark wrote:

On Mon, Dec 17, 2012 at 11:04 PM, Dave Airlieairlied@gmail.com wrote:

...

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

There have been already some ideas proposed to resolve this at the PM subsystem level [1]. And this problem is of course not only specific to platform drivers. The idea of having monolithic drivers, just because we can't get the suspend/resume sequences right otherwise, doesn't really sound appealing. SoC IPs get reused on multiple different SoC series, no only by single manufacturer. Whole graphics/video subsystems are composed from smaller blocks in SoCs, with various number of distinct sub-blocks and same sub-blocks repeated different number of times in a specific SoC revision. Expressing an IP as a platform device seems justified to me, often these platform devices have enough differences to treat them as such. E.g. belong in different power domain/use different clocks. Except there is big issue with the power management... However probably more important is to be able to have driver for a specific IP in a separate module.

And this suspend/resume ordering issue is not only about the platform devices. E.g. camera subsystem can be composed of an image sensor sub-device driver, which is most often an I2C client driver, and of multiple SoC processing blocks. The image sensor can have dependencies on the SoC sub-blocks. So even if we created monolithic driver for the SoC part, there are still two pieces to get s/r ordering right - I2C client and SoC drivers. And please don't propose to merge the sensor sub-device driver too. There has been a lot of effort in V4L2 to separate those various functional blocks into sub-devices, so they can be freely reused, without reimplementing same functionality in each driver. BTW, there has been a nice talk about these topics at ELCE [2], particularly slide 22 is interesting.

I believe the solution for these issues really needs to be sought in the PM subsystem itself.

I tend to think that sub-devices are useful just to have a way to probe hw which may or may not be there, since on ARM we often don't have any alternative.. but beyond that, suspend/resume, and other life-cycle aspects, they should really be treated as all one device. Especially to avoid undefined suspend/resume ordering.

[1] https://lkml.org/lkml/2009/9/9/373 [2] http://elinux.org/images/9/90/ELCE2012-Modular-Graphics-on-Embedded-ARM.pdf

Thanks, Sylwester

On Mon, Dec 17, 2012 at 10:21 PM, Rob Clark rob.clark@linaro.org wrote:

On Mon, Dec 17, 2012 at 11:04 PM, Dave Airlie airlied@gmail.com wrote:

...

...

Many developers showed interest in the first RFC, and I've had the opportunity to discuss it with most of them. I would like to thank (in no particular order) Tomi Valkeinen for all the time he spend helping me to draft v2, Marcus Lorentzon for his useful input during Linaro Connect Q4 2012, and Linaro for inviting me to Connect and providing a venue to discuss this topic.

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

I tend to think that sub-devices are useful just to have a way to probe hw which may or may not be there, since on ARM we often don't have any alternative..

You can probe the device tree from a normal DRM driver. For example in nouveau for PPC we probe the OF device tree looking for connectors. I don't see how sub-devices or extra platform drivers help with that, as long as the device tree is populated upfront somehow...

Stéphane

but beyond that, suspend/resume, and other life-cycle aspects, they should really be treated as all one device. Especially to avoid undefined suspend/resume ordering.

CDF or some sort of mechanism to share panel drivers between drivers is useful. Keeping it within drm, is probably a good idea, if nothing else to simplify re-use of helper fxns (like avi-infoframe stuff, for example) and avoid dealing with merging changes across multiple trees. Treating them more like shared libraries and less like sub-devices which can be dynamically loaded/unloaded (ie. they should be not built as separate modules or suspend/resumed or probed/removed independently of the master driver) is a really good idea to avoid uncovering nasty synchronization issues later (remove vs modeset or pageflip) or surprising userspace in bad ways.

...

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

yeah, let's not have multiple frameworks to do the same thing.. For fbdev, it is pretty clear that it is a dead end. For v4l2 (subdev+mcf), it is perhaps bit more flexible when it comes to random arbitrary hw pipelines than kms. But to take advantage of that, your userspace isn't going to be portable anyways, so you might as well use driver specific properties/ioctls. But I tend to think that is more useful for cameras. And from userspace perspective, kms planes are less painful to use for output than v4l2, so lets stick to drm/kms for output (and not try to add camera/capture support to kms).. k, thx

BR, -R

...

Dave. _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel

---

dri-devel mailing list dri-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/dri-devel

On Mon, Dec 24, 2012 at 7:37 AM, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

Hi Rob,

On Tuesday 18 December 2012 00:21:32 Rob Clark wrote:

...

On Mon, Dec 17, 2012 at 11:04 PM, Dave Airlie airlied@gmail.com wrote:

...

...

Many developers showed interest in the first RFC, and I've had the opportunity to discuss it with most of them. I would like to thank (in no particular order) Tomi Valkeinen for all the time he spend helping me to draft v2, Marcus Lorentzon for his useful input during Linaro Connect Q4 2012, and Linaro for inviting me to Connect and providing a venue to discuss this topic.

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

I tend to think that sub-devices are useful just to have a way to probe hw which may or may not be there, since on ARM we often don't have any alternative.. but beyond that, suspend/resume, and other life-cycle aspects, they should really be treated as all one device. Especially to avoid undefined suspend/resume ordering.

I tend to agree, except that I try to reuse the existing PM infrastructure when possible to avoid reinventing the wheel. So far handling suspend/resume ordering related to data busses in early suspend/late resume operations and allowing the Linux PM core to handle control busses using the Linux device tree worked pretty well.

...

CDF or some sort of mechanism to share panel drivers between drivers is useful. Keeping it within drm, is probably a good idea, if nothing else to simplify re-use of helper fxns (like avi-infoframe stuff, for example) and avoid dealing with merging changes across multiple trees. Treating them more like shared libraries and less like sub-devices which can be dynamically loaded/unloaded (ie. they should be not built as separate modules or suspend/resumed or probed/removed independently of the master driver) is a really good idea to avoid uncovering nasty synchronization issues later (remove vs modeset or pageflip) or surprising userspace in bad ways.

We've tried that in V4L2 years ago and realized that the approach led to a dead-end, especially when OF/DT got involved. With DT-based device probing, I2C camera sensors started getting probed asynchronously to the main camera device, as they are children of the I2C bus master. We will have similar issues with I2C HDMI transmitters or panels, so we should be prepared for it.

What I've done to avoid that so far is that the master device registers the drivers for it's output sub-devices before registering it's own device. At least this way I can control that they are probed first. Not the prettiest thing, but avoids even uglier problems.

On PC hardware the I2C devices are connected to an I2C master provided by the GPU, but on embedded devices they are usually connected to an independent I2C master. We thus can't have a single self-contained driver that controls everything internally, and need to interface with the rest of the SoC drivers.

I agree that probing/removing devices independently of the master driver can lead to bad surprises, which is why I want to establish clear rules in CDF regarding what can and can't be done with display entities. Reference counting will be one way to make sure that devices don't disappear all of a sudden.

That at least helps cover some issues.. although it doesn't really help userspace confusion.

Anyways, with enough work perhaps all problems could be solved.. otoh, there are plenty of other important problems to solve in the world of gpus and kms, so my preference is always not to needlessly over-complicate CDF and instead leave some time for other things

BR, -R

...

...

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

yeah, let's not have multiple frameworks to do the same thing.. For fbdev, it is pretty clear that it is a dead end. For v4l2 (subdev+mcf), it is perhaps bit more flexible when it comes to random arbitrary hw pipelines than kms. But to take advantage of that, your userspace isn't going to be portable anyways, so you might as well use driver specific properties/ioctls. But I tend to think that is more useful for cameras. And from userspace perspective, kms planes are less painful to use for output than v4l2, so lets stick to drm/kms for output (and not try to add camera/capture support to kms)..

Agreed. I've started to advocate the deprecation of FBDEV during LPC. The positive response has motivated me to continue doing so :-) For V4L2 the situation is a little bit different, I think V4L2 shouldn't be used for graphics and display hardware, but it still has use cases on the video output side for pure video devices (such as pass-through video pipelines with embedded processing for instance). As those can use subdevices found in display and graphics hardware, I'd like to avoid code duplication.

-- Regards,

Laurent Pinchart

-- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

On Thu, Dec 27, 2012 at 1:18 PM, Sascha Hauer s.hauer@pengutronix.de wrote:

On Thu, Dec 27, 2012 at 09:54:55AM -0600, Rob Clark wrote:

...

On Mon, Dec 24, 2012 at 7:37 AM, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

...

Hi Rob,

On Tuesday 18 December 2012 00:21:32 Rob Clark wrote:

...

On Mon, Dec 17, 2012 at 11:04 PM, Dave Airlie airlied@gmail.com wrote:

...

...

Many developers showed interest in the first RFC, and I've had the opportunity to discuss it with most of them. I would like to thank (in no particular order) Tomi Valkeinen for all the time he spend helping me to draft v2, Marcus Lorentzon for his useful input during Linaro Connect Q4 2012, and Linaro for inviting me to Connect and providing a venue to discuss this topic.

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

I tend to think that sub-devices are useful just to have a way to probe hw which may or may not be there, since on ARM we often don't have any alternative.. but beyond that, suspend/resume, and other life-cycle aspects, they should really be treated as all one device. Especially to avoid undefined suspend/resume ordering.

I tend to agree, except that I try to reuse the existing PM infrastructure when possible to avoid reinventing the wheel. So far handling suspend/resume ordering related to data busses in early suspend/late resume operations and allowing the Linux PM core to handle control busses using the Linux device tree worked pretty well.

...

CDF or some sort of mechanism to share panel drivers between drivers is useful. Keeping it within drm, is probably a good idea, if nothing else to simplify re-use of helper fxns (like avi-infoframe stuff, for example) and avoid dealing with merging changes across multiple trees. Treating them more like shared libraries and less like sub-devices which can be dynamically loaded/unloaded (ie. they should be not built as separate modules or suspend/resumed or probed/removed independently of the master driver) is a really good idea to avoid uncovering nasty synchronization issues later (remove vs modeset or pageflip) or surprising userspace in bad ways.

We've tried that in V4L2 years ago and realized that the approach led to a dead-end, especially when OF/DT got involved. With DT-based device probing, I2C camera sensors started getting probed asynchronously to the main camera device, as they are children of the I2C bus master. We will have similar issues with I2C HDMI transmitters or panels, so we should be prepared for it.

What I've done to avoid that so far is that the master device registers the drivers for it's output sub-devices before registering it's own device. At least this way I can control that they are probed first. Not the prettiest thing, but avoids even uglier problems.

This implies that the master driver knows all potential subdevices, something which is not true for SoCs which have external i2c encoders attached to unrelated i2c controllers.

well, it can be brute-forced.. ie. drm driver calls common register_all_panels() fxn, which, well, registers all the panel/display subdev's based on their corresponding CONFIG_FOO_PANEL defines. If you anyways aren't building the panels as separate modules, that would work. Maybe not the most *elegant* approach, but simple and functional.

I guess it partly depends on the structure in devicetree. If you are assuming that the i2c encoder belongs inside the i2c bus, like:

&i2cN { foo-i2c-encoder { .... }; };

and you are letting devicetree create the devices, then it doesn't quite work. I'm not entirely convinced you should do it that way. Really any device like that is going to be hooked up to at least a couple busses.. i2c, some sort of bus carrying pixel data, maybe some gpio's, etc. So maybe makes more sense for a virtual drm/kms bus, and then use phandle stuff to link it to the various other busses it needs:

mydrmdev { foo-i2c-encoder { i2c = <&i2cN>; gpio = <&gpioM 2 3> ... }; };

ok, admittedly that is a bit different from other proposals about how this all fits in devicetree.. but otoh, I'm not a huge believer in letting something that is supposed to make life easier (DT), actually make things harder or more complicated. Plus this CDF stuff all needs to also work on platforms not using OF/DT.

BR, -R

Sascha

-- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

On Friday 28 December 2012 01:04:04 Sascha Hauer wrote:

On Thu, Dec 27, 2012 at 01:57:56PM -0600, Rob Clark wrote:

...

On Thu, Dec 27, 2012 at 1:18 PM, Sascha Hauer wrote:

...

On Thu, Dec 27, 2012 at 09:54:55AM -0600, Rob Clark wrote:

...

On Mon, Dec 24, 2012 at 7:37 AM, Laurent Pinchart

This implies that the master driver knows all potential subdevices, something which is not true for SoCs which have external i2c encoders attached to unrelated i2c controllers.

well, it can be brute-forced.. ie. drm driver calls common register_all_panels() fxn, which, well, registers all the panel/display subdev's based on their corresponding CONFIG_FOO_PANEL defines. If you anyways aren't building the panels as separate modules, that would work. Maybe not the most *elegant* approach, but simple and functional.

I guess it partly depends on the structure in devicetree. If you are

assuming that the i2c encoder belongs inside the i2c bus, like: &i2cN {

foo-i2c-encoder {

  ....

};

};

and you are letting devicetree create the devices, then it doesn't quite work. I'm not entirely convinced you should do it that way. Really any device like that is going to be hooked up to at least a couple busses.. i2c, some sort of bus carrying pixel data, maybe some gpio's, etc. So maybe makes more sense for a virtual drm/kms bus, and then use phandle stuff to link it to the various other busses it

needs: mydrmdev { foo-i2c-encoder { i2c = <&i2cN>; gpio = <&gpioM 2 3> ... }; };

This seems to shift initialization order problem to another place. Here we have to make sure the controller is initialized before the drm driver. Same with suspend/resume.

It's not only i2c devices, also platform devices. On i.MX for example we have a hdmi transmitter which is somewhere on the physical address space.

I think grouping the different units together in a devicetree blob because we think they might form a logical virtual device is not going to work. It might make it easier from a drm perspective, but I think doing this will make for a lot of special cases. What will happen for example if you have two encoder devices in a row to configure? The foo-i2c-encoder would then get another child node.

Right now the devicetree is strictly ordered by (control-, not data-) bus topology. Linux has great helper code to support this model. Giving up this help to brute force a different topology and then trying to fit the result back into the Linux Bus hierarchy doesn't sound like a good idea to me.

I agree. The Linux device model is architectured around a control bus based tree, I don't want to change that. With devices hooked up on several busses we will have dependency issues anyway, regardless of how we describe them in DT. If we hook up the nodes from a data bus perspective we will run into control bus dependency issues. It's thus better in my opinion to keep the classic control bus based model and solve the data bus dependency issues.

...

ok, admittedly that is a bit different from other proposals about how this all fits in devicetree.. but otoh, I'm not a huge believer in letting something that is supposed to make life easier (DT), actually make things harder or more complicated. Plus this CDF stuff all needs to also work on platforms not using OF/DT.

Right, but every other platform I know of is also described by its bus topology, be it platform device based or PCI or maybe even USB based.

CDF has to solve the same problem as ASoC and soc-camera: subdevices for a virtual device can come from many different corners of the system. BTW one example for a i2c encoder would be the SiI9022 which could not only be part of a drm device, but also of an ASoC device.

On Tue, Jan 8, 2013 at 2:25 AM, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

Hi Rob,

On Thursday 27 December 2012 09:54:55 Rob Clark wrote:

...

What I've done to avoid that so far is that the master device registers the drivers for it's output sub-devices before registering it's own device.

I'm not sure to follow you here. The master device doesn't register anything, do you mean the master device driver ? If so, how does the master device driver register its own device ? Devices are not registered by their driver.

sorry, that should have read "master driver registers drivers for it's sub-devices.."

BR, -R

Hi Rahul,

On Wednesday 09 January 2013 13:53:30 Rahul Sharma wrote:

Hi Laurent,

CDF will also be helpful in supporting Panels with integrated audio (HDMI/DP) if we can add audio related control operations to display_entity_control_ops. Video controls will be called by crtc in DRM/V4L and audio controls from Alsa.

I knew that would come up at some point :-) I agree with you that adding audio support would be a very nice improvement, and I'm totally open to that, but I will concentrate on video, at least to start with. The first reason is that I'm not familiar enough with ALSA, and the second that there's only 24h per day :-)

Please feel free, of course, to submit a proposal for audio support.

Secondly, if I need to support get_modes operation in hdmi/dp panel, I need to implement edid parser inside the panel driver. It will be meaningful to add get_edid control operation for hdmi/dp.

Even if EDID data is parsed in the panel driver, raw EDID will still need to be exported, so a get_edid control operation (or something similar) is definitely needed. There's no disagreement on this, I just haven't included that operation yet because my test hardware is purely panel-based.

On 12/18/2012 06:04 AM, Dave Airlie wrote:

...

Many developers showed interest in the first RFC, and I've had the opportunity to discuss it with most of them. I would like to thank (in no particular order) Tomi Valkeinen for all the time he spend helping me to draft v2, Marcus Lorentzon for his useful input during Linaro Connect Q4 2012, and Linaro for inviting me to Connect and providing a venue to discuss this topic.

So this might be a bit off topic but this whole CDF triggered me looking at stuff I generally avoid:

I like the effort, right now it seems like x86 and arm display sub systems are quite different in terms of DRM driver (and HW) design. I think this is partly due to little information shared about these different architectures and ideas behind the choices made. I hope some discussion will light up both sides. And an early discussion will hopefully give you less pain when CDF drivers starts to get pushed your way.

The biggest problem I'm having currently with the whole ARM graphics and output world is the proliferation of platform drivers for every little thing. The whole ordering of operations with respect to things like suspend/resume or dynamic power management is going to be a real nightmare if there are dependencies between the drivers. How do you enforce ordering of s/r operations between all the various components?

Could you give an example? Personally I don't think it is that many. I might not have counted the plat devs in all arm drivers. But the STE one have one per HW IP block in the HW (1 DSS + 3 DSI encoder/formatters). Then of course there are all these panel devices. But I hope that when CDF is "finished" we will have DSI devices on the DSI bus and DBI devices on the DBI bus. I think most vendors have used platform devices for these since they normally can't be probed in a generic way. But as they are off SoC I feel this is not the best choice. And then many of the panels are I2C devices (control bus) and that I guess is similar to "x86" encoders/connectors? Another part of the difference I feel is that in x86 a DRM device is most likely a PCI device, and as such has one huge driver for all IPs on that board. The closest thing we get to that in ARM is probably the DSS (collection of IPs on SoC, like 3D, 2D, display output, encoders). But it doesn't fell right to create a single driver for all these. And as you know often 3D is even from a separate vendor. All these lead up to a slight increase in the number of devices and drivers. Right way, I feel so, but you are welcome to show a better way.

The other thing I'd like you guys to do is kill the idea of fbdev and v4l drivers that are "shared" with the drm codebase, really just implement fbdev and v4l on top of the drm layer, some people might think this is some sort of maintainer thing, but really nothing else makes sense, and having these shared display frameworks just to avoid having using drm/kms drivers seems totally pointless. Fix the drm fbdev emulation if an fbdev interface is needed. But creating a fourth framework because our previous 3 frameworks didn't work out doesn't seem like a situation I want to get behind too much.

I have no intention to use CDF outside KMS connector/encoder and I have not heard Laurent talk about this either. Personally I see CDF as "helpers" to create and reuse connector/encoder drivers between SoCs instead of each SoC do their own panel drivers (which would be about a hundred, times the number of supported SoCs). We probably need to discuss the connector/encoder mappings to CDF/panels. But I think we need to flush out the higher level details like control bus vs. data bus vs. display entities. While I like the generic way of the display entities, I also like the pure bus/device/driver model without too many generalizations. Do you have any support in x86 world that could be compared to mobile phone DSI/DBI/DPI panels? That is, different encoder/lcd-driver chips between the on chip/cpu/SoC CRTC and the external LCD depending on product (mobile/netbook/...) or is it all HDMI/DP/LVDS etc on x86? And if you do, how do you model/setup/share all those in DRM driver? Or it is manageable (< 10) and not up in the hundreds of different encoders/lcd-drivers?

/BR /Marcus

On Mon, Dec 24, 2012 at 11:09 AM, Laurent Pinchart laurent.pinchart@ideasonboard.com wrote:

On the topic of discussions, would anyone be interested in a BoF/brainstorming/whatever session during the FOSDEM ?

I will be at FOSDEM.. and from http://wiki.x.org/wiki/fosdem2013 it looks like at least Daniel will be there. If enough others are, it could be a good idea.

BR, -R

Hi Daniel,

On Sunday 06 January 2013 18:46:47 Daniel Vetter wrote:

On Thu, Dec 27, 2012 at 09:57:25AM -0600, Rob Clark wrote:

...

On Mon, Dec 24, 2012 at 11:09 AM, Laurent Pinchart wrote:

...

On the topic of discussions, would anyone be interested in a BoF/brainstorming/whatever session during the FOSDEM ?

I will be at FOSDEM.. and from http://wiki.x.org/wiki/fosdem2013 it looks like at least Daniel will be there. If enough others are, it could be a good idea.

Seconded. Jesse should be there, too, and from the Helsinki guys Ville and Andy should show up. Doesn't look like Jani will be able to make it. I think something on Sunday (to not clash with the X devroom) would be good.

Should we apply for an offical BOF/Is there a process for tahat? Adding Luc in case he knows ...

From the event website it looks like there are free rooms on Sunday, it would

be good if we could secure one of them.

Are there other X/display related topics that need to be discussed on Sunday ? How much time should we set aside ?