Hi,
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think? Maxime
On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
Hi,
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think?
I believe we should start with the group of the tests that we know that are reliably passing today on most of the platforms and then increase the list as the tests and drivers become more reliable.
For instance, many of these would be candidate to be filtered out for now https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
compared to the whole view of kms tests: https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms
Thanks, Rodrigo.
Maxime
Hi,
On Thu, 3 Mar 2022 at 10:53, Rodrigo Vivi rodrigo.vivi@intel.com wrote:
On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think?
I believe we should start with the group of the tests that we know that are reliably passing today on most of the platforms and then increase the list as the tests and drivers become more reliable.
For instance, many of these would be candidate to be filtered out for now https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
compared to the whole view of kms tests: https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms
We are running some of IGT on Panfrost + amdgpu + i915 as part of KernelCI as well: go to https://linux.kernelci.org/test/ and search for 'igt-gpu'. This gets run for mainline, next, and whatever other kernel trees push into i915.
There is a Grafana-based dashboard that the KernelCI team have been working on to visualise test runs, but it's currently having some backend issues so I can't show you a link for that. I did raise a suggestion in their design discussion for a proper testing dashboard for making it easier to see the status, so feel free to pile in there: https://github.com/kernelci/kernelci-project/discussions/28#discussioncommen...
Cheers, Daniel
Hi Daniel,
On Fri, Mar 04, 2022 at 08:45:07AM +0000, Daniel Stone wrote:
On Thu, 3 Mar 2022 at 10:53, Rodrigo Vivi rodrigo.vivi@intel.com wrote:
On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think?
I believe we should start with the group of the tests that we know that are reliably passing today on most of the platforms and then increase the list as the tests and drivers become more reliable.
For instance, many of these would be candidate to be filtered out for now https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
compared to the whole view of kms tests: https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms
We are running some of IGT on Panfrost + amdgpu + i915 as part of KernelCI as well: go to https://linux.kernelci.org/test/ and search for 'igt-gpu'. This gets run for mainline, next, and whatever other kernel trees push into i915.
I was mainly interested in KMS, but I saw that there's an igt-kms test as well, thanks!
There is a Grafana-based dashboard that the KernelCI team have been working on to visualise test runs, but it's currently having some backend issues so I can't show you a link for that. I did raise a suggestion in their design discussion for a proper testing dashboard for making it easier to see the status, so feel free to pile in there: https://github.com/kernelci/kernelci-project/discussions/28#discussioncommen...
I'll have a look, thanks Maxime
Hi Rodrigo,
On Thu, Mar 03, 2022 at 05:52:57AM -0500, Rodrigo Vivi wrote:
On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
Hi,
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think?
I believe we should start with the group of the tests that we know that are reliably passing today on most of the platforms and then increase the list as the tests and drivers become more reliable.
I can see why that would be an objective too, but I'm not sure it would cover mine. What I'd like this series to be is something we can ask upfront to new drivers being submitted to make sure that they are sane.
Whether or not old drivers pass that bar is a bit irrelevant to that objective (and this would actually create tasks for newcomers that are looking for something to work on).
So, yeah, I don't mind having failing tests on older drivers, I kind of even expect them to fail somehow. It would essentially be a bar to show what any driver should strive for, not the lowest common denominator.
Does that make sense?
For instance, many of these would be candidate to be filtered out for now https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
compared to the whole view of kms tests: https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms
So the set of tests that are always run would be the latter, right?
Maxime
dri-devel@lists.freedesktop.org
-
Daniel Stone -
Maxime Ripard -
Rodrigo Vivi