Hi Rodrigo,
On Thu, Mar 03, 2022 at 05:52:57AM -0500, Rodrigo Vivi wrote:
On Thu, Mar 03, 2022 at 10:05:07AM +0100, Maxime Ripard wrote:
Hi,
Back at XDC we floated the idea of creating a test suite for IGT that we expect any KMS driver to pass, similar to what v4l2-compliance and cec-compliance provide for v4l2 and CEC respectively.
I was looking at the list of tests, and it's fairly massive, so it's not clear to me what tests we could start this suite with. I can only assume all the KMS (but the chamelium ones) and fbdev related ones would be a good start?
What do you think?
I believe we should start with the group of the tests that we know that are reliably passing today on most of the platforms and then increase the list as the tests and drivers become more reliable.
I can see why that would be an objective too, but I'm not sure it would cover mine. What I'd like this series to be is something we can ask upfront to new drivers being submitted to make sure that they are sane.
Whether or not old drivers pass that bar is a bit irrelevant to that objective (and this would actually create tasks for newcomers that are looking for something to work on).
So, yeah, I don't mind having failing tests on older drivers, I kind of even expect them to fail somehow. It would essentially be a bar to show what any driver should strive for, not the lowest common denominator.
Does that make sense?
For instance, many of these would be candidate to be filtered out for now https://intel-gfx-ci.01.org/tree/drm-next/index.html?testfilter=kms
compared to the whole view of kms tests: https://intel-gfx-ci.01.org/tree/drm-next/shards-all.html?testfilter=kms
So the set of tests that are always run would be the latter, right?
Maxime