[Public]

-----Original Message----- From: Bjorn Helgaas helgaas@kernel.org Sent: Wednesday, March 30, 2022 7:45 AM To: linux-pci@vger.kernel.org Cc: Deucher, Alexander Alexander.Deucher@amd.com; Robert Hancock robert.hancock@calian.com; dri-devel@lists.freedesktop.org Subject: RX 5500 XT: PCIe link speed stuck at Gen1 2.5GT/s by default

See https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.f reedesktop.org%2Fdrm%2Famd%2F- %2Fissues%2F1447&data=04%7C01%7Calexander.deucher%40amd.com %7C01b4e1eed4ac4c97ab0408da1242abe9%7C3dd8961fe4884e608e11a82 d994e183d%7C0%7C0%7C637842374822556101%7CUnknown%7CTWFpbGZ sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn 0%3D%7C3000&sdata=elFgBq%2FoCfqIO98aaGeS3d1omBdJj%2BSH6HO 13oNaBWM%3D&reserved=0

This report was against DRM, but I'm mentioning it here because we've been talking about some link speed init issues lately, and AFAICT the gitlab reports don't show up anywhere in lore.

Just noticed this. The GPU driver calls pcie_bandwidth_available() to get the speed and link widths of the link to root port. The driver then limits the link speed and number of lanes to the max available on the link. There's no reason to run the link faster than the slowest link in the chain. For the most part this works fine because I think the PCIe spec envisions that the links will negotiate the fastest link available when the system comes up. But it seems that some platforms don't always do this. Maybe pcie_bandwidth_available() should return the max bandwidth based on the caps of the links rather than the current status. That said, I'm not sure how you would differentiate between the platforms that came up in a slow link t save power vs platforms that came up in a slow link because there was some problem negotiating a faster link or who, if anyone would be responsible for making sure the links are upgraded when needed.

Alex

Robert Hancock reported:

...

I'm using a RX 5500 XT card on an Asus PRIME H270-PRO motherboard, Intel i5-7500 CPU, with kernel 5.10.9 under Fedora 33. I noticed that in Linux, "lspci -vv" always showed the GPU PCIe link running at 2.5GT/s link speed and never seemed to change regardless of the application being run, while in Windows, GPU-Z shows the link running at the max supported 8GT/s speed when under graphical load.

It seems like the driver thinks that 2.5GT/s is the max allowable speed, based on the pp_dpm_pcie file:

...

cd

/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:00.0/0000:0 3:00.0/

...

...

cat pp_dpm_pcie

0: 2.5GT/s, x8 81Mhz * 1: 2.5GT/s, x8 619Mhz *

I'm assuming that something is going wrong with the PCIe link speed detection in the driver. Using the "amdgpu.pcie_gen_cap=0x70007" kernel command line option seems to result in the driver detecting the proper 8GT/s maximum speed.

lspci -vv output from booting without overriding the speed is attached.