https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #1 from Duncan (1i5t5.duncan@cox.net) --- Created attachment 288651 --> https://bugzilla.kernel.org/attachment.cgi?id=288651&action=edit automated boot-time dmesg dump

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #3 from Duncan (1i5t5.duncan@cox.net) --- CCed the two from MAINTAINERS bugzi would let me add. It wouldn't let me add amd-gfx@ or david1.zhou@, and Alex's gmail address according to bugzi isn't what's in MAINTAINERS.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #5 from Duncan (1i5t5.duncan@cox.net) --- Well, that didn't take long. Four konsole terminals open to do (various aspects of) a system update. Just a few seconds after I entered the (git-based) sync command, display-FREEZE!

Back on 5.6.0 now. I'll probably test again with rc4, perhaps earlier if I see a set of drm/amdgpu updates in mainline git.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #7 from Duncan (1i5t5.duncan@cox.net) --- Bisecting, but it's slow going when the bug can take 12+ hours to trigger, and even then I can't be sure a "good" is actually so.

So far (at 5.6.0-01623-g12ab316ce, ~7 bisect steps to go, under 100 commits "after"), the first few were all "good", while the one I'm currently testing obviously isn't "bad" in terms of this bug yet, but does display a nasty buffer-sync issue with off-frame read-outs and eventual firefox crashes trying to play 4k@30fps youtube in firefox, a bit of a struggle with this kit but usually OK (it's the 4k@60fps that's the real problem in firefox/chromium, tho it tends to be fine without the browser overhead in mpv/smplayer/vlc).

But I hadn't seen that issue with the full 5.7-rc1 thru rc3, so it was apparently already fixed with rc1. And no incidents of this bug, full system or full graphics lockups with a segfault in amdgpu_dm_atomic_commit_tail, during the bisect yet.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #9 from Duncan (1i5t5.duncan@cox.net) --- I'm not there yet but it's starting to look like a possibly dud bisect: everything showing good so far. Maybe I didn't wait long enough for the bug to trigger at some step and I'm running up the wrong side of the tree, or maybe it's not drm after all (I thought I'd try something new and limit the paths to drivers/gpu/drm/ and include/drm/, but that may have been a critical mistake). Right now there's only 3-4 even remotely reasonable candidates (out of 14 left to test... the rest being mediatek or similar):

4064b9827 Peter Xu mm: allow VM_FAULT_RETRY for multiple times

6bfef2f91 Jason Gunthorpe mm/hmm: remove HMM_FAULT_SNAPSHOT

17ffdc482 Christoph Hellwig mm: simplify device private page handling in hmm_range_fault

And maybe (but I'm neither EFI nor 32-bit)

72e0ef0e5 Mikel Rychliski PCI: Use ioremap(), not phys_to_virt() for platform ROM

Meanwhile, user-side I've gotten vulkan/mesa/etc updates recently. I'm considering checking out linus-master/HEAD again, doing a pull, and seeing if by chance either the last week's kernel updates or the user-side updates have eliminated the problem. If not I can come back and finish the bisect (or try just reverting those four on current linus-master/HEAD), before starting a new clean bisect if necessary. Just saved the bisect log and current pointer.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #11 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Duncan from comment #10)

I did get an apparent graphics crash at the bisect-point above, but it didn't dump anything in the log this time

Got a gpf dump with amdgpu_atomic_commit_tail, confirming it's the same bug. Still a couple bisect steps to go, but the EFI candidate's out now, leaving only three (plus mediatek and nouveau, and an amdgpu that says it was doc fix only), and the current round is testing between 406 and the 6bf/17f pair so I should eliminate at least one of the three this round:

4064b9827 Peter Xu mm: allow VM_FAULT_RETRY for multiple times

6bfef2f91 Jason Gunthorpe mm/hmm: remove HMM_FAULT_SNAPSHOT

17ffdc482 Christoph Hellwig mm: simplify device private page handling in hmm_range_fault

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #13 from Duncan (1i5t5.duncan@cox.net) --- Well, so much for /that/ bisect! Took me a few hours but then had the graphics stall twice in a few minutes... with the above commit reverted AND with memory compression off.

So it's back to square one, except I know that my originally chosen new memory compression options aren't involved. New bisect time.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #15 from Duncan (1i5t5.duncan@cox.net) --- Bug's in v5.8-rc1-226-g4333a9b0b too.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #17 from Duncan (1i5t5.duncan@cox.net) --- (In reply to rtmasura+kernel from comment #16)

Reporting I've had the same issue with kernel 5.7.2 and 5.7.4:

Thanks!

Jun 22 07:10:24 abiggun kernel: Hardware name: System manufacturer System Product Name/Crosshair IV Formula, BIOS 1102 08/24/2010

So socket AM3 from 2010, slightly older than my AM3+ from 2012. Both are PCIe-2.0.

What's your CPU and GPU?

As above my GPU is Polaris11 (AMD Radeon RX 460, arctic-islands/gcn4 series, pcie-3), AMD fx6100 CPU.

Guessing the bug is gpu-series code specific or there'd be more people howling, so what you're running for gpu is significant. It's /possible/ it may be specific to people running pcie mismatch, as well (note my pcie-3 gpu card on a pcie-2 mobo).

Jun 22 07:10:24 abiggun kernel: Workqueue: events_unbound commit_work [drm_kms_helper] 0010:amdgpu_dm_atomic_commit_tail+0x2aa/0x2310 [amdgpu]

That's the bit of the dump I understand, similar to mine...

If you can find a quicker/more-reliable way to trigger the crash, it'd sure be helpful for bisecting. Also, if you're running a bad kernel enough to tell (not just back to 5.6 after finding 5.7 bad), does it reliably dump-log before the reboot for you? I'm back to a veerrry--sloowww second bisect attempt, with for instance my current kernel having crashed three times now so it's obviously bugged, but nothing dumped in the log on the way down yet so I can't guarantee it's the _same_ bug (the bisect is in pre-rc1 code so chances of a different bug are definitely non-zero), and given the bad results on the first bisect I'm trying to confirm each bisect-bad with a log-dump and each bisect-good with at least 3-4 days no crash. But this one's in between right now, frequent crashing but no log-dump to confirm it's the same bug.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #19 from Duncan (1i5t5.duncan@cox.net) --- (In reply to rtmasura+kernel from comment #18)

09:00.0 VGA compatible controller: NVIDIA Corporation GP104GL [Quadro P4000] (rev a1)

0e:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XL/XT [Radeon RX Vega 56/64] (rev c3)

A few notes on that: The AMD Vega56 is used for this PC, the Quadro P4000 is disabled on my system and passed through to VMs.

So newer graphics, Vega56/gcn5 compared to my gcn4.

No VMs at all here so that can be excluded as a factor (unless it's a minor trigger similar to my zooming or video play).

I haven't found any way to trigger it. Seems completely random. Sat down this morning to update a VM (not the one with the nvidia passthrough) and it froze, wasn't any real graphical things going on other than normal KDE stuff.

KDE/Plasma here too. I think kwin exercises the opengl a bit more than some WMs, in part because it's a compositor as well. The bug most often hits here when playing video or using kwin's zoom effect, which exercise the graphics a bit.

So mostly kde/kwin triggers could lower the population hitting it and could be a factor, based on both of us running it.

Model name: AMD Phenom(tm) II X6 1090T Processor

Newer graphics, gcn5 to gcn4, older cpu, phenom ii to fx, than here.

So we know gcn4 and gcn5 are affected, and pcie2 bus with pcie3 cards and kde/kwin are common-factor possible triggers so far.

I would be happy to help with any testing, just let me know what information you need.

If you happen to run anything besides KDE/Plasma on X, duplicating (or failing to duplicate) the bug on non-kde and/or on wayland would be useful info. I only run KDE Plasma on X here. Well, that and CLI (on amdgpu-drm-framebuffer) more than some but not enough that I'd have expected to see it there, which I haven't.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #21 from rtmasura+kernel@hotmail.com --- OK. I've uninstalled the vast majority of KDE and am using a vanilla XFCE4. It's been about 12 hours on 5.7.4-arch1-1 and I have yet to have a crash. It is looking like it may be something with KDE.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #23 from rtmasura+kernel@hotmail.com --- Yeah, over 24 hours and still stable. And glad I could help, I rarely have anything I can give back to the community.

And wow, that much work. Truly, we all do appreciate your work, but I don't think most of us understand how much. Thank you from all of us :)

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #25 from rtmasura+kernel@hotmail.com --- Same kernel (5.7.4) and I'll try to reproduce it, and if it happens I'll turn off the screen tear and try to reproduce again

Let me know if that's anything I can provide you

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #27 from rtmasura+kernel@hotmail.com --- and another crash, chrome's good at causing them (watching youtube). Used -s "" for the setting which I think should set it to 'auto', and what I assumed was default. I've changed that to -s "off" to see if that helps.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

zzyxpaw@gmail.com changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |zzyxpaw@gmail.com

--- Comment #29 from zzyxpaw@gmail.com --- Just hit this on Archlinux with linux-5.7.6 on a Vega 64. So far I've had three crashes mostly occuring within the first few minutes of uptime. I'm not running kwin or chrome, just a light window manager (bspwm) and compton.

During the first two, steam's fossilize was running which lead me to suspect it was triggered by an interaction with that. However the third crashed before I even managed to start steam, so either I'm just lucky or my system is good at triggering this. @Duncan I'm not sure if you want to muddle your bisect results with a different system configuration, but I'm happy to help test commits if that would be helpful.

I've noticed the call traces reported in the kernel log are slightly different for each crash; I'm not sure if they're likely to be useful or not. Here's at least the one from my first crash:

Jun 27 14:04:40 erebor kernel: general protection fault, probably for non-canonical address 0x5dda9795528973db: 0000 [#1] PREEMPT SMP NOPTI Jun 27 14:04:40 erebor kernel: CPU: 14 PID: 193610 Comm: kworker/u32:14 Tainted: G OE 5.7.6-arch1-1 #1 Jun 27 14:04:40 erebor kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./AB350 Pro4, BIOS P4.90 06/14/2018 Jun 27 14:04:40 erebor kernel: Workqueue: events_unbound commit_work [drm_kms_helper] Jun 27 14:04:40 erebor kernel: RIP: 0010:amdgpu_dm_atomic_commit_tail+0x2aa/0x2310 [amdgpu] Jun 27 14:04:40 erebor kernel: Code: 4f 08 8b 81 e0 02 00 00 41 83 c5 01 44 39 e8 0f 87 46 ff ff ff 48 83 bd f0 fc ff ff 00 0f 84 03 01 00 00 48 8b bd f0 fc ff ff <80> bf b0 01 00 00 01 0f 86 ac 00 00> Jun 27 14:04:40 erebor kernel: RSP: 0018:ffffbcec0a4afaf8 EFLAGS: 00010206 Jun 27 14:04:40 erebor kernel: RAX: 0000000000000006 RBX: ffff9b71dbaed000 RCX: ffff9b7472e4b800 Jun 27 14:04:40 erebor kernel: RDX: ffff9b72504ea400 RSI: ffffffffc13181e0 RDI: 5dda9795528973db Jun 27 14:04:40 erebor kernel: RBP: ffffbcec0a4afe60 R08: 0000000000000001 R09: 0000000000000001 Jun 27 14:04:40 erebor kernel: R10: 0000000000000082 R11: 00000000000730e2 R12: 0000000000000000 Jun 27 14:04:40 erebor kernel: R13: 0000000000000006 R14: ffff9b71dbaed800 R15: ffff9b71a8fdb580 Jun 27 14:04:40 erebor kernel: FS: 0000000000000000(0000) GS:ffff9b747ef80000(0000) knlGS:0000000000000000 Jun 27 14:04:40 erebor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 27 14:04:40 erebor kernel: CR2: 000056460ce164b0 CR3: 0000000341c86000 CR4: 00000000003406e0 Jun 27 14:04:40 erebor kernel: Call Trace: Jun 27 14:04:40 erebor kernel: ? __erst_read+0x160/0x1d0 Jun 27 14:04:40 erebor kernel: ? __switch_to_asm+0x34/0x70 Jun 27 14:04:40 erebor kernel: ? __switch_to_asm+0x40/0x70 Jun 27 14:04:40 erebor kernel: ? __switch_to_asm+0x34/0x70 Jun 27 14:04:40 erebor kernel: ? __switch_to_asm+0x40/0x70 Jun 27 14:04:40 erebor kernel: ? rescuer_thread+0x3f0/0x3f0 Jun 27 14:04:40 erebor kernel: commit_tail+0x94/0x130 [drm_kms_helper] Jun 27 14:04:40 erebor kernel: process_one_work+0x1da/0x3d0 Jun 27 14:04:40 erebor kernel: ? rescuer_thread+0x3f0/0x3f0 Jun 27 14:04:40 erebor kernel: worker_thread+0x4d/0x3e0 Jun 27 14:04:40 erebor kernel: ? rescuer_thread+0x3f0/0x3f0 Jun 27 14:04:40 erebor kernel: kthread+0x13e/0x160 Jun 27 14:04:40 erebor kernel: ? __kthread_bind_mask+0x60/0x60 Jun 27 14:04:40 erebor kernel: ret_from_fork+0x22/0x40 Jun 27 14:04:40 erebor kernel: Modules linked in: snd_seq_midi snd_seq_dummy snd_seq_midi_event snd_hrtimer snd_seq fuse ccm 8021q garp mrp stp llc snd_usb_audio snd_usbmidi_lib snd_rawmidi snd_seq_de> Jun 27 14:04:40 erebor kernel: blake2b_generic libcrc32c crc32c_generic xor uas usb_storage raid6_pq crc32c_intel xhci_pci xhci_hcd Jun 27 14:04:40 erebor kernel: ---[ end trace cb5c0d96dd991657 ]--- Jun 27 14:04:40 erebor kernel: RIP: 0010:amdgpu_dm_atomic_commit_tail+0x2aa/0x2310 [amdgpu] Jun 27 14:04:40 erebor kernel: Code: 4f 08 8b 81 e0 02 00 00 41 83 c5 01 44 39 e8 0f 87 46 ff ff ff 48 83 bd f0 fc ff ff 00 0f 84 03 01 00 00 48 8b bd f0 fc ff ff <80> bf b0 01 00 00 01 0f 86 ac 00 00> Jun 27 14:04:40 erebor kernel: RSP: 0018:ffffbcec0a4afaf8 EFLAGS: 00010206 Jun 27 14:04:40 erebor kernel: RAX: 0000000000000006 RBX: ffff9b71dbaed000 RCX: ffff9b7472e4b800 Jun 27 14:04:40 erebor kernel: RDX: ffff9b72504ea400 RSI: ffffffffc13181e0 RDI: 5dda9795528973db Jun 27 14:04:40 erebor kernel: RBP: ffffbcec0a4afe60 R08: 0000000000000001 R09: 0000000000000001 Jun 27 14:04:40 erebor kernel: R10: 0000000000000082 R11: 00000000000730e2 R12: 0000000000000000 Jun 27 14:04:40 erebor kernel: R13: 0000000000000006 R14: ffff9b71dbaed800 R15: ffff9b71a8fdb580 Jun 27 14:04:40 erebor kernel: FS: 0000000000000000(0000) GS:ffff9b747ef80000(0000) knlGS:0000000000000000 Jun 27 14:04:40 erebor kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 27 14:04:40 erebor kernel: CR2: 000056460ce164b0 CR3: 0000000341c86000 CR4: 00000000003406e0

https://bugzilla.kernel.org/show_bug.cgi?id=207383

Duncan (1i5t5.duncan@cox.net) changed:

What |Removed |Added ---------------------------------------------------------------------------- Kernel Version|5.7-rc1, 5.7-rc2, 5.7-rc3 |5.7-rc1 - 5.7 - 5.8-rc1+

--- Comment #31 from Duncan (1i5t5.duncan@cox.net) --- (In reply to mnrzk from comment #30)

In some conditions, when amdgpu_dm_atomic_commit_tail calls dm_atomic_get_new_state, dm_atomic_get_new_state returns a struct dm_atomic_state* with an garbage context pointer.

Good! Someone with the bug who can actually read and work the code, now. Portends well for a fix. =:^)

I've also found that this bug exclusively occurs when commit_work is on the workqueue. After forcing drm_atomic_helper_commit to run all of the commits without adding to the workqueue and running the OS, the issue seems to have disappeared.

I see it always with the workqueue too, but not being a dev I simply assumed that was how it was; I had no idea it could be taken off the workqueue.

The system was stable for at least 1.5 hours before I manually shut it down (meanwhile it has usually crashed within 30-45 minutes).

You're seeing a crash much faster than I am. I believe my longest uptime before a crash with the telltale trace was something like two and a half days, with the obvious implications for bisect good since it's always a gamble that I've simply not tested long enough.

Perhaps there's some sort of race condition occurring after commit_work is queued?

Agreed, FWIW, tho you've taken it farther than I could, not being able to work with code much beyond bisect or modifying an existing patch here or there.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #33 from Michel Dänzer (michel@daenzer.net) --- (In reply to rtmasura+kernel from comment #24)

xfwm4 --replace --vblank=glx &

FWIW, I recommend

xfwm4 --vblank=xpresent

instead. --vblank=glx is less efficient and relies on rather exotic GLX functionality which can be quirky with Mesa.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #35 from Duncan (1i5t5.duncan@cox.net) --- (In reply to mnrzk from comment #34)

Has anyone tried 5.8-rc3? I've been testing it out for the past 3 hours and it seems stable to me.

I have now (well, v5.8.0-rc3-00017-g7c30b859a). Unfortunately got a freeze with our familiar trace fairly quickly (building kde updates at the time) so it's not fixed yet. =:^(

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #37 from mnrzk@protonmail.com ---

Notice, no drm tree, tho whatever buggy commit it is obviously affects drm/amdgpu.

Yeah, I kind of noticed that while I was just skimming through the commit history. Perhaps it's possible that the issue has existed for a while but became much more apparent since 5.7?

Whatever it is, keep up the good work; maybe you'll find some sort of clue while bisecting.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

Duncan (1i5t5.duncan@cox.net) changed:

What |Removed |Added ---------------------------------------------------------------------------- Attachment #290101|0 |1 is obsolete| |

--- Comment #39 from Duncan (1i5t5.duncan@cox.net) --- Created attachment 290105 --> https://bugzilla.kernel.org/attachment.cgi?id=290105&action=edit Partial git bisect log update #3

(In reply to Duncan from comment #38)

With luck I'll get a couple more bad ones right away and narrow it down quickly.

And so it is. 28 candidates ATM, several of which are OCFS2 or spelling fixes neither of which should affect this bug. Excluding those there are eleven left; the penultimate (next to last) one looks to be a good candidate:

5f2d5026b mm/Makefile: disable KCSAN for kmemleak b0d14fc43 mm/kmemleak.c: use address-of operator on section symbols 667c79016 revert "topology: add support for node_to_mem_node() to determine the fallback node" 3202fa62f slub: relocate freelist pointer to middle of object 1ad53d9fa slub: improve bit diffusion for freelist ptr obfuscation bbd4e305e mm/slub.c: replace kmem_cache->cpu_partial with wrapped APIs 4c7ba22e4 mm/slub.c: replace cpu_slab->partial with wrapped APIs c537338c0 fs_parse: remove pr_notice() about each validation 630f289b7 asm-generic: make more kernel-space headers mandatory 98c985d7d kthread: mark timer used by delayed kthread works as IRQ safe 4054ab64e tools/accounting/getdelays.c: fix netlink attribute length

My gut says it's 98c "kthread: mark ... delayed kthread... IRQ safe". Not a coder but the comment talks about delayed kthreads, we always see the workqueue in the traces, and mnrzk observes in comment #30 that forcing drm_atomic_helper_commit to run directly instead of using the workqueue seems to eliminate the freeze. If it's called from the amdgpu code and that commit changes the IRQ-safety assumptions the amdgpu code was depending on in the workqueue, where the unqueued context is automatically IRQ-safe...

Still could be wrong, but at 11 real candidates it's a 9% chance even simply statistically, and it sure seems to fit. Anyway, if it /is/ correct, the next few bisect steps should be bisect bad and thus go faster, narrowing it down even further.

Regardless, we're down far enough that someone that can actually read code might be able to take a look at that and the others now, so my bisect shouldn't be the /entire/ bottleneck any longer.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #41 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Alex Deucher from comment #40)

Does this patch help?

Booted to v5.7 with it applied now. We'll see. Since the bug can take awhile to trigger on my hardware, if the patch fixes it I won't know for days, and won't be /sure/ for say a week, the reason bisecting was taking so long.

(It wouldn't apply to current 5.8-rc4-plus-an-s390-pull. Too tired to figure out why ATM but if it's because it was there already, hopefully it was pulled in after v5.8-rc3 as I tested that and got the same graphics freeze with the characteristic trace, so if the patch was already in v5.8-rc3, it does /not/ fix the bug.)

As for bisecting, I've hard-crashed twice on the current step, apparently with a different bug, so while _this_ bug hasn't seemed to trigger yet, I haven't gotten the necessary confidence that it's a bisect-good. So hopefully this patch /does/ fix it, and I can put this entirely too frustrating bug-bisect behind me!

https://bugzilla.kernel.org/show_bug.cgi?id=207383

Christopher Snowhill (kode54@gmail.com) changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |kode54@gmail.com

--- Comment #43 from Christopher Snowhill (kode54@gmail.com) --- What about this patch?

https://cgit.freedesktop.org/~agd5f/linux/commit/?h=drm-fixes-5.8&id=6eb...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #45 from Fabian Möller (fabianm88@gmail.com) --- (In reply to Christopher Snowhill from comment #43)

What about this patch?

https://cgit.freedesktop.org/~agd5f/linux/commit/?h=drm-fixes-5. 8&id=6eb3cf2e06d22b2b08e6b0ab48cb9c05a8e1a107

Applying 6eb3cf2e06d22b2b08e6b0ab48cb9c05a8e1a107 to v5.7.7 fixed the issue for a RX5700/Navi10 under Wayland for me. It still produces the following log, which might be related to https://bugzilla.kernel.org/show_bug.cgi?id=206349.

------------[ cut here ]------------ WARNING: CPU: 2 PID: 1176 at arch/x86/kernel/fpu/core.c:109 kernel_fpu_end+0x19/0x20 Modules linked in: fuse xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo xt_addrtype iptable_filter iptable_nat nf_nat nf_conntrack nf_defrag_ipv4 br_netfilter overlay wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libblake2s blake2s_x86_64 ip6_udp_tunnel udp_tunnel libcurve25519_generic libchacha libblake2s_generic af_packet rfkill msr amdgpu amd_iommu_v2 gpu_sched ttm drm_kms_helper drm snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi wmi_bmof mxm_wmi snd_hda_intel nls_iso8859_1 agpgart igb deflate nls_cp437 snd_intel_dspcfg sp5100_tco ptp mousedev vfat efi_pstore fb_sys_fops evdev fat snd_hda_codec edac_mce_amd pstore mac_hid syscopyarea watchdog pps_core sysfillrect edac_core snd_hda_core sysimgblt dca i2c_piix4 backlight crc32_pclmul i2c_algo_bit ghash_clmulni_intel efivars k10temp snd_hwdep i2c_core thermal wmi pinctrl_amd tiny_power_button button acpi_cpufreq sch_fq_codel snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore atkbd libps2 serio loop cpufreq_ondemand tun tap macvlan bridge stp llc vboxnetflt(OE) vboxnetadp(OE) vboxdrv(OE) kvm_amd kvm irqbypass efivarfs ip_tables x_tables ipv6 nf_defrag_ipv6 crc_ccitt autofs4 xfs libcrc32c crc32c_generic dm_crypt algif_skcipher af_alg input_leds led_class hid_generic usbhid hid ahci xhci_pci libahci crc32c_intel xhci_hcd libata aesni_intel libaes crypto_simd nvme usbcore cryptd scsi_mod glue_helper nvme_core t10_pi crc_t10dif crct10dif_generic crct10dif_pclmul usb_common crct10dif_common rtc_cmos dm_snapshot dm_bufio dm_mod CPU: 2 PID: 1176 Comm: systemd-logind Tainted: G OE 5.7.7 #1-NixOS Hardware name: Gigabyte Technology Co., Ltd. X570 AORUS PRO/X570 AORUS PRO, BIOS F10c 11/08/2019 RIP: 0010:kernel_fpu_end+0x19/0x20 Code: 90 e9 db 9b 14 00 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 65 8a 05 5c 1a 3e 66 84 c0 74 09 65 c6 05 50 1a 3e 66 00 c3 <0f> 0b eb f3 0f 1f 00 0f 1f 44 00 00 8b 15 cd 59 57 01 31 f6 e8 2e RSP: 0018:ffffb9dbc1417660 EFLAGS: 00010246 RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000121b RDX: 0000000000000001 RSI: ffff90b685fa1cd4 RDI: 000000000002f980 RBP: ffff90b685fa0000 R08: 0000000000000000 R09: 0000000000000040 R10: ffffb9dbc14175b0 R11: ffffb9dbc14170a0 R12: 0000000000000001 R13: ffff90b685fa1da8 R14: 0000000000000006 R15: ffff90b5158f8400 FS: 00007fd7ab143880(0000) GS:ffff90b6bea80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00000e81091ef030 CR3: 00000007d80c4000 CR4: 0000000000340ee0 Call Trace: dcn20_validate_bandwidth+0x2c/0x40 [amdgpu] dc_commit_updates_for_stream+0xad7/0x1930 [amdgpu] ? amdgpu_display_get_crtc_scanoutpos+0x85/0x190 [amdgpu] amdgpu_dm_atomic_commit_tail+0xb4c/0x1fc0 [amdgpu] commit_tail+0x94/0x130 [drm_kms_helper] drm_atomic_helper_commit+0x113/0x140 [drm_kms_helper] drm_client_modeset_commit_atomic+0x1c9/0x200 [drm] drm_client_modeset_commit_locked+0x50/0x150 [drm] __drm_fb_helper_restore_fbdev_mode_unlocked+0x59/0xc0 [drm_kms_helper] drm_fb_helper_set_par+0x3c/0x50 [drm_kms_helper] fb_set_var+0x175/0x370 ? update_load_avg+0x78/0x630 ? update_curr+0x69/0x1a0 fbcon_blank+0x20d/0x270 do_unblank_screen+0xaa/0x150 complete_change_console+0x54/0xd0 vt_ioctl+0x126f/0x1320 tty_ioctl+0x372/0x8c0 ksys_ioctl+0x87/0xc0 __x64_sys_ioctl+0x16/0x20 do_syscall_64+0x4e/0x160 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x7fd7ab75d1c7 Code: 00 00 90 48 8b 05 b9 9c 0c 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 89 9c 0c 00 f7 d8 64 89 01 48 RSP: 002b:00007ffe9138fec8 EFLAGS: 00000246 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fd7ab75d1c7 RDX: 0000000000000001 RSI: 0000000000005605 RDI: 0000000000000015 RBP: 0000000000000015 R08: 0000000000000000 R09: 00000000ffffffff R10: 0000000000000001 R11: 0000000000000246 R12: 00007ffe9138ff38 R13: 0000000000000006 R14: 00007ffe91390050 R15: 00007ffe91390048 ---[ end trace eefc00b763354df8 ]---

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #47 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Duncan from comment #39)

28 candidates ATM, several of which are OCFS2 or spelling fixes neither of which should affect this bug. Excluding those there are eleven left; the penultimate (next to last) one looks to be a good candidate:

5f2d5026b mm/Makefile: disable KCSAN for kmemleak b0d14fc43 mm/kmemleak.c: use address-of operator on section symbols 667c79016 revert "topology: add support for node_to_mem_node() to determine the fallback node" 3202fa62f slub: relocate freelist pointer to middle of object 1ad53d9fa slub: improve bit diffusion for freelist ptr obfuscation bbd4e305e mm/slub.c: replace kmem_cache->cpu_partial with wrapped APIs 4c7ba22e4 mm/slub.c: replace cpu_slab->partial with wrapped APIs c537338c0 fs_parse: remove pr_notice() about each validation 630f289b7 asm-generic: make more kernel-space headers mandatory 98c985d7d kthread: mark timer used by delayed kthread works as IRQ safe 4054ab64e tools/accounting/getdelays.c: fix netlink attribute length

(... and comment #44)

[I]dea to just try patch-reverting them on top of 5.7 or current 5.8-rc, thus eliminating the apparently unrelated kernel-panics I''ve twice triggered at the current bisect step.

[Again noting that on my polaris11 the bug doesn't seem to be fixed, despite comment #45 saying it is on his navi10 with a patch/commit that I can see in 5.8-rc4+.]

So I tried this with the 11 above commits against 5.8.0-rc4-00025-gbfe91da29, which previously tested as triggering the freeze for me. Of the 11, nine clean-reversed and I simply noted and skipped the other two (3202fa62f and 630f289b7) for the moment. The patched kernel successfully built and I'm booted to it now. I just completed a system update (on gentoo so built from source), which doesn't always trigger the freeze, but seems to do so with a reasonable number of package updates on kernels with this bug perhaps 50% of the time. No freeze.

I'll now try some 4k youtube in firefox, the other stressor that sometimes seems to trigger it here, and perhaps combine that with an unnecessary rebuild (since my system's already current) of something big like qtwebengine. If that doesn't trigger a freeze I'll stay booted to this thing another few days and try some more, before being confident enough to declare that one of those nine commits triggers the bug on my hardware and reverting them eliminates it.

Assuming it is one of those 9 commits (down from 28, as I quoted above, at my last completed auto-bisect step) I'll reset and try manually bisecting on the 9. It's looking good so far, but other kernels have looked good at this stage and then ultimately frozen with the telltale gpf log, so it remains to be seen.

Meanwhile, nice to be on a current development kernel and well past rc1 stage, again. =:^) Bisect-testing otherwise long-stale pre-rc1 kernels with other kernel-crasher bugs to complicate things is *not* my definition of fun!

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #49 from Christopher Snowhill (kode54@gmail.com) --- One possibility that I hadn't considered when I was originally testing this. I use the GNOME 3 desktop on Arch, and have two monitors, one 3840x2160@60Hz, one 1920x1080@60Hz, both DisplayPort. One thing I haven't enabled since I switched back from my Nvidia GTX 960 backup card was Variable Refresh Rate, which I had previously enabled in my Xorg configuration.

I never experienced crashes like these on page flips on my Nvidia card, and am awaiting a crash any day now on the RX 480, assuming I haven't magically configured it away with the deletion of that Xorg config snippet which did nothing but enable VRR.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #51 from rtmasura+kernel@hotmail.com --- that didn't read well, with vblank_mode off for XFWM I don't have this issue at all.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

Stratos Zolotas (strzol@gmail.com) changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |strzol@gmail.com

--- Comment #53 from Stratos Zolotas (strzol@gmail.com) --- Hi everyone.

Don't know if it helps. I'm getting a similar issue on Opensuse Tumbleweed with kernel 5.7.7. Reverting to kernel 5.7.5 makes things stable for me. My GPU is RX580.

Ιουλ 09 21:17:39.718030 teras.baskin.cywn kernel: general protection fault, probably for non-canonical address 0x3e9478a9ecb3abc8: 0000 [#1] SMP NOPTI Ιουλ 09 21:17:39.718200 teras.baskin.cywn kernel: CPU: 1 PID: 141 Comm: kworker/u16:3 Tainted: G O 5.7.7-1-default #1 openSUSE Tumbleweed (unreleased) Ιουλ 09 21:17:39.718239 teras.baskin.cywn kernel: Hardware name: Gigabyte Technology Co., Ltd. To be filled by O.E.M./970A-DS3P, BIOS FD 02/26/2016 Ιουλ 09 21:17:39.718273 teras.baskin.cywn kernel: Workqueue: events_unbound commit_work [drm_kms_helper] Ιουλ 09 21:17:39.718306 teras.baskin.cywn kernel: RIP: 0010:amdgpu_dm_atomic_commit_tail+0x273/0x10f0 [amdgpu] Ιουλ 09 21:17:39.718339 teras.baskin.cywn kernel: Code: 43 08 8b 90 e0 02 00 00 41 83 c6 01 44 39 f2 0f 87 3a ff ff ff 48 83 bd a0 fd ff ff 00 0f 84 03 01 00 00 48 8b bd a0 fd ff ff <80> bf b0 01 00 00 01 0f 86 ac 00 00 00 48 b9 00 00 00 00 01 00 00 Ιουλ 09 21:17:39.718368 teras.baskin.cywn kernel: RSP: 0018:ffffb7cf4037bbe0 EFLAGS: 00010202 Ιουλ 09 21:17:39.718400 teras.baskin.cywn kernel: RAX: ffff8fb2a5e11800 RBX: ffff8fb28f2c2880 RCX: ffff8fb10ff8ec00 Ιουλ 09 21:17:39.718442 teras.baskin.cywn kernel: RDX: 0000000000000006 RSI: ffffffffc0b7f530 RDI: 3e9478a9ecb3abc8 Ιουλ 09 21:17:39.718482 teras.baskin.cywn kernel: RBP: ffffb7cf4037be68 R08: 0000000000000001 R09: 0000000000000001 Ιουλ 09 21:17:39.718519 teras.baskin.cywn kernel: R10: 000000000000014d R11: 0000000000000018 R12: ffff8fb2a35e7400 Ιουλ 09 21:17:39.718547 teras.baskin.cywn kernel: R13: 0000000000000000 R14: 0000000000000006 R15: ffff8fb112001000 Ιουλ 09 21:17:39.718584 teras.baskin.cywn kernel: FS: 0000000000000000(0000) GS:ffff8fb2aec40000(0000) knlGS:0000000000000000 Ιουλ 09 21:17:39.718620 teras.baskin.cywn kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Ιουλ 09 21:17:39.718652 teras.baskin.cywn kernel: CR2: 00007f364b9bc000 CR3: 000000042a2ae000 CR4: 00000000000406e0 Ιουλ 09 21:17:39.718683 teras.baskin.cywn kernel: Call Trace: Ιουλ 09 21:17:39.718715 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.718750 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.718784 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.718810 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.718840 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.718868 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.718894 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.718921 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.718946 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.718972 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.718999 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719026 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719062 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719088 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719122 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719149 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719177 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719203 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719229 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719254 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719280 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719306 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719333 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719359 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719383 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719408 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719433 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719465 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719490 teras.baskin.cywn kernel: ? __switch_to_asm+0x40/0x70 Ιουλ 09 21:17:39.719515 teras.baskin.cywn kernel: ? __switch_to+0x152/0x380 Ιουλ 09 21:17:39.719545 teras.baskin.cywn kernel: ? __switch_to_asm+0x34/0x70 Ιουλ 09 21:17:39.719572 teras.baskin.cywn kernel: ? __schedule+0x1fe/0x560 Ιουλ 09 21:17:39.719605 teras.baskin.cywn kernel: ? usleep_range+0x80/0x80 Ιουλ 09 21:17:39.719637 teras.baskin.cywn kernel: ? _cond_resched+0x16/0x40 Ιουλ 09 21:17:39.719664 teras.baskin.cywn kernel: ? __wait_for_common+0x3b/0x160 Ιουλ 09 21:17:39.719690 teras.baskin.cywn kernel: commit_tail+0x94/0x130 [drm_kms_helper] Ιουλ 09 21:17:39.719727 teras.baskin.cywn kernel: process_one_work+0x1e3/0x3b0 Ιουλ 09 21:17:39.719760 teras.baskin.cywn kernel: worker_thread+0x46/0x340 Ιουλ 09 21:17:39.719795 teras.baskin.cywn kernel: ? process_one_work+0x3b0/0x3b0 Ιουλ 09 21:17:39.719830 teras.baskin.cywn kernel: kthread+0x115/0x140 Ιουλ 09 21:17:39.719888 teras.baskin.cywn kernel: ? __kthread_bind_mask+0x60/0x60 Ιουλ 09 21:17:39.719920 teras.baskin.cywn kernel: ret_from_fork+0x22/0x40 Ιουλ 09 21:17:39.719952 teras.baskin.cywn kernel: Modules linked in: rfcomm fuse af_packet vboxnetadp(O) vboxnetflt(O) cmac algif_hash vboxdrv(O) algif_skcipher af_alg bnep dmi_sysfs msr it87 hwmon_vid squashfs xfs nls_iso8859_1 nls_cp437 vfat fat loop edac_mce_amd uvcvideo kvm_amd pktcdvd ccp videobuf2_vmalloc videobuf2_memops videobuf2_v4l2 kvm videobuf2_common snd_hda_codec_realtek snd_hda_codec_generic snd_usb_audio irqbypass snd_hda_codec_hdmi ledtrig_audio videodev btusb snd_usbmidi_lib btrtl snd_rawmidi btbcm snd_hda_intel btintel snd_seq_device snd_intel_dspcfg crct10dif_pclmul crc32_pclmul mc ghash_clmulni_intel bluetooth joydev snd_hda_codec aesni_intel ecdh_generic crypto_simd rfkill ecc cryptd snd_hda_core glue_helper efi_pstore fam15h_power pcspkr k10temp sp5100_tco snd_hwdep i2c_piix4 snd_pcm r8169 snd_timer realtek snd libphy soundcore tiny_power_button button acpi_cpufreq tcp_bbr sch_fq hid_logitech_hidpp hid_logitech_dj uas usb_storage hid_generic usbhid btrfs sr_mod cdrom blake2b_generic libcrc32c xor amdgpu Ιουλ 09 21:17:39.720011 teras.baskin.cywn kernel: ohci_pci amd_iommu_v2 gpu_sched i2c_algo_bit ttm drm_kms_helper raid6_pq crc32c_intel ata_generic syscopyarea xhci_pci sysfillrect sysimgblt fb_sys_fops xhci_hcd cec ohci_hcd rc_core ehci_pci ehci_hcd drm pata_atiixp usbcore sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua efivarfs Ιουλ 09 21:17:39.720045 teras.baskin.cywn kernel: ---[ end trace 573bd378072b1ec2 ]--- Ιουλ 09 21:17:39.720078 teras.baskin.cywn kernel: RIP: 0010:amdgpu_dm_atomic_commit_tail+0x273/0x10f0 [amdgpu] Ιουλ 09 21:17:39.720105 teras.baskin.cywn kernel: Code: 43 08 8b 90 e0 02 00 00 41 83 c6 01 44 39 f2 0f 87 3a ff ff ff 48 83 bd a0 fd ff ff 00 0f 84 03 01 00 00 48 8b bd a0 fd ff ff <80> bf b0 01 00 00 01 0f 86 ac 00 00 00 48 b9 00 00 00 00 01 00 00 Ιουλ 09 21:17:39.720133 teras.baskin.cywn kernel: RSP: 0018:ffffb7cf4037bbe0 EFLAGS: 00010202 Ιουλ 09 21:17:39.720163 teras.baskin.cywn kernel: RAX: ffff8fb2a5e11800 RBX: ffff8fb28f2c2880 RCX: ffff8fb10ff8ec00 Ιουλ 09 21:17:39.720193 teras.baskin.cywn kernel: RDX: 0000000000000006 RSI: ffffffffc0b7f530 RDI: 3e9478a9ecb3abc8 Ιουλ 09 21:17:39.720241 teras.baskin.cywn kernel: RBP: ffffb7cf4037be68 R08: 0000000000000001 R09: 0000000000000001 Ιουλ 09 21:17:39.720294 teras.baskin.cywn kernel: R10: 000000000000014d R11: 0000000000000018 R12: ffff8fb2a35e7400 Ιουλ 09 21:17:39.720322 teras.baskin.cywn kernel: R13: 0000000000000000 R14: 0000000000000006 R15: ffff8fb112001000 Ιουλ 09 21:17:39.720351 teras.baskin.cywn kernel: FS: 0000000000000000(0000) GS:ffff8fb2aec40000(0000) knlGS:0000000000000000 Ιουλ 09 21:17:39.720380 teras.baskin.cywn kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Ιουλ 09 21:17:39.720409 teras.baskin.cywn kernel: CR2: 00007f364b9bc000 CR3: 000000042a2ae000 CR4: 00000000000406e0 Ιουλ 09 21:19:54.107989 teras.baskin.cywn kernel: [drm:do_aquire_global_lock.isra.0 [amdgpu]] *ERROR* [CRTC:49:crtc-1] hw_done or flip_done timed out Ιουλ 09 21:20:04.348029 teras.baskin.cywn kernel: [drm:do_aquire_global_lock.isra.0 [amdgpu]] *ERROR* [CRTC:49:crtc-1] hw_done or flip_done timed out

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #55 from Stratos Zolotas (strzol@gmail.com) --- (In reply to Paul Menzel from comment #54)

Thank you for your report. How quickly can you reproduce it? If you could bisect the issue to pinpoint the culprit commit between 5.7.5 and 5.7.7, that’d be great. Maybe open even a separate bug report, in case they are unrelated. They can always be marked as duplicates later.

If you guide me on what to do I can report back in some hours (not on that system now). I had 4 crashes yesterday with kernel 5.7.7 in 3 hours doing daily stuff (not gaming or something like that). System was unresponsive, ssh to the box worked but reboot from console hangs also, only ALT+SysRq+B reboots the system. I booted with the previous kernel (5.7.5) and was stable for over 6-7 hours.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #57 from Anthony Ruhier (anthony.ruhier@gmail.com) --- To give some precision about the kernel version range, I'm staying on 5.6.19 for a while, which doesn't have the issue. It's pretty bad though, as it's EOL.

Only the 5.7 branch has it. So it's something that wasn't backported.

I also have multimonitors, with one with VRR, though I don't know if VRR changes anything in my case as the 2 other screens don't support it.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #59 from chancuan66@gmail.com --- (In reply to Paul Menzel from comment #54)

(In reply to Stratos Zolotas from comment #53)

...

Don't know if it helps. I'm getting a similar issue on Opensuse Tumbleweed with kernel 5.7.7. Reverting to kernel 5.7.5 makes things stable for me. My GPU is RX580.

[…]

Thank you for your report. How quickly can you reproduce it? If you could bisect the issue to pinpoint the culprit commit between 5.7.5 and 5.7.7, that’d be great. Maybe open even a separate bug report, in case they are unrelated. They can always be marked as duplicates later.

I am running the same setup as the comment. RX 580, Tumbleweed, have both kernels 5.7.5 and 5.7.7. On 5.7.7, it happens almost immediately after login. However, reverting to 5.7.5 does NOT stabilise, and the same problem arises somewhere between 1 to 10 minutes.

I didn't have this issue prior to installing the 5.7.7 kernel though...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #61 from Christopher Snowhill (kode54@gmail.com) --- It may be worth noting that I also haven't experienced this crash lately, and one of the things I did recently was update my motherboard BIOS, which included an update from AGESA 1.0.0.4 release 2, to 1.0.0.6am4.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #63 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Duncan from comment #62)

I've applied the three 320-and-followups revert-patches to v5.8-rc5-8-g0dc589da8 and just did the rebuild with them applied. Now to reboot to it and see if it still has our bug.

NB: The 3202fa62f followups are cbfc35a48 and 89b83f282. That should let anyone else with git and kernel building skills try reverting the three.

Still too early (by days) to call it nailed down as I've had it take 2-3 days to trigger, but no gfx freeze here yet on that v5.8-rc5+ with 320-and-followups reverted so far, despite playing 4k video to try to trigger it as it has previously on affected kernels. I'll be trying update builds (gentoo) later today or tomorrow, another previous trigger, so we'll see how it goes.

But initial results are good enough to let others know that may want to try it...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #65 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Duncan from comment #63)

NB: The 3202fa62f followups are cbfc35a48 and 89b83f282. That should let anyone else with git and kernel building skills try reverting the three.

Still too early (by days) to call it nailed down as I've had it take 2-3 days to trigger, but no gfx freeze here yet on that v5.8-rc5+ with 320-and-followups reverted so far, despite playing 4k video to try to trigger it as it has previously on affected kernels. I'll be trying update builds (gentoo) later today or tomorrow, another previous trigger, so we'll see how it goes.

I'm still not saying for sure, but that's actually looking like the culprit.

Today's gentoo update included a dep of qtwebengine, which changed ABI so qtwebengine needed rebuilt on top of it, and qtwebengine is chromium-based. And as anyone that's built chromium (or firefox for that matter) can tell you, at least on older fx-based hardware, it's several hours of near constant 100% all-cores.

While rebuilding qtwebengine (at a batch-nice of +19 so it doesn't interfere too badly with anything else I want to run), I was playing youtube videos at 1080p, not normally a problem by themselves (tho 4k can be, especially 4k60) but with qtwebengine building at the same time...

No freezes.

I'm going to run with the 320 commit and followups reverted a few more days before declaring it for sure the culprit, and I'm watching for Anthony's results as well, but the bug's sure doing a convincing job of hiding ATM if that commit isn't the culprit!

I'd say it's time to start reviewing the amdgpu code to see what relocating the slub freelist pointer to the middle of the object (what the 320 commit did according to its git log explanation) could tickle, when the work goes on the work queue to run later, since that's consistently what the logs say is the scenario and what mnrzk confirmed by forcing it /not/ to go to the work queue in comment #30.

Hopefully we can still get and confirm a proper codefix by 5.8.0 release. =:^)

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #67 from Anthony Ruhier (anthony.ruhier@gmail.com) --- No freeze for me too, and I compiled firefox yesterday, which usually triggers a freeze on 5.7, and nothing yet. That's some really good news if it stays true, thanks a lot Duncan!

FYI, I applied the revert on 5.7.8, I didn't want to run on 5.8.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #69 from Anthony Ruhier (anthony.ruhier@gmail.com) --- (In reply to Stratos Zolotas from comment #68)

(In reply to Stratos Zolotas from comment #60)

...

To make things looks more strange... I have a non-explicable development with this issue. When it appeared to me I was in the middle of upgrading some components on my system. I replaced my AMD FX-8350 with one AMD Ryzen

5

...

3600X and my Gigabyte GA-970a-ds3p motherboard with one Gigabyte X570 UD (along with new RAM dimms from 16GB to 32GB). RX580 stayed the same and

also

...

OS is the same (disks moved to the new motherboard, no re-install). Guess what... running with 5.7.7 for 48 hours now without issues.... problem has disappeared. I suspect a very rare combination of things maybe even not in the amdgpu driver itself... With 5.7.7 on my "old" configuration, I had the crash almost immediately after login like in the above comment.

Just to report that got the issue after some days with my new hardware setup, so it is still there, hope you guys pinpoint it soon!

You're talking about having the bug with 5.7.7 vanilla, right? Not with the revert of the commits quoted above?

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #71 from Anthony Ruhier (anthony.ruhier@gmail.com) --- Just to give some news, I can confirm that I haven't had any freeze since Wednesday. Usually, when my system just idled, it would quickly trigger the bug. That or doing something CPU intensive (like compiling firefox). But nothing since I reverted the 3 commits.

Really good job Duncan! Thanks a lot for your debug!

MB chipset: x470 CPU: ryzen 2700x GPU: vega64

https://bugzilla.kernel.org/show_bug.cgi?id=207383

Jeremy Kescher (jeremy@kescher.at) changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |jeremy@kescher.at

--- Comment #73 from Jeremy Kescher (jeremy@kescher.at) --- Confirming as well that 3202fa62f, cbfc35a48 and 89b83f282 are the commits that cause this regression.

Tested with 5.7.9, Radeon RX 480.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #75 from Kees Cook (kees@outflux.net) --- Hi!

First, let me say sorry for all the work my patch has caused! It seems like it might be tickling another (previously dormant) bug in the gpu driver.

(In reply to mnrzk from comment #30)

I've been looking at this bug for a while now and I'll try to share what I've found about it.

In some conditions, when amdgpu_dm_atomic_commit_tail calls dm_atomic_get_new_state, dm_atomic_get_new_state returns a struct dm_atomic_state* with an garbage context pointer.

I've also found that this bug exclusively occurs when commit_work is on the workqueue. After forcing drm_atomic_helper_commit to run all of the commits without adding to the workqueue and running the OS, the issue seems to have disappeared. The system was stable for at least 1.5 hours before I manually shut it down (meanwhile it has usually crashed within 30-45 minutes).

Perhaps there's some sort of race condition occurring after commit_work is queued?

If it helps to explain what's happening in 3202fa62f, the kernel memory allocator is moving it's free pointer from offset 0 to the middle of the object. That means that when the memory is freed, it writes 8 bytes to join the newly freed memory into the allocator's freelist. That always happened, but after 3202fa62f it began writing it in the middle, not offset 0. If the work queue is trying to use freed memory, and before it didn't notice the first 8 bytes getting written, now it appears to notice the overwrite... but that still means something is freeing memory before it should.

Finding that might be a real trick. :( However, if you've suffered through all those bisections, I wonder if you can try one other thing, which is to compile the kernel with KASAN:

CONFIG_KASAN=y CONFIG_KASAN_GENERIC=y CONFIG_KASAN_OUTLINE=y CONFIG_KASAN_STACK=y CONFIG_KASAN_VMALLOC=y

This will make things _slow_, which might mean the use-after-free race may never trigger. *However* it's possible that it'll catch a bad behavior before it even needs to get hit in a race that triggers the behavior you're seeing. (And note that swapping CONFIG_KASAN_OUTLINE=y for CONFIG_KASAN_INLINE=y might speed things up, but the kernel image gets bigger).

I'm going to try to read the work queue code for the driver and see if anything obvious stands out...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #77 from Kees Cook (kees@outflux.net) --- (Midair collision... you saw the same about the structure layout as I did. Here's my comment...)

(In reply to mnrzk from comment #30)

I've been looking at this bug for a while now and I'll try to share what I've found about it.

In some conditions, when amdgpu_dm_atomic_commit_tail calls dm_atomic_get_new_state, dm_atomic_get_new_state returns a struct dm_atomic_state* with an garbage context pointer.

It looks like when amdgpu_dm_atomic_commit_tail() walks the private objects list with for_each_new_private_obj_in_state(), it'll return the first object's state when the function pointer tables match. This is a struct dm_atomic_state allocation, which is 16 bytes:

struct drm_private_state { struct drm_atomic_state *state; };

struct dm_atomic_state { struct drm_private_state base; struct dc_state *context; };

If struct dm_atomic_state is being freed early, this would match the behavior seen: before 3202fa62f, .base.state would be overwritten with a freelist pointer. After 3202fa62f, .context will be overwritten.

In looking for all "kfree(.*state" patterns in drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c, I see a few suspicious things, maybe. dm_crtc_destroy_state() and amdgpu_dm_connector_funcs_reset() do an explicit kfree(state) -- should they use dm_atomic_destroy_state() instead? Or nothing at all, since I'd expect "state" to be managed by the drm layer via the .atomic_destroy_state callback?

I've also found that this bug exclusively occurs when commit_work is on the workqueue. After forcing drm_atomic_helper_commit to run all of the commits without adding to the workqueue and running the OS, the issue seems to have disappeared. The system was stable for at least 1.5 hours before I manually shut it down (meanwhile it has usually crashed within 30-45 minutes).

Is this the async call to "commit_work" in drm_atomic_helper_commit()?

There's a big warning in there:

/* * Everything below can be run asynchronously without the need to grab * any modeset locks at all under one condition: It must be guaranteed * that the asynchronous work has either been cancelled (if the driver * supports it, which at least requires that the framebuffers get * cleaned up with drm_atomic_helper_cleanup_planes()) or completed * before the new state gets committed on the software side with * drm_atomic_helper_swap_state(). ...

I'm not sure how to determine if amdgpu_dm.c is doing this correctly?

I can't tell what can interfere with drm_atomic_helper_commit() -- I would guess the race is between that and something else causing a kfree(), but I don't know the APIs here at all...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #79 from mnrzk@protonmail.com --- (In reply to Kees Cook from comment #78)

(In reply to mnrzk from comment #76)

...

If my understanding is correct, base would have previously been filled with the freelist pointer (since it's the first 8 bytes). Now since the freelist pointer is being put in the middle (rounded to the nearest sizeof(void*), or 8 bytes), it's being put in the last 8 bytes of *dm_state (or dm_state->context).

I'll place a void* for padding in the middle of struct dm_atomic_state* and if my hypothesis is correct, the padding will be filled with garbage data instead of context and the bug should be fixed. Of course, there would still be a use-after-free bug in the code which may cause other issues in the future so I wouldn't really consider it a solution.

Agreed: that should make it disappear again, but as you say, it's just kicking the problem down the road since now the failing condition is losing a race with kfree()+kmalloc()+new contents.

And if you want to detect without crashing, you can just zero the padding at init time and report when it's non-NULL at workqueue run time... I wonder if KASAN can run in a mode where the allocation/freeing tracking happens, but without the heavy checking instrumentation? Then when the corruption is detected, it could dump a traceback about who did the early kfree()... hmmm.

So far I've been testing it by passing my GPU to my VM via vfio-pci and attaching kgdb to the guest. To test if the context was invalid, I added a check to make sure the context pointer wasn't garbage data (by checking if dc_state was not null and the upper 16 bits were set on dc_state).

I wonder if there's any way to set a watchpoint to see where exactly the dm_atomic_state gets filled with garbage data.

Also, since I'm not too familiar with freelists, do freelist pointers look like regular pointers? On a regular pointer on a system with a 48-bit virtual address space, regular pointers would be something like 0xffffXXXXXXXXXXXX. I've noticed that the data being inserted never followed this format. Is this something valuable to note or is that just the nature of freelist pointers?

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #83 from Christian König (christian.koenig@amd.com) --- Instead of working around the bug I think we should concentrate on nailing the root cause.

I suggest to insert an use after free check into just that structure. In other words add a field "magic_number" will it with 0xdeadbeef on allocation and set it to zero before the kfree().

A simple BUG_ON(ptr->magic_number != 0xdeadbeef) should yield results rather quickly.

Then just add printk()s before the kfree() to figure out why we have this use after free race.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #85 from mnrzk@protonmail.com --- (In reply to Christian König from comment #83)

Instead of working around the bug I think we should concentrate on nailing the root cause.

I suggest to insert an use after free check into just that structure. In other words add a field "magic_number" will it with 0xdeadbeef on allocation and set it to zero before the kfree().

A simple BUG_ON(ptr->magic_number != 0xdeadbeef) should yield results rather quickly.

Then just add printk()s before the kfree() to figure out why we have this use after free race.

Fair point, I was just trying to confirm my hypothesis.

I realised why the test failed, adding 8 bytes of padding to the middle made the struct size 24 bytes. Since the freelist pointer is being added to the middle (12 bytes) and that's aligned to the nearest 8 bytes, the pointer ended up being placed at an offset of 16 bytes (context).

After making the padding an array of 2 void* and initialising it to {0xDEADBEEFCAFEF00D, 0x1BADF00D1BADC0DE}, the padding was eventually corrupted with the context being left intact and therefore, no crashing.

GDB output of dm_struct: { base = {state = 0xffff888273884c00}, padding = {0xdeadbeefcafef00d, 0x513df83afd3ad7b2}, context = 0xffff88824e680000 }

That said, I still don't know the root cause of the bug, I'll see if I can use KASAN or something to figure out what exactly freed dm_state. If anyone is more familiar with this code has any advice for me, please let me know.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #87 from mnrzk@protonmail.com --- Good news, I wrote a patch that fixed this bug on my machine and submitted it to the Linux kernel mailing list [1].

I've tested this for almost 12 hours with KASAN enabled and 3 hours with all debugging options disabled while watching videos and there have been no crashes. The longest it's taken for the bug to occur in the past for me was about 1 hour.

To anyone experiencing this bug, please test out the patch and report on whether on not it works. I think we'll need some Tested-bys in the LKML thread and in here before we can consider this bug fixed.

[1] https://lkml.org/lkml/2020/7/23/1123

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #89 from Christian König (christian.koenig@amd.com) --- (In reply to mnrzk from comment #87)

Good news, I wrote a patch that fixed this bug on my machine and submitted it to the Linux kernel mailing list [1].

You should probably send it to the amd-gfx@lists.freedesktop.org mailing list as well if you haven't already done so.

I'm not an expert on the DC state stuff, so Harry or Alex need to validate this patch. But of hand it looks like a nice catch to me.

Good work :)

https://bugzilla.kernel.org/show_bug.cgi?id=207383

laser.eyess.trackers@gmail.com changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |laser.eyess.trackers@gmail. | |com

--- Comment #91 from laser.eyess.trackers@gmail.com --- I wanted to comment on this bug because I believe I have been experiencing it based on a bug report I filed with amdgpu[1]. As of 5.7.8 on Arch Linux I no longer experiencing this bug regularly. Usually I could trigger it every 1-3 days. The biggest change I made was turning off adaptive_sync (VRR, Freesync, etc.) in my window manager. Now it's been almost a week and I haven't seen it. Right now I am on 5.7.9 and will keep running as long as possible until it crashes again, if it crashes again.

I see some discussion here about race conditions between memory allocations and atomic commits, and while I don't understand most of it, would I be correct in assuming that variable frame timing would exacerbate this bug? If so, I believe that is exactly what I am experiencing. I'd love to help test patches for this as they come in, but for now I want to add that VRR is an important part of the equation for this bug for me.

The bug report linked in [1] has more of my set up but all I'll say here is that I also have a multimonitor setup, each one supports VRR and they are at varying resolutions/refresh rates; two at 1440p 144Hz, one at 4k 60Hz.

1. https://gitlab.freedesktop.org/drm/amd/-/issues/1216

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #93 from mnrzk@protonmail.com --- (In reply to Nicholas Kazlauskas from comment #92)

This sounds very similar to a bug I fixed a year ago but that issue was with freeing the dc_state.

https://bugzilla.kernel.org/show_bug.cgi?id=204181

1. Client requests non-blocking Commit #1, has a new dc_state #1,

state is swapped, commit tail is deferred to work queue

2. Client requests non-blocking Commit #2, has a new dc_state #2,

state is swapped, commit tail is deferred to work queue

3. Commit #2 work starts before Commit #1, commit tail finishes,

atomic state is cleared, dc_state #1 is freed

4. Commit #1 work starts after Commit #2, uses dc_state #1, NULL pointer

deref.

This issue was fixed, but it occurred under similar conditions - heavy system load and frequent pageflipping.

However, in the case of dm_state things can't be solved in the same manner. Commit #2 can't free Commit #1's commit - only the commit tail for Commit #1 can free it along with the IOCTL caller.

I don't know if this is going down any of the deadlock paths in DRM core because that might trigger strange behavior as well with clearing/putting the dm_state.

If someone who can reproduce this issue can produce a dmesg log with the DRM IOCTLs logged (I think drm.debug=0x54 should work) then I should be able to examine the IOCTL sequence in more detail.

Yes, this actually seems quite similar to that bug. Perhaps it's something like that bug but with dm_state instead?

Also, some more observations I've made: While dm_state is encountering a use-after-free bug, it does not seem like state as a whole is. The KASAN bug report only states that reading from dm_state is invalid, but the same cannot be said about state.

Furthermore, dm_state seems to be used in two separate commits and is being freed after one commit is complete. This creates a race between the two commits where the completion of one commit before the other calls dm_atomic_get_new_state causes a use-after-free.

I think the bug works something like this. Keep in mind that I haven't worked with this code outside of this bug report so there may be a few misconceptions:

1. Client requests non-blocking Commit #1, has a new dm_state #1, state is swapped, commit tail is deferred to work queue

2. Client requests non-blocking Commit #2, has a new dm_state #2, state is swapped, commit tail is deferred to work queue

3. Commit #2 work starts before Commit #1, commit tail finishes, atomic state is cleared, dm_state #1 is freed

4. Commit #1 work starts after Commit #2, uses dm_state #1 (use-after-free), reads bad context pointer and dereferences freelist pointer instead.

So I would agree that this is very similar to the dc_state bug (I even based that explanation on yours). Perhaps that bug you fixed also affected dm_state as a whole but only caused an issue with dc_state at the time?

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #95 from Nicholas Kazlauskas (nicholas.kazlauskas@amd.com) --- Created attachment 290583 --> https://bugzilla.kernel.org/attachment.cgi?id=290583&action=edit 0001-drm-amd-display-Force-add-all-CRTCs-to-state-when-us.patch

So the sequence looks like the following:

1. Non-blocking commit #1 requested, checked, swaps state and deferred to work queue.

2. Non-blocking commit #2 requested, checked, swaps state and deferred to work queue.

Commits #1 and #2 don't touch any of the same core DRM objects (CRTCs, Planes, Connectors) so Commit #2 does not stall for Commit #1. DRM Private Objects have always been avoided in stall checks, so we have no safety from DRM core in this regard.

3. Due to system load commit #2 executes first and finishes its commit tail work. At the end of commit tail, as part of DRM core, it calls drm_atomic_state_put().

Since this was the pageflip IOCTL we likely already dropped the reference on the state held by the IOCTL itself. So it's going to actually free at this point.

This eventually calls drm_atomic_state_clear() which does the following:

obj->funcs->atomic_destroy_state(obj, state->private_objs[i].state);

Note that it clears "state" here. Commit sets "state" to the following:

state->private_objs[i].state = old_obj_state; obj->state = new_obj_state;

Since Commit #1 swapped first this means Commit #2 actually does free Commit #1's private object.

4. Commit #1 then executes and we get a use after free.

Same bug, it's just this was never corrupted before by the slab changes. It's been sitting dormant for 5.0~5.8.

Attached is a patch that might help resolve this.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

mnrzk@protonmail.com changed:

What |Removed |Added ---------------------------------------------------------------------------- Attachment #290485|0 |1 is obsolete| |

--- Comment #97 from mnrzk@protonmail.com --- Created attachment 290591 --> https://bugzilla.kernel.org/attachment.cgi?id=290591&action=edit drm/amd/display: Clear dm_state for fast updates

drm/amd/display: Clear dm_state for fast updates

Alright, the bug patch I mentioned in the last comment seems to be good after a few hours of testing.

Please try out this patch and see if it fixes the issue for the rest of you.

In the meantime, I'm doing more extended tests on this patch to confirm it works well enough before posting it on LKML.

Nicholas, I haven't tested your commit since I was too busy with this. I'll try it out if this one fails though.

Also, can you please review this patch to confirm that I'm not doing anything wrong here?

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #99 from mnrzk@protonmail.com --- (In reply to Nicholas Kazlauskas from comment #98)

As much as I'd like to remove the DRM private object from the state instead of just carrying it over I'd really rather not be hacking around behavior from the DRM core itself.

Maybe there's value in adding these as DRM helpers in the case where a driver explicitly wants to remove something from the state. My guess as to why these don't exist today is because they can be bug prone since the core implicitly adds some objects (like CRTCs when you add a plane and CRTCs when you add connectors) but I don't see any technical limitation for not exposing this.

I'm a little bit confused, is there anything particularly illegal or discouraged about the patch I sent? If so, how should I correct it?

Should I create some sort of DRM helper for deleting a private object and use that to delete the state's associated private object?

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #101 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Nicholas Kazlauskas from comment #95)

Created attachment 290583 [details] 0001-drm-amd-display-Force-add-all-CRTCs-to-state-when-us.patch

Just booted to 5.8-rc7 with this patched in locally (and the g320+ reverts /not/ patched in). So testing, but noting again that the bug can take a couple days to trigger on my hardware, so while verifying bug-still-there /might/ be fast, verifying that it's /not/ there will take awhile.

If this still bugs on me (and barring other developments first) I'll try mnrzk's patch in place of this one. Even if it's not permanent, getting it into 5.8 as a temporary fix and doing something better for 5.9 would buy us some time to develop and test the more permanent fix.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #103 from mnrzk@protonmail.com --- (In reply to Nicholas Kazlauskas from comment #95)

Created attachment 290583 [details] 0001-drm-amd-display-Force-add-all-CRTCs-to-state-when-us.patch

So the sequence looks like the following:

1. Non-blocking commit #1 requested, checked, swaps state and deferred to

work queue.

2. Non-blocking commit #2 requested, checked, swaps state and deferred to

work queue.

Commits #1 and #2 don't touch any of the same core DRM objects (CRTCs, Planes, Connectors) so Commit #2 does not stall for Commit #1. DRM Private Objects have always been avoided in stall checks, so we have no safety from DRM core in this regard.

3. Due to system load commit #2 executes first and finishes its commit tail

work. At the end of commit tail, as part of DRM core, it calls drm_atomic_state_put().

Since this was the pageflip IOCTL we likely already dropped the reference on the state held by the IOCTL itself. So it's going to actually free at this point.

This eventually calls drm_atomic_state_clear() which does the following:

obj->funcs->atomic_destroy_state(obj, state->private_objs[i].state);

Note that it clears "state" here. Commit sets "state" to the following:

state->private_objs[i].state = old_obj_state; obj->state = new_obj_state;

Since Commit #1 swapped first this means Commit #2 actually does free Commit #1's private object.

4. Commit #1 then executes and we get a use after free.

Same bug, it's just this was never corrupted before by the slab changes. It's been sitting dormant for 5.0~5.8.

Attached is a patch that might help resolve this.

So I just got around to testing this patch and so far, not very promising.

Right now I can't comment on if the bug in question was resolved but this just introduced some new critical bugs for me.

I first tried this on my bare metal system w/ my RX 480 and it boots into lightdm just fine. As soon as I log in and start up XFCE however, one of my two monitors goes black (monitor reports being asleep) but my cursor seems to drift into the other monitor just fine. So after that, I check the display settings and both monitors are detected. So I tried re-enabling the off monitor and then both monitors work fine.

After that, another bug: I now have two cursors, one only works on my right monitor and the other only stays in one position.

At this point, I recompiled and remade the initramfs, and sure enough, same issues. This time, however, changing the display settings didn't "fix" the issue with one monitor being blank; the off monitor activated, but the previously working one just froze.

I also tried this on my VM passing through my GPU w/ vfio-pci; similar issues. Lightdm worked fine but when I started KDE Plasma, it started flashing white and one of my monitors just became blank. This time, I couldn't enable the blank display from the settings, it just didn't show up. Xrandr only showed one output as well; switching HDMI outputs still only lets me use the monitor on the "working" HDMI port.

I don't exactly know how I would go about debugging this since there's just too many bugs to count. I also don't know if it would be worth it at all.

Do you have any idea why this would occur? This patch only seems to force synchronisation, I don't quite know why it would break my system so much.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #105 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Duncan from comment #102)

(In reply to Duncan from comment #101)

...

(In reply to Nicholas Kazlauskas from comment #95)

...

0001-drm-amd-display-Force-add-all-CRTCs-to-state-when-us.patch

Just booted to 5.8-rc7 with this patched in

So far building system updates so heavy cpu load while playing only moderate FHD video. No freezes but I have seen a bit of the predicted judder.

The 4k30 and 4k60 youtube tests will probably have to wait for tomorrow, tho, as I've been up near 24 now...

Still up... Here's the promised 4k youtube-in-firefox tests.

4k is a bit more stuttery than normal with the patch, but not near as bad as I expected it to be. I can normally run 4k60 at 80-85% normal speed with occasional stutters but without freezing the video entirely until I drop the speed down again as I often have to do if I try running over that. With the patch I was doing 70-75%. So there's definitely some effect on 4k60. Switching to the performance cpufreq governor from my default conservative, as usual, helps a bit, but not a lot, maybe 5%, tho the frame-freezes seem to recover a bit better on performance. In addition to long video freezes at the full 4k60 100%, even normally I'll sometimes get tab-crashes depending on the video. I didn't have any for this test but then I'm so used to not being able to run at full-speed that I didn't try it for long.

I can normally run 4k30 videos without much problem on default conservative. With the patch I was still getting some stuttering at 30fps on conservative, but it pretty much cleared up with on-demand. I did just have a tab-crash at 4k30, something I very rarely if ever see normally on 4k30, it normally takes 4k60 to trigger them, so it's definitely affecting it.

But... other than slowing down the usable 4k fps, I'm not seeing any of judder artifacts on the work (non-video-playing) monitor that I was seeing with the high system load but relatively low video load build testing with only FHD video. That surprised me. I expected to see more of that with the more demanding video. But apparently that's tied to CPU or memory load, not video load.

But nothing like the problems mnrzk's seeing with the patch, at all. Both monitors running fine in text mode, login, startx to plasma, running fine there too. Hardware cursor's fine. <shrug> The only thing I'm seeing is some slowdown and judder, as described above.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #107 from Paul Menzel (pmenzel+bugzilla.kernel.org@molgen.mpg.de) --- Everyone seeing this, it’d be great, if you tested

[PATCH] drm/amd/display: Clear dm_state for fast updates

and reported any noticeable performance regressions.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #109 from zzyxpaw@gmail.com --- I've been testing mnrzk's patch for about 12 hours as well, so far so good. No obvious performance degradation has appeared, at least that I can discern just by "feel". My testing has been interrupted a couple times by the new power-off-on-overtemperature feature while attempting to test heavier loads.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #111 from mnrzk@protonmail.com --- Yeah, no noticeable performance impact on my end either. I don't really see why it would cause a performance impact either. I could run a benchmark to compare but I don't really know what to benchmark specifically.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #113 from laser.eyess.trackers@gmail.com --- I have been using this patch for about 24 hours now, and there has not been any noticeable performance degradation. I have not experienced any crashes, but it was much harder for me to get this crash (1-3 days, if I was lucky), so I'm not sure what that means. At the very least nothing is worse.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #115 from Duncan (1i5t5.duncan@cox.net) --- So 5.8.0 has been out for a few hours with the patch-fix, and I see Greg K-H has it applied to the 5.7 stable tree as well as 5.4 LTS (the bug was in 5.4 but latent, not exposed until developments in 5.7), so they should be covered in their next releases.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #117 from Duncan (1i5t5.duncan@cox.net) --- For those on stable-series 5.4 and/or interested in related bugs...

FWIW, there's an (apparently different) atomic_commit_tail bug reported against 5.4.58 now, bug #208913, with the patch for this bug (which went into 5.4.56 after hitting a late 5.8-rc) originally listed as a potential trigger.

But the filer closed the bug and moved it to the gitlab instance on freedesktop.org https://gitlab.freedesktop.org/drm/amd/-/issues/1263 , where he said reverting the patch didn't cure his issue, so there's something else going on there.

Just posting this here as related, in case anyone here wants to follow it, since I came across it while checking on a different (not graphics-related) bug in 5.9-rc1. With any luck, however, the similar bug will help get a better longer term fix for both bugs, since the patch in 5.8 (backported to 5.7 and 5.4) was seen as a temporary bandaid, not a permanent fix.

https://bugzilla.kernel.org/show_bug.cgi?id=207383

--- Comment #119 from Duncan (1i5t5.duncan@cox.net) --- (In reply to Christopher Snowhill from comment #118)

Now experiencing this attempting to run Luxmark with literally any OpenCL runtime on my RX 480, be it ROCm, Clover, or AMDGPU Pro 20.45. Goody gumdrops, wlroots added support for adaptive sync / variable refresh rate 6 months ago, but Wayfire still hasn't added an option to control it, so it may or may not be in the mix.

Happens with both 5.10.4 and 5.4.86 kernels.

FWIW: While I'd normally be on 5.11-rc by now, I've been busy switching to wayland (on plasma/kwin), along with the usual disruption accompanying such a major upgrade, and thus have been focused on userspace bugs.

So did you try earlier 5.10 releases, 5.10.0-5.10.3, and were they fine? Because I'm still on 5.10.0 and haven't had an issue since this bug was originally resolved.

Maybe I better get with the program and try current 5.11-rcs...

https://bugzilla.kernel.org/show_bug.cgi?id=207383

akanar@pm.me changed:

What |Removed |Added ---------------------------------------------------------------------------- CC| |akanar@pm.me

--- Comment #121 from akanar@pm.me --- Is the linked issue possibly related to the issue discussed in this thread? If it is you may at least have a 100% consistent and easy way to test.

https://github.com/prusa3d/PrusaSlicer/issues/6677