Graphics Freeze: amdgpu *ERROR* [CRTC:67:crtc-0] flip_done timed out
Brief summary of the problem:
My Laptop often experiences random complete graphical freezes, requiring a hard reboot. Sound persists, but mouse and keyboard are unresponsive.
I experience this freeze 1-2 times per day, typically when launching an application. Ironically, the most reliable way I can make the system freeze is by running lspci -nn to get information for this bug report.
I can reproduce this bug on kernels 5.12.x-5.17.4 (All kernels shipped in Fedora 35).
Hardware description:
- Laptop Hardware Model: Dell Inspiron G5 5505
- CPU: AMD Ryzen 7 4800h
- GPU: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT]
- System Memory: 16 GB
- Type of Display Connection: Laptop Screen
System information:
- Distro name and Version: Fedora 35 Workstation
- Kernel version: 5.17.5-200.fc35.x86_64
- Desktop Environment: Gnome
- Display Protocol: Wayland
How to reproduce the issue:
The process of writing this bug report has given me the most reliable way to reproduce my system freeze. I boot up, and repeatedly run the lspci -nn command, it will freeze within running the command five times.
Attached Log Files:
Activity
mentioned in issue #2068
I upgraded to kernel 5.19.4 over the weekend, and still experience kernel freezes related to amdgpu drm. It's still random, but the freezes are notably more frequent with programs that interact with Wine/Proton (Steam & Bottles).
Here's the latest dmesg log from the last freeze: 22-08-30_dmesg.log
mentioned in issue #2135 (closed)
Latest freeze 3 minutes after a cold boot: 2022-09-19_dmesg.log
Running kernel 5.19.9 and linux-firmware 20220815-139
Please some developer reply so I know you've at least seen this bug report! I can ssh into my system during a freeze, maybe I can run live diagnostics or something? Anything to help find the root cause of this error.
I see a lot of:
May 06 22:32:32 fedorapc kernel: [drm] Fence fallback timer expired on ring gfx May 06 22:32:32 fedorapc kernel: [drm] Fence fallback timer expired on ring sdma0in your logs. This is often means that interrupts are not working properly on your platform. You might try a newer bios if one is available. Are you seeing any interrupts being delivered to any CPUs for amdgpu in /proc/interrupts?
I have been on the latest Dell BIOS 1.11.0 since April, but I see there was a new version released at the end of September, so I will upgrade and try that.
I do see interrupts for amdgpu. Here's the output of
cat /proc/interrupts | grep amd:interrupts.logFor reference, I'm currently on kernel 5.19.15
try kernel version 5.10.20 that one worked for me and not seeing
flip_done timeouthangs
mentioned in issue #2184 (closed)
I'll keep adding logs to this issue. It's really hard to pin down, because I've not found any specific action to trigger the freeze. The most common triggers are launching Discord, Minecraft, and Steam. But, I can go days without issues or it freezes within minutes of booting without even launching any programs.
Here's a recent log where it froze withing a minute of a cold boot: 2022-11-16-dmesg.log
This is Fedora 36, kernel 6.0.8-200.fc36, amd-firmware 20221109-144.fc36
Latest log from today: 2022-11-22-dmesg.log
This is Fedora 37, kernel 6.0.9-300.fc37, amd-gpu-firmware 20221109-144.fc37.
Only action taken was cold boot, login, then launch Firefox, resulting in immediate freeze.
mentioned in issue #1707 (closed)
Just got the exact problem with fedora 37 and linux 6.2.8 on a thinkpad x13 gen3 with APU r7 pro 6850U and a radeon 680M
amdgpu 0000:33:00.0: [drm] *ERROR* [CRTC:67:crtc-0] flip_done timed outFirmware : amd-gpu-firmware.noarch 20230310-148.fc37 @updates
It happened only when I tried to reenable the PSR afer flickering that should be fixed in 6.2.8 #2453 (closed) https://forums.lenovo.com/t5/Fedora/Random-screen-flickering-on-T14s-Gen-3-AMD-in-Fedora-37/m-p/5198472?page=8#5941992 And for me it crashed after only 1-2 min post-boot when using chrome so...............
So try disabling PSR with
amdgpu.dcdebugmask=0x10, for me doing it again avoided the problem.Edited by naguammentioned in issue #2453 (closed)
added DC label
juuni 22 22:39:31 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:63:crtc-1] hw_done or flip_done timed out juuni 22 22:39:41 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:66:crtc-2] hw_done or flip_done timed out juuni 22 22:39:51 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:69:crtc-3] hw_done or flip_done timed out juuni 22 22:40:02 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:63:crtc-1] hw_done or flip_done timed out juuni 22 22:41:14 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:66:crtc-2] hw_done or flip_done timed out juuni 22 22:41:28 Zen kernel: [drm:amdgpu_dm_atomic_check [amdgpu]] *ERROR* [CRTC:66:crtc-2] hw_done or flip_done timed outjust right now, first time ever such problem, on my desktop:
- Ryzen 5950X
- Vega 64
- linux-firmware 20230117.7e4f0ed-1 (had to downgrade linux-firmware yesterday from 20230404.2e92a49f-1 because that caused some issues with Ryzen 5950X), which previously worked fine.
- 6.3.5-2-MANJARO
- Xfce; X11
- mesa-git 23.2.0_devel.172685.ce07aabab12.d41d8cd98f00b204e9800998ecf8427e-1
Edited by D33M0NI'm still using Fedora, now version 38, with GNOME, mesa 23.1.3-1.fc38. Since kernel 6.2.x, this problem has started occurring multiple times a day: just logging into the desktop, just launching Firefox, launching Steam. It seems like any random thing might cause this freeze.
But today, I had a slightly different behavior, maybe the logs can help. I restarted a game in Steam, causing the same "[CRTC:67:crtc-0] flip_done timed out" error, but the system managed to recover, then freeze again completely a few minutes later.
Full dmesg log: 2023-07-12_dmesg_full.log
dmesg log, filtered for errors 2023-07-12_dmesg_err.log
Edited by Owen Tyndall@tyn_inks would you please try these options
- disable amdgpu.aspm=off
- fix pcie lane/gen, e.g. pcie_lane_cap=0x00400000 pcie_gen_cap=0x00000010
Unfortunately, I am still experiencing the same CRTC:67 error. For 6.4.x & 6.5.x kernels, it freezes about once every week. All freezes only occur when launching a program: launching Firefox, Thunderbird, Krita, Steam, and Discord all have caused this freeze.
This is an improvement over 6.3.x and earlier kernels, where I was experiencing this freeze several times a day, even multiple an hour.
Whatever this issue is (maybe a multitude of issues causing the same error) it goes back to previous lts kernels as well. Switching to 6.1-lts and 5.15-lts do not get rid of this error, it maybe only impacts the frequency of the error.
I'm running 6.5.5 right now. It's definitely better than 6.2-6.4, but only about as stable as 6.1 lts for me. The crashes are different now, so I'm pretty certain 6.5.5 fixed something, but there are more issues it seems. I can always get a crash now by running Wayland and plugging or unplugging an extra monitor. But this does not happen on X11.
When it crashes on X11, the system is still running, just the entire GUI is unresponsive, keyboard capslock light is also not responsive, but plugging USB in/out causes notification sounds.
When it crashes on Wayland, Plasma does something and a few seconds later the UI is all back to normal.
Looking at logs in journalctl I have tons of:
[drm:amdgpu_cs_ioctl [amdgpu]] *ERROR* Failed to process the buffer list -22!And:
[drm:dm_plane_helper_prepare_fb [amdgpu]] *ERROR* Failed to pin framebuffer with error -22A spattering of:
mce: [Hardware Error]: Machine check events logged [Hardware Error]: Corrected error, no action required. [Hardware Error]: CPU:1 (17:60:1) MC0_STATUS[-|CE|-|-|-|-|-|-|-|-]: 0x8000000000000000 [Hardware Error]: IPID: 0x000000b000000000 [Hardware Error]: Load Store Unit Ext. Error Code: 0, Load queue parity error. [Hardware Error]: cache level: RESV, tx: INSNAnd:
[drm] Fence fallback timer expired on ring sdma0 [drm] Fence fallback timer expired on ring gfx_high [drm] Fence fallback timer expired on ring gfx_high [drm] Fence fallback timer expired on ring gfx_low [drm] Fence fallback timer expired on ring gfx_low [drm] Fence fallback timer expired on ring sdma0 [drm] Fence fallback timer expired on ring gfx_low [drm] Fence fallback timer expired on ring gfx_low [drm] Fence fallback timer expired on ring gfx_low [drm] Fence fallback timer expired on ring sdma0 amdgpu 0000:09:00.0: [drm] *ERROR* [CRTC:67:crtc-0] flip_done timed outAnd:
[drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring gfx_low timeout, signaled seq=1573363, emitted seq=1573365 [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process kwin_x11 pid 2296 thread kwin_x11:cs0 pid 2319The actual
CRTC:67error isn't super common, but it's still there. These all happened on 6.5.5:Linux version 6.5.5-gentoo (root@grigibook) (gcc (Gentoo 13.2.1_p20230826 p7) 13.2.1 20230826, GNU ld (Gentoo 2.40 p5) 2.40.0) #1 SMP PREEMPT_DYNAMIC Thu Oct 5 21:27:38 BST 2023I suspect there are so many causes of the same symptoms, sadly.
@superm1, here are logs for 6.5.5 & 6.5.6, both from today, with the [CRTC:67:crtc-0] drm error.
For good measure, here's a log from 6.5.7 with the freeze as well.
On kernel 6.5.11, I'm still seeing the [CRTC:67:crtc-0] drm error, but the freeze behavior is slightly different.
Here's a log where the graphics froze after launching Steam, but then the screen turned on-off several times before freezing:
2023-11-14-dmesg_steam.logAnd here's one where the graphics froze after launching Firefox. The screen turned on-off once then successfully recovered. However, it would later freeze after launching Steam: 2023-11-14-dmesg_firefox_steam.log
Edited by Owen Tyndall
