Kernel backtrace in gk104_fifo_recover_engn on TU104 (2070 SUPER)
I hit this backtrace earlier. The kernel module is driving a secondary display. Backtrace happened while turning on that monitor (after a long period where the monitor was turned off completely using power button).
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: fifo: fault 01 [VIRT_WRITE] at 000000000b802000 engine 40 [GR] client 13 [GPC5/PROP_0] reason 00 [PDE] on channel 2 [01ff8f0000 systemd-logind[805]]
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: fifo: channel 2: killed
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: fifo: runlist 0: scheduled for recovery
Jan 18 09:09:09 hostname kernel: ------------[ cut here ]------------
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: timeout
Jan 18 09:09:09 hostname kernel: WARNING: CPU: 2 PID: 96314 at drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c:447 gk104_fifo_recover_engn+0x257/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: Modules linked in: tun fuse wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 libchacha poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libcurve25519_gen>
Jan 18 09:09:09 hostname kernel: i2c_smbus intel_spi spi_nor snd_pcm mtd snd_timer snd mxm_wmi mei_me soundcore ttm rfkill mei intel_pch_thermal wmi sch_fq_codel nfsd crc32c_intel it87 hwmon_vid coretemp i2c_dev efivarfs
Jan 18 09:09:09 hostname kernel: CPU: 2 PID: 96314 Comm: kworker/2:2 Tainted: G W T 5.10.7-gentoo #1
Jan 18 09:09:09 hostname kernel: Hardware name: Gigabyte Technology Co., Ltd. Z390 AORUS ULTRA/Z390 AORUS ULTRA-CF, BIOS F7 03/14/2019
Jan 18 09:09:09 hostname kernel: Workqueue: events nvkm_notify_work [nouveau]
Jan 18 09:09:09 hostname kernel: RIP: 0010:gk104_fifo_recover_engn+0x257/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: Code: 8b 40 10 48 8b 78 10 4c 8b 7f 50 4d 85 ff 75 03 4c 8b 3f e8 4b 34 6a c4 4c 89 fa 48 c7 c7 56 ab 73 c0 48 89 c6 e8 88 9e 9f c4 <0f> 0b e9 4c ff ff ff e8 dd 79 a3 c4 66 66 2e 0f 1f 84 00 00 00 00
Jan 18 09:09:09 hostname kernel: RSP: 0018:ffffa6d246a97c68 EFLAGS: 00010086
Jan 18 09:09:09 hostname kernel: RAX: 0000000000000000 RBX: ffff944345e3f000 RCX: 0000000000000000
Jan 18 09:09:09 hostname kernel: RDX: 0000000000000002 RSI: ffffffff85967b2f RDI: 00000000ffffffff
Jan 18 09:09:09 hostname kernel: RBP: 0000000000002a30 R08: 0000000000000000 R09: ffffa6d246a97aa0
Jan 18 09:09:09 hostname kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 0000000000000000
Jan 18 09:09:09 hostname kernel: R13: ffff944342cccc00 R14: 0000000000000040 R15: ffff944341330770
Jan 18 09:09:09 hostname kernel: FS: 0000000000000000(0000) GS:ffff9452be480000(0000) knlGS:0000000000000000
Jan 18 09:09:09 hostname kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 18 09:09:09 hostname kernel: CR2: 000027584a14a000 CR3: 0000000cf2a0c003 CR4: 00000000003706e0
Jan 18 09:09:09 hostname kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jan 18 09:09:09 hostname kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Jan 18 09:09:09 hostname kernel: Call Trace:
Jan 18 09:09:09 hostname kernel: gk104_fifo_recover_chan+0x1bb/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: gk104_fifo_fault+0x118/0x2b0 [nouveau]
Jan 18 09:09:09 hostname kernel: gv100_fault_ntfy_nrpfb+0x217/0x260 [nouveau]
Jan 18 09:09:09 hostname kernel: nvkm_notify_work+0x18/0x70 [nouveau]
Jan 18 09:09:09 hostname kernel: process_one_work+0x1d1/0x390
Jan 18 09:09:09 hostname kernel: ? process_one_work+0x390/0x390
Jan 18 09:09:09 hostname kernel: worker_thread+0x45/0x3b0
Jan 18 09:09:09 hostname kernel: ? process_one_work+0x390/0x390
Jan 18 09:09:09 hostname kernel: kthread+0x120/0x160
Jan 18 09:09:09 hostname kernel: ? kthread_park+0x90/0x90
Jan 18 09:09:09 hostname kernel: ret_from_fork+0x1f/0x30
Jan 18 09:09:09 hostname kernel: ---[ end trace 6d7b711290def5b3 ]---
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: fifo: engine 0: scheduled for recovery
Jan 18 09:09:09 hostname kernel: ------------[ cut here ]------------
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: timeout
Jan 18 09:09:09 hostname kernel: WARNING: CPU: 2 PID: 96314 at drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.c:447 gk104_fifo_recover_engn+0x257/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: Modules linked in: tun fuse wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 libchacha poly1305_x86_64 ip6_udp_tunnel udp_tunnel libblake2s blake2s_x86_64 libblake2s_generic libcurve25519_gen>
Jan 18 09:09:09 hostname kernel: i2c_smbus intel_spi spi_nor snd_pcm mtd snd_timer snd mxm_wmi mei_me soundcore ttm rfkill mei intel_pch_thermal wmi sch_fq_codel nfsd crc32c_intel it87 hwmon_vid coretemp i2c_dev efivarfs
Jan 18 09:09:09 hostname kernel: CPU: 2 PID: 96314 Comm: kworker/2:2 Tainted: G W T 5.10.7-gentoo #1
Jan 18 09:09:09 hostname kernel: Hardware name: Gigabyte Technology Co., Ltd. Z390 AORUS ULTRA/Z390 AORUS ULTRA-CF, BIOS F7 03/14/2019
Jan 18 09:09:09 hostname kernel: Workqueue: events nvkm_notify_work [nouveau]
Jan 18 09:09:09 hostname kernel: RIP: 0010:gk104_fifo_recover_engn+0x257/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: Code: 8b 40 10 48 8b 78 10 4c 8b 7f 50 4d 85 ff 75 03 4c 8b 3f e8 4b 34 6a c4 4c 89 fa 48 c7 c7 56 ab 73 c0 48 89 c6 e8 88 9e 9f c4 <0f> 0b e9 4c ff ff ff e8 dd 79 a3 c4 66 66 2e 0f 1f 84 00 00 00 00
Jan 18 09:09:09 hostname kernel: RSP: 0018:ffffa6d246a97c68 EFLAGS: 00010086
Jan 18 09:09:09 hostname kernel: RAX: 0000000000000000 RBX: ffff944345e3f000 RCX: 0000000000000000
Jan 18 09:09:09 hostname kernel: RDX: 0000000000000002 RSI: ffffffff85967b2f RDI: 00000000ffffffff
Jan 18 09:09:09 hostname kernel: RBP: 0000000000002a5c R08: 0000000000000000 R09: ffffa6d246a97aa0
Jan 18 09:09:09 hostname kernel: R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000b
Jan 18 09:09:09 hostname kernel: R13: ffff944342cccc00 R14: 000000000000000f R15: ffff944341330770
Jan 18 09:09:09 hostname kernel: FS: 0000000000000000(0000) GS:ffff9452be480000(0000) knlGS:0000000000000000
Jan 18 09:09:09 hostname kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jan 18 09:09:09 hostname kernel: CR2: 000027584a14a000 CR3: 0000000cf2a0c003 CR4: 00000000003706e0
Jan 18 09:09:09 hostname kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jan 18 09:09:09 hostname kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Jan 18 09:09:09 hostname kernel: Call Trace:
Jan 18 09:09:09 hostname kernel: gk104_fifo_recover_chan+0x1bb/0x270 [nouveau]
Jan 18 09:09:09 hostname kernel: gk104_fifo_fault+0x118/0x2b0 [nouveau]
Jan 18 09:09:09 hostname kernel: gv100_fault_ntfy_nrpfb+0x217/0x260 [nouveau]
Jan 18 09:09:09 hostname kernel: nvkm_notify_work+0x18/0x70 [nouveau]
Jan 18 09:09:09 hostname kernel: process_one_work+0x1d1/0x390
Jan 18 09:09:09 hostname kernel: ? process_one_work+0x390/0x390
Jan 18 09:09:09 hostname kernel: worker_thread+0x45/0x3b0
Jan 18 09:09:09 hostname kernel: ? process_one_work+0x390/0x390
Jan 18 09:09:09 hostname kernel: kthread+0x120/0x160
Jan 18 09:09:09 hostname kernel: ? kthread_park+0x90/0x90
Jan 18 09:09:09 hostname kernel: ret_from_fork+0x1f/0x30
Jan 18 09:09:09 hostname kernel: ---[ end trace 6d7b711290def5b4 ]---
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: fifo: engine 11: scheduled for recovery
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: bus: MMIO write of 00000001 FAULT at 00259c [ TIMEOUT ]
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: systemd-logind[805]: channel 2 killed!
Jan 18 09:09:09 hostname kernel: usb 1-3.4.4: USB disconnect, device number 13
Jan 18 09:09:09 hostname rtkit-daemon[915]: Supervising 7 threads of 5 processes of 1 users.
Jan 18 09:09:09 hostname rtkit-daemon[915]: Successfully made thread 96482 of process 2173 owned by '1000' RT at priority 5.
Jan 18 09:09:09 hostname rtkit-daemon[915]: Supervising 8 threads of 5 processes of 1 users.
Jan 18 09:09:09 hostname kernel: nouveau 0000:01:00.0: bus: MMIO write of 00000001 FAULT at 00262c [ TIMEOUT ]This presumably let to these debug logs appearing in my sway/wlroots logs:
nouveau: kernel rejected pushbuf: No such device
nouveau: ch2: krec 0 pushes 1 bufs 8 relocs 0
nouveau: ch2: buf 00000000 00000002 00000004 00000004 00000000
nouveau: ch2: buf 00000001 0000001e 00000004 00000004 00000000
nouveau: ch2: buf 00000002 00000008 00000002 00000002 00000002
nouveau: ch2: buf 00000003 0000000a 00000002 00000002 00000002
nouveau: ch2: buf 00000004 00000006 00000004 00000000 00000004
nouveau: ch2: buf 00000005 00000019 00000002 00000000 00000002
nouveau: ch2: buf 00000006 00000007 00000002 00000002 00000000
nouveau: ch2: buf 00000007 00000016 00000004 00000004 00000000
nouveau: ch2: psh 00000000 0000010724 0000010b20
nouveau: 0x200203fd
nouveau: 0x0a000000
nouveau: 0x05a00000
nouveau: 0x20090200
nouveau: 0x00000000
nouveau: 0x09800000
nouveau: 0x00000a00
nouveau: 0x000005a0
nouveau: 0x000000cf
nouveau: 0x00000040
nouveau: 0x00000001
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x2001054e
nouveau: 0x00000000
nouveau: 0x20010487
nouveau: 0x0fac6881
nouveau: 0x80000574
nouveau: 0x800101d5
nouveau: 0x200308e0
nouveau: 0x00010000
nouveau: 0x00000000
nouveau: 0x00b20000
nouveau: 0xa04108e3
nouveau: 0x000001a0
nouveau: 0x80008000
nouveau: 0x00000002
nouveau: 0x01000000
nouveau: 0x00000000
nouveau: 0xffffffff
nouveau: 0xffffffff
nouveau: 0xdd620180
nouveau: 0x00007f9c
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00001000
nouveau: 0xffffffff
nouveau: 0x00000080
nouveau: 0x6edeed00
nouveau: 0x2484cbe1
nouveau: 0x80008000
nouveau: 0x00005638
nouveau: 0x6edeed00
nouveau: 0x2484cbe1
nouveau: 0x51bcc060
nouveau: 0x00005638
nouveau: 0x00000003
nouveau: 0x00000000
nouveau: 0x80008000
nouveau: 0x00005638
nouveau: 0x51bcc360
nouveau: 0x00005638
nouveau: 0x930d4d00
nouveau: 0x00007ffe
nouveau: 0x53711538
nouveau: 0x00005638
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0xdd574c87
nouveau: 0x00007f9c
nouveau: 0x00000018
nouveau: 0x00000030
nouveau: 0x930d4cb0
nouveau: 0x00007ffe
nouveau: 0x80008000
nouveau: 0x00007ffe
nouveau: 0x6edeed00
nouveau: 0x2484cbe1
nouveau: 0x00008cd5
nouveau: 0x00000000
nouveau: 0x00000014
nouveau: 0x00000000
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0xdd55d826
nouveau: 0x00007f9c
nouveau: 0x930d4c50
nouveau: 0x00007ffe
nouveau: 0xffffffff
nouveau: 0x00000000
nouveau: 0x80008000
nouveau: 0x00007f9c
nouveau: 0x00003270
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x6edeed00
nouveau: 0x2484cbe1
nouveau: 0x20040478
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x20040360
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x20010674
nouveau: 0x0000003c
nouveau: 0x200308e0
nouveau: 0x00010000
nouveau: 0x00000000
nouveau: 0x00a80000
nouveau: 0xa01108e3
nouveau: 0x00000000
nouveau: 0x40000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x40000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0xbf800000
nouveau: 0xbf800000
nouveau: 0x3f800000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x200308e0
nouveau: 0x00010000
nouveau: 0x00000000
nouveau: 0x00ac0000
nouveau: 0xa00508e3
nouveau: 0x00000000
nouveau: 0x3f800000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x20024062
nouveau: 0x00000000
nouveau: 0x00b571c0
nouveau: 0x20024060
nouveau: 0x00000020
nouveau: 0x00000001
nouveau: 0xa009406c
nouveau: 0x00001001
nouveau: 0x54e24908
nouveau: 0x06000000
nouveau: 0x00400000
nouveau: 0x00010140
nouveau: 0xeb8009ff
nouveau: 0x8000059f
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x200104cd
nouveau: 0x00000000
nouveau: 0x200308e0
nouveau: 0x00010000
nouveau: 0x00000000
nouveau: 0x00b20000
nouveau: 0x200208e3
nouveau: 0x00000020
nouveau: 0x0000038e
nouveau: 0x20010700
nouveau: 0x00001008
nouveau: 0x20010704
nouveau: 0x00001008
nouveau: 0xa0050e02
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x0962e4ff
nouveau: 0x00000000
nouveau: 0x0962e4e0
nouveau: 0xa0050e02
nouveau: 0x00000001
nouveau: 0x00000000
nouveau: 0x0962e51f
nouveau: 0x00000000
nouveau: 0x0962e500
nouveau: 0x200308e0
nouveau: 0x00010000
nouveau: 0x00000000
nouveau: 0x00b20000
nouveau: 0xa04108e3
nouveau: 0x000001a0
nouveau: 0x80008000
nouveau: 0x00005638
nouveau: 0xd97f0c4e
nouveau: 0x00007f9c
nouveau: 0x00002800
nouveau: 0x00005638
nouveau: 0xd97f4bb2
nouveau: 0x00007f9c
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x536a0ad0
nouveau: 0x00005638
nouveau: 0x00000020
nouveau: 0x00000000
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x536a1ac0
nouveau: 0x00005638
nouveau: 0x536d8410
nouveau: 0x00005638
nouveau: 0xdbc441c7
nouveau: 0x00007f9c
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x536d6320
nouveau: 0x00005638
nouveau: 0x536d6338
nouveau: 0x00005638
nouveau: 0x00000000
nouveau: 0x00000000
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x536a0ad0
nouveau: 0x00005638
nouveau: 0x5369c4d0
nouveau: 0x00005638
nouveau: 0xdbc442f1
nouveau: 0x00007f9c
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x00000006
nouveau: 0x00000000
nouveau: 0x00000004
nouveau: 0x00000000
nouveau: 0x536a0ad0
nouveau: 0x00005638
nouveau: 0x80008000
nouveau: 0x00000000
nouveau: 0x00000020
nouveau: 0x00000000
nouveau: 0x000071c0
nouveau: 0x00000000
nouveau: 0x53dfba04
nouveau: 0x00005638
nouveau: 0x80008000
nouveau: 0x00005638
nouveau: 0xbb91e7d8
nouveau: 0x00007f9c
nouveau: 0x536b6470
nouveau: 0x00005638
nouveau: 0x536b6680
nouveau: 0x00005638
nouveau: 0x20040478
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x88888888
nouveau: 0x20010586
nouveau: 0x00000005
nouveau: 0x2002035d
nouveau: 0x00000000
nouveau: 0x00000004
nouveau: 0x80000585