GPU hung on i.MX6QP using v6.1 kernel, mesa 22.3.6, Qt5
Hi,
System information
- OS: Yocto sumo based (very old!)
- GPU: Vivante GC3000 inside of i.MX6QP
- Kernel version: v6.1 + some extra patches adding HW support and DCIC
- Mesa version: 22.3.6
- Desktop manager and compositor: based on https://wiki.qt.io/QtWayland (Qt5)
issue description
2023-03-02T09:43:20+0100 kernel: etnaviv-gpu 130000.gpu: recover hung GPU!
2023-03-02T09:43:21+0100 kernel: etnaviv-gpu 130000.gpu: recover hung GPU!
The exact steps to reproduce the problem is unknown. Initially (older kernel and mesa) it was possible to reproduce hungs by running glmark2-es2-drm
for a few hours/days, but that no longer works with mesa 22.3.6 (system can run glmark2-es2-drm
for weeks without any hung now). Qt on the other hand still produces hungs sporadically. When a hung appears, there is a high chance, that a second one happens shortly after (but not always). A similar machine running the same software, but being based on i.MX6Q (without P) seems to be unaffected. A colleague of mine pointed me to a patch from @lynxeye that adds brown-out detection support for the GPU regulator and no brownouts are detected for the hardware.
Logs
=== Register dump
0000000c = 00000099
00000000 = 00040900
00000004 = 7ffffffa Idle: FE- DE+ PE- SH+ PA+ SE+ RA+ TX+ VG+ IM+ FP+ TS+
00000008 = 00002200
00000014 = ffffffff
00000018 = 14010000
0000001c = e0287cad
00000020 = 00002000
00000024 = ffff5450
00000028 = 20140617
0000002c = 20575900
00000034 = c9799efb
00000038 = 00000000
00000070 = 00000000
00000100 = 00140021
00000104 = 00030000
00000108 = 000010f8
0000010c = 015d0880
00000400 = 00000000
00000404 = 00000000
00000408 = 00000000
0000040c = 00000000
00000410 = 00000000
00000414 = 3c000000
00000418 = 00000000
0000041c = 00000000
00000420 = 00000000
00000424 = 00000000
00000428 = 00000000
0000042c = 00030000
00000480 = 00000000
0000065c = 00000001
00000660 = 00000015 Cmd: [stall DMA: idle Fetch: idle] Req idle Cal idle
00000664 = 00001200 Command DMA address
00000668 = 48000000 FE fetched word 0
0000066c = 00000701 FE fetched word 1
00000670 = 00000000
=== Buffers
Num Name IOVA Size
0 reg 00000000 00000128 296
1 mmu 00000000 00005000 20480
* 2 ring 00001000 00001000 4096
3 cmd 0000a000 000005c8 1480
4 bomap 00000000 00000320 800
5 bo fd8c1000 00001000 4096
6 bo fe212000 00030000 196608
7 bo ff1d3000 00001000 4096
8 bo fd696000 0000a000 40960
9 bo fe5c6000 00028000 163840
Checking MMU entries... failed
Buf 5 Offset 00000000: 000002f6 70942000
Buf 6 Offset 00000000: 00000000 56867000
Buf 6 Offset 00001000: 00000000 56868000
Buf 6 Offset 00002000: 00000000 56869000
Buf 6 Offset 00003000: 00000000 5686a000
Buf 6 Offset 00004000: 00000000 5686b000
Buf 6 Offset 00005000: 00000001 5686c000
Buf 6 Offset 00006000: 00000001 5686d000
Buf 6 Offset 00007000: 00000000 5686e000
Buf 6 Offset 00008000: 00000000 5686f000
Buf 6 Offset 00009000: 00000000 56870000
Buf 6 Offset 0000a000: 00000000 56871000
Buf 6 Offset 0000b000: 00000000 56872000
Buf 6 Offset 0000c000: 00000000 56873000
Buf 6 Offset 0000d000: 00000000 56874000
Buf 6 Offset 0000e000: 00000000 56875000
Buf 6 Offset 0000f000: 00000000 56876000
Buf 6 Offset 00010000: 00000000 56877000
Buf 6 Offset 00011000: 00000000 56878000
Buf 6 Offset 00012000: 00000000 56879000
Buf 6 Offset 00013000: 00000000 5687a000
Buf 6 Offset 00014000: 03020100 5687b000
Buf 6 Offset 00015000: 07060504 5687c000
Buf 6 Offset 00016000: 0b0a0908 5687d000
Buf 6 Offset 00017000: 40404040 5687e000
Buf 6 Offset 00018000: 0c404040 5687f000
Buf 6 Offset 00019000: 100f0e0d 56880000
Buf 6 Offset 0001a000: 14131211 56881000
Buf 6 Offset 0001b000: 18171615 56882000
Buf 6 Offset 0001c000: 1c1b1a19 56883000
Buf 6 Offset 0001d000: 201f1e1d 56884000
Buf 6 Offset 0001e000: 24232221 56885000
Buf 6 Offset 0001f000: 40404025 56886000
Buf 6 Offset 00020000: 26404040 568c6000
Buf 6 Offset 00021000: 2a292827 568c7000
Buf 6 Offset 00022000: 2e2d2c2b 568c8000
Buf 6 Offset 00023000: 3231302f 568c9000
Buf 6 Offset 00024000: 36353433 568ca000
Buf 6 Offset 00025000: 3a393837 568cb000
Buf 6 Offset 00026000: 3e3d3c3b 568cc000
Buf 6 Offset 00027000: 0000003f 568cd000
Buf 6 Offset 00028000: 00000000 568ce000
Buf 6 Offset 00029000: 00000000 568cf000
Buf 6 Offset 0002a000: 00000000 568d0000
Buf 6 Offset 0002b000: 00000000 568d1000
Buf 6 Offset 0002c000: 00000005 568d2000
Buf 6 Offset 0002d000: 6562616c 568d3000
Buf 6 Offset 0002e000: 0000006c 568d4000
Buf 6 Offset 0002f000: 00000000 568d5000
Buf 7 Offset 00000000: ffec98bc 568d6000
Buf 8 Offset 00000000: 0001d67f 70bb5000
Buf 8 Offset 00001000: 00000000 70bb8000
Buf 8 Offset 00002000: 0001d680 708e3000
Buf 8 Offset 00003000: 00000000 708df000
Buf 8 Offset 00004000: 0001d681 708e4000
Buf 8 Offset 00005000: 00000000 70bbc000
Buf 8 Offset 00006000: 0001d682 70bbd000
Buf 8 Offset 00007000: 00000000 708e0000
Buf 8 Offset 00008000: 0001d683 708e6000
Buf 8 Offset 00009000: 00000000 70bbe000
Buf 9 Offset 00000000: 00000000 568d7000
Buf 9 Offset 00001000: 00000000 568d8000
Buf 9 Offset 00002000: 00000000 568d9000
Buf 9 Offset 00003000: 00000000 568da000
Buf 9 Offset 00004000: 00000000 568db000
Buf 9 Offset 00005000: 00000000 568dc000
Buf 9 Offset 00006000: 00000000 568dd000
Buf 9 Offset 00007000: 00000000 568de000
Buf 9 Offset 00008000: 00000000 568df000
Buf 9 Offset 00009000: 00000000 568e0000
Buf 9 Offset 0000a000: 00000000 568e1000
Buf 9 Offset 0000b000: 00000000 568e2000
Buf 9 Offset 0000c000: 00000000 568e3000
Buf 9 Offset 0000d000: 00000000 568e4000
Buf 9 Offset 0000e000: 00000000 568e5000
Buf 9 Offset 0000f000: 00000000 568e6000
Buf 9 Offset 00010000: 00000000 568e7000
Buf 9 Offset 00011000: 00000000 568e8000
Buf 9 Offset 00012000: 00000000 568e9000
Buf 9 Offset 00013000: 00000000 568ea000
Buf 9 Offset 00014000: 00000000 568eb000
Buf 9 Offset 00015000: 00000000 568ec000
Buf 9 Offset 00016000: 00000000 568ed000
Buf 9 Offset 00017000: 00000000 568ee000
Buf 9 Offset 00018000: 00000000 568ef000
Buf 9 Offset 00019000: 00000000 568f0000
Buf 9 Offset 0001a000: 00000000 568f1000
Buf 9 Offset 0001b000: 00000000 568f2000
Buf 9 Offset 0001c000: 00000000 568f3000
Buf 9 Offset 0001d000: 00000000 568f4000
Buf 9 Offset 0001e000: 00000000 568f5000
Buf 9 Offset 0001f000: 00000000 568f6000
Buf 9 Offset 00020000: 00000000 568f7000
Buf 9 Offset 00021000: 00000000 568f8000
Buf 9 Offset 00022000: 00000000 568f9000
Buf 9 Offset 00023000: 00000000 568fa000
Buf 9 Offset 00024000: 00000000 568fb000
Buf 9 Offset 00025000: 00000000 568fc000
Buf 9 Offset 00026000: 00000000 568fd000
Buf 9 Offset 00027000: 00000000 568fe000