BUG: kernel NULL pointer dereference in AMD driver
Kernel BUG appears in every dmesg output during startup.
PC freezes after using X, the sooner the heavier the load. I can freeze the PC by starting a web browser or running a phoronix benchmark. During the freeze I can often see what reminds a corrupted video memory on the display - uniform colored, somtimes blinking artifacts. I can safely reboot using SysRq so it's not completely unresponsive, only to the mouse input it seems. Relation between this kernel log and freeze is hypothetical and not proven. I also captured the following output shortly before crash (see last comment in bug #1965 (closed))
I'm experiencing this bug since 5.19.0-rc1 in every rc release so far - including the freeze. All 5.18.x versions run fine long-term (currently in use as a backup).
OS: Gentoo Base System release 2.8 x86_64 Host: MS-7B93 1.0 Kernel: 5.19.0-rc5-llvm Resolution: 3840x2160 (1x DP connected panel) DE: GNOME 42.3.1 WM: Mutter CPU: AMD Ryzen 7 3800X (16) @ 3.900GHz GPU: AMD ATI Radeon RX 5600 OEM/5600 XT / 5700/5700 XT Memory: 6971MiB / 64244MiB
lspci VGA: 2f:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev ca)
uname -a: Linux gentoo 5.19.0-rc5-tkg-llvm #1 (closed) TKG SMP PREEMPT Sun Jul 3 14:29:45 CEST 2022 x86_64 AMD Ryzen 7 3800X 8-Core Processor AuthenticAMD GNU/Linux
Custom kernel build, .config on request.
dmesg output shortly after start.
Jul 06 11:55:01 gentoo kernel: BUG: kernel NULL pointer dereference, address: 0000000000000008
Jul 06 11:55:01 gentoo kernel: #PF: supervisor read access in kernel mode
Jul 06 11:55:01 gentoo kernel: #PF: error_code(0x0000) - not-present page
Jul 06 11:55:01 gentoo kernel: PGD 0 P4D 0
Jul 06 11:55:01 gentoo kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI
Jul 06 11:55:01 gentoo kernel: CPU: 6 PID: 135 Comm: kworker/6:1 Tainted: G W 5.19.0-rc5-tkg-tt-llvm #1
Jul 06 11:55:01 gentoo kernel: Hardware name: Micro-Star International Co., Ltd. MS-7B93/MPG X570 GAMING PRO CARBON WIFI (MS-7B93), BIOS 1.G0 05/27/2022
Jul 06 11:55:01 gentoo kernel: Workqueue: events delayed_fput
Jul 06 11:55:01 gentoo kernel: RIP: 0010:dma_resv_add_fence+0x3b/0x1e0
Jul 06 11:55:01 gentoo kernel: Code: 49 89 f4 49 89 fe 48 85 f6 74 1f b8 01 00 00 00 f0 41 0f c1 44 24 38 85 c0 0f 84 14 01 00 00 8d 48 01 09 c1 0f 88 39 01 00 00 <49> 8b 44 24 08 48 c7 c1 f0 51 3e 82 48 39 c8 0f 84 ed 00 00 00 48
Jul 06 11:55:01 gentoo kernel: RSP: 0018:ffff888104dafc38 EFLAGS: 00010246
Jul 06 11:55:01 gentoo kernel: RAX: 0000000000000000 RBX: ffff88811ac850f8 RCX: 0000000000000000
Jul 06 11:55:01 gentoo kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff8881688c0158
Jul 06 11:55:01 gentoo kernel: RBP: ffff88811ac85918 R08: 0000000080200013 R09: 0000000080200012
Jul 06 11:55:01 gentoo kernel: R10: ffffea0004e7cdc0 R11: 0000000000000000 R12: 0000000000000000
Jul 06 11:55:01 gentoo kernel: R13: ffff8881688c0000 R14: ffff8881688c0158 R15: 0000000000000001
Jul 06 11:55:01 gentoo kernel: FS: 0000000000000000(0000) GS:ffff888feed80000(0000) knlGS:0000000000000000
Jul 06 11:55:01 gentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 06 11:55:01 gentoo kernel: CR2: 0000000000000008 CR3: 000000000500a000 CR4: 0000000000350ee0
Jul 06 11:55:01 gentoo kernel: Call Trace:
Jul 06 11:55:01 gentoo kernel: <TASK>
Jul 06 11:55:01 gentoo kernel: amdgpu_amdkfd_gpuvm_destroy_cb+0x8b/0x200 [amdgpu]
Jul 06 11:55:01 gentoo kernel: amdgpu_vm_fini+0x2d/0x740 [amdgpu]
Jul 06 11:55:01 gentoo kernel: ? idr_get_next+0x83/0x130
Jul 06 11:55:01 gentoo kernel: amdgpu_driver_postclose_kms+0x29d/0x440 [amdgpu]
Jul 06 11:55:01 gentoo kernel: drm_file_free+0x1f6/0x2a0
Jul 06 11:55:01 gentoo kernel: drm_release+0xf9/0x200
Jul 06 11:55:01 gentoo kernel: __fput+0xde/0x2f0
Jul 06 11:55:01 gentoo kernel: delayed_fput+0x28/0x40
Jul 06 11:55:01 gentoo kernel: process_one_work+0x1d9/0x300
Jul 06 11:55:01 gentoo kernel: worker_thread+0x33d/0x6f0
Jul 06 11:55:01 gentoo kernel: kthread+0x1f8/0x240
Jul 06 11:55:01 gentoo kernel: ? rcu_free_pool+0x30/0x30
Jul 06 11:55:01 gentoo kernel: ? kthreadd+0x2f0/0x2f0
Jul 06 11:55:01 gentoo kernel: ret_from_fork+0x1f/0x30
Jul 06 11:55:01 gentoo kernel: </TASK>
Jul 06 11:55:01 gentoo kernel: Modules linked in: snd_seq_dummy snd_seq mousedev hid_jabra joydev vfat fat amdgpu iommu_v2 drm_buddy drm_display_helper edac_mce_amd gpu_sched drm_ttm_helper edac_core ttm drm_kms_helper sysimgblt snd_hda_codec_generic syscopyarea kvm_amd sysfillrect ccp fb_sys_fops rng_core backlight snd_hda_intel igb cfbimgblt snd_intel_dspcfg cfbcopyarea dca snd_hda_codec cfbfillrect snd_usb_audio kvm fb snd_hda_core snd_usbmidi_lib snd_hwdep irqbypass snd_rawmidi snd_seq_device font rapl snd_pcm fbdev btusb i2c_piix4 btrtl snd_timer wmi_bmof btmtk k10temp btbcm btintel snd bluetooth soundcore button rfkill mc acpi_cpufreq usbhid squashfs loop sch_fq_codel fuse configfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel nvme xhci_pci xhci_pci_renesas nvme_core winesync nct6775 hwmon_vid wmi nct6775_core hwmon ipv6
Jul 06 11:55:01 gentoo kernel: CR2: 0000000000000008
Jul 06 11:55:01 gentoo kernel: ---[ end trace 0000000000000000 ]---
Jul 06 11:55:01 gentoo kernel: RIP: 0010:dma_resv_add_fence+0x3b/0x1e0
Jul 06 11:55:01 gentoo kernel: Code: 49 89 f4 49 89 fe 48 85 f6 74 1f b8 01 00 00 00 f0 41 0f c1 44 24 38 85 c0 0f 84 14 01 00 00 8d 48 01 09 c1 0f 88 39 01 00 00 <49> 8b 44 24 08 48 c7 c1 f0 51 3e 82 48 39 c8 0f 84 ed 00 00 00 48
Jul 06 11:55:01 gentoo kernel: RSP: 0018:ffff888104dafc38 EFLAGS: 00010246
Jul 06 11:55:01 gentoo kernel: RAX: 0000000000000000 RBX: ffff88811ac850f8 RCX: 0000000000000000
Jul 06 11:55:01 gentoo kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff8881688c0158
Jul 06 11:55:01 gentoo kernel: RBP: ffff88811ac85918 R08: 0000000080200013 R09: 0000000080200012
Jul 06 11:55:01 gentoo kernel: R10: ffffea0004e7cdc0 R11: 0000000000000000 R12: 0000000000000000
Jul 06 11:55:01 gentoo kernel: R13: ffff8881688c0000 R14: ffff8881688c0158 R15: 0000000000000001
Jul 06 11:55:01 gentoo kernel: FS: 0000000000000000(0000) GS:ffff888feed80000(0000) knlGS:0000000000000000
Jul 06 11:55:01 gentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 06 11:55:01 gentoo kernel: CR2: 0000000000000008 CR3: 000000000500a000 CR4: 0000000000350ee0Activity
Probably a duplicate of #2050 (closed)
I can confirm the error is gone with latest buddy allocator revert patches from #2050 (closed)
marked this issue as a duplicate of #2050 (closed)
marked this issue as related to #2050 (closed)
I'd like to reopen this issue as I've encountered it again.
My kernel is 5.19.0-rc5 with buddy allocator revert patches from #2050 (closed)
Yet still after some time running the kernel I get:
Jul 09 14:45:34 gentoo kernel: main[4462]: segfault at 38 ip 00000000008d97bb sp 00007f280874fba0 error 4 in mailsync.bin[400000+2972000] Jul 09 14:45:34 gentoo kernel: Code: 48 89 c7 e8 61 4c 00 00 c9 c3 55 48 89 e5 48 83 ec 30 48 89 7d f8 48 89 75 f0 48 89 55 e8 48 89 4d e0 4c 89 45 d8 48 8b 45 d8 <48> 8b 40 38 48 3b 45 e0 75 3f 48 8b 45 d8 48 89 c7 e8 13 bb fd ff Jul 09 14:45:34 gentoo systemd[1]: Started Process Core Dump (PID 27152/UID 0). Jul 09 14:45:34 gentoo systemd-coredump[27153]: elfutils disabled, parsing ELF objects not supported Jul 09 14:45:34 gentoo systemd-coredump[27153]: Process 4432 (main) of user 1000 dumped core. Jul 09 14:45:34 gentoo systemd[1]: systemd-coredump@1-27152-0.service: Deactivated successfully. Jul 09 14:47:28 gentoo dbus-daemon[1277]: [session uid=1000 pid=1277] Activating service name='org.gnome.GConf' requested by ':1.238' (uid=1000 pid=27704 comm="/usr/bin/gconf tool-2 -g /apps/gnome-screensaver/id") Jul 09 14:47:28 gentoo dbus-daemon[1277]: [session uid=1000 pid=1277] Successfully activated service 'org.gnome.GConf' Jul 09 14:47:56 gentoo kernel: BUG: kernel NULL pointer dereference, address: 0000000000000008 Jul 09 14:47:56 gentoo kernel: #PF: supervisor read access in kernel mode Jul 09 14:47:56 gentoo kernel: #PF: error_code(0x0000) - not-present page Jul 09 14:47:56 gentoo kernel: PGD 0 P4D 0 Jul 09 14:47:56 gentoo kernel: Oops: 0000 [#1] PREEMPT SMP NOPTI Jul 09 14:47:56 gentoo kernel: CPU: 0 PID: 7655 Comm: kworker/0:0 Tainted: G S 5.19.0-rc5-tkg-tt-llvm #1 Jul 09 14:47:56 gentoo kernel: Hardware name: Micro-Star International Co., Ltd. MS-7B93/MPG X570 GAMING PRO CARBON WIFI (MS-7B93), BIOS 1.G0 05/27/2022 Jul 09 14:47:56 gentoo kernel: Workqueue: events delayed_fput Jul 09 14:47:56 gentoo kernel: RIP: 0010:dma_resv_add_fence+0x3b/0x1e0 Jul 09 14:47:56 gentoo kernel: Code: 49 89 f4 49 89 fe 48 85 f6 74 1f b8 01 00 00 00 f0 41 0f c1 44 24 38 85 c0 0f 84 14 01 00 00 8d 48 01 09 c1 0f 88 39 01 00 00 <49> 8b 44 24 08 48 c7 c1 70 4f 3e 82 48 39 c8 0f 84 ed 00 00 00 48 Jul 09 14:47:56 gentoo kernel: RSP: 0018:ffff888131b4fc38 EFLAGS: 00010246 Jul 09 14:47:56 gentoo kernel: RAX: 0000000000000000 RBX: ffff88811bc850f8 RCX: 0000000000000000 Jul 09 14:47:56 gentoo kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff888309f90158 Jul 09 14:47:56 gentoo kernel: RBP: ffff88811bc85918 R08: 0000000080200012 R09: 0000000080200011 Jul 09 14:47:56 gentoo kernel: R10: ffffea0015170d80 R11: 0000000000000000 R12: 0000000000000000 Jul 09 14:47:56 gentoo kernel: R13: ffff888309f90000 R14: ffff888309f90158 R15: 0000000000000001 Jul 09 14:47:56 gentoo kernel: FS: 0000000000000000(0000) GS:ffff888feec00000(0000) knlGS:0000000000000000 Jul 09 14:47:56 gentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 09 14:47:56 gentoo kernel: CR2: 0000000000000008 CR3: 000000000500a000 CR4: 0000000000350ef0 Jul 09 14:47:56 gentoo kernel: Call Trace: Jul 09 14:47:56 gentoo kernel: <TASK> Jul 09 14:47:56 gentoo kernel: amdgpu_amdkfd_gpuvm_destroy_cb+0x8b/0x200 [amdgpu] Jul 09 14:47:56 gentoo kernel: amdgpu_vm_fini+0x2d/0x740 [amdgpu] Jul 09 14:47:56 gentoo kernel: ? idr_get_next+0x83/0x130 Jul 09 14:47:56 gentoo kernel: amdgpu_driver_postclose_kms+0x29d/0x440 [amdgpu] Jul 09 14:47:56 gentoo kernel: drm_file_free+0x1f6/0x2a0 Jul 09 14:47:56 gentoo kernel: drm_release+0xf9/0x200 Jul 09 14:47:56 gentoo kernel: __fput+0xde/0x2f0 Jul 09 14:47:56 gentoo kernel: delayed_fput+0x28/0x40 Jul 09 14:47:56 gentoo kernel: process_one_work+0x1d9/0x300 Jul 09 14:47:56 gentoo kernel: worker_thread+0x33d/0x6f0 Jul 09 14:47:56 gentoo kernel: kthread+0x1f8/0x240 Jul 09 14:47:56 gentoo kernel: ? rcu_free_pool+0x30/0x30 Jul 09 14:47:56 gentoo kernel: ? kthreadd+0x2f0/0x2f0 Jul 09 14:47:56 gentoo kernel: ret_from_fork+0x1f/0x30 Jul 09 14:47:56 gentoo kernel: </TASK> Jul 09 14:47:56 gentoo kernel: Modules linked in: msr snd_seq_dummy snd_seq mousedev hid_jabra joydev vfat fat amdgpu iommu_v2 edac_mce_amd drm_display_helper edac_core gpu_s ched drm_ttm_helper ttm drm_kms_helper kvm_amd sysimgblt syscopyarea sysfillrect fb_sys_fops ccp backlight cfbimgblt rng_core cfbcopyarea cfbfillrect fb igb kvm snd_hda_codec_generic snd_hda_intel font dca snd_intel_dspcfg fbdev irqbypass snd_usb_audio snd_hda_codec snd_usbmidi_lib rapl snd_hda_core snd_hwdep wmi_bmof snd_rawmidi btusb btrtl snd_seq_device k10temp i2c_piix4 btmtk snd_pcm btbcm btintel snd_timer bluetooth snd button soundcore acpi_cpufreq rfkill mc usbhid squashfs loop sch_fq_codel fuse configfs crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel nvme xhci_pci xhci_pci_renesas nvme_core winesync nct6775 hwmon_vid wmi nct6775_core hwmon ipv6 Jul 09 14:47:56 gentoo kernel: CR2: 0000000000000008 Jul 09 14:47:56 gentoo kernel: ---[ end trace 0000000000000000 ]--- Jul 09 14:47:56 gentoo kernel: RIP: 0010:dma_resv_add_fence+0x3b/0x1e0 Jul 09 14:47:56 gentoo kernel: Code: 49 89 f4 49 89 fe 48 85 f6 74 1f b8 01 00 00 00 f0 41 0f c1 44 24 38 85 c0 0f 84 14 01 00 00 8d 48 01 09 c1 0f 88 39 01 00 00 <49> 8b 44 24 08 48 c7 c1 70 4f 3e 82 48 39 c8 0f 84 ed 00 00 00 48 Jul 09 14:47:56 gentoo kernel: RSP: 0018:ffff888131b4fc38 EFLAGS: 00010246 Jul 09 14:47:56 gentoo kernel: RAX: 0000000000000000 RBX: ffff88811bc850f8 RCX: 0000000000000000 Jul 09 14:47:56 gentoo kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff888309f90158 Jul 09 14:47:56 gentoo kernel: RBP: ffff88811bc85918 R08: 0000000080200012 R09: 0000000080200011 Jul 09 14:47:56 gentoo kernel: R10: ffffea0015170d80 R11: 0000000000000000 R12: 0000000000000000 Jul 09 14:47:56 gentoo kernel: R13: ffff888309f90000 R14: ffff888309f90158 R15: 0000000000000001 Jul 09 14:47:56 gentoo kernel: FS: 0000000000000000(0000) GS:ffff888feec00000(0000) knlGS:0000000000000000 Jul 09 14:47:56 gentoo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 09 14:47:56 gentoo kernel: CR2: 0000000000000008 CR3: 000000000500a000 CR4: 0000000000350ef0mentioned in issue #2050 (closed)
I posted this on bugzilla.kernel.org but was redireted here. Kernel version is 5.19-rc5-next-20220708 without any revert patches (it seems the patch from #2050 (closed) is already applied). A very simple hsa test program triggers a NULL pointer dereference when opening /dev/kfd. Program:
#include <stdlib.h> #include <stdio.h> #include <hsakmt.h> #include <hsakmttypes.h> int main() { HSAKMT_STATUS ret; ret = hsaKmtOpenKFD(); if (ret != HSAKMT_STATUS_SUCCESS) { printf("hsaKmtOpenKDF failed with status %d\n", ret); } ret = hsaKmtCloseKFD(); if (ret != HSAKMT_STATUS_SUCCESS) { printf("hsaKmtClosesKDF failed with status %d\n", ret); } return 0; }Version of libhsakmt: 5.2.0+dfsg-1 (from the debian sid repository) The test program does not trigger any error with linux-5.18.5.
Error message:
Jul 9 10:39:32 lisa kernel: [ 35.814297] ------------[ cut here ]------------ Jul 9 10:39:32 lisa kernel: [ 35.814298] WARNING: CPU: 8 PID: 126 at drivers/gpu/drm/ttm/ttm_bo.c:704 ttm_bo_unpin+0x5a/0x70 [ttm] Jul 9 10:39:32 lisa kernel: [ 35.814307] Modules linked in: ccm(EN) rfcomm(EN) bnep(EN) cpufreq_conservative(EN) cpufreq_ondemand(EN) cpufreq_powersave(EN) cpufreq_userspace(EN) snd_ctl_led(EN) btusb(EN) btrtl(EN) snd_hda_codec_realtek(EN) btbcm(EN) btintel(EN) btmtk(EN) snd_hda_codec_generic(EN) ledtrig_audio(EN) bluetooth(EN) snd_hda_codec_hdmi(EN) snd_hda_intel(EN) snd_soc_dmic(EN) snd_intel_dspcfg(EN) snd_acp3x_rn(EN) snd_acp3x_pdm_dma(EN) snd_hda_codec(EN) jitterentropy_rng(EN) snd_soc_core(EN) uvcvideo(EN) snd_hwdep(EN) snd_hda_core(EN) videobuf2_vmalloc(EN) snd_pcm_oss(EN) videobuf2_memops(EN) sha512_generic(EN) videobuf2_v4l2(EN) nls_ascii(EN) snd_mixer_oss(EN) nls_cp437(EN) videodev(EN) ccp(EN) snd_rn_pci_acp3x(EN) snd_pcm(EN) joydev(EN) snd_acp_config(EN) msi_wmi(EN) vfat(EN) snd_timer(EN) ctr(EN) drbg(EN) snd(EN) ecdh_generic(EN) evdev(EN) sparse_keymap(EN) wmi_bmof(EN) fat(EN) ecc(EN) videobuf2_common(EN) serio_raw(EN) hid_multitouch(EN) soundcore(EN) snd_soc_acpi(EN) efi_pstore(EN) rng_core(EN) Jul 9 10:39:32 lisa kernel: [ 35.814329] k10temp(EN) battery(EN) wmi(EN) ac(EN) button(EN) video(EN) hid_sensor_accel_3d(EN) hid_sensor_magn_3d(EN) hid_sensor_prox(EN) hid_sensor_als(EN) hid_sensor_gyro_3d(EN) hid_sensor_trigger(EN) industrialio_triggered_buffer(EN) kfifo_buf(EN) industrialio(EN) hid_sensor_iio_common(EN) amd_pmc(EN) acpi_cpufreq(EN) mt7921e(EN) mt7921_common(EN) mt76_connac_lib(EN) mt76(EN) mac80211(EN) libarc4(EN) cfg80211(EN) rfkill(EN) ipmi_devintf(EN) ipmi_msghandler(EN) msr(EN) fuse(EN) configfs(EN) efivarfs(EN) autofs4(EN) ext4(EN) crc32c_generic(EN) crc32c_intel(EN) crc16(EN) mbcache(EN) jbd2(EN) usbhid(EN) amdgpu(EN) drm_ttm_helper(EN) ttm(EN) gpu_sched(EN) i2c_algo_bit(EN) drm_buddy(EN) drm_display_helper(EN) nvme(EN) xhci_pci(EN) drm_kms_helper(EN) xhci_hcd(EN) nvme_core(EN) r8169(EN) syscopyarea(EN) sysfillrect(EN) hid_sensor_hub(EN) sysimgblt(EN) t10_pi(EN) mfd_core(EN) fb_sys_fops(EN) hid_generic(EN) drm(EN) usbcore(EN) realtek(EN) i2c_hid_acpi(EN) mdio_devres(EN) psmouse(EN) Jul 9 10:39:32 lisa kernel: [ 35.814352] i2c_hid(EN) amd_sfh(EN) libphy(EN) crc64_rocksoft(EN) hid(EN) backlight(EN) crc64(EN) crc_t10dif(EN) i2c_piix4(EN) cec(EN) usb_common(EN) crct10dif_generic(EN) crct10dif_common(EN) i2c_designware_platform(EN) i2c_designware_core(EN) Jul 9 10:39:32 lisa kernel: [ 35.814359] CPU: 8 PID: 126 Comm: kworker/8:1 Tainted: G W E N 5.19.0-rc5-next-20220708 #89 Jul 9 10:39:32 lisa kernel: [ 35.814361] Hardware name: Micro-Star International Co., Ltd. Alpha 15 B5EEK/MS-158L, BIOS E158LAMS.107 11/10/2021 Jul 9 10:39:32 lisa kernel: [ 35.814362] Workqueue: kfd_process_wq kfd_process_wq_release [amdgpu] Jul 9 10:39:32 lisa kernel: [ 35.814479] RIP: 0010:ttm_bo_unpin+0x5a/0x70 [ttm] Jul 9 10:39:32 lisa kernel: [ 35.814485] Code: 00 00 83 ab 8c 01 00 00 01 48 85 ff 74 08 48 89 de e8 5a 44 00 00 48 8b bb 38 01 00 00 5b 48 81 c7 20 08 00 00 e9 f6 a6 18 c5 <0f> 0b 5b c3 0f 0b eb ac 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 00 Jul 9 10:39:32 lisa kernel: [ 35.814486] RSP: 0018:ffffb06b005cfd00 EFLAGS: 00010246 Jul 9 10:39:32 lisa kernel: [ 35.814487] RAX: 0000000000000000 RBX: ffff911c2410ac58 RCX: 0000000000000000 Jul 9 10:39:32 lisa kernel: [ 35.814488] RDX: ffff911bed7acc40 RSI: 0000000000000000 RDI: ffff911c2410ac58 Jul 9 10:39:32 lisa kernel: [ 35.814489] RBP: ffff911c2410ac00 R08: ffff911be0da5d68 R09: ffff911be0da5d68 Jul 9 10:39:32 lisa kernel: [ 35.814490] R10: ffff911bc7fb7140 R11: 0000000000000001 R12: ffff911be0da5128 Jul 9 10:39:32 lisa kernel: [ 35.814490] R13: ffff911c2410ac00 R14: ffff911bd83be800 R15: ffffd06aff804b05 Jul 9 10:39:32 lisa kernel: [ 35.814491] FS: 0000000000000000(0000) GS:ffff911e9e800000(0000) knlGS:0000000000000000 Jul 9 10:39:32 lisa kernel: [ 35.814492] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 9 10:39:32 lisa kernel: [ 35.814493] CR2: 000055704d312208 CR3: 000000012ce10000 CR4: 0000000000750ee0 Jul 9 10:39:32 lisa kernel: [ 35.814493] PKRU: 55555554 Jul 9 10:39:32 lisa kernel: [ 35.814494] Call Trace: Jul 9 10:39:32 lisa kernel: [ 35.814495] <TASK> Jul 9 10:39:32 lisa kernel: [ 35.814497] amdgpu_bo_unpin+0x15/0x80 [amdgpu] Jul 9 10:39:32 lisa kernel: [ 35.814586] amdgpu_amdkfd_gpuvm_free_memory_of_gpu+0x350/0x420 [amdgpu] Jul 9 10:39:32 lisa kernel: [ 35.814691] kfd_process_device_free_bos+0x98/0xe0 [amdgpu] Jul 9 10:39:32 lisa kernel: [ 35.814786] kfd_process_wq_release+0x27f/0x340 [amdgpu] Jul 9 10:39:32 lisa kernel: [ 35.814876] process_one_work+0x1bd/0x310 Jul 9 10:39:32 lisa kernel: [ 35.814880] ? rescuer_thread+0x390/0x390 Jul 9 10:39:32 lisa kernel: [ 35.814881] worker_thread+0x4b/0x390 Jul 9 10:39:32 lisa kernel: [ 35.814883] ? rescuer_thread+0x390/0x390 Jul 9 10:39:32 lisa kernel: [ 35.814884] kthread+0xd4/0x100 Jul 9 10:39:32 lisa kernel: [ 35.814886] ? kthread_complete_and_exit+0x20/0x20 Jul 9 10:39:32 lisa kernel: [ 35.814888] ret_from_fork+0x22/0x30 Jul 9 10:39:32 lisa kernel: [ 35.814891] </TASK> Jul 9 10:39:32 lisa kernel: [ 35.814891] ---[ end trace 0000000000000000 ]--- Jul 9 10:39:33 lisa kernel: [ 35.816088] BUG: kernel NULL pointer dereference, address: 0000000000000008 Jul 9 10:39:33 lisa kernel: [ 35.816091] #PF: supervisor read access in kernel mode Jul 9 10:39:33 lisa kernel: [ 35.816093] #PF: error_code(0x0000) - not-present page Jul 9 10:39:33 lisa kernel: [ 35.816095] PGD 0 P4D 0 Jul 9 10:39:33 lisa kernel: [ 35.816097] Oops: 0000 [#1] PREEMPT SMP NOPTI Jul 9 10:39:33 lisa kernel: [ 35.816100] CPU: 8 PID: 126 Comm: kworker/8:1 Tainted: G W E N 5.19.0-rc5-next-20220708 #89 Jul 9 10:39:33 lisa kernel: [ 35.816103] Hardware name: Micro-Star International Co., Ltd. Alpha 15 B5EEK/MS-158L, BIOS E158LAMS.107 11/10/2021 Jul 9 10:39:33 lisa kernel: [ 35.816104] Workqueue: events delayed_fput Jul 9 10:39:33 lisa kernel: [ 35.816107] RIP: 0010:dma_resv_add_fence+0x3e/0x1a0 Jul 9 10:39:33 lisa kernel: [ 35.816112] Code: 89 54 24 04 48 85 f6 74 21 48 8d 7e 38 b8 01 00 00 00 f0 0f c1 46 38 85 c0 0f 84 0e 01 00 00 8d 50 01 09 c2 0f 88 12 01 00 00 <49> 8b 46 08 48 3d 80 0a ad 85 0f 84 ec 00 00 00 48 3d 20 0a ad 85 Jul 9 10:39:33 lisa kernel: [ 35.816114] RSP: 0018:ffffb06b005cfc98 EFLAGS: 00010246 Jul 9 10:39:33 lisa kernel: [ 35.816116] RAX: 0000000000000000 RBX: ffff911bd03a8158 RCX: 0000000080200013 Jul 9 10:39:33 lisa kernel: [ 35.816117] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff911bd03a8158 Jul 9 10:39:33 lisa kernel: [ 35.816119] RBP: ffff911bd03a8000 R08: 0000000000000000 R09: 0000000000000000 Jul 9 10:39:33 lisa kernel: [ 35.816120] R10: ffff911bc7916900 R11: 0000000000000000 R12: ffff911bd0499400 Jul 9 10:39:33 lisa kernel: [ 35.816121] R13: ffff911be09e5128 R14: 0000000000000000 R15: 00000000c0991f00 Jul 9 10:39:33 lisa kernel: [ 35.816122] FS: 0000000000000000(0000) GS:ffff911e9e800000(0000) knlGS:0000000000000000 Jul 9 10:39:33 lisa kernel: [ 35.816124] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 9 10:39:33 lisa kernel: [ 35.816125] CR2: 0000000000000008 CR3: 000000010409c000 CR4: 0000000000750ee0 Jul 9 10:39:33 lisa kernel: [ 35.816127] PKRU: 55555554 Jul 9 10:39:33 lisa kernel: [ 35.816128] Call Trace: Jul 9 10:39:33 lisa kernel: [ 35.816129] <TASK> Jul 9 10:39:33 lisa kernel: [ 35.816131] amdgpu_amdkfd_gpuvm_destroy_cb+0x53/0x1c0 [amdgpu] Jul 9 10:39:33 lisa kernel: [ 35.816223] amdgpu_vm_fini+0x39/0x4e0 [amdgpu] Jul 9 10:39:33 lisa kernel: [ 35.816295] ? amdgpu_ctx_mgr_entity_fini+0x4b/0xd0 [amdgpu] Jul 9 10:39:33 lisa kernel: [ 35.816366] amdgpu_driver_postclose_kms+0x1cb/0x2b0 [amdgpu] Jul 9 10:39:33 lisa kernel: [ 35.816433] drm_file_free.part.0+0x201/0x250 [drm] Jul 9 10:39:33 lisa kernel: [ 35.816448] drm_release+0x60/0x110 [drm] Jul 9 10:39:33 lisa kernel: [ 35.816459] __fput+0x87/0x240 Jul 9 10:39:33 lisa kernel: [ 35.816461] delayed_fput+0x1a/0x30 Jul 9 10:39:33 lisa kernel: [ 35.816462] process_one_work+0x1bd/0x310 Jul 9 10:39:33 lisa kernel: [ 35.816464] ? rescuer_thread+0x390/0x390 Jul 9 10:39:33 lisa kernel: [ 35.816465] worker_thread+0x4b/0x390 Jul 9 10:39:33 lisa kernel: [ 35.816466] ? rescuer_thread+0x390/0x390 Jul 9 10:39:33 lisa kernel: [ 35.816467] kthread+0xd4/0x100 Jul 9 10:39:33 lisa kernel: [ 35.816469] ? kthread_complete_and_exit+0x20/0x20 Jul 9 10:39:33 lisa kernel: [ 35.816471] ret_from_fork+0x22/0x30 Jul 9 10:39:33 lisa kernel: [ 35.816473] </TASK> Jul 9 10:39:33 lisa kernel: [ 35.816473] Modules linked in: ccm(EN) rfcomm(EN) bnep(EN) cpufreq_conservative(EN) cpufreq_ondemand(EN) cpufreq_powersave(EN) cpufreq_userspace(EN) snd_ctl_led(EN) btusb(EN) btrtl(EN) snd_hda_codec_realtek(EN) btbcm(EN) btintel(EN) btmtk(EN) snd_hda_codec_generic(EN) ledtrig_audio(EN) bluetooth(EN) snd_hda_codec_hdmi(EN) snd_hda_intel(EN) snd_soc_dmic(EN) snd_intel_dspcfg(EN) snd_acp3x_rn(EN) snd_acp3x_pdm_dma(EN) snd_hda_codec(EN) jitterentropy_rng(EN) snd_soc_core(EN) uvcvideo(EN) snd_hwdep(EN) snd_hda_core(EN) videobuf2_vmalloc(EN) snd_pcm_oss(EN) videobuf2_memops(EN) sha512_generic(EN) videobuf2_v4l2(EN) nls_ascii(EN) snd_mixer_oss(EN) nls_cp437(EN) videodev(EN) ccp(EN) snd_rn_pci_acp3x(EN) snd_pcm(EN) joydev(EN) snd_acp_config(EN) msi_wmi(EN) vfat(EN) snd_timer(EN) ctr(EN) drbg(EN) snd(EN) ecdh_generic(EN) evdev(EN) sparse_keymap(EN) wmi_bmof(EN) fat(EN) ecc(EN) videobuf2_common(EN) serio_raw(EN) hid_multitouch(EN) soundcore(EN) snd_soc_acpi(EN) efi_pstore(EN) rng_core(EN) Jul 9 10:39:33 lisa kernel: [ 35.816495] k10temp(EN) battery(EN) wmi(EN) ac(EN) button(EN) video(EN) hid_sensor_accel_3d(EN) hid_sensor_magn_3d(EN) hid_sensor_prox(EN) hid_sensor_als(EN) hid_sensor_gyro_3d(EN) hid_sensor_trigger(EN) industrialio_triggered_buffer(EN) kfifo_buf(EN) industrialio(EN) hid_sensor_iio_common(EN) amd_pmc(EN) acpi_cpufreq(EN) mt7921e(EN) mt7921_common(EN) mt76_connac_lib(EN) mt76(EN) mac80211(EN) libarc4(EN) cfg80211(EN) rfkill(EN) ipmi_devintf(EN) ipmi_msghandler(EN) msr(EN) fuse(EN) configfs(EN) efivarfs(EN) autofs4(EN) ext4(EN) crc32c_generic(EN) crc32c_intel(EN) crc16(EN) mbcache(EN) jbd2(EN) usbhid(EN) amdgpu(EN) drm_ttm_helper(EN) ttm(EN) gpu_sched(EN) i2c_algo_bit(EN) drm_buddy(EN) drm_display_helper(EN) nvme(EN) xhci_pci(EN) drm_kms_helper(EN) xhci_hcd(EN) nvme_core(EN) r8169(EN) syscopyarea(EN) sysfillrect(EN) hid_sensor_hub(EN) sysimgblt(EN) t10_pi(EN) mfd_core(EN) fb_sys_fops(EN) hid_generic(EN) drm(EN) usbcore(EN) realtek(EN) i2c_hid_acpi(EN) mdio_devres(EN) psmouse(EN) Jul 9 10:39:33 lisa kernel: [ 35.816516] i2c_hid(EN) amd_sfh(EN) libphy(EN) crc64_rocksoft(EN) hid(EN) backlight(EN) crc64(EN) crc_t10dif(EN) i2c_piix4(EN) cec(EN) usb_common(EN) crct10dif_generic(EN) crct10dif_common(EN) i2c_designware_platform(EN) i2c_designware_core(EN) Jul 9 10:39:33 lisa kernel: [ 35.816521] CR2: 0000000000000008 Jul 9 10:39:33 lisa kernel: [ 35.816523] ---[ end trace 0000000000000000 ]--- Jul 9 10:39:33 lisa kernel: [ 35.895575] RIP: 0010:dma_resv_add_fence+0x3e/0x1a0 Jul 9 10:39:33 lisa kernel: [ 35.895585] Code: 89 54 24 04 48 85 f6 74 21 48 8d 7e 38 b8 01 00 00 00 f0 0f c1 46 38 85 c0 0f 84 0e 01 00 00 8d 50 01 09 c2 0f 88 12 01 00 00 <49> 8b 46 08 48 3d 80 0a ad 85 0f 84 ec 00 00 00 48 3d 20 0a ad 85 Jul 9 10:39:33 lisa kernel: [ 35.895588] RSP: 0018:ffffb06b005cfc98 EFLAGS: 00010246 Jul 9 10:39:33 lisa kernel: [ 35.895591] RAX: 0000000000000000 RBX: ffff911bd03a8158 RCX: 0000000080200013 Jul 9 10:39:33 lisa kernel: [ 35.895592] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff911bd03a8158 Jul 9 10:39:33 lisa kernel: [ 35.895593] RBP: ffff911bd03a8000 R08: 0000000000000000 R09: 0000000000000000 Jul 9 10:39:33 lisa kernel: [ 35.895594] R10: ffff911bc7916900 R11: 0000000000000000 R12: ffff911bd0499400 Jul 9 10:39:33 lisa kernel: [ 35.895595] R13: ffff911be09e5128 R14: 0000000000000000 R15: 00000000c0991f00 Jul 9 10:39:33 lisa kernel: [ 35.895597] FS: 0000000000000000(0000) GS:ffff911e9e800000(0000) knlGS:0000000000000000 Jul 9 10:39:33 lisa kernel: [ 35.895598] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 9 10:39:33 lisa kernel: [ 35.895599] CR2: 0000000000000008 CR3: 000000010409c000 CR4: 0000000000750ee0 Jul 9 10:39:33 lisa kernel: [ 35.895600] PKRU: 55555554 Jul 9 10:39:38 lisa kernel: [ 41.995638] [drm] free PSP TMR bufferBisection gives
548e7432dc2da475a18077b612e8d55b8ff51891 is the first bad commit commit 548e7432dc2da475a18077b612e8d55b8ff51891 Author: Christian König <christian.koenig@amd.com> Date: Fri Sep 24 10:55:45 2021 +0200 dma-buf: add dma_resv_replace_fences v2 This function allows to replace fences from the shared fence list when we can gurantee that the operation represented by the original fence has finished or no accesses to the resources protected by the dma_resv object any more when the new fence finishes. Then use this function in the amdkfd code when BOs are unmapped from the process. v2: add an example when this is usefull. Signed-off-by: Christian König <christian.koenig@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Link: https://patchwork.freedesktop.org/patch/msgid/20220321135856.1331-1-christian.koenig@amd.com drivers/dma-buf/dma-resv.c | 45 ++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 49 ++++-------------------- include/linux/dma-resv.h | 2 + 3 files changed, 54 insertions(+), 42 deletions(-)as the first commit that triggers the NULL pointer dereference.
It seems that the revert patch from #2050 (closed) is already applied in linux-next-20220708.
Edited by Bert KarwatzkiThere is an important detail missing from the error message above (right before the [ cut here ]):
Jul 9 10:39:32 lisa kernel: [ 35.810730] amdgpu 0000:03:00.0: amdgpu: Failed to map peer:0000:08:00.0 mem_domain:2 Jul 9 10:39:32 lisa kernel: [ 35.810751] amdgpu 0000:08:00.0: amdgpu: Failed to map peer:0000:03:00.0 mem_domain:2Could this be the cause of the later error?
Edit: This error was a kernel configuration problem (CONFIG_HSA_AMD_P2P was missing). But this does not solve the NULL pointer error.
Edited by Bert KarwatzkiMy error was caused by
void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, struct amdgpu_vm *vm) { struct amdkfd_process_info *process_info = vm->process_info; struct amdgpu_bo *pd = vm->root.bo; if (!process_info) return; /* Release eviction fence from PD */ amdgpu_bo_reserve(pd, false); amdgpu_bo_fence(pd, NULL, false); amdgpu_bo_unreserve(pd);when amdgpu_bo_fence is called with fence == NULL it calls dma_resv_add_fence with fence == NULL. But dma_resv_add_fence dereferences fence. My current workaround is:
*/ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence, bool shared) { struct dma_resv *resv = bo->tbo.base.resv; int r; r = dma_resv_reserve_fences(resv, 1); if (r) { /* As last resort on OOM we block for the fence */ dma_fence_wait(fence, false); return; } // Only call dma_resv_add_fence if fence != NULL: if (fence) dma_resv_add_fence(resv, fence, shared ? DMA_RESV_USAGE_READ : DMA_RESV_USAGE_WRITE); }What exactly is the purpose of calling amdgpu_bo_fence with fence == NULL?
Edited by Bert Karwatzkimentioned in issue #2074 (closed)
Please register or sign in to reply