Different value of pp_dpm_dcefclk and lower performance in games after reattach on 7900 XTX
Brief summary of the problem:
After reattaching GPU (tried device rescan and driver bind) I've noticed lowered performance in games and pp_dpm_dcefclk
return different value.
After system boot it returns:
cat /sys/bus/pci/devices/0000:03:00.0/pp_dpm_dcefclk
0: 145Mhz
1: 400Mhz *
2: 1563Mhz
after doing echo 1 > /sys/bus/pci/devices/0000:03:00.0/remove
and echo 1 > /sys/bus/pci/rescan
(or unbind, bind) it returns:
cat /sys/bus/pci/devices/0000:03:00.0/pp_dpm_dcefclk
0: 145Mhz *
1: 1563Mhz
And performance in Overwatch drops from stable 240 fps to unstable 100.
Hardware description:
- CPU: AMD Ryzen 9 7950x
- GPU: AMD Radeon 7900 XTX
- System Memory: 32GB
- Display(s): 2
- Type of Display Connection: DP,HDMI
System information:
- Distro name and Version: NixOS 24.05.20240425.7bb2ccd
- Kernel version: 6.8.6-zen1
- Custom kernel: Zen
- AMD official driver version: N/A
How to reproduce the issue:
- Check values of
pp_dpm_dcefclk
- Unbind/remove gpu
- Reattach/rescan
- Check values of
pp_dpm_dcefclk
again
Log files (for system lockups / game freezes / crashes)
Dmesg log after remove/rescan
[ 556.867553] amdgpu 0000:03:00.0: amdgpu: amdgpu: finishing device.
[ 556.895809] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895817] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895831] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895833] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895846] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895848] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895856] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895859] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895862] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895864] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895868] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895870] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895973] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895976] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895980] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895982] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895985] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895988] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.895991] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.895994] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.896001] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.896003] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.896007] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.896009] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.896035] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.896037] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 556.896040] amdgpu 0000:03:00.0: amdgpu: failed to clear page tables on GEM object close (-19)
[ 556.896043] amdgpu 0000:03:00.0: amdgpu: leaking bo va (-19)
[ 563.343035] pci 0000:03:00.0: [1002:744c] type 00 class 0x030000 PCIe Legacy Endpoint
[ 563.343053] pci 0000:03:00.0: BAR 0 [mem 0x00000000-0x0fffffff 64bit pref]
[ 563.343062] pci 0000:03:00.0: BAR 2 [mem 0x00000000-0x001fffff 64bit pref]
[ 563.343068] pci 0000:03:00.0: BAR 4 [io 0x0000-0x00ff]
[ 563.343073] pci 0000:03:00.0: BAR 5 [mem 0x00000000-0x000fffff]
[ 563.343078] pci 0000:03:00.0: ROM [mem 0x00000000-0x0001ffff pref]
[ 563.343085] pci 0000:03:00.0: Max Payload Size set to 256 (was 128, max 256)
[ 563.343161] pci 0000:03:00.0: PME# supported from D1 D2 D3hot D3cold
[ 563.343382] pci 0000:03:00.0: Adding to iommu group 14
[ 563.343410] pci 0000:03:00.0: vgaarb: bridge control possible
[ 563.343412] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[ 563.343440] pcieport 0000:02:00.0: ASPM: current common clock configuration is inconsistent, reconfiguring
[ 563.352768] pci 0000:03:00.0: BAR 0 [mem 0x00000000-0x0fffffff 64bit pref]: releasing
[ 563.352775] pci 0000:03:00.0: BAR 2 [mem 0x00000000-0x001fffff 64bit pref]: releasing
[ 563.352781] pci 0000:03:00.0: BAR 0 [mem 0xf000000000-0xf00fffffff 64bit pref]: assigned
[ 563.352788] pci 0000:03:00.0: BAR 2 [mem 0xf010000000-0xf0101fffff 64bit pref]: assigned
[ 563.352796] pci 0000:03:00.0: BAR 5 [mem 0xfcb00000-0xfcbfffff]: assigned
[ 563.352799] pci 0000:03:00.0: ROM [mem 0xfcc00000-0xfcc1ffff pref]: assigned
[ 563.352802] pci 0000:03:00.0: BAR 4 [io 0xf000-0xf0ff]: assigned
[ 563.352861] pci 0000:03:00.0: BAR 0 [mem 0xf000000000-0xf00fffffff 64bit pref]: releasing
[ 563.352864] pci 0000:03:00.0: BAR 2 [mem 0xf010000000-0xf0101fffff 64bit pref]: releasing
[ 563.352880] pcieport 0000:02:00.0: bridge window [mem 0xf000000000-0xf80fffffff 64bit pref]: releasing
[ 563.352882] pcieport 0000:01:00.0: bridge window [mem 0xf000000000-0xf80fffffff 64bit pref]: releasing
[ 563.352884] pcieport 0000:00:01.1: bridge window [mem 0xf000000000-0xf80fffffff 64bit pref]: releasing
[ 563.352891] pcieport 0000:00:01.1: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: assigned
[ 563.352894] pcieport 0000:01:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: assigned
[ 563.352896] pcieport 0000:02:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: assigned
[ 563.352899] pci 0000:03:00.0: BAR 0 [mem 0x880000000-0x88fffffff 64bit pref]: assigned
[ 563.352906] pci 0000:03:00.0: BAR 2 [mem 0x890000000-0x89fffffff 64bit pref]: assigned
[ 563.352913] pcieport 0000:00:01.1: PCI bridge to [bus 01-03]
[ 563.352916] pcieport 0000:00:01.1: bridge window [io 0xf000-0xffff]
[ 563.352919] pcieport 0000:00:01.1: bridge window [mem 0xfcb00000-0xfcdfffff]
[ 563.352922] pcieport 0000:00:01.1: bridge window [mem 0x880000000-0x89fffffff 64bit pref]
[ 563.352932] pcieport 0000:01:00.0: PCI bridge to [bus 02-03]
[ 563.352935] pcieport 0000:01:00.0: bridge window [io 0xf000-0xffff]
[ 563.352939] pcieport 0000:01:00.0: bridge window [mem 0xfcb00000-0xfccfffff]
[ 563.352942] pcieport 0000:01:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]
[ 563.352947] pcieport 0000:02:00.0: PCI bridge to [bus 03]
[ 563.352950] pcieport 0000:02:00.0: bridge window [io 0xf000-0xffff]
[ 563.352953] pcieport 0000:02:00.0: bridge window [mem 0xfcb00000-0xfccfffff]
[ 563.352957] pcieport 0000:02:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]
[ 563.392299] sysfs: cannot create duplicate filename '/devices/pci0000:00/0000:00:01.1/0000:01:00.0/0000:02:00.0/0000:03:00.0/resource0'
[ 563.392303] CPU: 6 PID: 14349 Comm: zsh Not tainted 6.8.6-zen1 #1-NixOS
[ 563.392306] Hardware name: ASUS System Product Name/TUF GAMING X670E-PLUS WIFI, BIOS 1636 07/28/2023
[ 563.392308] Call Trace:
[ 563.392310] <TASK>
[ 563.392312] dump_stack_lvl+0x64/0x80
[ 563.392317] sysfs_warn_dup+0x8a/0xa0
[ 563.392322] sysfs_add_bin_file_mode_ns+0x9f/0xd0
[ 563.392326] sysfs_create_bin_file+0x6d/0xa0
[ 563.392329] pci_create_attr+0xfb/0x180
[ 563.392333] pci_create_resource_files+0x42/0xb0
[ 563.392336] pci_bus_add_device+0x2f/0x90
[ 563.392340] pci_bus_add_devices+0x30/0x70
[ 563.392343] pci_bus_add_devices+0x5b/0x70
[ 563.392346] pci_bus_add_devices+0x5b/0x70
[ 563.392349] pci_bus_add_devices+0x5b/0x70
[ 563.392353] pci_rescan_bus+0x27/0x30
[ 563.392355] rescan_store+0x71/0xa0
[ 563.392359] kernfs_fop_write_iter+0x133/0x1d0
[ 563.392362] vfs_write+0x29b/0x470
[ 563.392367] ksys_write+0x6f/0xf0
[ 563.392370] do_syscall_64+0xc1/0x210
[ 563.392374] entry_SYSCALL_64_after_hwframe+0x79/0x81
[ 563.392377] RIP: 0033:0x7b7af13fc6b4
[ 563.392405] Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d 95 89 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 563.392407] RSP: 002b:00007ffd88d8f588 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 563.392410] RAX: ffffffffffffffda RBX: 00000000004d6820 RCX: 00007b7af13fc6b4
[ 563.392412] RDX: 0000000000000002 RSI: 00000000004d6820 RDI: 0000000000000001
[ 563.392413] RBP: 0000000000000002 R08: 0000000000000000 R09: 00007b7af12f9960
[ 563.392414] R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000002
[ 563.392416] R13: 00007b7af14de5c0 R14: 00007b7af14dbf00 R15: 00007b7af12f9960
[ 563.392420] </TASK>
[ 563.392476] amdgpu 0000:03:00.0: enabling device (0000 -> 0003)
[ 563.392509] [drm] initializing kernel modesetting (IP DISCOVERY 0x1002:0x744C 0x1DA2:0x471E 0xC8).
[ 563.392522] [drm] register mmio base: 0xFCB00000
[ 563.392523] [drm] register mmio size: 1048576
[ 563.396981] [drm] add ip block number 0 <soc21_common>
[ 563.396983] [drm] add ip block number 1 <gmc_v11_0>
[ 563.396985] [drm] add ip block number 2 <ih_v6_0>
[ 563.396986] [drm] add ip block number 3 <psp>
[ 563.396987] [drm] add ip block number 4 <smu>
[ 563.396988] [drm] add ip block number 5 <dm>
[ 563.396989] [drm] add ip block number 6 <gfx_v11_0>
[ 563.396990] [drm] add ip block number 7 <sdma_v6_0>
[ 563.396991] [drm] add ip block number 8 <vcn_v4_0>
[ 563.396997] [drm] add ip block number 9 <jpeg_v4_0>
[ 563.396998] [drm] add ip block number 10 <mes_v11_0>
[ 563.400440] amdgpu 0000:03:00.0: amdgpu: Fetched VBIOS from ATRM
[ 563.400442] amdgpu: ATOM BIOS: 113-3E4710U-O4O
[ 563.401026] amdgpu 0000:03:00.0: amdgpu: CP RS64 enable
[ 563.401510] [drm] VCN(0) encode/decode are enabled in VM mode
[ 563.401511] [drm] VCN(1) encode/decode are enabled in VM mode
[ 563.401673] amdgpu 0000:03:00.0: [drm:jpeg_v4_0_early_init [amdgpu]] JPEG decode is enabled in VM mode
[ 563.402048] amdgpu 0000:03:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported
[ 563.402060] [drm] GPU posting now...
[ 563.402081] amdgpu 0000:03:00.0: amdgpu: MEM ECC is not presented.
[ 563.402082] amdgpu 0000:03:00.0: amdgpu: SRAM ECC is not presented.
[ 563.402090] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[ 563.402101] amdgpu 0000:03:00.0: BAR 2 [mem 0x890000000-0x89fffffff 64bit pref]: releasing
[ 563.402103] amdgpu 0000:03:00.0: BAR 0 [mem 0x880000000-0x88fffffff 64bit pref]: releasing
[ 563.402116] pcieport 0000:02:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: releasing
[ 563.402118] pcieport 0000:01:00.0: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: releasing
[ 563.402119] pcieport 0000:00:01.1: bridge window [mem 0x880000000-0x89fffffff 64bit pref]: releasing
[ 563.402123] pcieport 0000:00:01.1: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]: assigned
[ 563.402125] pcieport 0000:01:00.0: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]: assigned
[ 563.402126] pcieport 0000:02:00.0: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]: assigned
[ 563.402128] amdgpu 0000:03:00.0: BAR 0 [mem 0x1000000000-0x17ffffffff 64bit pref]: assigned
[ 563.402134] amdgpu 0000:03:00.0: BAR 2 [mem 0xc00000000-0xc0fffffff 64bit pref]: assigned
[ 563.402141] pcieport 0000:00:01.1: PCI bridge to [bus 01-03]
[ 563.402143] pcieport 0000:00:01.1: bridge window [io 0xf000-0xffff]
[ 563.402145] pcieport 0000:00:01.1: bridge window [mem 0xfcb00000-0xfcdfffff]
[ 563.402147] pcieport 0000:00:01.1: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]
[ 563.402156] pcieport 0000:01:00.0: PCI bridge to [bus 02-03]
[ 563.402158] pcieport 0000:01:00.0: bridge window [io 0xf000-0xffff]
[ 563.402162] pcieport 0000:01:00.0: bridge window [mem 0xfcb00000-0xfccfffff]
[ 563.402164] pcieport 0000:01:00.0: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]
[ 563.402168] pcieport 0000:02:00.0: PCI bridge to [bus 03]
[ 563.402170] pcieport 0000:02:00.0: bridge window [io 0xf000-0xffff]
[ 563.402178] pcieport 0000:02:00.0: bridge window [mem 0xfcb00000-0xfccfffff]
[ 563.402180] pcieport 0000:02:00.0: bridge window [mem 0xc00000000-0x17ffffffff 64bit pref]
[ 563.402186] amdgpu 0000:03:00.0: amdgpu: VRAM: 24560M 0x0000008000000000 - 0x00000085FEFFFFFF (24560M used)
[ 563.402188] amdgpu 0000:03:00.0: amdgpu: GART: 512M 0x00007FFF00000000 - 0x00007FFF1FFFFFFF
[ 563.402197] [drm] Detected VRAM RAM=24560M, BAR=32768M
[ 563.402198] [drm] RAM width 384bits GDDR6
[ 563.402273] [drm] amdgpu: 24560M of VRAM memory ready
[ 563.402275] [drm] amdgpu: 15887M of GTT memory ready.
[ 563.402291] [drm] GART: num cpu pages 131072, num gpu pages 131072
[ 563.402349] [drm] PCIE GART of 512M enabled (table at 0x00000085FEB00000).
[ 563.402855] [drm] Loading DMUB firmware via PSP: version=0x07002900
[ 563.402974] [drm] Found VCN firmware Version ENC: 1.16 DEC: 5 VEP: 0 Revision: 6
[ 563.402985] amdgpu 0000:03:00.0: amdgpu: Will use PSP to load VCN firmware
[ 563.473437] [drm] reserve 0x1300000 from 0x85fc000000 for PSP TMR
[ 563.617330] amdgpu 0000:03:00.0: amdgpu: RAP: optional rap ta ucode is not available
[ 563.617332] amdgpu 0000:03:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available
[ 563.617403] amdgpu 0000:03:00.0: amdgpu: smu driver if version = 0x0000003d, smu fw if version = 0x0000003f, smu fw program = 0, smu fw version = 0x004e7900 (78.121.0)
[ 563.617405] amdgpu 0000:03:00.0: amdgpu: SMU driver if version not matched
[ 563.779379] amdgpu 0000:03:00.0: amdgpu: SMU is initialized successfully!
[ 563.779678] [drm] Display Core v3.2.266 initialized on DCN 3.2
[ 563.779680] [drm] DP-HDMI FRL PCON supported
[ 563.781756] [drm] DMUB hardware initialized: version=0x07002900
[ 563.793060] snd_hda_intel 0000:03:00.1: bound 0000:03:00.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu])
[ 564.014065] [drm] kiq ring mec 3 pipe 1 q 0
[ 564.021064] [drm] VCN decode and encode initialized successfully(under DPG Mode).
[ 564.021198] amdgpu 0000:03:00.0: [drm:jpeg_v4_0_hw_init [amdgpu]] JPEG decode initialized successfully.
[ 564.076639] amdgpu: HMM registered 24560MB device memory
[ 564.077920] kfd kfd: amdgpu: Allocated 3969056 bytes on gart
[ 564.077932] kfd kfd: amdgpu: Total number of KFD nodes to be created: 1
[ 564.077958] amdgpu: Virtual CRAT table created for GPU
[ 564.078602] amdgpu: Topology: Add dGPU node [0x744c:0x1002]
[ 564.078604] kfd kfd: amdgpu: added device 1002:744c
[ 564.078615] amdgpu 0000:03:00.0: amdgpu: SE 6, SH per SE 2, CU per SH 8, active_cu_number 96
[ 564.078618] amdgpu 0000:03:00.0: amdgpu: ring gfx_0.0.0 uses VM inv eng 0 on hub 0
[ 564.078620] amdgpu 0000:03:00.0: amdgpu: ring comp_1.0.0 uses VM inv eng 1 on hub 0
[ 564.078621] amdgpu 0000:03:00.0: amdgpu: ring comp_1.1.0 uses VM inv eng 4 on hub 0
[ 564.078622] amdgpu 0000:03:00.0: amdgpu: ring comp_1.2.0 uses VM inv eng 6 on hub 0
[ 564.078623] amdgpu 0000:03:00.0: amdgpu: ring comp_1.3.0 uses VM inv eng 7 on hub 0
[ 564.078624] amdgpu 0000:03:00.0: amdgpu: ring comp_1.0.1 uses VM inv eng 8 on hub 0
[ 564.078625] amdgpu 0000:03:00.0: amdgpu: ring comp_1.1.1 uses VM inv eng 9 on hub 0
[ 564.078626] amdgpu 0000:03:00.0: amdgpu: ring comp_1.2.1 uses VM inv eng 10 on hub 0
[ 564.078627] amdgpu 0000:03:00.0: amdgpu: ring comp_1.3.1 uses VM inv eng 11 on hub 0
[ 564.078628] amdgpu 0000:03:00.0: amdgpu: ring sdma0 uses VM inv eng 12 on hub 0
[ 564.078629] amdgpu 0000:03:00.0: amdgpu: ring sdma1 uses VM inv eng 13 on hub 0
[ 564.078630] amdgpu 0000:03:00.0: amdgpu: ring vcn_unified_0 uses VM inv eng 0 on hub 8
[ 564.078631] amdgpu 0000:03:00.0: amdgpu: ring vcn_unified_1 uses VM inv eng 1 on hub 8
[ 564.078632] amdgpu 0000:03:00.0: amdgpu: ring jpeg_dec uses VM inv eng 4 on hub 8
[ 564.078633] amdgpu 0000:03:00.0: amdgpu: ring mes_kiq_3.1.0 uses VM inv eng 14 on hub 0
[ 564.079757] [drm] ring gfx_32768.1.1 was added
[ 564.080024] [drm] ring compute_32768.2.2 was added
[ 564.080236] [drm] ring sdma_32768.3.3 was added
[ 564.080285] [drm] ring gfx_32768.1.1 ib test pass
[ 564.080350] [drm] ring compute_32768.2.2 ib test pass
[ 564.080375] [drm] ring sdma_32768.3.3 ib test pass
[ 564.081774] amdgpu 0000:03:00.0: amdgpu: Using BOCO for runtime pm
[ 564.082356] [drm] Initialized amdgpu 3.57.0 20150101 for 0000:03:00.0 on minor 0
[ 564.092024] amdgpu 0000:03:00.0: [drm] fb1: amdgpudrmfb frame buffer device