dGPU set as primary is going back and forth sleep/awake when no external displays connected to it

Brief summary of the problem:

When dGPU is set as primary via ENV{DEVNAME}=="/dev/dri/card0", TAG+="mutter-device-preferred-primary" and dGPU does not have displays directly connected to it (using iGPU eDP via PRIME) and dGPU has only Wayland clients (Gnome Wayland session, Firefix in Wayland mode) then dGPU goes to sleep and back awake every 7 seconds or so causing short UI freezes.

Hardware description:

• Machine Type: Laptop System: ASUSTeK product: ROG Strix G513QY_G513QY v: 1.0 (AMD Advantage Edition)
• CPU: 8-core model: AMD Ryzen 9 5900HX with Radeon Graphics
• iGPU: 08:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cezanne [1002:1638] (rev c4)
• dGPU (RX6800M): 03:00.0 Display controller [0380]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT / 6800M] [1002:73df] (rev c3)
• iGPU is connected to internal laptop screen via eDP, dGPU is set as primary for GNOME Wayland session
• System Memory: 64 GB

System information:

• Distro name and Version: Manjaro rolling
• Kernel version: Linux 5.17.6-1-MANJARO SMP PREEMPT Tue May 10 22:09:48 EEST 2022 x86_64 GNU/Linux

How to reproduce the issue:

• Create file /etc/udev/rules.d/61-mutter-primary-gpu.rules with content ENV{DEVNAME}=="/dev/dri/card0", TAG+="mutter-device-preferred-primary" where /dev/dri/card0 is the device node of the dGPU
• Boot into Gnome Wayland mode, open Gnome Terminal and run sudo journalctl -f, press Win button (Enter Gnome Workspace overview mode)
• Observe the log lines about dGPU waking up from sleep every 7 seconds or so

dGPU stops behaving like this when either of this happens

• External display is connected (attached journal shows this)
• sudo radeontop -b 3 is running
• glxgears in XWayland mode is running

Attached files:

Log files (for system lockups / game freezes / crashes)

• 5.17.6_journal_dgpu_react_to_extrnal_displays.log

mentioned in issue #1982 (closed)

From the original issue there was two asks that I don't believe were checked. This sounds like it's probably runtime PM going in and out. Can you please try these two things (separately)?

Does setting amdgpu.runpm=0 on the kernel command line help? Can you also try setting andgpu.ppfeaturemask=0xfff73fff on the kernel command line?

I tried both amdgpu.runpm=0 and amdgpu.ppfeaturemask=0xfff73fff separately. And in both cases issue is not reproducible.

Do you need any more info on this?

The dGPU is behaving as expected. When it's not in use, it powers down to save power. The powering up and down takes time. Disabling runtime power management or running a 3D application keeps the dGPU powered up. Is there a reason you want to make the dGPU the primary? You can make the APU the primary and still run games/apps on the dGPU when desired and save additional power. If you want the dGPU to not be powered down, you can use the amdgpu.runpm=0 parameter to disable it or you can dynamically enable/disable runtime power management using sysfs (https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-devices-power).

To force the dGPU on: sudo bash -c "echo on > /sys/devices/.../power/control"

To re-enable runtime power management: sudo bash -c "echo auto > /sys/devices/.../power/control"

Most of the time I use my laptop Plugged In that's why the power consumption is not critical. I have two 4k 60Hz external monitors. To get the best performance I connect them directly to the dGPU output via MST splitter USB-C alt mode because copying up to 2.8 GB of data per second from RAM over PCIe 3.0 bus does not seem very efficient to me. Ideally, when on Battery I would like it to automatically switch to iGPU but I don't think that's possible at the moment without restarting gnome-shell. At least I wanted it to stop freezing every 7 seconds.

I got you point about runtime power management. To build up on this idea:

• What would be the best way to prevent the dGPU from powering down but keep the dGPU power consumption to a minimum?
• Any way to configure the power down timeout? Looks like it's around 7 seconds by default.

• Any way to configure the power down timeout? Looks like it's around 7 seconds by default.

What you're looking for is: /sys/devices/.../power/control/autosuspend_delay_ms

The kernel sets that up by default to be 5 seconds for all devices IIRC. You can add to your /etc/udev/rules.d/61-mutter-primary-gpu.rules a longer time if it makes sense for your use case.

This issue hasn't had any activity since 2022-05-17. The AMD driver stack changes rapidly and contains lots of shared code across products so it's possible that it has already been fixed. Please upgrade to a current stable kernel and userspace stack and try again. If you still experience this issue with the latest driver stack, please capture relevant logging and open a new issue referring back to this one.

closed