External screen failure on Dell XPS 7390 2-in-1 (Ice Lake i7-1065G7 CPU) and Thunderbolt dock

[1.] One line summary of the problem:

After anywhere from 15min to two hours of use, external screens attached via the Thunderbolt dock go dark and have to be disabled/reenabled (or hard reboot) to get a signal again.

[2.] Full description of the problem/report:

After some time running 2 external monitors over the Thunderbolt 3 dock attached to the Dell XPS 7390 2-in-1, both external monitors suddenly lose their signal. The screens are still "seen" by the laptop, but display no picture (and the monitors go into power save thinking no device is attached).

The internal laptop screen continues to work with no issue.

Following messages can be seen in the journal:

[drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe C FIFO underrun

or

[drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe B FIFO underrun

Disabling and reenabling the screen in the GNOME display configuration sometimes fixes the issue, other times the entire device needs to be powered down and restarted for them to work again.

This issue persisted from kernel 5.4 all the way to the most recent version (5.5.9) on Arch Linux.

[3.] Keywords (i.e., modules, networking, kernel):

i915, firmware, intel UHD graphics

[4.] Kernel information

[4.1.] Kernel version (from /proc/version):

Linux version 5.5.9-arch1-2 (linux@archlinux) (gcc version 9.3.0 (Arch Linux 9.3.0-1)) #1 (moved) SMP PREEMPT Thu, 12 Mar 2020 23:01:33 +0000

[5.] Most recent kernel version which did not have the bug:

This issue persisted ever since I used the Thunderbolt dock with the device, from kernel 5.4 onwards.

[6.] Output of error message (found in dmesg):

[drm:intel_cpu_fifo_underrun_irq_handler [i915]] *ERROR* CPU pipe C FIFO underrun

[7.] A small shell script or example program which triggers the problem (if possible)

The problem cannot be reproduced easily, but seems to be triggered sometimes by a click or other activity in Chromium.

[8.] Environment

Arch Linux with GNOME desktop/Wayland

[8.1.] Software (add the output of the ver_linux script here)

[8.2.] Processor information (from /proc/cpuinfo):

cat /proc/cpuinfo

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 126
model name	: Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz
stepping	: 5
microcode	: 0x46
cpu MHz		: 825.986
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 27
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid md_clear flush_l1d arch_capabilities
bugs		: spectre_v1 spectre_v2 spec_store_bypass swapgs itlb_multihit
bogomips	: 2996.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

[8.3.] Module information (from /proc/modules):

i915 2449408 18 - Live 0x0000000000000000
intel_gtt 24576 1 i915, Live 0x0000000000000000
i2c_algo_bit 16384 1 i915, Live 0x0000000000000000
drm_kms_helper 237568 1 i915, Live 0x0000000000000000
drm 557056 10 i915,drm_kms_helper, Live 0x0000000000000000

[8.4.] Loaded driver and hardware information (/proc/ioports, /proc/iomem)

cat /proc/ioports

0000-0000 : PCI Bus 0000:00
  0000-0000 : dma1
  0000-0000 : pic1
  0000-0000 : iTCO_wdt
    0000-0000 : iTCO_wdt
  0000-0000 : timer0
  0000-0000 : timer1
  0000-0000 : keyboard
  0000-0000 : keyboard
  0000-0000 : rtc0
  0000-0000 : dma page reg
  0000-0000 : pic2
  0000-0000 : dma2
  0000-0000 : fpu
  0000-0000 : iTCO_wdt
    0000-0000 : iTCO_wdt
  0000-0000 : pnp 00:00
  0000-0000 : PNP0C09:00
    0000-0000 : EC data
  0000-0000 : PNP0C09:00
    0000-0000 : EC cmd
0000-0000 : PCI conf1
0000-0000 : PCI Bus 0000:00
  0000-0000 : pnp 00:00
  0000-0000 : ACPI PM1a_EVT_BLK
  0000-0000 : ACPI PM1a_CNT_BLK
  0000-0000 : ACPI PM_TMR
  0000-0000 : ACPI CPU throttle
  0000-0000 : ACPI PM2_CNT_BLK
  0000-0000 : pnp 00:02
  0000-0000 : ACPI GPE0_BLK
  0000-0000 : pnp 00:07
  0000-0000 : PCI Bus 0000:5a
  0000-0000 : 0000:00:02.0
  0000-0000 : PCI Bus 0000:01
    0000-0000 : PCI Bus 0000:02
      0000-0000 : PCI Bus 0000:04
  0000-0000 : PCI Bus 0000:2e
  0000-0000 : 0000:00:1f.4
    0000-0000 : i801_smbus

[8.5.] PCI information ('lspci -vvv' as root)

See lspci.txt

[8.6.] SCSI information (from /proc/scsi/scsi)

Empty

[8.7.] Other information that might be relevant to the problem (please look in /proc and include all information that you think to be relevant):

The driver generally has a problem with external monitors when the device is booted with the lid closed. The external monitors have no signal and the internal display turns on and off every second, with the journal printing:

Mär 09 11:05:03 notebook gnome-shell[2004]: Failed to post KMS update: Page flip of 91 failed, and no mode set available
Mär 09 11:05:04 notebook gnome-shell[2004]: Failed to post KMS update: Failed to set mode 2560x1440 on CRTC 91: Invalid argument

Although this seems to be a separate issue. Booting with the lid opened causes the issue not to appear.

[X.] Other notes, patches, fixes, workarounds:

None.

added Community kernel:5.4 labels

added platform: ICL label

added feature: display/watermark priority::medium severity::normal labels

I am having the same issue and so far documented my journey in the arch user forum. If putting everything together in the same format as above helps, let me know.

The issue still persists as of the recently released Kernel 5.7.

Here is the debug log of the drm driver during such an event:

[   80.807016] [drm:gen8_de_irq_handler.isra.0 [i915]] hotplug event received, stat 0x00000001, dig 0x00008889, pins 0x00000040, long 0x00000000
[   80.807043] [drm:intel_hpd_irq_handler [i915]] digital hpd on [ENCODER:221:DDI C] - short
[   80.807169] [drm:intel_dp_hpd_pulse [i915]] got hpd irq on [ENCODER:221:DDI C] - short
[   80.808923] [drm:intel_dp_hpd_pulse [i915]] channel EQ not ok, retraining
[   80.809552] [drm:intel_dp_start_link_train [i915]] Using LINK_BW_SET value 1e
[   80.810124] [drm:intel_dp_set_signal_levels [i915]] Using vswing level 0
[   80.810143] [drm:intel_dp_set_signal_levels [i915]] Using pre-emphasis level 0
[   80.810162] [drm:intel_dp_program_link_training_pattern [i915]] Using DP training pattern TPS1
[   80.812378] [drm:intel_dp_set_signal_levels [i915]] Using vswing level 0
[   80.812396] [drm:intel_dp_set_signal_levels [i915]] Using pre-emphasis level 0
[   80.815371] [drm:intel_dp_start_link_train [i915]] clock recovery OK
[   80.815390] [drm:intel_dp_program_link_training_pattern [i915]] Using DP training pattern TPS7
[   80.838345] [drm:drm_mode_addfb2 [drm]] [FB:263]
[   80.848566] [drm:intel_dp_start_link_train [i915]] Channel EQ done. DP Training successful
[   80.849261] [drm:intel_dp_hpd_pulse [i915]] [CONNECTOR:222:DP-1] Link Training Passed at Link Rate = 810000, Lane count = 4
[   80.849496] [drm:intel_dp_hpd_pulse [i915]] got esi 02 00 00
[   80.943929] [drm:drm_mode_addfb2 [drm]] [FB:258]

@grOGH I too have experienced this since getting the XPS7390 2-in-1 and WD19TB dock.

One thing that I noticed is that it doesn't seem to affect external non-TB monitors. For example, with 2 monitors connected to the WD19TB, and 1 monitor connected via a simple USB-C to DP adapter, the USB-C/DP monitor stays alive when the WD19TB connected ones die.

Additionally, it seems like in my case switching to a VT from Sway and back again seems to bring them back to life, for a time. They die pretty quickly again after that (much quicker than after a reboot), and even if they don't die outright often flicker.

I haven't been able to find a triggering cause either. Things are quite stable after a reboot and seem to deteriorate after that.

Yes, the issue only appears with Thunderbolt. It is not just the displays - checking the logs revealed that all USB devices attached to the dock get briefly disconnected, including external HDD drives which get write corruption errors.

I also have a USB-C port replicator that works flawlessly.

I would assume it is a bug in the Thunderbolt driver, or maybe somehow related to this issue: Random crashes in IntelliJ IDEA on Ice Lake CPU systems

Good catch, I am experiencing similar issues with Jetbrains IDEs as well.

Hi there, I too have an XPS 7390 2-in-1 and a WD19TB with the described problems but I have the same problem on various USB-C hub-adapters. Since I have one HDMI monitor and it didn't work very well on the WD19TB I switched back to different USB-C adapters I have laying around. They are from different vendors like QacQoc and HP. The adapters work quiet well on my work HP Zbook 15 G4 so they shouldn't be the issue.

After a normal boot of the XPS the HDMI screen is dead. I have to switch to a virtual terminal and reconnect the monitor several times to get the vt to display. If the vt display works I can switch back to Wayland/Xorg and the monitor works. Sometimes I have flickering on the external monitor which can be resolved by going back to the vt and reconnect the monitor.

How can I assist in further debugging since not having the external monitor or having a buggy system kind of sucks :D

I am using arch linux wayland and kde plasma

Out of curiosity I just retried the WD19TB and got this

Jun 19 12:58:06 rtfm kernel: [drm:drm_dp_mst_dpcd_read [drm_kms_helper]] *ERROR* mstb 00000000eb5492a9 port 1: DPCD read on addr 0x4b0 for 1 bytes NAKed
Jun 19 12:58:06 rtfm kernel: [drm:drm_dp_mst_dpcd_read [drm_kms_helper]] *ERROR* mstb 00000000eb5492a9 port 2: DPCD read on addr 0x4b0 for 1 bytes NAKed
Jun 19 12:58:06 rtfm kernel: [drm:drm_dp_mst_dpcd_read [drm_kms_helper]] *ERROR* mstb 00000000eb5492a9 port 3: DPCD read on addr 0x4b0 for 1 bytes NAKed

Another thing I just noticed: If I switch to vt directly after wayland boot and then plug in the USB-C dock the external monitor is recognized immediately and works fine. Didn't try this with the WD19TB till now.

The frustrating problem described here also occurs on the newer Dell XPS 13 9300 model with the WD19TB dock although the two screens usually only go black for about 6 secs and then both or one recovers. I can go many hours without seeing the issue but then it may occur 4 times in a few minutes. Sometimes neither recover and I have to unplug the TB cable, wait, and then reconnect it. Sometimes even that will not recover the monitors and I have to reboot. The intermittent blackouts seem to occur more often when the laptop is under heavy load.

Another issue, probably related, is that the monitors sometimes do not turn back on after the laptop has suspended. I get the same ERROR CPU pipe A FIFO underrun (for A, B, and C), page flip failed, and mstb errors quote by others above.

The dock is driving 2 1440p monitors each connected via DP. I am running current Arch Linux with kernel 5.7.10 and have used fwupd to install latest BIOS and WD19TB firmwares. Setting i915.enable_guc=2 seems to have reduced the frequency of this/these problems but certainly has not eliminated it.

Same observation here. Additionally: It appears I can make the monitors work for extended periods of time by plugging in and doing a cold boot cycle. (Rebooting is not enough, it will have my screens flicker immediately after the DE comes up).

Apparently, it is possible to temporarily fix this issue by getting the WD19TB into a partially "broken" state.

For that, you need to power up the device before connecting it to the dock, and then hotplug the dock while it is running. Sometimes, the dock doesn't fully recognize the connection - the LED on the connector cable is on, but the LED on the power button of the dock remains off. Once the dock is in this state, the issue seems not to appear. All device connections and displays continue to work normally in this "degraded" state.

So keep hotplugging the dock until the dock power button LED is off, and then you're good. If its on, the issue still happens, even with kernel 5.8.

Guys, I have found a simple work around that fixes this and all issues with Linux and the WD19TB dock. I have written about it here.

Thanks for your research - so the MST is the issue.

However, can you try the following: there was a BIOS update in September for the 7390 2-in-1 (and other Ice Lake laptops I presume) from Dell that also updates the internal Thunderbolt controller firmware.

Can you try applying the update and run the dock again with MST enabled? You need a Windows installation for the update, it is not applied with fwupd.

After the update, I only had one instance where the entire dock crashed and rebooted (visible in the USB reconnects in dmesg), but no simple display failure yet. I'll update if it happens again and try with MST disabled.

As per that link, mine is an XPS 13 9300 (same Ice Lake i7-1065G7 CPU, which is why the same MST issue arises I suspect) and am running the latest Dell BIOS 1.2.0 and the latest firmwares via fwupd. I do not have Windows installed on this machine and no access to any Windows systems.

I was convinced after only 2 days (but forced myself to wait a full week before reporting this) that disabling MST cleared the faults because not only did the critical screen blackouts and dock disconnects stop happening, but when resuming from standby, or suspend, or reboot then the screens would simply connect immediately and gracefully turn on, rather than blink/bounce all over the place and randomly fail to connect as I was accustomed to.

You need a Windows installation for the update, it is not applied with fwupd.

Nah, it can be done via a FAT-formatted USB from the BIOS (hit "F12"(? I think, maybe "F7") on boot to see the menu.

@mark.blakeney Unfortunately, disabling mst didn't solve the problem for me. I still have random faults.

I am using XPS 13 9300 with the latest BIOS update (1.2.0). Also, I tested with 2 different docks - Dell TB16 and Lenovo Thunderbolt Essential Gen 2. Both docks have the same problem.

@vladimir.didenko sorry but you will see every other participant here is referring to problems with the Dell WD19TB Thunderbolt 3 dock like myself so my only claim is that the problems with that dock and Dell XPS Ice Lake laptops are addressed with the i915.enable_dp_mst=0 option (and possibly latest firmware, late kernel, etc).

@mark.blakeney Disabling mst isn't a workaround. Aside from the fact that as you mention, you needed to buy an extra dongle to get two displays working, some displays require mst to hit the higher data rates.

For me 1.6.2 BIOS has be remarkably better. The time between seeing the issue is in days for me now.

Note that users who only want to run a single external screen from the dock only need to add the kernel option. The can continue to use the dock's HDMI, or either DP port.

The TB3->DP cable I mention is only required if you want the dock to run a second screen, e.g. swap the second DP->DP to a TB3->DP cable.

I can now confirm that even with the latest updates, the issue still persists in a different form.

The screens will go blank, but recover after a few seconds. However, now the dock crashes randomly, disconnecting all devices and screens, then reboots and everything reconnects again.

I am not sure what is responsible - the a0 microcode, the BIOS update, or Kernel 5.9.1, but at least something is happening.

@grOGH what you are calling a "different form" is how the problem has always been for me. As I said in my description, sometimes the screens just black out intermittently and for this case you see nothing reported in the journal, other times the entire dock fleetingly disconnects and you see all the USB etc disconnect/reconnects in the journal. These 2 different forms of the issue have always existed for me, i.e. through all CPU microcode versions I have installed (i.e. 0x78, 0x96, 0x9e, and 0xa0), and the 25 Arch kernel updates I have seen in that time (i.e. 5.7.4.arch1-1 -> 5.9.1.arch1-1), and the 3 BIOS updates to 1.2.0. I suspect most people who report "my screens black out intermittently" have not bothered to note the subtle difference in the two cases.

These problems go completely away (and the screens also restore gracefully and 100% reliably after suspend/standby) when I add the i915.enable_dp_mst=0 option. Did you try it?!

I did, but the dock then can't drive my 1440p screen properly. I am waiting for a TB->DP cable to be delivered so I can try your solution in full.

That being said, while it still faults sometimes, stability has improved greatly. Once I either hot-plug the system while its running or the dock crashes and recovers, I don't experience any more faults until the next cold boot.

Checking in to say that since disabling MST, the displays are now working flawlessly. Driving the dock with one DP->DP and one TB3->DP cable for two screens and it works.

Thank you for your research!

@mark.blakeney It seems that i915.enable_dp_mst=0 really helps but it looks there is an additional issue. I have found that if I am using USB Type C Hub then I still see this issue even with mst disabled. I tested it with HP USB-C Hub and default Dell USB-C to USB-A adapter. If I plug any of these adapters into a laptop or docking station everything is working really unstable. At the same time, it looks like (though I didn't test it for a long time) 2 thunderbolt stations could be connected to the laptop without any issue.

Unfortunately, I reported success too early. When yesterday my setup (mst disabled and no USB hub connected) worked stable today I again faced random faults. I know that my docking station is not WD19TB but I still believe that there is a common issue affecting different HW.

FYI, Dell released a new BIOS 1.3.0 today for the XPS 13 9300 which lists fixes against the WD19TB dock. I installed it then removed my kernel option and put my screens back on dual DP but the screen blackouts and disconnects immediately returned. :(

I think the DP MST problem is within the Linux kernel (i915?) with Ice Lake & later architectures.