[CI][IGT] igt@* - dmesg-warn ERROR Link Training Unsuccessful

added CI feature: display/DP platform: BXT platform: ILK platform: KBL priority::high severity::normal + 1 deleted label

Marta Löfstedt @marta said:

FYI the old bugs are bug 102359 and bug 103287

Marta Löfstedt @marta said:

note that this is related to bug 103313

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Since, this is a highly unstable issue, it is dangerous to draw any conclusion. However, from IRC:
<marta> tsa, so far all new linktraining tests I have filed from CI_DRM_3311 and forward has been on KBL6 and KBL1, is there anything special about them?
<tsa> nothing that I know. Same FW, same KVM switch type (but diff KVM for 1 and 6)

Marta Löfstedt @marta said:

More from IRC #intel-gfx:

<marta> shashank_sharma, ^^ could you comment if different KVM could affect the frequency of the link-training on KBL?
...
<tsa> marta: but they are the same kvm model, just that we need 2 to connect 7 hosts
<tsa> same kind of hdmi dummy attached, even
<shashank_sharma> marta: marta, there is a fair possibility, as KVM acts as a sink for LSPCON/HDMI devices
<shashank_sharma> marta: in fact while doing some experiments on LSPCON + KVM tsa and jsaa_ also observed some difference in behavior, to the good or the bad side
marta> shashank_sharma, we still have to support all setups, but maybe this is a lead to get yet another LSPCON FW upgrade?
<shashank_sharma> marta: I would agree as a long term goal, but currently focus should be on direct link training failures, KVM would be an added variable
<jsaa> marta: kvmäs used are http://www.aten.com/global/en/products/kvm/desktop-kvm-switches/cs1794/#.WgVv_dNL9aQ
<shashank_sharma> marta: are these boards RVPs or NUCs ?
<tsa> NUCs
<tsa> KBL7i5NUC, full height model
<tsa> exact model NUC7i5BNH
<shashank_sharma> tsa/marta: As per MCA, there is a Phy tuning binary, which should be provided with every MCA LSPCON. If that binary is not configured properly, there can be random Link training issues
<shashank_sharma> As per them, during commercialization, these binaries are flashed as per HW
<shashank_sharma> As the NUC devices come with pre-installed windows, I would assume they are with proper PHY tuning binaries
<shashank_sharma> so I was hoping for an answer "RVP" but you broke my heart :P
<jsaa> shashank_sharma: NUC reliable for CI that is why mojority those.
<jsaa> majority
<shashank_sharma> agree !
<jsaa> but many others: https://intel-gfx-ci.01.org/hardware.html
<shashank_sharma> jsaa: But looks like the link training failures are on NUCs only
<shashank_sharma> so need to talk to MCA again on these errors
<shashank_sharma> interestingly, I have same model kbl NUC NUC7i5BNH and I could not see any LT issues

Shashank Sharma @contactshashanksharma said:

The latest reports indicates some different problem all-together. For this particular KBL all the tests which can do link training are failing. This seems like some HW failure !

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Just one new this morning, but maybe we are covering all with this and other bugs?

KBL2:
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3342/shard-kbl2/igt@gem_exec_nop@basic-sequential.html

Anyways, previous ideas that the linktraining issue was limited to specific machines have been disproven.

Hector Velazquez said:

This test have the same dmesg-warn on CFL QA
with IGT-Version: 1.20-g7d75119 (x86_64) (Linux: 4.14.0-rc8-drm-tip-ww45-commit-1342299+ x86_64)

test list:
igt@kms_plane_multiple@atomic-pipe-b-tiling-x
igt@kms_plane_multiple@atomic-pipe-b-tiling-y
igt@kms_plane_multiple@atomic-pipe-b-tiling-yf
igt@kms_plane_multiple@legacy-pipe-b-tiling-x
igt@kms_plane_multiple@legacy-pipe-b-tiling-y
igt@kms_plane_multiple@legacy-pipe-b-tiling-yf
igt@kms_plane_multiple@atomic-pipe-b-tiling-none
igt@kms_properties@plane-properties-legacy
igt@pm_rpm@modeset-non-lpsp-stress
igt@pm_rpm@modeset-non-lpsp-stress-no-wait
igt@pm_rpm@fences-dpms

dmesg-warn:
[ 5784.225188] [drm:intel_dp_get_link_train_fallback_values [i915]] ERROR Link Training Unsuccessful

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

DRM_3361 kbl7:

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3361/shard-kbl7/igt@kms_cursor_crc@cursor-128x128-sliding.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3361/shard-kbl7/igt@kms_flip_tiling@flip-changes-tiling-y.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3361/shard-kbl7/igt@kms_atomic@crtc_invalid_params.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3361/shard-kbl7/igt@pm_rpm@gem-execbuf-stress.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Only 2 new subtests this morning CI_DRM_3369 KBL7, but if you check:
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3369/shards.html
there is a lot more, but that is already covered by this or other bugs.

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3369/shard-kbl7/igt@kms_cursor_legacy@all-pipes-forked-bo.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3369/shard-kbl7/igt@kms_fbcon_fbt@fbc-suspend.html

Marta Löfstedt @marta said:

new subtests on CI_DRM_3371 KBL4

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@gem_ctx_bad_destroy@invalid-default-ctx.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@gem_mmap_gtt@basic-copy.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@drv_hangman@error-state-capture-render.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@prime_self_import@reimport-vs-gem_close-race.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@kms_pipe_crc_basic@nonblocking-crc-pipe-c.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3371/shard-kbl4/igt@kms_cursor_legacy@cursor-vs-flip-legacy.html

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4006/shard-kbl3/igt@kms_frontbuffer_tracking@fbc-rgb101010-draw-mmap-wc.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4006/shard-kbl3/igt@kms_plane_multiple@atomic-pipe-c-tiling-none.html

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4007/shard-kbl2/igt@kms_chv_cursor_fail@pipe-b-128x128-left-edge.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4007/shard-kbl2/igt@perf_pmu@init-busy-vcs0.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

The 4.15.0-rc1 fire is now under control so I will filing new linktraining issues here again:

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3415/shard-kbl2/igt@kms_color@ctm-0-5-pipe0.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3415/shard-kbl2/igt@drv_selftest@live_guc.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3415/shard-kbl2/igt@drv_selftest@live_contexts.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4027/shard-kbl2/igt@kms_universal_plane@universal-plane-pipe-c-sanity.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3446/shard-kbl3/igt@kms_flip@flip-vs-fences.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3446/shard-kbl3/igt@kms_cursor_crc@cursor-256x85-sliding.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3446/shard-kbl3/igt@kms_universal_plane@disable-primary-vs-flip-pipe-a.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3446/shard-kbl3/igt@prime_busy@wait-hang-bsd.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3446/shard-kbl3/igt@kms_atomic_interruptible@legacy-pageflip.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Shashank Sharma @contactshashanksharma uploaded an attachment:

Hello,
I am on vacation between 06 Dec - 16 Dec with limited Phone and no mail access.
Please expect delay in response.

If this is something urgent, please contact my manager : Indranil, Mukherjee

Regards
Shashank

~~Attachment 136031~~, "attachment-29094-0.html":
attachment-29094-0.html

Shashank Sharma @contactshashanksharma said:

(In reply to shashank.sharma@intel.com from comment 27)

Created attachment 136031 [details]
attachment-29094-0.html

Hello,
I am on vacation between 06 Dec - 16 Dec with limited Phone and no mail
access.
Please expect delay in response.

If this is something urgent, please contact my manager : Indranil, Mukherjee

Regards
Shashank

Please ignore, wrong OOO message :) !

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3478/shard-kbl1/igt@perf_pmu@event-wait-rcs0.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3476/shard-kbl6/igt@gem_caching@reads.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3476/shard-kbl6/igt@gem_workarounds@basic-read-context.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3476/shard-kbl6/igt@kms_plane@plane-position-hole-dpms-pipe-b-planes.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4056/shard-kbl7/igt@kms_plane@plane-panning-top-left-pipe-a-planes.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4056/shard-kbl7/igt@kms_vblank@query-forked-busy.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4056/shard-kbl7/igt@kms_cursor_legacy@cursor-vs-flip-varying-size.html

Marta Löfstedt @marta said:

Not much this morning maybe we are running out of tests to fail on:
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4057/shard-kbl7/igt@kms_flip@flip-vs-fences-interruptible.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4057/shard-kbl7/igt@kms_cursor_legacy@cursora-vs-flipa-toggle.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3499/shard-kbl3/igt@kms_cursor_crc@cursor-64x21-onscreen.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3499/shard-kbl3/igt@gem_wait@invalid-buf.html
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3499/shard-kbl3/igt@drm_read@short-buffer-block.html

Marta Löfstedt @marta said:

https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4059/shard-kbl7/igt@kms_busy@extended-pageflip-hang-oldfb-render-b.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4059/shard-kbl7/igt@kms_flip@bo-too-big-interruptible.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4058/shard-kbl7/igt@gem_exec_params@rs-invalid-on-blt-ring.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4058/shard-kbl7/igt@core_get_client_auth@simple.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4058/shard-kbl7/igt@gem_mmap_wc@copy.html
https://intel-gfx-ci.01.org/tree/drm-tip/IGT_4058/shard-kbl7/igt@kms_ccs@pipe-b-bad-aux-stride.html

Marta Löfstedt @marta said:

only one new this morning:
https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3501/shard-kbl4/igt@kms_ccs@pipe-b-ccs-on-another-bo.html

Marta Löfstedt @marta said:

Marta Löfstedt @marta said:

Eero Tamminen @eero-t said:

We started seeing link training errors in dmesg on our perf lab last week (16th onwards), on KBL GT3e NUC which is connected to KVM, but not on KBL GT2 NUC, which isn't connected to KVM.

This happened on all tracks, Mesa, X, kernel, so it seems to have been triggered by something else than kernel change (firmware?).

KVM connector is DVI, which is connected to KBL i7 7567U HDMI output through HDMI->DVI converter:
----------------------------------
$ sudo cat /sys/kernel/debug/dri/0/i915_display_info
...
connector 47: type DP-1, status: connected
name:
physical dimensions: 530x300mm
subpixel order: Unknown
CEA rev: 3
DPCD rev: 12
audio support: yes
DP branch device present: yes
Type: HDMI
ID: MC2800�s{�
HW: 2.2
SW: 1.66
Max TMDS clock: 600000 kHz
Max bpc: 16
----------------------------------

Other (non-KBL) devices connected to the same KVM with similar converter don't show same issue, just KBL.

Shashank Sharma @contactshashanksharma said:

From the connector status, I can see:
DP branch device present: yes
Type: HDMI
ID: MC2800�s{�
HW: 2.2
-> SW: 1.66 (This is Old LSPCON FW)

LSPCON FW needs to be V1.72, to get rid of most(if not all) Link training issues