xe_oa: exec_queue unusable after closing OA stream

Mesa is reporting the following: if OA metrics are collected for an exec_queue, after the OA stream is closed, future batch buffers submitted on the exec_queue do not complete. KMD sees this hang and resets the GPU. This is being seen on Xe2+ platforms. The hangs are not seen if OA stream is not closed.

Cc: @zehortigoza, @fjdegroo, @aswarup

added feature: Perf/OA platform: BMG platform: LNL labels

assigned to @adixit

diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index 6d17f89a132d2..7eaa0d164c193 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -774,7 +774,8 @@ static int xe_oa_configure_oar_context(struct xe_oa_stream *stream, bool enable)
        /* Modify stream hwe context image with regs_context */
        err = xe_oa_modify_ctx_image(stream, stream->exec_q->lrc[0],
-                                    regs_context, ARRAY_SIZE(regs_context));
+                                    regs_context,
+                                    enable ? ARRAY_SIZE(regs_context) : 1);
        if (err)
                return err;
 

The change above fix the issue but I don't have any documentation backing it.

changed the description

To illustrate what @zehortigoza's patch above does, it is equivalent to the patch below (also @zehortigoza's earlier patch since @zehortigoza was investigating this issue):

diff --git a/drivers/gpu/drm/xe/xe_oa.c b/drivers/gpu/drm/xe/xe_oa.c
index a959722a66fd..9d5ed1b146d1 100644
--- a/drivers/gpu/drm/xe/xe_oa.c
+++ b/drivers/gpu/drm/xe/xe_oa.c
@@ -762,12 +762,14 @@ static int xe_oa_configure_oar_context(struct xe_oa_stream *stream, bool enable)
 			stream->oa->ctx_oactxctrl_offset[stream->hwe->class] + 1,
 			enable ? OA_COUNTER_RESUME : 0,
 		},
+/*
 		{
 			RING_CONTEXT_CONTROL(stream->hwe->mmio_base),
 			regs_offset + CTX_CONTEXT_CONTROL,
 			_MASKED_FIELD(CTX_CTRL_OAC_CONTEXT_ENABLE,
 				      enable ? CTX_CTRL_OAC_CONTEXT_ENABLE : 0)
 		},
+*/
 	};
 	struct xe_oa_reg reg_lri = { OAR_OACONTROL, oacontrol };
 	int err;

The code above is referring to Bit 8 in CTXT_SR_CTL register (Bspec: 60314). So when OA stream is opened, we write 1 to bit 8 and when OA stream is closed we write 0 to bit 8 (not directly to the register but to the context image). So it seems if we skip writing 0 when OA stream is closed, we don't see these hangs.

Specification mention about enabled this bit in golden image so maybe it should never moved from 1 -> 0 but I'm don't have any documentation backing it up.

I could not reproduce this using IGT but have sent out a test case to check for this here:

https://patchwork.freedesktop.org/series/138937/

Posted fix for this from @zehortigoza: https://patchwork.freedesktop.org/series/139063/

closed with commit 0c8650b0

mentioned in commit 8135f1c0

mentioned in commit drm/nouveau@0c8650b0