Fix GL_DRAW_COOKIE_INVALID error with QEMU/virtio-gpu on Apple Silicon
On Asahi Linux (Linux for Apple Silicon, both the Arch and Fedora based versions) trying to use Spice+OpenGL with virtio-gpu with QEMU crashes with a GL_DRAW_COOKIE_INVALID error.
It seems the UTM (emulator for iOS) folks ran into this same issue last year as well, and patched it by simply removing the check in spice_qxl_gl_scanout
(see attached patch below). I have tried this patch on both Arch and Fedora and it solved the problem, my VMs are now properly GL accelerated.
In this RedHat bug - https://bugzilla.redhat.com/show_bug.cgi?id=1674324 - they discuss the same crash, though their solution is different and it seems they reason it happens because of 2D calls being translated to 3D.
From 02cae4922d55c488cce790835ee2fccd61e34dbd Mon Sep 17 00:00:00 2001
From: osy <osy@turing.llc>
Date: Sun, 18 Dec 2022 23:20:48 -0800
Subject: [PATCH 11/11] red-qxl: remove cookie assertion on scanout
The original check ensures we do not have an outstanding GL_DRAW. However,
in QEMU, there is no guarantee that a scanout cannot happen while the async
GL_DRAW has not returned a result yet. This introduces a race where if a
scanout is called while there is an outstanding GL_DRAW, QEMU will crash.
The removal of this check enforces a new contract with the SPICE client.
Every GL_DRAW must be matched with a GL_DONE (i.e. with a call to
`spice_display_channel_gl_draw_done()` in the client) even if the render
context is no longer valid. Otherwise, the assertion crash will happen in
the next call to `spice_qxl_gl_draw_async()`.
---
server/red-qxl.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/server/red-qxl.c b/server/red-qxl.c
index dbfcd440..8cdf86f5 100644
--- a/server/red-qxl.c
+++ b/server/red-qxl.c
@@ -703,7 +703,6 @@ void spice_qxl_gl_scanout(QXLInstance *qxl,
spice_return_if_fail(qxl != NULL);
QXLState *qxl_state = qxl->st;
- spice_return_if_fail(qxl_state->gl_draw_cookie == GL_DRAW_COOKIE_INVALID);
pthread_mutex_lock(&qxl_state->scanout_mutex);