Skip to content

Xnest: fix user specified color depth and memleak

Retrieving visuals offered by upstream Xserver is broken in several ways:

a) duplicate elimination breaks out too fast: when a duplicate is found, it doesn't just skips that one, it completely breaks out the loop, so subsequent upstream visuals aren't considered anymore. that's leading to (unpredictable) limit on available color depths (depending on the order reported by upstream sever)

b) buffer overflow when user specificed different depth/class than default one: xnestOpenScreen() looks into the wrong table: it's local visuals[] array, instead of the global (non-dedup'ed) list fetched by xlib. The visuals[] array is much smaller (deduplicated) than the xnestVisuals[] array, and xnestDefaultVisualIndex is likely to point outside of visual[]'s bounds.

To make it actually work against an Xorg upstream server, the upstream server needs fix for another bug in the DIX:

https://gitlab.freedesktop.org/xorg/xserver/-/issues/1741
https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/1644

Signed-off-by: Enrico Weigelt, metux IT consult info@metux.net

Merge request reports