Skip to content

Rework BGRA resources handling

Ryan Neph requested to merge ryanneph/virglrenderer:bgra-rework into master

After observing some failure cases in the current BGRA emulation code, I took the opportunity to investigate further to see if anything could be simplified.

In doing so, I discovered that the use of glTextureView() on any eglimage-backed BGR* resource will result in an unintended red/blue channel swap. This is because these eglimage-backed resources are allocated via minigbm with BGR data ordering (e.g. DRM_FORMAT_A8R8G8B8) and imported into the host driver through a series of operations following dmabuf-->eglimage-->glTexture.

Conversely, any textures created through the normal glTexImage*D() or glTexStorage*D() methods are allocated with RGB* data ordering, by our request according to the internalformat we define in vrend_formats.c. For these "internal" textures, in order to maintain maximum compatibility across GL and GLES hosts, especially for the use case where a guest GL app runs on a host GLES driver, maintaining the GL_RGBA8 byte order is critical. Many things break if we deviate from this, such as multisampled bgra resources.

With vrend's current format organization architecture, whenever we create a glTextureView() of one of these eglimage-backed bgr* resources, we end up applying a view with internalformat GL_RGBA8 onto a texture that the host driver considers to already have the BGR order, causing the host driver to perform a red/blue swizzle internally for every read/write on such a resource. One might imagine we can add some special cases around the glTextureView() calls to modify the internalformat passed whenever an eglimage-backed bgr* resource is involved, but that begs the question: what internalformat do we choose in such cases? There is no GL_BGRA8 internal format or equivalent for the SRGB case. This leads me to conclude that texture views on eglimage-backed bgr* resources are simply unsupported.

This leads us to question how we can work around the issue. In this MR I have proposed one possible approach that is still a bit laden with special cases, but the motivations for each one are much stronger than in my previous proposal in !539 (merged). It is very likely that this proposal can be cleaned up further now that the situation is well understood.

  • commits 1-2 are simple cleanups
  • 3-12 remove uses of glTextureView() on eglimage-backed bgr* resources and add workarounds to fill the gaps
  • 13-16 fix the gbm format mapping and apply CPU data reordering to ALL bgr* resources, only on GLES hosts
  • 17-18 fixes glClearTexSubImageEXT() for ALL bgr* resources, only on GLES hosts, and enable the test in CI.

Merge request reports