qmlglsink fails to display content on recent Android devices
Summary of the issue:
In trying to get a pipeline with a qmlglsink (qt5) instance on Android, I got blank content even though the pipeline (simple with videotestsrc ! glupload ! glcolorconvert ! qmlglsink) was fully functional and running (verified through buffer callbacks). Android logcat logs showed that the vertex shader code was not being properly linked with the following warnings:
...: QOpenGLShader::compile(Fragment): 3:1: S0032: no default precision defined for variable 'v_texcoord'
...: \*\*\* Problematic Fragment shader source code \*\*\*
...: #ifdef GL_KHR_blend_equation_advanced
...: #extension GL_ARB_fragment_coord_conventions : enable
...: #extension GL_KHR_blend_equation_advanced : enable
...: #endif ...: #ifndef GL_FRAGMENT_PRECISION_HIGH
...: #define highp mediump
...: #endif
...: #line 1
...: varying vec2 v_texcoord;
...: uniform sampler2D tex;
...: uniform float opacity;
...: vec4 swizzle(vec4 texel, int components\[4\]) {
...: return vec4(texel\[components\[0\]\], texel\[components\[1\]\], texel\[components\[2\]\], texel\[components\[3\]\]);
...: }
...: vec3 swizzle(vec3 texel, int components\[3\]) {
...: return vec3(texel\[components\[0\]\], texel\[components\[1\]\], texel\[components\[2\]\]);
...: }
...: vec2 swizzle(vec2 texel, int components\[2\]) {
...: return vec2(texel\[components\[0\]\], texel\[components\[1\]\]);
...: }
...: vec2 swizzle2(vec3 texel, int components\[3\]) {
...: return vec2(texel\[components\[0\]\], texel\[components\[1\]\]);
...: }
...: vec2 swizzle2(vec4 texel, int components\[4\]) {
...: return vec2(texel\[components\[0\]\], texel\[components\[1\]\]);
...: }
...: vec3 swizzle3(vec4 texel, int components\[4\]) {
...: return vec3(texel\[components\[0
...: shader compilation failed:
...: "3:1: S0032: no default precision defined for variable 'v_texcoord'"
...: QOpenGLShader::link: Link failed because of missing fragment shader.
...: QOpenGLShaderProgram::uniformLocation(u_transformation): shader program is not linked
...: QOpenGLShaderProgram::uniformLocation(opacity): shader program is not linked
...: QOpenGLShaderProgram::uniformLocation(swizzle_components): shader program is not linked
...: QOpenGLShaderProgram::uniformLocation(tex): shader program is not linked
...: QOpenGLShaderProgram::uniformLocation(yuv_offset): shader program is not linked
Expected Behavior
Expected behavior was to see the pipeline output on the Qt widget, which is what was observed on the desktop Linux environment
Observed Behavior
Pipeline was in running state, all qmlglsink function were being invoked up to the QtGLVideoItemInterface::setBuffer, verified through various debugging log additions. The update methods was also calling QtGLVideoItem::updatePaintNode, which was frorming the texture and sending it back to Qt. Android logcat, however, showed the above warning
Setup
This was with the qmlglsink compiled as per the README.md in gst-plugins-good/ext/qt with Android NDK 27.0.11718014 and open-source Qt 5.15.14. The Android code properly instantiates a GstGLVideoItem and the same coder works in Linux. Gstreamer compilation was done through cerbero with its latest version. Qt compilation detects EGL 2.0 for their android compilation. Android device was an up-to-date Samsung Galaxy S20+.
Steps to reproduce the bug
In my setup, the bug was universally observed without an yexpcetions.
Solutions you have tried
The final solution ended up adding a default precision to the vertex shader code in gstqsgmaterial.cc as follows:
Change:
#define texcoord_input \
"varying mediump vec2 v_texcoord;\\n"
to
#define texcoord_input \
"precision mediump float;\\n" \
"varying mediump vec2 v_texcoord;\\n"
which solves the issue.