amcviddec: slow performance on some android devices

Hello together,

my report might not be a real bug. Maybe it's just a configuration issue. Nevertheless i could not get any help from posting at http://gstreamer-devel.966125.n4.nabble.com/Bad-performance-of-H264-stream-playback-on-android-tc4689899.html. So i would like to try it here.

I face the problem that a h264 video stream works fine on my LG G4, but very poor on my HTC u11 and Samsung Galaxy S7.

I used tutorial-3 from the android examples and modified it to show a H264 rtp stream. I used the following pipeline on android side:

data->pipeline = gst_parse_launch("udpsrc port=5000 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H264 ! decodebin ! 
autovideosink", &error); 

For sending a h264 stream I use the following command on a Windows 7 machine:

gst-launch-1.0 -v ksvideosrc do-timestamp=true ! videoconvert ! x264enc tune=zerolatency ! "video/x-h264,profile=baseline,width=1280,height=720,framerate=30/1,bitrate=300" ! rtph264pay config-interval=1 ! udpsink host=192.168.8.104 port=5000 sync=false

The only difference is on which device the android application is running. It's the same network setup etc. I am using GStreamer 1.15.1. First I though it is due to the reason that the Samsung device is based on an Exynos SOC, which is using some amcviddec-omxexynosavcdec decoder, while the LG G4 is based on Snapdragon using some amcviddec-omxqcomvideodecoderavc. Then I bought a HTC u11, which also has a Snapdragon SOC like the G4 and it's decoder is even named exactly like on the LG G4 (amcviddec-omxqcomvideodecoderavc).

But in case of the HTC I realized that the amc hardware decoders are still not used. It fails with some message "amcvideodec-omxqcomvideodecoderavc0:sink> pad peer query failed". Instead avdec_h264 was suggested by decodebin like on the Samsung Galays S7. Additionally I got the following errors on startup of tutorial-3 complaining about some unkown color_formats:

W/VideoCapabilities: Unrecognized profile 2130706434 for video/avc 
E/GStreamer+amc: 0:00:00.100972812 0x7fa1fdb000 
../sys/androidmedia/gstamc.c:2069:accepted_color_formats Unknown color 
format 0x7fa30c06, ignoring 
W/GStreamer+amc: 0:00:00.100993906 0x7fa1fdb000 
../sys/androidmedia/gstamc.c:1732:scan_codecs video/avc decoder has unknown 
color formats, only direct rendering will be supported 

This was surprising me since the decoders are named the same and I do not get this error on LG G4. Then I verified with the help of Android Media API, that the HTC supports exactly the same color formats than the LG G4. Except that the HTC is supporting some more color formats, which are NOT known to GStreamer.

To force gstreamer using amc hardware decoders I modified inside tutorial-3 project the decoder ranks with some method(1) and set the environment variable GST_AMC_IGNORE_UNKNOWN_COLOR_FORMATS(2) suppressing this "unknown color stuff":

(1)

static void enable_factory (const gchar *name, gboolean enable) { 
    GstRegistry *registry = NULL; 
    GstElementFactory *factory = NULL; 
    GstRegistry* reg = gst_registry_get(); 
    if (!reg) 
    { 
        GST_DEBUG("No registry found for %s", name); 
        return; 
    } 
    GstPluginFeature* feat = gst_registry_lookup_feature(reg, name); 
    if(feat == NULL) { 
        GST_DEBUG("No feat found for %s", name); 
        return; 
    } 
    if (enable) { 
        GST_DEBUG("RAISE rank %s", name); 
        gst_plugin_feature_set_rank(feat, GST_RANK_PRIMARY + 1); 
    } 
    else { 
        GST_DEBUG("Disable plugin %s", name); 
        gst_plugin_feature_set_rank(feat, GST_RANK_NONE); 
    } 
    gst_object_unref(feat); 
    return; 
} 

(2):

 static { 

        try { 
            Os.setenv("GST_AMC_IGNORE_UNKNOWN_COLOR_FORMATS", "yes", true); 
            //Os.setenv("GST_DEBUG", "5", true); 
            Os.setenv("GST_DEBUG_DUMP_DOT_DIR", 
"/data/data/org.freedesktop.gstreamer.tutorials.tutorial_3/files", true); 
        } catch (ErrnoException e) { 
            e.printStackTrace(); 
        } 

        System.loadLibrary("gstreamer_android"); 
        System.loadLibrary("tutorial-3"); 
        nativeClassInit(); 
    } 

In the end I could achieve that HTC u11 and Samsung Galays S7 are building up the same pipeline like the LG G4 (only with decoder ranks modification and env variable). The pipeline itself is still determined by decodebin. I verified this by comparing both pipelines by generating the corresponding visual representations (dot files in attachment).

Nevertheless the HTC and Samsung show very poor performance. I can observe a delay of 1-2 seconds and a lots of artifacts during playback and shuttering. Playback is not smooth at all.

I can attach logs of the tutorial 3 android app without modification of my rank and env variable. I thought that the error might be more obvious there, because gstreamer decides in this case on its own to prefer software decoding: https://www.dropbox.com/s/daq4x13zyhi2vtt/htc_u11full.txt?dl=0

Thank you already very much in advance. As you might see I tried already a lot, but sadly without having success...