webrtcbin: video pixelated

Submitted by Dmitriy

Link to original bug (#797280)

Description

Hello.

I using gstwebrtc-demos for experiments.
I no modifications this demo, video are chronically pixelated.

My run demo on Intel Core i5-2300. Tested on Fedora 28 and AltLinux 7.
GStreamer 1.14.1-1.14.4 and git master branch.

On localhost for vp8:
chrome -> gstreamer video pixeleted after few seconds and not restoring. FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out: Sub-class should implement drain()
firefox -> gstreamer not pixelated.

Via internet for vp8:
chrome -> gstreamer video pixeleted after few seconds and not restoring. FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out: Sub-class should implement drain()
firefox -> gstreamer pixelated periodical and restoring. FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out: Sub-class should implement drain()

On localhost for vp9:
chrome -> gstreamer video not pixeleted, after one minute decode error. Logged vp9dec0 Failed to decode frame
firefox -> gstreamer not pixelated.

Via internet for vp9:
chrome -> gstreamer video not pixeleted, after one minute decode error. Logged vp9dec0 Failed to decode frame
firefox -> gstreamer pixelated periodical and restoring. FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out: Sub-class should implement drain()

On localhost for h264:
chrome -> gstreamer not working.

Via wi-fi for h264:
ios safari -> gstreamer pixelated periodical and restoring.

I modified demo for transfer video hd 720p. I took 720p video clip (~ 2000 kbit/s), split the video and audio into different files.
Replace videotestsrc on filesrc ! decodebin.

On localhost:
chrome -> gstremer video pixeleted after few seconds and not restoring.
gstreamer -> chrome video from file normally reproduced in good quality and not pixelated. Periodically, the video freezes(slows down), but continues to play.

I modified demo for connection gstreamer <-> gstreamer and show packets-lost statistics.

On localhost on CPU Intel Core i5-2300 both videos pixelated.
On localhost on CPU Intel Core i7-7500U not pixelated, video can play for hours. But sometimes it is suddenly pixelated and not restored.

Via internet video starts to pixelate even without packet loss.
Without vaapi logged:
FIXME videodecoder gstvideodecoder.c:933:gst_video_decoder_drain_out: Sub-class should implement drain()

For vaapi logged:
0:01:05.094888406 31361 0x164a190 WARN basesink gstbasesink.c:3008:gst_base_sink_is_too_late:<sink> warning: A lot of buffers are being dropped.
0:01:05.094911959 31361 0x164a190 WARN basesink gstbasesink.c:3008:gst_base_sink_is_too_late:<sink> warning: There may be a timestamping problem, or this computer is too slow.
0:01:09.812852771 31361 0x164a190 WARN basesink gstbasesink.c:3008:gst_base_sink_is_too_late:<sink> warning: A lot of buffers are being dropped.
0:01:09.812895157 31361 0x164a190 WARN basesink gstbasesink.c:3008:gst_base_sink_is_too_late:<sink> warning: There may be a timestamping problem, or this computer is too slow.

or

0:00:46.437781309 30576 0x7fcdbc086a80 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:21.974949098 too big, reset skew
0:00:54.432961894 30576 0x7fcdbc086a80 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:07.929154280 too big, reset skew
0:00:55.091775442 30576 0x7fcdbc086a80 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.005567476 too big, reset skew
0:00:55.751770789 30576 0x7fcdbc086a80 WARN rtpjitterbuffer rtpjitterbuffer.c:570:calculate_skew: delta - skew: 0:00:01.006104407 too big, reset skew
0:00:56.032690630 30576 0x7fcd8c0044a0 WARN videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<vp8dec0> decreasing timestamp (0:00:46.420374279 < 0:00:47.392941755)
0:00:56.035278829 30576 0x7fcd8c0044a0 WARN videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<vp8dec0> decreasing timestamp (0:00:46.454371755 < 0:00:47.392941755)
0:00:56.037620125 30576 0x7fcd8c0044a0 WARN videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<vp8dec0> decreasing timestamp (0:00:46.487322322 < 0:00:47.392941755)
0:00:56.039805150 30576 0x7fcd8c0044a0 WARN videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<vp8dec0> decreasing timestamp (0:00:46.520118718 < 0:00:47.392941755)
0:00:56.042165373 30576 0x7fcd8c0044a0 WARN videodecoder gstvideodecoder.c:2780:gst_video_decoder_prepare_finish_frame:<vp8dec0> decreasing timestamp (0:00:46.553719923 < 0:00:47.392941755)

I tried to minimize the load on the processor. I check gstreamer(h264) <-> gstreamer(h264) via internet using vaapi for decoder and encoder.
Video is almost not pixelated. Only sometimes and immediately restored. Packet loss is about the same as in previous tests.

How to make the video just slow down when there is a shortage of computer resources and then continued to play?