gstv4l2videodec.c: apparent incorrect processing (?) of frames in gst_v4l2_video_dec_loop(decoder) after 2020.10.19 commit
The cause of the symptoms described in issue #1103 (closed) appears to be a possible code error introduced in @slomo 's commit bcb3428e
Before that 19 Oct 2020 commit, the oldest frame in the buffer was processed in the video decryption loop. After then the frame was chosen as the one with frame number stored as GST_BUFFER_TIMESTAMP (buffer) /GST_SECOND.
However, if the first call to the loop is after a codec event (AU formed by a VCL NAL preceded by SPS PPS SEI etc) the loop is exited early at
ret = gst_buffer_pool_acquire_buffer (pool, &buffer, NULL);
g_object_unref (pool);
if (ret == GST_V4L2_FLOW_RESOLUTION_CHANGE) {
GST_WARNING_OBJECT (decoder, "Received resolution change");
g_atomic_int_set (&self->capture_configuration_change, TRUE);
return;
}
This return introduces delays so that at the next call to the loop,
GST_BUFFER_TIMESTAMP(buffer) is ahead of the frames in the buffer, and two or three
frames are orphaned (and later picked up by the Garbage collection when they are 100 frames too late, with an incorrect message telling the user to report a codec error, repeated for each orphan frame)
If extras delays are introduced by adding code before the return to see what is going on,
even more frames are lost in this way.
surely the correct behavior is to process the oldest frame in the buffer, provided its frame number is not too far behind (100 behind maybe?) the frame number stored in GST_BUFFER_TIMESTAMP(buffer) ?
see #1103 (closed) for a test file (20 secs of BBB, h264 video only, as processed through Apple's AirPlay) and pipeline that demonstrates the issue.