subparse: problem with seeking video subtitiles
GStreamer 1.18.2
I am describing two bugs here, the more serious one only applies on Mac (I'm testing on Big Sur) while the less serious one applies everywhere (but you would usually not notice it).
I find that subparse on Mac Big Sur (and probably earlier Mac versions?) gets confused by seeking. I attach a simple subtitle file Test.srt in "subrip" format, obviously you can put these subtitles on whatever video you might have at hand.
The pipeline looks like this:
gst-launch-1.0
textoverlay name=ov ! autovideosink
filesrc location=my-video.mp4 ! decodebin ! videoconvert ! videoscale ! ov.video_sink
filesrc location=Test.srt ! subparse ! ov.text_sink
but of course I am doing this programmatically and this pipeline works fine if you don't "seek" it. And I don't think it's possible to seek with gst-launch?
However if you programmatically seek this pipeline to 8 seconds with GST_DEBUG=subparse:7 then subparse produces errors. I have attached a file subparse_log.txt showing the crucial lines.
The crucial lines from the source are these, at line 1060 in the function parse_subrip in gstsubparse.c, dealing with "state 2" (expecting subtitle text):
if (in_seg) {
state->start_time = clip_start;
state->duration = clip_stop - clip_start;
} else {
state->state = 0;
return NULL;
}
That is, if we are out of segment (parsing lines before the ones we are interested in) then throw away the subtitle text and transition immediately to state 0 (expecting sequence number). IMHO this is wrong, the next thing we are in fact going to see is either another line of subtitle text or a blank line. I'll call this the segment problem and it applies everywhere.
The problem is then compounded by the fact that in state 0 the parser accepts almost anything - even a blank line - as a valid sequence number, and transitions to state 1 (expecting timestamps). I'll call this the Mac strtoull problem and it only applies on Mac, because the reason is that this code:
const gchar *nptr = ""; gchar *endptr; errno = 0; guint64 res = g_ascii_strtoull(nptr, &endptr, 10);
on Windows & Linux does not set errno, but on Mac it sets errno to 22 (EINVAL).
Therefore in "case 0" of the parser on Mac, this line applies:
else if (id == 0 && errno == EINVAL) state->state = 1;
These two factors cause the parsing errors to cascade, often destroying the first 2 or 3 subtitles that we did want to see.
Looking at the log I've attached, we see the segment event, start time 8 secs, and then:
State 0. Parsing line '1'
State 1. Parsing line '00:00:01,000 --> 00:00:05,000'
parse_subrip_time: parsing timestamp '00:00:01,000'
parse_subrip_time: parsing timestamp '00:00:05,000'
State 2. Parsing line 'Test message 1'
// segment problem: at this point we transition to state 0
// which is wrong - we should still be in state 2,
// waiting for blank line.
State 0. Parsing line ''
// Mac strtoull problem: here we wrongly transition to
// state 1 because the blank line we just saw has
// been wrongly accepted as a valid sequence number.
State 1. Parsing line '2'
error parsing subrip time line '2'
State 0. Parsing line '00:00:07,000 --> 00:00:12,000'
// This is not accepted as a sequence number because it
// starts with a digit, so does not set errno.
// So we are still in state 0.
State 0. Parsing line 'Another test message'
// Mac strtoull problem: however this is accepted as a
// sequence number because it does set errno!
// so we transition to state 1.
State 1. Parsing line 'on two lines'
error parsing subrip time line 'on two lines'
It seems to me that two fixes are needed:
-
Segment problem: The parser should only transition from state 2 to state 0 when it sees a blank line.
-
Mac strtoull problem: In order to re-synchronise after any parse error, it should only transition from state 0 to state 1 when it sees a line with a single decimal number on it.
The reason why this has not been noticed on Windows or Linux is because these have the segment error only, and that is far less serious on its own, in fact I think it would only report a parse error if it saw an out-of-segment subtitle like this:
00:00:07,000 --> 00:00:12,000
I think we first met in the year...
2005
where it would parse "2005" erroneously as a sequence number and then report a parse error on the next line (which is blank) where it would be expecting to see timestamps.
Can anyone suggest a workaround?
My Humax TV hard disk recorder shows the same symptoms : after a seek, it is often the case that several subtitles go missing before they get back in sync. I wonder why!