audiomixer producing GST_BUFFER_FLAG_GAP
I have this pipeline doing audiomixing which works fine:
GST_DEBUG="*:3,GST_TRACER:7" GST_TRACERS="buffer(filter=audioconvert)" gst-launch-1.0 autoaudiosrc is-live=true provide-clock=false ! "audio/x-raw,format=S32LE,layout=interleaved,rate=44100,channels=2,channel-mask=(bitmask)0x3" ! deinterleave name=d d.src_0 ! tee name=ch0_audio d.src_1 ! tee name=ch1_audio interleave name=ilv ! "audio/x-raw,format=S32LE,layout=interleaved,rate=44100,channels=2,channel-mask=(bitmask)0x3" ! queue ! audioconvert ! autoaudiosink audiomixer start-time-selection=zero name=leftamix sink_0::volume=0 sink_1::volume=0 ! "audio/x-raw,channels=1,channel-mask=(bitmask)0x1" ! queue ! ilv.sink_0 audiomixer start-time-selection=zero name=rightamix sink_0::volume=1 sink_1::volume=1 ! "audio/x-raw,channels=1,channel-mask=(bitmask)0x2" ! queue ! ilv.sink_1 ch0_audio. ! queue ! leftamix. ch0_audio. ! queue ! rightamix. ch1_audio. ! queue ! leftamix. ch1_audio. ! queue ! rightamix.
However as soon as I add interpipesink/src
(from RidgeRun, based on appsrc/sink
) in between it stops working: no sound and all buffers seem to be marked with GST_BUFFER_FLAG_GAP
GST_BUFFER_FLAG_GAP (2048) – the buffer has been created to fill a gap in the stream and contains media neutral data (elements can switch to optimized code path that ignores the buffer content).
So, it seems like the mixer is discarding the frames.
GST_DEBUG="*:3,GST_TRACER:7" GST_TRACERS="buffer(filter=audioconvert)" gst-launch-1.0 autoaudiosrc is-live=true provide-clock=false ! "audio/x-raw,format=S32LE,layout=interleaved,rate=44100,channels=2,channel-mask=(bitmask)0x3" ! interpipesink name=audio_src sync=false async=false interpipesrc listen-to=audio_src is-live=true stream-sync=passthrough-ts format=time ! deinterleave name=d d.src_0 ! tee name=ch0_audio d.src_1 ! tee name=ch1_audio interleave name=ilv ! "audio/x-raw,format=S32LE,layout=interleaved,rate=44100,channels=2,channel-mask=(bitmask)0x3" ! queue ! audioconvert ! autoaudiosink audiomixer start-time-selection=zero name=leftamix sink_0::volume=0 sink_1::volume=0 ! "audio/x-raw,channels=1,channel-mask=(bitmask)0x1" ! queue ! ilv.sink_0 audiomixer start-time-selection=zero name=rightamix sink_0::volume=1 sink_1::volume=1 ! "audio/x-raw,channels=1,channel-mask=(bitmask)0x2" ! queue ! ilv.sink_1 ch0_audio. ! queue ! leftamix. ch0_audio. ! queue ! rightamix. ch1_audio. ! queue ! leftamix. ch1_audio. ! queue ! rightamix.
Adding more debug aggregator:6 gives me this message: 0:00:00.834373796 12456 0x563d8cfec300 DEBUG audioaggregator gstaudioaggregator.c:1755:gst_audio_aggregator_fill_buffer:rightamix:sink_0 Buffer before segment or current position: 35721 < 35721 which comes from here: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/gst-libs/gst/audio/gstaudioaggregator.c#L1934
if (pad->priv->position >= pad->priv->size) {
/* Empty buffer, drop */
pad->priv->dropped += pad->priv->size;
pad->priv->position = 0;
pad->priv->size = 0;
GST_DEBUG_OBJECT (pad,
"Buffer before segment or current position: %" G_GUINT64_FORMAT
" < %" G_GINT64_FORMAT, end_output_offset, aagg->priv->offset);
return FALSE;
}
First I would say that the debug message doesn't really match what is happening, I would expect it to tell me something about pad->priv->position >= pad->priv->size
.
Second, what is the problem? And what is causing it?
Also, if I change interpipesrc
(based on appsrc
) to is-live=false
the GST_BUFFER_FLAG_GAP
disappears, but I still get no sound.
If I also change interpipesink
to sync=true
(based on appsink
) it starts working (I get sound).
What is the difference between live and non-live behavior that causes this?
My end application will use alsasrc
so I believe I need thing to behave well with live sources...