audio aggregator: Remove never triggered condition
Copied from https://gitlab.freedesktop.org//gstreamer/gst-plugins-base/-/merge_requests/1178
At this point, there is a buffer, which means that there is some data left in it. But we've read all that is in there. So we're certain that the next byte written from that buffer will be after the current output offset.
Add a g_assert() just to be safe
This came from my attempt to write a unit test to trigger this code. But I don't think it's possible.
Please review this MR very very very carefully because I may have missed something.