aggregator: fix iteration direction in skip_buffers

Subclasses use the pad segment to determine whether a buffer should be skipped, we thus don't want to check if a buffer needs to be skipped before processing the segment it's part of.