pad: Accumulate live/non-live latency values separately
And only ever use the non-live values if all pads are non-live, otherwise only use the results of all live pads. It's unclear what one would use the values for in the non-live case, but by this we at least pass them through correctly then. This is a follow-up for 794944f7, which causes wrong latency calculations if the first pad is non-live but a later pad is actually live. In that case the live values would be accumulated together with the values of the non-live first pad, generally causing wrong min/max latencies to be calculated.