Keep track of the time we spend pausing the jitterbuffers when they were
buffering and distribute this elapsed time to the jitterbuffers.
Also keep the latency in nanosecond precision.