All the boxes in the 'trak' container should be parsed regardless of whether its duration is shorter than the duration of the longest track.
For example, qtdemux can not parse a video track from sample mp4 created with the following command: gst-launch-1.0 videotestsrc num-buffers=300 ! x264enc ! h264parse ! qtmux name=mux ! queue ! filesink location=sample.mp4 audiotestsrc num-buffers=3000 ! fdkaacenc ! aacparse ! mux.