vtenc: encoding interlaced ProRes (and H.264?) always fails on Apple M1 Max (and likely Pro)
GStreamer version: 1.20.3.
Operating system: macOS 12.5.1.
Hardware: MacBookPro18,2 (likely earlier MacBookPro18,3 too), NOT Macmini9,1.
Reproduce: run the following command (in a writable directory):
gst-launch-1.0 videotestsrc num-buffers=100 ! video/x-raw, interlace-mode=interleaved, field-order=top-field-first ! videoconvert ! vtenc_prores ! qtmux ! filesink location=videotestsrc.mov
Expected: the resulting file is a valid movie of 3.3s of test video.
Actual: the resulting file is empty (0 bytes), the terminal output is attached: ProRes-interlaced_M1-Max_2022-09-07.txt.
Notes:
- In practice, our application uses GStreamer APIs to write video files in ProRes formats, and it works fine on earlier Apple machines (including M1 Mini of 2020), but not on ones with the newer M1 Pro/Max chips.
- There are independent indications that this is a bug on the AVFoundation side: 1, 2 (the links are optimized for Chrome, Edge).
- This used to affect interlaced H.264 encoding as well but I am unable to reproduce this using a similar pipeline for some reason.
Workaround: we fixed this in our GStreamer 1.18.5+ by this simple patch: gst-plugins-bad_applemedia_vtenc_interlaced_no-HW-accel.patch (it works for the latest v1.20.* too, with auto-found offsets). We hope this patch can be applied to the stock GStreamer so that we don't need to rebuild it ourselves when integrating the latest version.