dup() and fcntl(_, F_DUPFD{_CLOEXEC}) have different behavior
After we call asynchronous atomic commit IOCTL (with non-blocking flag) , out_fence is set to fd which represents the out fence.
The thing is if we will try to duplicate this fd using fcntl() , it will fail (return -1), but dup() will succeed.
More surprisingly if we put ALOGE("Something");
in between of atomic commit IOCTL and fcntl(), it will successfully duplicate fd.
I was able to reproduce such scenarios on PinePhonePro (Rockchip) kernel v5.17+.
Looking into dup() and fcntl() syscall implementation I can see additional checks in fcntl() and can't see them in dup(), so it may be some in-kernel processes wasn't finished prior to returning from drm commit() IOCTL or something else.
Anyway, I'll always use dup() in drm_hwcomposer until someone fixes this issue.