This matches what we do for uses_sample_qualifier, and what we do in ir_set_program_inouts.cpp as well.