Garbled HDMI audio. Transfer failed: File descriptor in bad state

I have a TV connected to the HDMI port of my A770, the detected properties are:

andrew@andrew-gentoo-pc card2 % cat /proc/asound/card2/eld\#2.0
monitor_present         1
eld_valid               1
codec_pin_nid           0x4
codec_dev_id            0x0
codec_cvt_nid           0x3
monitor_name            HKC-TV
connection_type         DisplayPort
eld_version             [0x2] CEA-861D or below
edid_version            [0x3] CEA-861-B, C or D
manufacture_id          0x6321
product_id              0x0
port_id                 0x0
support_hdcp            0
support_ai              0
audio_sync_delay        0
speakers                [0x1] FL/FR
sad_count               3
sad0_coding_type        [0x1] LPCM
sad0_channels           2
sad0_rates              [0xe0] 32000 44100 48000
sad0_bits               [0x60000] 16 20
sad1_coding_type        [0xa] E-AC-3/DD+ (Dolby Digital Plus)
sad1_channels           8
sad1_rates              [0xe0] 32000 44100 48000
sad2_coding_type        [0x2] AC-3
sad2_channels           6
sad2_rates              [0xe0] 32000 44100 48000
sad2_max_bitrate        640000

Sending audio to the TV results in this mess: 3106__2023-05-15_21_23_28_ This is KDE Plasma's speaker testing utility, we can only hear front left (but audio is garbled), front right does not do anything at all.

• There is also a monitor connected using DisplayPort, this audio connection works fine.
• The HDMI connection to the TV works fine under Windows, so this cannot be a hardware problem.
• The HDMI connection is listed as a DisplayPort connection in the eld info, whether this is actually a problem and if it is related to the audio issue I don't know.
• Sending ac3 audio (which is listed as supported and also works under Windows) results in no sound at all.
• Happens both with kernel version 6.3.2 and the latest drm-tip.