I think that works as designed.

Some modes doesn't offer enough bandwidth in the default configs to transmit
all of the audio data. So for those modes we need to increase the number of
audio packets per line, but since we haven't implemented that the low bandwidth
modes can cause audio glitches.