Comment # 5 on bug 104597 from Mario Kleiner

Can you try what happens if you add the following snippet
inside the <device> section of your /etc/drirc and restart compton?

<application name="compton" executable="compton">
    <option name="allow_rgb10_configs" value="false"/>
</application>

The output of xdpyinfo and glxinfo could also be useful.

To disable the new rgb10 support globally, one can add this snippet instead:

<option name="allow_rgb10_configs" value="false"/>