I found out that it's still related to the hardware cursor.
When I set Option "SWCursor" "true" in Xorg config, moving windows is smooth.

The mouse uses 1000Hz polling rate, if that makes any difference.
To spot the stuttering best, set acceleration profile to flat for libinput.

Btw: There is also a little problem with Redshift and hardware cursor, it turns
more yellow/orange than the rest of the screen. Using software cursor works
around this problem as well.