The issue seems to be that kwin or DDX allocates a texture of the size 2*1600x1200 and your hardware only supports 2560x2560 for rendering and 2048x2048 for texturing. Unfortunately I'm not an expert on X server internals.