Ok, so it is glReadPixels() that is waiting for vsync. I added some
measurements around that call and it is definitely where the delay happens.

For good measure I removed the mapping (step 2) and the extra render (step 4).
So it does seem that glReadPixels() is simply broken and isn't doing the nice
asynchronous read is it supposed to.

Oddly enough I could not reproduce the problem with a simple test program. So
there is something else required to provoke the issue.

Ideas?