Michel, Chris, I just noticed, that the delay is only there when starting the
X.Org X server for the first time. Could you please make sure, that your
systems are rebooted when trying this?

(During my tests I do not use a graphical login manager, and just log in on a
terminal and execute `exec startx`.)