ofcourse at current state of technology Xorg will be unable to fit single chip, yet it's current modular nature and fact that it's (still) completely open-source, and uses pieces of system which are also open-source developed (i.e. http://wiki.opengraphics.org/ )
... could allow it to fit in several 'off-the-shelf' fpga's.