To restart X server, you only need to restart gdm.

/etc/init.d/gdm restart

* Stopping GNOME Display Manager… [ ok ]
* Starting GNOME Display Manager… [ ok ]

By master