I'm using Debian sid with the xserver-xorg from Debian etch (due to debconf problems), and it works fine just running with "x11vnc -many" from the terminal.