However, with help of one friend, the bug was reproduced with xterm instead of an SDL client, and also on Silicon Motion SM710. So, I'm reassigning (or at least attempting to reassign) this bug to xserver-xorg-core.
Correct repro steps:
1. Switch to a tty vt.
2. X -ac :1
3. DISPLAY=:1 xterm
4. ctrl+c on the tty vt where you launched xterm.
5. Observe X crashing.
Same thing happens if a window manager is launched.
I presume SDL crashes immediately because it opens a client connection to X.org, closes it soon afterwards, and then opens another one. Thus, after closing the first connection, there is a crash and the second one (the one that actually opens a window) never happens. This is just a guess.