Argh. I recall having similar problems. It might be the screensaver process segfaulting. Is there anything useful in ~/.xsession-errors?