This may be something you already tried, but a friend of mine had this
issue so
I am suggesting it anyway.
Are you sure you need the 304.x version of the drivers? If you have a newer
graphics card, then using this older driver may cause problems. You can
check
this by running nvidia-debugdump --list to see your device name and then
searching for it in this list <http://www.nvidia.com/object/IO_32667.html>,
or, probably easier, selecting it on this page
<http://www.nvidia.com/Download/index.aspx?lang=en-us>.
You can then search for the Debian package matching the suggested version.
Good luck, video drivers tend to be finicky...
Thom