recovering wrecked system (potato->unstable)

[Graham Seaman <graham@codix.net>]

Hi,

I've probably wrecked my system and will need to reformat
and start over, but am hoping I'll be able to recover without.
But I'm stuck - any advice appreciated (as you will see from below,
I don't have much idea what I'm doing).

I had an old potato system. 
Problem 1:

I tried to upgrade X to unstable (xserver-common, xlibs etc) (yes, I know,
silly move). The first result was that xdm failed to start with 'Can't
open RGB_DB'. I copied rgb.txt to /usr/X11R6/lib/X11/; now I have xdm,
xfs, windowmaker etc all running, but a completely blank screen. All the
parameters for my video chipset (ATI Rage Pro) seem to have been set
correctly. $REALSTARTUP is not quoted, though the symptoms sound similar.
I can still get into console mode, where everything works ok.
   
Problem 2:

I thought the problem might be due to running an old kernel (2.2.17),
since xlibs installation gave me a warning 'Kernel too old for Mesa DRI'.
I attempted to install the 2.4.14-1 image. Rather than booting from the
hard drive, I tried to create a boot floppy. I'm unable to boot from
this, getting a kernel panic: 'VFS: Cannot open root device 'hdb3' or
03:43'.
However, I can still boot from my old boot floppy using hdb3.

Any suggestions? 
Thanks
Graham