I wanted a live Debian system on my USB key.
The Debian Live option is too static for my taste, I wanted a real live
system, upgradable via apt-get etc...
One option is to use a large enough USB drive and do a plain Debian install
on it. But my USB drive is only 128MB so it was not possible.
I saw someone on the web has done something like that using squashfs+unionfs
and so you can do apt-get update and then to store the resulting state back
on the drive, you do some kind of "commit".
I didn't want to go down that route, so instead I've used a "plain normal
Debian system", but using jffs2 as a file system, which has the advantage of
being compressed and writable.