Glenn McGrath wrote:
Im not confident about squashfs, i think the best solution is initrd.romfs.gz for normal installs, but squashfs or cramfs might be good for lowmem installs, i havent looked into squashfs yet, not sure when i will get time.
Hi,I'm pleased that people (thank you Drew) are considering using Squashfs in this way. I'll be happy to answer any questions that people have with Squashfs, including any suggestions for improvement...
Why are you not confident about Squashfs? (this isn't a flame, I am genuinely interested). You mentioned in an earlier thread that you didn't like the fact that it wasn't yet in the mainstream kernel (even 2.5). I agree that this is the major problem with using Squashfs as a boot filesystem. Unfortunately, I have no control over this, and in fact I have not yet asked for it to be included - I don't want to annoy the various people who can do this, when there's a feature freeze on 2.5 (and if you've been reading the lkml, it appears rather easy to annoy the necessary person, with the result it will never go in). I'm waiting for 2.7 to open, and I'll try then.
As a number of people have commented, the right filesystem/compression scheme to use is very much a question of the environment you expect to be running in, and therefore whether overhead on disk or memory is more important. I'm not going to tell you guys how to suck eggs, but the following is a comparison I did today, of the various filesystems/compression schemes and how well Squashfs fits in with this.
I took an initrd root filesystem I had handy (called ro-dev, for a PPC405 embedded board I'm working on), which contains the usual mix of modules, binaries (mostly symbolic links to busybox) and start up shell scripts. I created an ext2 filesystem, cramfs, squashfs and romfs filesystems, and gzipped/bzipped2 the ext2 and romfs filesystems. This is the result, in ascending order of size:
1699840 Feb 19 13:42 ro-dev.romfs 815104 Feb 19 13:21 ro-dev.cramfs 706401 Feb 19 13:25 ro-dev.squashfs 690770 Feb 19 13:34 ro-dev.ext2.gz 672955 Feb 19 13:42 ro-dev.romfs.gz 634925 Feb 19 13:17 ro-dev.ext2.bz2 624237 Feb 19 13:42 ro-dev.romfs.bz2As romfs doesn't do any compression, this is obviously the biggest, but in addition, because the filesystem is made of smallish binaries, the bzip2 compressed romfs image is only 36% of the uncompressed romfs size.
The cramfs image is much bigger than the other images. This is because cramfs doesn't compress metadata and compresses in units of 4K, which is much smaller than the others (as shown later, compression block size is key to the compression ratio).
The squashfs image and the gzipped ext2 and romfs images are clustered around 690K. This result is because all compress in units of 32K. Squashfs as a filesystem compresses the metadata and compresses each file data page in 32K blocks. This is the major reason why squashfs gives much greater compression ratios than cramfs. The slightly worse compression ratio for squashfs (2% bigger than the gzipped ext2 fs) is because squashfs compresses file pages individually, and in the case of this initrd, there are a lot of files less than 32K. In other cases, squashfs filesystems are smaller than the gzipped ext2 image. Obviously, the gzipped romfs and ext2 image will be uncompressed in memory.
The question of why Squashfs uses gzip compression has been brought up. The simple reason is because only gzip decompression is supported by the kernel, which is why initrds, cramfs, squashfs and jffs2 use it.
The other thing that has to be taken into account is the decompress memory overhead. Gzip compresses in 32K blocks, and a 'top' of a gunzip of one of the above --best compressed files showed gunzip required roughly 700K. Bzip2 --best compression compresses in 900K blocks, and according to the man page, requires 3700K to decompress. When a kernel filesystem is initialised/inserted, it grabs the necessary workspace to do zlib decompression, this is a per filesystem copy, and is retained for the life of the filesystem. Moving over to bzip2 compression would considerably increase the necessary kernel memory overhead, or at least necessitate a different approach to the workspace allocation. The PPMd decompressor (again according to a brief peruse of the documentation) requires considerably more workspace again.
Making bzip2 compress in 100K blocks, which reduces decompress memory requirements to 500K, brings the compression ratio much closer to that of gzip.
675457 Feb 19 13:35 ro-dev.ext2.bz2Though I'm obviously biased in favour of Squashfs, I'm an advogate of using the best tool for the job. I wrote squashfs to be better than cramfs, because otherwise there's was little point in writing and releasing it. As for its use for booting, I don't want you to mistake this for an advert for squashfs. A couple of questions regarding squashfs's role have been raised, and I simply wanted to clear up any confusion.
Regards Phillip LougherI didn't subscribe to this mailing list, and came across it while browsing, and I therefore had to extract your email from the newsgroup, and reconstruct the mail header to hopefully get this properly threaded, apologies if anything goes wrong...