Bug#553707: RFH: lzma -- future of Debian squashfs-lzma
Michael Prokop <mika@debian.org> wrote:
JFYI: We (the grml team) have working squashfs 4 + lzma patches for
kernel 2.6.31 and an according squashfs-tools package providing
lzma support. Please let me know if we can assist in any way.
I believe these patches are based on the OpenWRT Squashfs patches.
In that case I strongly advise against using them in Debian.
The Squashfs 4.0 LZMA filesystems generated by the OpenWRT patches
are incompatible with my 'official' Squashfs LZMA 4.0 filesystems
and will not (and cannot, see below) be officially supported.
While is OK for an embedded system to use incompatible Squashfs
versions (they can just regenerate the filesystem), it is unwise
for a distribution because users will find they cannot read
their LZMA compressed 4.0 filesystems in the future.
A little bit of background is probably necessary to understand
the situation. First I should mention that I've never been contacted
by the OpenWRT guys and the this is based on a study of their patches.
LZMA support in Squashfs has always been problematic because LZMA
decompression support hasn't been provided by the mainline kernel.
While Squashfs was also out of tree I choose not to officially
support LZMA compression because doing so would have presented
an additional barrier to its mainlining. This led to numerous
third-party patches to Squashfs 3.x providing LZMA support with
a private LZMA decompression implementation. Having a private
LZMA decompression implementation is OK as long as you have
no intention of trying to mainline the patches (they'll never
be accepted).
The situation today is different, Squashfs and an LZMA decompression
implementation has been mainlined. This means it is now possible
to have Squashfs LZMA support mainlined using the in-kernel
LZMA decompression library. This is what I have been working
towards, and an initial implementation is now ready (see the
original email for the link).
The OpenWRT guys have produced an incompatible Squashfs 4.0 LZMA
implementation using their own private LZMA decompression implementation
(they do not use the in-kernel LZMA decompression library). The
distinction is important because they have modified the LZMA header in
their implementation to be incompatible with what is expected
by the in-kernel LZMA library and by the standard LZMA SDK
(i.e. the official LZMA SDK and the in-kernel LZMA library cannot
read their LZMA compressed data).
The standard LZMA header is as follows
0 1 Special LZMA properties (lc,lp, pb in encoded form)
1 4 Dictionary size (little endian)
5 8 Uncompressed size (little endian).
OpenWRT have optimised the LZMA header, and have removed
the 8 bye uncompressed size field. The in-kernel LZMA
library cannot read their LZMA compressed data because it
expects this field to be present. What this means is their
LZMA compressed filesystems cannot be supported by any
mainlined Squashfs LZMA implementation.
Phillip
Reply to: