[debian-knoppix] cloop file format change proposal
Dear Knoppix/cloop developers
as you may know I utilize cloop for my own bootable GNU/Linux on CD
(one based on SuSE7.3 and another on debian/woody distributions)
valentyn+knoppix <at> nospam.openoffice.nl
once suggested:
A better method for all of the indexing hassle would probably be a rewrite
of cloop where:
- a version number is included in the header
- the index is at the end of the file, followed by
- a number that indicates the exact position of the start of the index.
while Klaus objected:
think that stuff like version information or indexes really belong to
the beginning of a file, not at the end, so you still can access them if
later parts of the file are corrupted.
seek times are another issue.
while file curruption should be better detected by MD5-sums and the
associated
data is not available then anyway there are some alternatives (2 and 3)
below.
looking at the final data layout following the header this would look like
block 1 data, block 2 data, ..., block n data, block 1 size, ...,
block n size, offset of block 1 size (64 bit)
n being the total number of blocks
This is advantageous because filesize-8 can be easily found/seeked
and all block info can be read afterwards at once.
we could even run compress on this index-data too which could save some
more KB but would require keeping the decompressed Data in RAM ourselves.
Alternative 2 would be
block 1 size, block 1 data, block 2 size, block 2 data, ..., block n data,
0
+easy to code
+low mem
+integrity with end missing or corrupted
-bad seek times
Alternative 3 would be
value of m, block 1 size, block 2 size, ..., block m size, block 1 data,
block 2 data, ..., block m data, block m+1 size, block m+2 size, ...,
block n size, 0
with m<n e.g. m=511 to m=4096 seem reasonable
this is not very different from the current format, except that
fewer blocks are accumulated before writing them to disk
Alternative 2 can be seen as a special case of this with implicit m=1
+adequate seek times
+integrity
+somewhat low mem
in all the previous examples "block size" (17 to 32 bit would suffice)
could be replaced by absolute offset from the beginning (64 bit)
at compression time or when first reading this before decompression.
Of cause all those alternatives are incompatible with current file format
so one had to make sure not to use cloop-0.6x with such an image.
But some format change has to be done anyway to allow for large files
e.g. when mastering a compressed 4.3GB DVD
When you have decided on which new file-format you prefer
(better suggestions welcome)
I will volunteer to implement a working 64-bit cloop as well as an
interface-compatible(pipeable) create_compressed_fs
btw: you could have mentioned somewhere
http://packages.debian.org/unstable/misc/cloop-src.html
or some other means of obtaining the latest version (e.g. CVSRoot)
Bernhard M. Wiedemann
_______________________________________________
debian-knoppix mailing list
debian-knoppix@linuxtag.org
http://mailman.linuxtag.org/mailman/listinfo/debian-knoppix
Reply to: