Hi everyone,
the current Hurd causes quite a lot of file corruption. The symptoms
are 4096 byte long, zero filled blocks at page borders.
The attached program holefinder.c detects such holes as well as holes at the
end of a file which might be shorter than 4096 bytes.
Compilation:
gcc -o /usr/local/bin/holefinder holefinder.c
Usage:
holefinder FILE
or for a complete tree
find . -type f -exec holefinder \{\} \;
Example output:
ulysses:/gnu# find . -type f -exec holefinder \{\} \;
./var/log/lastlog: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71 (1476)
./var/spool/exim/db/retry: 1
./bin/emacs-20.3: 304, 305, 306, 307, 308, 309, 310, 311, 312, 318, 319, 320, 322, 335, 336, 337, 569, 570, 571, 572, 573, 574
./lib/libreadline.so.2.1: 37, 38
./lib/gconv/EBCDIC-ES-A.so: 1
./lib/gconv/EBCDIC-ES.so: 1
./lib/gconv/EBCDIC-IS-FRISS.so: 1
./lib/gconv/EBCDIC-UK.so: 1
./lib/gconv/libCNS.so: 1
./lib/gconv/SJIS.so: 6, 8, 9
./lib/libreadline.a: 14
./lib/libstyle.so.1.3: 807, 815, 816, 817, 818, 819, 820, 821, 822
./boot/kernel: 135, 136
...
The first line says that the file lastlog consists of 70 blocks of 4096 zero
bytes followed by a 1476 bytes long zero block.
Note that most of the above zero blocks are probably fine and of no concern.
However, I have examples of not correct zero blocks. See the following
example, which are holes in my hurd source tree, native compilation.
./hurd-20000103/debian/tmp/lib/libpager.so.0.2: 3
./hurd-20000103/debian/tmp/lib/libdiskfs.so.0.2: 9, 12
./hurd-20000103/debian/tmp/lib/libps.so.0.2: 4, 8
./hurd-20000103/debian/tmp/lib/libnetfs.so.0.2: 6, 10
./hurd-20000103/debian/tmp/hurd/proc: 2
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs.a: 34, 36
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs_pic.a: 42, 48
./hurd-20000103/debian/hurd-dev/usr/include/ftpconn.h: 0
./hurd-20000103/build/libdiskfs/boot-start_pic.o: 10, 13
./hurd-20000103/build/libdiskfs/init-init_pic.o: 6, 7, 9, 10
./hurd-20000103/build/libnetfs/append-std-options_pic.o: 8
./hurd-20000103/build/libnetfs/nref_pic.o: 4, 5
./hurd-20000103/build/libps/msgUser.c: 8
./hurd-20000103/build/libps/libps_pic.a: 108, 109
./hurd-20000103/build/libtrivfs/append-args_pic.o: 9
./hurd-20000103/build/proc/ourmsg_U.h: 2 (3528)
./hurd-20000103/build/proc/proc: 2, 12
./hurd-20000103/build/ufs/alloc.o: 4
./hurd-dev_20000103_hurd-i386.deb: 35
Although some of them may be valid, the holes in the header files (.h) are
certainly not. After running the search a second time, without changing
anything, I got even more holes. It seems that reading the files and causing
a lot of disk activity develops more holes.
./hurd-20000103/debian/tmp/lib/libpager.so.0.2: 3
./hurd-20000103/debian/tmp/lib/libdiskfs.so.0.2: 9, 12
./hurd-20000103/debian/tmp/lib/libps.so.0.2: 4, 5, 8
./hurd-20000103/debian/tmp/lib/libnetfs.so.0.2: 6, 8, 10
./hurd-20000103/debian/tmp/lib/libpipe.so.0.2: 4 (456)
./hurd-20000103/debian/tmp/hurd/proc: 2
./hurd-20000103/debian/tmp/hurd/exec: 8
./hurd-20000103/debian/tmp/hurd/term: 4
./hurd-20000103/debian/tmp/hurd/ufs.static: 38, 122
./hurd-20000103/debian/tmp/hurd/ext2fs.static: 6, 76, 161
./hurd-20000103/debian/tmp/hurd/mach-defpager: 6
./hurd-20000103/debian/tmp/hurd/fwd: 1 (308)
./hurd-20000103/debian/tmp/sbin/nfsd: 3
./hurd-20000103/debian/tmp/bin/vmstat: 1
./hurd-20000103/debian/tmp/bin/ftpdir: 1
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs.a: 2, 34, 36, 83
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs_pic.a: 42, 46, 48
./hurd-20000103/debian/hurd-dev/usr/lib/libnetfs.a: 8
./hurd-20000103/debian/hurd-dev/usr/lib/libpipe.a: 2
./hurd-20000103/debian/hurd-dev/usr/include/ftpconn.h: 0, 1
./hurd-20000103/build/libdiskfs/boot-start_pic.o: 10, 13
./hurd-20000103/build/libdiskfs/init-init_pic.o: 6, 7, 9, 10
(I terminated here)
This is pretty serious. I can not upload a new hurd package, as I don't
trust the generated packages in the native compilation and can't cross
compile (see last bug report). This means people will have boot problems, as
serverboot can't boot from partitions created with e2fsprogs 1.18.
Thanks,
Marcus
PS A third run under Linux, note the info files. And compare with the first
run, for example.
./hurd-20000103/debian/tmp/lib/libpager.so.0.2: 3
./hurd-20000103/debian/tmp/lib/libdiskfs.so.0.2: 9, 12
./hurd-20000103/debian/tmp/lib/libps.so.0.2: 4, 5, 8
./hurd-20000103/debian/tmp/lib/libnetfs.so.0.2: 6, 8, 10
./hurd-20000103/debian/tmp/lib/libpipe.so.0.2: 4 (456)
./hurd-20000103/debian/tmp/hurd/proc: 2
./hurd-20000103/debian/tmp/hurd/exec: 8
./hurd-20000103/debian/tmp/hurd/term: 4
./hurd-20000103/debian/tmp/hurd/ufs.static: 38, 122
./hurd-20000103/debian/tmp/hurd/ext2fs.static: 6, 76, 161
./hurd-20000103/debian/tmp/hurd/mach-defpager: 6
./hurd-20000103/debian/tmp/hurd/fwd: 1 (308)
./hurd-20000103/debian/tmp/sbin/nfsd: 3
./hurd-20000103/debian/tmp/bin/vmstat: 1
./hurd-20000103/debian/tmp/bin/ftpdir: 1
./hurd-20000103/debian/tmp/usr/info/hurd.info-2.gz: 3 (2696)
./hurd-20000103/debian/tmp/usr/info/hurd.info-4.gz: 3 (1751)
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs.a: 2, 34, 36, 83
./hurd-20000103/debian/hurd-dev/usr/lib/libdiskfs_pic.a: 42, 46, 48
./hurd-20000103/debian/hurd-dev/usr/lib/libnetfs.a: 8
./hurd-20000103/debian/hurd-dev/usr/lib/libpipe.a: 2
./hurd-20000103/debian/hurd-dev/usr/include/ftpconn.h: 0, 1
./hurd-20000103/build/libdiskfs/boot-start_pic.o: 10, 13
./hurd-20000103/build/libdiskfs/init-init_pic.o: 6, 7, 9, 10
./hurd-20000103/build/libnetfs/append-std-options_pic.o: 8
./hurd-20000103/build/libnetfs/nref_pic.o: 4, 5
./hurd-20000103/build/libps/msgUser.c: 8
./hurd-20000103/build/libps/libps_pic.a: 108, 109
./hurd-20000103/build/libtrivfs/append-args_pic.o: 9
./hurd-20000103/build/proc/ourmsg_U.h: 2 (3528)
./hurd-20000103/build/proc/proc: 2, 12
./hurd-20000103/build/ufs/alloc.o: 4
./hurd-dev_20000103_hurd-i386.deb: 35
--
`Rhubarb is no Egyptian god.' Debian http://www.debian.org Check Key server
Marcus Brinkmann GNU http://www.gnu.org for public PGP Key
Marcus.Brinkmann@ruhr-uni-bochum.de, marcus@gnu.org PGP Key ID 36E7CD09
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ brinkmd@debian.org