Re: ..weeding out bad debs off mirrors, was: Can I get some help, please?

[carlos@fisica.ufpr.br (Carlos Carvalho)]

Arnt Karlsen (arnt@c2i.net) wrote on 27 May 2007 23:44:
 >On Fri, 25 May 2007 14:13:45 -0300, Carlos wrote in message 
 >> One can use the checksums in indices/md5sums.gz to verify the files
 >> locally. Very useful indeed... I do it periodically, because a mirror
 >> is a significant responsibility.
 >
 >..aye, but _precisely_how_ do you do it?  Playing with Rafal's
 >'zcat indices/md5sums.gz |md5sum -c |grep -v ": OK" |grep -v ' etc
 >from the mirror root, I keep finding new innovative variations of
 >": Failure".

 zcat indices/md5sum.gz | md5sum -c > /some/path/md5sum-output 2> /dev/null

You can send stderr to a file if you want but it's not necessary.

Some *hours* later (for a full mirror)...

 grep -v ': OK$' /some/path/md5sum-output

should give you

/some/path/md5sum-output:Archive_Maintenance_In_Progress: FAILED open or read
/some/path/md5sum-output:indices/md5sums.gz: FAILED

if your mirror is correct.

I also check that all the files in the tree are in md5sums (the
converse is already done by md5sum, which flags non-existent files).
The only ones that aren't should be those in project/trace. Use this:

 find \! -type d \! -type l | sed -e 's:./::' | sort > /some/path/debian-files
 zcat indices/md5sums.gz | awk '{print $2}' | sort > /some/path/debian-md5sum

then

 join -v1 /some/path/debian-files /some/path/debian-md5sum

points out the intruders.