Re: Script that partially completes mirror from CDs
[please direct all followups to debian-cd, and don't cc me]
Ok, if youre not satisfied with copydir, then maybe you can use something
from my script below.
Regards,
/Karl
-----------------------------------------------------------------------
Karl Hammar Aspö Data karl@kalle.csb.ki.se
Lilla Aspö 2340 +46 173 140 57 Networks
S-742 94 Östhammar +46 70 511 97 84 Computers
Sweden Consulting
-----------------------------------------------------------------------
From: Antonio Rodriguez <arodriguez@worldnet.att.net>
Subject: Re: Script that partially completes mirror from CDs
Date: Sun, 26 Nov 2000 10:48:59 -0500
> I checked it, but I didn't see any clear way opf doing it.
>
> Karl Hammar wrote:
> >
> > Have you tried copydir from the mirrodir package?
> >
> > Regards,
> > /Karl
> >
> > -----------------------------------------------------------------------
> > Karl Hammar Aspö Data karl@kalle.csb.ki.se
> > Lilla Aspö 2340 +46 173 140 57 Networks
> > S-742 94 Östhammar +46 70 511 97 84 Computers
> > Sweden Consulting
> > -----------------------------------------------------------------------
> >
> > From: Antonio Rodriguez <arodriguez@worldnet.att.net>
> > Subject: Script that partially completes mirror from CDs
> > Date: Sat, 25 Nov 2000 07:41:05 -0500
> >
> > > Hello all. Does any one has a script that would
> > > 1. Look into the debian site, get instructions about
> > > directory structure (packages).
> > > 2. Look into CDs -binaries- loaded in /cdrom, or /whatever.
> > > 3. Upon finding a match, will retrieve it from CD and put it in
> > > the partial local mirror in the right place.
> > >
> > > This would greatly reduce the amount of traffic, since many people run
> > > partial mirrors, and some times for some reason these get wiped off.
> > > I myself need one, but the truth is that I am not sure how to write
> > > such a script. Any help will be greatly appreciated.
> > > Thanks,
> > > Antonio.
...
------ cut ----------------------------------------------------------------
#!/bin/sh
# Copyright: Karl Hammar
# Copyright terms: GPL
# upd.mirror tries to do the same as mirror/mirrordir,
# but the files are transferred via tape
# or more specifically:
# it tries to make a source and a destination directory tree be identical
# and when the source and the destination are on different hosts
# and when a direct network connection is not usable but we can excange e.g. tapes
# two files are identical when:
# they have the same content (or as an approximation the same md5sum)
# they have the same size, permissions, uid, gid, and modify date
# they have the same name and the same location within the tree
# this program ignores pipe, socket, char and block special files
# we assume two files are the same when command below gives same output:
# ordinary files:
# ls -l --full-time | cut -b-11,17-
# i.e. omit #links field, and make ls independant of when it is run
# directories:
# ls -ld --full-time | cut -b-11,17-34,69-
# same permission, uid, gid and name, ignore #links, size, and mdate
# symbolic links:
# ls -l --full-time | cut -b17-43,69-
# i.e. (omit permission and date) they have the same size, uid, gid and it points to the same file
# hard links (treated like ordinary files, tar/cpio takes care of theese)
# we quietly assume size < 99999999
# -rw-r--r-- 1 karl users 244414 Thu Jul 06 09:25:51 2000 linux_scsi_24.ps
# drwxr-xr-x 2 karl users 4096 Tue Sep 21 16:41:39 1999 kurs/
# lrwxrwxrwx 1 karl users 14 Sat Sep 16 15:45:38 2000 pub -> /home/ftp/pub/
# 12345678901234567890123456789012345678901234567890123456789012345678901234567890
# 1 2 3 4 5 6 7
# -rw-r--r-- karl users 244414 Thu Jul 06 09:25:51 2000 linux_scsi_24.ps
# drwxr-xr-x karl users kurs
# karl users 14 pub -> /home/ftp/pub/
src=/mnt
dst=/home/ftp/pub/linux
tmp=$HOME/tmp
tape=/dev/nst0
case "$1" in
1) # step 1, at the destination host run this and copy files $tmp/ls.?d to source host manually
# assume no file name contains any newlines
cd $dst
find . -type f -xdev | quoteline | sort | xargs ls -l --full-time | cut -b-11,17- > $tmp/ls.fd
find . -type d -xdev | quoteline | sort | xargs ls -ld --full-time | cut -b-11,17-34,69- > $tmp/ls.dd
find . -type l -xdev | quoteline | sort | xargs ls -l --full-time | cut -b17-43,69- > $tmp/ls.ld
;;
2) # step 2, at the source host, make sure above files arrived and run this
# assume no file name contains any newlines
cd $src
find . -type f -xdev | quoteline | sort | xargs ls -l --full-time | cut -b-11,17- > $tmp/ls.fs
find . -type d -xdev | quoteline | sort | xargs ls -ld --full-time | cut -b-11,17-34,69- > $tmp/ls.ds
find . -type l -xdev | quoteline | sort | xargs ls -l --full-time | cut -b17-43,69- > $tmp/ls.ls
cd $tmp
# assume new date implies different file
diff ls.fd ls.fs > f.diff
grep '^<' f.diff | cut -b66- > rm.theese
grep '^>' f.diff | cut -b66- > do.cp
# assume that neither filename contain ' -> '
diff ls.ld ls.ls > l.diff
grep '^<' l.diff | cut -b30- | sed -e 's/ -> .*$//' >> rm.theese
grep '^>' l.diff | cut -b30- | sed -e 's/ -> .*$//' > do.ln
diff ls.dd ls.ds > d.diff
grep '^<' d.diff | cut -b32- > rm.dir
grep '^>' d.diff | cut -b32- > do.mkdir
sed -e 's/^ *//' do.cp do.ln do.mkdir | sort -r > to.tape
;;
2b)
cd $src
tar cfC $tape $tmp rm.theese rm.dir
cat $tmp/to.tape | cpio -o 2>&1 > $tape | grep -v '^cpio: .*: truncating inode number$'
mt -f $tape rewind
;;
3) # step 3, back at the destination host, run this to update local mirror
# when updating do:
cd $tmp
tar xf $tape
cd $dst
cat $tmp/rm.theese | quoteline | xargs rm
sort -r $tmp/rm.dir | quoteline | xargs rmdir
mt -f $tape fsf
cpio -idmu < $tape
mt -f $tape rewind
;;
esac
Reply to: