Re: Backup Scripts

[Alan Chandler <alan@chandlerfamily.org.uk>]

On Friday 01 April 2005 00:19, Tom Allison wrote:
> OK, I've been examining the various packages available to me for data
> backup and I was wondering if anyone might have used something like:
>
> bacula
> backupninja
> backup21
> backup-manager
>
> These are some "stock" packages I have managed to find so far.

I use a combination of rsync -axHq , cp -alf and mv

let me explain.

BASIC PRINCIPALS

1) Nearky every thing is backed up disk to different disk every night.

2) My home directory is divided into three areas
a)  directory mydocs     - anything in here is kept with a versioning control 
system (see below)
b) directory nobak - is not backed up (generally holds items which I don't 
care whether they are lost and which would use up valuable backup space if I 
did
c) the rest - backed up as per point 1)

3) /etc - treated like mydocs in 2 a) above

HOW

I use rsync backup-dir to notice changes in mydoc and etc and to put them into 
daily, then weekly, then monthly, then CD archive directories

So in /etc/cron.daily I have a file called backup with the following sort of 
contents.

 #!/bin/sh
#       $Id: backup,v 1.2 2002/12/08 08:14:42 alan Exp $
#       AKC - see below for history

logger -t "Backup:" "Backup started"
ARCH=/bak/archive

if [ -d $ARCH/daily.6 ] ; then
        if [ ! -d $ARCH/weekly.1 ] ; then mkdir -p $ARCH/weekly.1 ; fi
# Now merge in stuff here with what might already be there using hard links
        cp -alf $ARCH/daily.6/* $ARCH/weekly.1
# Finally loose the rest
        rm -rf $ARCH/daily.6 ;

fi
# Shift along snapshots
if [ -d $ARCH/daily.5 ] ; then mv $ARCH/daily.5 $ARCH/daily.6 ; fi
if [ -d $ARCH/daily.4 ] ; then mv $ARCH/daily.4 $ARCH/daily.5 ; fi
if [ -d $ARCH/daily.3 ] ; then mv $ARCH/daily.3 $ARCH/daily.4 ; fi
if [ -d $ARCH/daily.2 ] ; then mv $ARCH/daily.2 $ARCH/daily.3 ; fi
if [ -d $ARCH/daily.1 ] ; then mv $ARCH/daily.1 $ARCH/daily.2 ; fi
if [ -d $ARCH/snap ] ; then mv $ARCH/snap $ARCH/daily.1 ; fi

# Collect new snapshot archive stuff doing daily backup on the way

mkdir -p $ARCH/snap

logger -t "Backup:" "Starting root backup"

# Backup root file system - keep track of changes to etc

rsync -aHxq --delete --exclude-from /etc/backup/exclude --backup 
--backup-dir=$ARCH/snap/kanger/etc/ /etc/ /bak/root/etc/
rsync -aHxq --delete --exclude "etc/" / /bak/root/
rsync -aHxq --delete /boot/ /bak/root/boot/

...


The first peice is about merging the oldest daily archive into a weekly one 
with the cp -alf command followed by the rm -rf command.  This creates 
additional links to the files if they don't already exist in the destination 
directory, else it copies them (overwriting the previous content).  This 
means it is normally very fast.  What that means is that for a file that is 
constantly changing I have daily snapshots up to one week old, and then 
(using similar techniques with backup in cron.weekly and cron.monthly) I 
create weekly level snapshots up to one month old and then monthly snapshots 
up to 6 months old.  There after the latest file (older than six months) is 
sitting in an archive for writing to CD0.

The second piece (using rsync) is showing how to backup the root partition 
using rsync, and adding to the change longer term archive all changes to /etc 
(except the list of files held in /etc/backup/exclude).

I have a similar schema for my home directory (on another partition).
-- 
Alan Chandler
http://www.chandlerfamily.org.uk