Re: Backup

[mi <michael.wordehoff@gmx.de>]

Hola Benito !

Para un backup simple (non-incrementell), utiliso un script qual hace un 
tarball de fuentes differentes.
Es configurado por dos listas:
Una lista de fiecheros y ordenarios para backup, y una lista con los de 
excluir. Es possible hacer algunas listas para backups differentes.
Las listas hacerse ejecutar ('sourcing'), es possible que se utilisa 
bash-commands. No es importante conocer Bash, tambien se functiona bien 
con senderos simplos. Te hago un attachment con un README (lo siento, 
solamente en ingles). Si quieres te mando el scripto con algunos exemplos.


saludos,

	--mi




Benito Pérez López wrote:
> Hola a todos
> Estoy empezando a familiarizarme un poco con la realización de backups, 
> ya sea utilizando tar, cpio, mt... o con herramientas tipo Amanda.
> He querido empezar utilizando tar y mt, pero no he encontrado mucha
> información sobre como hacer este tipo de cosas, solo cosas sueltas.
>Conoceis algún howto (a parte del Linux Backup Howto) que pueda 
orientarme?


------------------------------------------------------------------------

packup <choice> <options>
packup -s <choice1 choice2 choice3 ...>

Backup files and directories from a list into a compressed archive.
'packup' basically is a single GNU-'tar' command line.
By default it archives but doesn't follow SymLinks.

invocation parameters
---------------------
You must specify <choice>, which is a custom backup set selection,
corresponding to a specific configuration file name. Some examples:
system  ... Essentials for emergency system restauration to some degree.
user ...... User's home directories and Documents, including Graphics.
	    Photos are already compressed, and they keep 'tar' and maybe 
	    your running system very busy. Simply copy them instead.   
webcache .. Internet web (proxy or browser) cache.
short ..... Most important and recently changed data only. 


invocation options
------------------
Note that single short options (-x -y) can _not_ be put together (like -xy).
Their order does not matter. 
There are two invocation modes for 'packup':

I.) For the first mode, add options only _after_ a _single_ choice argument:

--list-packages (or -l) 
Before processing the backup, a list of installed packages will be stored under 
the configured path $PKGLIST. Recently only Debian support.

--force-overwrite (or -f)
While finishing backup, don't ask about previous tarballs of the same name;
just overwrite them instead.

--interactive (or -i)
Let you see all file-lists and ask questions before doing anything.
Always proposing a default.

II.) For the second mode, mulitple <choice>'s must be preceded by this option:

--silent-loop (or -s) <choice1 choice2 choice3 ...>
'packup' will process a list of space seperated <choice> strings, creating 
a seperate tar.bz2 for each <choice>. Implies the option -f. 
You must specify this flag first of all, _before_ amy choice string.
Very few (but still some) output. No questions.

configuration files
-------------------
Upon invocation, 2 files are sourced, each with a list of pathes:
A  'backup' file with all the directories and files to archive, and
an 'exclude' file with sub-directories and files to exclude from the first list. 
They are called <choice>_backup and <choice>_exclude by deafult. 
On run-time, these files will create the proper file-lists for tar dynamically. 

* Example:
* The dynamic source file for "system" backup is called system_backup.
* Sourcing it, 'packup' creates a "system_backup.list" which is parsed to 'tar'. 
* The same with system_exclude and system_exclude.list.

To customize your backup file lists, edit or create the 'source files'. 
'Sourced' means, they are invoked from inside the main program, 
and their output then will build the file list for tar.
This means, within these source filel,  you'll have to implement single pathes
through 'cat' or 'echo' commands . However, you also can put any other command
and of course even little scripts there, using shell-expressions and -expansions,
to output more sophisticated selections: 
eg. exclude compressed graphics, kernelimages, redundant files like thumbnails;
or backup only documents changed since a specific date.
Please don't let a source file create empty lines in the file-list output, also
make sure the output lines don't have any trailing <space>.

If you create or change a backup-list, don't forget to check the MOUNT: line 
on top of the file.

I suggest you to include the <choice>_backup and _exclude files and the 
main script itself in the <choice>_backup file to be archived anyway.
Of course, there are drafts for all that.

output files
------------
The backup archive is bzip2 compressed by default.  
packup creates a temporary file before removing any previous tarball.

You can advise packup to first write out a list of installed packages,
if needed updating first the database. This feature recently only is implemented 
for Debian Systems. The list will be stored outside the tarball.

The proceeding of tar will be logged to <choice>_log_<date> using the system 
clock time. 

The new tarball, the dynamically built file-lists and the log file(s) are 
created in a temporary directory. The default is /tmp. Depending on invocation
options and/or your answer to question propmpts, they are moved to the real 
Backup-Directory when tar (and compression) has finished successfully.

To extract archives with ORIGINAL OWNERSHIP, you need to do it as root.
You can extract the archive with tar -jx -p --same-owner -f <filename>

mounts
------
Any partitions (drives) that need to be mounted first can be listed in the 
MOUNT: headline of the <choice>_backup list. These will also be unmounted again
whenever a <choice> backup is finished. Only local mounts are implemented by now
(- but feel free to extend the 'mount_handler' function :-)


    Have fun !


Last update: Die 11 Nov 2003 02:33:45 CET


------------------------ packup FAQ ---------------------------


----- Mount handler
    
*  Should i list a regularly auto-mounted partition in the backup_list header,
just to be on the safe side ?

>>

I don't recommend to do so, because such a partition will be unmounted when
the backup finished successfully, but may be assumed to be still (auto)mounted
later. You would have to mount it again manually. 
( Feel free to modify the mount_handler function :-)

----- Excluding files and directories

* What is the meaning of the 'short_backup' ?

>>

By now, packup tool does no incremental backup.
If you don't want to run the bigger 'main' backups too often, you can create a 
'short' backup list which contains an excerpt from these, and use 'find' options
in the 'short_exclude' file to compare time-stamps against the creation date of
these 'main' backups, so that only 'newer' files are processed. For example:

find /boot  -newer /Archiv/WOODY/Woody_system.tar.bz2 -type f

Note that you need the '-type f' here because find will spit out the directory 
names else, which would 'tar' cause to backup them completely again.


* How to use shell filename-expansion and regular expressions ?

>>

In _backup or in _exclude, just use 'ls' or 'find' as usual.
For example, this grabs only the config's and the README's in several
custom kernel subdirectories of /boot:
ls /boot/kernel-2.*/{README,config}*

Only in the exclude file list, it is possible to pass filename expansion jokers
to 'tar' directly. For example, you can include into a 'cat << EOF' section:
/var/log/*{,/*}[0-9]	# to exclude rotated logs, and 
/var/log/*.gz   	# to exclude compressed logs.

You can do more fine-tuning by combining expressions with 'find' options.
A simple way is piping 'find' through 'grep', like:
find /boot  -newer /Archiv/WOODY/Woody_system.tar.bz2 | grep ".config"


* There are doubles in my backup-list, what's wrong ?

>>

Perhaps your regular expressions or find options select stuff that is
archived within a top directory's path entry already ?
See the next question for an example.
If a backup list spits out pathes double, 'tar' will archive them double.
Anyway, it's no loss; but working thing out cleanly will make it faster. 


* I'd like to 'short-backup' all recently changed stuff of a top directory, 
but some sub-directories I want to backup total anyway. Is that possible ?

>> 

The problem is, simply specifying these directories and _additionally_ run the
'find -newer' in the top-directory will create doubles. Instead, i recommend to
use the 'find -prune' option in the _backup list, for example like this:

find /root -path /root/Scripts -prune -o -newer /Archiv/WOODY/Woody_system.tar.bz2 -type f -print

This spits out anything in /root newer than the last '*_system' backup,
but excluding the directory /root/Scripts from this search, which you can insert
into the _backup file now as simple path to backup it completely.



---------------- Appendix: Copying partitions with cpio ------------------

* I'm getting a fair amount of disk seek timeout messages in the kernel log
and I'd like to rebuild the system on a new HD.  But I'd also like to have
it down for the least amount of time possible.
Can I plug the new hd at hdb, make the partition structure I want, put the
install image somwhere on any disk/partitions, then boot to a floppy, give
some input to the loader and have it start installing locally ?

>>

Sure. Attach the new hd as hdb, partition it and copy everything from hda.

mount /dev/hdb2 /mnt
cd /
find . -xdev | cpio -oc > /mnt/hda.cpio.gz

then mount all of your newly created partitions on hdb in the root dir, 
go to the mountpoint and unpack with cpio -icdm < /mnt/hda.cpio.gz

Then edit fstab and grub menu.lsto and you should be good to go !
( if I didn't forget anything).

p.s.  A Boot floppy probably will be helpful !