On 15/06/10 11:36, Lisi wrote:
On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote:There are many many ways to make take backups beyond having a disk big enough to hold the data.Would you feel inclined to elaborate? I'm trying to solve this problem for my granddaughter's large HDD, and am not keen to have to buy a 300GB external drive. Tar would still require a fairly large medium, would it not. :-( Lisi
I find it interesting that the discussion is about the size of the media on which the data is stored rather than the size of the dataset(s) itself.
I am not by any means running a professional shop, but at home I do have desktops and laptops to take care of, and I do have a major external site that I look after and some smaller stuff that I am starting to do professionally.
I have been using approximately the same approach for several years, but I have just started to formalize it by documenting what I do in a private tiddly wiki (see http://www.tiddlywiki.org) that I can refer to and keep up to date (previously it was jottings in a notebook that I used to keep this sort of information in).
I realise that I have to compromise because of the limited storage capacity I have in all locations (although my two 1TB disks I hope to take delivery of in the next few days is going to ease some worries :-) ), so I define some of my storage as single disk and some as software raid 1 (using mdadm).
I first define the set of key data stores that I wish to protect and for each one indicate their size, whether they need to be operationally protected by being on raid and how frequent/recent a backup of that I need. For frequency I generally work on daily, weekly and monthly (because of the ability to drop a backup script in /etc/cron.daily etc) although I also have datasets that are accessed by Americans (I am in the UK) so I also have /etc/cron.d entries that specify 9-10am times as opposed to the middle of the night for the /etc/cron.daily and similar.
I use that to develop a backup plan - where I can copy the datasets locally on the same machine, (mainly I tend to do that with database dumps in the first instance), across between two machines, or offsite from machines across the internet.
For convenience I tend to use rsync -a sometimes with the --delete option sometimes not. Internally between machines I like to have rsyncd running, defining the local datasets with a module in rsyncd.conf - and specifically my wifes Windows XP desktop, and my Windows 7 laptop have rsync running as a service so I can back these machines up from linux.
Once I know what my requirements are for dataset storages including the operational copy and the backups, I then define what I need in terms of raid, non-raid, and an approximation of the volumes. I tend to use LVM on raid or on a single disk. I try and avoid using LVM to span volumes, although in the past it has been necessary and it DID bite me.
(I was using PVMOVE to move data off a disk I wanted to release. PVMOVE seems to be VERY sensitive to other things happening as well, and I stupidly got bored with waiting for it to finish and started another task on the same volume group. It screwed up and lost the data from that - I can't remember if it was just the LV that I was trying to move off the PV or whether it was the entire volume group)
One key store is my longer term archive. I wrote about that in my 2005 blog entry
http://www.chandlerfamily.org.uk/blog/2005/apr/backup-and-archiving-home(this store is both on raid, and backed up on another machine - sadly I can't afford to back it up off site as well).
The real magic command is "cp -alf" which essentially merges a shorter term store with a longer term one, making new entries where the shorter store has a file that isn't in the longer term store, and overwriting it where the shorter term store has a file with the same name as the longer one.
-- Alan Chandler http://www.chandlerfamily.org.uk