Re: Backups - was Re: LVM
On 15/06/10 11:36, Lisi wrote:
On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote:
There are many many ways to make take backups beyond having a disk big
enough to hold the data.
Would you feel inclined to elaborate? I'm trying to solve this problem for my
granddaughter's large HDD, and am not keen to have to buy a 300GB external
drive. Tar would still require a fairly large medium, would it not. :-(
I find it interesting that the discussion is about the size of the media
on which the data is stored rather than the size of the dataset(s) itself.
I am not by any means running a professional shop, but at home I do have
desktops and laptops to take care of, and I do have a major external
site that I look after and some smaller stuff that I am starting to do
I have been using approximately the same approach for several years, but
I have just started to formalize it by documenting what I do in a
private tiddly wiki (see http://www.tiddlywiki.org) that I can refer to
and keep up to date (previously it was jottings in a notebook that I
used to keep this sort of information in).
I realise that I have to compromise because of the limited storage
capacity I have in all locations (although my two 1TB disks I hope to
take delivery of in the next few days is going to ease some worries :-)
), so I define some of my storage as single disk and some as software
raid 1 (using mdadm).
I first define the set of key data stores that I wish to protect and for
each one indicate their size, whether they need to be operationally
protected by being on raid and how frequent/recent a backup of that I
need. For frequency I generally work on daily, weekly and monthly
(because of the ability to drop a backup script in /etc/cron.daily etc)
although I also have datasets that are accessed by Americans (I am in
the UK) so I also have /etc/cron.d entries that specify 9-10am times as
opposed to the middle of the night for the /etc/cron.daily and similar.
I use that to develop a backup plan - where I can copy the datasets
locally on the same machine, (mainly I tend to do that with database
dumps in the first instance), across between two machines, or offsite
from machines across the internet.
For convenience I tend to use rsync -a sometimes with the --delete
option sometimes not. Internally between machines I like to have rsyncd
running, defining the local datasets with a module in rsyncd.conf - and
specifically my wifes Windows XP desktop, and my Windows 7 laptop have
rsync running as a service so I can back these machines up from linux.
Once I know what my requirements are for dataset storages including the
operational copy and the backups, I then define what I need in terms of
raid, non-raid, and an approximation of the volumes. I tend to use LVM
on raid or on a single disk. I try and avoid using LVM to span volumes,
although in the past it has been necessary and it DID bite me.
(I was using PVMOVE to move data off a disk I wanted to release. PVMOVE
seems to be VERY sensitive to other things happening as well, and I
stupidly got bored with waiting for it to finish and started another
task on the same volume group. It screwed up and lost the data from
that - I can't remember if it was just the LV that I was trying to move
off the PV or whether it was the entire volume group)
One key store is my longer term archive. I wrote about that in my 2005
(this store is both on raid, and backed up on another machine - sadly I
can't afford to back it up off site as well).
The real magic command is "cp -alf" which essentially merges a shorter
term store with a longer term one, making new entries where the shorter
store has a file that isn't in the longer term store, and overwriting it
where the shorter term store has a file with the same name as the longer
- From: "Gerald C.Catling" <firstname.lastname@example.org>
- Re: LVM
- From: Andrew Sackville-West <email@example.com>
- Backups - was Re: LVM
- From: Lisi <firstname.lastname@example.org>