Re: An experiment in backup

On Fri, Jan 16, 2015 at 8:14 AM, Rob Owens <rowens@ptd.net> wrote:

On Thu, Jan 15, 2015 at 08:47:01PM -0800, Kevin O'Gorman wrote:
> On Thu, Jan 15, 2015 at 8:41 PM, David Christensen <
> > There are two basic kinds of "backups":
> >
> > 1. File system -- e.g. a copy of the files and directories on an mounted
> > and operating drive.
> >
> > 2. Raw binary image -- e.g. a copy of the bytes on a drive taken when the
> > drive is powered, but the partitions, volumes, file systems, etc., are not
> > mounted.
> >
> >
> > For system drives, the former won't work; you need the later. I connect a
> > large hard drive (to hold the images), boot Debian installation media into
> > rescue mode, and use 'dd' to backup/ restore system drive raw binary images.
> >
> > I was hoping for some details on why this won't work on system drives, or
> conditions under which it just might. Another user has suggested I read
> https://help.ubuntu.com/community/BackupYourSystem/TAR which suggests that
> it actually should work.

Image backups are definitely easier for doing disaster recovery of an
entire machine. And when you have that kind of problem, you may really
appreciate having to do less work / make fewer decisions.

But filesystems backups can be used for disaster recovery. I've done
it. One potential problem is that on a running system, things change.
So at the start of your backup, you backup file A. At the end of the
backup, you backup file Z. But in the middle of the backup, both file A
and file Z have changed. And some software requires that file A and
file Z be in sync. When you restore, those files are not in sync and
you could have a problem.

In practice, I haven't seen this be a problem much on home desktop
machines. But that's not to say it couldn't be a problem.

Another thing to consider is hardware changes. This can make certain
devices be named differently when you restore. eth0 becomes eth1, and
/etc/network/interfaces doesn't have a stanza for eth1. /dev/dvd
becomes /dev/dvd1, and your cd burner was set to look for /dev/dvd which
no longer exists. These things can be fixed in the /etc/udev/rules.d
directory.

UIDs of disk partitions will change. If /etc/fstab references UIDs, you
need to update it. Same for /boot/grub/grub.cfg, although for that you
run update-grub2 from the restored system (you'll need to boot with a
live cd and chroot, or you'll need to boot with Super Grub Disk or
similar). You will also need to install the bootloader on the new hard
disk. 'grub-install /dev/sda'

The UUID and Grub issues don't show up when restoring from an image
backup, but the network card and cd burner issues can.

There are a lot of free software backup solutions available. I would
recommend using one of those, unless this endevour is more for learning
experience than anything else.

Backuppc may be overkill for your case, but it's pretty good. It will
do file pooling and compression, so keeping multiple backups of one or
more machines doesn't take up much disk space.

This is partly a learning experience, and partly to take control of what's happening.

I have plenty of backup capacity. Aside from using compression I see no need

to worry about optimizing storage. Instead, I like having each backup be self-contained
and easily identifiable.

I have roughly 32TB of 2-TB disks dedicated to backups (!) plus smaller older ones.
My three machines are quite modest in size. Except for one with a huge stripe array
for temporary stuff related to a hobby of mine, not subject to backups. I have
backups running back for years, covering machines I recycled as much as a decade ago.

All my drives are such that I have a USB drive dock for them, or they come with a

USB interface. USB seems likely to stick around for a while.

My hardware is stable enough I'm not worried about naming confusions. Almost

everything is GPT partitions. Everything is identified by UUIDs -- those have no

need to change and I know how to set them on an existing partition to match anything
in /etc/fstab.

I back up the GPT itself (both copies) and every partition other than swap. Just in
case I want it later.

I've not tried other solutions. I worry that the ones folks seem to like most do more
than I need or want in terms of management. I want my stuff where I can see it, so

to speak, and where I can use the ancient tools (tar, dd, gzip and so on) to work with it.

I have already spent too much of my life dealing with incompatibilites, problems with
software and format changes and a whole raft of other stuff. I want it simple, and I
want it to be obvious. I use directory and file names to help make it so, and every

backup includes the scripts that were used to make it. It's all bash script and tools

that were known at the time I started using Linux -- roughly 1993.

I'm a bit ashamed I've never tried (or needed) a system restore before. So whatever
this problem is, it's probably lurking in all of my backups. I may just make image
backups of my bootable partitions real quick now, pending resolution of this difficulty.

Thanks for all the input, everyone. I'm still trying to figure out what went wrong.