Re: backing up backups
On 4/10/22 19:58, Default User wrote:
Hello!
My setup:
- single home x86-64 computer running Debian 11 Stable, up to date.
- one 4-Tb external usb hard drive to use as a backup device, labeled MSD1.
- another identical usb hard drive, labeled MSD2, to use as a copy of the
backups on MSD1.
- the computer and all storage devices are formatted ext4, not encrypted.
- two old Clonezilla disk images from when I installed Debian 11 last year
(probably irrelevant).
- Timeshift to daily back up system EXCEPT for data directories.
- Back in Time to daily back up data directories.
- Borgbackup to also daily back up data directories.
- Rsync to frequently backup any changed data between the daily Back in
Time and Borgbackup backups of data directories, using this command:
sudo rsync -aAXHxvv --delete --info=progress2,stats2,name2 --exclude-from
"/home/default/rsync_exclude_list.txt" /home
/media/default/MSD1/rsync_backups_of_host_home_directory_only
Each type of backup is in a separate subdirectory on MSD1 (Timeshift, Back
in TIme, Rsync, etc.).
It "seems" to work okay, BUT . . .
Then I try to use rsync to make an identical copy of backup device MSD1 on
an absolutely identical 4-Tb external usb hard drive,
labeled MSD2, using this command:
sudo rsync -aAXHxvv --delete --info=progress2,stats2,name2
/media/default/MSD1/ /media/default/MSD2
See 'man 1 rsync'. You have a slash at the end of SRC, but not at the
end of DEST. I would add a slash after "MSD2":
sudo rsync -aAXHxvv --delete --info=progress2,stats2,name2
/media/default/MSD1/ /media/default/MSD2/
I have experienced USB connection failures with external drives,
resulting in filesystem corruption. I prefer mobile racks and caddies
for my backup drives, and black SATA 6 GBps cables with locking connectors:
https://www.startech.com/en-us/hdd/drw150satbk
https://www.cablematters.com/pc-187-156-3-pack-straight-60-gbps-sata-iii-cable.aspx
Here's the problem. Although the size and number of items in each
subdirectory on MSD1 and MSD2 seem to be identical, more space in total
seems to be taken up on MSD2 than on MSD1:
df -h /dev/sdb1
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 3.6T 68G 3.4T 2% /media/default/MSD1
df -h /dev/sdc1
Filesystem Size Used Avail Use% Mounted on
/dev/sdc1 3.6T 69G 3.4T 2% /media/default/MSD2
df /dev/sdb1
Filesystem 1K-blocks Used Available Use%
Mounted on
/dev/sdb1 3844517880 71255588 3577896764 2%
/media/default/MSD1
df /dev/sdc1
Filesystem 1K-blocks Used Available Use%
Mounted on
/dev/sdc1 3844517880 71906088 3577246264 2%
/media/default/MSD2
I have tried "everything", even re-formatted MSD2 and re-done:
sudo rsync -aAXHxvv --delete --info=progress2,stats2,name2
/media/default/MSD1/ /media/default/MSD2
but the results were exactly the same.
I doubt that this is a problem with hard links, as I am using the "H"
option in rsync.
Now , pardon me for thinking that a copy of backups should not take up any
more or less space than the original. And I consider backups to be much
too important to not be absolutely "correct". So, what SHOULD be done so
that MSD2 will always be EXACTLY the same as MSD1?
Is there a way to do this using rsync?
Do you pause Timeshift, Back in Time, Borgbackup, etc., before using
rsync(1) to copy MSD1 to MSD2?
Have you compared MSD1 to MSD2 using diff(1) afterwards? If diff(1) is
happy, I would call it good. If diff(1) finds differences, figure out why.
I suppose I could just make images of MSD1, using dd for example. But then
each time wouldn't I just be backing up not just the changed data, but all
the data, AND backing up all of the free space on MSD1 also. Aside from
wasting TONS of space, each time it could take many hours or even days to
do. Which means it just won't get done. So much for redundancy!
I agree that dd(1) is overkill and impractical for this use-case.
So . . . what IS the correct way to make "backups of backups"?
For ext4 backups and ext4 copies of backups, I use rsync(1).
Are you doing any integrity checking? E.g. generating a checksum file
of the backup contents? Without such, you are exposed to bit rot.
David
Reply to: