Re: backup of backup or alternating backups?
On 2024-10-08, Andy Smith wrote:
> When you have hundreds of millions of files in rsnapshot it really
> starts to hurt because every backup run involves:
>
> - Deleting the oldest tree of files;
rsnapshot can rename it apart and delete it after backup is done. Thus
involving only the backup system
> - Walking the entire tree of the most recent backup once to cp -l it and
> then;
rsnapshot only renames directories when rotating backups then does rsync
with hard links to the newest
> This rsnapshot I have is really quite slow with only two 7200rpm HDDs.
> It spends way longer walking its data store than actually backing up any
> data. I could definitely make it speedier by switching to something
> else. But I like rsnapshot for this particular case.
On 7200rpm HDDs I was using xfs over RAID1 and the slowest/blocking part
was the deletion
> Although it probably matters most how many files you have only in the
> most recent backup iteration rather than the entire rsnapshot store. For
> me that is approx 5.8 million.
I don't remember but I should have been around your volume.
rsync uses metadata so it also depends on the filesystem. Some are
quicker. I think metadata is quite like the index used by other backup
systems.
Reply to: