Re: Limitations of rsnapshot-style backups (Was: Re: lazy old guy asks question)
Hi,
On Tue, Sep 02, 2025 at 09:05:39AM -0400, Dan Ritter wrote:
> David Christensen wrote:
> > a. Set the ZFS backup file system property "dedup". This will enable
> > block-level de-duplication, which can de-duplicate data more than hard links
> > alone.
>
> This is generally not a good thing to recommend; one of the
> authors of the system wrote a good article which should
> definitely be read before turning on dedup:
>
> https://despairlabs.com/blog/posts/2024-10-27-openzfs-dedup-is-good-dont-use-it/
I evaluated zfs dedup with my real data as part of deciding what to do,
and it became clear I would need to significantly increase the hardware
I was dedicating to the task and I would also need to re-think the
remote places I am storing further copies (for even more expense).
Having said that, there were other factors in my decision. The
cross-source dedup is not a huge factor. That is, dedup done on all data
across all hosts being backed up isn't amazing. My figures show that 1.6
TB in rsnapshot came out as 920 GB in restic. If we assume that turning
off zfs dedup loses the dedup between backuo sources, but the
snapshotting continues to allow only diffs within the backups for each
source to be stored, then it's less than double the capacity needed. And
again, as I said, capacity isn't so difficult.
So yeah in summary, probably don't even consider zfs dedup but
do consider zfs.
Thanks,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
Reply to: