Re: else or Debian (Re: ZFS performance (was: Re: deduplicating file systems: VDO with Debian?))
On Thu, 2022-11-10 at 22:37 +0100, Linux-Fan wrote:
> hw writes:
>
> > On Wed, 2022-11-09 at 19:17 +0100, Linux-Fan wrote:
> > > hw writes:
> > > > On Wed, 2022-11-09 at 14:29 +0100, didier gaumet wrote:
> > > > > Le 09/11/2022 à 12:41, hw a écrit :
>
> [...]
>
> > > > I'd
> > > > have to use mdadm to create a RAID5 (or use the hardware RAID but that
> > > > isn't
> > >
> > > AFAIK BTRFS also includes some integrated RAID support such that you do
> > > not necessarily need to pair it with mdadm.
> >
> > Yes, but RAID56 is broken in btrfs.
> >
> > > It is advised against using for RAID
> > > 5 or 6 even in most recent Linux kernels, though:
> > >
> > > https://btrfs.readthedocs.io/en/latest/btrfs-man5.html#raid56-status-and-recommended-practices
> >
> > Yes, that's why I would have to use btrfs on mdadm when I want to make a
> > RAID5.
> > That kinda sucks.
> >
> > > RAID 5 and 6 have their own issues you should be aware of even when
> > > running
> > > them with the time-proven and reliable mdadm stack. You can find a lot of
> > > interesting results by searching for “RAID5 considered harmful” online.
> > > This
> > > one is the classic that does not seem to make it to the top results,
> > > though:
> >
> > Hm, really? The only time that RAID5 gave me trouble was when the hardware
>
> [...]
>
> I have never used RAID5 so how would I know :)
>
> I think the arguments of the RAID5/6 critics summarized were as follows:
>
> * Running in a RAID level that is 5 or 6 degrades performance while
> a disk is offline significantly. RAID 10 keeps most of its speed and
> RAID 1 only degrades slightly for most use cases.
It's fine when they pay for the additional disks required by RAID1 or 10.
> * During restore, RAID5 and 6 are known to degrade performance more compared
> to restoring one of the other RAID levels.
When that matters, don't use this RAID level. It's not an issue about keeping
data save.
> * Disk space has become so cheap that the savings of RAID5 may
> no longer rectify the performance and reliability degradation
> compared to RAID1 or 10.
When did that happen? Disk space is anything but cheap and the electricty
needed to run these disks is anything but cheap. Needing more disks for the
same capacity is anything but cheap because a server can fit only so many disks,
and the more disks you put into a server, the more expensive that server gets.
You might even need more server, further increasing costs for hardware and
electricity.
> All of these arguments come from a “server” point of view where it is
> assumed that
>
> (1) You win something by running the server so you can actually
> tell that there is an economic value in it. This allows for
> arguments like “storage is cheap” which may not be the case at
> all if you are using up some thightly limited private budget.
You're not gona win anything when your storage gets too expensive.
> (2) Uptime and delivering the service is paramount.
That can get expensive very quickly.
> Hence there
> are some considerations regarding the online performance of
> the server while the RAID is degraded and while it is restoring.
> If you are fine to take your machine offline or accept degraded
> performance for prolonged times then this does not apply of
> course.
Sure, when you have issues like that, find a different solution.
> If you do not value the uptime making actual (even
> scheduled) copies of the data may be recommendable over
> using a RAID because such schemes may (among other advantages)
> protect you from accidental file deletions, too.
Huh?
> Also note that in today's computing landscape, not all unwanted file
> deletions are accidental. With the advent of “crypto trojans” adversaries
> exist that actually try to encrypt or delete your data to extort a ransom.
>
When have unwanted file deletions been exclusively accidential?
> > More than one disk can fail? Sure can, and it's one of the reasons why I
> > make
> > backups.
> >
> > You also have to consider costs. How much do you want to spend on storage
> > and
> > and on backups? And do you want make yourself crazy worrying about your
> > data?
>
> I am pretty sure that if I separate my PC into GPU, CPU, RAM and Storage, I
> spent most on storage actually. Well established schemes of redundancy and
> backups make me worry less about my data.
Well, what did they say: Disk space has become cheap. Yeah, for sure ... :)
> I still worry enough about backups to have written my own software:
> https://masysma.net/32/jmbb.xhtml
> and that I am also evaluating new developments in that area to probably
> replace my self-written program by a more reliable (because used by more
> people!) alternative:
> https://masysma.net/37/backup_tests_borg_bupstash_kopia.xhtml
> >
cool :)
> [...]
> >
> > Is anyone still using ext4? I'm not saying it's bad or anything, it only
> > seems that it has gone out of fashion.
>
> IIRC its still Debian's default.
Hm, I haven't really used Debian in a long time. There's probably no reason to
change that. If you want something else, you can always go for it.
> Its my file system of choice unless I have
> very specific reasons against it. I have never seen it fail outside of
> hardware issues. Performance of ext4 is quite acceptable out of the box.
> E.g. it seems to be slightly faster than ZFS for my use cases.
> Almost every Linux live system can read it. There are no problematic
> licensing or stability issues whatsoever. By its popularity its probably one
> of the most widely-deployed Linux file systems which may enhance the chance
> that whatever problem you incur with ext4 someone else has had before...
>
I'm not sure it's most widespread. Centos (and Fedora) defaulted to xfs quite
some time ago, and Fedora more recently defaulted to btrfs (a while after Redhat
announced they would remove btrfs from RHEL altogether). Centos went down the
drain when it mutated into an outdated version of Fedora, and RHEL is probably
isn't any better.
So assuming that RHEL and Centos may be more widespread than Debian because
there's lots of hardware supporting those but not Debian, I wouldn't think that
ext4 is most widespread and xfs is more common until btrfs has replaced it.
> > I'm considering using snapshots. Ext4 didn't have those last time I
> > checked.
>
> Ext4 still does not offer snapshots.
awww
It's a shame that btfrs is going so dead slow that it might be replaced with
something new before ever getting there, leaving Linux without a fully featured
file system while we've been waiting on it for the last 10 years.
> The traditional way to do snapshots
> outside of fancy BTRFS and ZFS file systems is to add LVM to the equation
> although I do not have any useful experience with that.
Ugh. Don't even try it, LVM sucks badly. It works, but's unflexible to the
point of being not only completely useless but even detremental in practise.
The idea was to provide faster storage for VM images than files residing in a
file system. Well, screw that. I can copy and/or move a VM image in a file
just as easy as any other file even from one machine to another. Good luck
trying that with LVM. I lost a whole VM that way once. And are the files any
slower? It doesn't seem so.
Snapshots? IIRC only on the same LVM volume, and when that is full, you're out
of luck. It's a way to waste like 60% of your disk space because you have to
keep spare space in case you want to make another snapshot or VM. You could
make one with dd, but that's unwieldy.
> Specifically, I am
> not using snapshots at all so far, besides them being readily available on
> ZFS :)
Well, for me they seem to be a really good option for incremental backups :)
Reply to: