Re: ZFS performance (was: Re: deduplicating file systems: VDO withDebian?)
On Fri, 2022-11-11 at 21:26 +0100, Linux-Fan wrote:
> hw writes:
>
> > On Thu, 2022-11-10 at 23:05 -0500, Michael Stone wrote:
> > > On Thu, Nov 10, 2022 at 06:55:27PM +0100, hw wrote:
> > > > On Thu, 2022-11-10 at 11:57 -0500, Michael Stone wrote:
> > > > > On Thu, Nov 10, 2022 at 05:34:32PM +0100, hw wrote:
> > > > > > And mind you, SSDs are *designed to fail* the sooner the more data
> > you
> > > > > > write
> > > > > > to
> > > > > > them. They have their uses, maybe even for storage if you're so
> > > > > > desperate,
> > > > > > but
> > > > > > not for backup storage.
>
> [...]
>
> > Why would anyone use SSDs for backups? They're way too expensive for that.
>
> I actually do this for offsite/portable backups because SSDs are shock
> resistant (dont lose data when being dropped etc.).
I'd make offsite backups over internet. If you can afford SSDs for backups,
well, why not.
> The most critical thing to acknowledge about using SSDs for backups is that
> the data retention time of SSDs (when not powered) is decreasing with each
> generation.
Do you mean each generation of SSDs or of backups? What do manufacturers say
how long you can store an SSD on a shelf before the data on it has degraded?
> Write endurance has not become critical in any of my SSD uses so far.
> Increasing workloads have also resulted in me upgrading the SSDs. So far I
> always upgraded faster than running into the write endurance limits. I do
> not use the SSDs as caches but as full-blown file system drives, though.
I use them as system disks because they don't mind being switched off and on and
partly because they don't need as much electricity as hard disks. If it wasn't
for that, I'd use hard disks for system disks. I don't use them for storage,
they're way too small and expensive for that. Fortunately, system disks can be
small; the data is on the server anyway.
There are exceptions, like hard drives suck for laptops and SSDs are much better
for them, and things that greatly benefit from low latencies, lots of IOOPs,
high transfer rates.
> On the current system, the SSDs report having written about 14 TB and are
> specified by the manufacturer for an endurance of 6300 TBW (drive size is 4
> TB).
Wow you have expensive drives.
> The small (drive size about 240GB) ones I use for backup are much less
> durable.
You must have quite a lot of them. That gets really expensive.
> For one of them, the manufacturer claims 306TBW, the other has
> 360 TBW specified. I do not currently know how much data I have written to
> them already. As you can see from the sizes, I backup only a tiny subset of
> the data to SSDs i.e. the parts of my data that I consider most critical (VM
> images not being among them...).
Is that because you have them around anyway because they were replaced with
larger ones, or did you actually buy them to put backups on them?
> [...]
>
> > There was no misdiagnosis. Have you ever had a failed SSD? They usually
> > just
> > disappear. I've had one exception in which the SDD at first only sometimes
> > disappeared and came back, until it disappeared and didn't come back.
>
> [...]
>
> Just for the record I recall having observed this once in a very similar
> fashion. It was back when a typical SSD size was 60 GiB. By now we should
> mostly be past this “SSD fails early with controller fault” issues. It can
> still happen and I still expect SSDs to fail with less notice compared to
> HDDs.
Why did they put bad controllers into the SSDs?
> When I had my first (and so far only) disk failure (on said 60G SSD) I
> decided to:
>
> * Retain important data on HDDs (spinning rust) for the time being
>
> * and also implement RAID1 for all important drives
>
> Although in theory running two disks instead of one should increase the
> overall chance of having one fail, no disks failed after this change so
> far.
I don't have any disks that aren't important. Even when the data can be
recovered, it's not worth the trouble not to use redundancy. I consider
redundacy a requirement, there is no storing anything on a single disk. I'd
only tolerate it for backups when there are multiple backups when it can't be
avoided. Sooner or later, a disk will fail.
Reply to: