Bzzzz wrote:
> Bob Proulx wrote:
> > completely out of disk space. For that the reasonable amount of
> > disk space reserved is an absolute value that a system might need
> > on that partitions. That part really shouldn't be a percentage of
> > the disk but should be a finite reserved amount.
>
> It isn't very important as tune2fs accepts decimals for -m,
> so you can easily reduce the original %age (eg: on a 39GB
> partition, -m 0.1 gives 9488 reserved blocks).
> You can also use the -r switch, specifying the exact number
> of reserved blocks you want.
Sure. But the question was about using a 5% or smaller minfree.
> > The other is more subtle to understand. In the "old" days of
> > spinning disks the allocation algorithm will try to defrag files
> > on the fly by allocating them appropriately. That algorithm needs
> > a certain percentage of disk space free to use scattered
> > throughout the drive. For that algorithm it really should be a
> > percentage. For that algorithm people would benchmark the system
> > performance and determine a good "knee" in the performance curve
> > at various amounts of disk fullness. The knee in the curve would
> > usually occur somewhere around the 5% free amount. Therefore
> > setting it to 10% would guarentee good performance. Setting it to
> > 5% would allow more use of space on bigger disks but keep
> > performance from getting too bad.
>
> However, these blocks won't be free for use except if you use
> a defrag pgm, such as e2defrag.
What do you mean when you say "these blocks won't be free ..." without
defragmenting? Please explain. If you have references to share that
explained the details that would be great.
Also as I understand it use of e2defrag is not recommended. Using
e2defrag "may destroy data, depending on the feature bits turned on in
the filesystem; it does not know how to treat many of the newer ext3
features."
http://en.wikipedia.org/wiki/Ext3#Defragmentation
For the newer file systems like ext3, ext4 and others they perform
both delayed allocation and block reservation. Those are techniques
used by the file system to reduce fragmentation. It doesn't eliminate
it but it is greatly reduced as long as there is sufficient free disk
space already (a reason for a reasonable minfree) such that the
operating system can avoid fragmentation.
https://en.wikipedia.org/wiki/File_system_fragmentation#Preventing_fragmentation
Which is one of the reasons why performance is reduced when disks get
to be very full. And as such it is one of the reasons that keeping a
reasonable minfree is a good thing. Because it enforces operating at
a less than full disk.
But fragmentation is much more of an issue with spinning media. I
haven't fully pre-computed how this is changed with SSDs but
fragmentation should be much less of an issue with an SSD.
> 100% - 5% = 95% == 100% usable, so if you reach 100% (of the 95%)
> you're pwned except if you free the remaining sectors.
Please explain how having a full disk (which reduces performance and
increases fragmentation) causes your machine to be "pwned".
> > Of course now with SSDs that standard thinking needs to be thought
> > out again. I haven't seen any benchmark data for full SSDs. I
> > imagine that it will have much flatter performance curves up to
> > very full on an SSD. It would super awesome if someone has
> > already done this performance benchmarking and would post a link
> > to it so that we could all learn from it.
>
> There is no difference, as the embedded logic first gather
> the whole sectors list before any operation takes place (not
> exactly, but as the difference is counted in ns...)
I am not sure but I think you might be talking about delayed
allocation and block reservation.
> > So my thinking is that if it is a 3T spinning hard drive then I
> > would still keep minfree at 5% (or 10%) for reasons of performance
> > until and unless I see benchmark data showing otherwise. For any
> > size of SSD I think it would be okay to reduce that to any smaller
> > percentage that still reserved at least 500M (my best guess, may
> > need a better guess) of disk space for the system to operate for
> > log files and temporary files and other normal continuous activity.
>
> EXT4 has more protections about that:
> https://www.redhat.com/archives/ext3-users/2009-January/msg00026.html
Thank you for that reference. It has good information that I will
repeat here. And due to this author I would consider it authoritative
for ext4 file systems. It directly addresses the original question
that started this thread.
Theodore Tso wrote to ext3-users mailing list on 23 Jan 2009:
Alex Fler wrote:
> On large FS like 100gb default value of "Reserved block count" takes
> 5% of usable disk, can this value be safely changed to 1% and not
> affect a performance ? Is a reservation size of 1gb enough for 100gb
> disk ? And when we have even larger filesystem like 1Tb default
> "Reserved block count" is 50GB, is it an absolutely minimum must
> have reserved number of space for disk performance, or it's just a
> legacy concept which can be adjusted?
If you set the reserved block count to zero, it won't affect
performance much except if you run for long periods of time (with lots
of file creates and deletes) while the filesystem is almost full
(i.e., say above 95%), at which point you'll be subject to
fragmentation problems. Ext4's multi-block allocator is much more
fragmentation resistant, because it tries much harder to find
contiguous blocks, so even if you don't enable the other ext4
features, you'll see better results simply mounting an ext3 filesystem
using ext4 before the filesystem gets completely full.
If you are just using the filesystem for long-term archive, where
files aren't changing very often (i.e., a huge mp3 or video store), it
obviously won't matter.
That question and answer parallels the original poster's question.
That makes perfect sense. Setting the reserved block count has no
immediate effect since it is simply a variable that holds a value.
But it does allow operating at lower disk free values.
If you then go ahead and operate for long periods of time at very full
disk levels then you will be subject to fragmentation problems. The
above answer still specifies above 95% as almost full and likely to
trigger fragmentation problems.
But for example if you were to write one large file you wouldn't. Or
write a large file and soon delete it then you wouldn't. There are
lots of ways to use the file system briefly that won't cause problems.
It is the continued operation with only small amounts of free space
when it is hard for the file system to avoid fragmentation. There
will be many files written in parallel since it is a multitasking
system and there will always be log files and other activity.
That answer from tytso was in 2009 and it would still be useful and
interesting to see how using recent SSD storage would change our
pre-compiled operating model.
Bob
Attachment:
signature.asc
Description: Digital signature