Re: Storage server
Am Sonntag, 16. September 2012 schrieb Stan Hoeppner:
> On 9/16/2012 7:38 AM, Martin Steigerwald wrote:
> > I have always recommended to leave at least 10-15% free, but from a
> > discussion on XFS mailinglist where you took part, I learned that
> > depending on use case for large volumes even more free space might be
> > necessary for performant long term operation.
>
> And this is due the allocation group design of XFS. When the
> filesystem is used properly, its performance with parallel workloads
> simply runs away from all other filesystems. When using LVM in the
> manner I've been discussing, the way the OP of this thread wants to
> use it, you end up with the following situation and problem:
>
> 1. Create 1TB LVM and format with XFS.
> 2. XFS creates 4 allocation group
> 3. XFS spreads directories and files fairly evenly over all AGs
> 4. When the XFS gets full, you end up with inode/files/free space
> badly fragmented over the 4 AGs and performance suffers when
> reading these back, or when trying to write new, or modify existing 5.
> So you expand the LV by 1TB and then grow the XFS over the new space
> 6. This operation simply creates 4 new AGs in the new space
> 7. New inode/extent creation to these new AGs is fast and reading back
> is also fast.
> 8. But, here's the kicker, reading the fragmented files from the first
> 4 AGs is still dog slow, as well as modifying metadata in those AGs
>
> Thus, the moral of the story is that adding more space to an XFS via
> LVM can't fix performance problems that one has created while reaching
> the "tank full" marker on the original XFS. The result is fast access
> to the new AGs in the new LVM sliver, but slow access to the original
> 4 AGs in the first LVM sliver. So as one does the LVM rinse/repeat
> growth strategy, one ends up with slow access to all of their AGs in
> the entire filesystem. Thus, this method of "slice/dice" expansion
> for XFS is insane.
>
> This is why XFS subject matter experts and power users do our best to
> educate beginners about the aging behavior of XFS. This is why we
> strongly recommend that users create one large XFS of the maximum size
> they foresee needing in the long term instead of doing the expand/grow
> dance with LVM or doing multiple md/RAID reshape operations.
>
> Depending on the nature of the workload, and careful, considerate,
> judicious use of XFS grow operations, it is safe to grow an XFS without
> the performance problems. This should be done long before one hits the
> ~90% full mark. Growing before it hit ~70% is much better. But one
> should still never grow an XFS more than a couple of times, as a
> general rule, if one wishes to maintain relatively equal performance
> amongst all AGs.
Thanks for your elaborate explaination.
I took note of this for my Linux Performance analysis & tuning trainings.
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7
Reply to: