Re: Storage server
On 20120910_053746, Stan Hoeppner wrote:
> On 9/9/2012 3:25 PM, Paul E Condon wrote:
>
> > I've been following this thread from its beginning. My initial reading
> > of OP's post was to marvel at the thought that so many things/tasks
> > could be done with a single box in a single geek's cubicle.
>
> One consumer quad core AMD Linux box of today can do a whole lot more
> than what has been mentioned.
>
> > I resolved
> > to follow the thread that would surely follow closely. I think you,
> > Stan, did OP an enormous service with your list of questions to be
> > answered.
>
> I try to prevent other from shooting themselves in the foot when I see
> the loaded gun in their hand.
>
> > This thread drifted onto the topic of XFS. I first learned of the
> > existence of XFS from earlier post by you, and I have ever since been
> > curious about it. But I am retired, and live at home in an environment
> > where there is very little opportunity to make use of its features.
>
> You might be surprised. The AG design and xfs_fsr make it useful for
> home users.
>
> > Perhaps you could take OP's original specification as a user wish list
> > and sketch a design that would fulfill the wishlist and list how XFS
> > would change or resolve issues that were/are troubling him.
>
> The OP's issues don't revolve around filesystem choice, but basic system
> administration concepts.
>
> > In particular, the typical answers to questions about backup on this list
> > involve rsync, or packages that depend on rsync, and on having a file
> > system that uses inodes and supports hard links.
>
> rsync works with any filesystem, but some work better with rsync
> workloads. If one has concurrent rsync jobs running XFS is usually best.
Rsync features that invoke hard links are commonly used to do
de-duplication in backup systems that are designed with extX file
system in mind. Other parts of rsync work without hardlinks in the
file system. But, I think a common desire of people seeking advice
here is that there be some sort of automatic, easy to administer,
de-duplication.
>
> > How would an XFS design
> > handle "de-duplication"?
>
> Deduplication isn't an appropriate function of a filesystem.
The wording of this question was too terse. I should have said
something like:
How would a backup system design that uses XFS implement
de-duplication?
I know that file-systems don't do de-duplication, but the rsysc
program does do de-duplication in the case of extX file system. What
alternative method for achieving de-duplication might be substituted
for rsync?
>
> > Or is de-duplication simply a bad idea in very
> > large systems?
>
> That's simply a bad, very overly broad question.
Yes, but de-duplication is a feature that it highly touted as a "good
thing". Is there some easy way to have de-duplication *and* the
benefits of XFS in a single, optimized design of a backup system?
--
Paul E Condon
pecondon@mesanetworks.net
Reply to: