Bug#832443: ITP: mergerfs -- mergerfs is a union filesystem geared towards simplifing storage
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Hi Konstantin,
On Mon, 2016-07-25 at 19:06 +0300, Konstantin Khomoutov wrote:
> > mergerfs is a union filesystem geared towards simplifing storage
>
> I supposed it should be "simplifying" in both cases.
>
THanks. Corrected in my packaging.
> > and management of files across numerous commodity storage devices. It
> > is similar to mhddfs, unionfs, and aufs.
> >
> > Some salient features include
> > * Runs in userspace (FUSE)
> > * Configurable behaviors
> > * Support for extended attributes (xattrs)
> > * Support for file attributes (chattr)
> > * Runtime configurable (via xattrs)
> > * Safe to run as root
> > * Opportunistic credential caching
> > * Works with heterogeneous filesystem types
> > * Handling of writes to full drives
>
> What does this mean? Drives with no more room to write data left?
mergefs allows union mount, created out of multiple drives.
Take this example
tmpfs -> /tmp-ram - 4G
/dev/sdb2 -> /tmp-disk - 10G
mergerfs can help create a unified mount:
/tmp-ram:/tmp-disk /tmp mergerfs
So, for a 5 GiB file, mergerfs would intelligently write it to the block based
backing device.
Here are some stats:
rrs@learner:~$ df -h /tmp
Filesystem Size Used Avail Use% Mounted on
tmp-ram:media/SSHD/tmp-disk:tmp-disk 483G 415G 64G 87% /tmp
2016-07-25 / 22:25:24 ♒♒♒ ☺
rrs@learner:~$ for x in 1 2 3 4 5 6 7 8 9; do dd if=/dev/zero of=/tmp/$x.img
bs=1M count=500; done
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 2.55788 s, 205 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.6153 s, 325 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 8.25114 s, 63.5 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.84161 s, 285 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 2.11159 s, 248 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 14.8864 s, 35.2 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.77783 s, 295 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 1.74245 s, 301 MB/s
500+0 records in
500+0 records out
524288000 bytes (524 MB, 500 MiB) copied, 14.1321 s, 37.1 MB/s
2016-07-25 / 22:26:57 ♒♒♒ ☺
The actual data is internally moved to the optimal backing device, based on
policies explained in the following link, under section POLICIES
https://github.com/trapexit/mergerfs
rrs@learner:~$ ls /tmp
1.img
2.img
3.img
4.img
5.img
6.img
7.img
8.img
9.img
.....snipped.....
2016-07-25 / 22:28:25 ♒♒♒ ☺
rrs@learner:~$ ls /media/SSHD/tmp-disk/
1.img 9.img
2.img
3.img
4.img
5.img
6.img
7.img
8.img
.....snipped.....
2016-07-25 / 22:28:33 ♒♒♒ ☺
rrs@learner:~$ ls /tmp-ram/
2016-07-25 / 22:28:43 ♒♒♒ ☺
We get good throughput and at the same time efficiency.
This is just one of the many use cases.
- --
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-----BEGIN PGP SIGNATURE-----
iQIcBAEBCgAGBQJXlkZWAAoJEKY6WKPy4XVpe0YQALgKImgpU3OM3z51JjvLRu5a
bWKlspcMC1AhPmvvlHlyHUjkPIzWsJKsOexlslwwExjW6S6TTXoeHfEkfp8Xua23
aC1N1xbI6tvB6pEAfnijSA85SQ0QguMxmRNrXmDVJ6LyXTaU3yewaWIrjkXoJw8R
E/RKssxTwPKvAc5fd1hmcMWxBKEL+kJO1JmrDmbNdgWn01rseSghnd0cr+A1HVuE
YebysE5qkvjet2lruqR80jVXH6DJJnzl6cBI5Te1+dksk9BHCkuz0IP2lbM1Z7Bq
e520E5S2Tx6A+Knmtssg3G48zYWhw0DH78/DyNo+7UTcx8GC1TKKKW/tQgDdm77H
jBzpucnF4jb8Ike67czdEYif76sC6xne+IUWHCuUPJCqVENMOG1TN69qCWrh8F0f
0tK8qR0tM7g9n3A2Tx7BG1J5hVFiMcMONUzfwH+vXPnVip9y3CDvwzwk5vIsifBV
kRFWAZstAI04TQlaE2DU7AFMWi4WHh06ScpaWOL2eNT4wSDNU6JlxdF0msKNI5A+
aoKkEEOoI4u85SFbttqPdB1PiBtTClv8mB68ollw4MzUcKbmF5byR56KeCUsT0Wo
NaaXlLiMBEPquqaRHac+3DTRemZkoSQGQyLhXBahra6RN5d2SE7WMeJMvBcc8fGr
MFgEdiU20rUSZGYmmhwY
=wIOu
-----END PGP SIGNATURE-----
Reply to: