[Freedombox-discuss] Tahoe-LAFS is not a filesystem
Folks:
I'm a developer on the Tahoe-LAFS project (https://tahoe-lafs.org )
and I would be happy to see it used in Freedombox. The projects have
many goals in common. I'm also the founder of a startup that sells
Tahoe-LAFS ciphertext storage service: https://leastauthority.com .
I'm just writing this brief note to try to explain something -- Tahoe,
the Least-Authority File System fits better for file sharing,
streaming or network-attached-storage than it does for a normal, local
filesystem.
It should probably be moved from here, where it is currently listed
next to MooseFS:
http://wiki.debian.org/FreedomBox/ExampleProjects#Filesystems
To here:
http://wiki.debian.org/FreedomBox/ExampleProjects#File_sharing
or maybe here or here or here:
http://wiki.debian.org/FreedomBox/ExampleProjects#Streaming
http://wiki.debian.org/FreedomBox/ExampleProjects#Network_Attached_Storage
Where it can be listed next to things like ownCloud, SparkleShare,
BitTorrent, Tin Can Jukebox, etc.
The confusion is that Tahoe-LAFS *is* a file system, in a couple of
senses. It has arbitrarily nestable directories and files (unlike a
lot of newfangled storage systems), and it has an SFTP server, which
means you can point sshfs at it and have a reasonably good access to
it directly in your kernel VFS. You could also use PyFilesystem to
integrate it into your local filesystem.
*But*, while the compatibility and correctness of Tahoe-LAFS and its
SFTP server are pretty good, its performance characteristics as a
distributed, secure, fault-tolerant storage system are sometimes a bad
match for the expectations of user-space programs that access their
storage through the POSIX filesystem API. Some things work fine -- it
depends on your specific use case, but some access patterns that are
normal and efficient on a local filesystem like ext4 are painfully
slow or even infeasible on Tahoe-LAFS. For example, opening a file for
append, appending a few bytes, and then closing it again, and then
doing this over and over 10,000 times, will take about O(N) time and
O(N) space on ext4 (about 10,000 disk operations and about 10,000
units of storage on disk), but take about O(N**2) time and O(N**2)
space on Tahoe-LAFS (about 100,000,000 network operations and about
100,000,000 units of storage on the backend), which means the program
that does something like that will never finish and might use up a lot
of your secure cloud storage space.
Everybody wants to use Tahoe-LAFS through FUSE as though it were a
local filesystem, but nobody actually uses Tahoe-LAFS through FUSE, to
my knowledge. All the people who actually Tahoe-LAFS end up using it
less like it is a "filesystem" like ext4 and more like it is an
"application" for backing up, sharing, or hosting files.
(N.B. the tahoe-lafs developers actively try to support whatever it is
that our users demand, and so we continue to support and improve its
behavior when it is treated as though it were a local filesystem, even
though I don't exactly recommend it.)
Regards,
Zooko
Reply to: