[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Ideas for object-based git-like storage on Linux



Roger Leigh wrote:
> There are lots of Debian people out there using git, and some of them
> have expressed interest over the years in having the ability to use
> git as a filesystem in its own right (#477942 is an example of one
> in a package I maintain).
> 
> I've finally got down to it and written all my thoughts on the topic
> down in a mostly-organised form, which you can find at
> 
>   http://www.codelibre.net/~rleigh/hashlink.pdf
> 
> This paper looks at the concept of object-based storage, and the
> creation of "hashlinks", essentially symlinks which use hashes
> rather than pathnames to refer to a file.

You may be interested in my git-annex program, which implements just
such a thing, although in user space, not kernel space.
http://git-annex.branchable.com/

joey@gnu:~/lib/sound/misc>ls -l dj_mooch_-_misc_-_01_-_dirty_as_mud.mp3
lrwxrwxrwx 1 joey joey 113 Nov  9 14:09 dj_mooch_-_misc_-_01_-_dirty_as_mud.mp3 -> ../.git/annex/objects/SHA1:717566db4265b4b3a986ba84e797df56c25923be/SHA1:717566db4265b4b3a986ba84e797df56c25923be

Note that once you have a hashlink, you can know that if another
filesystem elsewhere has the same hashlink, the content can be retrieved
from there. So the deduplication and general content management can
be done cross-filesystem. That is a basis of much of the good stuff
git-annex can do. For example:

joey@gnu:~/lib/big/new>ls -l debian-6.0.0-amd64-i386-netinst.iso
lrwxrwxrwx 1 joey joey 145 Feb  6 14:10 debian-6.0.0-amd64-i386-netinst.iso -> ../.git/annex/objects/WORM:1296961374:432142336:debian-6.0.0-amd64-i386-netinst.iso/WORM:1296961374:432142336:debian-6.0.0-amd64-i386-netinst.iso
joey@gnu:~/lib/big/new>git log --pretty=oneline debian-6.0.0-amd64-i386-netinst.iso
97dc2f3a5b5aec7671c7f282b8ea42a89f90b44a pre passport-sync
joey@gnu:~/lib/big/new>cat debian-6.0.0-amd64-i386-netinst.iso
cat: debian-6.0.0-amd64-i386-netinst.iso: No such file or directory
joey@gnu:~/lib/big/new>git annex get debian-6.0.0-amd64-i386-netinst.iso
get debian-6.0.0-amd64-i386-netinst.iso (copying from turtle...) 


Also, Josh Triplett has some ideas about integrating git with dpkg for
fully versioned systems. I'm not sure if he's ready to make them
public, but you may find talking with him interesting.

-- 
see shy jo

Attachment: signature.asc
Description: Digital signature


Reply to: