Re: Ideas for object-based git-like storage on Linux
Roger Leigh <rleigh@codelibre.net> writes:
> On Sun, Feb 06, 2011 at 10:23:00PM -0400, Joey Hess wrote:
>> Roger Leigh wrote:
>> > There are lots of Debian people out there using git, and some of them
>> > have expressed interest over the years in having the ability to use
>> > git as a filesystem in its own right (#477942 is an example of one
>> > in a package I maintain).
>> >
>> > I've finally got down to it and written all my thoughts on the topic
>> > down in a mostly-organised form, which you can find at
>> >
>> > http://www.codelibre.net/~rleigh/hashlink.pdf
>> >
>> > This paper looks at the concept of object-based storage, and the
>> > creation of "hashlinks", essentially symlinks which use hashes
>> > rather than pathnames to refer to a file.
>>
>> You may be interested in my git-annex program, which implements just
>> such a thing, although in user space, not kernel space.
>> http://git-annex.branchable.com/
>
> I've been meaning to give git-annex a whirl for a while, but I'm
> afraid I've lacked the time to get intimately acquainted with it.
> From what I understand, in terms of what a single user could do with
> it, it's looking pretty equivalent, the major difference being that
> it's entirely in userspace. It's definitely on my TODO list.
>
> I wanted to look at if it was possible to make multi-user store and
> provide a lightweight kernel interface to access it. It might well
> be possible to use git-annex as the storage backend for an initial
> implementation!
All you need is a little bit of fuse code. There really is no need to
invent a new kernel interface for this and something like this is best
kept in userspace to keep the complexity managable.
> Following the suggestion to look at log structured filesystems, I
> took a look at things like Plan9's Venti. It's good stuff; my main
> objection to most being that they are append-only, with no provision
> for GC of no longer referenced data. I would consider that a
> requirement for a general purpose store with rapid turnover of data,
> especially if you're going to store working copies as well as things
> of "commit quality", and even things you commit can be temporary for
> rebasing etc.
>
>
> Regards,
> Roger
MfG
Goswin
Reply to: