Re: [Nbd] NBD, sparse files & TRIM

[Paul Clements <paul.clements@...856...>]

On Wed, Dec 29, 2010 at 10:32 AM, Wouter Verhelst <w@...112...> wrote:
> Hi Folkert,
>
> (List added)
>
> On Tue, Dec 28, 2010 at 04:23:06PM +0100, folkert wrote:
>> Hi,
>>
>> Please consider adding TRIM support to NBD.
>
> I'd been thinking that would be a great option, too; it's just that I
> wasn't aware of a userspace interface to tell the kernel to revert a bit
> of a file back to a sparse state.
>
>> Since XFS can "punch hole in a file" and it seems other Linux
>> filesystems may be able to do so too in the near future (see
>> http://lwn.net/Articles/415889/ ) this can be nicely integrated.
>
> This of course fixes that problem, and I'd be happy to do something like
> that.
>
> It would, however, require an update of the protocol, with an extra
> packet to allow the client to communicate to the server that a given
> region of the device is not needed anymore.
>
> Since nbd 2.9.18, I've changed the handshake protocol a little, which
> allows me to more easily provide backwards compatibility with previous
> versions of NBD, even as I add options. So I could imagine the
> following:
>
> - The server sets a flag if it supports a TRIM command. "Supports" can
>  be interpreted fairly broadly here: it might know about the command,
>  but also know that the filesystem on which we're running does not
>  support sparse files; or it might be compiled on a system (FreeBSD or
>  some such) that doesn't have the fallocate() option in question. In
>  such cases, nbd-server might not set the flag.
> - If set, nbd-client performs another ioctl() to communicate to the
>  kernel that the TRIM command is supported. Since this is just a flag
>  we're setting, I suggest a somewhat generic NBD_SET_OPTS ioctl which
>  takes a bitmask or something (so that it can easily be extended in the
>  future, should the need arise).

Yes, sounds reasonable.

> - The kernel can now issue TRIM packets.
>
> The packet itself could be something like:
>
> .type: 3 (NBD_CMD_TRIM, or perhaps NBD_CMD_DELETE, doesn't really matter
> all that much)
> .from: offset to trim
> .len: length of stuff to trim

Maybe NBD_CMD_DISCARD, since that's the block layer name for it. (TRIM
is ATA, PUNCH is SCSI).

Let me put this on my todo list. It shouldn't take too long to code up.

--
Paul

> Thoughts?
>
> --
> The biometric identification system at the gates of the CIA headquarters
> works because there's a guard with a large gun making sure no one is
> trying to fool the system.
>  http://www.schneier.com/blog/archives/2009/01/biometrics.html
>