Re: [Nbd] how does NBD handle disconnections?

[Wouter Verhelst <w@...112...>]

On Wed, May 16, 2007 at 02:00:44PM +0200, Tomasz Chmielewski wrote:
> How does NBD handle disconnections?

Not very well.

> For example, we have a nbd device mounted on a client, the client 
> writes, and we kill a server for 15 minutes.
> 
> What will the client do?

Scream and yell. And die a horrible death.

> Will the process accessing the nbd disk just freeze for 15 minutes 
> (waiting for IO), and happily continue once nbd client can reconnect to 
> the server?

This is the behaviour that I would _want_ to occur; but because of some
implementation specifics in the kernel currently, the actual behaviour
is that the kernel will treat such a device as one that hasn't been
connected yet; i.e., you get something like this in dmesg once something
tries to write to the device:

nbd0: Attempted send on closed socket
end_request: I/O error, dev nbd0, sector 0

Proper reconnects aren't possible, either. The client now has a
'-persist' option which will prompt it to try to reconnect immediately
when the device is disconnected; but as the kernel doesn't lock the
device, that still gives you a (short) window during which I/O errors,
as above, may occur.

I discussed this with the NBD kernel module's maintainer, and the idea
is now that the NBD kernel device will be modified so that it will
indeed lock a device for as long as a nbd-client userspace process has
the device node open. This will remove the race described above, though
"killing the server for fifteen minutes" (I assume for a reboot or so?)
will still be problematic (but then at least fixable). However that
hasn't been done yet, AFAIK.

Obviously, if you're interested in working on that, you'd be welcome.
AAUI, this shouldn't be all that hard for someone knowledgeable about
kernel internals; but since I'm not such a person, well.

-- 
Shaw's Principle:
	Build a system that even a fool can use, and only a fool will
	want to use it.