Re: [Nbd] tech documentation

[Wouter Verhelst <wouter@...3...>]

On Thu, Oct 19, 2006 at 07:29:19AM -0500, Phil Howard wrote:
> On Thu, Oct 19, 2006 at 01:09:25PM +0200, Wouter Verhelst wrote:
> | Err, SCTP is that protocol that provides connectionless reliable
> | networking, right? Okay, that would be a good idea, but it would still
> | require patches from someone :)
> 
> No.  That's RDP.
> 
> SCTP is not connectionless.  They just call it an association because
> the semantics are different than a TCP connection.  But in important
> ways, the association still has the value of a connection.  For example,
> when all processes holding the file descriptors close those descriptors
> or otherwise just go away, the system network stack will shutdown the
> association.  That's something UDP cannot do.
[...]
> It _MAY_ be of benefit to NBD (one of the reasons I wanted to know
> more about the protocol).  It can allow multiple requests that are
> order independent to be queued from the client to the server and not
> slow down all of them if one packet is lost, while the lost one is
> being recovered by the peer stacks.  Order is retained within a
> stream, but is not enforced between them.

Yes, that could be a benefit, though I'm not quite sure it would in fact
have a positive effect NBD througput speed: most NBD client requests are
to read or write 1024 bytes (which is not enforced anywhere, it's just
the default); there would probably be some additional overhead in
creating a new SCTP stream for each of those requests, I guess? That
overhead might outweigh the benefit you get from not having to stop the
TCP stream cold every time a packet gets lost. You could of course
increase the number of requests per stream, but that would require quite
some additional bookkeeping (not sure that's worth it), or increase the
request size (but then you increase the average amount of useless data
that gets pumped over the network, not very good for througput).

[...]
> | If anyone is going to do this, make sure it can be disabled at compile
> | time as well as at run time. Not every operating system supports SCTP
> | yet, and I imagine you may also want to have a server that supports both
> | SCTP and TCP clients at the same time. Also, patches would have to be
> | against svn trunk, not against whatever is the latest "stable" version
> | out there.
> 
> By all means there should be an option, not only in compiling the tools,
> but also in the kernel configuration.  There may also need to be an
> option at kernel config time, compile time, and/or run time to place an
> upper limit on the number of SCTP streams.  I cannot judge, yet, what
> a reasonable default would be.

Okay.

> FYI, I am planning to look into implementing an NBD client directly into
> the QEMU emulator in system mode.  This would allow an emulated system
> to have one or more block devices mapped onto the network block devices
> served from nbd-server anywhere without using NBD in the host kernel.
> And the nbd-server can even be on the same machine running QEMU without
> the deadlock issues (nbd-server runing in the guest OS under QEMU would,
> of course, still have those issues).  This is another reason I wanted to
> see technical protocol details.

Right. I hope the blog post I pointed you to is sufficient; there really
isn't much else.

-- 
<Lo-lan-do> Home is where you have to wash the dishes.
  -- #debian-devel, Freenode, 2004-09-22