Re: [Nbd] Starting over?

[Wouter Verhelst <w@...112...>]

On Wed, Jun 03, 2015 at 09:32:40PM +0100, Alex Bligh wrote:
> 
> On 3 Jun 2015, at 17:24, Wouter Verhelst <w@...112...> wrote:
> 
> > What I think is the main issue with nbd-server, though, is the fact that
> > it uses the stack as a state machine. That is, we read out one request
> > from the network (e.g., a read request), call a function to handle it
> > (expread), which checks if we do copy-on-write and calls rawexpread,
> > which does a seek and reads data into a buffer, and then we unwind the
> > stack and eventually (in mainloop()) we send the reply to the client.
> > 
> > Instead, it should be refactored so that it reads a request into a
> > buffer, and returns to mainloop if we block for I/O, so that we can
> > handle multiple requests at the same time.
> 
> Yeah I rewrote negotiation as a state machine (the idea being to publish
> it under something more liberal than GPL) and it was truly fugly.

My dabbling came to the same conclusion.

But I think that's okay. We can keep negotiation as a blocking thing --
we need do it only once, anyway -- and implement a state machine for the
data pushing phase. The "negotiation" would be a single state in the
whole state machine, then.

> I agree using the stack for state (as you put it) is not nice. But
> if the idea is merely to handle simultaneous requests, threading
> might be an alternative route. IE have one thread handling the
> socket, and a bunch of workers handling the requests.

My dabbling was going down the "start one thread per request" route. It
became complicated because of my misunderstanding that a FLUSH needed to
ensure that all outstanding requests have flushed; but since it's clear
now that that was a misunderstanding, I suppose we can go down that
road easily.

> This would be trivial in golang.

Sure, but golang is portability--; x86* and arm* only, AFAICS, whereas C
works everywhere. There's also a GThreadPool API in glib, which
simplifies things a bit. We'd just have to do this:

main thread:
- read request off socket
- if request.length > 0, read data
- g_thread_pool_push(pool, buffers, &error)

in the thread
- get buffers (we have a pointer to those buffers as a function
  argument)
- handle request (when we need to read or write, use pread() or
  pwrite() so multiple threads can safely deal with the same file at
  different offsets)
- lock a mutex (for the socket)
- send the reply
- clean up (free buffers, unlock mutex, ...)
- exit function

That's not *too* hard, right?

-- 
It is easy to love a country that is famous for chocolate and beer

  -- Barack Obama, speaking in Brussels, Belgium, 2014-03-26