Re: [Nbd] Question about the expected behaviour of nbd-server for async ops

[Goswin von Brederlow <goswin-v-b@...186...>]

Alex Bligh <alex@...872...> writes:

> Goswin,
>
>> But since all requests are done synchronously there are no pending
>> requests and closing the socket is all there is left to do. So currently
>> that behaviour is as it should be.
>
> I'm not sure what you mean by "synchronously". The current client issues
> requests and processes replies asynchronously, i.e. there may be more
> than one outstanding.

The server doesn't, currently. So it will have replied to all requests
before it reads the NBD_CMD_DISC.

> It happens that an exception is made for NBD_CMD_DISC, certainly
> when accessed by a filing system, because prior to unmounting it
> waits until the disk queues are flushed.
>
> However, if (for instance) you do an "nbd-client -d" whilst running
> a raw dd, then there might be an in-flight reply; if there's not,
> its only because of synchronisation at the bio layer.

If the server where asynchronous then yes. Now there might be an
in-flight request but it will be completed before the server dies. And I
believe that behaviour should remain.

>> Here is what i think should happen:
>> - on recieving a NBD_CMD_DISC request you shutdown(fd, SHUT_RD)
>> - process and reply any pending requests
>> - fsync() /* implicit flush, just to be nice */
>> - shutdown(fd, SHUT_WR)
>> - close(fd)
>> - exit()
>
> That alone doesn't help (I am not sure we do the shutdown but
> it might be an improvement). The issue is that there may be
> replies in the tcp channel which the client has not read when
> it hits close. I don't think shutdown blocks until everything
> is sent, does it?

It (or close) blocks with SO_LINGER set or goes into background
otherwise. Also lingering is allways done on exit.

>From all google can find me the kernel will still try to send any
remaining data in the outgoing socket buffer till the SO_LINGER timeout
or tcp timeout kills the socket for good. There seem to be no way to
"wait for all data to be send" on a socket prior to closing it.

>>> Right.
>>
>> I was thinking of a buggy or malicious client. Say there is a bug in the
>> linux kernel so it sends the NBD_CMD_DISC followed by a NBD_CMD_READ.
>> Then we tear down the connection and never reply to the READ. Is that
>> better than replying with an error to the READ?
>
> We tear down the client, and exit the process. The socket is closed,
> so the client will get EPIPE. If the client is buggy or malicious,
> that's no better than it deserves!

You forget the network latenz. The client can send additional comands
before the server can tear down the socket. Remember the assumption is a
buggy or malicious client. Clearly a correct client should never ever
send anything after NBD_CMD_DISC. It should probably even shutdown the
writing side of its socket.

MfG
        Goswin