Re: Requirements for an NBD client library in userspace

On Sat, May 11, 2019, 15:39 Richard W.M. Jones <rjones@redhat.com> wrote:

On Sat, May 11, 2019 at 03:28:28PM +0300, Nir Soffer wrote:
> On Sat, May 11, 2019, 14:03 Richard W.M. Jones <rjones@redhat.com> wrote:
>
> > On Tue, Apr 30, 2019 at 04:43:03PM +0300, Nir Soffer wrote:
> > > On Tue, Apr 30, 2019, 15:46 Richard W.M. Jones <rjones@redhat.com>
> > wrote:
> > >
> > > > I believe there is no simple NBD client library. Am I wrong about
> > > > this? Let's assume I'm not for the sake of the rest of this email ...
> > > >
> > > > We do however have at least 4 places this could be consumed if it
> > existed:
> > > >
> > > > - fio's proposed NBD engine
> > > > https://www.spinics.net/lists/fio/msg07831.html
> > > >
> > > > - nbdkit-nbd-plugin
> > > > https://github.com/libguestfs/nbdkit/tree/master/plugins/nbd
> > > >
> > > > - Martin K's forthcoming VMware liberation tool
> > > >
> > > > - qemu, maybe?
> > > > https://github.com/qemu/qemu/blob/master/block/nbd-client.c
> > >
> > >
> > > oVirt imageio, replacing pure python client, used to implement nbd-http
> > > proxy
> > >
> > https://github.com/oVirt/ovirt-imageio/blob/master/common/ovirt_imageio_common/nbd.py
> >
> > It looks like imageio is Python 2. I'm trying to write libnbd
> > bindings for Python now, but TBH supporting Python 2 is turning out to
> > be painful, because there are plenty of Python 3 features it would be
> > nice to use ("b" boolean conversions, proper bytes handling and
> > PyUnicode_FSConverter are three particular features). Do you really
> > need Python 2? What is the roadmap for using Python 3 with imageio?
> >
>
> We plan to drop python 2 for 4.4. I don't that anyone care about python 2
> now.

I wrote some initial, very preliminary bindings, see top commit(s) here:

https://github.com/rwmjones/libnbd

Simple synchronous calls appear to work. However they don't work for
asynchronous calls because the idiom of:

char buf[512];

conn = nbd_get_connection (nbd, 0);
hid = nbd_aio_pread (conn, buf, sizeof buf, 0);
/* ... various calls to poll/wait here ... */
if (nbd_aio_command_completed (conn, hid)) {
/* now we can use the data in buf ... */
}

doesn't translate well into Python and passing buffers in and out of C
code. That's TBD.

buf can come from the python layer, using the buffer protocol.

Py_buffer b;

if (!PyArg_ParseTuple(args, "s*", &b))

return NULL;

nbd_pread(h, &b.buf, b.len, offset);

PyBuffer_Release(&b);

The argument on the python side can be bytearray(), mmap(), or memoryview().

If you want to provide the simplest interface returning bytes:

pread(offset, length) -> bytes

You can add a version receiving a buffer:

pread_into(offset, buf) -> n

Like socket.recv(), socket.recv_into().

In imageio nbd client we implement read() and readinto().

Adding readinfo() to imageio show 12% improvement, see:

https://github.com/oVirt/ovirt-imageio/commit/dbc3463349128357030903addb3f5d1ad464f30b

Buffer size is also important, another reason to let the client provide the buffer. See:

https://github.com/oVirt/ovirt-imageio/commit/d9ad5ea75013598074e07587c6902767fcbcb4a2

The imageio code goes appear to be doing anything asynch if I'm
understanding the code correctly.

imageio is using only synchronous code. I'm not sure how are we going to map

http to nbd using async api.

It will be interesting to compare libnbd with imageio pure python nbd client.

We can use imageio example nbd-client:

https://github.com/oVirt/ovirt-imageio/blob/91609bf5b56e1e9a85bf1edae4b916659514c7fd/examples/nbd-client#L72

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines. Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v