Re: Requirements for an NBD client library in userspace
On Sat, May 11, 2019 at 06:48:15PM +0300, Nir Soffer wrote:
> On Sat, May 11, 2019, 15:39 Richard W.M. Jones <rjones@redhat.com> wrote:
>
> > On Sat, May 11, 2019 at 03:28:28PM +0300, Nir Soffer wrote:
> > > On Sat, May 11, 2019, 14:03 Richard W.M. Jones <rjones@redhat.com>
> > wrote:
> > >
> > > > On Tue, Apr 30, 2019 at 04:43:03PM +0300, Nir Soffer wrote:
> > > > > On Tue, Apr 30, 2019, 15:46 Richard W.M. Jones <rjones@redhat.com>
> > > > wrote:
> > > > >
> > > > > > I believe there is no simple NBD client library. Am I wrong about
> > > > > > this? Let's assume I'm not for the sake of the rest of this email
> > ...
> > > > > >
> > > > > > We do however have at least 4 places this could be consumed if it
> > > > existed:
> > > > > >
> > > > > > - fio's proposed NBD engine
> > > > > > https://www.spinics.net/lists/fio/msg07831.html
> > > > > >
> > > > > > - nbdkit-nbd-plugin
> > > > > > https://github.com/libguestfs/nbdkit/tree/master/plugins/nbd
> > > > > >
> > > > > > - Martin K's forthcoming VMware liberation tool
> > > > > >
> > > > > > - qemu, maybe?
> > > > > > https://github.com/qemu/qemu/blob/master/block/nbd-client.c
> > > > >
> > > > >
> > > > > oVirt imageio, replacing pure python client, used to implement
> > nbd-http
> > > > > proxy
> > > > >
> > > >
> > https://github.com/oVirt/ovirt-imageio/blob/master/common/ovirt_imageio_common/nbd.py
> > > >
> > > > It looks like imageio is Python 2. I'm trying to write libnbd
> > > > bindings for Python now, but TBH supporting Python 2 is turning out to
> > > > be painful, because there are plenty of Python 3 features it would be
> > > > nice to use ("b" boolean conversions, proper bytes handling and
> > > > PyUnicode_FSConverter are three particular features). Do you really
> > > > need Python 2? What is the roadmap for using Python 3 with imageio?
> > > >
> > >
> > > We plan to drop python 2 for 4.4. I don't that anyone care about python 2
> > > now.
> >
> > I wrote some initial, very preliminary bindings, see top commit(s) here:
> >
> > https://github.com/rwmjones/libnbd
> >
> > Simple synchronous calls appear to work. However they don't work for
> > asynchronous calls because the idiom of:
> >
> > char buf[512];
>
>
> > conn = nbd_get_connection (nbd, 0);
> > hid = nbd_aio_pread (conn, buf, sizeof buf, 0);
> > /* ... various calls to poll/wait here ... */
> > if (nbd_aio_command_completed (conn, hid)) {
> > /* now we can use the data in buf ... */
> > }
> >
> > doesn't translate well into Python and passing buffers in and out of C
> > code. That's TBD.
> >
>
> buf can come from the python layer, using the buffer protocol.
>
> Py_buffer b;
>
> if (!PyArg_ParseTuple(args, "s*", &b))
> return NULL;
>
> nbd_pread(h, &b.buf, b.len, offset);
>
> PyBuffer_Release(&b);
>
> The argument on the python side can be bytearray(), mmap(), or memoryview().
>
> If you want to provide the simplest interface returning bytes:
>
> pread(offset, length) -> bytes
This is how the synchronous version works already. The problem is
binding the AIO version where the caller must pass in a buf pointer
that persists while the command is being run across multiple API
calls.
But if imageio doesn't need AIO then it's not urgent to address this.
> You can add a version receiving a buffer:
>
> pread_into(offset, buf) -> n
>
> Like socket.recv(), socket.recv_into().
>
> In imageio nbd client we implement read() and readinto().
>
> Adding readinfo() to imageio show 12% improvement, see:
> https://github.com/oVirt/ovirt-imageio/commit/dbc3463349128357030903addb3f5d1ad464f30b
>
> Buffer size is also important, another reason to let the client provide the
> buffer. See:
> https://github.com/oVirt/ovirt-imageio/commit/d9ad5ea75013598074e07587c6902767fcbcb4a2
>
>
> > The imageio code goes appear to be doing anything asynch if I'm
> > understanding the code correctly.
> >
>
> imageio is using only synchronous code. I'm not sure how are we going to map
> http to nbd using async api.
>
> It will be interesting to compare libnbd with imageio pure python nbd
> client.
> We can use imageio example nbd-client:
> https://github.com/oVirt/ovirt-imageio/blob/91609bf5b56e1e9a85bf1edae4b916659514c7fd/examples/nbd-client#L72
libnbd can't do structured replies or read allocation status at the
moment. Also TBD.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
Reply to: