Re: Status flags definition for NBD_REPLY_TYPE_BLOCK_STATUS
On Sat, Jul 27, 2024 at 08:02:35PM GMT, Connor Kuehl wrote:
> Hello,
>
> I am implementing the client-side of the NBD protocol and I've gotten
> a little stuck on the NBD_REPLY_TYPE_BLOCK_STATUS structured reply.
>
> The protocol description[1] states:
>
> > * `NBD_REPLY_TYPE_BLOCK_STATUS` (5)
> >
> > *length* MUST be 4 + (a positive integer multiple of 8). This reply
> > represents a series of consecutive block descriptors where the sum
> > of the length fields within the descriptors is subject to further
> > constraints documented below. A successful block status request MUST
> > have exactly one status chunk per negotiated metadata context ID.
> >
> > The payload starts with:
> >
> > 32 bits, metadata context ID
> >
> > and is followed by a list of one or more descriptors, each with this
> > layout:
> >
> > 32 bits, length of the extent to which the status below
> > applies (unsigned, MUST be nonzero)
> > 32 bits, status flags
>
> I can't seem to find the definition for these status flags. It's likely
> I may have just missed it. Any pointers will be much appreciated.
The status flags are defined per-metacontext. The spec defines the
"base:allocation" metacontext:
https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md#baseallocation-metadata-context
which in turn defines flag 0x01 for NBD_STATE_HOLE (clear by default
or if the extent is known to be allocated, set only if the extent is
known to be unallocated), and flag 0x02 for NBD_STATE_ZERO (clear by
default or if the extent is known to contain non-zero data, set only
if the extent is known to read as zero).
If you are implementing NBD_BLOCK_STATUS on top of files,
lseek(SEEK_HOLE) can be used to set NBD_STATE_HOLE|NBD_STATE_ZERO in
parallel (either both bits are clear or both bits are set). But there
are some other storage mechanisms where the distinction between bits
is meaningful (for example, iSCSI has the notion of some hardware that
supports the notion of allocation but does not promise whether
unallocated regions read as zero; conversely, qcow2 files have a
notion of regions of a disk image that read as zero but which are
already allocated so that writing to those regions won't expand the
qcow2 file size).
There are other defined metacontexts linked from the NBD spec; in
particular, QEMU defines two metacontexts at
https://gitlab.com/qemu-project/qemu/-/blob/master/docs/interop/nbd.txt
where "qemu:dirty-bitmap:XXX" defines 0x01 to be set if an extent is
known to be dirty in the bitmap named XXX, and where "qemu:allocation"
is an integer representing the depth of where the allocation comes
from (0 for unallocated, 1 for the current image, 2 for the first
backing layer, 3 for the second backing layer, and so forth).
There may be other NBD metacontexts out there, but so far, none of
them have submitted a patch to be included in the mention from the NBD
spec. One proposal that I have considered (but have not actually
coded up) would be a series of NBD metacontexts allowing the ability
to expose Zoned Block Devices (ZBD,
https://zonedstorage.io/docs/linux/overview) across NBD; in
particular, this would include metacontexts that reveal information
such as the current 64-bit append location within a given zone
(getting 64-bit block status would require the use of extended
headers, currently mentioned as experimental in spec but currently
implemented by qemu and libnbd).
For what it's worth, the nbdkit project can be used for testing
interoperability with your client implementation (among other things,
nbdkit provides it easy to write a shell script server that advertises
various block status combinations for "base:allocation" for testing
how your client reacts). To date, no one has contributed patches to
nbdkit to support the emulation of any NBD metacontext other than
"base:allocation", but if it would help integration testing, that may
be a worthwhile feature to add to nbdkit.
https://libguestfs.org/nbdkit.1.html
I can also point you to the libnbd project that may serve as a way to
incorporate an existing client implementation into your project rather
than writing a client from scratch.
https://libguestfs.org/libnbd.3.html
>
> Thank you,
>
> Connor
>
> [1] https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
Reply to: