Re: question on NBD idempotency
On Wed, Dec 11, 2024 at 09:12:13PM -0800, Connor Kuehl wrote:
> > I'm trying to develop a Kubernetes CSI driver that, among other
> > things, will be creating and tearing down NBD connections to other
> > hosts in the cluster, and am looking for idempotency design ideas.
>
> Hey Eric,
>
> This sounds cool! Is this pushed to a public repo somewhere that
> people can follow along?
Not just follow - contributions are welcome too!
https://gitlab.com/kubesan/kubesan
To date, the project has been working on a demo on an OpenShift
cluster with KubeSAN as its CSI driver and backed by a shared lvm
Volume Group on top of a LUN from fibre channel SAN storage (it is
also possible to create a shared Volume Group with iSCSI storage that
will also benefit). Using just lvm thin LVs managed by KubeSAN, I was
able to migrate a kubevirt virtual machine from one node to another
within the cluster, even though lvm can only activate thin LVs on one
node at a time - NBD was key to providing the illusion of RWX
(ReadWriteMany) access that kubevirt requires for the duration of the
migration. With thin LVs, it will also be a straightforward step to
provide CSI Snapshots. CSI cloning will be a bit more complex, and
may provide another opportunity for NBD exports to be in use.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc.
Virtualization: qemu.org | libguestfs.org
Reply to: