Revisiting partman-nbd
Hi,
Back in spring, I did some preliminary work on a 'partman-nbd', to
support an installation to the Network Block Device. I didn't finish it
then, for two reasons:
- Just as I was working on it, a patch was submitted to the Linux kernel
to make NBD support partition tables. At the time, I was working on
figuring out how to make partman understand that an NBD device is this
'weird' thing that, like an LVM volume, can be formatted but not
partitioned. Since that patch came up, I postponed the work on
partman-nbd
- I did not have proper support to actually boot off an NBD device yet,
so I made sure to properly support that first.
Now the NBD partitioning support has made it into 2.6.26, and the
root-on-NBD stuff has been in lenny for quite a while; I just uploaded a
package which also understands root-on-partitioned-NBD a few days ago.
So today, I thought I'd revisit partman-nbd, and see what needs to be
done to make it work. I'm still having some issues apparently, though,
and I thought I'd toss them up here. I feel I should make clear that I'm
not aiming for any of this to make it into the lenny installer --
squeeze is soon enough. But since I have some spare time now...
Because of some implementational details[1], when you load the NBD module,
the kernel will create device nodes for all the NBD devices that it
supports. It /used/ to be the case that those devices would report a
size equal to the maximum block device size as supported by the
particular kernel you're running on, but this would confuse the LVM
utilities such that it'd open the (unconnected) NBD device nodes, and
try to read some data from them, resulting in read errors (since there's
nothing to read). As such, the kernel was changed around the same time
the partitioning support was added to report a size of 0 for unconnected
NBD devices.
Unfortunately, this now seems to confuse partman; we apparently can't be
right for everyone. When loading the NBD kernel module before
/lib/partman/init.d/30parted is run, we will get the following error
message:
Unable to determine geometry of file/device /dev/nbd0. You should not
use Parted unless you REALLY know what you're doing!
followed by "Warning!" and the options 'Ignore' or 'Cancel' for the user
to choose; and this is repeated for each and every NBD device that is
available. Unless kernel parameters are used to change the defaults,
that means 16 of them. Obviously loading the NBD kernel module /after/
30parted is run doesn't help -- the next time partman is restarted,
we'll get those warnings anyway, because then some (though not all) NBD
devices may have been connected, which will (obviously) prevent
unloading the module...
I tried creating a /lib/partman/init.d/24nbd-devices which would
precreate the =dev=nbd0 etc directories in /var/lib/partman/devices; but
it would appear that /l/p/i/30parted just ignores their existence, and
tries (and fails) to open the device nodes anyway.
When checking the code, I do see that partman does check for a few
things, such as whether a particular device node is part of a multipath
or SATA-RAID setup, and skips them in those cases. While I could of
course suggest a change be made to partman so that similar checks would
be performed for NBD, I can't help but think that it would be much more
efficient if there were a way for a partman module to signal that a
particular device node should be skipped; perhaps by creating the device
directory and creating a file "skip" in that directory, or something
similar. init.d/30parted could then contain a line like this:
test -f /var/lib/partman/devices/$dirname/skip && continue
it could then be the responsibility of partman-dmraid, partman-nbd, and
so on, to mark the devices that need to be skipped before 30partman is
ran.
Thoughts?
[1] To connect an NBD device node, nbd-client must open() the device
node, and then call some ioctl()s on it; there is no 'master' device
node that is used to create others, or some such. So the device node
needs to exist, even if it is not in use, since you cannot open() a
device node that does not exist.
--
<Lo-lan-do> Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22
Reply to: