On Sun, Sep 4, 2016 at 9:28 AM Lars Wirzenius <
liw@liw.fi> wrote:
Hi, all.
I am the original author vmdebootstrap. It started as a quick hack to
let me create disk images for virtual machines without doing a manual
install with debian-installer. I didn't want to do it manually,
because I needed six images that only varied with Debian release and
CPU architecture: I was setting up a CI system test Obnam on Debian
oldstable, stable, and unstable, on i386 and amd64 targets.
Much to my surprise, a few other people seemed to like vmdebootstrap.
This led to some feature requests and other maintenance burden I
wasn't ready for, and as a result I turned over the project to Neil
Williams. Neil took good care of it for a couple of years, adding things
like grub support, and foreign architecture support, and other things I
haven't yet found. Many thanks to Neil.
Neil has now asked me to take vmdebootstrap back, and I have agreed to
do so, partly because I need it at work and at home.
I would like to see vmdebootstrap grow into a tool that adequately
handles all of Debian's image creation needs. I am currently thinking
particularly of the following use cases:
* Create VM images for libvirt + kvm (which is what I use at home).
* Create VM images for all the cloud services: OpenStack, AWS, etc. I
don't know what they all are (I've only used OpenStack and AWS), and
I'm not on expert on any of them (but we use OpenStack at work). As
such, I'll certainly need help with them.
* Create disk images for actual hardware. I have installed two of my
servers at home like this: I generate a disk image with
vmdebootstrap, write that to a USB stick with dd, and then boot off
the stick. The initial image is very barebones, and I use Ansible to
do any additional configuration (and specifically I don't use the
vmdebootstrap customization script).
I imagine this would also be useful to many who want to hack on ARM
devboards, and such, but again, that isn't something I'm an expert
on.
These are the overall classes, but each specific use may have its
unique requirements. For example, there's been some recent discussion
of how customizable partitioning should vmdebootstrap support. My
opinion is that it should support anything users need, even if the
tool becomes more complex. As an example, I have a personal need to be
able to create a bootable image that has /boot in cleartext and
everything else encrypted, with LVM.
There may be other use cases that I haven't thought of yet. Please
tell me. There may be features vmdebootstrap currently lacks that you
really, really want. Tell me.
I wish to maintain vmdebootstrap so that it serves its users and that
requires it to be quite flexible. The challenge for me is to come up
with a way to be flexible while keeping complexity at a manageable
level.
I also want vmdebootstrap to be pleasant to use. What hinders you from
enjoying vmdebootstrap? Tell me.
The other challenge is that I think the code quality is not great. It
started as a quick hack, without automated tests, and that's how it
was when I gave the project to Neil. Some tests have since been added,
but not enough, I feel. This is not Neil's fault in any way: the mess
he inherited was something I feel ashamed of. I intend to improve the
code quality massively. I want a thorough unit test suite, and I want
to extend the integration tests so that the images they create also
get tested.
This is all a lot of work. I would like help from others. A lot of
things are mysterious to me. Would you like to help? What are concrete
things you yourself could do or would like to do? Tell me.
I am aware that there are other tools than vmdebootstrap. The most
common other choice seems to be bootstrap-vz. I haven't used it; it's
possible that it could be enhanced to replace vmdebootstrap and that
my project should thus be dropped. From a cursory skim of the
documentation, it seems to not target bare hardware at all, and that's
one of the things I need myself. If you have opinions on this, tell
me.
My own next actions with regards to vmdebootstrap is to review the
currently open bugs, the code base, and find enough small changes to
make to warrant an upload to change Maintainer/Uploaders.
--
I want to build worthwhile things that might last. --joeyh
Hello Lars and all
I am the original author of bootstrap-vz.
Just this morning (around half an hour before you wrote your mail)
I decided that I would no longer be able to maintain the project,
since I simply don't have enough free time to allot to it
(or rather, I have other projects I'd rather spend my time on).
Additionally we are switching to a dockerized setup at work, which means
the amount of real-world problems I encounter,
whose solutions could improve bootstrap-vz, is reduced.
The co-maintainer of bootstrap-vz is Tiago Ileve, whom I wrote
this morning to offer the project.
Regretfully he declined, because of no free time in general.
Would you be interested in taking over bootstrap-vz instead?
I was planning on making an open offer to the mailing list,
but I think you'd be the perfect fit.
I'd like to think that the code quality, maintainability and
documentation of bootstrap-vz is top notch, which would ease the transition.
You mentioned hardware support - bootstrap-vz was originally
intended for virtualized environments, but I have myself been
close to implementing support for e.g. RaspberryPi mostly because
I believed it would be pretty easy (though I didn't get around to doing it).
I think general hardware support would be fairly trivial as well, there already is support
for both grub and extlinux bootloaders. Disks can be partitioned with
msdos, gpt or not at all.
> As an example, I have a personal need to be able to create a bootable
> image that has /boot in cleartext and everything else encrypted, with LVM.
I never got around to implementing LVM support, but I am fairly certain that the
> Create VM images for libvirt + kvm (which is what I use at home).
> The initial image is very barebones, and I use Ansible to do any additional configuration
Regarding maintainability: bootstrap-vz is around 7k lines of code,
most of that code is part of the common architecture and the common tasks.
Provider specific code averages at about 200 lines, so there is loads of reuse going on.
> I want a thorough unit test suite
> and I want to extend the integration tests so that the images they create also get tested.
Done... well, the testing harness is done and you can bootstrap machines
from scratch and start them automatically.
I never got around to SSHing into the machines, and running test commands,
which is like... running a marathon and then sitting down and
drinking a beer right in front of the finish line.
I have distilled the bootstrapping, image copying, booting and teardown into to this line:
with boot_manifest(manifest_data, boot_vars) as instance:
For EC2 this includes a temporary VPC and subnet creation,
for virtualbox it means creation of a virtual machine through VBoxManage and on docker
the image is automatically imported to the repo.
I hope I am not trying to sell bootstrap-vz too hard, I just believe that the
reproducability and ease of configuration of images offers something
that other bootstrappers do not.
Also: Please know that I will not just abruptly stop my involvment in bootstrap-vz,
I will most certainly be available for questions and would love to help the transition
process as much as possible.
So, what do you think? :-)