On Mon, 11 Jan 2016 16:17:12 +0100 Manuel Traut <manut@mecka.net> wrote: > Hi Neil, > > > > we wrote some software to generate customized Debian > > > rootfilesystem images. The project is called ELBE and is > > > available as open-source (GPLv3) [1]. > > > > *Another debootstrap wrapper* ?? Please no. > > no not just a debootstrap wrapper. It includes the functionality to > generate e.g. a sysroot for a cross-toolchain, licence files needed > to distribute the images. A lot of supported Image formats (cpio, > msdoshd, gpt, UBI, tar.gz, squashfs, ..), pbuilder integration, > update-checker based on the generated source.xml, update-generators > and daemons. ELBE is a kind of toolkit for developing and maintaining > Debian based target images, including self-developed applications. So, yes, another debootstrap wrapper with lots of necessary features. :-) > The main focus of ELBE is to reproduce the same target image no > matter on which Debian Host PC it is running. You just need to archive > a single ISO file. That should be manageable with any of the existing tools, although how to compare the target image for some measure of "equality" is non-trivial. Same package list, same configuration - can be done and should work for all tools of this kind. vmdebootstrap has a test suite which has the beginnings of this kind of check - fast checks during development, build checks for releases. > > > ELBE uses debootstrap, python-apt, qemu-user-static & chroot, > > > mtd-utils and python-parted to generate target images. Currently > > > amd64, armel, armhf and i386 are tested successfully. > > > > There was a session at DebConf15 which identified about 15 existing > > wrappers that do much the same thing already. > > Is the output of the session available? Which wrappers exist, what > features or intention do they have and where are they hosted? http://saimei.acc.umu.se/pub/debian-meetings/2015/debconf15/Creating_bootable_Debian_images.webm http://annex.debconf.org/debconf-share/debconf15/slides/creating-bootable-debian-images/#/ > > > I'm quite sure, a lot of people wrote scripts for doing this. > > > > ... and packages already in Debian. > > We didn't find one, as we started with this in 2007. But of course > we would also like to have the elbe packages in Debian. > > > > To ensure that the same target image can be generated again, the > > > target image generation is done in a defined Debian installation, > > > typically running in KVM. We call this machine 'initvm'. So that's an extra layer outside - vmdebootstrap is used this way when building live images because that allows the build to happen without a real root user being needed. However, it seems like extra encumbrance to do that by default. Personally, I much prefer "one tool one job" and not one monolithic lump of code. This leads to developing wrappers and helpers along side the main tool to cope with the otherwise endless permutations of options, configurations and use cases. > I didn't know about vmdebootstrap and just had a quick look at the > source. > > Please correct me if i'm wrong. > > A lot of the code looks very familiar for me (elbe is also written in > python). I suspected as much, hence my original query about adding another version of the same core functionality. > It covers more or less the featureset of the 'elbe > buildchroot' command [0]. However I see limitations in vmdebootstrap > regarding supported Image formats, multiple mirrors, incremental runs > (just add a single package and regenarate the images). However on the > other hand a lot of the code is more or less the same (e.g. > grub-install, creating partitions). Image formats can be added. Multiple mirrors are not actually supported by debootstrap, so that comes down to a customisation script. (Or, as is currently being developed, a process of generating a "common" tarball of the base for lots of images, customising in parallel and preparing the final squashfs using a single tool which calls code from the vmdebootstrap module. Incremental runs are a source of horrible bugs. debootstrap itself doesn't handle incremental runs particularly well. Each run would need to be independently tested before it could be considered reliable. Adding packages to existing images isn't a job for the debootstrap wrapper - it could have a helper script to do a mount but we haven't seen a use-case for that yet. vmdebootstrap has targz, squashfs, qcow2 support. GPT is supported if the tools underneath support it (--part-type option but this is missing from the manpage, so that's something to fix). So yes, image formats additions are welcome in vmdebootstrap. Wrappers and helpers to solve use cases for multiple mirrors or image manipulation after creation (not just adding packages) would be welcome too. > The big difference between 'elbe buildchroot' and 'vmdebootstrap' is > the user interface. vmdebootstrap works with a lot of parameters and > shell scripts. 'elbe buildchroot' passes the same information in a > single XML file. vmdebootstrap options can be handled via a config file as well. Using the log file options, the config is dumped as a config file for re-use later, if required. See also the --dump-config option. > I see the potential to merge the sourc of both projects. We could > integrate the additional functionalities from 'elbe buildchroot' into > vmdebootstrap and just call this from our code. Or vmdebootstrap can > be just another frontend to elbepack/elbeproject.py (that is > currently part of the elbe-common debian binary package). Yay! There are some features you've described that can be folded into vmdebootstrap, there are some features that should be wrappers around vmdebootstrap or helpers to do things unrelated to building the original image(s). I'm keen to keep the vmdebootstrap utility itself relatively clean in terms of just building images. Other tasks like tweaking images and customising a base for lots of similar images needs to be done with helpers. It sounds like your script is doing things outside of what vmdebootstrap itself should be doing, so calling vmdebootstrap code from your code and including whatever changes we need to match the functionality would seem appropriate. Please ensure that you also add unit tests for the new functionality so that support which is "just for you" is not going to get broken by later changes for someone else. > Or integrate the additional functionality of 'elbe buildchroot' into > vmdebootstrap source and generate a python-vmdebootstap module both > and perhaps some other projects can use. vmdebootstrap 1.0 has/is a python module, it's not packaged separately at this time - a new binary package (with fewer dependencies) is possible. vmdebootstrap has a mailing list for discussion of development ideas: vmdebootstrap-devel@lists.alioth.debian.org https://lists.alioth.debian.org/mailman/listinfo/vmdebootstrap-devel -- Neil Williams ============= http://www.linux.codehelp.co.uk/
Attachment:
pgpzxer10fyZz.pgp
Description: OpenPGP digital signature