[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Bug#962140: ITP: bdebstrap -- YAML config based multi-mirror Debian chroot creation tool



Am Donnerstag, den 11.06.2020, 12:02 +0200 schrieb Johannes Schauer:
> Hi,
> 
> Quoting Benjamin Drung (2020-06-11 10:52:43)
> > > > bdebstrap is an alternative to debootstrap and a wrapper around
> > > > mmdebstrap to support YAML based configuration files. It
> > > > inherits
> > > > all
> > > > benefits from mmdebstrap. The support for configuration allows
> > > > storing
> > > > all customization in a YAML file instead of having to use a
> > > > very
> > > > long
> > > > one-liner call to mmdebstrap. It also layering multiple
> > > > customizations
> > > > on top of each other, e.g. to support flavors of an image.
> > > 
> > > Just curious, how does it compare to vmdb2, besides using
> > > mmdebstrap 
> > > instead of debootstrap.
> > 
> > Before developing bdebstrap, I evaluated vmdb2 and borrowed the
> > idea of
> > using YAML.
> > 
> > The big difference besides mmdebstrap/deboostrap is that vmdb2
> > creates
> > a disk image and bdebstrap create a tarball or squashfs image.
> > 
> > This serves us two use cases:
> > 
> > 1) building live systems to use for booting over the network
> > 
> > 2) installing the tarball on two disks (the OS on a 2.5" disk and
> > the
> > /boot directory on an SD card). Work in progress for the install
> > script:
> > https://github.com/bdrung/bdebstrap/blob/install-image/install-image
> 
> additionally, it seems to inherit the following properties from
> mmdebstrap:
> 
>  - building an image with SOURCE_DATE_EPOCH set produces bit-by-bit
>    reproducible output:
> 
>       $ ./bdebstrap -c examples/Debian-unstable.yaml --name example1
> --env SOURCE_DATE_EPOCH=1591868595

Yes, and it goes a bit further: If no SOURCE_DATE_EPOCH is specified,
it will set it to the current timestamp and record it in its output
config.yaml. So you have a configuration for bit-by-bit rebuild of the
image.

>  - building an image does not require superuser privileges

Yes. That was one reason for us to switch from debootstrap to
mmdebstrap.

Let me tell the story of bdebstrap for whom is interested in it: When I
joined IONOS (it was called ProfitBricks back then), we had a Shell
script that was used for building Debian live systems, which basically
called debootstrap, installed some package, copied configurations and
did other modifications to the resulting root tarball. That live system
used a custom built initrd with a pre-build busybox binary. I ported it
to use initramfs-tools for building the initrd and added a tmpfs boot
mode to initramfs-tools (see Debian bug #864777). Thanks to the
flexibility of initramfs-tools, all the needed modifications could be
done via scripts. The next big step was to replace the tmpfs boot mode
with live-boot (which uses a squashfs image and tmpfs overlays). Our
build script was still custom, long and required root permission to
run. During all this time, I looked at different build tools and then
mmdebstrap came along. I tried it out and it seemed to be a very good
fit. Johannes Schauer helped in a timely manner to address all those
bug reports that I opened for mmdebstrap (thanks for that). It took me
quite some git commits to split and reorder our build script so that it
can call mmdebstrap and use hooks for all the customization at the
different stages. The build shell script ended up to more or less just
contain a multi-line call to mmdebstrap as well as the support for
flavors. Since mmdebstrap had such simple user interface and only
produce one output file/directory, I decided to write bdebstrap for
using YAML files and put multiple files (config.yaml, manifest, and
others) in the output directory. Now our live images can be build with
only open source tools and packages from Debian. The README.md / man
page of bdebstrap describes the minimal bits for building such live
system. Our in-house YAML configuration is over 200 lines long.
bdebstrap is intended to stay and I have enough other work besides
reworking our build system again.

-- 
Benjamin Drung

DevOps Engineer and Debian & Ubuntu Developer
Platform Integration (IONOS Cloud)

1&1 IONOS SE | Greifswalder Str. 207 | 10405 Berlin | Germany
E-mail: benjamin.drung@cloud.ionos.com | Web: www.ionos.de

Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 24498

Vorstand: Dr. Christian Böing, Hüseyin Dogan, Dr. Martin Endreß, Hans-
Henning Kettler, Arthur Mai, Matthias Steinberg, Achim Weiß
Aufsichtsratsvorsitzender: Markus Kadelke


Member of United Internet

Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte
Informationen enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat
sind oder diese E-Mail irrtümlich erhalten haben, unterrichten Sie
bitte den Absender und vernichten Sie diese E-Mail. Anderen als dem
bestimmungsgemäßen Adressaten ist untersagt, diese E-Mail zu speichern,
weiterzuleiten oder ihren Inhalt auf welche Weise auch immer zu
verwenden.

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient of this e-mail, you are hereby
notified that saving, distribution or use of the content of this e-mail 
in any way is prohibited. If you have received this e-mail in error,
please notify the sender and delete the e-mail.

Attachment: signature.asc
Description: This is a digitally signed message part


Reply to: