Hi,
Quoting Simon McVittie (2024-06-27 15:59:01)
> On Thu, 27 Jun 2024 at 11:46:51 +0200, Helmut Grohne wrote:
> > I don't quite understand the need for a Dockerfile here. I suspect that
> > this is the obvious way that works reliably, but my impression was that
> > using podman import would be easier.
>
> Honestly, the need for a Dockerfile here is: I already knew how to build
> containers from a Dockerfile, and I didn't read the documentation for
> the lower-level `podman import` because `podman build` can already do
> what I needed.
>
> I see this as the same design principle as why we encourage package
> maintainers to use dh, even when building trivial "toy" packages like
> hello, and in preference to implementing debian/rules at a lower level
> in trivial cases. To build a non-trivial container with multiple layers,
> you'll likely need a Dockerfile (or docker-compose, or some similar thing)
> *anyway*, so a typical user expectation will be to have a Dockerfile, and
> anyone building a container will likely already have learned the basics
> of how to write one; and then we might as well follow the same procedure
> in the trivial case, rather than having the trivial case be different and
> require different knowledge.
I have never in my life written a Dockerfile and so far I've only used "podman
import" instead. Your explanation makes sense to me. I had no idea that "podman
build" is on a higher plumbing level. As a container noob it was always more
easy for me to write:
mmdebstrap [my customizations] unstable | podman import - debian
If I understand what you are saying, then what should instead be done is to
write a Dockerfile receiving a vanilla tarball and then do the customizations
via the Dockerfile?
Can a Dockerfile be read from stdin? It's a small wrinkle to me that I would
then need to create a private temporary directory with a Dockerfile first
instead of just shoving it in over a pipe.
> > Do I understand correctly that in this variant, you intend to use podman
> > without its image management capabilities and rather just use --rootfs
> > spawning two podman containers on the same --rootfs (one after another)
> > where the first one installs dependencies and the second one isolates the
> > network for building?
>
> Maybe that; or maybe use its image management, tell the first podman command
> not to delete the container's root filesystem (don't use --rm), and then
> there's probably a way to tell podman to reuse the resulting filesystem
> with an additional layer in its overlayfs for the network-isolated run.
>
> Please note that I am far from being an expert on podman or the
> "containers" family of libraries that it is based on, and I don't
> know everything it is capable of. Because Debian has a lot of pieces
> of infrastructure we have built for ourselves from first principles,
> I've had to spend time on understanding the finer points of sbuild,
> schroot, lxc and so on, so that I can replicate failure modes seen on
> the buildds and therefore fix release-critical bugs in the packages that
> I've taken responsibility for (and occasionally also try to improve the
> infrastructure itself, for example #856877 which recently passed its
> 7th birthday). That comes with an opportunity cost: the time I spent
> learning about schroot is time that I didn't spend learning about OCI.
>
> One of the reasons I would like to have fewer Debian-specific pieces in
> our stack is so that other Debian developers don't have to do what I
> did, and can instead spend their time gaining transferrable knowledge
> that will be equally useful inside and outside the Debian bubble (for
> example the best ways to use OCI images, and OCI-based tools like
> Docker and Podman, which have a lot of overlap in how they are used even
> though they are rather different behind the scenes).
Thank you for this text as well as the one in your initial email in which you
caution against more Debian-isms with only very few maintainer(s) maintaining
them. As the author of the unshare backend I am guilty of having added another
Debian-specific thing instead of re-using existing solutions. Maybe my defense
can be that when I wrote that code in 2018, there was no podman in Debian yet?
I am not attached to the unshare code. I gladly throw it out for something
better. The less code I have to maintain the better for me. I do not dislike
podman either and I am happy that in contrast to docker, there is no persistent
service running in the background.
What I wanted to mainly bring up in this email are the following things:
Creating build chroots from things that are signed with the Debian archive
keyring is important to me. Even though, as Holger pointed out, the Debian
images that one can download can be reproduced independently, I rather make
sure that I receive what I think I receive by relying on creating my chroot via
mmdebstrap/apt verified by my local keyring. Maybe in the future debian.org can
publish build chroots signed by the archive keyring at which point I may change
my position on this. But until then, I really heavily prefer to download the
GPG signed stuff from our mirrors instead of something from an image registry
that we do not control.
If we change things around, I'd prefer whatever change is done comes with
non-negligible advantages and few (preferably none) regressions. The unshare
backend is theoretically (not in practice because the schroot backend cannot do
it) able to give you an environment where you have all your namespaces unshared
but you are on the outside of the chroot directory. This gives processes on the
outside the opportunity to work on those on the inside (because they have the
required privileges). One enticing application I see for this feature is to be
able to build inside chroots that do not even have apt installed. This can be
possible because apt can install build dependencies in the chroot from the
outside, if given the chroot directory and the necessary privileges. As far as
I'm aware (and please correct me if I'm wrong) podman does not offer this
functionality? So with a podman backend, my build chroot has to include apt
because the build dependencies have to be installed somehow? One
solution/workaround to this problem would be something else you said earlier:
create the container as part of build dependency installation. In such a
scenario, the podman container could be created by running
mmdebstrap [options installing b-d] unstable | podman import -
And then the resulting container would have essential and the build
dependencies installed but would not have apt in it. This would not work with a
Dockerfile, right?
Last point: people on this list were very excited about using an established
container technology in our tooling instead of cooking something up ourselves
and rightly so. I agree with that sentiment. The excitement can probably also
be seen by there existing 13 independent software packages that do "debian
package building in docker":
https://wiki.debian.org/SystemBuildTools#Package_build_tools
But, if everybody is so excited about this, where are the sbuild contributors
implementing this? As is hopefully obvious from my above questions, I have no
clue about containers, so I'd be the wrong person to work on this. But we have
implementations like the one in #867176 since 2017 and nobody stepped up to
maintain it since then. This is very curious to me. People (including on this
list) are very excited about docker/podman but then there is no code and
longterm maintainership that follows. I'd be happy to review patches and help
integrating podman into sbuild in any of the three ways I outlined in my other
mail. But I need help with that and that help didn't arrive yet. So like the
others on this list I am with you Simon, that it would be nice to have less
Debian-isms and more use of cross-distro tools. But in practice, what we have
are the Debian-isms maintained by only very few and nobody putting their long
term efforts into implementing something that re-uses cross-distro approaches
in sbuild... :/
In essence: somebody please help! :)
Thanks!
cheers, joschAttachment:
signature.asc
Description: signature