Re: Please, minimize your build chroots
On Sun, 2023-01-29 at 12:28:37 +0200, Adrian Bunk wrote:
> On Sun, Jan 29, 2023 at 05:00:56AM +0100, Guillem Jover wrote:
> > On Sat, 2023-01-28 at 21:35:01 +0200, Adrian Bunk wrote:
> > > Removing tzdata from the essential set made sense, and I do see your
> > > point that tzdata does not fit the "totally broken" definition of
> > > "required".
> >
> > > What we need are technical discussions like whether packages like tzdata
> > > should also be removed from the build essential set, or whether tzdata
> > > being a part of the build essential set should be expressed by making
> > > the build-essential package depend on tzdata.
> >
> > I guess my question is, what makes tzdata build essential, besides
> > that it's currently kind-of implicitly there? To me it does not look
> > like it has the properties to be considered as such, besides that if
> > we lower its priority (as it deserves) it makes a bunch of packages
> > FTBFS.
>
> It has historically been part of the build essential set.
(So the "that it's currently kind-of implicitly there".)
> It is used in the build of many packages.
Given the number of packages that currently declare a dependency on
tzdata (34~), the ones that seem to have the most potential to pull it
for others are language bindings such as python3-dateutil, python3-tz
ruby-tzinfo, etc, which handle timezone stuff anyway and would keep
pulling it. So I find this assertion hard to believe. And the following
points seem to be based on this fundamental assumption.
I think though, we might be able to easily answer at least how many
builds ended up with tzdata installed (even if it ended up not being
used), if Santiago still has the build logs around, those would be
easily grep-able.
(Whether it was used at all could be checked after each build through
atime I guess. But this seems all like a lot of wasted effort TBH.)
And for reference, Santiago seems (I might be wrong) to have filed
around 46 reports about this, which seems like a very tiny speck in
the grand scheme of things.
> There would be so many invisible undeclared build dependencies of
> packages using it who have it pulled in by other packages that random
> changes in the dependency tree might cause many packages to FTBFS at
> any time if it does not stay build essential.
I fail to see how this is any different from any other transitive
dependency being pulled automatically due to some implementation
detail somewhere.
> It is required to provide glibc functionality that is frequently
> used during the build.
This seems like a reiteration of the above "it's used a lot". There
are other things that are used a lot during builds and yet they are
not build-essential (python3 for an obvious example).
> > So, for one, this is getting in the way of making our minimal
> > (non build) systems smaller.
>
> No, it is not.
>
> There are 3 different topics:
>
> 1. making minimal (non build) systems smaller
>
> Being able to exclude tzdata from a minimal system was achieved when it
> was removed from the essential set in stretch.
> debootstrap not installing it by default would make that easier.
> Whether build-essential depends on tzdata does not make any difference.
Sure, you can, but it's not the _default_, so it *is* getting in the way.
And some seem to be even trying to tie it into the build-essential set by
proxy, wanting to bolt Priority:required into it… People keep bringing up
in this discussion the sanctity of these default installations, so it
again seems unfair to then disregard it when it's not convenient.
> 2. normal systems
>
> In a normal (non-minimal) installation not having tzdata installed
> would be a bug harming many users, no matter what priority it will
> have.
Sure, this can easily be achieved by making it Priority:important,
so I don't see this being a concern.
> 3. build essential
>
> That's separate from 1. and 2.
Sure again (as long as it is not kept as Priority:required "for
reasons").
> > > I have so far not seen any technical arguments why removing tzdata from
> > > the build essential set would be better for Debian than keeping it there.
> > > Removing tzdata reduces the size of a chroot that has the build
> > > dependencies for the hello package installed by ~ 0.5%, this size
> > > decrease does not strike me as a sufficient reason for reducing the
> > > build essential set.
> >
> > I don't see how this can be a pure technical decision, TBH. To me this
> > looks more like either a matter of trade-offs,
>
> It is a tradeoff between less work and saving ~ 0.5% space in build
> chroots.
That would be the trade-off you see, yes. :)
> > or IMO more importantly
> > of clean definition of a specification (which seems rather more
> > important than the other concerns). The work to spot these problems has
> > already been done, and the changes required are trivial and minimal
> > (disregarding any severity consideration here).
>
> The work has been done for packages that do FTBFS today.
>
> I would guess ~ 90% of the packages that did have tzdata installed
> during Santiagos builds did not have or need a direct build dependency
> because something else pulled it in.
> It is unknown how many of these would have a latent FTBFS bug due to an
> undeclared direct build dependency.
This is again usual business when it comes to any transitive dependency
that is an implementation detail, for any existing package. I still
fail to see how tzdata is some kind of special case here.
> Do we have any packages that build successfully but are broken without
> tzdata installed during the build?
Do we perform or require similar checks whenever we change or remove
transitive dependencies elsewhere? I guess this could easily be caught
up by the reproducible builds though.
> >...
> > I appreciate the minimalism and simplicity of the definition. I've
> > also heard from time to time complains that we even require a C/C++
> > compiler as build-essential, when for many packages that's not even
> > needed (although a C compiler is currently needed by dpkg-dev to
> > determine the host arch).
>
> I would also complain that dpkg-dev pulls the full perl into the build
> essential set.
(If the idea of rewriting dpkg-dev in some other language was ever
_entertained_, I don't see why we would not also at the same time
consider removing perl from the build-essential set, yes. If the
complaint is though about perl-base vs perl, I'm happy to evaluate how
much effort that would involve. :)
> The build essential set is so huge that I wouldn't even be surprised if
> at some point in the future this discussion becomes moot because some
> package in the build essential set gains a dependency on tzdata.
>
> Perhaps some localtime related functionality could justify a dependency
> of perl or perl-modules-5.36 on tzdata, which would keep it in the build
> essential set due to dpkg-dev being build essential?
This again seems like a gratuitous dependency just to keep this around,
affecting every other installation.
> > Policy also has this to say:
> >
> > ,---
> > If build-time dependencies are specified, it must be possible to build
> > the package and produce working binaries on a system with only
> > essential and build-essential packages installed and also those
> > required to satisfy the build-time relationships (including any
> > implied relationships). […]
> > `---
> >...
>
> I don't think this is helpful for the discussion whether packages that
> were previously part of the essential set should stay part of the build
> essential set.
Why? This has happened in the past and will keep happening. An example
that immediately comes to mind is the perl CGI modules, which got split
out from perl and that was not a big social issue at all (as should be
expected!). I also easily recall this even happening within the essential
set, when dpkg switched from xz-utils to liblzma and packages had started
to assume the former would be present. The build-essential set has always
been way more dynamic than the essential one.
So I'm having a very hard time understanding what has people riled up
about tzdata, as this very special thing that needs to be preserved
there at all costs, while there's been this much time, effort, emotion
and words spent on these threads compared to the amount of time required
to add the trivial dependency to all current failing packages and
probably any potential future ones.
I'm also finding the notion that some seem to be trying to push here
(even if not in an explicit or intended way) that the whole
build-essential set should have such a strict and monolithic composition
(including apparently all of its transitive dependencies) in a similar
way how we supposedly handle the essential set, to be worrisome, and
given the energy spent here, wasteful.
> "Important: yes" e2fsprogs is an even bigger problem here:
> It is rarely used during the build of a package and shouldn't stay part
> of the build essential set for that reason, but as long as apt refuses
> to remove it when it is already installed I don't think such
> a nearly-essential package can be removed from the build essential
> set because something like
> Build-Conflicts: e2fsprogs
> would then be both permitted and likely cause problems.
In the abstract I think this theoretical problem is important to
support from the tooling, so I've filed #1030020 against apt, so that
we can control a way to allow automatic removal from discardable
systems. After that I might file reports against build-drivers such as
sbuild or pbuild to make use of that possibility when they know they
are building on such discardable system.
But for the particular case you bring up, there has never been the
need, and I think I'd find it rather upsetting if any package declared
a Build-Conflicts against a Protected/Important:yes package, because
in essence that means you cannot safely build them anymore on your main
system, and I'd be hard pressed to question why any package would ever
need such Build-Conflicts to the point I think this should be considered
a bug on those packages.
So this seems like a non-issue to me.
Regards,
Guillem
Reply to: