On Sun, 2022-07-17 at 11:34 +0100, Simon McVittie wrote: > On Sun, 17 Jul 2022 at 00:56:14 +0100, Luca Boccassi wrote: > > Piuparts has been enhanced with a new test case that covers the > > moratorium on moving files manually between their location in the > > root > > directories and /usr. > > Thanks for doing this, it will help a lot. > > > A MR is pending for debootstrap [1] to make use of this new [...] > > file flag when --variant=buildd is passed, so that buildds can > > continue > > to be unmerged-usr until the CTTE says otherwise, as per decision > > above. > > I don't think we actually ever said that buildds must continue to be > unmerged-/usr. In #914897 we resolved that for Debian 11, build > systems > can validly be either merged-/usr or unmerged-/usr, implicitly > meaning > that packages that are mis-built on either merged-/usr or unmerged- > /usr > buildds have a bug (#994388 recommended that this bug is treated as > RC). > In #994388 we confirmed that this continues to apply during the > bookworm-is-testing cycle. > > In #914897 this narrowly defeated the alternative of mandating that > build > systems must be merged-/usr for Debian 11 (which, with hindsight, > would > have been too disruptive at that time, and I'm glad we didn't choose > it). > > I personally agree that buildds should stay unmerged-/usr until the > known > bugs that would be triggered by merged-/usr buildds have been raised > to > RC severity, though. #992645 is a typical example. I'll try to have a > look > at those bugs today. > > The QA systems that I had in mind when I suggested that we need an > opt-out were things like autopkgtest and piuparts, rather than > buildds, > to ensure that we can detect this scenario: > > * a package (let's say ncftp, #992645) is built on a merged-/usr > buildd > * it builds successfully, but the contents are incompatible with > unmerged-/usr systems > - for example in #992645 the binary gets /usr/bin/tar hard-coded > into it, > but that path only exists on merged-/usr > * a QA test (autopkgtest, piuparts or similar) installs the package > into > an unmerged-/usr environment > * the package fails to install, or installs but fails to work > - I don't think ncftp has a test that exercises its tar > integration, > but please imagine it did... > * as per #994388, the TC think this should be treated as a RC bug > > The opt-out is needed in order to allow autopkgtest and piuparts to > create > the necessary unmerged-/usr environment to be able to run that test. > If > they can only create merged-/usr bookworm environments, then that > test > would not be implementable. So the reasoning behind adding the override to debootstrap in -- variant=buildd mode is twofold. From a very practical perspective, it means it's supereasy to allow all those builds and tools you mentioned to run in unmerged-usr, there's no extra config or code changes required anywhere else, it just works. From a strategic point of view, catching all those class of bugs you mentioned is not trivial, as you said if there's no autopkgtest to exercise the feature (which is the case for many many many packages) it will be difficult. And essentially, it's all for naught, as they only manifest when built on merged-usr and used on unmerged-usr. By keeping the buildds in umerged-usr state for Bookworm, and switching them as soon as it is released, it seems to me we can avoid caring about them at all. That's because we will do all builds on unmerged-usr, until we are sure everything is switched over (and the autopkgtest/piuparts can also be turned over) in Bookworm+1, then there's nowhere left that runs them in unmerged-usr systems, and thus we can happily forget about it and spend the time on other issues. Wouldn't this work? Am I missing something? > > Once deboostrap is updated and deployed on the buildds, a "usrmerge > > | > > usr-is-merged" dependency will be added to the Priority: essential > > init-system-helpers package, and uploaded to unstable to complete > > the > > transitions for all installations that are older than buster or > > that > > have been manually installed as unmerged-usr. Any system not > > upgraded > > will be considered unsupported, and any package that doesn't work > > on > > the only supported layout will be considered RC-buggy, as per > > decision > > above. No special installation or upgrade path will be necessary, > > the > > normal apt upgrade/install process works as expected, and the > > transition happens from a maintainer script and it does not require > > to > > be ordered before or after any other package installation or > > upgrade. > > This all seems like a reasonable course of action to me. The usr-is- > merged > package avoids the dependency impact of the Perl modules that > usrmerge > needs when it does the actual transition. > > After usrmerge has been installed successfully, is it straightforward > to > replace it with usr-is-merged so that usrmerge's Perl dependencies > can be > autoremoved? Yes, once an essential package has a usrmerge | usr-is-merged dependency it's as easy as: # apt remove usrmerge Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: usr-is-merged The following packages will be REMOVED: usrmerge The following NEW packages will be installed: usr-is-merged 0 upgraded, 1 newly installed, 1 to remove and 0 not upgraded. Need to get 4256 B of archives. After this operation, 26.6 kB disk space will be freed. Do you want to continue? [Y/n] Get:1 http://deb.debian.org/debian sid/main amd64 usr-is-merged all 27 [4256 B] Fetched 4256 B in 0s (135 kB/s) debconf: delaying package configuration, since apt-utils is not installed Selecting previously unselected package usr-is-merged. (Reading database ... 24027 files and directories currently installed.) Preparing to unpack .../usr-is-merged_27_all.deb ... Unpacking usr-is-merged (27) ... Setting up usr-is-merged (27) ... (Reading database ... 24030 files and directories currently installed.) Removing usrmerge (27) ... # > > The patch from user uau that the dpkg maintainer rejected in the > > past > > has been submitted to the existing bug [2] for completeness (with > > permission from the author). > > If I remember correctly, Julian Andres Klode was developing a version > of this patch that replaced the hard-coding of the /usr merge with a > more general way to declare "directory A is an alias for directory B" > and have it stored in the dpkg database, so that dpkg has the > mechanism > and some Essential package like init-system-helpers or base-files > could > have the policy, in the hope that this would be more acceptable to > the > dpkg maintainer. Is that code available? I asked Julian a week ago or so on IRC if he had worked on that patch at all, and the answer was no. Besides, I think [0] and everything else make it very evident at this point that the form and function of the patch attached to the dpkg bug are not really the problem. There are more fundamental issues of project management and organization at play, and thus we really do not want those issues to be in the way of the goal of fully moving to merged-usr. -- Kind regards, Luca Boccassi [0] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#132
Attachment:
signature.asc
Description: This is a digitally signed message part