On Mon, 2022-07-18 at 17:53 +0200, Johannes Schauer Marin Rodrigues wrote: > Hi all, > > Quoting Luca Boccassi (2022-07-18 16:42:03) > > On Mon, 2022-07-18 at 16:04 +0200, Johannes Schauer Marin Rodrigues > > wrote: > > > Hi Luca & Helmut, > > > > > > Quoting Helmut Grohne (2022-07-18 14:32:31) > > > > On Sun, Jul 17, 2022 at 12:56:14AM +0100, Luca Boccassi wrote: > > > > > A MR is pending for debootstrap [1] to make use of this new package and > > > > > file flag when --variant=buildd is passed, so that buildds can continue > > > > > to be unmerged-usr until the CTTE says otherwise, as per decision > > > > > above. > > > > > > I don't think I have access to the mail to which Helmut replied but I guess [1] > > > is this merge request? > > > > > > https://salsa.debian.org/installer-team/debootstrap/-/merge_requests/71 > > > > Yes - the thread with the first mail is on the CTTE mailing list: > > https://lists.debian.org/debian-ctte/2022/07/threads.html > > > > btw, any reason you dropped the list from CC? > > I just didn't think that mmdebstrap-specific stuff would be on-topic on > tech-ctte. Luca agreed for me to send my full reply to their mail back to the > list in public. > > > > > Can I ask you to also work with the mmdebstrap maintainer to make sure > > > > that merged-/usr and opting out of it works well there? Specifically, I'd > > > > like to have it work without pulling the usrmerge package (and perl > > > > module dependencies) both in a merged and unmerged (buildd) layout. > > > > mmdebstrap used to have switches, but now mostly ignores /usr-merge (and > > > > thus doesn't merge). If we do nothing, it'll pull in usrmerge and its > > > > perl module dependencies with no easy way to opt out. I'm not sure what > > > > the solution is, but I'm positive that you'll find a reasonable > > > > compromise once talking to one another. mmdebstrap has tried to stay away > > > > from custom set-up code, which makes this slightly non-trivial. > > > > > > [SNIP RANT] > > > > > > Anyways, I think what makes the situation a bit better as far as mmdebstrap is > > > concerned is, that we only need to be able to build an unmerged chroot until > > > the transition is complete, correct? Does this mean that the following can be a > > > solution for everybody that wants to build an unmerged buildd chroot? > > > > > > mmdebstrap --variant=buildd \ > > > --setup-hook='echo "this system will not be supported in the future" > "$1/etc/unsupported-skip-usrmerge-conversion"' \ > > > unstable chroot.tar > > > > > > I can throw this into one of the hook scripts in /usr/share/mmdebstrap/hooks > > > and then the CLI invocation would be shorter if desired but I guess this will > > > only be run from scripts anyways? This will still install the usrmerge package > > > and its dependencies though, more on that below. > > > > If you can also add set it up so that usr-is-merged gets installed, > > then you avoid the usrmerge pkg and its extra dependencies as well. But > > yes that should work. > > Yes, that seems useful. But then usr-is-merged being installed would be a lie. > I guess we accept that hack? I think so, as long as it is understood that the system is unsupported from Bookworm onward. The flag file was added specifically to allow CIs/etc to stay back as indicated by the CTTE, so it's fine to use it as such. > Also, what is the relationship between the usr-is-merged package and the > usrmerged package? Both were/are built by src:usrmerge but its changelog > doesn't indicate when or why the naming changed so I'm left a bit confused > about the relationship between these two packages. Since your deboootstrap > merge request changed from usrmerged to usr-is-merged I guess the latter is the > correct choice? It was renamed following a request on #debian-ftp while it was in NEW, as the feedback was that 'usrmerge' and 'usrmerged' were too similar and thus easily confused. The 'usrmerged' one can be disregarded and will be de-crufted. > > > > > Once deboostrap is updated and deployed on the buildds, a "usrmerge | > > > > > usr-is-merged" dependency will be added to the Priority: essential > > > > > init-system-helpers package, and uploaded to unstable to complete the > > > > > transitions for all installations that are older than buster or that > > > > > have been manually installed as unmerged-usr. Any system not upgraded > > > > > > > > I think this setup is prone to breakage. While apt prefers the first > > > > alternative, it doesn't have to pick it and other resolvers are even > > > > more prone to picking non-default alternatives. Imagine one of the perl > > > > modules being uninstallable. Even apt will pick the usr-is-merged > > > > package in that scenario. Are you ware of this potential problem? Do you > > > > have any ideas on how to handle it? To be clear: I do not think this > > > > aspect to be a show-stopper. Let's just talk about it on a technical > > > > level. > > > > > > > > Let me propose something strange without having thought through the > > > > implications in depth (i.e. it may be a totally bad idea): > > > > > > > > What would happen if we were to replace the usr-is-merged package with a > > > > "trivial-usr-merge" package? This package would actually perform a > > > > /usr-merge in simplified conditions. Like usr-is-merged, it would refuse > > > > to perform a merge on actual systems. However in the case of > > > > non-container chroots (those not running init), it would perform the > > > > merge in a simpler way that doesn't require additional perl modules and > > > > doesn't come with the same amount of safety-guarantees. > > > > > > > > A benefit of this approach would be that I could say mmebstrap > > > > --include=trivial-usr-merge. Possibly though, this is a bad idea for > > > > reasons I am not presently understanding well. Thank you for > > > > considering. > > > > > > This will not work with mmdebstrap as of today because the packages from the > > > --include option are only installed after the essential set has been installed. > > > This was fine until today because the use-case of choosing between dependency > > > alternatives in the essential set didn't come up yet. > > > > > > If I understand correctly, then you see a trivial-usr-merge package as a > > > solution for mmdebstrap only, correct? Because debootstrap already does this > > > with the magic setup code it ships this would only be useful for chroot > > > creation tools that are not debootstrap. I think instead of a trivial-usr-merge > > > package, the following two options would work better. Either > > > > > > a) I add a setup hook script to mmdebstrap. If mmdebstrap is called with that > > > then the script will take care of doing the merged-/usr setup and installing > > > the usr-is-merged package before essential is installed so that apt will not > > > attempt installing the usrmerge package. > > > > > > b) Make convert-usrmerge work with perl-base only. Would it be difficult to do > > > so? I just tried running convert-usrmerge just with perl-base installed and as > > > far as I can see it would only need removal of autodie (and adjusting the code > > > accordingly) and embedding the code of File::Find::Rule into the usrmerge > > > package. > > > > > > Option b) will make option a) useless but requires that the usrmerge > > > maintainers accept such a patch. > > > > > > What do you think? > > > > I do not know how feasible the second one is. Ubuntu ended up vendoring > > a lot of modules into the package to be able to make it depend only on > > perl-base, and Md wasn't too keen on doing that since among other > > things maintenance and security support become a nightmare, hence why > > he added the empty metapkg instead. Given this subthread is private you > > can try and ask him on #debian-systemd which is where we talked about > > these things in the past couple of weeks. There was a bug open on this: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=985957 > > Okay. I'm not going to put my time into usrmerge then. > > Instead, I worked on option a) and came up with this diff to mmdebstrap: > > https://paste.debian.net/1247588/ > > Essentially, I'm installing the usr-is-merged dummy package before essential > gets installed. That way, apt will see that the dependency "usrmerge | > usr-is-merged" of init-system-helpers is already satisfied and not attempt > installing the usrmerge package. To summarize the changes, here is the new man > page section this diff creates: > > MERGED-/USR > mmdebstrap will create a merged-/usr chroot or not depending on whether > packages setting up merged-/usr (i.e. the usrmerge package) are > installed or not. In Debian, the essential package init-system-helpers > is will depend on the usrmerge package, starting with Debian 12 > (Bookworm). > > Before Debian 12 (Bookworm), to force mmdebstrap to create a chroot > with merged-/usr using symlinks, either explicitly install the usrmerge > package: > > --include=usrmerge > > or setup merged-/usr using the debootstrap-method which takes care of > the architecture specific symlinks and installs the usr-is-merged > package. > > --hook-dir=/usr/share/mmdebstrap/hooks/merged-usr > > To force mmdebstrap to create a chroot without merged-/usr even after > the Debian 12 (Bookworm) release, you can use the following hook: > > --hook-dir=/usr/share/mmdebstrap/hooks/no-merged-usr > > This will write "this system will not be supported in the future" into > /etc/unsupported-skip-usrmerge-conversion inside the chroot and install > the usr-is-merged package to avoid the installation of the usrmerge > package and its dependencies. > > > Does that sound okay to you and does the patch look like it does the right > thing? Yes, without knowing much about mmdebstrap, the diff looks good to me. I'd only ask that in the comment of the script hooks/no-merged- usr/setup00.sh if you could please mention explicitly that it creates an unsupported system. Maybe even print a warning when it's called. Also, I assume it is creating the metapackage on-the-fly because it doesn't have the downloaded packages available at that point? Not a problem, just double checking. > > Personally I'm of the opinion that basic filesystem layout setup should > > be done by the image builder tools, before any package gets unpacked > > let alone configured, so I'd prefer option a) but it is not a strong > > opinion and if you can convince Md to go that way, it will be fine by me. > > Would it change your opinion if I told you that with current versions of dpkg > and apt this "basic filesystem layout setup" is not needed anymore at all? So > if we ignore merged-/usr we already are in a position today where we can run > dpkg and apt on a completely empty directory and they will just do the right > thing. Knowing that this is technically possible, does it change your opinion > about which approach is superior? The one that needs magic code of a bootstrap > script or the one where all the magic is done by the package manager? > > The approach that merged-/usr took in expecting that it's okay to have some > magic setup code destroyed this property and now we need some magic setup code > again. This is why I like that the usrmerge package will become > pseudo-essential because it restores the nice property that no "basic > filesystem layout setup" needs to be done by the chroot creation tool. > > Thanks! Honestly it would not, no - IMHO basic image provision should come before any packaging tool is even around. But it is just an opinion and not a strong one, and it doesn't really affect the roadmap chosen here. If it was possible to do otherwise I wouldn't have a problem with that if there was consensus for it. -- Kind regards, Luca Boccassi
Attachment:
signature.asc
Description: This is a digitally signed message part