On Sun, Jul 11, 2021 at 12:04:41PM -0700, Otto Kekäläinen wrote: > So to have complete testing we should test with: No. piuparts works in a minimal chroot to test a very specific thing: If your packages can be installed/removed cleanly or if e.g. their maintainer script have bugs. Any of your listed commands will do for this (well, as your 'upgrade' involves removes any dist-upgrade one) as that is mostly about the interactions they have with dpkg. Your problem is that you are trying to interpret a success in these tests incorrectly: This is NOT testing if for all your users the resolvers will decide to install -10.5 and associated stuff – this depends on their unique environment and piuparts tests only in a very specific minimal environment, which is pretty much unlike any user system. You can interpret piuparts results this way if your upgrades are clean and simple – aka: not requiring the removal of other packages – as if there are no choices to be made, the solution is the same for everyone. > > The behaviour is different for the "normal" usecase of upgrade: > > Upgrading from point release to point release, potentially unattended, > > where packages gain sometimes new dependencies/recommends we really > > should install along with the (security) upgrade rather than holding > > back forever which apt-get does. > > Every non-interactive invocation of apt prominently warns that it is > not intended to be run in a non-interactive fashion. But here you > indicate that indeed apt is for that, and that apt even is superior > because of better dependency resolution logic. *sigh* You can opt into this behaviour with apt-get. It is also available via libapt, which both apt and apt-get (and aptitude and …) are just front ends of. I didn't think you would be interested in the finer details below and just gave a short general perspective. The super fine-print here is btw that non-interactive != unattended. We have very many interactive users who run apt (or any others) basically unattended as you can easily confirm in support forums. software-centers do that to some extend as well as while a user interactively picks the apps to install, the details of the installation set are not shown/ considered. The different commands (upgrade, full-upgrade, …) in the different tools (apt-get, apt, aptitude, …) all have their individual strong and weak points as they are to be used for different things. "apt-get upgrade" is a pretty safe operation, but it can only do clean upgrades. As such, it can be used most of the time in point releases and (as in the release notes) to remove some stress from the next step for tools and users actually reading the output alike. "apt upgrade" is not removing packages – an action usually removing features and hence extremely dangerous for a user – to a point that the user prefers installing new packages rather than upgrading already installed packages if that would require removing others. This has no real bearing in the point release case it is meant to be for as those tend to require at most the installation of new package without the added hurdle of wanting to remove others or collapse into the most common case of being solved in the previous stricter constrains, but it seemingly has effects on the case you 'misuse' it for. "full-upgrade" (it actually doesn't matter if apt or apt-get or if you call it dist-upgrade) is free to do whatever, which includes deciding against upgrading packages and just removing them if that is deemed more reasonable. That is why the release notes recommend to do an "apt-get upgrade" (or equivalent) first, but it isn't strictly necessary. I hardly ever do it and users who don't read the release notes will not either. Most Debian based distros do not have this advice either even if they import Debian packages wholesale. Those "interfaces" can be provided by different tools and resolvers there in, libapt e.g. allows talking to external resolvers like aspcud, which deals on buildds with backports/experimental. While the constrains on the allowed actions remain the same for them, they offer different approaches to tackling hard choices within these constrains. Like minimizing removals and new-installs counts (= foreshadowing the next section). So, this isn't about who is superior in general as none is or we would all be using this hammer for everything, including screwing. It is "just" about using the right tool for the job at hand. > I guess I will need to re-read this chapter a couple of times, because > I didn't understand yet why this behaviour is related to the fact that > 'apt' *removed* mariadb-server while 'apt-get' did not remove it. You make it sound like that would be the only difference in the two interactions (including two commands each): -85 upgraded, 0 newly installed, 0 to remove and 40 not upgraded. -39 upgraded, 38 newly installed, 5 to remove and 0 not upgraded. +123 upgraded, 31 newly installed, 0 to remove and 2 not upgraded. +1 upgraded, 0 newly installed, 5 to remove and 0 not upgraded. So, based on the numbers alone (and what else really matters to a machine) the second set of commands is clearly superior as they manage to upgrade the same amount of packages with less new bugs^Wpackages to install. Optimisation criteria based resolvers like aspcud will be so proud of libapts heuristic one for doing the right thing here… Your problem is that mariadb stuff is not an upgrade and it also can't be co-installed with what is already installed, so a user/resolver can pick any possible solution – not just the one solution you would prefer in which galera-4 is installed. The second set of commands instead decides to keep galera-3 and upgrade it. That makes a few other choices unavailable (like installing mariadb-5.10), but that is not important as on this specific system nothing of note depends on them anyhow (= remember: minimal chroot). That can be drastically different on other user systems and that is why it was always a bad idea to construct upgrade paths in which packages must be removed as the choice to not do it can end up being viewed as the better pick depending on what else is (not) installed on the given system. > > I have said before that the mariadb packaging style is choosing the worst > > of both worlds by being versioned and mutually exclusive which makes it > > likely to run into situations in which a(ny) resolver will pick > > Yes I agree that it is cumbersome, but this is how the mysql packaging > in Debian has been for 10+ years, and MariaDB needs to be compatible > with that, otherwise things get too complex for our users. Also, > convincing MySQL packagers and MariaDB upstream to adopt versionless > package names is hard, as everything has worked fine for the past 10 > years and it would be a rather big change in the packaging, and people > need a clear reason why to do the effort of coordinating such a > packaging scheme change with compatible packages and millions (?) of > users without causing more problem than what it solves. You know, fossil fuel worked for years and has millions of users, too. The "problem" always existed, you just didn't thought ice melting and burning fossils would be connected, so to speak. Note that I am not saying you should "adopt versionless package names". You already have (e.g. mariadb-server). You also have versioned. And you have it so that different versioned ones can not be co-installed which is offering a choice – a choice you don't want but none the less offer. If that problem is worse than your other problems, I don't know. I naively believe it could be resolved with alternatives and so on, but I don't know the situation nor do I want it to know enough to provide a solution to you on a silver plater. All I can do is spot obvious flaws from my point of view of dependency (and upgrade) problem resolution as if I look to close I could equally well become the maintainer of all of the ~60k source packages in Debian. That isn't going to work. All I am really saying is that there is no bug in apt here: Upgrade problems are by definition problems of the packages involved as even if they were a bug in apt, we could not change apt in stable to fix it. But again, the presented problem here is not a bug in apt, both choices are equally valid as far as dependencies are concerned and that is all the resolvers care about. It might be surprising that two different paths lead to different results, but given the individual constrains that is, if not expected, at least understandable in hindsight. Best regards David Kalnischkies
Attachment:
signature.asc
Description: PGP signature