[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: [idea]: Switch default compression from "xz" to "zstd" for .deb packages

On Tue, Sep 19, 2023 at 11:39:49AM +0200, Johannes Schauer Marin Rodrigues wrote:
> I was about to say that zdebootstrap by Adam Borowski used to be a thing four
> years ago but now I see another commit from two days ago so maybe it's still
> alive and usable?
> https://git.sr.ht/~kilobyte/zdebootstrap

At the moment, all it can do is unpack packages, but a blocker for even
continuing with dpkg would be registering triggers.

Other problems:
 * the base set includes diverts now; I would need to know them
   _beforehand_.  This means available from the control tarball, or
   preferably even Packages.  Shipping a list would notoriously get
   out of sync, thus is only an option for past packages.
 * parallel debootstrapping is for obvious reasons utterly incompatible
   with usrmerge or any other kind of random package sabotaging the layout
   behind dpkg's back.  When/if DEP17 gets implemented, such alias
   mangling would be doable but symlinks would still be a nasty thing.

I'd also need to see what Pre-Depends can't be cheated away; as far as
I've found so far, all are needed for upgrades or preinst, otherwise
degrading them to Depends is workable -- but potential for breakage is
huge.  Likewise cheating on preinst.

Also I'd really need to mark zdebootstrapped systems as tainted somehow,
as they wouldn't be trustworthy until a lot of polishing and testing is

> There is also my package mmdebstrap which gives you a Debian chroot a few times
> faster than debootstrap does. Here are some benchmarks from my laptop:
>     | variant   | mmdebstrap | debootstrap  |
>     | --------- | ---------- | ------------ |
>     | essential | 9.52 s     | n.a          |
>     | apt       | 10.98 s    | n.a          |
>     | minbase   | 13.54 s    | 26.37 s      |
>     | buildd    | 21.31 s    | 34.85 s      |
>     | standard  | 23.01 s    | 48.83 s      |

That's so insanely bad...  if we only could put dpkg developers' time to
some productive task instead of you-know-what.  Even without breaking any
of current guarantees, you can easily:
 * parallelize unpack stages
 * replace the status database with something not terrible

WRT the second point: the only benefit of the current scheme is that, on a
filesystem that corrupts the data on crash even if you do everything right,
the user can do hail-mary manual recovery.  If you stop caring about ext2
and vfat, we can do much better.  And even dropping the flat text file
format wouldn't be required if we 1. extended the "updates" log, 2. didn't
rewrite the status file so often.  Of course, this would require all
readers to understand "updates" at which point we can just as well go with
a binary database, but it's not like good key-value stores are far between,
nor hard to devise from scratch.

It's depressing how lesser distributions can be so much faster, despite us
being supposed to be the bestest, prettiest, and so on :p

(I'd continue this discussion, but ATM stuck in Abu Dhabi for 14 hours
with no power source, then needing a week of sleep...)

⣾⠁⢠⠒⠀⣿⡁ Packager's rule #1: upstream _always_ screws something up.  This
⢿⡄⠘⠷⠚⠋⠀ is true especially if you're packaging your own project.

Reply to: