Re: Current issues with the Go Ecosystem in Debian [Was: DebConf19 pkg-go BoF minutes (Finally!)]
Hi,
Just a quick heads up about this one (sorry about the top posting, but I
don't see any particular part I should reply to).
While at FOSDEM, I talked to Paul Grevers (elbrus) - or rather was
talked to by him. He urges us to move forward on this, and especially,
not to try to fix everything on our own, but reach out to ftp-masters,
RM and security team early in the process. Rust / Haskell / OCaml teams
as well, if possible, to try to fix all issues in a compatible - if not
identical - way.
I'm happy to reach out to those teams, but I'd like us (Go team) to at
least make sure we agree on the issues and what the ways forward could be.
Cheers,
On 30/01/2020 15:30, Clément Hermann wrote:
> On 29/01/2020 14:34, Clément Hermann wrote:
>> ---
>>
>> # Minutes of the Debconf19 Go Packaging Team BoF
> <skip irrelevant stuff>
>
>> DECISION: nodens will get in touch with someone from the release team to
>> discuss the dak issue. Further discussion to happen on the Mailing List.
>> Update post-BoF: nodens talked with Ivod - Detailed email on the issue
>
> So I've discussed the matter with Ivo during DebConf19 in Curitiba, and tried
> to map the issues properly. Hopefully this gives a good overview of the issues
> and we can discuss how to move forward.
>
> TL;DR: It's a complicated matter as there are several teams involved, but by no
> means unfixable. We might want to start a discussion with the involved teams
> (ftp-masters, release, rust…) first to be sure we're on the same page.
>
> If anyone is comfortable with python and wants to help on the dak issue,
> it would be great! I don't really know python myself.
>
> Anyway, here it is.
>
> ---
> # Trying to map the Go ecosystem issues in Debian
>
> Why is it an issue when it's not the first language where you end up with
> statically linked binaries?
>
> What where the problems during Buster Release cycle?
>
> - security team: go is actually (widely) used by people, and for sensitive
> packages (Docker, Prometheus).
> - Haskell/OCAML have theorically the same issues, but not the same exposure.
> - Rust share some of the issues, but on a smaller scale.
>
> # unstable->testing migration
>
> Issue: we need to know, for a given package that is updated, what needs to be
> rebuilt (binNMU).
>
> Currently, we know it by using `Built-Using:` field. This is generated by dh_golang at
> build time, using `${misc:Built-Using}` substvar.
>
> Note: this is a policy violation, we need to change this field.
> The field used by Rust is `X-Cargo-Built-Using`.
>
> DISCUSSION: check with the Rust team? It would make sense to use the same, as we
> might then share tooling. Alternatively, just make our own and use
> X-Go-Built-Using (and tooling can check `X-[^-]+-Built-Using`
>
> ## How it can be a problem: exemple of the issue during Buster freeze
>
> During the freeze, people kept uploading new versions of packages to unstable,
> which meant that whenever a package in testing needed a fix, it would end up
> building with a more recent version of libraries than what was in testing… So
> we had to upload fake new releases to make the package go back to the previous
> version and respect the freeze.
>
> OCAML & haskell also have the issue, but they also have very stricts deps,
> which means they have to coordinate a lot wrt. package migration.
>
> In Go and Rust ecosystem, individual packages can migrate.
>
> DISCUSS: find a way to prevent this (migrating when a Build-depends is different
> than in testing).
> - Maybe using autopkgtest?
> - Britney should prevent somehow the migration of the rdepends until the
> build-depends has migrated?
> - just upload non-fixes to experimental during the freeze?
>
> # stable updates (through security update or point release)
>
> Everything has to be rebuilt and manually uploaded (either by Security Team or
> Release team for a point release).
>
> Same tool than for testing migration can be used in this case: we should be
> able to map easily what needs to be rebuilt.
>
> Problem is, it gets particularly messy when it's a security update: dak
> database is public for the main archive, but not the security archive. Indeed,
> there might be embargoed issues, so this database needs to be kept private (and
> thus separate).
>
> The issue is, in case of BinNMU, you still need the orig tarball, but if it's
> the first security update, you don't have it in the local database, and dak
> doesn't know how to check if it exists in another database, and even less how to
> get it.
>
> It means that if you upload a new revision, you also need the orig tarball. If
> you do a binnmu, source isn't uploaded, and they don't have it in the security
> archive, which means it can't be uploaded (dak will screem at you)
>
> ## potential solutions:
> DISCUSS:
> 1. make a full upload every time an upload is made on the SA for a package for
> the first time (so not a binnmu), and then you can make binnmu on this
> package afterwards?
>
> 2. (preferred) have some way in dak to detect that the source actually exists -
> even is it's a different db.
> → IOW: When I'm the security archive, if I don't have source, check the main
> archive and get the source from it if it's there.
>
> Maybe Security and Release Teams would be OK with 1., if we provide tooling to
> help ? It sure seems the easiest solution.
>
> ## Suggestion on how to move forward by ivodd:
>
> In the case we want to fix the issue on dak side - which would benefit not only
> Go, but other similar ecosystems in Debian - the suggested path forward is to
> try to create 2 archives using the interactive shell to replicate the problem.
> Note that we'll probably need help (info) from ftp team/security team because
> the databases will be empty initially.
>
> Ivodd is working on a test that will send a binNMU, which is part of the
> failure case. ETA unknown, so please don't wait up on that.
>
> ACTION: replicate the failure on a test dak instance.
> - 2 archives (one non-public)
> - upload a binNMU in the non-public one
> - it will fail because source is missing
> - Then, it's "just" a matter of patching dak to look for the source in other archives
>
> ### References for dak
>
> - lives at https://salsa.debian.org/ftp-team/dak
> - check https://salsa.debian.org/ftp-team/dak/tree/master/integration-tests for
> interactive shell test version (will import what's needed in local database
> etc)
> - tests are good examples (see 2000-xxx, which generate suites etc)
> - debian/control shows all you need to setup a local version
> - https://salsa.debian.org/ftp-team/dak/blob/master/docs/debian-specific.rst
> shows what steps are needed for security archive handling (lots of manual
> work)
>
Reply to: