Thanks for the CC, please continue to do that. On Mon, 2019-12-02 at 19:08 +0100, Ansgar wrote: > If you discard binaries from sourceful uploads, they won't be available > for NEW processing. The intention was to discard after NEW processing, exactly at the time when binaries are copied into the archive directory and database. > If you discard binaries after they were in NEW, you create race > conditions as a file with the same name will be uploaded by buildds. > (Also I suspect it would make version tracking to enforce newer uploads > always have a higher version than previous uploads, even when binaries > were removed in between, more complicated.) I'm not familiar enough with the internals of dak & NEW but perhaps accept to NEW and accept from NEW should not insert the binaries into the database/filesystem? > It also means maintainers would have to upload binaries twice some > times (before and after NEW processing). That would only be needed for bootstrap scenarios so I don't think it is too much of an imposition, would just need to announce this requirement on d-d-a and document it in the devref. > I'm not sure if it isn't better to eventually switch to source-only > uploads for NEW instead. That would mean the binaries wouldn't be available for lintian autorejects. I didn't check if any autorejects are binary-only. > > I think the right way to implement that is that the ArchiveUpload > > _install_to_suite function that installs an upload to a suite should > > ignore such binaries when the discard binaries option is on. > > No, that is before NEW. OK, could you detail where the relevant parts are? I found _install_to_suite by grepping for where the acceptance mails are sent and then following the surrounding code to where the copying happens. > If we had reproducible builds, we really wouldn't need to throw away > maintainer-built binaries anyway as buildds would produce the same > result ;-) I think build-dep version skew after the NEW processing period is very likely to mean that buildds and maintainer builds are likely to get different binaries. That could also happen for non-NEW uploads but is of course much less likely. > But yes, anything that was in the archive should be kept for audit > purposes (we even keep rejected stuff). What is the dak option for the audit files location? Once the discarding is in place, perhaps we should also have a mailing list that announces all maintainer-built binary-only uploads? > > Do we want to flag the maintainer-built binary-only uploads for future > > audit purposes such as to ensure they are only used for bootstrap? > > What sort of bootstrap? I was mainly thinking of languages that build-dep on themselves. You can deal with those by having the maintainer build the package using upstream binaries or by having the maintainer cross-build from another architecture. Once the maintainer binaries are available somewhere, dak could enforce doing another bootstrap on the buildds without including the maintainer binaries in the archive or incoming. This is similar to how we bootstrap new architectures, import the binaries but rebuild them all before installing them in the archive. > Please keep in mind that some packages will not get built by buildds > anyway, such as some non-free stuff (not building them on buildds is the > default for non-free). Good point, perhaps we could solve this by simply not discarding binaries from sourceful uploads for contrib and non-free stuff? Alternatively, perhaps we could have a .changes option for that? > > Do we want to disallow maintainer-built binary-only uploads by DMs? > > I don't think we should treat DMs much different from DDs. OK. My main concern was that until we have some sort of repro builds setup DMs could easily sneak changes into binaries. -- bye, pabs https://wiki.debian.org/PaulWise
Attachment:
signature.asc
Description: This is a digitally signed message part