[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Standardized way of extracting additional build-time artefacts (was: Re: RFC: Standardizing source package artifacts build paths)

(note, this is a barely structured brain dump)

On Tue, Mar 10, 2020 at 08:10:55AM +0100, Niels Thykier wrote:
> >> Though, can you elaborate a bit on why the above approach would be
> >> better than a standard ENV variable a la AUTOPKGTEST_ARTIFACTS and some
> >> easy way to declare additional artifacts to be extracted?
> > 
> > Mainly, I'd prefer something declarative with glob patterns (a bit like
> > debian/clean or Gitlab-CI's artifacts:paths) rather than having to write
> > logic like these pseudo-patches:

Having such directory be hidden inside
/build/foo-1.2.3/debian/.build/artefacts/ is kind of a mouthful, but if
that gets to be standardized I think it would be awesome (builders
(sbuild, pbuilder, …) decide on the first '/build/foo-1.2.3/' part of
the path and they know of it; package building happens with CWD in that
place, so build tools should just try to stick to relative paths
'./debian/.build/artefacts/'; everything should Just Work that way).

One thing that strikes me of this proposal, is that you were trying to
"hide" that .build directory from the maintainer; doing this would be
going against that design decision.  This is the only "concern" I have
with the proposal.  Probably this can be avoided by providing a dh_

> Ack, I get the part of having a declarative mechanism for selecting files.

And then builder could just take out the whole directory.  If that gets
to be (g|x|)zipped or not would be an implementation detail of the
builders (sbuild, pbuilder, …) and of whatever frontend (launchpad,
buildd + wanna-build, …) is used.

> Just to clarify something related.  Should debhelper and other tools by
> default archive "certain files of possible interest" (e.g. config.log)?
> Or should we limit it to "on request only"?

That would be some nice automatism indeed, but I think it's something
for "later".  If you do, please consider these bits:
 * naming the files: you risk clashing with maintainer-set file names
 * deciding on whether to put those files there only on failure or all
   the time

> The former makes it simpler for people that are interested in the
> "default" parts but also bloats the archive for people that are
> interested in just one file.

"bloating" is indeed important.  If we start doing this, frontends need
to decide on a retaining policy.  Do we want maintainers to have a say
on this?  Like, adding a metadata file to the artifacts to indicate any
interest on those files (this is a successful build: keep for x
days/keep until next successful build + y days, etc etc).

> > I don't have any particular opinion on whether artifacts should be
> > collected into debian/.build/artifacts/, into $DPKG_ARTIFACTS/, or
> > directly into some sort of archive (Gitlab and Jenkins seem to use zip
> > files, which have the advantage of being seekable, so web frontends
> > can presumably read individual logs directly out of the zip file if
> > that's desirable).
> Thanks for clarifying.  This answered the question I was trying to write. :)

I think I took care of those thoughts above, but to reiterate:
 * IMHO ./debian/.build/artefacts/ (or artifacts? :P) is a cool and
   accessible place for all interested software
 * perhaps, you could consider using
   ${DPKG_ARTEFACTS:-$PWD/debian/.build/artefacts} so that some builders
   can override the directory if they find it more convenient for some
   reason, but otherwise I'd rather stick to a stable, non-changable
 * I think eventual tarball/compression should be left as a matter for
   the build driver (sbuild, pbuilder, …).

                        Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540      .''`.
More about me:  https://mapreri.org                             : :'  :
Launchpad user: https://launchpad.net/~mapreri                  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-

Attachment: signature.asc
Description: PGP signature

Reply to: