[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#982598: incomplete logs for autopkg tests



Control: reassign -1 debci
Control: retitle -1 trunkate huge logs at the start instead of the end

Hi,

On Fri, 12 Feb 2021 20:35:11 +0100 Paul Gevers <elbrus@debian.org> wrote:
> reassign -1 debci

Oops.

> Hi Matthias,
> 
> On 12-02-2021 10:54, Matthias Klose wrote:
> > Package: release.debian.org
> 
> It's not the release team that runs ci.debian.net. Reassigning
> appropriately.
> 
> > As seen with glibc autopkg tests [1], the Debian CI infrastructure doesn't store
> > complete build logs, cutting these to 20MB (uncompressed), resulting in ~450k
> > compressed logs.  This might not be important for successful tests, but it
> > doesn't provide any information for failed tests, as the summary at the end is
> > always cut.  Looking at the Ubuntu CI testers, you see that the glibc log for a
> > successful test can reach 150MB, compressed 3.5GB [2].  With a glibc patch to
> > not stop on test failures with the first pass [3], logs for failed tests can
> > also reach that size.
> > 
> > The outcome of tests is used by the release team to make decisions about the
> > upcoming release [4].  Pointing out to a failed log which doesn't provide useful
> > information is not helpful, and does cost volunteer time to analyze.  In this
> > case it turned out to be the flaky test infrastructure, and retries resulted in
> > a successful test.  Good luck with that for a real regression ...
> > 
> > As pointed out in another context, "machine time is cheap, volunteer time is
> > not" [5], as well as machine storage is cheap compared to volunteer time, so
> > please stop cutting the autopkg test logs.
> 
> Unfortunately, we're hitting infrastructure issues if we don't cap the
> logs [1]. However, I think we should save the last part (and not the
> first part) if we have to cap, because normally the failure happens in
> the end.
> 
> Paul
> 
> [1]
> https://salsa.debian.org/ci-team/debci/-/commit/9240d93a3e8a017c3037777d4d1604808fbc1d0f
> 

Attachment: OpenPGP_signature
Description: OpenPGP digital signature


Reply to: