[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Bug#995277: unblock: golang-github-klauspost-compress/1.11.7-2



Hi Paul,

On Sun, Oct 3, 2021 at 12:39 PM Paul Gevers <elbrus@debian.org> wrote:
Hi Reinhard,

> The consistent OOM is surprising given that you state that the worker
> has 250GB of RAM. Looking at the logs,
> I note that the tests are being passed the option -p 160 by the
> dh-golang helper, so it will build
> and run test executables concurrently. That confirms to me that we are
> indeed running on these 250GB/160 core workers.

We only have one armhf host, so *all* armhf tests are run there. I'm
also not seeing this problem back in the logs:
https://ci.debian.net/munin/ci-worker-armel-01/ci-worker-armel-01/memory.html.
But maybe the incident is too short to be caught by the once per 5
minutes poll.

I think this might be the case. The whole package fails (or passes) usually in less than 5 minutes.
These incidents would be very, very spiky. Probably something that captures the maximum
values in the 5 minute interval would be more useful here. 

> Is it possible that armhf is setting up ulimits that limits the amount
> of memory the test may allocate?

root@ci-worker-armel-01:~# ulimit
unlimited

> As far as I understand, all golang packages use autodep8 to declare the
> tests,
> which doesn't support adding the Architecture field.

I think you're right, but maybe we should fix autodep8 to enable the
change like the other overwrites in section `PACKAGE-SPECIFIC
CONFIGURATION` of $(man autodep8).

> In order to get
> around this,
> I guess I could remove the Testsuite field from debian/control and add a
> debian/tests/control that looks similar to what autodep8 generates, but adds
> the Architecture: !armhf  restriction.

Maybe until autodep8 is fixed? I fear that this may cause future trouble
if/when autodep8 support for golang gets updated.
Sure, I've cobbled up https://salsa.debian.org/ci-team/autodep8/-/merge_requests/25 for this.

I've went ahead and implemented my limiting idea anyways:
https://salsa.debian.org/go-team/packages/golang-github-klauspost-compress/-/commit/836de264a300e102a54055c1e3d629f82a1e8a1b
Maybe Adrian is right and with too many goroutines going on at the same time,
the test executable does exhaust the 3GB memory limitation on a 32bit architecture.
debhelper has this convenient --max-parallel switch that I used in combination with
setting GOMAXPROCS=8 to limit the number of goroutines in the test.

the most recent test on armhf passes with no noticeable slowdown. Maybe we could go
higher than 8, but finding out the optimal number does not seem like a good use of
our time.

Thanks for your help and input!

--
regards,
    Reinhard

Reply to: