Re: How to replicate the CI environment?
Dear Paul,
thanks to your feedback.
Il 22/12/21 21:05, Paul Gevers ha scritto:
Hi,
On 21-12-2021 08:35, Antonio Valentino wrote:
My question is: is there a way to replicate exactly the environment
used to run tests in CI on my machine or on a porterbox?
I don't think so. We recently received bug #1001992 to improve the
situation, but I guess in this case, you can't emulate the powerful
environment I guess.
Do you have docker or virtualbox images of the armhf runner somewhere
that I could use to debug my issue?
No, we have an arm64 worker from equinix which comes with Debian 10
provisioned. We upgraded that to Debian 11. And we run plain autopkgtest
on that with the lxc backend such that we can do a different
architecture too.
OK this is probably something that I could try to do in my emulated
docker environment.
It would be important to me what is e.g. the available RAM and, maybe,
the output of ulimit.
Do you have some specific configuration for the lxc environment?
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000273
[2] https://tracker.debian.org/pkg/satpy
[3]
https://ci.debian.net/data/autopkgtest/testing/armhf/s/satpy/17766835/log.gz
[4] https://github.com/pytroll/satpy/issues/1883
If you give us some instructions on what to do and what you're after, I
could spend a few minutes extracting some more information from the
testbed. This would be done from the shell I get by passing --shell
--shell-fail to autopkgtest.
well, looking at [1] there seems to be several failures but according to
the discussion in [2] it seems that the root cause could be a specific
test fixture used in test_modis_l1b.test_load_longitude_latitude that
creates a quite large array. Once the memory is saturated also unrelated
tests start failing.
My plan would be to start disabling tests in tests_modis_l1b to
understand which are the ones that actually trigger the problem.
This approach already worked to fix the same kind of issue on i386.
Unfortunately this could be a quite long process, that's why I would
kike to be able to replicate the behavior locally.
If you could provide the output of:
$ python3 -m pytest -v --pyargs satpy -k "not test_retrieve \
and not test_offline_retrieve and not test_retrieve_all \
and not test_download_script \
and not test_start_time and not test_end_time \
and not test_mimic_TPW2_nc and not test_fy4a_for_one_resolution \
and not tests_modis_l1b"
it would be very helpful.
[1]
https://ci.debian.net/data/autopkgtest/testing/armhf/s/satpy/17802934/log.gz
[2] https://github.com/pytroll/satpy/issues/1883
kind regards
--
Antonio Valentino
Reply to: