On Fri, Nov 27, 2015 at 11:44:35AM +0100, Petter Reinholdtsen wrote: > [Antonio Terceiro] > > Hello, > > Hi. > > > The debian-edu test suite is timing out on ci.debian.net. > > What is the timeout value on ci.debian.net? The defaults for adt-run. there are different timeouts for different parts of the process. from adt-run(1): There are five timeouts affected by five values of which: short: sup‐ posedly short operations like setting up the testbed's apt and checking the state (default: 100s); install: installation of packages including dependencies (default: 3,000s); test: test runs (default: 10,000s); copy: copy files/directories between host and testbed (default: 300s); and build: builds (default: 100,000s). > > I noted the test suite invokes debootstrap several times, however > > autopkgtest/DEP-8 is supposed to test the packages as installed, and I > > don't quite understand what is the point of testing several full > > debootstraps every time. Is there anything that really needs to be > > tested with a full debootstrap and not with just installing the > > relevant packages on an existing clean testbed? > > The debian-edu package is a set of dynamically created metapackages that > should be installable, and the autopkgtest scripts are verifying that > they are indeed installable. Not all the metapackages are > co-installable, and installing several of them in the same testbed do > not make sense. This is the reason separate debootstrap environments > are used. running different tests against fresh testbeds is builtin in autopkgtest since the beginning. You can just do Test: debian-edu-checks Depends: debian-edu-foo Test: debian-edu-checks Depends: debian-edu-bar [...] Which will install what is in Depends:, and then -- assuming the installation works -- call debian/tests/debian-edu-checks. If you have to controll the installation somehow (i.e. you need to call apt-get install explicitly with some parameters, or do something before that), you can do like this: Test-Command: debian/tests/debian-edu-install-metapackage debian-edu-foo Test-Command: debian/tests/debian-edu-install-metapackage debian-edu-bar Then you can concentrate your logic in the debian-edu-install-metapackage script, which will receive the metapackage(s) to install as command line parameters. Each test that is declared in its own block gets a fresh, empty testbed to start from. So no, you don't need to debootstrap every time. This has the added feature that on a failure, just scrolling to the bottom of the log will tell you immediately which metapackages failed and which didn't. > And some of the metapackages are huge (installing around 1.6 GiB of > software, the last time I checked), so it will take a long time to > install them. > > > Also, it will help if you have separate tests (i.e. separate entries > > in debian/tests/control) for each flavor. Since the autopkgtest > > timeout is per-test, if you pack several very slow procedures into a > > single test you risk getting timeouts. > > This is a good idea. I did not do it originally as I was not aware of > such timeout problem, and wanted to keep code duplication to a minimum. My suggestion above covers this. > > I am blacklisting debian-edu for now, and will revisit that when this > > bug is closed. > > Sad to hear this. I'm not happy about it either, but right now the tests are broken and the system can test hundreds or even thousands of packages during the time in which debian-edu would run and fail midway 100% of the time. -- Antonio Terceiro <terceiro@debian.org>
Attachment:
signature.asc
Description: PGP signature