[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

testing packages at build-time



Hello Debian policy,

Ancient policy [1] frowned upon running automated check of runtime
behavior of packages in debian/rules to save time for the autobuilders,
and say that such test should be run by maintainers manually before
uploading.

According to some comment of James Troup and other autobuilders admins,
I believe that autobuilder time is a less scarce ressource than porters
time, so I would move for a revision of this practice for the following
reasons:

1°) Maintainers can only tests binary packages they will upload. They can
only test an autobuild package once it is part of unstable and is 
maybe  already causing trouble.

2°) During toolchain transition, a package can be miscompiled in a way
that let it suffer for runtime problem while the autobuilders does not
report problems. Optimisation errors is an example.

3°) Once a package is successfully autobuild, it will soon be used by
the autobuiders themself to build other packages that Build-Depends on
it. That can lead to second-degree, or even third-degree damage that
require the upload of three packages to fix the breakage. That the sort
of things that delay migration to testing.

4°) When a package fails to build, the maintainer is notified and can
read the buildlog to see what has happened. Sometimes it is possible
to rebuild the package manually on one of the Debian port machine using
fixed versions of the build tools. Since the package has not been
uploaded already for this architecture, it can be uploaded without
bumping the version, which is simpler and nicer.

The most important packages (glibc, gcc) already include such automated 
checks. 

Of course the checks must be of run-time behavior and not of package
content like lintian does.

I see two possibility to implement this proposal:

1°)  Let maintainers run tests in the build or binary target.
Eventually we add a notest DEBBUILD_OPTION to disable it.

2°)  We add a test target in debian/rules. Autobuilders will need to
be modified to take advantage of this. We can then go farther and
implement special testing facility.

I am sorry for the long post, but I do believe we can make toolchain
transitions and release easier with a proper automated test
architecture for the autobuilder.

Cheers,
-- 
Bill. <ballombe@debian.org>

Imagine a large red swirl here. 

[1] I was unable to find the reference. Maybe it was part of the old
packaging manual. Any pointer welcome. The passage was something like
"... You should not run checks in the Debian rules to avoid slowdown
the autobuilders. Instead run the tests manually before uploading the
package."

Attachment: pgpxMA8sX0yXk.pgp
Description: PGP signature


Reply to: