Re: Automatic testing of Debian packages
Hello
When it comes to testing you have to ask yourself what you want
to test.
We have a couple of different types of tests:
* Component test:
Tests if a program component (object, function etc) is working as
specified. Normally this kind of tests are implemented in the same
programming language as the function itself to test each function
and combinations of them. This is very function specific and must
be implemented per project.
* Function testing
Tests if a program work as specified. This is also program specific.
There are more but theese two are the biggest parts.
Both of these tasks are project specific and should be managed per
packge (or similar). It can of course be automated using a test suite
but each test case have to be implemented per package (or per program
in the package).
There are however some general things that could be tested. Some
of them are tested already using linda/lintian but more could be
done.
Upgrade of packages have been mentioned, install, uninstall and
purge is another thing. These are common things and you can see
it as a function test of a deb package. A test that check if the
package conform to the deb specification.
So you have to state what you want tested. Different types of tests
require different types of framework etc.
Regards,
// Ola
On Fri, Apr 08, 2005 at 10:37:27AM +0200, Mads Lindstrøm wrote:
> Hi
>
> I am writing this email as I think there could be done more to
> automatically test Debian packages. This would improve quality, as
> discovering errors is the first step in correcting them. I am not a
> Debian maintainer, but have been using Debian for 4+ years. As I am not
> an insider, I am likely to overlook some relevant information and I
> would be happy if people will correct me.
>
> I have looked at the Debian homepage to investigate what kind of
> auto-testing are performed. I found the auto-builder network
> (http://www.nl.debian.org/devel/buildd/), Lintian
> (http://packages.debian.org/unstable/devel/lintian) and Pbuilder
> (http://www.netfort.gr.jp/~dancer/software/pbuilder-doc/pbuilder-doc.html).
>
> The above mentioned software seems to test whether packages will compile
> or not. Furthermore, the software testes if packages comply by Debian
> policy. However, I would think that more could be done. It would be nice
> if packages were automatically tested under different setups, like
> language settings or other things that may affect a package. Also each
> package maintainer could supply a script which should test if the
> package ran properly. This script could then be run automatically on
> different architectures and setups (langauge settings and different
> settings for packages which the tested package depends upon).
>
> Do there exist, any other kind of automatic tests that I have not
> mentioned here?
>
> Anything else I have missed?
>
> Which errors are typical for Debian packages? This is important as
> certain kinds of error will properly newer be caught automatically.
>
> Any other comments would also be appreciated.
>
>
> Mads Lindstrm
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
>
>
> --
> To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
>
>
--
--------------------- Ola Lundqvist ---------------------------
/ opal@debian.org Annebergsslingan 37 \
| opal@lysator.liu.se 654 65 KARLSTAD |
| +46 (0)54-10 14 30 +46 (0)70-332 1551 |
| http://www.opal.dhs.org UIN/icq: 4912500 |
\ gpg/f.p.: 7090 A92B 18FE 7994 0C36 4FE4 18A1 B1CF 0FE5 3DD9 /
---------------------------------------------------------------
Reply to: