Re: Automated testing - design and interfaces
On Fri, Nov 18, 2005 at 03:35:25PM +0000, Ian Jackson wrote:
> Bill Allombert writes ("Re: Automated testing - design and interfaces"):
> > piuparts is a first answer to that: it allows maintainer to check that
> > their package will install, remove and upgrade in a clean environnement.
>
> Yes, and piuparts is a good thing. But, it doesn't allow the
> maintainer to easily test that their package _works_. We currently
> rely on the developer doing some ad-hoc testing. This can easily be
> improved.
piuparts is in very early development phase and it has already quite
expanded the possibility of testing Debian. I am sure there are room
to add new feature for yet more testing.
> > > > If Ubuntu want to improve the testing coverage, you could start by
> > > > submitting patches to packages that don't run test-suite in
> > > > debian/rules. That would profit both Debian and Ubuntu and there are
> > > > lot of work to do there.
> > >
> > > Running the upstream test suite in debian/rules usually isn't the
> > > answer to packaging mistakes, library mismanglements, and the like.
> >
> > There is no reason to restrict debian/rules to upstream test-suite?
>
> Yes, that's true. But I thought you meant that we should submit
> patches to make packages run the upstream test suite (if any) in
> debian/rules.
>
> If that's not what you meant I'm afraid I don't follow you. Can you
> explain what you meant ?
My fault, I wrote literally half of the sentence. What I meant is that
nothing prevent you to run arbitrary test in your 'debian/rules build'
target even if you have to add more Build-Depends. The test is run each
time the package is build. This practice is encouraged but the buildd
admins, is performed automatically for all platforms even unofficial one,
and actually prevent buggy packages to be uploaded.
You can obviously run the upstream test-suite, but you can also write
a new test-suite, add it to the diff.gz and run it in 'debian/rules build',
whether it is Debian specific or not.
Of course, non-Debian specific test-suite should be pushed upstream.
Maybe developers will propose helper packages that provide simple
test framework or standard tests for libraries like we have debhelper,
dpatch, cdbs, etc. but at this time the issue is rather to write
more tests than to speculate.
OTOH, having a dedicated test network that duplicate the buildd network
is going to be very hard to set up.
(I would not mind be CCed since I am only subscribed to -policy).
Cheers,
--
Bill. <ballombe@debian.org>
Imagine a large red swirl here.
Reply to: