Bug#656595: debian-policy: Mandatory provide of testsuites or demos for libraries
Package: debian-policy
Severity: normal
Dear Maintainer,
What led up to the situation?
What lead me here was the fact that libogre 1.7.3-3 that is basically a package
that does not work at all made it into the testing branch. This should not be
possible.
What I learnt from the policy that the demo and testsuite had been stripped due
to containing sections that were not licensed acceptable to main repo. Lack of
testing on the maintainers path. Worse lack of means for me as a end user to
simply test a library and find out if it working.
Result was a library that got threw usable as a non working library and into
testing due to being rarely used package. Excuse that is rarely used should
not cut it. This is also not the first time I have seen a rarely used package
get from unstable to testing without being caught as defective. Most have cost
me many hours trying to work out what has gone wrong with what I am working on.
*** My proposed policy change***
If upstream library source package contains a testsuite. This testsuite must
be packaged as package-testsuite unless it cannot be ship by any one of the
rules over main, contrib or non-free. This even include the case that the
library ends up in main and the testsuite ends up in non-free or contrib with
the source files split in 2.
If upstream package contains demos same rule as testsuite except its package-
demos.
If upstream library source package does not contain testsuite or demos or due
to licensing they cannot be provide by the debain system a package-testsuite-
min must be created. If testsuite or demos packages exist testsuite-min does
not have to be created.
Package-testsuite-min will contain at least little program that inits the
library and shut-down down the library done in the min amount of code. Enough
to prove the library will at east basically fire up and has some chance of
providing some functionality. Extra debian license compadible tests can be
added to this file.
Before sending a package upstream to unstable at least one of the following has
to be run testsuite, demos or testsuite-min by the package maintainer to prove
the library. What has been run with any failures. What has been run on
library to test to be record in the package description "tested by: *name*
using *testsuite/demo/testsuite-min* on *arch/archs*" with optional "failures
recorded in *file*" if there are any failures.
Recommended if possible for maintainers to integrate package-testsuite to deb8
*** End ***
This is to remove delete being used as a solution with testsuites. Reason must
provide is issues with libraries sometimes can be complier. If library
provided testsuite runs proper and testsuite built by current complier does not
you know you most likely have a complier failure on your hands.
Reason why this is a possible security flaw is there are most likely a lot of
libraries who test-suite is not being built or used by the maintainers. 10
days in unstable is not going to test the library as completely as one proper
run of the libraries test suite.
deb8 is also not going to cut it as more libraries add items like opencl
support. Reason why a library might fail test suite due to hardware
difference. Hardware difference the end user need to be able to testsuite to
know what is going on. Again dependable testsuite that is not going to give
false reading because the user damaged the complier strangely.
Also deb8 building from source is not going to help you in case of defective
complier locally.
Yes this will mean more disc space. To address some of the disc space issue I
suggest to more sub repo parts. tests and tests-nonfree. These are the repos
were all testsuites and demos go. Testsuites and demos don't need to be
replaced out to every mirror site.
This provide formal proof a test was done on a package by a maintainer that is
confirm-able in case of a failed package. This also clearly suggest to
maintainers what I was told that it would not be a problem if more programs
were using the library is not suitable ever.
A package without a tested by: *name* using *testsuite/demo/teststuite-min*
should be rejected after 12 months from this policy change. This should give
people enough time to alter system.
So yes this fault is security and stability of Debian. If my alteration to
policy is not something else mandatory about testing has to go in to make it
clear in policy sending items upstream without any form testing will not be
tolerated. Library makers should not depend on application makers to test
there libraries completely for them.
Peter Dolding
-- System Information:
Debian Release: wheezy/sid
APT prefers stable
APT policy: (700, 'stable'), (650, 'testing'), (600, 'unstable'), (500, 'proposed-updates'), (400, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 3.1.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Reply to: