Effort to test Debian Med repository
[Reply-To set to debian-qa]
Hi,
Thorsten Alteholz started an effort to test the quality of the Debian
Med repository[*] and announced this in a private mail group. I've got
permission to quote in public and I hereby do so because I'm convinced
that this effrot belongs to debian-qa where some tasks are done in a
similar way and others might be interesting for a more general public.
The initial mail is attached and I'm now answering to his fully quoted
response to my reply on his first mail.
On Wed, Aug 31, 2011 at 07:44:52PM +0200, Thorsten Alteholz wrote:
>
> On Tue, 30 Aug 2011, Andreas Tille wrote:
>
>> On Tue, Aug 30, 2011 at 07:32:26PM +0200, Thorsten Alteholz wrote:
>>> - target called 'get-orig-source' in debian/rules
>>
>> I'm not convinced that the availability of this target tells us
>> something. If the watch file is working properly there is no real need
>> for such a target (IMHO).
>
> It might be easier for everybody to have the same workflow for every
> package. The watch file just gets the released tarfile. In lots of
> packages this is not enough, as there has to be done some repacking. So I
> think this target is the only way to achieve such an unified workflow.
I agree that a common workflow is a good idea. It is also correct that
not always uscan can be used because a source tarball needs to be
rewritten. So having something like
make -f debian/rules get-orig-source
could help. However, I honestly think that it is wasting time to reach
the goal by injecting such a target into each rules file. Moreover it
is hard to maintain if you assume that uscan might change some options
or gets a new option which we would consider useful. Having the very
same uscan command in say 75% of our rules files is just duplication of
code which sucks.
So if it is about a common workflow only I would rather suggest to write
a wrapper script which checks for a get-orig-source target in
debian/rules and builds this or just call uscan otherwise. Speaking
about uscan might have new options it might be also feasible to have an
uscan option say --use-get-orig-source-if-exists (or some better name)
which could solve the problem as well (just a half-brew idea - might be
this is stupid).
>>> - debain/watch available
>>
>> Good - but IMHO there is just such an effort - you should discuss this
>> with QA at debian-qa@lists.debian.org.
>
> They just check whether there is a watch file and whether it works. They
> do not check whether it is possible to build a package from the obtained
> data (all packages that get their sources from a repository might have a
> watch-file but that might not be usable as there is no released tar file
> -> mgltools).
So how exactly are you doing more sophisticated checks than the current
check for watch files?
>>> - version of downloaded file is equal to version from changelog
>>
>> Something like this is IMHO implemented in PET[1]
>
> Oh, that is interesting. I didn't know that.
That's why I insisted on posting to debian-qa. :-)
>>> - mergeWithUpstream is set on debian directory
>>
>> This is only for git maintained packages, right?
>
> No, up to now the page is only about packages in svn.
Ahhh, this might be caused by my ignorance of svn-buildpackage. It's
definitely not set on packages I injected if nobody else did so. Do you
think this is an important quality measure? If yes, why not just
iterating through our SVN and just set it. I do not see any harm for
my svn-buildpackage - ignorant workflow.
>>> - package can be build with sid-pbuilder
>>
>> Cool, how are you doing this?
>
> There is an option for svn-buildpackage to call pbuilder
> (--svn-builder="pdebuild ....") for the real work.
Well, yes. My question was rather in the direction like: Are you just
automatically fetching all source archives and fire up pbuilder to test
this or what exactly are you doing?
>>> - lintian check
>>
>> Should be possible to obtain from http://lintian.debian.org/
>> (or even UDD - if not this should be done).
>
> Yes, but these data are only from packages that are already uploaded.
> This might differ from the contents of the repository.
> Also from my experience these data are not always created with the latest
> version of lintian.
Ahhh, seems you are really building the packages from SVN. That's cool.
>>> - piuparts check
>>
>> Are you running piuparts yourself or just fetching other piuparts
>> results.
>
> No, I call it myself and use the same archive as pbuilder.
... against your just created packages?
>> My concerns are: IMHO the tests should be done centrally for *all*
>> Debian packages and we simply extract the results - most preferably via
>> UDD as data storage interface. I have the feeling that parts of the job
>> are just done - so we should definitely coordinate with
>> debian-qa@lists.debian.org.
>
> In a perfect world, there should be no need to gather these informations
> for uploaded packages. Everything should be already checked before the
> upload. So my approach is to provide some data for development and to
> show potential trouble spots.
> If you want to create an all green package, you just need to go from the
> left to right and fix the corresponding issue. The "official" information
> are only available from different pages and you have to collect them
> first.
>From your first mail it did not became clear to me that you are
reimplementing a full package building and testing workflow based
on the current SVN status. IMHO that's really something which is
not yet done.
>> (I actually do not see any need for a
>> private mail exchange - also the debian-med list would have been fine.)
>
> I just wanted to receive flames from a small group before being roasted
> by the whole group :-).
So if you ask me you are afraid of flames but rather are wasting some
kudos you might have just received.
[not really relevant for debian-qa any more]
>> In principle yes - however we need to weight priorities with other
>> important tasks. For instance I would really love to get GinkgoCADx in
>> which is blocked due to complex dependencies and I would think this
>> might our users help more than having get-orig-source targets in every
>> rules file (just to find an example).
>
> Yes, of course, but isn't there already somebody working on these
> dependencies?
Yes, there is, but
http://release.debian.org/migration/testing.pl?package=ginkgocadx
looks complex and might need more manpower. BTW, this was just an
example for not wasting time on some details. Your general effort is
probably more important for the QA aof Debian teams and specifically
for Debian Med.
Kind regards
Andreas.
[*] http://debian-med.alteholz.de
--
http://fam-tille.de
Hi,
during the last couple of days, I wrote a script to test the packages from
the repository. Mainly it looks for the following stuff:
- target called 'get-orig-source' in debian/rules
- debain/watch available
- download of orig.tar.gz with debain/rules or debain/watch possible
- version of downloaded file is equal to version from changelog
- downloaded file is put in a directory called tarballs
- mergeWithUpstream is set on debian directory
- package can be build with sid-pbuilder
- lintian check
- piuparts check
At the moment the script runs for about 7 hours and the results are
available at http://debian-med.alteholz.de
So, now please let me hear your opinion about that. Do you think these
informations are usefull and the script should be run on a regular basis?
Are these enough tests or should there be others? Should we try to get rid
of all red boxes? Shall only the current maintainer care for his packages
or should everybody mess around with it?
Thorsten
Reply to: