Hi Chris, On 04-05-18 01:35, Chris Lamb wrote: >>> ie. 75 out of "top" 100 packages according to popcon are missing >>> autopkgtests. >> >> Yes, go provide patches to add them ;) But let's make them smart. > > Well, you're pushing at an open door with me with the "patches > welcome" call to arms :) > > But is there not value to even the smallest test here? I've caught > a ludicrous number of idiotic mistakes in my packages and code in > general with even the dumbest of "smoke" tests. I meant something differently than one can easily read in my words. What I mean (biased by stuff I do at work) is that it is easy to generate reference data for your test and to check the current output to that. This is great to bootstrap a smoke test, but often they are too sensitive to changes and also trigger on progression. Selecting the piece of the reference data you care about and testing only against that is what I meant with "smart". > Indeed, the return-on-investment versus clever tests is often > scary and that's before we start trading clichés such as "the > perfect is the enemy of the good" etc. etc. I rather have tests than no tests. For sure. We can fix bugs in them, sure. What I wanted to at least have said once is that one can also make them easily too poor. I'll stop warning about that ;) >> https://ci.debian.net/status/ > > (I note that these are statistics about packages that actually have > tests.) That is why I said "a bit". But if one knows how many packages there are, one can get the ratio. By the way, I would for myself only look at the passing part because a large part of the FAILS have never passed and thus are nearly worthless for the bystander. Paul
Attachment:
signature.asc
Description: OpenPGP digital signature