automatic testing of dist-upgrades (was: Re: Ideas for additional large scale tests)
[opening a new thread, since it was the original idea]
On Thu, Mar 01, 2007 at 02:59:59PM +0100, Stefano Zacchiroli wrote:
> I'm more worried about how you choose the environment you want to
> dist-upgrade. Various criteria that come to my mind (braindump):
> - base system only (pro: easy to set up, cons: tests a too small set of
> package)
> - \forall task, base system + 1 task (pro: still easy to set up, tests a
> lot more packages, I guess all tasks taken together are a fair share
> of the packages in the archive, cons: do not upgrade issues induced by
> inter-task relationships)
> - base system + a set of random selected packages (pro: easy to set up,
> tests inter-task issues, cons: non-idempotent, i.e. not easy to
> reproduce, no guarantees/idea about how much of the testing domain has
> been effectively tested)
I would add:
- base + most popular packages (arch-wise, modulo together-installability)
- base + largest set of packages installable together (there's more than one)
the above sets can be found with the aid from the edos project, I had a quick
look at their tools but can't find an obvious solution.
> PS hello world, from a QA team newcomer :)
hi there! :)
filippo
--
Filippo Giunchedi - http://esaurito.net
PGP key: 0x6B79D401
random quote follows:
Beauty isn't worth thinking about; what's important is your mind.
You don't want a fifty-dollar haircut on a fifty-cent head.
-- Garrison Keillor
Reply to: