[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Ideas for additional large scale tests



On Thu, Mar 01, 2007 at 02:27:54PM +0100, Filippo Giunchedi wrote:
> how about testing automatic dist-upgrades like stable->testing?

I love this idea: as a maintainer is quite a burden to take care of
this, a massive testing effort would help a lot of DDs out there!

> of course this is not trivial as we want to run this in a
> noninteractive/preseeded fashion, but better than nothing :)

I don't see this as the major problem to tackle (but still a problem).

I'm more worried about how you choose the environment you want to
dist-upgrade. Various criteria that come to my mind (braindump):
- base system only (pro: easy to set up, cons: tests a too small set of
  package)
- \forall task, base system + 1 task (pro: still easy to set up, tests a
  lot more packages, I guess all tasks taken together are a fair share
  of the packages in the archive, cons: do not upgrade issues induced by
  inter-task relationships)
- base system + a set of random selected packages (pro: easy to set up,
  tests inter-task issues, cons: non-idempotent, i.e. not easy to
  reproduce, no guarantees/idea about how much of the testing domain has
  been effectively tested)

Cheers

PS hello world, from a QA team newcomer :)

-- 
Stefano Zacchiroli -*- Computer Science PhD student @ Uny Bologna, Italy
zack@{cs.unibo.it,debian.org,bononia.it} -%- http://www.bononia.it/zack/
(15:56:48)  Zack: e la demo dema ?    /\    All one has to do is hit the
(15:57:15)  Bac: no, la demo scema    \/    right keys at the right time

Attachment: signature.asc
Description: Digital signature


Reply to: