Re: Cluster available for testing
Hi,
Dr. Jesus wrote (24 Aug 2010 23:24:35 GMT) :
>> - Are you interested in building downstream systems that use
>> live-build -like T(A)ILS- in addition to the standard Debian images?
>> I'm pretty sure it would add value to your testing effort to deal
>> with complicated configurations that make use of the full live-build
>> potential.
> I'm indifferent to what images get tested, but adding T(A)ILS to the
> image testing suite will impact non-T(A)ILS testing.
Sure.
Anyway: I guess we T(A)ILS developpers would be quite happy to have
the automated builds done on your cluster to begin with. Testing built
images is another matter I'll discuss bellow.
An important question I have is the following: is it imaginable to
make the build results (logs at least, built images would be better)
available to T(A)ILS developers? Without this feedback I doubt
autobuilds can be useful at all for us.
Note that this needs to be taken with a grain of salt as we did not
discuss it yet, so please consider I'm only talking in my own name
here.
> That might be a non-issue, I'm only pointing it out because I don't
> know if testing is resource intensive or not.
As far as I know, the state of the art in autotesting Live systems is
quite basic:
1. build an image
2. boot the resulting image in a virtual machine
3. check the final screen state vs. the expected one
4. notify the developers on failure, optionally publishing the boot
video + screenshots online so that they can detect false positives
and get a glimpse of what's happening.
#1 is quick and cheap provided you can build in RAM (~8GB RAM are
needed to build in RAM a somehow complete desktop Live system).
#2 and #3 are quick and cheap too with nowadays virtualization-enabled
#hardware.
I presume your cluster won't have a hard time dealing with this.
On the other hand there is ongoing work to test in a more thorough way
the autobuilt images: unit testing, functionality testing. E.g.
Michael Prokop talked about autotesting in KVM during his talk at
DebConf 10. Such deeper testing greatly interests us T(A)ILS people.
It will probably make autobuilding+testing a bit longer task (more CPU
cycles) but as we're mainly interested in testing potential leaks to
the network the added cost is likely to stay low.
[1] http://penta.debconf.org/dc10_schedule/events/547.en.html
Bye,
--
intrigeri <intrigeri@boum.org>
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr-fingerprint.asc
| Every now and then I get a little bit restless
| and I dream of something wild.
Reply to: