On Mon, 27 Apr 2015 21:17:32 +0200 Holger Levsen <holger@layer-acht.org> wrote: > Hi Neil, > > thanks for sharing your interesting plans and sorry for the late > reply... > > On Freitag, 20. Februar 2015, Neil Williams wrote: > > > what is this lava stuff? > > https://packages.debian.org/jessie/lava-server > > Continuous integration system currently in use for kernel and > > bootloader testing on ARM but open to other architectures as well. > > [...] > [...] > > Debian has a number of QA tasks based around individual packages but > > little support for QA tasks involving architectures other than > > amd64, > > that's absolutly the case and I'm glad you are working on improving > these areas. > > which reminds me to point out: piuparts.d.o shall be extended this > year to test other architectures as well. Help on this is much > welcome, I'd be glad to mentor and I know Andreas has some patches > flying around - but still, someone new working on this would be > awesome and I'd be really glad to get her or him started. > > That said, hardware offers for non-x86 and !ppc64 piuparts-slaves > would also be very welcome...! You'll have to remind me what the requirements are for piuparts - presumably an up to date schroot would be enough? I'm currently implementing schroot support inside the new LAVA dispatcher. This could provide piuparts slaves on ARM devices - although it's more likely that the devices would be doing more than just piuparts and that a piuparts job would be scheduled amongst other schroot tests. (A device offering schroot would only offer schroot and wouldn't be involved in tests which would replace the kernel, OS etc.) > > tasks involving multiple packages interacting or disparate systems > > or reproducible testing of large scale upgrades. > > jenkins.d.n has 384 tests (involving 5 debian suites) in the category > chroot- installations, which also tests installations and upgrades of > package combinations, see > https://jenkins.debian.net/view/chroot-installation/ > > do you have ideas what to add? Chroot tests suffer from limitations with daemons (changed port numbers if the same daemon is running outside etc.) and are subject to whatever the running kernel can offer. I'm thinking of tests where two independent (possibly virtual) machines are tested over a particular protocol. Possibly whether a client handles service interruption cleanly when the daemon is upgraded etc. This allows the test writer to specify which kernel to use at each end. The limitation at the moment is the bootloader - without custom hardware, we can't allow tests to replace the bootloader but we can allow interaction with an existing bootloader to change the boot operations. Everything after that point is up for grabs. > > > > Debian Installer test parallelisation on x86 and ARM - utilise > > > > support from FAI to completely automate testing of DI, possibly > > > > including support for video capture cards (where hardware is > > > > available). > > > FAI as in fully automatic install or something else? > > Yes, fully automatic install so that DI builds can be tested on > > lots of different devices at the same time, automatically. > > so FAI as in "fully automatic install" but not as in src:fai in > Debian? that's confusing. src:fai has a slightly different focus - this is intended to be part of the CD image build test cycle at each release so that the installer itself is tested rather than achieving a set of install operations. I had a start on this for ARM over the Jessie release weekend [0]. Think of it as automated installer validation. Where src:fai preseed files and other support can be used, that can be part of it. It could also be used to test src:fai itself. [0] http://linux.codehelp.co.uk/?p=208 > > What has become apparent within the LAVA usage is that LAVA is one > > piece of a solution for implementing CI in this way. Once the > > results are available, the data needs to be presented in a way that > > is relevant to one specific audience. Teams are now using LAVA as > > the data source and building custom frontends to be able to close > > the CI loop and get relevant information back to developers in as > > short a timeframe as possible, directly after commits or merges. > > e.g. http://kernelci.org/ > > > > So something like ci.debian.net could incorporate this data or > > another frontend could be designed which collates the raw LAVA data > > into a format suitable and meaningful to Debian folk. > > hm, I see. So somewhat similar to reproducible.debian.net which > basically is just a collection of static pages created by > jenkins.d.n :) Possibly static, possibly generated from exported data (as kernelci.org does). -- Neil Williams ============= http://www.linux.codehelp.co.uk/
Attachment:
pgp7B_sOMBd5x.pgp
Description: OpenPGP digital signature