[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Timeline for official Debian cloud images for the Stretch release



On Thu, 31 Mar 2016 19:18:23 +0200
Thomas Goirand <zigo@debian.org> wrote:

> Hi Steve,
> 
> On 03/30/2016 02:55 AM, Steve McIntyre wrote:
> > IME it's much easier to automate using VMs if possible, but I'm open
> > to being convinced otherwise.  
> 
> Sure, but how do I get a new VM for it? Should I write a mail to
> dsa@ ?

One of the benefits of lava.debian.net is that the machine running the
VM can be local to the tester with the results being published through
lava.debian.net. That's useful for some tests but tends to lead to that
machine only being available when the local tester is available. I'm
currently looking at adding a VM host box and that would be available
to everyone using lava.debian.net.

Bare metal boxes require a lot more infrastructure to automate
(remotely automated power control for one) but are the life blood of
most of the current LAVA instances.
 
> Not sure if you want more insights here, but I'll give them (as maybe,
> others will happily read). Feel free to skip if you have no time.
> 
> On 03/30/2016 02:55 AM, Steve McIntyre wrote:
> > We could do something like that quite readily, I hope. Neil Williams
> > has already started setting up a Debian instance of LAVA [1] which
> > is designed to do exactly this kind of thing! See lave.debian.net if
> > you're interested.
> > 
> > [1] https://wiki.linaro.org/LAVA  
> 
> I was at his presentation of LAVA in Portland.

Things have moved on in the meantime, I've submitted a request
for another LAVA talk in Cape Town.

> As long as I get a new
> reseted Debian on which I can ssh to, then I can do the rest and
> install OpenStack on it, and test a Debian image.

Yes. Your LAVA job would download the image to use for the base VM, add
some support scripts that include files from git repositories which you
maintain, boot that VM and give you multiple automated SSH connections
into it. Each connection then runs it's own test scripts. (In the
documentation, this method is termed Secondary Connections.) Once
inside the VM, the test writer has a free hand. (If you wanted to
reboot the VM that would need to be handled by LAVA rather than simply
killing the VM.)

Note: interactive SSH sessions (hacking sessions) are supported too but
require some network configuration to allow you to access the machine.
With a local box that is trivial, for a hosted box it requires
permission to modify the firewall.

> Otherwise, a VM
> that can be rebooted (and reinstalled from scratch?) with some kind
> of API will do. Maybe LAVA can provide such an API?

Using LAVA to build images tends to not make the best use of LAVA -
Jenkins is usually better suited to running the debootstrap wrapper you
choose for your image build.

Rebooting a VM needs to be done in the test job rather than by scripts
in the test itself as LAVA is monitoring the VM to determine whether
the job has gone into a tailspin and needs to be killed.
 
> Here's an example of a test run of OpenStack and its functional test
> suite, called Tempest:
> https://mitaka-jessie.pkgs.mirantis.com/job/openstack-tempest-ci/64/consoleFull
> 
> As you can see, this is a test using Jessie, and using the *next*
> version of OpenStack (called Mitaka), to be released in a few days,
> and currently living in Experimental.

How many of the dependencies live in the base VM image is up to the
test writer but it would make tests much faster if the build scripts
created a VM image with things like mysql-server-core pre-installed.
(Speed is a useful goal if the tests are to be commit-based to be able
to get a result back to the developer within a usable timeframe.)

This is a similar job used within the LAVA software team for our own
test suite:
https://staging.validation.linaro.org/scheduler/job/143951/complete_log?debug=on&section=test

The results of the job can be seen here:
https://staging.validation.linaro.org/results/143951/lava-server-pipeline-unit-tests

This is one of the longer jobs we use as running this test takes a fair
amount of preparation. The test took 18 minutes. We also run local
tests using VM images with the base packages pre-installed.

This is a test on an ARMv8 board using secondary connections:
https://staging.validation.linaro.org/scheduler/job/141505.0

(It doesn't launch a VM, it provides multiple ssh connections to a bare
metal device running a user-specified NFS root filesystem.) This is the
kind of test that just cannot be done in a build-based system or single
VM.

One thing I will mention, if a particular device is not available on
lava.debian.net, it can be added. In the meantime, check one of the
other instances of LAVA - if that is validation.linaro.org or
staging.validation.linaro.org, there is a process by which community
members can submit jobs to those instances.

https://staging.validation.linaro.org/static/docs/v2/development.html#register-with-linaro-as-a-community-contributor

Linaro is also looking at bare-metal ARMv8 testing of cloud-type
systems but that involves a large investment in hardware and
infrastructure.

> I'll have to setup something using Stretch instead, and whatever
> version of OpenStack that will be in it (maybe Mitaka, maybe the next
> one called Newton). It shouldn't be too hard to do in Stretch,
> provided there's no new added bugs, which I don't think there is, as
> I can already run all unit tests at build time (this env. will only
> run functional tests).
> 
> This currently uses the Cirros image, because it's smaller (a few
> megs), so it's faster to test OpenStack. In the context of testing
> the Debian image, I could switch to the Debian image: it's only 1 or
> 2 lines to change, as this was what I was doing to begin with.

LAVA is in the midst of a large migration from LAVA V1 to LAVA V2.
lava.debian.net only runs V2 test jobs and tracks the packages as
uploaded to jessie-backports. Now is the right time to provide input of
your use-cases, your Test Plans and what you want to do with a system
which can provide automated access to VMs and bare metal.

Steve has the role of coordinating lots of use cases from teams in
Linaro and from the community. So talk to us, take a look at what can
currently be done and help us improve the documentation. Let's expand
tests within Debian from in-build tests and package-based tests to tests
which are not constrained within a single system and not constrained by
pre-configured base configuration.

-- 


Neil Williams
=============
http://www.linux.codehelp.co.uk/

Attachment: pgpHlBjWw0Xpg.pgp
Description: OpenPGP digital signature


Reply to: