Hi,
since October 2012 I've been slowly working on setting up jenkins.debian.net
to run more tests on Debian than what is done so far with lintian and
piuparts.
After presenting this service at DebConf13 (see [1] for the video) it's now
time to announce it to the general Debian development audience.
Before explaining some details of jenkins.debian.net let's start straight with
the important message: jenkins.d.n is useless if it's reports are ignored and
not acted upon. For this to happen, individual teams (or developers) need to
look after the tests they care about, this probably means writing and
maintaining tests as well as caring about the results.
So, what is jenkins [2] and what does it do? Very much simplified it boils
down to:
1. it runs stuff (via cron, via triggers)
2. stuff is treated as failing if the exit code is 1
3. stuff is treated as unstable if output matches some patterns
4. stuff is good, if exit code is 0
5. Failures and status changes cause notifications via irc and email
Tests run on jenkins.d.n:
As of today, 283 different jobs exist in total, each representing one test
each. The most interesting currently are:
44 g-i-installation jobs (graphical debian-installer)
- rescue mode in 12 languages
- four different desktop installs on wheezy, jessie + sid (testing preseeded
installations, then boot the installed system, run iceweasel, apt-get more
in xterm and poweroff)
- several debian-edu profiles in squeeze+wheezy doing similar tests.
All of these jobs produce both screenshots and a video. A current example of
breakage can be seen at [3] currently - see below for another occurance of
the same problem, exposing the root cause of it too).
168 debian-installer jobs:
- all 114 d-i.git repos of packages are build on every commit
- all 51 installation manuals are build on every commit
47 chroot-installation jobs:
- base debootstrap of sid, jessie + wheezy
- four different desktops (Gnome, KDE, LXDE, Xfce4) in sid, jessie + wheezy as
well as a combined full desktop (all desktops plus selected apps) and a
developer install (installing all build depends for the full desktop)
- Haskell jobs - maintained by the Debian Haskell team (their teams page
on the Debian wiki [4] nicely shows how to embedd jenkins results)
- Ruby jobs - maintained by the Debian Ruby team, still in development...
The root for the g-i breakage linked above can also be seen in [5]
(look at the console log) which nicely shows that debootstrap of jessie is
currently broken.
For some one these jobs there are IRC notifications to #debian-qa,
#debian-boot and #debian-edu - I believe some of them should also be send via
email, eg to debian-devel@, qa@, boot@ and @edu.
jenkins.d.n software setup:
The complete setup is described in the jenkins.d.n about page [6], should you
ever want to look it up again :) Also that page has more details than this
mail.
The base system is Debian wheezy plus the long term support jenkins.deb from
jenkins-c-i.org. All of the jenkins jobs are generated from job definitions
kept in git, there is no manual setup involved whatsoever. [7]
Clone the jenkinds.debian.net git repo (see [8]) and the INSTALL file to learn
how to set up a similar system (or this system again, eg. as a debian.org
service one day).
The two most important directories in that git repo are ./job-cfg and ./bin
(which is copied to /srv/jenkins/bin/ and includes the bash scripts which the
jobs are running+testing.)
Currently also a patched version of jenkins-job-builder is used (see [9]), but
I plan to get these patches upstream as well and one day run
jenkins-job-builder from Debian stable.
All tests currently are only run on amd64.
Patches for all of this are very very very much welcome. In the beginning I'll
want to review patches before merging and applying them to the system, but
what I'm really looking for are other people wanting and able to share this
responsibility.
jenkins.d.n hardware setup:
Thanks to profitbricks.com for sponsoring the setup:
The (virtual) hardware jenkins.debian.net is running on, using 6 cores and
12 GB RAM currently, has been kindly sponsored by Profitbricks.com who are
proud to support us. Conrad Wood (Deputy CTO) explains why: "We use Debian
in our data centers, so we rely on its quality, that's one reason why we are
proud to support this. We actively support open source projects by committing
our patches upstream and Debian is a major platform for our development".
Profitbricks is also happy to give us more ressources for jenkins.d.n if/when
we need them, plus I already have three other offers for more real and virtual
hardware - so if you want to offer some help, please offer something else than
hardware, we really have enough ressources for now and next year as well.
Beta software disclaimer & big plans
Right now, all of this is still quite beta and suffering from being mostly
set up and maintained by one person, me. So some jobs are not as stable and/or
well written as they should be. Patches really really welcome.
Nonetheless, as a result of some discussions happening at DebConf13, the
release team plans to use this setup, eg for running autopkgtests, in the
future - I'm definitly excited to support this.
Other goals for next year:
- build a team maintaining jenkins.d.n
- have more teams write jenkins.d.n jobs
- have more teams use jenkins.d.n, that is, care about the results
And then there is a rather long TODO list in that jenkins.d.n. git repo too.
Please halp! :-)
Feedback, questions, ideas, pull requests? Please send them to
debian-qa@lists.debian.org! If you want to discuss ideas on some other
specific list, please do and please keep debian-qa@ in CCed.
cheers,
Holger
About Profitbricks
Founded in 2010, Germany based ProfitBricks is a cloud computing IaaS 2.0
provider with a focus on the B2B-market. ProfitBricks sets the bar on
offering its customers a low-cost cloud computing platform based on the
newest, high standard virtualization technology, delivering innovative
features like live vertical scaling and full virtual networking within a
software defined data center approach. ProfitBricks is committed to giving
back to the community e.g. by extensively enhancing and patching the KVM
kernel.
About Debian
The Debian Project was founded in 1993 by Ian Murdock to be a truly free
community project. Since then the project has grown to be one of the largest
and most influential open source projects. Thousands of volunteers from all
over the world work together to create and maintain Debian software.
Available in 70 languages, and supporting a huge range of computer types,
Debian calls itself the universal operating system.
[1] http://meetings-archive.debian.net/pub/debian-meetings/2013/debconf13/high/995_An_introduction_to_jenkins.debian.net.ogv
[2] http://jenkins-ci.org/
[3] http://jenkins.debian.net/job/g-i-installation_debian_sid_daily_xfce/262/artifact/results//snapshot_000700.png
[4] https://wiki.debian.org/Haskell
[5] http://jenkins.debian.net/job/chroot-installation_jessie_bootstrap/99/
[6] http://jenkins.debian.net/userContent/about.html
[7] lies: jenkins' main config.xml has been configured using a browser and all
Jenkins plugins (see the INSTALL file in git for a list) were installed
manually - but at least this is backed up in a git repo via the SCM Sync
Configuration Plugin...
[8] git clone git://git.debian.org/users/holger/jenkins.debian.net.git
[9] git clone git://github.com/h01ger/jenkins-job-builder/
Attachment:
signature.asc
Description: This is a digitally signed message part.