[ Please note the cross-post and Reply-To ]
Hi folks,
As promised, here's a quick summary of what was discussed at the Cloud
Images BoF session in Cape Town.
Thanks to the awesome efforts of our video team, the session is
already online [1]. I've taken a copy of the Gobby notes too,
alongside my small set of slides for the session. [2]
We spoke about the state of cloud images in Debian, and how we'd like
to progress with creating *official* Debian images.
Stuff we're currently doing
===========================
* Debian images in many clouds with lots of variation
* Major cloud platforms have custom Debian images
* Lots of variation, no “official” images
Stuff we should be doing
========================
* We want to have official images produced by Debian:
+ for better support
+ because users cannot necessarily trust the 3rd party additions
+ to protect the Debian “name”
+ because they are a trusted starting point
- for users who do not care to build their own custom image (yet?), and
- for users to base a custom image on.
* We absolutely do *not* want to prevent people making and using the
custom images. Debian is Free Software, and we *want* people to do
whatever they want and need with Debian.
Official image requirements
===========================
* For an image to be *official*, it must be produced by Debian people
on Debian machines. That should be possible for all the cloud
platforms, we believe.
* The contents of an official image are important. Clearly, it must
contain things only from Debian main. If drivers or platform
packages are not in main, that is a problem. Virtualbox extensions
could be a problem here. Other images could be described as "based
on" Debian, maybe, but that doesn't mean much. Are we happy for
people to describe their images as "Debian", but not "official
Debian"?
* Is there a way to check an image loaded remotely, after
installation? That depends on the cloud platforms - each has
different features in this area. We'd like for users to be able to
validate that what they have *is* a Debian image.
* What image(s) should we provide?
Thinking of two main options thus far:
+ A simple stable image
+ An image with some backports included (backported kernel,
backported cloud platform tools). Maybe called "Jessie Refresh"
or similar? It's common for cloud platform folks to want updates
for a faster development cycle.
Each has its pros and cons. The most important thing is that
whatever image(s) we produce, we must label clearly so users
understand what they're getting. Could look at these as similar to
Blends, maybe.
* cloud-init:
+ Should we include cloud-init? (A package that does the basic
setup during image boot, to configure your image correctly for
the cloud platform you're running on). Most people want this, but
it seems not everybody. Official images should probably go this
way.
+ Does any cloud platform require contrib or non-free bits for
cloud-init? Most not, but apparently at least Rackspace needs a
specific agent which is not free and it's difficult to even get
hold of a copy. There are ways to do things better here, they're
just not using those methods.
* It would be nice to not have to provide *lots* of different images,
one per cloud platform - that's painful. However, there are some
issues with that plan. Apparently cloud-init can be painfully slow
if things are not configured specifically to use the right data
sources for a specific platform. e.g. enabling the EC2-style data
source (as wanted for AWS images) will cause a 90s timeout for
users on any other platforms. :-( Need more investigation?
* Question: could we use d-i preseeding here? We're not using d-i,
we're building and running complete images so that's not helpful.
* Security updates:
+ Images need to be regenerated whenever any of the included
packages are updated. We're not doing that yet, but should be.
Should images be configured to automatically do a full upgrade on
first boot? Could be controlled by cloud-init - different people
have different requirements here.
+ Should unattended-upgrades be installed in official images by
default? Makes sense for the platform providers, but lots of
users may not want this - e.g. if you have a long-running cloud
instance running infrastructure like a database server, you're
not going to want that to be restarted without careful
planning.
This is a wider question anyway - we should be thinking
about this for all Debian installations, maybe. But it's a
question that cloud people want to try and solve now... Another
issue - if you configure your image to do updates at a specific
time, what happens when 1,000+ machines in a large cloud all try
to upgrade simultaneously?
* QA
+ Official images need good QA
+ Need automation so we can validate our images sensibly. Tt the
moment we don't have much testing going on at all, and that's
bad.
* Plans
+ Suggested a timeline for building official images, etc. That's
not been followed at all so far. We want to get some real images
building and testing, but we've not made any progress so far.
Proposed Sprint
===============
Zach Marano from Google has offered to host a cloud sprint in Seattle
later in the year, to help solve some of our issues. Seattle makes
sense as a location - the three biggest cloud platforms all have their
major development teams based there. We want everybody with an
interest in this area to be able to take part in the discussions and
work sessions - we want to get people working together to improve
things, particularly the official Debian images. More details coming
soon on the cloud list.
Documentation
=============
* For *unofficial* images, for them to be called Debian, what should
we demand of people? Document the changes that have been made, at
the very least. We want to protect our users, and also Debian's
name.
* Once we have standard official images, it should become clearer.
* The best people are already doing the right things here.
* Suggestion that people should use bootstrap-vz and make the
manifest available, as an example. People should be saying what
tooling they used, and what config -> reproducible, trustable
images. Maybe a single git tree showing all modifications, but
that's difficult unless everybody's using the same tools.
Tooling
=======
* The plethora of tools here does not serve users well.
* Riku's session at Debconf15 counted image generating tools and got
into double digits. Everybody starts off scratching their own itch,
solving a simple problem. Over time we need better answers and good
flexible tools to make things work better.
* Maybe in time we'll only bless images as official if they're using
the "right" tool. Don't want to have the argument *now* about which
tools are best, but this is definitely something that should be
discussed at the sprint. A good honest comparison of the features
of the various tools would be awesome!
Near future
===========
Come up with a minimal policy of what an image needs to look like, for
Stretch.
* measure existing images against that policy to call these
"official".
* ensure the tool can add the preferred packages for custom images.
* create a trusted place from where to start.
What packages should be in our cloud images?
============================================
There's not been very much discussion about this so far on the cloud
list, and we should fix this. Lots of variation possible:
* screen vs tmux and other alternative packages
* There's been some worry that the vagrant images are including too
much, e.g. Gnome and KDE libraries being pulled in due to broken
dependencies (pinentry in particular)
* Most people want a *minimal* installation. Tiny images are
preferred with nothing installed extra by default. Let tools like
ansible or puppet add the extra pieces where they're wanted. Every
extra 1% multiplies up with each new install. Some people are
paying per MB here.
* If some folks want some less-minimal images, they should build
their own images with their own selection of nifty tools. There's a
lot of personal choice here and a lot of people won't agree on the
"small list" of extras that should be included.
* It's not just about image size. Look at the security footprint -
complexity becomes problematic. Cloning and stacking images is a
common task that the cloud providers tend to make easy.
* DSA runs cloud images too on a Ganeti installation too. Should they
be "official" images? Up to them. :-)
* Suggestion that the minimal image should just be enough to run apt,
plus *maybe* openssh-server. ssh is maybe *not* entirely required
(could be pulled in via cloud-init, or alternatively removed) but
most people are happy to add it, it seems.
Misc discussion points
======================
* If we go with a single image, the cloud providers may need to be
able to pull in extra packages on demand if it helps the user.
* Should an image look like a d-i installed system? e.g. should it
use grub rather than syslinux? There have been problems with
syslinux, apparently. There's a good point that we might want to
provide as close as possible to a "standard" Debian for cloud
images here to not surprise users. But should that include nanp
etc.? Would we consider an image official if it used sysvinit
instead of systemd?
* The policy for "official" could/should not lock down what we
*choose* to produce, necessarily. Let's not be too prescriptive
here.
* Make sure that our tooling is easy for users to use to create their
own tweaked images. There's a difference in usage here. Many people
will expect to create their own images with their own changes if
they're going to deply a large cloud. In opposition to that, a lot
of people *won't* know what to do to make their own images, or
won't care and will want to use our officially provided images as a
known-stable, known-working system that will do what they
need. We'll be providing a standard image as a trusted place for
people to start.
Next steps
==========
* Organise the sprint - the DPL is happy to approve funding for this
as/when we ask him
* Need to define the official image policy soon, due to the Stretch
freeze. People want clear rules here - join in the discussion on
the cloud list.
[1] http://meetings-archive.debian.net/pub/debian-meetings/2016/debconf16/Cloud_Images_BoF.webm
[2] http://www.einval.com/~steve/talks/Debconf16-cloud-images-bof/
--
Steve McIntyre, Cambridge, UK. steve@einval.com
"Since phone messaging became popular, the young generation has lost the
ability to read or write anything that is longer than one hundred and sixty
characters." -- Ignatios Souvatzis
Attachment:
signature.asc
Description: Digital signature