[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Official Debian image request: Google Compute Engine (was: Re: Please let's not talk about "clouds")

[+David - please correct me if I get any details wrong in this email.]

Hi Lucas,

On Wed, Apr 24, 2013 at 12:08 AM, Lucas Nussbaum <leader@debian.org> wrote:
> Right. Note, however, that this is not very different from HP shipping a
> server with "Debian" pre-installed. So ideally, we would have a policy
> that addresses both cases.
> Does it sound doable that the requester would provide a script or
> another complete description (e.g. Puppet/Chef) of the customizations
> made from a standard Debian installation (either using d-i or
> debootstrap)?

Let's try this out now. :) As you can tell from my other recent posts
here, Google would like to collaborate with Debian to provide official
Debian images in the Google Compute Engine cloud. Please let us know
if Debian is okay with us doing the plan I describe in this email as
an official Debian image, of course always open to feedback from
Debian and adjustments as appropriate. If the request needs to go
somewhere else, please redirect appropriately.

I've said most of this in prior emails, but here are the details:

Technical details

* The images are built with Anders' script build-debian-cloud
(https://github.com/andsens/build-debian-cloud), which was called
ec2debian-build-ami until we worked with him to add Google Compute
Engine support. Compared with Debian's already-official images for
Amazon EC2, we've adapted this to produce Google's image format
(documented at https://developers.google.com/compute/docs/building_image)
and to add the Google-specific integration code in the next two bullet
points. All of the code we contributed to build-debian-cloud is
released under the Apache License 2.0 and has been merged upstream. It
works for squeeze and for wheezy and uses debootstrap at its core.

* The build downloads and installs three Google-specific debs from a
repository Google runs, all of which are released under the Apache
License 2.0 and which contain only human readable config files and
scripts (no compiled binaries for which source would be needed). One
deb does the rough equivalent of the ec2 init scripts in Anders'
codebase and other first-boot or startup logic, including running any
user-provided startup script. The second deb includes a python daemon
to manage IP addresses in response to customer add/remove commands and
a cron job to manage ssh access (only by default) in line with the
Google Compute Engine documentation. The last deb provides Google's
python script for users to make their own customized images based on
standard Google images. All of these are installed in reasonable
locations: /etc/init.d/ plus a call to update-rc.d, /etc/init/*.conf
for Upstart support (the sysvinit scripts will exit 0 if Upstart is
present), /etc/cron.d for the cron job, and /usr/share/google/ for the

* The build also installs Google's gsutil and gcutil command-line
tools, which allow access to Google Cloud Storage and Google Compute
Engine from within the VM (quite useful for tasks like downloading
startup scripts or managing other VMs). Since these are not currently
packaged, it installs them under
/usr/local/share/google/{gsutil,gcutil} and symlinks them into
/usr/local/bin. These are also released under the Apache License 2.0.

* Further discussion is appropriate to determine what packaging model
makes sense for the Google-specific tools. We look forward to that
conversation to find a solution that works for both Debian and Google.
As mentioned, they are already free software.

* We are working on adding arbitrary kernel support to Google Compute
Engine as soon as we can, but until that's ready, all images run with
a Linux 3.3.8 kernel provided by us with module loading and direct
memory access disabled for security purposes, rather than the
installed kernel. This should be fixed relatively early in the Wheezy
cycle, but not before Wheezy's release. Source code is already
published for the kernel and /proc/config.gz shows the exact

* Though we haven't made this change yet, we may set the default
Debian mirror for Google Compute Engine images to be one which lives
inside our cloud, to save our customers money on bandwidth. We're
using ftpsync to handle the transfer from upstream mirrors.

Technical policy details

* We're currently thinking of an official naming scheme of the form
debian-X_Y_Z-vYYYYMMDD for each image. (Unfortunately periods are not
allowed in image names, so underscores would work instead.) This is
meant to be similar to our other official images but is open to change
if Debian has a strong preference.

Process details

* Anyone in the debian-cloud Google Compute Engine project can upload
images on Debian's behalf accessible to any Google Compute Engine

* As a Debian developer working at Google on this, I've been the only
one to upload images there so far. I am willing to maintain these in
the near term, but am equally open to hand this off to someone else or
grow the maintainer team to include me and others together.

* We're quite happy to grant access to Debian developers who are not
employed at Google to help with image maintenance.

* If Debian wants the non-DD Googlers to refrain from uploading images
to debian-cloud as part of official status, we can accept that.

* There is a debian-cloud-experiments project which we strongly
encourage Debian to use for validation before publishing to the world.

* We will be including the Debian images in our internal testing
procedures to help ensure a good experience for everyone.

Process policy details

* Our goal is that these images would be supportable by both Debian
and Google via usual support channels. Except for the additions we
indicated and (temporarily) the different kernel, everything is
standard Debian in the same way that the official Amazon EC2 images

- Jimmy Kaplowitz
jimmy@debian.org / jkaplowitz@google.com

Reply to: