Re: Moving GCE scripts to Python

2014/1/22 Jimmy Kaplowitz <jkaplowitz@google.com>

Neat. Yeah, a GCE image is simply a raw bootable disk named disk.raw, compressed in a gzipped GNU-format tarball with a filename ending in .tar.gz. We also encourage creating disk.raw as a sparse file and using GNU tar's S flag to minimize image size and add time. The tarball is necessary but hopefully the code is general enough to handle that. Certain bits of our image snapshotting tool gcimagebundle expect everything aside from the bootloader to be in a single MSDOS partition number 1.

I think this is the case for virtualbox provider.

We make various tweaks, such as installing the various integration software I've mentioned before, pointing to our Debian mirror, and (with Tomasz's pull request into the google GitHub fork) the host machine's NTP server, etc. Nothing that changes the essence.

Some of those (setting up a mirror, installing packages) can be done:

1) with plugins (but as it is a requirement for you this is not the best choice)

2) in your plugin task, use the library part that manage package install etc.. if you look at cloud-init plugin task for example, you will see how to add a source (here debian backports) and packages to the image (here "sudo").

The rest of the plugin task can focus on copying some files in the image, setting ntp etc...

The strategy you suggest sounds worth trying indeed.

- Jimmy
On Jan 21, 2014 10:32 PM, "olivier sallou" <olivier.sallou@gmail.com> wrote:
2014/1/21 Tomasz Rybak <tomasz.rybak@post.pl>
As Jimmy wrote in his email from 2014-01-14, I started
looking at GCE-related parts of build-debian-cloud.

I assume that the course for now is changing the scripts
to work in Python, similar to what's being done with EC2
and VirtualBox parts. Should we branch from andsens/python
and work on it, or do something else? Also, who'll create
the main branch (GCE-python-WIP?), into which we would
pull proposed changes? I think the best solution would be
to create such branch in repository
https://github.com/google/build-debian-cloud

As for the work to do, I think we'll need to:
1. change gce file to proper manifest
2. move tasks from tasks/gce to providers/gce and
rewrite them in Python
3. integrate cloud-init when appropriate

If the base image requirement is a raw image file and GCE only adds startup/management scripts for boot etc... you may only develop a plugin and use VirtualBox provider which is in fact a quite generic one (not only virtualbox).

I personnaly use VirtualBox provider for my KVM machines and use the opennebula plugin for the OpenNebula contextualization (will be modified soon to use cloud-init too).

Then, for GCE, it would be, for the user, only a matter of user VirtualBox provider (raw format) and activating the GCE plugin.

Olivier

This is rough idea, and I have not touched
packaging of
https://github.com/GoogleCloudPlatform/compute-image-packages

Have I missed something? I assume we need to have
more detailed plan of moving to Python so anyone
can see what is to be done and volunteer to some
tasks ;-) For now I just want to start discussion
to see what I forgot about.

Best regards.

--
Tomasz Rybak GPG/PGP key ID: 2AD5 9860
Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860
http://member.acm.org/~tomaszrybak

--
gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438

gpg key id: 4096R/326D8438  (keyring.debian.org)

Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438