Heh, it's great to see others are able to figure out my framework already, without documentation. I should really get on with writing it though... :-)On 22 January 2014 08:23, olivier sallou <olivier.sallou@gmail.com> wrote:
2014/1/22 Jimmy Kaplowitz <jkaplowitz@google.com>Neat. Yeah, a GCE image is simply a raw bootable disk named disk.raw, compressed in a gzipped GNU-format tarball with a filename ending in .tar.gz. We also encourage creating disk.raw as a sparse file and using GNU tar's S flag to minimize image size and add time. The tarball is necessary but hopefully the code is general enough to handle that. Certain bits of our image snapshotting tool gcimagebundle expect everything aside from the bootloader to be in a single MSDOS partition number 1.
I think this is the case for virtualbox provider.We make various tweaks, such as installing the various integration software I've mentioned before, pointing to our Debian mirror, and (with Tomasz's pull request into the google GitHub fork) the host machine's NTP server, etc. Nothing that changes the essence.
Some of those (setting up a mirror, installing packages) can be done:1) with plugins (but as it is a requirement for you this is not the best choice)2) in your plugin task, use the library part that manage package install etc.. if you look at cloud-init plugin task for example, you will see how to add a source (here debian backports) and packages to the image (here "sudo").The rest of the plugin task can focus on copying some files in the image, setting ntp etc...The strategy you suggest sounds worth trying indeed.
- Jimmy
On Jan 21, 2014 10:32 PM, "olivier sallou" <olivier.sallou@gmail.com> wrote:2014/1/21 Tomasz Rybak <tomasz.rybak@post.pl>As Jimmy wrote in his email from 2014-01-14, I started
looking at GCE-related parts of build-debian-cloud.
I assume that the course for now is changing the scripts
to work in Python, similar to what's being done with EC2
and VirtualBox parts. Should we branch from andsens/python
and work on it, or do something else? Also, who'll create
the main branch (GCE-python-WIP?), into which we would
pull proposed changes? I think the best solution would be
to create such branch in repository
https://github.com/google/build-debian-cloud
As for the work to do, I think we'll need to:
1. change gce file to proper manifest
2. move tasks from tasks/gce to providers/gce and
rewrite them in Python
3. integrate cloud-init when appropriateIf the base image requirement is a raw image file and GCE only adds startup/management scripts for boot etc... you may only develop a plugin and use VirtualBox provider which is in fact a quite generic one (not only virtualbox).I personnaly use VirtualBox provider for my KVM machines and use the opennebula plugin for the OpenNebula contextualization (will be modified soon to use cloud-init too).Then, for GCE, it would be, for the user, only a matter of user VirtualBox provider (raw format) and activating the GCE plugin.Olivier
This is rough idea, and I have not touched
packaging of
https://github.com/GoogleCloudPlatform/compute-image-packages
Have I missed something? I assume we need to have
more detailed plan of moving to Python so anyone
can see what is to be done and volunteer to some
tasks ;-) For now I just want to start discussion
to see what I forgot about.
Best regards.
--
Tomasz Rybak GPG/PGP key ID: 2AD5 9860
Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860
http://member.acm.org/~tomaszrybak
--gpg key id: 4096R/326D8438 (keyring.debian.org)Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438--gpg key id: 4096R/326D8438 (keyring.debian.org)Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438
> If the base image requirement is a raw image file and GCE only adds startup/management scripts for boot etc... you may only develop a plugin and use VirtualBox provider which is in fact a quite generic one (not only virtualbox).I would strongly suggest to refrain from doing that. If the VirtualBox provider is indeed very generic, things should be abstracted into task sets. I have not abstracted much of it until now, since I wasn't aware of the commonalities between providers (given that there are only to - or three now with kvm). If You add GCE as a separate provider, I can take a look at it and create some new tasksets that should make the task resolving a bit easier.
The advantages of having a provider rather than a plugin are manifold.
- Plugins tasks will be resolved *after* the provider tasks, meaning they will be able to remove some provider tasks if they do something more specific.
- You can enforce plugin compatibility in the manifest schemas by looking at the provider string, having a plugin look at what other plugins are loaded is just messy.
Disadvantages of creating a provider as a plugin:
- Any changes you make because the virtualbox provider doesn't quite fit will become hard to understand - one provider adding a task and the plugin removing that same task... you might end up with a tasklist that is completely different from virtualbox (once you are done).
- Changes to the virtualbox provider will now have to be carefully made because other providers suddenly rely on it
- You will have a hard time adding special things to the manifest since the vbox provider applies its own manifest schema
> In your plugin task, use the library part that manage package install etc.. if you look at cloud-init plugin task for example, you will see how to add a source (here debian backports) and packages to the image (here "sudo").What he said! The package API can save you a lot of trouble. Btw, you can add trusted keyrings to apt through the manifest as well.Anders
gpg key id: 4096R/326D8438 (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438