Re: Building GCE images with bootstrap-vz

On 24 February 2014 20:25, Tomasz Rybak <tomasz.rybak@post.pl> wrote:

Dnia 2014-02-17, pon o godzinie 17:57 -0800, Jimmy Kaplowitz pisze:

> Congratulations! It's great to see this. I hope to try it soon, and
> join the polishing effort once I get a bit more time. Other Debian
> cloud folks may also be interested in collaborating.
>
>
> Image registration and upload is not critical to have in the script
> before merging - for the official images we've been building, we've
> done various rounds of tests before publishing the results, so the
> process was more complex anyway. It's definitely a desirable feature,
> but doesn't need to be viewed as a merge blocker.

Here is the list of all tasks from non-Python GCE build-debian-cloud
parts that I've omitted:
21-prefer-backports-kernel
26-remove-goog-apt-source
30-grub - I've commented out some of the serial console configuration
35-disable-ipv6
39-set-hostname
75-compress-image and 80-save-image - I use contant name disk.raw and
image.tar.gz
95-register-image

I guess 21-* could be moved to more generic plugin to set apt
preferences, to be used by all providers.

30 (providers.gce.tasks.boot.ConfigureGrub) contains lines setting
serial console, I just haven't tested them. I had also some problems
with setting GRUB timeout to 0, so it is also commented out.

Best regards.

--
Tomasz Rybak GPG/PGP key ID: 2AD5 9860
Fingerprint A481 824E 7DD3 9C0E C40A 488E C654 FB33 2AD5 9860
http://member.acm.org/~tomaszrybak

> Code does not seem very sophisticated. Two issues that seemed the most important were:

> printing the list of plugins that will be used (I know that they are in log, but having them on screen would be nice)

You can just use --debug. You think plugins loaded should be logged as <info> rather than <debug>?

> and my lack of intuition where to put some information.

I can understand that. It's hard to write a guide for that though, since there can be a lot of different scenarios.

Maybe a FAQ for the most basic cases would be useful? Developers could maybe extrapolate from that.

> For example I was wondering whether to put packages to install in manifest or into new plugin.

Only use plugins when it is *really* a plugin, e.g. something cross-platform. If it's package installation that is closely tied with GCE but optional, make it a switch in the manifest.

Come to think of it: There currently isn't a provider specific settings section. That could be useful for things like: VirtualBox Guest Additions, EC2 startup scripts, GCE packages etc.

> As for package installation - do you think it would be good idea to move code choosing which kernel package to install to common plugin?

I do. -- For squeeze on EC2 we need a specific xen kernel package. But that should just be an exception to the rule.

> I had problems with kpartx -a; it did not create files in /dev/mapper quickly enough, so the next steps (either mkfs or grub-install) were not able to find mapped devices and failed.

Are you 100% sure that this is in fact the case and that it wasn't some other side-effect that caused kpartx -a to suddenly work.

I ask because I had a very similar problem, which I then thought I solved by - you guessed it - sleeping... only later on to find out that the problem was actually with the partitioning the failed in special circumstances. I can't for the life of me remember what exactly it was, nnngh!

> Do you suggest adding more generic plugin, with potential line in manifest with NTP server address?

Is NTP optional on GCE or is it a must have because of some other stuff?

If it is optional, I'd suggest a plugin as well. Just hack it together in a minimal way, you can always add options later on :-)

> Here is the list of all tasks from non-Python GCE build-debian-cloud parts that I've omitted:

> 35-disable-ipv6

Might be useful to have in common, since AWS doesn't support it yet either.

> 39-set-hostname

Should also be in common/. The set-hostname task is actually next on my todo list.

p.s.: Try out the development branch. I've added quite a few fixes. Also: I think you might benefit from the minimize_size plugin using zerofree. It zeroes out unallocated disk space, which increases the effect of compression A LOT. I tested it on an almost barebones setup with ssh, sudo and nfs-client: (gzipped) 118 MB :-)