Re: Oracle Cloud Infrastructure
(The statements and opinions in this e-mail are mine, not Google's.)
On Tue, Jan 01, 2019 at 07:24:19AM +0100, Thomas Goirand wrote:
> > Why didn't we use cloud-init? For pretty much the same reason the other
> > cloud vendors haven't, and there was fair bit of discussion about it at
> > the cloud-init summit back in August.
>
> I'm not sure what "other cloud vendors" you're referring to, but AWS use
> it, and so are the vast majority of OpenStack based services (according
> to Jeremy, that's about 72 cloud providers).
Azure installs waagent in its Debian image; AWS installs awscli in its
Debian image ; Google installs a number of GCE-specific packages in
its Debian Image. It may not be the "official Debian image", but
guess which one gets recommended for use by customers in their
official cloud documentation?
> > The Redhat based distributions are running comparatively ancient
> > versions of cloud-init, and we can't inject newer versions in the images
> > used on our platforms, or we fall afoul of their branding rules and
> > wouldn't be able to call it "CentOS", for example.
>
> You will have the very same problem with Debian (except maybe that
> cloud-init isn't that old? I'm not sure what you call old, so I can't
> tell about this specific point...)
So that's going to be the problem. The cloud space is moving super
fast, with new features being added as Microsoft, Amazon, Google,
Oracle, et.al., compete with teach other. These new features will
often require the support in the guest OS, whether it's in the kernel,
or userspace (or, most often, both). If the official guest OS image
won't allow those enhancements, then the cloud companies will either
need to encourage their customers to use distros that *do* allow those
enhancements (read: Ubuntu), or they will provide "value-added"
variants of the distros that they decide they want to support ---
because that's what all of their competitors will be doing, in order
to provide the best possible customer exprience.
> So, if there's a way to have an image work with cloud-init, then there
> is a chance that we can produce a Stretch official image at some point
> for the Oracle cloud. Otherwise, your only hope is to have your agent
> packaged in Debian, or simply not hope for having the word *official*
> stamped (ie: it's going to be your own version of Debian).
I suspect what we will find is that for many cloud companies, being
able to allow customers to use all of the advanced features of their
cloud environment is going to be way more important than having the
word "official" stamped on their distro image.
On Tue, Jan 01, 2019 at 07:27:16AM +0100, Thomas Goirand wrote:
> > "Run-and-done" also doesn't support disk snapshots and disk resizing
> > properly.
>
> You didn't react to my suggestion about having some kind of signal sent
> by Qemu to the kernel. Isn't this a better idea than having a userland
> agent doing it?
You need *both*. There will be some way of sending the signal from
the Cloud environment to the kernel --- for example, via a virtio-scsi
extension. But then, you need to have a way of having the guest
kernel send a signal to userspace so that a database such as MySQL or
Postgress to freeze their Database journal, and then ask the guest
kernel to freeze their journalling file systems, and then tell the
guest kernel to send a signal back to the Cloud environment saying,
"it's OK to proceed with the block device snapshot". Then when the
snapshot is done, the Cloud environment will need to send an "OK to
unfreeze" message which has to get propagated to userspace so the
database can resume their journal.
In the ideal world, this would be standardized and be the same across
all hosting / cloud environments, and the userspace hooks would also
be standardized. The problem is that if you wait for the OpenStack
standardization process to grind away slowly, and then wait for
years(!) for the stable/enterprise distros to get it shipped into a
cloud-init (that today is fundamentally architecturally deficient for
this sort of feature; so ix-nay on using a cloud-init plugin) --- the
companies that just throw support into their agent will be able to let
their customers take advantage of this cloud feature years before
their competitors.
So at least for the short-term, the need for cloud-specific agents is
going to be huge, and good luck trying to get cloud companies to agree
to degrade customer functionality just for the sake of "Debian
official" getting stamped on the image. Or worse, telling customers
they should use "Debian Official" images and then use "curl | bash" to
get the functionality back. For both the cloud company and the
customer, using the "Debian as enhanced by {AWS|Azure|Google|Oracle}"
is going to be more secure and more easy for the customer to use.
Cheers,
- Ted
Reply to: