Re: Oracle Cloud Infrastructure
On 12/29/18 12:55, Thomas Goirand wrote:
I may be misunderstanding your comment, but so far as runtime vs
baked-in, we've been attempting to make everything as secure as possible
from the get-go, and assume that customers ignore any good practices or
suggestions we document. Given the security risk of the iSCSI endpoint,
we don't want to be taking risks there.
It looks like I'm missing exactly what you're doing. Could you please
explain what you're doing with iSCSI exactly? Maybe some configuration
example, etc. I have to admit I don't know much about it, apart from
having configured Cinder and let the magic happen with tgtd... :)
The actual configuration changes for iSCSI are relatively minor. Just
tweaks to resiliency, iscsi pings etc, based on experiences in our
operating environment. Back when we launched, we just ran as default,
we've been spending a couple of years tweaking things to strike the
right balance between resiliency and performance.
The security concerns around the endpoint are just that because it's a
cloud environment, root has to be unauthenticated, or we'd either have
to engineer ways to inject in to images the (launch-time?) generated
credentials, etc. iptables gives you the option to restrict traffic
based on UID, so we just lock it down to UID 0.
One of the other design goals for the platform was "secure by default",
and we're trying to encourage customers to think about defence in depth,
putting the firewall on by default at each server, with a particular
configuration is ideal.
We do provide a launch options via the API (and in the web console) that
allow customers to enable or disable features in the instance agent,
with the configuration spit out via a metadata endpoint that the agent
checks on start-up.
In reference to "runtime" configuration above, and in keeping with "do
as little as possible", part of the intent with the instance agent is to
have the agent become responsible for initial run-time configuration for
features outside of cloud-init.
The very point of cloud-init is to gather all the features cloud
providers need, so that we get a single tool to manage them all. So the
correct course of action would have been to contribute to cloud-init,
rather than implement things outside of it. Why didn't Oracle take that
path and decided it was best to do things outside of cloud-init?
I promise you, we don't make decisions based on "how do we make life
hardest for ourselves", and believe me, with FIPS etc to consider,
producing our own agent is most definitely not the easiest option.
We've worked with the main cloud-init developers repeatedly over the
last two years, with bug fixing and the like. OCI has been repeatedly
funding and submitting work upstream to various projects as part of the
cloud build-out, based on issues we've found at scale, features we need
etc. Those fixes and improvements have been going in to everything from
the kernel down to iPXE. It's our strong preference to do things that
way vs build it in-house (I know, you're probably not used to
associating such a thing with Oracle)
Why didn't we use cloud-init? For pretty much the same reason the other
cloud vendors haven't, and there was fair bit of discussion about it at
the cloud-init summit back in August. RedHat doesn't backport features.
The Redhat based distributions are running comparatively ancient
versions of cloud-init, and we can't inject newer versions in the images
used on our platforms, or we fall afoul of their branding rules and
wouldn't be able to call it "CentOS", for example. What you _can_ do
there is add in extra packages, to a limited degree, so it's perfectly
fine to add an instance agent, it's not okay to add in an updated
version of cloud-init. When they added cloud-init to the redhat
repositories, they basically tied all the cloud-vendor's hands. CentOS,
RedHat etc. make up a large slice of the Linux server market, so if we
relied on cloud-init for this runtime configuration, we'd be stuck
without important feature support for customers. The most frustrating
part of that, of course, is that cloud-init only serves one purpose.
Customers that use it are all going to be customers that are operating
in some kind of cloud environment.
There are also other considerations such as Windows, feature release
co-ordination etc, runtime features (one thing Google's instance agent
supports is injecting new SSH keys to instances, which cloud-init as a
"run and done" doesn't support.)
If you have a source package to be reviewed, please give the link to the
.dsc file, so we (or even, *I*) can review the package, and sponsor the
upload to Debian if it is in good enough shape. Then you'll get to
maintain the package directly in Debian until ... well, forever! :)
Awesome, I'll chase up on that one with co-workers as soon as I'm in the
office this week.
Would it be acceptable to be installing the instance-agent as a deb
package via cloud-init, on start-up?
Since you are controlling what the metadata server pushes to your
instance via cloud-init, I guess you could do whatever you want. Though
I'm guessing that this isn't what you're asking.
If your question is:
- can I do: "curl $foo | bash" at startup and hardwire it in rc.local
then my answer would be: of course not! Please do things properly, have
your agent packaged and uploaded to Debian. We have a bit more than a
month to have this happen if you don't want to miss Buster.
curl | bash :shudder: I hope I never see us do that.
I'd like to make sure we're singing from the same hymn sheet. What's the
route forwards? Do we do the work and provide patches for review, do we
need to be contracting with the Debian group to get the work done, or is
the route forwards something else entirely?
IMO, the route forward is:
1/ Get your instance agent packaged, and give me (or someone else) the
link to the source package for review. Probably best would even be if
the packaging was done in Salsa as a new repository here:
https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_cloud-2Dteam_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=ZscEEr9nw3LSVvXoPtSeEPOB99CD0eOeziaYFoP_AX8&m=fKlH8BO1tnqF-BJl-j4mhx-XlF_dxhnsAcaM6bcN2NY&s=_MGmkWEn1vFQNQ4hd4i6CtE01S5OzcCNa-SpyGabxoI&e=
(or in another group, but there is fine...)
2/ Get patches done to debian-cloud-images so it produces an Oracle image.
3/ Get Steve McIntire (aka: Sledge) to also publish your image in our
web site.
4/ Ultimately, get in touch with SPI once all of the above is done, so
that you can give a login / pass to Oracle, so that when we produce a
new Debian image (maybe because of a point release, or a security
update) then it gets uploaded automatically in your could. We can deal
with that last part later.
I hope this helps,
Cheers,
Thomas Goirand (zigo)
That's awesome, thanks. We'll get working on it. I'll speak to our
instance agent devs about 1), I've already dropped them links and
started a conversation with them on slack. The others will probably
come back to me in one form or the other. I'll get started and see what
I can produce.
Paul
Reply to: