[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Oracle Cloud Infrastructure



On 12/29/18 12:55, Thomas Goirand wrote:

I may be misunderstanding your comment, but so far as runtime vs
baked-in, we've been attempting to make everything as secure as possible
from the get-go, and assume that customers ignore any good practices or
suggestions we document.  Given the security risk of the iSCSI endpoint,
we don't want to be taking risks there.
It looks like I'm missing exactly what you're doing. Could you please
explain what you're doing with iSCSI exactly? Maybe some configuration
example, etc. I have to admit I don't know much about it, apart from
having configured Cinder and let the magic happen with tgtd... :)

The actual configuration changes for iSCSI are relatively minor. Just tweaks to resiliency, iscsi pings etc, based on experiences in our operating environment.  Back when we launched, we just ran as default, we've been spending a couple of years tweaking things to strike the right balance between resiliency and performance.

The security concerns around the endpoint are just that because it's a cloud environment, root has to be unauthenticated, or we'd either have to engineer ways to inject in to images the (launch-time?) generated credentials, etc.  iptables gives you the option to restrict traffic based on UID, so we just lock it down to UID 0.

One of the other design goals for the platform was "secure by default", and we're trying to encourage customers to think about defence in depth, putting the firewall on by default at each server, with a particular configuration is ideal.

We do provide a launch options via the API (and in the web console) that
allow customers to enable or disable features in the instance agent,
with the configuration spit out via a metadata endpoint that the agent
checks on start-up.

In reference to "runtime" configuration above, and in keeping with "do
as little as possible", part of the intent with the instance agent is to
have the agent become responsible for initial run-time configuration for
features outside of cloud-init.
The very point of cloud-init is to gather all the features cloud
providers need, so that we get a single tool to manage them all. So the
correct course of action would have been to contribute to cloud-init,
rather than implement things outside of it. Why didn't Oracle take that
path and decided it was best to do things outside of cloud-init?

I promise you, we don't make decisions based on "how do we make life hardest for ourselves", and believe me, with FIPS etc to consider, producing our own agent is most definitely not the easiest option.  We've worked with the main cloud-init developers repeatedly over the last two years, with bug fixing and the like. OCI has been repeatedly funding and submitting work upstream to various projects as part of the cloud build-out, based on issues we've found at scale, features we need etc.  Those fixes and improvements have been going in to everything from the kernel down to iPXE.  It's our strong preference to do things that way vs build it in-house (I know, you're probably not used to associating such a thing with Oracle)

Why didn't we use cloud-init?  For pretty much the same reason the other cloud vendors haven't, and there was fair bit of discussion about it at the cloud-init summit back in August. RedHat doesn't backport features.  The Redhat based distributions are running comparatively ancient versions of cloud-init, and we can't inject newer versions in the images used on our platforms, or we fall afoul of their branding rules and wouldn't be able to call it "CentOS", for example.  What you _can_ do there is add in extra packages, to a limited degree, so it's perfectly fine to add an instance agent, it's not okay to add in an updated version of cloud-init.  When they added cloud-init to the redhat repositories, they basically tied all the cloud-vendor's hands. CentOS, RedHat etc. make up a large slice of the Linux server market, so if we relied on cloud-init for this runtime configuration, we'd be stuck without important feature support for customers.  The most frustrating part of that, of course, is that cloud-init only serves one purpose.  Customers that use it are all going to be customers that are operating in some kind of cloud environment.

There are also other considerations such as Windows, feature release co-ordination etc, runtime features (one thing Google's instance agent supports is injecting new SSH keys to instances, which cloud-init as a "run and done" doesn't support.)

If you have a source package to be reviewed, please give the link to the
.dsc file, so we (or even, *I*) can review the package, and sponsor the
upload to Debian if it is in good enough shape. Then you'll get to
maintain the package directly in Debian until ... well, forever! :)
Awesome, I'll chase up on that one with co-workers as soon as I'm in the office this week.
Would it be acceptable to be installing the instance-agent as a deb
package via cloud-init, on start-up?
Since you are controlling what the metadata server pushes to your
instance via cloud-init, I guess you could do whatever you want. Though
I'm guessing that this isn't what you're asking.

If your question is:
- can I do: "curl $foo | bash" at startup and hardwire it in rc.local

then my answer would be: of course not! Please do things properly, have
your agent packaged and uploaded to Debian. We have a bit more than a
month to have this happen if you don't want to miss Buster.
curl | bash :shudder:  I hope I never see us do that.
I'd like to make sure we're singing from the same hymn sheet. What's the
route forwards?  Do we do the work and provide patches for review, do we
need to be contracting with the Debian group to get the work done, or is
the route forwards something else entirely?
IMO, the route forward is:
1/ Get your instance agent packaged, and give me (or someone else) the
link to the source package for review. Probably best would even be if
the packaging was done in Salsa as a new repository here:

https://urldefense.proofpoint.com/v2/url?u=https-3A__salsa.debian.org_cloud-2Dteam_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=ZscEEr9nw3LSVvXoPtSeEPOB99CD0eOeziaYFoP_AX8&m=fKlH8BO1tnqF-BJl-j4mhx-XlF_dxhnsAcaM6bcN2NY&s=_MGmkWEn1vFQNQ4hd4i6CtE01S5OzcCNa-SpyGabxoI&e=

(or in another group, but there is fine...)

2/ Get patches done to debian-cloud-images so it produces an Oracle image.

3/ Get Steve McIntire (aka: Sledge) to also publish your image in our
web site.

4/ Ultimately, get in touch with SPI once all of the above is done, so
that you can give a login / pass to Oracle, so that when we produce a
new Debian image (maybe because of a point release, or a security
update) then it gets uploaded automatically in your could. We can deal
with that last part later.

I hope this helps,
Cheers,

Thomas Goirand (zigo)

That's awesome, thanks.  We'll get working on it.  I'll speak to our instance agent devs about 1), I've already dropped them links and started a conversation with them on slack.  The others will probably come back to me in one form or the other.  I'll get started and see what I can produce.

Paul


Reply to: