Re: Debian images on Microsoft Azure cloud
On Wed, Nov 18, 2015 at 8:12 AM, Martin Zobel-Helas
<martin.zobel-helas@credativ.de> wrote:
> Hi,
>
> On Wed Nov 11, 2015 at 15:18:51 -0500, Brian Gupta wrote:
>> (Note: Although some of you may know that I am a member of the Debian TM
>> team, I am raising the following issues as a long-time participant in the
>> debian-cloud group/mailing-list. I also apologize upfront for the length of
>> this email, and for any inevitable omissions.)
>
> I see some conflict of interest here, but i will answer the technical
> questions.
Responding here as an individual member of the TM team. (Not on behalf of the
TM team, as I've been swamped and we haven't had a chance to discuss).
While I'm not sure if I have a "conflict" of interest, I will admit that I am
hoping to kill more than one bird with a single stone. First, I'd like to
help you resolve your TM request, in a way that is agreeable to the Debian
Project and your team.
Second, as there is a growing consensus developing for Debian to be publishing
official cloud images, and at least one of the teams involved in publishing
Debian images has expressed an interest in building/blessing cloud images,
I'm hoping that this discussion promotes the development of a more fleshed out
set of standards and policies related to Debian cloud images. This will
hopefully make everyone's lives easier in the future.
I'll also add that it's not the TM team's role to privately set technical
policy, which is why we advocated to have this discussion in public
with the appropriate technical teams.
> First of all, i want to stress out, that i didn't request the trademark for the
> name "Official Debian images on Microsoft Azure cloud". I am happy to help here
> that we, at some future point, might reach that status, but as per discussion
> (where never a final decission was made!) during DebConf15 we mostly agreed
> that we should be careful what we call "Official Debian". Therefor we would
> like to use "Debian Jessie/Debian Wheezy build for Microsoft Azure".
So my position here, which is being shaped by ongoing discussions, is that you
are asking the Debian TM team for blessing or sanctioning to call a product you
built, "Debian". That's about as "legally official" a blessing as one can get.
Going forward, if one of our teams (say debian-cd) decided they wanted to also
build Azure images, do you feel that there should be two separate sets of
images, built by two separate sets of DDs? Which is official, if they both
received the blessing of the project? From a user's point of view this could
lead to great confusion, and would make the job of the TM team quite a bit
harder, especially if there were disagreements about how the images are
built.
At this point in time, the only "non-official" blessing I would prefer the TM
team be involved in would be perhaps for temporary transitional images
that may not quite be ready for official status, but the DDs are committed
to resolving any issues, the issues aren't considered showstoppers, and the
technical teams involved support that plan. (Definitely open to feedback here,
and this may change over time as we evolve our policies.)
>> 1) the image includes only software available in Debian [2]
>
> Check. Our image only includes software available in Debian, except waagent.
> waagent is available in Debian itself, but not the version we currently need
> for the image, see my initial mail for more information.
>
>> 2) the image generation process is controlled solely by Debian [2]
>
> Check. Only DD have write access to the Jenkins instance used to generate
> images and control the scripts used by the process. Apart from the usual vendor
> operation staff, of cause.
>
>> 3) the image is generated using tools available in Debian, or maintained [2]
>> by Debian
>
> Check. The tools are maintained by DD.
>
>> 4) Only DFSG-compliant Software in the image. Only main enabled, with
>> perhaps a temporary exception for backports [3], for specific enablement
>> software
>
> Check.
>
>> 6) the images most provide a user experience (in terms of default
>> choice of packages, or of default configuration) identical to other
>>means of installing Debian. Differences must be documented and justified.
>>[4]
>
> Check.
>
>> 7) Debian kernel [5]
>
> Check. For Wheezy we need to use the kernel from backports.
>
>> 8) Built using Debian infrastructure [6] (I think this should be modified to
>> have a caveat, "to as much an extent as possible")
>
> In general I support this idea. But for the current process of building those
> images is based on a contract our company have with Microsoft. This would
> violated the DMUP that clearly says: "Don't use Debian Facilities for private
> financial gain or for commercial purposes, including consultancy or any other
> work outside the scope of official duties or functions for the time being,
> without specific authorization to do so." The process of modifing the DMUP
> should be discussed elsewhere. The publishing of images requires login
> credentials to the vendors publishing API. In most cases those credentials are
> in some way linked to credit card data.... Do I really need to say more?
> Currently building images for whatever vendor requires root permissions on
> debian.org boxes. While I have them, using them would be an abuse of my DSA
> position. Also we eat our own dogfood and use Azure images to build Azure
> images.
It's my opinion that if the relevant teams believe that these tasks
should be run on Debian hardware, then we will support that decision.
Many people are paid to "work on Debian" so I'm not sure I understand
the issue? It's also likely that the work could fall under "official duties"
if they were blessed by the appropriate team(s), which is a likely outcome
of this discusison.
>> There are other considerations as well that I'm not sure if we've addressed
>> before.
>>
>> 1) Should we require that the images only point to Debian repos, and/or
>> official
>> mirrors? If not, what are the requirements here?
>
> That idea is complete nonsense. a) We have several layers of checksums and
> cryptographical signatures on the Debian archive and apt requiring the correct
> archive signing keys, so apt would start to complain immediately. What we could
> do as requirement is that every vendor needs to list all imported keys from
> "apt-key list" in the published build log of the image. b) In most cases
> vendors offering Debian run mirrors internally, which are available with much
> better connection than our official ones. Those can be verified by apt (see
> (a)). Cloud vendors usually bill for external traffic, sometimes only one
> direction, sometimes both. So your idea would result in our users needing to
> pay even more money to the cloud vendors. While the cloud vendors might support
> your idea, I personaly (without any hat on) think it is a very bad idea.
To be clear I am NOT proposing that a Debian mirror can't exist within
a public cloud.
Perhaps I have a misunderstanding of how official mirrors work, (Or even may be
mistaken in my believe that there are such things as "official mirrors"?)
My view would be that images should point to "official repos/mirrors",
and if an image building team wanted to point to different repos, they
would get the blessing of the team responsible for overseeing our mirror
network. (DSA?).
IE: Why can't these mirrors become "official mirrors", for use with a
specific public cloud, if they follow Debian's rules, and don't have
random arbitrary packages in them?
>> 2) Require public review of images/plans (where? I think debian-cloud
>> and debian-cd are the right places, but there may be others)
>
> I like the idea in general. Will we be able to support the review process for
> all different vendors? Will we be able to verify images / review images for
> cloud systems that are not that widely used as Azure, AWS, GCE or Openstack?
I don't think there is a formal process, but there has been countless
discussions shaping decisions made for AWS and GCE on the debian-cloud
mailing lists. I know the AWS images are announced to the list, and
peer reviewed.
I can also say with certainty that both AWS and GCE went through an
initial public vetting on list. As for vetting images for unpopular systems,
I don't know the answer, but I think we can cross that bridge when we
come to it.
Cheers,
Brian
>> 4) Documentation? Is it enough to just put it in wiki.d.o, in the cloud
>> section?
> started on https://wiki.debian.org/MicrosoftAzure.
>
>> Other questions:
>>
>> 1) bootstrap-vz is used to build the AWS and GCE images. bootstrap-vz has
>> also had support for Azure for at least two years. Is there a reason the
>> same tool wasn't used?
>
> The answer to this is quite simple: At the time we started to create images for
> Azure, bootstrap-vz was not in shape for generating Azure images that worked.
> For the demonstration purpose during DebConf15 we needed an image and Thomas
> openstack-debian-images script generated an image that was more or less out of
> the box usable for Azure. So we continued to use that script. Long term we plan
> to support both scripts.
>
> Best regards,
>
> Martin
> --
> Martin Zobel-Helas
> Technischer Leiter Betrieb
> Tel.: +49 (2161) 4643-0
> Fax: +49 (2161) 4643-100
> E-Mail: martin.zobel-helas@credativ.de
> pgp fingerprint: 6B18 5642 8E41 EC89 3D5D BDBB 53B1 AC6D B11B 627B
> http://www.credativ.de
>
> credativ GmbH, HRB Mönchengladbach 12080
> USt-ID-Nummer: DE204566209
> Hohenzollernstr. 133, 41061 Mönchengladbach
> Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
Reply to: