Re: Debian images on Microsoft Azure cloud
On Fri, Nov 20, 2015 at 7:04 PM, Anders Ingemann <anders@ingemann.de> wrote:
> On Fri, Nov 20, 2015 at 11:57 AM Narcis Garcia <debianlists@actiu.net>
> wrote:
>>
>> To cross that bridge is good, being aware all the time about what's
>> exactly the thing.
>> In the meanwhile, a "derivative" image can be named as "Debian based".
>>
>> "Official mirrors" shouldn't contain differences to debian.org
>> repositories. Otherwise they should be named "Debian based" too.
>>
>>
>>
>> __________
>> I'm using this express-made address because personal addresses aren't
>> masked enough at lists.debian.org archives.
>>
>> El 20/11/15 a les 04:42, Brian Gupta ha escrit:
>> > On Wed, Nov 18, 2015 at 8:12 AM, Martin Zobel-Helas
>> > <martin.zobel-helas@credativ.de> wrote:
>> >> Hi,
>> >>
>> >> On Wed Nov 11, 2015 at 15:18:51 -0500, Brian Gupta wrote:
>> >>> (Note: Although some of you may know that I am a member of the Debian
>> >>> TM
>> >>> team, I am raising the following issues as a long-time participant in
>> >>> the
>> >>> debian-cloud group/mailing-list. I also apologize upfront for the
>> >>> length of
>> >>> this email, and for any inevitable omissions.)
>> >>
>> >> I see some conflict of interest here, but i will answer the technical
>> >> questions.
>> >
>> > Responding here as an individual member of the TM team. (Not on behalf
>> > of the
>> > TM team, as I've been swamped and we haven't had a chance to discuss).
>> >
>> > While I'm not sure if I have a "conflict" of interest, I will admit that
>> > I am
>> > hoping to kill more than one bird with a single stone. First, I'd like
>> > to
>> > help you resolve your TM request, in a way that is agreeable to the
>> > Debian
>> > Project and your team.
>> >
>> > Second, as there is a growing consensus developing for Debian to be
>> > publishing
>> > official cloud images, and at least one of the teams involved in
>> > publishing
>> > Debian images has expressed an interest in building/blessing cloud
>> > images,
>> > I'm hoping that this discussion promotes the development of a more
>> > fleshed out
>> > set of standards and policies related to Debian cloud images. This will
>> > hopefully make everyone's lives easier in the future.
>> >
>> > I'll also add that it's not the TM team's role to privately set
>> > technical
>> > policy, which is why we advocated to have this discussion in public
>> > with the appropriate technical teams.
>> >
>> >> First of all, i want to stress out, that i didn't request the trademark
>> >> for the
>> >> name "Official Debian images on Microsoft Azure cloud". I am happy to
>> >> help here
>> >> that we, at some future point, might reach that status, but as per
>> >> discussion
>> >> (where never a final decission was made!) during DebConf15 we mostly
>> >> agreed
>> >> that we should be careful what we call "Official Debian". Therefor we
>> >> would
>> >> like to use "Debian Jessie/Debian Wheezy build for Microsoft Azure".
>> >
>> > So my position here, which is being shaped by ongoing discussions, is
>> > that you
>> > are asking the Debian TM team for blessing or sanctioning to call a
>> > product you
>> > built, "Debian". That's about as "legally official" a blessing as one
>> > can get.
>> >
>> > Going forward, if one of our teams (say debian-cd) decided they wanted
>> > to also
>> > build Azure images, do you feel that there should be two separate sets
>> > of
>> > images, built by two separate sets of DDs? Which is official, if they
>> > both
>> > received the blessing of the project? From a user's point of view this
>> > could
>> > lead to great confusion, and would make the job of the TM team quite a
>> > bit
>> > harder, especially if there were disagreements about how the images are
>> > built.
>> >
>> > At this point in time, the only "non-official" blessing I would prefer
>> > the TM
>> > team be involved in would be perhaps for temporary transitional images
>> > that may not quite be ready for official status, but the DDs are
>> > committed
>> > to resolving any issues, the issues aren't considered showstoppers, and
>> > the
>> > technical teams involved support that plan. (Definitely open to feedback
>> > here,
>> > and this may change over time as we evolve our policies.)
>> >
>> >>> 1) the image includes only software available in Debian [2]
>> >>
>> >> Check. Our image only includes software available in Debian, except
>> >> waagent.
>> >> waagent is available in Debian itself, but not the version we currently
>> >> need
>> >> for the image, see my initial mail for more information.
>> >>
>> >>> 2) the image generation process is controlled solely by Debian [2]
>> >>
>> >> Check. Only DD have write access to the Jenkins instance used to
>> >> generate
>> >> images and control the scripts used by the process. Apart from the
>> >> usual vendor
>> >> operation staff, of cause.
>> >>
>> >>> 3) the image is generated using tools available in Debian, or
>> >>> maintained [2]
>> >>> by Debian
>> >>
>> >> Check. The tools are maintained by DD.
>> >>
>> >>> 4) Only DFSG-compliant Software in the image. Only main enabled, with
>> >>> perhaps a temporary exception for backports [3], for specific
>> >>> enablement
>> >>> software
>> >>
>> >> Check.
>> >>
>> >>> 6) the images most provide a user experience (in terms of default
>> >>> choice of packages, or of default configuration) identical to
>> >>> other
>> >>> means of installing Debian. Differences must be documented and
>> >>> justified.
>> >>> [4]
>> >>
>> >> Check.
>> >>
>> >>> 7) Debian kernel [5]
>> >>
>> >> Check. For Wheezy we need to use the kernel from backports.
>> >>
>> >>> 8) Built using Debian infrastructure [6] (I think this should be
>> >>> modified to
>> >>> have a caveat, "to as much an extent as possible")
>> >>
>> >> In general I support this idea. But for the current process of building
>> >> those
>> >> images is based on a contract our company have with Microsoft. This
>> >> would
>> >> violated the DMUP that clearly says: "Don't use Debian Facilities for
>> >> private
>> >> financial gain or for commercial purposes, including consultancy or any
>> >> other
>> >> work outside the scope of official duties or functions for the time
>> >> being,
>> >> without specific authorization to do so." The process of modifing the
>> >> DMUP
>> >> should be discussed elsewhere. The publishing of images requires login
>> >> credentials to the vendors publishing API. In most cases those
>> >> credentials are
>> >> in some way linked to credit card data.... Do I really need to say
>> >> more?
>> >> Currently building images for whatever vendor requires root permissions
>> >> on
>> >> debian.org boxes. While I have them, using them would be an abuse of my
>> >> DSA
>> >> position. Also we eat our own dogfood and use Azure images to build
>> >> Azure
>> >> images.
>> >
>> > It's my opinion that if the relevant teams believe that these tasks
>> > should be run on Debian hardware, then we will support that decision.
>> > Many people are paid to "work on Debian" so I'm not sure I understand
>> > the issue? It's also likely that the work could fall under "official
>> > duties"
>> > if they were blessed by the appropriate team(s), which is a likely
>> > outcome
>> > of this discusison.
>> >
>> >>> There are other considerations as well that I'm not sure if we've
>> >>> addressed
>> >>> before.
>> >>>
>> >>> 1) Should we require that the images only point to Debian repos,
>> >>> and/or
>> >>> official
>> >>> mirrors? If not, what are the requirements here?
>> >>
>> >> That idea is complete nonsense. a) We have several layers of checksums
>> >> and
>> >> cryptographical signatures on the Debian archive and apt requiring the
>> >> correct
>> >> archive signing keys, so apt would start to complain immediately. What
>> >> we could
>> >> do as requirement is that every vendor needs to list all imported keys
>> >> from
>> >> "apt-key list" in the published build log of the image. b) In most
>> >> cases
>> >> vendors offering Debian run mirrors internally, which are available
>> >> with much
>> >> better connection than our official ones. Those can be verified by apt
>> >> (see
>> >> (a)). Cloud vendors usually bill for external traffic, sometimes only
>> >> one
>> >> direction, sometimes both. So your idea would result in our users
>> >> needing to
>> >> pay even more money to the cloud vendors. While the cloud vendors might
>> >> support
>> >> your idea, I personaly (without any hat on) think it is a very bad
>> >> idea.
>> >
>> > To be clear I am NOT proposing that a Debian mirror can't exist within
>> > a public cloud.
>> >
>> > Perhaps I have a misunderstanding of how official mirrors work, (Or even
>> > may be
>> > mistaken in my believe that there are such things as "official
>> > mirrors"?)
>> >
>> > My view would be that images should point to "official repos/mirrors",
>> > and if an image building team wanted to point to different repos, they
>> > would get the blessing of the team responsible for overseeing our mirror
>> > network. (DSA?).
>> >
>> > IE: Why can't these mirrors become "official mirrors", for use with a
>> > specific public cloud, if they follow Debian's rules, and don't have
>> > random arbitrary packages in them?
>> >
>> >>> 2) Require public review of images/plans (where? I think debian-cloud
>> >>> and debian-cd are the right places, but there may be others)
>> >>
>> >> I like the idea in general. Will we be able to support the review
>> >> process for
>> >> all different vendors? Will we be able to verify images / review images
>> >> for
>> >> cloud systems that are not that widely used as Azure, AWS, GCE or
>> >> Openstack?
>> >
>> > I don't think there is a formal process, but there has been countless
>> > discussions shaping decisions made for AWS and GCE on the debian-cloud
>> > mailing lists. I know the AWS images are announced to the list, and
>> > peer reviewed.
>> >
>> > I can also say with certainty that both AWS and GCE went through an
>> > initial public vetting on list. As for vetting images for unpopular
>> > systems,
>> > I don't know the answer, but I think we can cross that bridge when we
>> > come to it.
>> >
>> > Cheers,
>> > Brian
>> >
>> >>> 4) Documentation? Is it enough to just put it in wiki.d.o, in the
>> >>> cloud
>> >>> section?
>> >> started on https://wiki.debian.org/MicrosoftAzure.
>> >>
>> >>> Other questions:
>> >>>
>> >>> 1) bootstrap-vz is used to build the AWS and GCE images. bootstrap-vz
>> >>> has
>> >>> also had support for Azure for at least two years. Is there a reason
>> >>> the
>> >>> same tool wasn't used?
>> >>
>> >> The answer to this is quite simple: At the time we started to create
>> >> images for
>> >> Azure, bootstrap-vz was not in shape for generating Azure images that
>> >> worked.
>> >> For the demonstration purpose during DebConf15 we needed an image and
>> >> Thomas
>> >> openstack-debian-images script generated an image that was more or less
>> >> out of
>> >> the box usable for Azure. So we continued to use that script. Long term
>> >> we plan
>> >> to support both scripts.
>> >>
>> >> Best regards,
>> >>
>> >> Martin
>> >> --
>> >> Martin Zobel-Helas
>> >> Technischer Leiter Betrieb
>> >> Tel.: +49 (2161) 4643-0
>> >> Fax: +49 (2161) 4643-100
>> >> E-Mail: martin.zobel-helas@credativ.de
>> >> pgp fingerprint: 6B18 5642 8E41 EC89 3D5D BDBB 53B1 AC6D B11B 627B
>> >> http://www.credativ.de
>> >>
>> >> credativ GmbH, HRB Mönchengladbach 12080
>> >> USt-ID-Nummer: DE204566209
>> >> Hohenzollernstr. 133, 41061 Mönchengladbach
>> >> Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer
>> >
>>
>
> Just my 5 cents:
>> "Official mirrors" shouldn't contain differences to debian.org
>> repositories. Otherwise they should be named "Debian based" too.
>
> Isn't that what we have GPG package signatures for? In the end, the real
> showstopper would be the installation of public keys that are not controlled
> by Debian. As long as I know that the only keys software is verified with
> are official Debian ones I couldn't care less where I get my "data" from -
> or at least that's how I think it should be, I am not pretending to know the
> official stance on this.
Ok, I think I stand corrected. The requirement should then perhaps be, no
third party keys? (I guess I was mistaken when I thought there was a known
trust issue with third party repos, in that they could be configured to serve
out-of-date packages to leave open backdoors into running servers.)
Apologies,
Brian
> --
> Anders Ingemann
Reply to: