This one time, at band camp, Lucas Nussbaum said:
> Hi,
>
> As previously mentioned[1], I've been talking with DSA about the state
> of Debian services on Debian infrastructure. DSA asked me to move the
> discussion to a public list, which is what I'm doing here.
> What follows is my current state of mind on this question. I'm open to
> changing my mind based on the project's feedback.
>
> [1] https://lists.debian.org/debian-project/2013/12/msg00015.html
>
>
> The underlying questions that I'd like to answer are:
>
> Q1. How much should we push for Debian services (services useful to
> Debian) to be hosted on Debian infrastructure?
>
> Q2. Should we seek other hosting opportunities to ease Debian services
> development and hosting? Should I authorize the use of Debian money to
> fund infrastructure not managed by DSA, in the case of a useful service
> that DSA has been unable to accept in the infrastructure it manages?
>
> Q3. What should be our definition of "official services"?
>
>
> Background
> ==========
>
> First, it's important to state that DSA has been doing a fantastic job on
> maintaining our core infrastructure for the last few years. The level of
> quality and professionnalism of their work clearly surpasses what can be
> found in most organizations, volunteer or not.
Thank you.
I am speaking largely for myself, but I've discussed this mail with the
team. Where I use 'we', I speak for the team.
> However, there has been several episodes, involving the development of new
> services or their move to Debian infrastructure, that generated a lot of
> frustration and demotivation on both sides (DSA and service developers).
> As examples, one can think of codesearch, or the fedmsg GSOC project.
> There are also services that have been developed outside of Debian's
> infrastructure, with AFAIK no plans to move them to Debian infrastructure.
> The recurring pattern seems to be that, when DSA is approached to move the
> service to Debian infrastructure, their evaluation of the service's design
> results in changes requests or constraints that the service maintainers
> consider too hard to satisfy.
We think this might come from a fundamental mismatch between what DSA
are willing to host and what some people would like to design. We
don't think that that's a bad thing: we don't have to host everything
that relates to Debian, any more than the glibc maintainers have to look
after everything written in C.
We are, of course, happy to look after things once they get a little
further along and look like they'll prove useful to the project.
> I'm worried that this situation is harmful for the project. Similarly to
> how we increased the amount of collaborative packages maintainance over
> the years, we should improve on collaborative service maintainance, and we
> should move away from the myriad of unofficial services maintained by a
> sole DD, and hosted on this DD's personal machine.
Absolutely agreed.
> The obvious way to do that would be to facilitate the hosting of emerging
> services on Debian infrastructure, and I think that we should all work
> together to identify what could be done towards that.
We don't agree here. There are many ways to meet the goal of service
ownership being collaborative, only one of which is to have everything
hosted by DSA. Another might be that service owners look after their
own hosting and system administration duties themselves, in a team,
and have nothing to do with DSA. Another might be that DSA act in
an advisory capacity, and assist new services in starting up, and then
recuse ourselves and allow the service owners to look after their service
themselves. Another might be that there is a mixed mode of support.
I don't think jumping straight to a solution that puts all of the
responsibility for every idea for a service in Debian on DSA shoulders is
either the only way to go or even a good way to go. There are plenty
of bad ideas that should be allowed to wither on the vine, and there
are always going to be services that have been designed in such a way as
to be difficult to integrate into DSA-managed infrastructure. We are,
after all, a reasonably small team of volunteers. Pretending that we
can support an infinite number of services or an infinite variety of
designs is just going to end in disappointment for someone.
> I've given some thought to this myself, and came up with the following
> ideas. Some of them are probably really bad ideas, but let's try to
> brainstorm a bit:
>
> 1. to improve communication between DSA and service maintainers
> + have an archived, public list where (prospective) service maintainers
> can engage with DSA about stuff they are planning/thinking about.
> (Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?)
> + have DSA provide collective answers more often, rather than a set of
> individual answers. It's often not clear if something is a final
> decision, or the opinion of a DSA member.
> + redirect some discussions from IRC to a mailing list, esp. when
> things look like policy decisions (on service design, etc)
>
> 2. to improve the tracking of services
> + request wiki pages from maintainers that detail who are the contact
> points, what the service does, what are its requirements. State a
> "service level agreement" (informative of expectations, not punitive,
> of course)
> + have service maintainers create similar pages for services in
> development, to provide some kind of "incubation process" during which
> DSA can provide feedback.
>
> 3. to provide a place to experiment with new services
> + create a "Debian cloud" with virtual machines to develop new services
> (maybe providing manually-created VMs would be enough -- I'm not sure we
> need a complex infra such as OpenStack).
>
> Additionally, one suggestion from DSA is to involve them as earlier as
> possible in the design process.
Most of this sounds pretty good. I think you might be expecting an
unreasonable level of service-desk like automatism in some cases ("don't
provide individual opinions"), but meh, these things mostly come out in
the wash. I'll probably continue providing an individual opinion, and
I'll continue going along with the team when I have a minority opinion.
Providing a list of expectations for new services is on our TODO list,
and is being actively talked about.
> I'm now trying to answer my own questions:
>
> > Q1. How much should we push for Debian services (services useful to
> > Debian) to be hosted on Debian infrastructure?
>
> Fragmentation in terms of hosting is harmful, and we should all try to
> get our services hosted on Debian infrastructure, because that's the
> easiest way to ensure that the maintainance of such services can be shared
> or transferred between developers.
>
> Now, for some services, it seems that doing this has become so hard that
> it harmed the service itself, and badly demotivated the service
> maintainers. I don't think that we should allow that to happen. We should
> stay open to other hosting solutions, when this is necessary.
Sure. Services designed to be run solely by the service owner should
probably be run by the service owner. That's not necessarily a bad
thing. If I go write something in brainfuck, I should expect that I'd
get a vanishingly small team to comaintain it with me. Life is full of
opportunities to learn the consequences of design decisions. Whether
or not Debian finances have to pay for strange design decisions is up
to you, of course.
Again, I think it's useful to keep in mind that we are actually talking
about two things here - whether DSA should host all services that are
'useful to the project' (wherever you draw that line), and whether
DSA should host what might be called 'speculative services'. You use
fedmsg as an example of where DSA did badly (and I agree, communication
was poor), but I'd put fedmsg squarely in the 'speculative' side of
things. So this makes me think that you believe it to be DSA's job to
host and manage services in incubation, and that I disagree with.
For 'important' services, I think we all agree that it would be good
if DSA managed them, for all the usual reasons that we've all said.
Where we draw the line for what is 'important' and who gets to decide
what is 'important' can be another thread for another day. That said,
with responsibility comes authority and, as mentioned, we don't believe
we are required to accept every service that's developed.
> > Q2. Should we seek other hosting opportunities to ease Debian services
> > development and hosting? Should I authorize the use of Debian money to
> > fund infrastructure not managed by DSA, in the case of a useful service
> > that DSA has been unable to accept in the infrastructure it manages?
>
> For services development (something for which we don't have a good
> answer yet), I think that we should explore getting specific sponsored
> hosting. As you know, Amazon provides Debian with free credits on the
> AWS Cloud. Those credits are normally used for archive rebuilds, but
> have already been used by a GSOC project (PTS rewrite), and more
> recently, to start work on an archive-wide autopkgtest setup. A few more
> virtual machines could be accomodated, and I will investigate if we
> could extend that even more. Also, codesearch.d.n is hosted on
> Rackspace's Cloud[1]. I will also follow up on that.
> [1] http://people.debian.org/~stapelberg//2013/12/25/codesearch-rackspace.html
>
> Question to DSA: would you be willing to manage the allocation of
> virtual machines on such cloud infrastructures? That would ease the
> monitoring of which services are being developed, and help with getting
> involved early in the design process.
I, speaking solely for myself, would not. As we've seen from the
recent openssl issue, untrusted hypervisors are untrustworthy. I know,
shocking, right? See http://www.openssl.org/news/secadv_hack.txt for
reference.
In the stages we're talking about (early development of a service),
people can probably live with that level of insecurity, and can probably
also manage day to day sysadmin issues on their own. I think everyone
in the team would be happy to help people with specific "how is apache
configured" issues, of course. Having responsibility for those services,
and doing things like account management are much less interesting.
That said, we are very interested in finding sponsors similar to
bytemark who are prepared to provide hardware (servers and storage)
for expanding our ganeti clusters. We're happy to provide VMs to
people if/when they ask for them, subject to the need and purpose (ie,
people should get VMs from a provider for their own non-Debian projects,
people should provide their own resources for package maintenance, etc.).
> > Q3. What should be our definition of "official services"?
>
> [ The debian.org DNS domain name is often associated with "official"
> services, whereas the debian.net DNS domain name is often associated
> with "unofficial" services. Besides the d.o/d.n distinction, I'm not
> aware of another definition of "official services" in Debian ]
>
> Even if this is highly preferable, I don't think that official services
> (.d.o services) should necessarily be running on Debian hardware managed
> by DSA, provided that:
> - the service is clearly useful and used
> - the service has a sustainable maintainance model (active team +
> instructions on how to contribute, run a local copy, etc. + DFSG-free)
> - the service's design does not raise security or scalability concerns
I think you attach excessive importance to the TLD of a service. I
doubt that most of our users would notice if the service was in the
debian.xxx domain, to be honest, but the difference between .org and
.net is certainly something I doubt casual people to be even vaguely
interested in.
We prefer that services not supported by DSA be accommodated in debian.net
so as to empower the service owners to have full control of the DNS
entry for <service>.debian.net. Conversely, we prefer that service in
debian.org be supported by DSA. We wish to avoid confusion (and the
per-service wiki page can help).
Finally, please note that there are a number of debian.net services
hosted on Debian infrastructure.
Cheers,
--
-----------------------------------------------------------------
| ,''`. Stephen Gran |
| : :' : sgran@debian.org |
| `. `' Debian user, admin, and developer |
| `- http://www.debian.org |
-----------------------------------------------------------------
Attachment:
signature.asc
Description: Digital signature