Debian services and Debian infrastructure
Hi,
As previously mentioned[1], I've been talking with DSA about the state
of Debian services on Debian infrastructure. DSA asked me to move the
discussion to a public list, which is what I'm doing here.
What follows is my current state of mind on this question. I'm open to
changing my mind based on the project's feedback.
[1] https://lists.debian.org/debian-project/2013/12/msg00015.html
The underlying questions that I'd like to answer are:
Q1. How much should we push for Debian services (services useful to
Debian) to be hosted on Debian infrastructure?
Q2. Should we seek other hosting opportunities to ease Debian services
development and hosting? Should I authorize the use of Debian money to
fund infrastructure not managed by DSA, in the case of a useful service
that DSA has been unable to accept in the infrastructure it manages?
Q3. What should be our definition of "official services"?
Background
==========
First, it's important to state that DSA has been doing a fantastic job on
maintaining our core infrastructure for the last few years. The level of
quality and professionnalism of their work clearly surpasses what can be
found in most organizations, volunteer or not.
However, there has been several episodes, involving the development of new
services or their move to Debian infrastructure, that generated a lot of
frustration and demotivation on both sides (DSA and service developers).
As examples, one can think of codesearch, or the fedmsg GSOC project.
There are also services that have been developed outside of Debian's
infrastructure, with AFAIK no plans to move them to Debian infrastructure.
The recurring pattern seems to be that, when DSA is approached to move the
service to Debian infrastructure, their evaluation of the service's design
results in changes requests or constraints that the service maintainers
consider too hard to satisfy.
I'm worried that this situation is harmful for the project. Similarly to
how we increased the amount of collaborative packages maintainance over
the years, we should improve on collaborative service maintainance, and we
should move away from the myriad of unofficial services maintained by a
sole DD, and hosted on this DD's personal machine.
The obvious way to do that would be to facilitate the hosting of emerging
services on Debian infrastructure, and I think that we should all work
together to identify what could be done towards that.
I've given some thought to this myself, and came up with the following
ideas. Some of them are probably really bad ideas, but let's try to
brainstorm a bit:
1. to improve communication between DSA and service maintainers
+ have an archived, public list where (prospective) service maintainers
can engage with DSA about stuff they are planning/thinking about.
(Maybe recycle debian-admin@l.d.o, or use debian-services-admin@l.d.o?)
+ have DSA provide collective answers more often, rather than a set of
individual answers. It's often not clear if something is a final
decision, or the opinion of a DSA member.
+ redirect some discussions from IRC to a mailing list, esp. when
things look like policy decisions (on service design, etc)
2. to improve the tracking of services
+ request wiki pages from maintainers that detail who are the contact
points, what the service does, what are its requirements. State a
"service level agreement" (informative of expectations, not punitive,
of course)
+ have service maintainers create similar pages for services in
development, to provide some kind of "incubation process" during which
DSA can provide feedback.
3. to provide a place to experiment with new services
+ create a "Debian cloud" with virtual machines to develop new services
(maybe providing manually-created VMs would be enough -- I'm not sure we
need a complex infra such as OpenStack).
Additionally, one suggestion from DSA is to involve them as earlier as
possible in the design process.
I'm now trying to answer my own questions:
> Q1. How much should we push for Debian services (services useful to
> Debian) to be hosted on Debian infrastructure?
Fragmentation in terms of hosting is harmful, and we should all try to
get our services hosted on Debian infrastructure, because that's the
easiest way to ensure that the maintainance of such services can be shared
or transferred between developers.
Now, for some services, it seems that doing this has become so hard that
it harmed the service itself, and badly demotivated the service
maintainers. I don't think that we should allow that to happen. We should
stay open to other hosting solutions, when this is necessary.
> Q2. Should we seek other hosting opportunities to ease Debian services
> development and hosting? Should I authorize the use of Debian money to
> fund infrastructure not managed by DSA, in the case of a useful service
> that DSA has been unable to accept in the infrastructure it manages?
For services development (something for which we don't have a good
answer yet), I think that we should explore getting specific sponsored
hosting. As you know, Amazon provides Debian with free credits on the
AWS Cloud. Those credits are normally used for archive rebuilds, but
have already been used by a GSOC project (PTS rewrite), and more
recently, to start work on an archive-wide autopkgtest setup. A few more
virtual machines could be accomodated, and I will investigate if we
could extend that even more. Also, codesearch.d.n is hosted on
Rackspace's Cloud[1]. I will also follow up on that.
[1] http://people.debian.org/~stapelberg//2013/12/25/codesearch-rackspace.html
Question to DSA: would you be willing to manage the allocation of
virtual machines on such cloud infrastructures? That would ease the
monitoring of which services are being developed, and help with getting
involved early in the design process.
> Q3. What should be our definition of "official services"?
[ The debian.org DNS domain name is often associated with "official"
services, whereas the debian.net DNS domain name is often associated
with "unofficial" services. Besides the d.o/d.n distinction, I'm not
aware of another definition of "official services" in Debian ]
Even if this is highly preferable, I don't think that official services
(.d.o services) should necessarily be running on Debian hardware managed
by DSA, provided that:
- the service is clearly useful and used
- the service has a sustainable maintainance model (active team +
instructions on how to contribute, run a local copy, etc. + DFSG-free)
- the service's design does not raise security or scalability concerns
Feedback, comments, ideas?
Thanks,
- Lucas
Reply to: