Re: Question to all candidates: how is Debian doing?
Hi Lucas
On 2022/03/17 17:54, Lucas Nussbaum wrote:
As someone who used to care a lot about Debian, but who hasn't been able
to pay much attention to the project lately, I was wondering:
I don't think anyone would hold it against you that you've got busy with
other stuff, having a life outside Debian is also considered very
healthy these days.
How is Debian doing currently?
Excellent question! A few weeks ago I saw a headline "Is Mozilla ok?",
and while I've thought about it on different levels for a while, it was
the first time that I thought in the exact words "Is Debian OK!?" and
mean to write something about it (possibly in a blog post, possibly in a
"bits from the DPL" mail), but as with this mail, it ended up in various
forms of drafts and I never made it half way with it, at least not yet.
So starting with a tl;dr, I think Debian is doing ok. It's not doing
great, but it is ok.
When we ask how Debian is doing, it's also useful to qualify what we're
asking. Is the Debian project (our structure, project members, larger
community...) ok? Is the Debian distribution (what features our users
need, severity of bugs, are we living up to our promises, etc...) ok?
On the positive side, we are chugging along quite well. Packages (and
lots of new packages) get uploaded, old crud eventually gets deleted
(last release was pretty good in this front), bugs get fixed, since 2005
the project has managed to release every 2 years, the website team has
great plans to make the website more friendly (*poke to www team to make
some public update please*), we finally have a functioning community
team (after some iterations and speedbumps), we now have the fasttrack
project (although still quite young) to deal with things that move to
fast for our usual archives, we're slowly but surely improving community
processes that people have complained about for a while (like our
current and previous GR to improve voting).
Our finances are also really good, our donors show lots of confidence in
us. Our corporate sponsors are already great, but I'm constantly amazed
by the generosity of our individual donors! There are people who donate
a $1000 at a time, some a few $100 every month, and sometimes even a
sporadic $4 donation from the same person. It's all very valuable and
appreciated! One person even donated $100,000 worth of shares to Debian
(was worth $140,000 when I checked last week) which was extremely
generous. Even though we've been spending a lot, our available funds are
also the highest they've ever been, last year we surpassed the $1m mark
in available funds for the first time.
That's great. As DPL, that allowed me to feel comfortable saying yes to
every single request every DD has made (which I did, and even some none-DDs.
I'll focus on the challenging aspects further down since that is a
seperate question.
What are the recent successes I might have missed?
I'll list just a few things since you got busy, there's probably a lot more.
We're getting a bit better at working with industry. We have a person
from Lenovo helping out with hardware support on their latest hardware,
we just today had a DD join from Microsoft, and Microsoft also covered
our LWN subscriptions for the last year (thanks!). There's lots of ways
big Linux users out there are helping us out, Hetzner gave us a huge
discount on our backup server hosting, Lenovo gave us a significant
discount on some servers we bought for DSA hosted stuff, and the list
goes on.
Our local groups initiative is also taking form again. I can't wait to
see more from this, covid put a real dampener on this, but between the
Debian reunion even in Germany and DebConf22, I hope there will be some
great local team packs made that can be sent around the world well
before the end of the year.
We've moved from Alioth to Salsa (GitLab instance) in 2017. This created
a big leap forward in how easy it is to make contributions to Debian.
Since then, Gnome, KDE, and many other free software projects have also
implemented a GitLab instance, it's now a very familiar and common way
to do things in the free software world, and I think this was a
significant and important change for us, even though it came with its
own set of speedbumps and challenges too.
We've gained a riscv64 port. Along with the lowrisc project to make a
fully open source CPU, it opens up the possibility to have a truly and
fully free hardware and software stack using Debian. It seems like it
may still be some years before you could easily buy a
phone/e-reader/router/laptop/desktop/server/etc with a riscv64 cpu that
can run Debian, but the foundations are being laid, and I consider that
critically important. Hardware is increasingly being locked down, and we
don't know how long it will be before you have to contact your
manufacturer in order to get an unlock code in order to install an
alternative operating system on a typical laptop (as it is with many
phones right now). This is an area in which I hope we'll grow in more
and can really shine in the future.
There's a lot happening in the machine learning world too, too much to
mention here. But I'm glad that some DDs have already taken the time to
think about how this affects Debian, and there's an early draft that
exists of a Debian Machine Learning Policy, which can be read at
https://salsa.debian.org/deeplearning-team/ml-policy/-/blob/master/ML-Policy.rst
Debian gained secureboot support, this took a bunch of big pushes but
besides the benefits of secureboot itself, it makes dual-booting or new
installations a lot easier for non-expert users, who previously we had
to explain how to get into their firmware setup to disable it (lots of
varied systems out there made this difficult in some times since many
novices struggled even getting into their firmware), so
for multiple reasons, this was also an important milestone.
There's reproducible builds, an effort to ensure that a binary built
from source is 100% reproducible, which means that builds can be
verified and trusted not to have been poisoned at some point during
compilation. The core members of reproducible builds are all Debian
Developers, but the project now extends across many Linux distributions
and other software projects. It's a huge Debian success story, even
though we're not 100% reproducible ourselves yet. The release team now
also require binary packages in stable releases to be built on Debian
infrastructure from source, so no more binaries in stable releases that
were built on people's laptops (or in some weird cases, even built in
Ubuntu).
There's https://fabre.debian.net/ - an initiative to make a friendly
interface for browsing the BTS.
We now host a debuginfod service, which allows you to debug software
without having to download their debuginfo packages by retreiving it
online (more info on https://wiki.debian.org/Debuginfod). Our instance
is one of the largest debuginfod services out there.
The above two services are two of a whole bunch of services ran by
project volunteers. During my first term of being DPL, I received lots
of requests for the project to pay for services that DDs host at various
providers that run under the debian.net domain. Some of these really
expensive, so I worked with debian.ch to get us some accounts at
providers so that we can create instances for our DDs and host and pay
for them ourselves, streamlining a lot of admin and at least if a DD
dissapears for a while and we need to make some serious security fix, we
can also gain access to the VM. Not very original, but we formed a team
called the debian.net team to assist with services run on debian.net
(some details: https://wiki.debian.org/Teams/DebianNet). As an aside,
looking at that page I was just reminded that rsync.net provides 500GB
of backup space for every DD, which is plenty of space to backup typical
things hosted under debian.net domains.
Before that, I was also struggling to figure out how and where to host a
PeerTube instance. PeerTube is a federated video sharing platform that
uses webtorrent to scale out so that many users can watch a video at the
same time without needing a lot of server infrastructure. I wanted to
install this so that DebConf videos are more discoverable and so that
local teams can easier share locally produced content without having to
upload it to YouTube. PeerTube fedirates on a network called "the
fediverse", and it turns out there was a bunch of other federated
services that debianites also wanted to implement. So we founded the
debian.social project (https://wiki.debian.org/Teams/DebianSocial) that
hosts services like our PeerTube instance
(https://peertube.debian.social/) and a few others that are too much
detail for this email at this point. It's also the project under which
we installed our Jitsi server (https://jitsi.debian.social/), jitsi is a
free software platform for making group video calls, it's been used
quite widely in the project since the start of the pandemic.
Covid came with a whole slew of challenges for the project, not being
able to meet in person has been really tough. 2020 was set to be our
biggest year in terms of MiniConfs. But we don't give up easily, and
gave a shot at our first ever mini DebConf that was entirely online.
Besides a few hickups, it was a big success, and we learned a lot to
make future online events a lot better. DebConf20 then ended up being
our first ever completely online DebConf. We also ended up donating all
the proceeds from DC20 for a PeerTube streaming feature, that will make
it easier for future Debian (and others) to stream small events in the
future (https://bits.debian.org/2020/10/debian-donation-peertube.html).
Maybe a bit subjective, but I think our look and feel has improved quite
a bit over the last few releases. Debian just "feels" a lot more
polished. We have a lot less papercuts on our desktops on the live
media, our default artwork has been pretty good for a few releases now.
Our live media has also gained the Calamares installer. While I don't
consider this a big piece of progress, it at least makes our
installation media a lot more useful until we as a project have a better
long-term strategy for our installer.
There are also entire teams full of achievements that I didn't get to
here (Debian Med team has been great and very relevant during covid!).
There's also so many smaller things that happened that I can't get into,
for example, APT finally hit v1.0, you can now setup dkim for your
@debian.org email address, we now have a much more loopy sponsors loop
for DebConf (https://peertube.debian.social/w/aEjdorA9M71tvm558YxyAP), etc.
For people who are very busy, I'd also suggest subscribing to
https://bits.debian.org - this is where our publicity team posts as
often as they can. But they also don't get to everything, if someone
reading this has something that you think the project (and the world)
should know about, get in touch with them on #debian-publicity, or even
better yet, write something for them (anyone who has access to salsa can
help write a story for bit.debian.org).
Overall, Debian has been very busy the last 5 years, and there's been
many changes, which always surprise me when there are the few people who
claims that nothing ever happens in Debian.
Where did we fail or under-perform?
(I'm going to try to cover these at the same time because there might be
some large overlap, and also in the interest of time I spend on this
mail :) )
In a previous DPL talk from me, I explained that Debian is a bottomless
pit of problems. This might sound harsh, or mean, but if you look at our
scope of work, we're affected by just about every problem that exists in
computer science and the general computing world. I suppose at least
we're not too concerned about quantum computing problems... yet.
Besides our countless technical challenges, we're also affected by many
social problems in the world. The less privileged someone is, the more
likely it is that they are earning less for doing the same kind of work.
It's difficult to convince someone to work for free on challenging
technical work when they are also struggling to pay their own bills.
There is some positive edge to this though, there are also many people
who have been able to make a career they wouldn't have been able to
otherwise, because of free software (I count myself into this category).
While I think we under-perform in the area of diversity, I do think we
can (and will) improve. I think that putting more effort into local
teams will help a lot. Helping more people learn about Debian, how to
use Debian and all the wonderful things you can do with it will spark
more and more interest, and as people in different areas become more
successful in their careers using Debian, it will inspire more people
from their local area to join in. On that note, it would be great if we
could also help people more on their Debian careers somehow.
Taking an educated guess before, I've estimated that we need 2-3 times
the volunteers we have now to pull off the goals of the Debian project
on the level that we want to. As someone pointed out to me recently,
this isn't unique to Debian or free software, this is often the case in
commercial software too. I was glad that some of my instincts were also
mirrored in a more scientific study of Debian, Kaylea Champion presented
some very interesting data at DebConf21 in her talk "Detecting At-Risk
Projects in Debian"
(https://peertube.debian.social/w/49JyBRR33c4d4oS1SvzK2U). While I would
take some conclusions with a grain of salt, it certainly provides some
food for thought in terms of matching up where we spend our energy the
most optimally.
A lot of our processes fall short. And I'm tempted to write out a long
list of examples of that, but again in the interest of getting this mail
sent out at a reasonable time, I'll do that some other time. A few
recent events specifically bother me. The usrmerge situation is very
unfortunate, it doesn't seem like there's a clear right way out of it
yet (I admit I'm about 20 bugmails behind on that right now, so
hopefully I'm wrong and something has changed). Our on boarding
processes are difficult to navigate, I'd love to help on that at some
point, but I know that wouldn't be possible for a DPL during a DPL term,
there's just too many little things to take care of, I hope to spend
some time on that after I'm DPL. Exiting has gotten a little better, if
someone wants to retire from Debian it's now just a few clicks to enter
emeritus status. The processes for firing someone from the project are a
lot more problematic. There's some barely started conversations on this
recently when it comes to DAM and CT reform, hopefully after our current
vote, we'll have some more bandwidth to take it on.
I very, very much enjoy using the software that we're upstream for
(dpkg, apt, d-i, dh, etc), but I feel we're not doing enough to support
these. I hope that when the world situations improve that we can have
more sprints for these, encourage developers for these to speak more at
events and ensure that each bit of upstream software we're responsible
for has a team behind it and not a single person carrying most of the load.
When it comes to money, I think we should really consider a kind of
grants system, where we collectively decide on a piece of work that
someone can do in exchange for a set amount of money. This could help us
solve some more long-standing issues that we don't get time for, and
help someone out. At the same time, I don't think that would compromise
us as a volunteer project, the project direction would still be
determined by the collective of volunteers (as apposed to some external
organization or entity).
> Which big challenges do you see ahead of us?
There's just so much change, and I don't think we can even anticipate
all the changes that are going to come. Having the right pieces in place
to deal with change is going to become more and more important.
A small part of me is also concerned that consumer computing products
are going to continue being more locked down (hopefully future open
hardware can help counter that, and I think we should be a part of that).
One part that has changed significantly over the last more than a decade
is firmware. It used to be something that was shipped with your hardware
that you could update in many cases if it fixed a bug. Now, it is
something that's increasingly loaded using software from disk, this
creates some significant problems for us. For example, on our default
live media many wireless network cards doesn't work. This /used/ to be
much less of a problem when we could tell people "Oh just install and
then install the iwlwifi from non-free afterwards", but more and more
consumer hardware doesn't have a wired ethernet port anymore. In the
past, if we didn't have the right display driver, we could launch
graphics in a degraded performance mode (like a vesa driver). On many
chips this isn't even possible anymore. So where we could do an install
first and then install just a non-free piece of firmware for graphics
afterwards, live media would now just give a black screen for those
cards. The ac97 audio architecture that's been in use for a long time
seems to be making way for the new intel audio, which also need non-free
firmware to be loaded in order to work. This has just been getting
increasingly worse, and not at all better. Ideally, I would have really
much appreciated if the FSF and OSI could lobby hardware manufacturers
to change this. Some people think that such an excercise would be
futile, but at least it would be doing /something/ in the positive
direction, and I'm fairly positive that some companies could be
convinced to be more free-software-friendly, sometimes even just moving
that dial can be beneficial long-term. Until we find actual solutions,
we might also have to consider making our images with non-free firmware
on easier to find for our users, along with very clear information that
media containing those files do not conform to our usual promises like
our social contract.
So in a nutshell, I think being able to install on physical hardware is
going to remain being an important challenge, and we should co-ordinate
and work on it from various angles.
Are there opportunities that we could leverage?
I think so, every problem also brings opportunity. In the case of
firmware above, perhaps it would be useful for Debian to fund reverse
engineering of firmware where it seems plausible. Perhaps that should be
done under a consortium for that goal that could get some specific
sponsorship from all the companies who would like to see that goal
materialized.
I could list a bunch more, but it's 23:38 here right now and I've spend
quite a lot of time on this mail already so I'll mostly leave it at that.
When it comes to opportunities, I think most long-time DDs have some
good ideas on how to leverage them, but we all get busy and bogged down
with our own areas of interest and problems. It's why in-person meetings
are also so crucial for us as a project, because it's often where people
get exposed to both problems and ideas outside of their personal Debian
bubble.
I wish I could chat some more about the topics you've asked about, but
time for bed here, thanks for your questions!
-Jonathan
Reply to: