[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Question to all candidates: how is Debian doing?

Hi Lucas

On 2022/03/17 17:54, Lucas Nussbaum wrote:
As someone who used to care a lot about Debian, but who hasn't been able
to pay much attention to the project lately, I was wondering:

I don't think anyone would hold it against you that you've got busy with other stuff, having a life outside Debian is also considered very healthy these days.

How is Debian doing currently?

Excellent question! A few weeks ago I saw a headline "Is Mozilla ok?", and while I've thought about it on different levels for a while, it was the first time that I thought in the exact words "Is Debian OK!?" and mean to write something about it (possibly in a blog post, possibly in a "bits from the DPL" mail), but as with this mail, it ended up in various forms of drafts and I never made it half way with it, at least not yet.

So starting with a tl;dr, I think Debian is doing ok. It's not doing great, but it is ok.

When we ask how Debian is doing, it's also useful to qualify what we're asking. Is the Debian project (our structure, project members, larger community...) ok? Is the Debian distribution (what features our users need, severity of bugs, are we living up to our promises, etc...) ok?

On the positive side, we are chugging along quite well. Packages (and lots of new packages) get uploaded, old crud eventually gets deleted (last release was pretty good in this front), bugs get fixed, since 2005 the project has managed to release every 2 years, the website team has great plans to make the website more friendly (*poke to www team to make some public update please*), we finally have a functioning community team (after some iterations and speedbumps), we now have the fasttrack project (although still quite young) to deal with things that move to fast for our usual archives, we're slowly but surely improving community processes that people have complained about for a while (like our current and previous GR to improve voting).

Our finances are also really good, our donors show lots of confidence in us. Our corporate sponsors are already great, but I'm constantly amazed by the generosity of our individual donors! There are people who donate a $1000 at a time, some a few $100 every month, and sometimes even a sporadic $4 donation from the same person. It's all very valuable and appreciated! One person even donated $100,000 worth of shares to Debian (was worth $140,000 when I checked last week) which was extremely generous. Even though we've been spending a lot, our available funds are also the highest they've ever been, last year we surpassed the $1m mark in available funds for the first time.

That's great. As DPL, that allowed me to feel comfortable saying yes to every single request every DD has made (which I did, and even some none-DDs.

I'll focus on the challenging aspects further down since that is a seperate question.

What are the recent successes I might have missed?

I'll list just a few things since you got busy, there's probably a lot more.

We're getting a bit better at working with industry. We have a person from Lenovo helping out with hardware support on their latest hardware, we just today had a DD join from Microsoft, and Microsoft also covered our LWN subscriptions for the last year (thanks!). There's lots of ways big Linux users out there are helping us out, Hetzner gave us a huge discount on our backup server hosting, Lenovo gave us a significant discount on some servers we bought for DSA hosted stuff, and the list goes on.

Our local groups initiative is also taking form again. I can't wait to see more from this, covid put a real dampener on this, but between the Debian reunion even in Germany and DebConf22, I hope there will be some great local team packs made that can be sent around the world well before the end of the year.

We've moved from Alioth to Salsa (GitLab instance) in 2017. This created a big leap forward in how easy it is to make contributions to Debian. Since then, Gnome, KDE, and many other free software projects have also implemented a GitLab instance, it's now a very familiar and common way to do things in the free software world, and I think this was a significant and important change for us, even though it came with its own set of speedbumps and challenges too.

We've gained a riscv64 port. Along with the lowrisc project to make a fully open source CPU, it opens up the possibility to have a truly and fully free hardware and software stack using Debian. It seems like it may still be some years before you could easily buy a phone/e-reader/router/laptop/desktop/server/etc with a riscv64 cpu that can run Debian, but the foundations are being laid, and I consider that critically important. Hardware is increasingly being locked down, and we don't know how long it will be before you have to contact your manufacturer in order to get an unlock code in order to install an alternative operating system on a typical laptop (as it is with many phones right now). This is an area in which I hope we'll grow in more and can really shine in the future.

There's a lot happening in the machine learning world too, too much to mention here. But I'm glad that some DDs have already taken the time to think about how this affects Debian, and there's an early draft that exists of a Debian Machine Learning Policy, which can be read at https://salsa.debian.org/deeplearning-team/ml-policy/-/blob/master/ML-Policy.rst

Debian gained secureboot support, this took a bunch of big pushes but besides the benefits of secureboot itself, it makes dual-booting or new installations a lot easier for non-expert users, who previously we had to explain how to get into their firmware setup to disable it (lots of varied systems out there made this difficult in some times since many novices struggled even getting into their firmware), so
for multiple reasons, this was also an important milestone.

There's reproducible builds, an effort to ensure that a binary built from source is 100% reproducible, which means that builds can be verified and trusted not to have been poisoned at some point during compilation. The core members of reproducible builds are all Debian Developers, but the project now extends across many Linux distributions and other software projects. It's a huge Debian success story, even though we're not 100% reproducible ourselves yet. The release team now also require binary packages in stable releases to be built on Debian infrastructure from source, so no more binaries in stable releases that were built on people's laptops (or in some weird cases, even built in Ubuntu).

There's https://fabre.debian.net/ - an initiative to make a friendly interface for browsing the BTS.

We now host a debuginfod service, which allows you to debug software without having to download their debuginfo packages by retreiving it online (more info on https://wiki.debian.org/Debuginfod). Our instance is one of the largest debuginfod services out there.

The above two services are two of a whole bunch of services ran by project volunteers. During my first term of being DPL, I received lots of requests for the project to pay for services that DDs host at various providers that run under the debian.net domain. Some of these really expensive, so I worked with debian.ch to get us some accounts at providers so that we can create instances for our DDs and host and pay for them ourselves, streamlining a lot of admin and at least if a DD dissapears for a while and we need to make some serious security fix, we can also gain access to the VM. Not very original, but we formed a team called the debian.net team to assist with services run on debian.net (some details: https://wiki.debian.org/Teams/DebianNet). As an aside, looking at that page I was just reminded that rsync.net provides 500GB of backup space for every DD, which is plenty of space to backup typical things hosted under debian.net domains.

Before that, I was also struggling to figure out how and where to host a PeerTube instance. PeerTube is a federated video sharing platform that uses webtorrent to scale out so that many users can watch a video at the same time without needing a lot of server infrastructure. I wanted to install this so that DebConf videos are more discoverable and so that local teams can easier share locally produced content without having to upload it to YouTube. PeerTube fedirates on a network called "the fediverse", and it turns out there was a bunch of other federated services that debianites also wanted to implement. So we founded the debian.social project (https://wiki.debian.org/Teams/DebianSocial) that hosts services like our PeerTube instance (https://peertube.debian.social/) and a few others that are too much detail for this email at this point. It's also the project under which we installed our Jitsi server (https://jitsi.debian.social/), jitsi is a free software platform for making group video calls, it's been used quite widely in the project since the start of the pandemic.

Covid came with a whole slew of challenges for the project, not being able to meet in person has been really tough. 2020 was set to be our biggest year in terms of MiniConfs. But we don't give up easily, and gave a shot at our first ever mini DebConf that was entirely online. Besides a few hickups, it was a big success, and we learned a lot to make future online events a lot better. DebConf20 then ended up being our first ever completely online DebConf. We also ended up donating all the proceeds from DC20 for a PeerTube streaming feature, that will make it easier for future Debian (and others) to stream small events in the future (https://bits.debian.org/2020/10/debian-donation-peertube.html).

Maybe a bit subjective, but I think our look and feel has improved quite a bit over the last few releases. Debian just "feels" a lot more polished. We have a lot less papercuts on our desktops on the live media, our default artwork has been pretty good for a few releases now. Our live media has also gained the Calamares installer. While I don't consider this a big piece of progress, it at least makes our installation media a lot more useful until we as a project have a better long-term strategy for our installer.

There are also entire teams full of achievements that I didn't get to here (Debian Med team has been great and very relevant during covid!).

There's also so many smaller things that happened that I can't get into, for example, APT finally hit v1.0, you can now setup dkim for your @debian.org email address, we now have a much more loopy sponsors loop for DebConf (https://peertube.debian.social/w/aEjdorA9M71tvm558YxyAP), etc.

For people who are very busy, I'd also suggest subscribing to https://bits.debian.org - this is where our publicity team posts as often as they can. But they also don't get to everything, if someone reading this has something that you think the project (and the world) should know about, get in touch with them on #debian-publicity, or even better yet, write something for them (anyone who has access to salsa can help write a story for bit.debian.org).

Overall, Debian has been very busy the last 5 years, and there's been many changes, which always surprise me when there are the few people who claims that nothing ever happens in Debian.

Where did we fail or under-perform?

(I'm going to try to cover these at the same time because there might be some large overlap, and also in the interest of time I spend on this mail :) )

In a previous DPL talk from me, I explained that Debian is a bottomless pit of problems. This might sound harsh, or mean, but if you look at our scope of work, we're affected by just about every problem that exists in computer science and the general computing world. I suppose at least we're not too concerned about quantum computing problems... yet.

Besides our countless technical challenges, we're also affected by many social problems in the world. The less privileged someone is, the more likely it is that they are earning less for doing the same kind of work. It's difficult to convince someone to work for free on challenging technical work when they are also struggling to pay their own bills. There is some positive edge to this though, there are also many people who have been able to make a career they wouldn't have been able to otherwise, because of free software (I count myself into this category).

While I think we under-perform in the area of diversity, I do think we can (and will) improve. I think that putting more effort into local teams will help a lot. Helping more people learn about Debian, how to use Debian and all the wonderful things you can do with it will spark more and more interest, and as people in different areas become more successful in their careers using Debian, it will inspire more people from their local area to join in. On that note, it would be great if we could also help people more on their Debian careers somehow.

Taking an educated guess before, I've estimated that we need 2-3 times the volunteers we have now to pull off the goals of the Debian project on the level that we want to. As someone pointed out to me recently, this isn't unique to Debian or free software, this is often the case in commercial software too. I was glad that some of my instincts were also mirrored in a more scientific study of Debian, Kaylea Champion presented some very interesting data at DebConf21 in her talk "Detecting At-Risk Projects in Debian" (https://peertube.debian.social/w/49JyBRR33c4d4oS1SvzK2U). While I would take some conclusions with a grain of salt, it certainly provides some food for thought in terms of matching up where we spend our energy the most optimally.

A lot of our processes fall short. And I'm tempted to write out a long list of examples of that, but again in the interest of getting this mail sent out at a reasonable time, I'll do that some other time. A few recent events specifically bother me. The usrmerge situation is very unfortunate, it doesn't seem like there's a clear right way out of it yet (I admit I'm about 20 bugmails behind on that right now, so hopefully I'm wrong and something has changed). Our on boarding processes are difficult to navigate, I'd love to help on that at some point, but I know that wouldn't be possible for a DPL during a DPL term, there's just too many little things to take care of, I hope to spend some time on that after I'm DPL. Exiting has gotten a little better, if someone wants to retire from Debian it's now just a few clicks to enter emeritus status. The processes for firing someone from the project are a lot more problematic. There's some barely started conversations on this recently when it comes to DAM and CT reform, hopefully after our current vote, we'll have some more bandwidth to take it on.

I very, very much enjoy using the software that we're upstream for (dpkg, apt, d-i, dh, etc), but I feel we're not doing enough to support these. I hope that when the world situations improve that we can have more sprints for these, encourage developers for these to speak more at events and ensure that each bit of upstream software we're responsible for has a team behind it and not a single person carrying most of the load.

When it comes to money, I think we should really consider a kind of grants system, where we collectively decide on a piece of work that someone can do in exchange for a set amount of money. This could help us solve some more long-standing issues that we don't get time for, and help someone out. At the same time, I don't think that would compromise us as a volunteer project, the project direction would still be determined by the collective of volunteers (as apposed to some external organization or entity).

> Which big challenges do you see ahead of us?

There's just so much change, and I don't think we can even anticipate all the changes that are going to come. Having the right pieces in place to deal with change is going to become more and more important.

A small part of me is also concerned that consumer computing products are going to continue being more locked down (hopefully future open hardware can help counter that, and I think we should be a part of that).

One part that has changed significantly over the last more than a decade is firmware. It used to be something that was shipped with your hardware that you could update in many cases if it fixed a bug. Now, it is something that's increasingly loaded using software from disk, this creates some significant problems for us. For example, on our default live media many wireless network cards doesn't work. This /used/ to be much less of a problem when we could tell people "Oh just install and then install the iwlwifi from non-free afterwards", but more and more consumer hardware doesn't have a wired ethernet port anymore. In the past, if we didn't have the right display driver, we could launch graphics in a degraded performance mode (like a vesa driver). On many chips this isn't even possible anymore. So where we could do an install first and then install just a non-free piece of firmware for graphics afterwards, live media would now just give a black screen for those cards. The ac97 audio architecture that's been in use for a long time seems to be making way for the new intel audio, which also need non-free firmware to be loaded in order to work. This has just been getting increasingly worse, and not at all better. Ideally, I would have really much appreciated if the FSF and OSI could lobby hardware manufacturers to change this. Some people think that such an excercise would be futile, but at least it would be doing /something/ in the positive direction, and I'm fairly positive that some companies could be convinced to be more free-software-friendly, sometimes even just moving that dial can be beneficial long-term. Until we find actual solutions, we might also have to consider making our images with non-free firmware on easier to find for our users, along with very clear information that media containing those files do not conform to our usual promises like our social contract.

So in a nutshell, I think being able to install on physical hardware is going to remain being an important challenge, and we should co-ordinate and work on it from various angles.

Are there opportunities that we could leverage?
I think so, every problem also brings opportunity. In the case of firmware above, perhaps it would be useful for Debian to fund reverse engineering of firmware where it seems plausible. Perhaps that should be done under a consortium for that goal that could get some specific sponsorship from all the companies who would like to see that goal materialized.

I could list a bunch more, but it's 23:38 here right now and I've spend quite a lot of time on this mail already so I'll mostly leave it at that.

When it comes to opportunities, I think most long-time DDs have some good ideas on how to leverage them, but we all get busy and bogged down with our own areas of interest and problems. It's why in-person meetings are also so crucial for us as a project, because it's often where people get exposed to both problems and ideas outside of their personal Debian bubble.

I wish I could chat some more about the topics you've asked about, but time for bed here, thanks for your questions!


Reply to: