[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: What will improve Debian most?

On Sat, Mar 28, 2009 at 11:59:43PM +0000, Steve McIntyre wrote:

] Campaigning period: Sunday, March 8th 00:00:00 UTC, 2009  
]                        - Saturday, March 28th 23:59:59 UTC, 2009

Hmmm... Cutting it fine...

> Depending on what you're measuring, we are still growing very quickly.
> The size of the archive and the number of packages are still going up
> quickly. 

Here's what that actually looks like:

    release  sources  packages  src growth  bin growth
    hamm     1115     1524
    slink    1580     2269      42%         49%
    potato   2647     3889      68%         71%
    woody    5218     8273      97%         113%
    sarge    8727     15195     67%         84%
    etch     10220    18051     17%         19%
    lenny    12123    22311     19%         24%

(limited to main, and packages is for i386)

I wouldn't say that's particularly quickly; but given the varying release
times, it's a bit hard to really tell. Correcting for that:

    release  date        days  s.p.d  p.p.d  sg.p.a  pg.p.a
    hamm     1998-07-24
    slink    1999-03-09  228   2.04   3.27   75%     89%
    potato   2000-08-14  524   2.04   3.09   43%     46%
    woody    2002-07-20  705   3.65   6.22   42%     48%
    sarge    2005-06-06  1052  3.34   6.58   20%     23%
    etch     2007-04-08  671   2.23   4.26   9%      10%
    lenny    2009-02-14  678   2.81   6.28   10%     12%

{s,p}.p.d = (net) new source/i386 packages per day.

{s,p}g.p.a = annualised growth in source/i386 packages; so eg the 42/49%
growth in slink gets annualised to 75/89% since it took under 8 months
to release slink. Formula is (100%+growth)^(365/days)-100%

Dropping from 75% to 40% to 20% to 10% in source package growth per year
is a bit sad -- but it's certainly interesting that the (net) number of
new source packages per day has stayed fairly constant.

Translating those percentages into a doubling rate (how many months to
double the number of source/binary packages) gives:

    release  dbl src  dbl pkg
    slink    15       13
    potato   23       22
    woody    24       21
    sarge    47       39
    etch     97       89
    lenny    90       73

Moore's law is about 18 months, which means we're not scaling at the same
rate technology is, and haven't been close since woody's release. There's
three ways of looking at that, some or all of which might be valid:

    * it's a good thing, extra packages are a cost when trying to work out
      what to install, so it's better to scale by combining functionality 
      into common packages

    * upstream isn't scaling with Moore's law, there's not that many
      new packages, because there just isn't that much new software

    * we haven't figured out how to offload enough of the scaling work onto
      technology, so we're not scaling with Moore's law

Personally, there seems to be an increasing number of things I'd like
to try that just aren't packaged for Debian, so I don't find the first
two answers alone entirely satisfying. Obviously YMMV.

> Sustaining exponential growth in terms of project size itself
> *is* difficult, however, and that's where we have started to tail off.

Well, the above growth doesn't look exponential either. But project size
is worse, because it's basically stagnant since 2002 (going by vote.d.o's
developer count at DPL election):

    2000     347
    2001       ?
    2002     939
    2003     831
    2004     911
    2005     965
    2006     972
    2007    1036
    2008    1075
    2009    1043 -- estimated from projectb

There are also about 84 DMs currently, and some number of people
maintaining packages by sponsorship.

For comparison, if my LaunchPad fu is any good, Ubuntu core has some 55
members (upload to Ubuntu's main), MOTU has some 120 members (upload to
Ubuntu's universe), and REVU has something like 948 members (reviewed
upload to universe). I'm not sure what that should imply for Debian --
maybe it means the 8x factor between MOTU and REVU is something we should
be able to duplicate between DDs and sponsored uploaders, eg.

> >    Over the next twelve months, what single development/activity/project
> >    is going to improve Debian's value the most? By how much? How will
> >    you be involved?
> I believe that the most important thing we need to do is to work out
> exactly what we want the Debian project to be, in terms of membership.

That seems very meta, as did Zack's answer. As a Debian user, or as a
free software developer, I just don't see a lot of benefit in that, and
I can't really see how other users/hackers who aren't directly involved
in Debian would do any better.

Likewise with:

> I think that in the last 12 months one of the biggest improvements has
> been making significant changes to our core teams. [...]

I realise this is seen as a huge thing within the project -- finally,
the cabal is overthrown, long live the new cabal and all -- but what's
the externally visible benefit? I guess there's the 1%-2% increase in
package growth above for lenny compared to etch? Is there anything else?

> In terms of strict numbers, I won't pretend to know what the
> difference will be. It's not going to be measureable immediately,
> anyway.

Presumably an important part of any leadership role is the ability
to prioritise different goals, and to justify those priorities. Isn't
estimating, quantifying and measuring costs and benefits pretty much
the state of the art in doing that well?

> What do *you* think is the best thing we could do?

I don't know; I've lost a lot of confidence in Debian's capabilities.
Which is a shame, because I don't think there's ever been a time when
its potential to make an impact has been higher -- both directly, and
through Ubuntu's userbase.

The biggest single benefit to users that I can think of would be providing
a good way of packaging and maintaining web applications for deployment
on servers -- dealing with the inevitable security updates, being
compatible with multiple web servers (and fastcgi, etc), being compatible
with multiple databases and automatically configuring them, supporting
multiple sites on a single system, supporting user-based access control,
etc. Some of that's already done (particularly the database support),
other bits not so much -- in my experience, outdated packages have been
the showstopper. Getting that done well would affect a signficant number
of server installs for both Debian and Ubuntu, in a fairly notable,
user-visible way; and hosting web apps is one of the areas proprietary
software competes with free software on a relatively level playing field.

Free software's doing pretty well, so I can't think of any big wins
for it. A moderate sized one could, I think, be had by partnering with
proprietary vendors who build things Debian users want to run -- eg,
vmplayer, lots of webapps etc -- and having them officially packaged and
distributed by Debian. Benefits: makes things easier for Debian users,
improves free software's position when people compare installing/running
that software on proprietary platforms against Debian, and gives
the vendors an opportunity to see benefits of open source development
techniques, which will encourage them to adopt some of them, increasing
the amount of software freedom in the world.

Certainly there are lots of things purely within Debian that could be
better; but if there aren't some big externally visible wins that Debian's
going to have, it all seems a bit pointless. Why spend time making Debian
twice as successful at achieving (almost) nothing, when there are so
many other projects out there making huge differences to people's lives?

> >Bonus question: in retrospect, what single activity/etc over the past
> >twelve months improved Debian the most? By how much? (Can you really
> >justify that?) How were you involved?
> Numbers? Meh: what do you think? :-)

I can't actually think of anything much over the past year that's been
a huge win. Certainly nothing on the scale of the ssh vulnerability,
and I'm not sure of anything even on the scale of losing m68k.

Anyway, a few things that come to mind, and some corresponding numbers:

  equivs bug #449542 had its patch applied and uploaded by NMU, which
  made me happy. According to popcon, equivs has a vote thats about 1.39%
  the size of perl-base's. On my system there's about 300 packages that
  get a vote; and I'd count the patch itself as a moderate feature. So if
  that counts as a 20% improvement to equivs, which is 1/300th of 1.39%
  of Debian systems, that's a 0.00093% improvement for the project.

  debhelper got a new "dh" interface, which was kinda cool. Its vote is
  about 7.02% of perl-base, but the new feature's probably a bit obscure,
  so count it as a 5% improvement to the package. That's a 0.00117%
  improvement to Debian.

  goplay got mentioned in the release notes and sounds kind of
  cool. It's not used much yet though it seems -- its vote is only
  0.13% of perl-base. Now it's a completely new package, so say a 100%
  improvement score for its users, resulting in a 0.00043% improvement
  of the distro as a whole.

  The armel port got released. AIUI the benefit of armel over arm
  is incremental: some new hardware's supported, programs run a bit
  faster, but it's not especially exciting to people who don't dream in
  Arm assembler. So call it a 5% improvement for the average arm user,
  which according to popcon is about 1.29% of Debian users (about half
  of whom are running armel already), for a 0.065% improvement.

What's that mean? It means the addition of "dh" improved Debian's user
experience by slightly more than the equivs bugfix, but not much. Does
that sound sensible? It fits my thoughts, but maybe I'm way off. "dh"
has the advantage that it might make Debian easier to develop, leading to
more bugs getting fixed in future; you might or might not want to count
that in your comparison. It means goplay was only worth about half of
"dh" or the equivs patch -- even though it's a major, useful new feature,
nobody's taking advantage of it. It means the armel port is worth about
70 improvements on the scale of the equivs patch or "dh". If 10 people
were involved in the armel rollout, they should've each spent about 7
times as much effort getting it done as was put into dh or that equivs
patch for their time to have been well spent.

Those seem like useful things to be able to quantify and reason about
to me.


Attachment: signature.asc
Description: Digital signature

Reply to: