[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Debian needs more buildds. It has offers. They aren't being accepted.

Anthony Towns <aj@azure.humbug.org.au> writes:

> On Thu, Feb 12, 2004 at 09:11:43PM -0600, Steve Langasek wrote:
> > Furthermore, what should our expectations be when one of the port
> > maintainers in question has a standing objection to anyone NMUing his
> > packages?
> As RM, my policy has been to not be worried about the buildds as long as
> http://buildd.debian.org/stats/graph2-week.png stays above around about
> 95%. It's appropriate to show mild concern for arches that aren't bunched
> in the top group (which is arm, m68k and mips* atm), and appropriate
> to check what's going on if an arch drops below 95% and start worrying
> about other action (like modifying the testing scripts to ignore out of
> date binaries on that arch, pestering the buildd maints, and eventually
> dropping the arch from being a release candidate) if it stays that way
> for more than a few days, or drops below 90%.
> At the moment, there's no cause for alarm about any of the architectures,
> let alone calls for the removal of the guys managing it.
> Cheers,
> aj

Taking a look at www.buildd.net I get the following figures for how
up-to-dat archs are:

arch    need build      building        dep-wait        SUM  +  failed
arm     0               156             41              197     16
m68k    55              40              35              204     74
mipsel  57              167             64              288     7
mips    213             145             66              424     8

Exagerating one could subtitle the colums as follows:

        (slow cpu)      (slow admin) (bad ordering)             (brokenness)

Having not enough cpu power calls for more buildds:

- Arm has got 2 new ones just recently, from the graphs I guess they
  got activated around Feb 7th. So the hardware seems fine.

- Mips has buildd offers that could have cushioned the effect of the
  existing buildds having hardware and kernel problems.

- Mipsel could use some buildds too. I have one with 64MB ram that
  could build preferably smaller packages freeing the big machines for
  XFree, kde, gnome, qt,... It, being a XXS1500 board, is also well
  maintained kernel wise by the manufacturer. (unlike the cobald cubes

The huge amounts of building packages indicates that the admin does
not manage packages as fast as they are build. Given the archs
(currently) can't even keep up with sid (and thats only going to get
more and more packages) means there should be more people involved. A
delay in managing packages can have a ripple effect of making a lot of
packages fail that need to be put in dep-wait. That not only slows
down the buildd but also creates more work for the admin.
[Note that m68k has ~4 packages per buildd building and arm 39 each]

The dep-wait you can add to needs-build. Those are waiting for another
build to finish or a package they Build-Depend on to be fixed.

The failed column at last could be seen as a measure how maintained
the port is by the general DD community (and how flakey the toolchain
is of cause). I wonder how well that number reflects reality. How many
of the "building" packages for arm, mips and mipsel should actually be
in failed? [also m68k seems to get all the automake timestamp skew
FTBFS bugs because patch takes so long :(, but that will change with
2.6 and mikrosecond timestamps ]

I do see problems there. One, the arm buildds getting organized by
Martin, is solved. The m68k port too has organized a lot of new
hardware since the compromise and is catching up, strugeling to get it
all setup and added. Today I got asked for the subnets my buildds are
in so things seem to get moving again.

But mips and mipsel are just hanging on. They hardly recover and every
minor problem will put them right back down into critical regions.

What if Ryan falls sick for a week or James? Things will just stand
still. It doesn't even mater how good a job they do or how good they
communicate (or not). Mips, Mipsel and Arm show problems now and are a
single point of failure, a catastrophe waiting to happen.  [M68k on
the other hand is just slow but its steadily and unstoppable crawling
back up]

Or the current tar problem. Several buildds have the broken tar
installed, 16 package builds already hit the rather obscure
bug. Before the compromise aparently any buildd admin could have
excluded those buildds temporarily from wanna-build and prevent any
more packages being build broken. After recognising the problem it
would have been patched up in minutes. Now the responce time and thus
damage is far worse. I call that a problem too.


Reply to: