On Fri, Apr 13, 2001 at 11:29:54AM +0100, Julian Gilbey wrote:
> On Fri, Apr 13, 2001 at 03:20:50PM +1000, Anthony Towns wrote:
> > Suppose we do. First, what benefit does this provide to the user who
> > gets a copy of the new woody release in, say, six months on CD. Can they
> > rely on every binary having a manpage? No, because it's not a real "must".
> The problem you seem to see is that users will not realise that the
> meaning of policy directives has changed.
No, the problem is that it doesn't provide any benefits to the user over
having a "should". A user buys a Debian woody (or woody+1, or whatever)
CD at some point after it's released. Can he rely on every binary having
a manpage? No. Can he rely on most binaries having manpages? Yes. Can he
file bugs about this? Yes. Can he submit a manpage of his own and have it
included? Yes.
None of these answers are any different to how it is now.
Changing policy in this manner has no benefit to users.
> No, but they can't now either because it's only a "should". At
> present, lots of binaries don't have manpages. My suggestion would
> ensure that by the time woody+1 is released, *almost* every binary
> will have a manpage.
Or it'd ensure that ftpmaster and the release manager would ignore policy,
along with more maintainers. In any event, most binaries have manpages
already.
> At present, there is no incentive for maintainers to add missing
> manpages;
I'm sorry? There's no incentive for maintainers to fix bugs or improve
their packages? Are you deluded?
And you're not adding incentives, you're adding *disincentives*. Carrots
are all very well, but we're talking about sticks here. Punishment for
not doing what ever "we" want them to do.
If there's not a clear and obvious reason why it's better to do what
policy says than not independently of policy saying it, it shouldn't
be in policy. If there is a clear and obvious reason, then that's the
incentive you're looking for.
> > Second, what happens to, say, security NMUs of packages without manpages?
> > Do they get rejected out of hand, because of RC bugs? Or do the security
> > team have to write manpages as well as do the fix and release advisories?
> > What about NMUs to fix real RC bugs, like the app-defaults thing? Do
> > they have to write manpages as well as fix the bugs as well? What about
> > maintainers who just want to fix normal bugs promptly? Do I have to, say,
> > write manpages for ping6 and traceroute6 and tracepath and tracepath6
> > when I make an upload of iputils.dsc to set its Maintainer: to debian-qa?
> As I said when discussing this originally:
> (1) NMUs may be exempt from these rules for the very reasons you
> point out.
So I should orphan all my packages and maintain them by NMU.
What a pain.
> (3) Maintainers who just want to fix normal bugs promptly *will* have
> to also make sure that their packages are policy-compliant before
> they upload them. Otherwise we are back to the current setup
> where policy is ignored by a significant number of maintainers.
So if I don't have the time or interest to write a manpage for some stupid
program, you'd rather just ignore the cute patch for case insensitive
searching or the new upstream update that I've packaged, or whatever it
might be.
> Either policy is being written by a small group of maintainers sitting
> in an ivory tower with nothing better to do with their time, and has
> no relevance to the rest of the project, or policy is an important
> tool for ensuring that there is a modicum of consistency between
> packages. The latter is the reason why Debian is so much better than
> many other distros, as Chris Rutter once told a visitor to our stand:
> we require packages to comply to quite well defined specs.
Well, quite frankly it looks more like the former to me these days.
What's it mean to be in an ivory tower? That you're not getting down into
the muck and helping with real problems. That you're not listening to the
people you're trying to order around. That you're distanced from them. That
you think you're superior to them.
How does saying "you can't upload packages anymore if you don't write
manpages for all your binaries" help those manpages get written? It
doesn't. It either takes time away from volunteers who'd rather be doing
other things, or it makes volunteers decide not to waste time on improving
their packages because they're not willing to do what you dictate.
As I said in my previous message, -policy seems to be uninterested
in bothering to actually investiage the issues they're talking about:
actually working out which packages would be removed from the distro after
a policy proposal is accepted is just Too Hard, and looking at packages
themselves isn't worth the effort when there's a Standards-Version field
that policy declares should be correct, and therefore clearly must be.
> Only the truth is that we don't, really. Maintainers who want to
> follow policy do, and those who don't, don't. And there's nothing
> encouraging them to do so.
And this is unjustified, arrogant nonsense.
Build dependencies are not required at the moment: 60% of packages
include them anyway (that's ignoring the x% of packages that don't need
build-dependencies at all).
You complain, eg, that no one's bothered to update "Standards-Version"
since 3.1.0. Looking through the changelog for policy it doesn't even
seem like there's been anything particularly crucial that's changed
since then: there's been some changes to the way policy's been layed
out (MUST and including the packaging manual), a number of bugfixes,
a fair bit of X related stuff, and some new optional things like
DEB_BUILD_OPTIONS. Further, most of this hasn't been particularly
publicised, because everyone on this list is more interested trying
to find ways of punishing people who don't follow policy than trying
to make it easier to follow by resurrecting the weekly policy proposal
summary or helping Shaleh fix lintian.
> Incidentally, the reason that app-defaults required RC bugs and NMUs
> was not because policy said so, but because things would badly break
> otherwise. That's why policy had to be modified so rapidly on this
> point.
Yes. That's why app-defaults was completely correct to be a MUST.
By contrast, it's also why not having a manpage *isn't* RC: not having
a manpage doesn't break anything.
> > "In order to fix the typo in this bug, you must read the current policy
> > document, add a manpage, move to /usr/share/doc, add a symlink from
> > /usr/doc, update your description, contact the translation team to make
> > sure your package's description and manpages are translated into French,
> > German and Japanese, register all your documents with doc-base, register
> > your programs with update-menus, ensure lintian doesn't register any
> > errors, and then you may upload".
> - You must check the current upgrading-checklist to check whether
> there's anything which might affect you
> - Yes, you must add manpages if necessary. Not hard using tools like
> pod2man.
If it's so easy, no doubt you'll be writing a few manpages every day and
sending it off to maintainers.
Everything's easy if someone else has to do it.
> Are you seriously suggesting that it's OK for maintainers at this
> point in time to:
> - not bother to check for policy updates on a monthly or bimonthly
> basis if they're actively maintaining packages (actually, I think
> that a posting of the new upgrading-checklist section to
> -devel-announce on release of a new policy version would be a very
> good idea)
The only list people have to read is debian-devel-announce. If they
haven't checked up on policy recently and anyone cares, they should be
able to expect to receive a bug telling them about it.
> > You can't dictate that maintainers will be motivated to fix their packages
> > no matter how much you might want them to. It's a mistake to try.
> Why can't we?
Because that's *not how people work*.
That's why we have capitalism and democracy and such, instead of feudalism
and monarchy.
> If their package doesn't get in with policy bugs, then
> the onus is on them to fix them if they want to upload a new version.
Or to go spend their time doing something that's appreciated, like hacking
on the kernel, or helping out with another distro, or similar.
> If their package does get in but they just get bugs filed, where's the
> incentive to fix them? Look how many times people ask: "why hasn't my
> package moved into testing yet?" People *want* their packages to get
> into the distribution.
Yes. And look how annoyed people are when their packages don't go into
testing. Notice how every second thread on -devel (well, nearly) is
entitled "testing is broken!".
Further, notice how the reasons for not putting something into testing
are *very* conservative. They're things that are already RC bugs one
way or another: either because the package doesn't build everywhere, or
because some of its dependencies aren't (or won't) be satisfied somewhere,
or similar.
*Further*, it doesn't stop people from uploading to either unstable or
experimental: that is, it doesn't stop people from doing anything they
used to do. Sure, you mightn't be able to get your package into testing
now, but you certainly weren't able to before either, because testing
didn't use to exist. And all the reasons for rejecting a package from
testing have always been reasons for removing a package from frozen when
a release is coming up.
> > Someone has to do the work. The maintainer's already indicated s/he
> > isn't that interested in writing manpages by the very fact that s/he
> > hasn't written any.
> Agreed. But if there were a big incentive....
An incentive would be "Write a manpage, and get given a cheque for
$30US". Or "For every manpage added before July, FooCorp will donate
$50US to the FSF (ironic, hey?)".
> > If you want manpages as badly as you seem to: *you*
> > write them.
> No. We're deciding as a project that manpages are a Good Thing and
> that binaries have to have them.
The way the project decides something is a good idea is by implementing it
and making it work.
Once there are only a handful of packages that don't have manpages for
all their binaries, *then* it might be worth considering making it a
requirement for uploads, but by that time it's equally worth making it
a requirement for all packages.
> In your setup, there will never be a time when all packages will have
> the required manpages, as there will always be maintainers who can't
> be bothered.
That's what NMUs are for. And I've never once said all packages have to
conform to something before we make it a MUST, just almost all of them.
> It is always possible to ask for help, say from -devel or -qa.
It's also possible to give help without being asked.
It's also possible to get things done without having to threaten people
with dire consequences if they don't do what you want.
> Absolutely! I totally agree! The policy cabal rules Debian! Yeah!!
>
> Anyone can join these discussions. Any developer can floor a
> proposal. They are not closed. The mailing list is public, archived
> and open. But you seem to be suggesting that policy should
> effectively not have any power at all. That's absurd.
It's not absurd at all; -policy has *never* had any power in the past. New
uploads gets stopped by ftpmaster if they feel it's appropriate, packages
get removed from frozen if the release manager feels it's appropriate, and
people try to follow policy merely because it's a good idea.
That's all the power policy has ever had, up until I made the
MUST/SHOULD/MAY proposal and formalised the old "important" severity.
That is: the good parts of policy would get followed, the bad parts would
get ignored. Whatever happened, happened.
When I made the MUST/SHOULD/MAY policy it was because I (as acting release
manager) found the burden of deciding which packages to remove from frozen
was too much: I couldn't keep my decisions consistent, and it just took
too much time. I figured distributing that workload and having it written
down somewhere obvious would help, and -policy seemed the right place.
Now it doesn't. This group doesn't appear to me to be doing a good job
of handling that power at all. In particular, it's not doing an even
slightly good job of knowing when *not* to use it.
> > Maybe the entire idea of "MUST" is wrong, and -policy can't be trusted
> > with fairly and reasonably deciding what's acceptable for release and
> > what's not.
> <silly>
> I think I'm going to upload a command-line tool package which doesn't
> have a copyright file, which puts its binaries in /bin, /usr/games and
> /usr/bin/X11, which doesn't do the "right thing" in lots of other
> ways. But it's OK, because it works, and policy doesn't matter
> anyway.
> </silly>
Please, stop arguing with me for a moment and just try to get what I'm
saying.
Clearly, the line has to be drawn somewhere. We do have to have a minimal
acceptable standard for inclusion in Debian. If you really think I'm
saying anything else, I guess there's no point my bothering to reply to
your messages: you're obviously not reading a thing I write.
> > > So one day you have a minor bug report against your package, the next
> > > day it becomes an RC bug report with the threat of your package being
> > > pulled.
> > "the next day" ?
> Turn of phrase meaning "suddenly".
I wouldn't think "Three months later" could reasonably be considered
"suddenly".
> > One day you have a minor bug report against your package.
> > Three months later, while every other maintainer except you has gone and
> > done stuff about it, and a policy proposal listing the packages still
> > violating the guideline (yours and a handful of others), and saying why
> > consistency in this area is crucially important, passes, the bug gets
> > upgraded to serious, and someone offers to NMU for you.
> And how many packages *still* use /usr/doc after over 18 months
> despite constant discussion on mailing lists and bugs being filed?
Huh?
How many packages have symlinks in /usr/doc? I'd hope all of them, since
policy mandates it.
How many packages reference /usr/doc when looking for documentation? Again,
I'd hope all of them, since, again, that's what policy mandates. (See 12.5)
And instead of asking rhetorical questions of that sort, why don't you
ssh to auric and work out the answer yourself? There are 260 unique
packages that put files in /usr/doc according to sid's i386 Contents
file [0]. That's about 3% of packages, by binary package. Seems like a pretty
trivial amount to me, and more due to maintainers who've gone completely
AWOL, or packages that no one cares about, than any conspiracy of apathy
against policy.
>>>> I have another suggestion. Let's get rid of the "Standards-Version" field
>>>> in debian/control and insist all packages must match current policy.
>>> No and yes.
>>> I think the Standards-Version is good when we look at a package to
>>> determine what the state of policy was when it was last uploaded and
>>> what might need to be modified to bring it up to date. So I wouldn't
>>> want to lose that information.
>> You'll note that phrase: "might need to be modified to bring it up to date".
> Yes, absolutely. It's indicative.
It means they might not need to be modified to be brought up to date too;
either because the standards-version simply hasn't been updated, or because
none of the changes in policy since then have affected that package.
It means that you'll need to do further investigation to work out just how
indicative they are.
And when you're getting a 50% error rate, that's not very indicative at
all, imo.
> > Skipping down, we see:
> > > This is exactly the situation at present. In sid/main, there are
> > > Standards-Versions ranging from 2.1.0 through to 3.5.8 (which is quite
> > > an achievement, really, given current version is only 3.5.2!). Out of
> > > approximately 4000 packages, there are close to 300 source packages
> > > with version < 3.0.0 and over 1000 (that's a quarter) with version
> > > less than 3.1.0.
> > Skipping further down, you then use this as an indiciation that no one listens
> > to policy, and that therefore harsher measures must be undertaken.
> And how many packages still don't use /usr/share/doc? Yes, the
> Standards-Version is indicative only. But there is plenty of other
> evidence that policy is effectively being ignored by many
> maintainers.
It's pretty easy to get a list of the maintainers doing this ignoring
[1]. If you have a look through it, you'll notice the names of lots
of people who just aren't really about anymore. You'll see the names
of people who obviously don't ignore policy, like Raul Miller, Wichert
Akkerman, Ian Jackson, Michael Alan Dorman, Martin Schulze, Herbert Xu,
etc. (The only reason my name's not there is because someone's NMUed
the only packages of mine that'd be affected).
There are 117 such maintainers, apparently, of around 777 maintainers all
up. 99 of them maintain other packages that don't include usr.doc stuff.
18 people out of 777 is 2%.
> > [...]
> > package doesn't use /usr/share/doc. You'd be wrong. A quick check [0]
> > (which would tend towards underestimating) lists 134 such packages, which
> > is around half of the 280 packages with standards versions < 3.0.0 in sid.
> That's still pretty bad.
No doubt you'd also say that 1 package using /usr/doc still is pretty bad.
> And there's another 130 packages or so which
> have version number >= 3.0.0 which still use /usr/doc.
Using the same counting method, there's actually 46 [2]. The difference is
made up by packages who has binary packages with different names than
its source package.
> Your point?
Work out your numbers, don't make them up?
> > This group seems quite happy to both make poor assumptions about the
> > correctness of other packages (treating standards-version fields as
> > more canonical than they are; not wanting to bother checking which
> > packages would be affected by new MUSTs in policy), and if -policy
> > is holding itself to such a low standard concerning other packages,
> > it seems at best hypocritical to demand package maintainers adhere to
> > the significantly higher standard that's being proposed.
> Agreed. We've made mistakes and have to figure out ways of improving
> them. That doesn't mean no-one else should do better.
Then try removing the log in your own eye before criticising the speck
in someone else's?
> > > I don't think that it's reasonable to demand that all packages in the
> > > archive are constantly updated to match current policy, with two
> > > provisos:
> > > (2) No package should be more than [fill in appropriate time] old
> > > policy-wise, eg., require all packages to have Standards-Version
> > > >= 3.0.0 in woody
> > How about we just require them to meet all the MUSTs in current policy,
> > whether they've bothered to update the standards-version field or not?
> > Because we already do that, you'll note.
> How to we require it? By filing RC bugs after the maintainer hasn't
> done anything about it for 18+ months, followed by NMUs.
You can file RC bug reports for any MUST in policy, whether they've been
added yesterday or haven't been acted upon for twelve months.
We enforce them by having bugsquash days every now and then.
We enforce them by throwing those packages out before we release.
> I'm
> proposing that we shift the responsibility towards the maintainer.
It's already the maintainer's responsibility.
You're proposing that we reject any good work a maintainer does, unless
he also fulfills your pet policies.
Why do I say "pet policies" or use some similar dimunitive? Because
you're not willing to apply it to all packages.
> > > > If you want to make policy apply inconsistently to different packages,
> > > > well, I think that's just stupid, no matter how many times it's proposed.
> > > This is exactly the situation at present. In sid/main, there are
> > > Standards-Versions ranging from 2.1.0 through to 3.5.8 (which is quite
> > > an achievement, really, given current version is only 3.5.2!).
> > There are two such packages: lprng and ax25-tools; everything else has a
> > valid standards-version. Packages can be buggy, maintainers can make typos,
> > it's nothing to get excited about.
> Hence the exclamation mark. Lintian would have caught it.
So? What harm does it do anyone?
> > > Out of
> > > approximately 4000 packages, there are close to 300 source packages
> > > with version < 3.0.0 and over 1000 (that's a quarter) with version
> > > less than 3.1.0.
> > And, equally, old standards-versions are also nothing to get excited
> > about. Those numbers just plain aren't meaningful.
> So let's encourage people to make them meaningful. When correct,
> they're useful.
No, they're not: every benefit we could've obtained from them is already
obtainable by automated checking. See the shell commands in the footnotes
of this message for example.
> > > (1) Many maintainers ignore policy changes and don't apply them to
> > > their packages. The above statistics give some indication of the
> > > extent of this problem.
> > No, they don't. Further, there are more helpful ways of solving this
> > problem than saying "get lost, we don't want your packages" to people
> > who aren't subscribed to this list. Restarting Joey Hess's "Policy
> > Proposals" sumamries and posting them regularly would help immensely,
> > I'd be willing to bet. Restarting the archive wide lintian laboratory
> > and using that to find things that need work, would probably help too.
> Agreed.
Then, instead of worrying about what other people do, why don't *you* do
something and work on either of those things? No one'll complain about
either, and they'll both be effective at easing your concerns.
> But I don't think these are mutually exclusive.
So start on the non-controversial option, and see if that turns out to
be enough?
> > Lintian alone is a pretty impressive measure at thwarting this.
> Only if it's enforced.
Only if it's *used*.
Like I said, your heart's in the right place, but you're obsessing about
enforcing things. We're all volunteers here, and we're all trying to do
the best we can: make it easy to do what's right, and you'll find things
go well; try to force people to do things, and you won't.
> > > (2) We don't enforce policy dictates except by filing RC bugs at a
> > > very late stage in the process, once we've decided that the
> > > requirement should be an absolute.
> > This isn't a "problem", it's what you want to have happen prefaced by a
> > "not".
> Huh? I want it so that when we decide to make it RC, there's very
> little work left to do.
From a user's point of view, looking at the CD she's just bought and is
about to install, how does (2) make any difference at all? Why should she
care even slightly when you filed RC bugs?
If a user's only going to care indirectly about something, then it
itself isn't a problem. Problems are what users see, everything else is
a solution waiting to happen.
If you can't tell what things affects users directly and what things
don't, well, that probably indicates you have ensconsed yourself in
something of an ivory tower.
> > > (3) The above two points mean that it is hard to maintain a high
> > > quality of packages in any automated fashion, and that a large
> > > burden is put on a small number of developers who try to fix the
> > > problems thereby caused, for example the /usr/doc ->
> > > /usr/share/doc transition is still not complete.
> > It's impossible to maintain high quality packages in an automated fashion.
> > Other maintainers aren't robots, you know.
> I know. But there are ways of helping the process.
There are lots of way sof helping lots of things. One of the best ways is
working out what's actually wrong with them in the first place, not just
picking on things which seem like they ought to be wrong.
> > You'll note that the /usr/share/doc transition was meant to take until
> > now, btw, and that we've got 97% of the way there, pretty much exactly
> > as hoped. Without doing any coordinated NMUs at all yet.
> True.
Think about this for a while.
We got /usr/doc -> /usr/share/doc done exactly right, without enforcing
anything.
Think about it.
> > A restatement as a problem might be:
> > (4) Our distribution isn't very consistent in areas in which it could
> > be. For example, many of our packages don't have manpages for all
> > their binaries.
> > This can be addressed by, eg, getting interested volunteers to write
> > manpages and submit them to the package maintainers, which is exactly
> > what -qa is doing.
> Or by ensuring that maintainers do so.
Yes. There are lots of possible ways of addressing problems.
Try to be a little more openminded than just obsessing on one.
> > There is actually language for this, and it was given earlier in this very
> > thread.
> > Some examples from policy:
> > (with a weakened "should")
> Those predate the defined meanings of "must" and "should" as far as I
> know. Yuck.
Nothing in policy predates the defined meanings of "must" and "should":
I went through every line of policy and made sure things made sense.
And whether you think a way of phrasing something is "Yuck" or not
doesn't really effect a user directly either.
> > > (b) The release manager decides upon a minimum acceptable
> > > Standards-Version for each release once (a) has been implemented.
> > And this goes *precisely* against what you've been saying earlier. The
> > minimum acceptable policy for packages to comply with at release is the
> > current version of policy. Even a difference of 0.0.0.1 is unacceptable.
> I *never* said that. Incidentally, minimum difference would be in the
> major patch level.
No, that's not the case. Not complying with policy version X, but
complying with version X - 0.0.0.1 should be impossible. If it's not,
and someone manages it, then they're buggy if they don't comply with the
current policy, and if it's a MUST they don't meet, they'll get removed.
Yes, I realise that knowing that a package was last checked against
policy version X-0.0.0.1 probably means it doesn't need to be checked
against policy version X. That doesn't really matter.
> I'm proposing that we say that all packages have to meed a minumum
> version of policy by the time the release happens, using NMUs if
> necessary. This does not really happen at present, except when QA
> decide to ensure that something happens.
No. This is because policy has never been used to determine whether a
package gets released or not before. MUSTs are all that's necessary to
ensure packages are consistent with the appropriate parts of policy.
It isn't appropriate to insist that all packages match all policy
guidelines in any version of policy in order to be released. As a policy
wonk, I formally object to that. As release manager, I'll ignore any of
the policy groups recommendations to that effect. As an ftpmaster I won't
be a part of implementing such checks. As a developer I won't be a part
of helping maintain a distribution that so abuses its volunteer's time
and effort. It is not feasible, it's not useful, it's not productive,
it's just plain power mongering.
I've explained this time and time again. There seems little point
continuing: you don't seem to be following what I've said in any of
these discussions, and I'm sure anyone who's watching has already been
convinced of anything they're going to be convinced of.
> We got it wrong. We used the wrong criteria to define the meaning of
> policy.
As release manager (ie, the guy who gets to decide what we release), I
can assure you we didn't. If you want to use other criteria, you're quite
simply wrong.
> Replace "manpages" by "/usr/share/doc" and you'll have a much better
> time understanding what I'm suggesting.
Then I'll refer you back to the point that the /usr/share/doc transition
has been completed except for a handful of packages, in the timeframe
that we wanted, without any requirements for enforcement.
> > [0] zcat sid/main/source/Sources.gz |
> > awk '/^Package:/ {P=$2} /^Standards-Version:/ {print P, $2}' |
> > while read a b; do
> > if dpkg --compare-versions $b lt 3.0.0 &&
> > zgrep -q "^usr/share/doc.*/$a$" sid/Contents-i386.gz;
> > then
> > echo "$a $b has usr.share.doc";
> > fi;
> > done
> Pretty inefficient; you'd do better to check for /usr/doc first.
*shrug* Efficiency doesn't really matter in the slightest for this. I'd've
been quite happy to leave it running on auric overnight, if appropriate.
The point is getting something that's mostly accurate, which is where
your statistics based on the standards-version failed miserably.
Cheers,
aj
[0] zgrep ^usr.doc sid/Contents-i386.gz | sed 's,^.*/\([^/]*\)$,\1,' |
sort -u | wc
[1] zgrep ^usr.doc sid/Contents-i386.gz | sed 's,^.*/\([^/]*\)$,\1,' |
sort -u | while read a; do
grep "^$a " /org/ftp.debian.org/ftp/indices/Maintainers;
done | sed 's/ */ /g' | cut -d\ -f2- |
sed 's/(.*>//;s/ <.*>//;s/^.*(\(.*\))/\1/' | sort -u
[2] zcat sid/Contents-i386.gz | sed -n 's,^usr/doc.*/\([^/]*\)$,\1,p' |
sort -u | while read a; do
x=`grep-dctrl -F Package -s Standards-Version -n -X "$a" org-testing/data/unstable/Sources | head -1`;
if dpkg --compare-versions "$x" gt "3.0.0"; then echo "$a $x"; fi;
done
--
Anthony Towns <aj@humbug.org.au> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. GPG signed mail preferred.
``_Any_ increase in interface difficulty, in exchange for a benefit you
do not understand, cannot perceive, or don't care about, is too much.''
-- John S. Novak, III (The Humblest Man on the Net)
Attachment:
pgpqnS05AO6sq.pgp
Description: PGP signature