Re: Bug#397939: Proposal: Packages must have a working clean target

[Manoj Srivastava <srivasta@debian.org>]

Hi,


        In all of the following discussion, no one has ever said
 anything about *WHY*  policy states that clean must undo what build
 does. Unless we are clear on the rationale for dictum, trying to
 resolve the issue is like playing blind man's bluff.

        There are several reasons for wanting a clean target:
 a) one should be able to iteratively tweak the source code, and so
    the build/clean cycle should be idempotent.
 b) The build process must be reproducible -- and this is where I see
    the aautotools recommendation failing.  If we are supposed to be a
    part of the free software community, we expect suers to contribute
    back to us. One of the major areas where such help is needed is
    debugging -- so someone building in a non-buildd or a non-one-shot
    build process , should have the same, reproducible results as the
    developer does.

    So, if I ship some code, and I am on IRC with a user, they do
    ./debian/rules clean, install some changes I suggest, we should
    get the same compiler results -- without such a common baseline,
    collaboration is made harder.

        If idempotency and reproducibility are the goals that we are
 striving for, any rules and policy dictums we come up with must
 reflect that.  If we relax the idempotency and reproducibility
 requirements, we must be sure that we are not endangering the use
 cases that the rules were put into place to protect. If we do hurt
 these use cases, we must have a clear idea why these use cases are
 not so important.

        Flailing around without laying down the ground rules is going
 to lead to sub-optimal policy.

On Sat, 11 Nov 2006 13:52:06 +0000, Stephen Gran <sgran@debian.org> said: 

> This one time, at band camp, Russ Allbery said:
>> Stephen Gran <sgran@debian.org> writes:
>> > This one time, at band camp, Russ Allbery said:
>> 
>> >> As previously discussed, it's very difficult to comply with this
>> >> directive as written if one is following the autotools-dev
>> >> recommendations for how to regenerate the various autotools
>> >> files.  Before putting too much weight on this directive, I'd
>> >> really like to find some way of reconciling that, since right
>> >> now it's a frequently-violated dictate of Policy.

        There is a simple way of reconciling that, which is what I
 use.  I keep up with the build process of my packages, and I run
 autotoools on my packages before the fact, and I feed the results
 back upstream.  There is no need to replace the config.* files on the
 fly, since they are fairly up to date to start with. If those files
 are so very out of date, then very probably there is all kinds of bit
 rot elsewhere in the package too.

>> > That's probably a bug in the autotools-dev recommendation, rather
>> > than a problem with policy, though.  For handling
>> > config.{sub,guess}, the freeradius package has a reasonable
>> > method for cleaning up after itself.  For rerunning autotools at
>> > build time, well, I tend to think that's a mistake, but we've had
>> > that fight before and I'm not really interested in reopening it.

>> Well, by contending that this is a bug in the autotools-dev
>> recommendation, you're reopening that fight.  :)

        This is not a fight. This is reopening the issue and
 determining what is the right thing to do.  If we are afraid to
 reexamine issues when suboptimality is pointed out to us, we should
 stop trying to determine policy.  Correct technical policy, and not
 fear of autotools devs, should be guiding us here.

> Well, not precisely.  Policy says clean should clean up after the
> build, and the autotools-dev recommendation makes it
> programmatically difficult to do so.  In the disagreement between
> these two, I tend to think policy is probably more likely to be
> correct.  If the autotools-dev recommendation is altered to make it
> simpler to programmatically clean up to the starting point in the
> clean target, then they are no longer disjunct.

>> Really not reopening the fight means providing some acceptable path
>> for those who would really prefer not to create massive Debian
>> diffs and, more importantly, to test building the package from
>> *source* rather than an intermediate source form partly generated
>> by the Debian developer.

        Building packages from source is good -- but in this case, it
 does introduce another variable as far as auto-tools and helper
 package versions.  We can say that we find this acceptable -- and
 that package should build depend on autotools, and _always_ run
 autotools in the build process.  This would tend to find bugs in
 those packages quickly, too.

        Building from sources improves the package quality in the long
 run, but does impact reproducibility, since now the tool chain
 components have increased in number.

        This also stands for lex, yacc, and swig output files.

        However, I wonder about the value of artificially reducing the
 size of the diffs. there are tools for examing the diffs, and for
 separating out the diffs on autogenerated files from diffs on true
 source files. 

> I can imagine an argument for removing all of the intermediate files
> (Makefile.in, configure, etc) in the clean target and rerunning the
> autotools stuff from the build target.  This would provide a
> relatively small diff (provided upstream doesn't ship the
> intermediate files), although I am not sure of the wisdom of this
> approach.

        In my experience, upstream ships ancient versions of
 intermediate files that often no longer work on hardware that Debian
 supports.  This has been my experience with fvwm, and before the
 current release, make.

> It would be nice if we could support all sorts of forms of rebuilds,
> but in practice, what we tend to take seriously is the sort of FTBFS
> bugs that will affect the autobuilders.  Since they build from an
> intermediate form generated by Debian developers (debian/rules), I
> am not sure how much we should worry about other use cases.

        I think we should worry a lot about other sue cases, or give
 up the pretense that we are a free software distribution.  The
 buildds are just delivering a binary only distribution to the end
 users; the free part comes from delviering source cod4e that our end
 users can tweak and modify and rebuild from.

        Policy should not be just concerned with releases and
 delivering binary packages, but with the overall goals of the
 distribution as a free software project.

        manoj
-- 
Never forget what a man says to you when he is angry.
Manoj Srivastava <srivasta@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C