Re: RFC: Package build automation tools (debmake replacement?)
Hi,
This is the revised document ...
manoj
=============================================================================
Request for requirements: Package build automation tools
Manoj Srivastava
The Debian Developers List1
V1.0, Wed Feb 26 1997
This is an effort to get the Debian developer community involved in
determining the perceived short coming of the current package build
automation tools (think debmake). Ideally, this would lead to the con-
struction of a replacement of the current set of tools. It would be
preferable to have an open design and development process that works
on the basis of consensus. There is going to be a parallel effort to
refurbish dpkg and friends as well, and we should keep track of devel-
opments in that (higher priority) process. Once this tool is completed
it may well be the standard way of building packages, which we do not
yet have, and needs to designed with care. This document endeavors to
list all the comments that have been made on the developers list
recently, by is by no means complete. Your comments are critical.
-----------
1. This document owes a lot to the contributions of Andy
Mortimer, Ben Pfaff, Chow Chi-Ming, Christoph Lameter,
David Engel, Fabien Ninoles, Ian Jackson, Mark Eichin,
Michael Alan Dorman, Rob Browning and Richard G. Roberto.
______________________________________________________________________
Table of Contents:
1. About this Effort
1.1. The status of this effort
1.1.1. The status the Author
1.2. The scope of this endeavor
1.2.1. Relationship with the dpkg team
2. Wish lists and other requirements
3. Implementation Issues
______________________________________________________________________
1. About this Effort
_____ ____ _____
===== ==== =====
This part of the document deals with meta issues like the status of
this effort, and the scope of the undertaking, etc.
1.1. The status of this effort
=== ====== == ==== ======
Over the last few months, there has been a growing perception in the
Debian development community that the packaging system, though good,
needed to be tweaked. The package debmake sprang out of such a
perception, but now there was percieved a need for modifying that
package as well.
Earlier this month, the board of directors decreed that the status of
dpkg and friends had reached critical status, and a task force be
formed to address the issues involved. The developer community gave it
self an ultimatum, and dpkg (or the packaging tools in general) re-
development was moved to a high priority; and package build automation
tools like debmake were pushed lower down the priority list.
However, not all the developers can be on the dpkg re-development
team, for the reason of sheer numbers, or issues with time commitment,
skill, or predeliction. Some of that number could possibly assist in a
parallel effort to refurbish debmake, and this is what this effort is
about.
It make sense, at least in the phase of trying to define what we are
trying to build to not limit ourselves to any particular
implementation. However, deciding whether debmake would or would not
provide a suitable base to build upon should best be left to after
we decide the desirable features of the tool we are building. I
think requirements should define the implementation, not the other
way around. In other words, if the new tool has to be written from
scratch, so be it.
It would be preferable to have an open design and development process
that works on the basis of some kind of consensus, with deadlock
breaking the task of the VP engineering, I think.
1.1.1. The status the Author
=== ====== === ======
The Author has not been enchartered by anyone for this task, and would
gladly step down in favor of anyone willing to take over this task.
I am just an interested developer, concerned about debmake, which,
IMHO, despite being a wonderful package, could do with a few
enhancements. Christoph does not have a major amount of time right
now, and I wrangled reluctant blessings from Bruce and him (which is
*not* a charter) to incorporate stuff if I actually manage to come up
with an improvement ;-). If this tool is easier to write from sctratch
rather than modifying debmake, well, that's the way the cookie
crumbles.
I'm just trying to get people involved in helping me design what is
percieved to be needed. Something like this effort needs at least a
compiler, a point man, and I invite anyone who feels commited to
improving debmake to step forward, and I'll happily stand down.
1.2. The scope of this endeavor
=== ===== == ==== ========
We are trying here to develop package build automation tools, or, in
other words, a helper tool suite for the package developer, somewhat
like debmake does currently.
We are not trying to re-write dpkg, that is the task of another team.
We do need someone to keep this process moving, and to limit the scope
of the effort so we don't bite off more than we could chew, and we
don't tread on the toes of the people Bruce and Ian assign to dpkg and
friends, which is a higher priority item.
1.2.1. Relationship with the dpkg team
============ ==== === ==== ====
We need to track the efforts of the dpkg team when they get going,
since changes there will affect the helper tool suite cnsiderably.
2. Wish lists and other requirements
____ _____ ___ _____ ___________
==== ===== === ===== ============
Finally, here is the current requirements list, in no particular order
(I have tried to order it partially from the general to the specific)
1. Make it easier to make a package from scratch. This should be
geared towards enabling a novice maintainer to build a Debian
packagewith the least effort possible and allow a gradual
development of package maintainence skills.
2. Make it easier to keep a simple debian/rules file up to date, by
automating a lot of it in a transparent manner.
3. Make it easier for packages to track standards.
4. Create an intelligent template
5. Create installation script templates
6. The tools, especially the update tool, if different from the
initial template creation tool, should be idempotent, and by no
means loose user configuration.
7. To the greatest extent possible, packaging policy shall be
plemented by the packaging system.
8. Helper programs to automate various aspects of the process. (These
may or may not be run time tools)
9. If multiple tools are present, then this package shuld bear in mind
name space pollution, and implement a common prefix, like dpkg has
done.
10.
A lint-type tool which checks the constructed package.
11.
Consult a configuration file which can modify the action of the
tool (guide the creation of templates, for example). For simple
packages, it may be enough to create such a configuration file, run
the tool, and create the package. (implementation note: mention has
been made of M4, Autoconf, Automake, and dist, as well as RPM spec
files; we could throw in lex/yacc for completeness).
12.
Implement a version number that is kept with the templates
(debian/Rules/version), and the upgrade tool should not modify
anything unless the version number in that file is less than or
equal to the version number built into the tool -- on upgrade, the
version number is brought into sync). This will prevent accidental
downgrades of the rules. This may also be used to take special
actions for older versions of the templates, if needed.
13.
On the same lines, maybe there could be a policy file that should
be looked at, to guide the action of the tool.
14.
An automated tool for uploading packages
15.
Automatically install DEBIAN files
16.
The action taken by the tools should be auditable, without
requiring super user access, and preferably without running as yet
unaudited programs.
17.
The output of the tools should be audit-able.
18.
The user may easily override parts of what actions are taken, and
modify them (customize actions). This should not be an all or
nothing option, finer granularity than that is desired. The details
of this wil be decided partly in the implementation phase, but I
envisage that a typical binary target in the rules file may have
half a dozen _chunks_, any one of which maybe separately over-ridden
by the user.
19.
The utilities/helper tools should be limited in their actions
(preferably limited to only modifying files in one directory? And
possibly subdirectories thereof Under no circumstance should they
modify anything outside the Debian subdirectory except possibly
/tmp No mucking in system directories. If possible, schemes for
enforcing this should be investigated.
20.
There should be a no exec option, which should not require super
user privilege to execute
21.
If at all possible, the commands to be run should also be available
statically (that is, in a file somewhere), and not only available
inside an executable image. Even in a shell or perl script, the
actual actions taken could quite likely be hard to determine (a
series of case statements would make tracing arduous).
22.
The tool should preferably not depend on anything not in the base
section.
23.
There should be no requirement that the tool be installed on a
system in order to build a package from the sources _even if_ the
developer had used the assistance of this tool.
24.
Either automate, or provide hints and templates to install common
files like info, documentation, cron, init, etc.
25.
Either automate, or provide hints and templates to compress
documents and man pages (ignoring whatever files, for example HTML
files, that the current policy says should be exempted, this also
includes file size criteria). This should of course have to track
policy, and keep up with the changes.
26.
Merely installing a new version of the package should *never* break a
package build process. This takes precedence over propagating new
27.
The preference is for automation of routine tasks, to the extent
that it does not make the rules/script too complex for easy
auditing.
28.
It would perhaps be preferable to have this tool broken up into
parts, which could then have alternates or additions over time,
permitting developers to tailor the help tools used for the package
(a share library searcher is not required for a .pm only package,
or a pure lisp add-on). If so, there should be a wrapper that
calls a default set in sequence so that the ordinary developer does
not have to cope with all these tools. _Example:__ dpkg-buildpackage.
Even if this is not done immediately, the design should be flexible
enough to allow this development. Whether this is implemented
remains an implementation issue.
3. Implementation Issues
This is empty by design, further along into the process this shall be
populated.
--
"You can't have filenames longer than 14 chars. You can't even think
about them!" --Larry Wall in Configure from the perl distribution
Manoj Srivastava <url:mailto:srivasta@acm.org>
Mobile, Alabama USA <url:http://www.datasync.com/%7Esrivasta/>
Reply to: