[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: RFC: Package build automation tools (debmake replacement?)


	There has been no new suggestion for the package build
 automation system for weeks now, so I guess it is time to move into a
 the next fun phase: implementation issues.

	I have been swamped with work recently which has precluded
 anything beyond maintaining my current packages, so it might be a few
 days before I can polish up the ideas I have been toying with,
 however, I am re posting the document so far to remind people we are
 still committed to redesigning our package build automation stuff
 (can someone come up with a less stuffy name?)

 working on pkg-order as well

  Request for requirements: Package build automation tools
  Manoj Srivastava
  The Debian Developers List1
  V1.1, Mon Mar 31 1997

  This is an effort to get the Debian developer community involved in
  determining the perceived short coming of the current package build
  automation tools (think debmake). Ideally, this would lead to the con-
  struction of a replacement of the current set of tools. It would be
  preferable to have an open design and development process that works
  on the basis of consensus. There is going to be a parallel effort to
  refurbish dpkg and friends as well, and we should keep track of devel-
  opments in that (higher priority) process. Once this tool is completed
  it may well be the standard way of building packages, which we do not
  yet have, and needs to designed with care. This document endeavors to
  list all the comments that have been made on the developers list
  recently, by is by no means complete. Your comments are critical.

    1. This document owes a lot to the contributions of Andy
  Mortimer, Ben Pfaff, Chow Chi-Ming, Christoph Lameter,
  David Engel, Fabien Ninoles, Ian Jackson, Mark Eichin,
  Michael Alan Dorman, Rob Browning and Richard G. Roberto.


  Table of Contents:

  1.      About this Effort

  1.1.    The status of this effort

  1.1.1.  The status the Author

  1.2.    The scope of this endeavor

  1.2.1.  Relationship with the dpkg team

  2.      Wish lists and other requirements

  3.      Implementation Issues

  1. About this Effort
     _____ ____ _____
     ===== ==== =====

  This part of the document deals with meta issues like the status of
  this effort, and the scope of the undertaking, etc.

  1.1.  The status of this effort
       === ====== == ==== ======

  Over the last few months, there has been a growing perception in the
  Debian development community that the packaging system, though good,
  needed to be tweaked. The package debmake sprang out of such a
  perception, but now there was percieved a need for modifying that
  package as well.

  Earlier this month, the board of directors decreed that the status of
  dpkg and friends had reached critical status, and a task force be
  formed to address the issues involved. The developer community gave it
  self an ultimatum, and dpkg (or the packaging tools in general) re-
  development was moved to a high priority; and package build automation
  tools like debmake were pushed lower down the priority list.

  However, not all the developers can be on the dpkg re-development
  team, for the reason of sheer numbers, or issues with time commitment,
  skill, or predeliction. Some of that number could possibly assist in a
  parallel effort to refurbish debmake, and this is what this effort is

  It make sense, at least in the phase of trying to define what we are
  trying to build to not limit ourselves to any particular
  implementation. However, deciding whether debmake would or would not
  provide a suitable base to build upon should best be left to after
  we decide the desirable features of the tool we are building. I
  think requirements should define the implementation, not the other
  way around. In other words, if the new tool has to be written from
  scratch, so be it.

  It would be preferable to have an open design and development process
  that works on the basis of some kind of consensus, with deadlock
  breaking the task of the VP engineering, I think.

  1.1.1.  The status the Author
          === ====== === ======

  The Author has not been enchartered by anyone for this task, and would
  gladly step down in favor of anyone willing to take over this task.

  I am just an interested developer, concerned about debmake, which,
  IMHO, despite being a wonderful package, could do with a few
  enhancements. Christoph does not have a major amount of time right
  now, and I wrangled reluctant blessings from Bruce and him (which is
  *not* a charter) to incorporate stuff if I actually manage to come up
  with an improvement ;-). If this tool is easier to write from sctratch
  rather than modifying debmake, well, that's the way the cookie

  I'm just trying to get people involved in helping me design what is
  percieved to be needed.  Something like this effort needs at least a
  compiler, a point man, and I invite anyone who feels commited to
  improving debmake to step forward, and I'll happily stand down.

  1.2.  The scope of this endeavor
       === ===== == ==== ========

  We are trying here to develop package build automation tools, or, in
  other words, a helper tool suite for the package developer, somewhat
  like debmake does currently.

  We are not trying to re-write dpkg, that is the task of another team.
  We do need someone to keep this process moving, and to limit the scope
  of the effort so we don't bite off more than we could chew, and we
  don't tread on the toes of the people Bruce and Ian assign to dpkg and
  friends, which is a higher priority item.

  1.2.1.  Relationship with the dpkg team
         ============ ==== === ==== ====

  We need to track the efforts of the dpkg team when they get going,
  since changes there will affect the helper tool suite cnsiderably.

  2.  Wish lists and other requirements
      ____ _____ ___ _____ ___________
      ==== ===== === ===== ============

  Finally, here is the current requirements list, in no particular order
  (I have tried to order it partially from the general to the specific)

  1. Make it easier to make a package from scratch. This should be
     geared towards enabling a novice maintainer to build a Debian
     packagewith the least effort possible and allow a gradual
     development of package maintainence skills.

  2. Make it easier to keep a simple debian/rules file up to date, by
     automating a lot of it in a transparent manner.

  3. Make it easier for packages to track standards.

  4. Create an intelligent template

  5. Create installation script templates

  6. The tools, especially the update tool, if different from the
     initial template creation tool, should be idempotent, and by no
     means loose user configuration.

  7. To the greatest extent possible, packaging policy shall be
     plemented by the packaging system.

  8. Helper programs to automate various aspects of the process. (These
     may or may not be run time tools)

  9. If multiple tools are present, then this package shuld bear in mind
     name space pollution, and implement a common prefix, like dpkg has

     A lint-type tool which checks the constructed package.

     Consult a configuration file which can modify the action of the
     tool (guide the creation of templates, for example).  For simple
     packages, it may be enough to create such a configuration file, run
     the tool, and create the package. (implementation note: mention has
     been made of M4, Autoconf, Automake, and dist, as well as RPM spec
     files; we could throw in lex/yacc for completeness).

     Implement a version number that is kept with the templates
     (debian/Rules/version), and the upgrade tool should not modify
     anything unless the version number in that file is less than or
     equal to the version number built into the tool -- on upgrade, the
     version number is brought into sync). This will prevent accidental
     downgrades of the rules. This may also be used to take special
     actions for older versions of the templates, if needed.

     On the same lines, maybe there could be a policy file that should
     be looked at, to guide the action of the tool.

     An automated tool for uploading packages

     Automatically install DEBIAN files

     The action taken by the tools should be auditable, without
     requiring super user access, and preferably without running as yet
     unaudited programs.

     The output of the tools should be audit-able.

     The user may easily override parts of what actions are taken, and
     modify them (customize actions).  This should not be an all or
     nothing option, finer granularity than that is desired. The details
     of this wil be decided partly in the implementation phase, but I
     envisage that a typical binary target in the rules file may have
     half a dozen _chunks_, any one of which maybe separately over-ridden
     by the user.

     The utilities/helper tools should be limited in their actions
     (preferably limited to only modifying files in one directory? And
     possibly subdirectories thereof Under no circumstance should they
     modify anything outside the Debian subdirectory except possibly
     /tmp No mucking in system directories. If possible, schemes for
     enforcing this should be investigated.

     There should be a no exec option, which should not require super
     user privilege to execute

     If at all possible, the commands to be run should also be available
     statically (that is, in a file somewhere), and not only available
     inside an executable image. Even in a shell or perl script, the
     actual actions taken could quite likely be hard to determine (a
     series of case statements would make tracing arduous).

     The tool should preferably not depend on anything not in the base

     There should be no requirement that the tool be installed on a
     system in order to build a package from the sources _even if_ the
     developer had used the assistance of this tool.

     Either automate, or provide hints and templates to install common
     files like info, documentation, cron, init, etc.

     Either automate, or provide hints and templates to compress
     documents and man pages (ignoring whatever files, for example HTML
     files, that the current policy says should be exempted, this also
     includes file size criteria). This should of course have to track
     policy, and keep up with the changes.

     Merely installing a new version of the package should *never* break a
     package build process.  This takes precedence over propagating new

     The preference is for automation of routine tasks, to the extent
     that it does not make the rules/script too complex for easy

     It would perhaps be preferable to have this tool broken up into
     parts, which could then have alternates or additions over time,
     permitting developers to tailor the help tools used for the package
     (a share library searcher is not required for a .pm only package,
     or a pure lisp add-on).  If so, there should be a wrapper that
     calls a default set in sequence so that the ordinary developer does
     not have to cope with all these tools. _Example:__ dpkg-buildpackage.
     Even if this is not done immediately, the design should be flexible
     enough to allow this development. Whether this is implemented
     remains an implementation issue.

  3.  Implementation Issues

  This is empty by design, further along into the process this shall be

 "No problem is so formidable that you can't walk away from it."
Manoj Srivastava               <url:mailto:srivasta@acm.org>
Mobile, Alabama USA            <url:http://www.datasync.com/%7Esrivasta/>

Reply to: