[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

DpkgV2 specs



A long time ago word got out that a couple of people were thinking
of rewriting dpkg. We promised to release the design specification
when we felt it had reaced a reasonably stable state to satisfy
all the curiosity and get your input. That time has now arrived!

Attached to this mail is the text-version of the spec. You can also
browse it online at http://www.debian.org/~bcollins/dpkg2/ .

I'ld like to stress a few points first:
* we do NOT intend to force people to switch to dpkgv2 at any moment.
* I'm not doing this as DPL, merely because I think it's a fun project

At this point we're starting to work on a developers guide that will
contain more detailed information about the various parts of dpkgv2
that are useful for people implementing dpkgv2, who want to write
modules for it, or are just interested in how it works.

If you have any comments please don't hestitate to tell us at
dpkg2@lists.wiggy.net . Oh, and we'll read debian-devel as well,
but I don't want to increase the traffic there too much.

Finally, we want a new name for the project. Unfortunately all
our imagination is spent in improving dpkgv2, so suggestions
are welcome..

On behalve of the dpkgv2,
  Wichert.


                DPKGv2 Design and Technical Specifications
                ------------------------------------------

                     Ben Collins <bcollins@debian.org>

                   Wichert Akkerman <wichert@linux.com>

       $Id: dpkg2-spec.sgml,v 1.26 1999/07/16 16:41:49 wichert Exp $


-------------------------------------------------------------------------------


Abstract
--------

     This document contains a design specification for DPKGv2. This
     specification is currently in it's earliest form and is by no means
     complete. It is intended as an outline of the DPKGv2 project and gives
     insight to the goals, reasons, and people behind the project, as well
     as some technical details of the software. For more detailed,
     developer oriented documentation, please refer to the _DPKGv2
     Developers Guide_.


Copyright Notice
----------------

     Copyright © 1999 Ben Collins

     This manual is free software; you may redistribute it and/or modify it
     under the terms of the GNU General Public License as published by the
     Free Software Foundation; either version 2, or (at your option) any
     later version.


-------------------------------------------------------------------------------


Contents
--------

     1.        Note from the Authors

     2.        Design Goals
     2.1.      Modular Structure (Basic Overview)
     2.2.      Coding Practice
     2.3.      A Seperation of Ways (dpkg, dselect and build tools)
     2.4.      Configuration Management
     2.5.      Frontend Interfaces
     2.6.      A Note on Debian Policy

     3.        Implementation
     3.1.      Source Layout
     3.2.      Build Tools
     3.3.      Library and Modules

     4.        Technical
     4.1.      Library Usage (frontend development)
     4.2.      Module Interfaces

     5.        Example frontend (dpkg binary internals)

     A.        Figures
     A.1.      Libraries and Modules


-------------------------------------------------------------------------------


1. Note from the Authors
------------------------

     Currently the DPKGv2 project is in it's infancy. As such, this spec is
     subject to considerable changes prior to completion of the project. As
     the project nears a stable release, this spec will evolve into a
     complete white paper of the DPKGv2 API.

     What DPKGv2 hopes to accomplish is revolutionize Debian's packaging
     system just as much as the original dpkg. Over time we have come to
     find we need more than what the current dpkg system has or can offer.
     Building on or trying to incorporate these changes into the existing
     source was considered extremely daunting to say the least. While Dpkg
     is a great achievement functionally, it's source code is
     unapproachable by most programmers. With all the features that users
     and developers are asking for in dpkg, it is better that it be handled
     from scratch in order to have certain design goals in mind while
     developing it.

     Since the free software community is at a crossroads at this point,
     this project will also aim to tackle the diverse package formats that
     have been developed over time. This is not to say that there will ever
     be only one format, but there should be a central way to access all
     the formats available as well as allow new ones to be incorporated
     easily, irregardless of the underlying system (or distribution). The
     design scope of this project should instill cohesiveness by utilizing
     common frontends across all formats through this method.

     In the next few chapters you will get a feel for the basic philosophy
     behind the goals and technical workings of this project.

     We look forward to your comments.

     Respectfully, The DPKGv2 Developers


-------------------------------------------------------------------------------


2. Design Goals
---------------

     This chapter overviews the general design goals of DPKGv2. Remember
     that specific functionality is not as important here as the underlying
     structure of the API. The design includes the ability to extend the
     capabilities of the library without regard to the frontend programs
     used by the users. It also includes the basic need for functionality
     being seperated into components in order to allow programs not dealing
     directly with package management, to use and interact with specific
     parts of the package system. Abstraction of the core functions will be
     a must.


2.1. Modular Structure (Basic Overview)
---------------------------------------

     DPKGv2's initial design concentrates on splitting the core functions
     of a package manager into several easily manageable and well defined
     subsystems. All of these subsystems will be accesible via the core
     library. The library obtains it's functionality through modules (see
     Section A.1, `Libraries and Modules').

2.1.1. Core Libraries
---------------------

     The core library gives the frontend programs access to all of the
     available package management functions, independant of the frontend's
     interface with the user. For example, the library doesn't care if it
     is a gtk, qt, curses, or command line based frontend, the API allows
     the program to make use of what ever UI it decides on. The core
     library is also unaware of specific package formats and other details
     handled by the modules.

     By making the library API independant of the specific package
     implementations, the frontend can be used for any combination of
     module setup. This simplifies frontend development and leaves the
     majority of the work up to the library and modules.

2.1.2. Core Modules
-------------------

     The modules work behind the scenes, giving the library it's intended
     functionality. This allows for the addition of features post-install,
     and by third-party developers, thus adding capabilities without
     requiring a recompile of the core library or frontend programs. Each
     module type will have a well defined API to ensure that frontend
     programs need no knowledge of the capabilties supplied by them. This
     will also allow for all frontend programs to benefit from module
     additions and centralize the structure of package management, giving
     way to locally customized systems, alternative archive formats, and
     endless expansion, now and in the future.


2.2. Coding Practice
--------------------

     The success or failure of a project such as this, depends heavily on
     the approachability of the source to developers outside of the core
     group. Making it easy for others to debug, test and develop for the
     project is the only way to be successful in this area. For this
     reason, practical coding will be adhered to.

2.2.1. Code Management
----------------------

     Basic code management will be instilled from the project inception.
     Source code will be kept in a CVS repository (anonymous access will be
     available once there is a sizeable code base). All core developers
     will have read/write access to the repository and patches from outside
     the core group will be accepted as long as they meet the requirements
     of coding style and stability. Generally speaking, third-party modules
     will not be accepted into the main source tree. An alternative source
     tree for third party modules may be maintained later in the project's
     life.

2.2.2. Cross Platform Compatibility
-----------------------------------

     When at all possible, the library and core modules will support as
     many of the standard platforms (yes, this even means proprietary
     platforms). Also, coding practices will be aimed at using standard
     (POSIX and ANSI compliant) functions from the C library. The more
     places this can run, the better chance of it being used and accepted
     outside of the Debian project. The core library will also not depend
     on anything other than the standard C library and will _not_ use
     external programs to perform functions. This reduces the overhead of
     external dependencies which could break the library inadvertently and
     reduces the effort in porting to new platforms.

2.2.3. DPKGv1 Compability
-------------------------

     DPKGv2's initial goal is to supply DPKGv1 compatibility through a base
     set of modules. It is this basic functionality that will be present in
     the main source tree. Once this is achieved, it is expected that a new
     package format will be derived and allow for simultaneous use of both
     formats. It is also expected that new features will be easily added to
     the current format during the developement phase and transistion to a
     new format.

     The old package format will be developed in the mod_deb2 module. A
     mod_deb3 module has been set aside for development of the new package
     format.

2.2.4. Multilingual Support
---------------------------

     DPKGv2 will utilize gettext for supporting locales other than the
     default of english.


2.3. A Seperation of Ways (dpkg, dselect and build tools)
---------------------------------------------------------

     Currently DPKGv1 focuses on the dpkg binary. Although there is a
     libdpkg, most programs exec dpkg since the API for libdpkg is not well
     defined nor documented. For a package manager to be truely open and
     extendable, a central library must be available for programs to use.
     This prevents duplication of code and incompatibilties between
     different implementations. In DPKGv2, the dpkg binary is not the
     central focus of the development. The dpkg program developed with the
     DPKGv2 project is mainly for transition and to serve as an example for
     frontend development with the library. It is expected that better
     frontends will be developed seperately from this project using the
     DPKGv2 library (curses, GTK, KDE, etc.).

     For this same reason, the dselect program will not be included with
     the core source, since it should be replaced by a better interface.
     This will help concentrate the projects efforts onto the real code
     that handles package and frontend interaction.

     Also, since this project is all about package management for systems,
     it is not within it's design scope to develop tools for building
     packages. It is the opinion of the DPKGv2 developers that package
     building should remain in it's own seperate group of programs so that
     local practices and distribution specific policies can be kept
     seperate from general installation and management.


2.4. Configuration Management
-----------------------------

     Yes, DPKGv2 hopes to incorporate internal configuration functions.
     This coincides with the debian-admintool effort. Since packages and
     configuration are so closely tied together, the merger will benefit
     both sides. Configuration management will be developed in the core
     library to be used in conjunction with the other functionality and
     should prove an integral part of future advances in packaging by
     allowing for non-interactive installs, centralized repositories for
     data, network administration and more.

     Several key things will be involved in the config functions. First, it
     will handle all of the persistent data storage for the package system
     (db modules). Data storage will be configurable with modules (ie, db2,
     plain, LDAP...). It will also evolve into a network management
     interface, which could allow for remote frontend interfaces.

     Some key features include the ability to defer obtaining configuration
     information for packages (package format will need to enable this) as
     well as post-installation configuration. Data stored in the system
     will be accessible between packages and programs. The library will
     also enable the development of programs that solely handle system
     configuration. The rest of the package system (including modules) will
     be able to store internal configuration data via this functionality as
     well.


2.5. Frontend Interfaces
------------------------

     DPKGv2's source tree will only come with a `dpkg' compatible binary
     (command line) program. All other frontends will need to be developed
     seperately, but should not be affected by changes in the libraries
     (once the API stabalizes), nor additions/changes in the modules. Once
     the frontends complete the skeletal framework for utilizing the API,
     the rest is left up to the libraries.


2.6. A Note on Debian Policy
----------------------------

     Since DPKGv2's design is not geared toward a package format, it is not
     expected to affect or be affected by policy. The only policy changes
     forseen would be an official implementation of DPKGv2 as the standard
     package manager for Debian, which is not going to be in the near
     future. Extensive testing as an alternative package manager will be
     needed to ensure the stability that is required to support the
     distribution.


-------------------------------------------------------------------------------


3. Implementation
-----------------


3.1. Source Layout
------------------

     The base of the source tree has 5 major branches with several
     subdirectories. The main branches are for the dpkg binary (dpkg/),
     build includes, the libdpkg2 library, the modules (which contains a
     subdir for each core module) and the documentation directory. Other
     subdirectories for utility sources and NLS support are present.


3.2. Build Tools
----------------

     DPKGv2's sources use the GNU autoconf and autoheader tools for
     maintaining the configure and Makefile's. This adds immediate ability
     for cross-platform development. In conjunction with this, the libtool
     suite has been implemented for building object files and linking the
     library and binaries.

     Since the entire source will be written C, we will be using GNU gcc
     for most of the development, but any C compiler should work for this
     purpose. The core library also makes use of the libltdl library
     provided with libtool as a dlopen wrapper and is included with the
     main source and statically linked with the core library. This provides
     a simple way of supporting many platforms with modules while keeping
     the source fairly clean and readable.

     GNU's gettext library is included in the source for systems that don't
     have it directly installed on the system.


3.3. Library and Modules
------------------------

     The core library will be virtually self-standing except for being
     linked with the C library (this includes libdl for the dlopen
     functions where applicable). The versioning scheme for the library
     will follow libtool's suggested syntax of current.revision.age. Here
     is an excerpt from the libtool documentation explaining this
     versioning:

   current
          The most recent interface number that this library implements.
   revision
          The implementation number of the current interface.
   age
          The difference between the newest and oldest interfaces that
          this library implements. In other words, the library implements
          all the interface numbers in the range from number current -
          age to current.


-------------------------------------------------------------------------------


4. Technical
------------

     (this is where we get to the nitty gritty...somewhat, save details for
     dev guide)


4.1. Library Usage (frontend development)
-----------------------------------------

     The core library is linked to the frontends and provides total control
     of the package system. (need details on the API)


4.2. Module Interfaces
----------------------

     There are currently 5 planned module interfaces which give the
     libraries their functionality. (still need to add details to each one)

4.2.1. Package Formats
----------------------

     dpkg_pkg_*.so -> Used to handle a single package format. These are
     functions for packing, unpacking and configuring packages.

4.2.2. Hooks
------------

     dpkg_hook_*.so -> This class of modules is used in seperates stages in
     the installation and removing of a package. To be exact: before and
     after installation and removal of a package, and just before
     installing or removing a single file/directory from a package. The
     module can decide if we should a) proceed normally, b) skip this file,
     c) use a different filename, or d) abort with an error.

4.2.3. Logging
--------------

     dpkg_log_*.so -> Used to supply logging facilities.

4.2.4. Data Storage
-------------------

     dpkg_db_*.so -> Used to store configuration info as well as persistent
     package status used by the archive modules.


-------------------------------------------------------------------------------


5. Example frontend (dpkg binary internals)
-------------------------------------------

     (here's where we break down the dpkg program provided by DPKGv2 and
     how it uses the libraries in a somewhat non-technical way)


-------------------------------------------------------------------------------


A. Figures
----------


A.1. Libraries and Modules
--------------------------

              +-----------------------+   )
              |    User Interface     |   |
              +---------------------x-+   |
              |                     | |   |- DPKGv2 or 3rd-party Frontend
              |      Application    | |   |
              |                     | |   |
              +-x---x---x---x---x---|-+   )
                |   |   |   |   |   |--------) User Interface functions
              +-x---x---x---x---x-+-x-+   )
              |                       |   |
              |        Library        |   |- Core DPKGv2 Library
              |                       |   |
              +---+---+---+---+---+---+   )
                   ||| ||| ||| |||
                   ooo ooo ooo ooo        )- DPKGv2 or 3rd-party, Modules


-------------------------------------------------------------------------------


     DPKGv2 Design and Technical Specifications

     Ben Collins <bcollins@debian.org>
     Wichert Akkerman <wichert@linux.com>

     $Id: dpkg2-spec.sgml,v 1.26 1999/07/16 16:41:49 wichert Exp $

Attachment: pgp72FNTnf8PU.pgp
Description: PGP signature


Reply to: