[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Design Documents


	Sorry for the delay, guys, but things kinda blew up in
 australia and france simultaneously, and I was on the phone at all
 kinds of strange hours. Add to it that I was sure murphy's law would
 make me take a trip right when I'm planning to move into my new house ;-).

	Anyway, version 0.04 is on it's way. I've expanded on the
 first two sections, I think I'll leave the disgram as it is, and just
 add explanations as text.

	I need help fleshing out section 3; I think if the people
 coding the modules contribute a few paragraphs per module we may meet
 Jason's weekend deadline yet.

	Again, please point out errors and omissions.

	I think when I'm through this, I'll try putting together an
 topological sort template. Jason, how are you doing on the diagram of
 the backing store (err, cache)? I am wondering if I should instead
 just write a topological sort that uses the backing store data (it
 may require an additional linked list or two threading though the

	I think I'll wait for a day or so before I convert this
 document into SGML and check it in.


 How many programmers does it take to change a light bulb? None.  It's
 a hardware problem.
Manoj Srivastava               <url:mailto:srivasta@acm.org>
Mobile, Alabama USA            <url:http://www.datasync.com/%7Esrivasta/>

        The Deity project design document Version 0.04
        === ===== ======= ====== ======== ======= ====

0. Introduction

 Deity is supposed to be a replacement for dselect, and not a
 replacement for dpkg. However, since addition functionality has been
 required for deity, and given the fact that this is very closely
 related to dpkg, it is not unreasonable to expect that additional
 functionality in the underlying dpkg would also be requested.

 Diety/dselect are the first introduction that people have to Debian,
 and unfortunately this first impression contributes greatly to the
 public perception of the distribution. It is imperative that this be
 a showcase for Debian, rather than frighten novices away (which has
 been an accusation often levelled at the current system).

1. Requirements
  a) Deity should be a replacement for dselect. Therefore it should
     have all the functionality that dselect has currently. This is
     the primary means of interaction between the user and the package 
     management system, and it should be able to handle all tasks
     involved in installing, upgrading, and routine management without 
     having the users take recourse to the underlying management

  b) It should be easier to use and less confusing for novice
     users. The primary stimulus for the creation of Deity was the
     perceived intractability,  complexity, and non-intuitive
     behavior of the existing user interface, and as such, human
     factors must be a primary mandate of Deity.

  c) It should be able to group packages more flexibly, and possibly
     allow operations based on a group. One should be able to select,
     or deselect, a coherent group of related packages
     simultaneously, allowing one to add, remove, or upgrade
     functionality to a machine as one step.

  d) This would allow Deity to handle "standard installations",
     namely, one could then install a set of packages to enable a
     machine to fulfill specific tasks.  Define a few standard  
     installations, and which packages are included therein. The
     packages should be internally consistent.

  e) Make use of a keywords field in package headers; provide a
     standard list of keywords for people to use. This could be the
     underpinning to allow the previous two requirements to work
     (though the developers are not constrained to implement the
      previous requirements using keywords)

  f) Use dependencies, conflicts, and reverse dependencies to properly
     order packages for installation and removal. This has been a
     complaint in the past that the installation methods do not really 
     understand dependencies, causing the upgrade process to break, or 
     allowing the removal of packages that left the system in an
     untenable state by breaking the dependencies on packages that
     were dependent on the package being removed. A special emhasis is 
     placed on handling pre-dependencies correctly; the target of a
     predependency has to be fully configured before attempting to
     install the pre-dependent package. Also, "configure immediately"
     requests mentioned below should be handled.

  g) Handle replacement of a package providing a virtual package with
     another (for example, it has been very difficult replacing
     sendmail with smail, or vice versa), making sure that the 
     dependencies are still satisfied. 

  h) Handle source lists for updates from multiple sources. Deity
     should also be able to handle diverse methods of acquiring new
     packages; local filesystem, mountable CD-ROM drives, FTP
     accesible repositories are some of the methods that come to mind.
     Also, the source lists can be separated into categories, such as
     main, contrib, non-us, non-local, non-free, my-very-own,
     etc. Deity should be set up to retrive the Packages files from
     these multiple source lists, as well as retrieving the packages

  i) Handle base of source and acquire all Packages files underneath.
     (possibly select based on architecture), this should be a simple
     extension of the previous requirement.

  j) Handle remote installation (to be implemented maybe in a future
     version, it still needs to be designed). This would ease the
     burden of maintaining multiple Debian machines on a site. In the
     authors opinion this is a killer difference for the distribution,
     though it may be too hard a problem to be implemented with the
     initial version of Deity. However, some thought must be given to
     this to enable Deity to retain hooks for future functionality, or 
     at least to refrain from methods that may preclude remote
     activity. It is desirable that adding remote installation not
     require a redesign of Deity from the ground up.

  k) Be scalable. Dselect worked a lot better with 400 packages, but
     at last count the number of packages was around twelve hundred
     and climbing. This also requires Deity to pay attention to the
     needs of small machines which are low on memory (though this
     requirement shall diminish as we move towards bigger machines, it 
     would still be nice if Debian worked on all old machines where
     Linux itself would work).

  l) Handle install immediately requests. Some packages, like
     watchdog, are required to be working for the stability of the
     machine itself. There are others which may be required for the
     correct functioning of a production machine, or which are mission 
     critical applications. Deity should, in these cases, upgrade the
     packages with minimal downtime; allowing these packages to be one 
     of potentially hundreds of packages being upgraded concurrently
     may not satisfy the requirements of the package or the
     site. (Watchdog, for example, if not restarted quickly, may cause 
     the machine to reboot in the midst of installation, which may
     cause havoc on the machine)

2. Procedural description
  a) Set Options: this process handles setting of user or site
     options, and configuration of all aspects of Deity. It allows the 
     user to set the location and order of package sources, allowing
     them to set up source list details, like ftp site locations,
     passwords, etc. Display options may also be set.

  b) Updates: build a list of available packages, using source lists
     or a base location and trawling for Packages files (needs to be
     aware of architecture). This may involve finding and retrieving
     Packages files, storing them locally for efficiency, and parsing
     the data for later use. This would entail contacting various
     underlying access modules (ftp, cdrom mounts, etc) Use a backing
     store for speed. This may also require downloading the actual
     package files locally for speed.

  c) Local status: Build up a list of packages already installed. This 
     requires reading and writing the local?? status file. For remote
     installation, this should probably use similar mechanisms as the
     Packages file retrieval does. Use the backing store for
     speed. One should consider multiple backing stores, one for each

  d) Determine forward and reverse dependencies. All known dependency
     fields should be acted upon, since it is fairly cheap to do
     so. Update the backing store with this information.

  e) Selection: Present the data to the user. Look at Behan Webster's
     documentation for the user interface procedures. (Note: In the
     authors opinion deletions and reverse dependencies should also be
     presented to the user, in a strictly symmetric fashion; this may
     make it easier to prevent a package being removed that breaks

  f) Installation/configuration etc.
     Build a list of events. Simple topological sorting gives order of
     packages in dependency order. At certain points in this ordering,
     predependencies/immediate configure directives cause an break in
     normal ordering. We need to insert the uninstall/purge directive
     in the stream (default: as early as possible).

  g) Take the order of installations and removals and build up a
     stream of events to send to the packaging system (dpkg). Execute
     the list of events if succesful. Do not partially install 
     packages and leave system in broken state. Go to step 2e as

3. Modules and interfaces

 a) The user interface module: Look at Behan Webster's documentation.

 b) widget set, related closely to above
    Could some one prewsent design decisions of the widget set here?

 c) Update Module
    Distinct versions of the same package are recorded separately, but 
    if multiple Packages files contain the same version of a package,
    then only the forst one is recorded. For this reason, the least
    expensive update source should be listed first (local file system
    is better than a remote ftp site)

     i) FTP methods
    ii) mount and file traversal module(s)?
   iii) Other methods ???

 d) Status file parser/generator.

 e) Package file parser/generator (related to above). Handle multiple
    Packages files, from different sources. Each package contains a
    link back to the packages file structure that contains details about
    the origin of the data. 

 f) Dependency module
    i) dependency/conflict determination and linking
   ii) reverse dependency generator. Maybe merged with above
  iii) package ordering/event generation module

 g) module to interact with dpkg

4. Data flow and conversions analysis.
                                                       __\|ftp modules|
                                                      /  /|___________|
                                    _ ____________   /     ________________
                                    |    update   | /     |mount/local file|
        |==========================>|   module    |/_____\|  traversals    |
        |                           |_____________|      /|________________|
        |                             ^        ^
        |                             |        |               ______________
  ______|_______    _ _____ ______    |   _____v________      \|            |
 |Configuration |   |configuration|   |   |Packages Files|  ===|Status file |
 |  module      |<=>|    data     |   |   |______________| /  /|____________|
 |______________|   |_____________|   |        ^          /
         ^                            |        |         /
         |                            | _______v_______|/_
         |                            | |              |    ________________
         |                            | |              |/_\|   Dependency  |
         |                            | |backing store |\ /|     Module    |
         |                            | |______________|  _|_______________|
         |                             \       ^          /|       ^
         |                              \      |         /         |
         |                              _\|____v_______|/__    ____v_______
         |_____________________________\| User interaction|    |    dpkg   |
                                       /|_________________|<==>|  Invoker  |

 dpkg also interacts with status and available files.

Reply to: