Taking time off
Hi guys,
I am discovering that aquiring a new house, and having some
remodeling done, and moving into it, are taking way more time and
energy than I anticiated. To top it off, I seem to be coming down
with the flu that my wife got at work. Moving day is the 17th, I'll
probably have decreasing amount of time for deity till then.
I've been trying to work on the design document, but have
gotten significantly less done on it than I wanted. I'm attaching the
latest version here, and would gladly accept patches.
manoj
--
What hath Bob wrought?
Manoj Srivastava <url:mailto:srivasta@acm.org>
Mobile, Alabama USA <url:http://www.datasync.com/%7Esrivasta/>
The Deity project design document Version 0.05
=== ===== ======= ====== ======== ======= ====
0. Introduction
Deity is supposed to be a replacement for dselect, and not a
replacement for dpkg. However, since addition functionality has been
required for deity, and given the fact that this is very closely
related to dpkg, it is not unreasonable to expect that additional
functionality in the underlying dpkg would also be requested.
Diety/dselect are the first introduction that people have to Debian,
and unfortunately this first impression contributes greatly to the
public perception of the distribution. It is imperative that this be
a showcase for Debian, rather than frighten novices away (which has
been an accusation often levelled at the current system).
1. Requirements
a) Deity should be a replacement for dselect. Therefore it should
have all the functionality that dselect has currently. This is
the primary means of interaction between the user and the package
management system, and it should be able to handle all tasks
involved in installing, upgrading, and routine management without
having the users take recourse to the underlying management
system.
b) It should be easier to use and less confusing for novice
users. The primary stimulus for the creation of Deity was the
perceived intractability, complexity, and non-intuitive
behavior of the existing user interface, and as such, human
factors must be a primary mandate of Deity.
c) It should be able to group packages more flexibly, and possibly
allow operations based on a group. One should be able to select,
or deselect, a coherent group of related packages
simultaneously, allowing one to add, remove, or upgrade
functionality to a machine as one step.
d) This would allow Deity to handle "standard installations",
namely, one could then install a set of packages to enable a
machine to fulfill specific tasks. Define a few standard
installations, and which packages are included therein. The
packages should be internally consistent.
e) Make use of a keywords field in package headers; provide a
standard list of keywords for people to use. This could be the
underpinning to allow the previous two requirements to work
(though the developers are not constrained to implement the
previous requirements using keywords)
f) Use dependencies, conflicts, and reverse dependencies to properly
order packages for installation and removal. This has been a
complaint in the past that the installation methods do not really
understand dependencies, causing the upgrade process to break, or
allowing the removal of packages that left the system in an
untenable state by breaking the dependencies on packages that
were dependent on the package being removed. A special emhasis is
placed on handling pre-dependencies correctly; the target of a
predependency has to be fully configured before attempting to
install the pre-dependent package. Also, "configure immediately"
requests mentioned below should be handled.
g) Handle replacement of a package providing a virtual package with
another (for example, it has been very difficult replacing
sendmail with smail, or vice versa), making sure that the
dependencies are still satisfied.
h) Handle source lists for updates from multiple sources. Deity
should also be able to handle diverse methods of acquiring new
packages; local filesystem, mountable CD-ROM drives, FTP
accesible repositories are some of the methods that come to mind.
Also, the source lists can be separated into categories, such as
main, contrib, non-us, non-local, non-free, my-very-own,
etc. Deity should be set up to retrive the Packages files from
these multiple source lists, as well as retrieving the packages
themselves.
i) Handle base of source and acquire all Packages files underneath.
(possibly select based on architecture), this should be a simple
extension of the previous requirement.
j) Handle remote installation (to be implemented maybe in a future
version, it still needs to be designed). This would ease the
burden of maintaining multiple Debian machines on a site. In the
authors opinion this is a killer difference for the distribution,
though it may be too hard a problem to be implemented with the
initial version of Deity. However, some thought must be given to
this to enable Deity to retain hooks for future functionality, or
at least to refrain from methods that may preclude remote
activity. It is desirable that adding remote installation not
require a redesign of Deity from the ground up.
k) Be scalable. Dselect worked a lot better with 400 packages, but
at last count the number of packages was around twelve hundred
and climbing. This also requires Deity to pay attention to the
needs of small machines which are low on memory (though this
requirement shall diminish as we move towards bigger machines, it
would still be nice if Debian worked on all old machines where
Linux itself would work).
l) Handle install immediately requests. Some packages, like
watchdog, are required to be working for the stability of the
machine itself. There are others which may be required for the
correct functioning of a production machine, or which are mission
critical applications. Deity should, in these cases, upgrade the
packages with minimal downtime; allowing these packages to be one
of potentially hundreds of packages being upgraded concurrently
may not satisfy the requirements of the package or the
site. (Watchdog, for example, if not restarted quickly, may cause
the machine to reboot in the midst of installation, which may
cause havoc on the machine)
k) Incorporate time stamps into the meta data maintained about the
package, so that an audit trail can be construvted about each
package. This means at least time stamping installations,
upgrades, and deletions, it may also extend to having an record
of the last N upgrades of any package (installation time-version
pairs). This would add to the status file format.
2. Procedural description
a) Set Options: this process handles setting of user or site
options, and configuration of all aspects of Deity. It allows the
user to set the location and order of package sources, allowing
them to set up source list details, like ftp site locations,
passwords, etc. Display options may also be set.
b) Updates: build a list of available packages, using source lists
or a base location and trawling for Packages files (needs to be
aware of architecture). This may involve finding and retrieving
Packages files, storing them locally for efficiency, and parsing
the data for later use. This would entail contacting various
underlying access modules (ftp, cdrom mounts, etc) Use a backing
store for speed. This may also require downloading the actual
package files locally for speed.
c) Local status: Build up a list of packages already installed. This
requires reading and writing the local?? status file. For remote
installation, this should probably use similar mechanisms as the
Packages file retrieval does. Use the backing store for
speed. One should consider multiple backing stores, one for each
machine.
d) Determine forward and reverse dependencies. All known dependency
fields should be acted upon, since it is fairly cheap to do
so. Update the backing store with this information.
e) Selection: Present the data to the user. Look at Behan Webster's
documentation for the user interface procedures. (Note: In the
authors opinion deletions and reverse dependencies should also be
presented to the user, in a strictly symmetric fashion; this may
make it easier to prevent a package being removed that breaks
dependencies)
f) Installation/configuration etc.
Build a list of events. Simple topological sorting gives order of
packages in dependency order. At certain points in this ordering,
predependencies/immediate configure directives cause an break in
normal ordering. We need to insert the uninstall/purge directive
in the stream (default: as early as possible).
g) Take the order of installations and removals and build up a
stream of events to send to the packaging system (dpkg). Execute
the list of events if succesful. Do not partially install
packages and leave system in broken state. Go to step 2e as
needed.
3. Modules and interfaces
a) The user interface module: Look at Behan Webster's documentation.
b) widget set, related closely to above
Could some one prewsent design decisions of the widget set here?
c) Update Module
Distinct versions of the same package are recorded separately, but
if multiple Packages files contain the same version of a package,
then only the forst one is recorded. For this reason, the least
expensive update source should be listed first (local file system
is better than a remote ftp site)
This module should interact with the user interface module to set
and change configuration parameters for the modules listed
below. It needs to record that information in an on disk data
file, to be read on future invocations.
i) FTP methods
ii) mount and file traversal module(s)?
iii) Other methods ???
d) Status file parser/generator. The status file records the current
state of the system, listing the packages installed, etc. The
status file is also one method of communicating with dpkg, since
it is perfectly permissible for the user to use deity to request
packages be updated, put others on hold, mark other for removal,
etc, and then run dpkg -BORGiE on a file system.
e) Package file parser/generator (related to above). Handle multiple
Packages files, from different sources. Each package contains a
link back to the packages file structure that contains details about
the origin of the data.
f) Dependency module
i) dependency/conflict determination and linking
ii) reverse dependency generator. Maybe merged with above
iii) package ordering/event generation module
g) module to interact with dpkg
4. Data flow and conversions analysis.
____________
__\|ftp modules|
/ /|___________|
_ ____________ / ________________
| update | / |mount/local file|
|==========================>| module |/_____\| traversals |
| |_____________| /|________________|
| ^ ^
| | | ______________
______|_______ _ _____ ______ | _____v________ \| |
|Configuration | |configuration| | |Packages Files| ===|Status file |
| module |<=>| data | | |______________| / /|____________|
|______________| |_____________| | ^ /
^ | | /
| | _______v_______|/_
| | | | ________________
| | | |/_\| Dependency |
| | |backing store |\ /| Module |
| | |______________| _|_______________|
| \ ^ /| ^
| \ | / |
| _\|____v_______|/__ ____v_______
|_____________________________\| User interaction| | dpkg |
/|_________________|<==>| Invoker |
|___________|
dpkg also interacts with status and available files.
The backing store and the associated data structures are the
core of deity. All modules essentially revolve around the backing
store, feeding it data, adding and manipulating links and
relationships between data in the backing store, allowing the user to
interact with and modify the data in the backing store, and finally
writing it out as the status file and possibly issuing directives to
dpkg.
The other focal point for deity is the user interface,
Reply to: