Re: Design Documents
Hi,
Sorry for the delay, guys, but things kinda blew up in
australia and france simultaneously, and I was on the phone at all
kinds of strange hours. Add to it that I was sure murphy's law would
make me take a trip right when I'm planning to move into my new house ;-).
Anyway, version 0.04 is on it's way. I've expanded on the
first two sections, I think I'll leave the disgram as it is, and just
add explanations as text.
I need help fleshing out section 3; I think if the people
coding the modules contribute a few paragraphs per module we may meet
Jason's weekend deadline yet.
Again, please point out errors and omissions.
I think when I'm through this, I'll try putting together an
topological sort template. Jason, how are you doing on the diagram of
the backing store (err, cache)? I am wondering if I should instead
just write a topological sort that uses the backing store data (it
may require an additional linked list or two threading though the
data).
I think I'll wait for a day or so before I convert this
document into SGML and check it in.
manoj
--
How many programmers does it take to change a light bulb? None. It's
a hardware problem.
Manoj Srivastava <url:mailto:srivasta@acm.org>
Mobile, Alabama USA <url:http://www.datasync.com/%7Esrivasta/>
The Deity project design document Version 0.04
=== ===== ======= ====== ======== ======= ====
0. Introduction
Deity is supposed to be a replacement for dselect, and not a
replacement for dpkg. However, since addition functionality has been
required for deity, and given the fact that this is very closely
related to dpkg, it is not unreasonable to expect that additional
functionality in the underlying dpkg would also be requested.
Diety/dselect are the first introduction that people have to Debian,
and unfortunately this first impression contributes greatly to the
public perception of the distribution. It is imperative that this be
a showcase for Debian, rather than frighten novices away (which has
been an accusation often levelled at the current system).
1. Requirements
a) Deity should be a replacement for dselect. Therefore it should
have all the functionality that dselect has currently. This is
the primary means of interaction between the user and the package
management system, and it should be able to handle all tasks
involved in installing, upgrading, and routine management without
having the users take recourse to the underlying management
system.
b) It should be easier to use and less confusing for novice
users. The primary stimulus for the creation of Deity was the
perceived intractability, complexity, and non-intuitive
behavior of the existing user interface, and as such, human
factors must be a primary mandate of Deity.
c) It should be able to group packages more flexibly, and possibly
allow operations based on a group. One should be able to select,
or deselect, a coherent group of related packages
simultaneously, allowing one to add, remove, or upgrade
functionality to a machine as one step.
d) This would allow Deity to handle "standard installations",
namely, one could then install a set of packages to enable a
machine to fulfill specific tasks. Define a few standard
installations, and which packages are included therein. The
packages should be internally consistent.
e) Make use of a keywords field in package headers; provide a
standard list of keywords for people to use. This could be the
underpinning to allow the previous two requirements to work
(though the developers are not constrained to implement the
previous requirements using keywords)
f) Use dependencies, conflicts, and reverse dependencies to properly
order packages for installation and removal. This has been a
complaint in the past that the installation methods do not really
understand dependencies, causing the upgrade process to break, or
allowing the removal of packages that left the system in an
untenable state by breaking the dependencies on packages that
were dependent on the package being removed. A special emhasis is
placed on handling pre-dependencies correctly; the target of a
predependency has to be fully configured before attempting to
install the pre-dependent package. Also, "configure immediately"
requests mentioned below should be handled.
g) Handle replacement of a package providing a virtual package with
another (for example, it has been very difficult replacing
sendmail with smail, or vice versa), making sure that the
dependencies are still satisfied.
h) Handle source lists for updates from multiple sources. Deity
should also be able to handle diverse methods of acquiring new
packages; local filesystem, mountable CD-ROM drives, FTP
accesible repositories are some of the methods that come to mind.
Also, the source lists can be separated into categories, such as
main, contrib, non-us, non-local, non-free, my-very-own,
etc. Deity should be set up to retrive the Packages files from
these multiple source lists, as well as retrieving the packages
themselves.
i) Handle base of source and acquire all Packages files underneath.
(possibly select based on architecture), this should be a simple
extension of the previous requirement.
j) Handle remote installation (to be implemented maybe in a future
version, it still needs to be designed). This would ease the
burden of maintaining multiple Debian machines on a site. In the
authors opinion this is a killer difference for the distribution,
though it may be too hard a problem to be implemented with the
initial version of Deity. However, some thought must be given to
this to enable Deity to retain hooks for future functionality, or
at least to refrain from methods that may preclude remote
activity. It is desirable that adding remote installation not
require a redesign of Deity from the ground up.
k) Be scalable. Dselect worked a lot better with 400 packages, but
at last count the number of packages was around twelve hundred
and climbing. This also requires Deity to pay attention to the
needs of small machines which are low on memory (though this
requirement shall diminish as we move towards bigger machines, it
would still be nice if Debian worked on all old machines where
Linux itself would work).
l) Handle install immediately requests. Some packages, like
watchdog, are required to be working for the stability of the
machine itself. There are others which may be required for the
correct functioning of a production machine, or which are mission
critical applications. Deity should, in these cases, upgrade the
packages with minimal downtime; allowing these packages to be one
of potentially hundreds of packages being upgraded concurrently
may not satisfy the requirements of the package or the
site. (Watchdog, for example, if not restarted quickly, may cause
the machine to reboot in the midst of installation, which may
cause havoc on the machine)
2. Procedural description
a) Set Options: this process handles setting of user or site
options, and configuration of all aspects of Deity. It allows the
user to set the location and order of package sources, allowing
them to set up source list details, like ftp site locations,
passwords, etc. Display options may also be set.
b) Updates: build a list of available packages, using source lists
or a base location and trawling for Packages files (needs to be
aware of architecture). This may involve finding and retrieving
Packages files, storing them locally for efficiency, and parsing
the data for later use. This would entail contacting various
underlying access modules (ftp, cdrom mounts, etc) Use a backing
store for speed. This may also require downloading the actual
package files locally for speed.
c) Local status: Build up a list of packages already installed. This
requires reading and writing the local?? status file. For remote
installation, this should probably use similar mechanisms as the
Packages file retrieval does. Use the backing store for
speed. One should consider multiple backing stores, one for each
machine.
d) Determine forward and reverse dependencies. All known dependency
fields should be acted upon, since it is fairly cheap to do
so. Update the backing store with this information.
e) Selection: Present the data to the user. Look at Behan Webster's
documentation for the user interface procedures. (Note: In the
authors opinion deletions and reverse dependencies should also be
presented to the user, in a strictly symmetric fashion; this may
make it easier to prevent a package being removed that breaks
dependencies)
f) Installation/configuration etc.
Build a list of events. Simple topological sorting gives order of
packages in dependency order. At certain points in this ordering,
predependencies/immediate configure directives cause an break in
normal ordering. We need to insert the uninstall/purge directive
in the stream (default: as early as possible).
g) Take the order of installations and removals and build up a
stream of events to send to the packaging system (dpkg). Execute
the list of events if succesful. Do not partially install
packages and leave system in broken state. Go to step 2e as
needed.
3. Modules and interfaces
a) The user interface module: Look at Behan Webster's documentation.
b) widget set, related closely to above
Could some one prewsent design decisions of the widget set here?
c) Update Module
Distinct versions of the same package are recorded separately, but
if multiple Packages files contain the same version of a package,
then only the forst one is recorded. For this reason, the least
expensive update source should be listed first (local file system
is better than a remote ftp site)
i) FTP methods
ii) mount and file traversal module(s)?
iii) Other methods ???
d) Status file parser/generator.
e) Package file parser/generator (related to above). Handle multiple
Packages files, from different sources. Each package contains a
link back to the packages file structure that contains details about
the origin of the data.
f) Dependency module
i) dependency/conflict determination and linking
ii) reverse dependency generator. Maybe merged with above
iii) package ordering/event generation module
g) module to interact with dpkg
4. Data flow and conversions analysis.
____________
__\|ftp modules|
/ /|___________|
_ ____________ / ________________
| update | / |mount/local file|
|==========================>| module |/_____\| traversals |
| |_____________| /|________________|
| ^ ^
| | | ______________
______|_______ _ _____ ______ | _____v________ \| |
|Configuration | |configuration| | |Packages Files| ===|Status file |
| module |<=>| data | | |______________| / /|____________|
|______________| |_____________| | ^ /
^ | | /
| | _______v_______|/_
| | | | ________________
| | | |/_\| Dependency |
| | |backing store |\ /| Module |
| | |______________| _|_______________|
| \ ^ /| ^
| \ | / |
| _\|____v_______|/__ ____v_______
|_____________________________\| User interaction| | dpkg |
/|_________________|<==>| Invoker |
|___________|
dpkg also interacts with status and available files.
Reply to: