[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

PROPOSAL: automatic installation and configuration


I have done a few experiments about automatic configuration of packages
and have now some working code. I want to describe my ideas and how to
integrate in the distribution. You can download my experimental but
almost working code at:


My idea of automatic installation is that if I need to install many similar
machines I could install manually the first, remember all the answer to
each configuration question, then go to the other machines and just type
install without any more prompting. At the moment this is only a dream
given the terrible way the install scripts are designed and written.

So I've written a little package which could be used by developers in
their postinst scripts to allow the automatic installation and
configuration of a debian system with a very minimal changes to the
existing code.

The overall goal of my project are:

    * make possible a totally automatic installation without prompting

    * save automatically all answers during a manual installation for later
      reuse in subsequent automatic, semi-automatic or manual installations

    * be able select different levels of interaction with the user, which
      could correspond to user expertise levels, but not necessarily

    * have the least impact on existing install scripts. Obvioulsy they
      have to be modified but the changes are usually very small

    * provide a uniform and consistent interface for user interaction

    * use a graphical interface when possible or the old plain tty, at
      user choice

    * save important installation messages, which currently are only shown
      on the screen with a 'press enter' prompt, for later inspection

    * use a flexible database structure with more levels of configuration
      files (script, debian, site, host, package, updates, etc.) and
      inheritance of configuration values

    * use only ascii databases and base programs for ease of maintenance
      and independence from installed packages or esotheric software

My proposal doesn't attempt to address the aspects of package configuration,
like the recent proposal of Goswin Brederlow did. I'm only interested in
replicating automatically the sequence of answers that users must currently
do by hand. A structured configuration system could provide these features
and many more but is is a much more ambitious project and would require much
more effort and developement time. My proposed API could however be reused
in a subsequent configuration project.


Of the about 700 installed packages in my system I found that only 70
scripts need to interact with the user. Most of the interaction of those
scripts falls in three cathegories:

1)  echo "message"
    echo -n "enter value:"; read answer

2)  echo "message"
    echo -n "do you want... ? (y/n)"; read answer
    if [ $answer = y ]; then ...

    This is the most frequent case. I believe that 90% of the prompts
    are for yes/no questions

3)  echo "read this message"
    echo -n "press enter"; read enter

    This is also a very common case

Obviously each scripts implements the above code in a completely different
and clever way. My idea is that if we replace all the above fragments in
every script with calls to a standardized interface we could intercept all
the user interaction and implement whatever control we want. So I wrote two
commands which could be used to easily replace the above code providing
an easier uniform method of getting answers from the user and a transparent
interface with a simple flat ascii configuration database. The commands are:

dpkg-getconfig <varname> [options...]


    -\?|-h|--help                       print this help
    -a|--auto|--from-db                 don't ask, get value from db
    -b|--bool|--boolean                 accept only true/false values
    -c|--choice <values...> "."         accept choice between values
    -C|--check <check-command>          input check command
    -d|--default <default>              set default value
    -D|--DEFAULT <default>              force default value
    -e|--expert                         set query level = expert
    -f|--file|--database <db>           set database file
    -i|--important                      set query level = important
    -l|--level <query-level>            set the query level
    -L|--lines|--text                   accept multi-line text
    -m|--msg|--message <message>        message text (- = stdin)
    -o|--once|--missing                 set query level = once
    -O|--output|--outfile <filename>    write answer to filename
    -p|--prompt <prompt>                prompt message
    -P|--package <package>              querying package name
    -s|--set|--set-value <value>        don't ask, write value to db
    -t|--title|--subject <title>        set window title
    -v|--var|--variable <varname>       set variable name
    -V|--verbose                        set query level = verbose
    -y|--yesno|--yes-no                 accept only yes/no values

dpkg-message [options...] <message>


    -\?|-h|--help                       print this help
    -a|--auto|--to-mailbox              don't show, save in mailbox
    -e|--expert                 	set notify level = expert
    -f|--file|--database <db>           set database file
    -i|--important                      set notify level = important
    -l|--level <query-level>            set notify level
    -m|--msg|--message <message>        messsage text (- = stdin)
    -n|--nowait                 	don't wait for Enter
    -o|--output|--mailbox <mailbox>     default mailbox
    -O|--OUTPUT|--MAILBOX <mailbox>     force mailbox
    -t|--title|--subject <title>        window title or mail subject
    -V|--verbose                        set notify level = verbose
    -w|--wait|--sleep <n>               wait n seconds after message

Using only these two commands the above script fragments could be easily
rewritten as:

1a) answer=$(dpkg-getconfig PACKAGE_VARIABLE \
		--default default --message "message" --prompt "prompt)

1b) answer=$(dpkg-getconfig PACKAGE_VARIABLE \
		--choices "aaa" "bbb" "ccc" . \
		--default "ccc" --message "message" --prompt "prompt)

1c) answer=$(dpkg-getconfig PACKAGE_VARIABLE \
		--lines --message "message" --prompt "prompt)

2)  answer=$(dpkg-getconfig PACKAGE_VARIABLE \
		--yesno --default y --message "message" --prompt "prompt)

3)  dpkg-message --title "title" --message "message"

The benefits, besides the simplification of the code and the uniform user
interface, are that all values entered by the user can be automatically
saved into a database, and that by setting a global configuration variable,
transparent to the scripts, the values can be later retrieved automatically
from the database instead of prompting the user.

The messages printed with dpkg-message are shown and automatically saved
in a mailbox, and can be later be examined with elm or any mail program.
The messages are appended directly to the mailbox and don't require an
installed and working MTA. So after the installation is finished one can
find many nice mail messages telling him exactly what the developer wanted
to notify. In this way nothing gets lost and you don't need to pay
attention to the terminal during the installation.

The level of interaction with the user can be controlled by the combination
of two values, an installation mode, or prompt level, or user level, which
can be one of the following:

    auto		don't ask nothing, use defaults or fail
    important		ask only very important questions
    once		ask 'once' questions only the first time
    normal		ask all normal questions every time (default)
    expert		ask also questions for expert users
    verbose		ask everything, for masochists only

and a corresponding importance value for each question which can be:

    auto	        don't ask, read from db or script defaults
    important		important question
    once		normal question, but ask only once
    normal		normal question (default)
    expert		question for expert users
    verbose		noise question

The user is prompted only if the install mode is equal or higher than the
query mode. For example a normal question(3) is asked only it the install
mode id normal(3) or higher. If one chooses to see only the important(1)
questions a normal or expert question will never asked but only retrieved
from the database or from the script defaults. The script can specify two
defaults for a query, with different priority respect the database value.
For example:

    dpkg-getconfig VAR_NAME --DEFAULT $x --default $y

will return, or propose to the user, $x if not empty, or the database
default if not empty, or $y. This is rarely used but sometimes handy.
If the user is prompted he will have anyway the last choice. 

If the user selects the "auto" install mode and has stored answers or 
scripts defaults for every question the system can be installed in a 
totally automatic way while he is playing tetris on another console.

The anwers can be easily and automatically collected by running a manual
installation the first time and are stored in an ascii file which can be
edited with vi or any standard text editor.
The format of the database is very simple, it is actually one or more
bash scripts which are sourced in sequence. The updates db collects all
the user answers and is loaded last so that its values override any setting
from other files.
The fact that the database is a script makes possible to implement easily
whatever inheritance and priority scheme is needed. My suggestion is that
we use the following files:

    config.db		the main config file which sources all the others
    debian.db		debian official defaults (read-only)
    site.db		site defaults
    host.db		host specific settings
    package.db		an optional specific db for each package
    updates.db		collects all user answers

The database and its format could be easily changed in the future if all
the scripts use only the dpkg-* API, so it is really not very important.
I choose bash scripts becaues they are handy for experimenting with new
ideas but we could in future use a more efficient and fast format.

For consistency the names of the configuration variables should be in
uppercase and should follow the following conventions:


for example:

    NET_ETH0_NETMASK	for the variable ETH0_NETMASK owned by package net


    CONFIG_DEFAULT_PAPERFORMAT	    for a common variable shared between
			many packages and not owned by any of them

All the common configuration variables should begin with the prefix CONFIG,
assuming that there will never be a package with that name.


I have tested the scriprs on my slink. I haven't done a complete install
because it would have meant modifying and repackaging a lot of things, but
I tested it with dotfile-bash and I seems to work fine. I will show this
example to illustrate how to change postinst scripts. This is the original
dotfile-bash postinst:

set -e
if [ ! -f /usr/X11R6/lib/X11/dotfile/bash/bytecompile ]
	echo " Byte compiling modules makes them run significantly faster"
	echo -n "Do you want to bytecompile this module now? (Y/n) ? "; read answer
	case $answer
		echo "you can bytecompile this module later by typing:" 
		echo  "dotfile bash bytecompile";; 
		/usr/bin/tclsh /usr/X11R6/lib/X11/dotfile/Generator/dotfile.tcl /usr/X11R6/lib/X11/dotfile/Generator /usr/X11R6/lib/X11/dotfile/bash bytecompile ;;
if [ -x /usr/bin/update-menus ] ; then update-menus ; fi

This is a simple yes/no question which could have been rplaced by the
following code:

    	answer=$(dpkg-getconfig DOTFILE_BASH_BYTECOMPILE \
	       --yesno --default y --package dotfile-bash \
	       -m "Byte compiling modules makes them run significantly faster"\
	       -p "Do you want to bytecompile this module now?")

I choose however to make the code a little more complicated to show also
how to simplify the installation from the user point of view. This is the
new script modified for use of dpkg-getconfig:

set -e
if [ ! -f /usr/X11R6/lib/X11/dotfile/bash/bytecompile ]; then
    bytecompile=$(dpkg-getconfig CONFIG_DOTFILE_BYTECOMPILE \
	--message "\
Byte compiling modules makes them run significantly faster.
Do you want to bytecompile every module?" \
	--choices "yes  - byte-compile every package" \
		  "no   - don't compile anything" \
		  "ask  - prompt for each package" \
		  "auto - use package default" . \
	--default yes --once --package dotfile-bash)
    case "$bytecompile" in
	    bytecompile=$(dpkg-getconfig DOTFILE_BASH_BYTECOMPILE  \
	       --yesno --default y --auto --package dotfile-bash)
	    bytecompile=$(dpkg-getconfig DOTFILE_BASH_BYTECOMPILE \
	       --yesno --default y --package dotfile-bash \
	       -m "Byte compiling modules makes them run significantly faster"\
	       -p "Do you want to bytecompile this module now?")
    case "$bytecompile" in
	    /usr/bin/tclsh /usr/X11R6/lib/X11/dotfile/Generator/dotfile.tcl \
		/usr/X11R6/lib/X11/dotfile/Generator \
		/usr/X11R6/lib/X11/dotfile/bash bytecompile
	    dpkg-message --title "${0##*/}" --message - <<-EOF
		The dotfile-bash module has not been byte-compiled.

		You can bytecompile this module later by typing:
		dotfile bash bytecompile
if [ -x /usr/bin/update-menus ] ; then update-menus ; fi

It is a little more complicated than the original because it now offers some
extra choices about byte-compiling, but it is a good example of what could
be done with dpkg-getconfig and dpkg-message.

The multiple prompting for byte-compilations, which makes so many people
angry, can be easily solved by defining a common configuration variable
CONFIG_DOTFILE_BYTECOMPILE which controls the overall compilation policy and
a variable specific for each package, for example DOTFILE_BASH_BYTECOMPILE,
which controls the byte-compilation of the specific package.
In the above example CONFIG_DOTFILE_BYTECOMPILE is asked with the "--once"
option, that is only if it is not aready defined, and the compilation of
dotfile-bash is asked only if CONFIG_DOTFILE_BYTECOMPILE is "yes" or "ask".
Note also that the script provides its own defaults in case the user selects
a totally automatic installations or just chooses to press enter.
In this way if one doen't want to be bothered by dotfile compilation he can
set CONFIG_DOTFILE_BYTECOMPILE=n once in the config db and he will never be
prompted again. The same thing could be done for emacs add-ons.

In the examples directory of the debian package you can find some other
examples of postinst scripts and configurations files.

The dpkg-getconfig provides also a nice graphical interface if whiptail or
dialog is installed. Unfortunately gdialog can't be used because of its bugs.
Also whiptail and dialog have their problems but they can be worked around.
The graphical interface is choosen automatically by dpkg-getconfig but can
also be controlled by the user with the DIALOG environment variable.
Selecting DIALOG=none will revert to the old plain tty mode.


My proposal is that the above mechanism is adopted in the official debian
distribution, once it is finished and tested. This has the following

1)	changing the policy about the installation scripts. From a certain
	debian version no package should be allowed to interact directly
	with the user and all prompting should be done trough an API.
	I suggest that by policy all scripts are executed with </dev/null.

2)	modifying a lot of install scripts. This seems an almost impossible
	task but it could actually be done in a short time and with very
	little effort because each developer will have to change only his
	few scrips and the changes are usually very simple. The orphaned
	packages could be distributed between active developers. Since the
	changes are very easy anyone should be able to modify also scripts
	written by others.

3)	modifying the packages so that they can always provide some
	reasonable default for variables which can't be asked ot which
	the user don't want or don't know how to ask. Think of an
	inexperienced user installing debian. How could he answer to
	complex questions about sendmail, ip-masquerading, etc.
	It should be possible to installa a standars machine knowing
	only few basic things (hostname, ip, dns and mailserver).
	This is a long term goal but in my opinion it is important
	that a package could be installed totally automatically,
	although not configured optimally. The expert user knows
	anyway what to change.

4)	split each postinst script into a the real postinst and a separate
	config script which can be invoked at any time to reconfigure the
	package. This is also what Goswin and other have proposed.

6)	Assign different prompting levels to various packages and questions

7)	Use the config db also for the first step of the installation from
	the rescue floppies. It should be possible to load a config file
	from a floppy or from the net and find all the values needed for
	the base installation.

The first two steps are in my opinion essential. The other require more
work for developers and some discussiona. Thy are also important but not
so urgent.


Massimo Dal Zotto

|  Massimo Dal Zotto               email: dz@cs.unitn.it               |
|  Via Marconi, 141                phone: ++39-0461534251              |
|  38057 Pergine Valsugana (TN)      www: http://www.cs.unitn.it/~dz/  |
|  Italy                             pgp: finger dz@tango.cs.unitn.it  |

Reply to: