[srivasta@datasync.com: LONG: Synopsis: various configuration management proposals]
I found this goodie while cleaning up my mailbox. It's a must-read if
you haven't read it earlier.
Wichert.
----- Forwarded message from Manoj Srivastava <srivasta@datasync.com> -----
To: debian-policy@lists.debian.org, debian-devel@lists.debian.org
Cc: Wichert Akkerman <wichert@wiggy.ml.org>
Subject: LONG: Synopsis: various configuration management proposals
From: Manoj Srivastava <srivasta@datasync.com>
Date: 28 Jul 1998 23:19:04 -0500
Hi,
Since this has come up in the past, and we had lots of nice
suggestions before, I am posting a giant collection of old and new
proposals. I beg that people review all this, and Wichert, please see
if you can boil all of this down to one (or more) proposals.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Ian Jackson <ian@chiark.greenend.org.uk>
Yes, we do need something like this.
Properties that it needs to have include (and not all of these are
true of your proposal), in no particular order:
1. Questions may need to be `independent' of any particular package.
2. Only a particular package can determine which questions need to be
asked in what order; in particular, following questions can depend on
the previous ones. This means that we can't specify the questions in
advance in a file. Instead, we have to put the information in
command-line arguments to the query program.
3. Questions should have a `name' that is textual (not numeric), and
is separate from the prompt string. Given 1. the name should probably
have a hierarchical structure. Given 2. there needs to be a way to
put arbitrary `parameters' into the `name'.
4. There should be a way to specify how `important' it is that a
question be asked, and an environment variable or something to specify
how willing we are to prompt, so that we can tune the level of
prompting.
5. The interface should be suitable for changing the UI later (eg,
plain-text, fullscreen text, X or whatever).
6. The database format used to cache answers should be editable by
humans.
7. The query program should be the same as the retrieve-question
program, so that the database of previous questions acts as a cache
for the user.
8. If the query program cannot prompt but the arguments say it needs
to then it should indicate this with a nonzero exit status, which will
(hopefully) cause the script to bomb out.
9. Valid responses should be specified by regexp (preferably a
reasonably fully-featured regexp like a Perl one) not a glob.
10. Metacharacters in prompts and data should work completely
correctly.
Ian.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Ian Jackson <ian@chiark.greenend.org.uk>
Oliver Elphick writes ("Re: Proposal: Automatic query servicing for dpkg installation scripts "):
...
> Are there any scripts that actually build their questions? Isn't it
> usually (always?) the case that the question is in the script as a text
> string, though the logic may skip it? Where parameters are needed,
> I should like to be able to pass them in, so that the question
> database would have to contain something like a printf format string.
Yes. For example, the scripts in the base setup procedure that ask
for your IP address et al. (`What is the address of system iguana'.)
You might have a fancy nameserver configuration which would prompt for
zones to primary and secondary, or something.
> If the questions are out in the database, they can much more easily be
> internationalised; is this a goal of the project?
Certainly it is a goal. However, I think we should distinguish
question strings (which are essentially static) from question
name,answer pairs.
> >3. Questions should have a `name' that is textual (not numeric), and
> >is separate from the prompt string. Given 1. the name should probably
> >have a hierarchical structure. Given 2. there needs to be a way to
> >put arbitrary `parameters' into the `name'.
>
> The way I wrote the script, it could have any arbitrary text, except for
> the delimiter character; I thought of it as a number, because that means
> less typing.
We need more structure for this, so just numbers are not appropriate.
Obviously the documentation should define the syntax for question
names, and the implementation should enforce it.
> >4. There should be a way to specify how `important' it is that a
> >question be asked, and an environment variable or something to specify
> >how willing we are to prompt, so that we can tune the level of
> >prompting.
>
> Presumably this is when there is no default in the database?
> Is this to enable the installation scripts to ask only `important'
> questions, whereas a different context would prompt more readily?
Yes.
> I think I need some meaningful names attached to the different levels
> of prompting so that I can understand how you are thinking.
Welcome to your new Debian GNU/Linux installation.
You may choose now in a very broad way how much you want to be asked
about the installation and configuration process. If after the
installation you want to configure programs in more detail, please
run the `dconfig' utility.
[E] Express setup. Only the most vital questions are asked.
[C] Custom setup. Any reasonably-relevant question will be asked.
[D] Detailed configuration. Only experts will understand many of
the questions.
[N] Absolutely no prompting. Not recommended; some programs will
not be correctly configured.
[L] Load configuration data from another system.
If you are asked a question you do not understand, choose the
default option if there is one. This will always be reasonable.
> My script was using different calling names to distinguish its function;
> I think I can do better by duplicating file pointers, but I haven't
> really thought how yet. I need to distinguish output of the question to
> the screen from output of the answer to the calling script.
A program like this can legitimately access /dev/tty (or DISPLAY or
whatever). At some point dpkg scripts' stdin will be connected to
/dev/null and its stdout and stderr directed into a logfile.
...
> >10. Metacharacters in prompts and data should work completely
> >correctly.
>
> Again, this is more difficult to achieve in shellscript; however, what
> should metacharacters be doing in this context? Correctness implies a
> specification to measure against. Are we talking about shell or
> regexp metacharacters here?
I mean that there should be no characters which cannot be handled both
as parts of query strings or in data items. So, the database format
and mechanisms for getting data in and out need an escaping mechanism.
Ian.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
"James R. Van Zandt" <jrv@vanzandt.mv.com>
Andreas <ad@papyrus.hamburg.com> writes:
>wouldn't it be better to not ask any question at all as part of the
>core installation process?
Yes.
I suggest that we need, primarily, three things:
1) The package installer should be able to get configuration
information from several places:
- a same-host database
- a database on a specified host
- default answers supplied with the package
- from the sysadmin (interactive)
2) The sysadmin should be able to specify a set of rules for where the
information should come from. The source could depend on
- the host being installed to
- the package being installed
- the specific question for a specific package
- whether the package is new or an upgrade
- default
There could also be several default methods, of specified priorities.
The package installer would try each in turn. If using a database,
the installer would supply answers like "expect". Any method could
fail (including interactive, if the sysadmin does not know an answer),
and the installer should leave that package in an unconfigured state.
Providing this fallback is critical, because it lets the installer
handle a mixed collection of ordinary and database-aware packages.
3) There should be several ways to load a database. The installer
should be able to monitor interactive installations and record the
sysadmin's answers. If the installer is told to get answers from a
database, but a configuration script asks a question for which the
database has no answer, then that configuration fails. However, the
question should also be added to the database. The sysadmin could use
a tool like the kernel's menuconfig to review these questions and
supply answers later. He should also be able to copy answers from one
database to another on a per-package basis.
We should invent a debian/questions file which supplies the possible
configuration questions, legal answers for the multiple choice
questions, and default answers. (The kernel configuration files might
be a good place to start.) All this data would be automatically added
to the database at package installation time. Developers should also
have a tool to generate an initial debian/questions file from the
database.
There are several ways to employ these tools:
Current Debian Installation Style - The only source of information
is "interactive".
RedHat Installation Style - The only source of information is
"package defaults".
Hands-off Installation - Specify no sources of information, so all
interactive configurations would fail. The sysadmin would answer the
questions in a later interactive configuration pass, by filling in the
answers to recorded questions, by fetching answers from a database on
another host, or some combination.
Copycat Installation - Specify a database on another machine as first
priority, and interactive as second priority.
Many-Host Installation - Specify the local database as first
priority (preloaded with host-specific data, like the IP number),
and a database on another machine as second priority.
- Jim Van Zandt
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Ian Jackson <ian@chiark.greenend.org.uk>
Raul Miller writes ("Re: Proposal: Automatic query servicing for dpkg installation scripts"):
> Ian Jackson <ian@chiark.greenend.org.uk> wrote:
> > 2. Only a particular package can determine which questions need to be
> > asked in what order; in particular, following questions can depend on
> > the previous ones. This means that we can't specify the questions in
> > advance in a file. Instead, we have to put the information in
> > command-line arguments to the query program.
>
> But a package should be able to grandfather responses from previous
> package instances. We have plenty of instances where new package
> names are needed for what is essentially the same package. [Not
> that it's always appropriate to grandfather responses...]
Yes. The way I see this working is to have the questions have names
that are in principle independent from package names. So you might
have the question `mail-transfer-agent.visible-domain' for the `what
is this machine's visible mail domain'; all packages that needed to
know the answer to this could ask it with:
dquestion-query --qname mail-transfer-agent.visible-domain \
--qtext "What is this machine's visible mail domain ?" \
--do-query:usually --default "`hostname -f`" \
--nocase --accept '[0-9a-z][-0-9a-z]*(,[0-9a-z][-0-9a-z]*)*'
Perhaps the question string itself would be stored in a file somewhere
else (to help with i18n).
The program would compare the --use-default with an environment
variable, so that for example if DQUESTION_ALLOW_QUERY were set to
`always' or `usually', then it would prompt, but not for `rarely' or
`never'. If --do-query:always is set and
DQUESTION_ALLOW_QUERY=disable then it fails. (Better names for these
variables and options and stuff are probably required.)
If there was a value in the database it would use that (perhaps there
would have to be a DQUESTION_ALLOW_REQUERY to allow you to reconfigure
a package later, and that would use the current value as default).
The database would be indexed by the qname, which would usually start
with a package name, but wouldn't have to change if the package did.
Ian.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Jason Gunthorpe <jgg@gpu.srv.ualberta.ca>
On Tue, 19 May 1998, Oliver Elphick wrote:
> Well, having looked at COAS, it seems not to dictate any particular data
> format; it has to cope with a large number of independently developed
> packages, each with its own format. To add a new one means writing a
> module which plugs into COAS to handle the new package. Once COAS is
> available, this can easily be done.
Specificaly the information I was talking about is the schema sub
directory. It is not quite what I was expecting (last I looked at coas it
was not for this detail) and I'm really not terribly enthused about it.
What I was thinking (and what some of the COAS docs claimed) was that it
was much like the SNMP MIB descriptions. Let me elaborate..
For each possible question we would describe the form that the answer can
take and we would describe what we are asking. SNMP uses ASN.1 notation
and looks something like
sysORIndex OBJECT-TYPE
SYNTAX INTEGER (1..2147483647)
MAX-ACCESS not-accessible
STATUS current
DESCRIPTION
"The auxiliary variable used for identifying instances of
the columnar objects in the sysORTable."
::= { sysOREntry 1 }
COAS provides the SYNTAX information but completely ingores the
description - your format completely ingores syntax. What we need is
description, syntax, defaults and so on as we scale up to more interactive
systems
Maybe like
mailname OBJECT-TYPE
SYNTAX STRING
STATUS mandatory
DEFAULT "debian.org"
DESCRIPTION
"The externaly visable mail name of the system"
::= { .. }
I don't know if we want to use ASN.1 - It seems like a nice idea to use
existing defined things however. We could probably even use the same form
as the Package files, or we could use SGML, or something totally new - it
really doesn't matter, any thing I say about ASN.1 can be applied just as
well if it is in some other form.
Like the way SNMP works the ASN.1 description would serve only as the
information that the UI tools use to understand what they are viewing.
The actual answers to the questions could be stored in some arbitary
database - this is something the packager does not have to deal with.
As we get more fancy we can extend the description to encompass a wider
range of database options [say native config files for the package] and to
encompass actual management.
So, to the package writer the tasks required of them would be to write a
Description of the configuration questions and then in their script
instead of prompting they would call a query program which would prompt
depending on various conditions.
Now, how this solves Ian's points,
#1 - I read this as meaning the questions are seperable - they would
be present in the Description file (MIB in SNMP notation) We could
make them avail for download via http just like changelogs are now.
#2 - The package has control here, it's script is performing the queries
into the database. As we scale up to pre-configuration the package
can optionally provide a UI Plugin that provides it's logic functions
If it does not then the user will simply have more things to answer
than is stricly relevent to their choices.
#3 - If we follow the SNMP model then each quesiton has BOTH a name and
a numeric string arranged in a heigharchy (debian.mail.mailname for
instance)
#4 - This is just a field in the 'MIB'
#5 - It is, because each question has a structured form we can have a
generic GUI that can browse the question tree. With options to
install 'Plugins' in the GUI we can scale even more so.
#6 - Implementation detail
#7 - ??
#8 - Implementation detail (related to #4)
#9 - This is handled more generally by typing the responses in ASN.1 -
this is MUCH more suited to a generic GUI than a regexp, but it does
not exclude the possibility of a regexp type.
#10 - This is not covered in the SNMP model. I suggest a a substition
system in the description feild that allows lookup and substition
of other database entries and to a lesser extent environment
variables (env vars limit the usefullness of a GUI tool)
Perhaps $(debian.mail.mailname) and #($FOO) ?
There are also a few other interesting benifits
#0 - We are -VERY- similar to something estabilished, we don't create
new languages or even so much new concepts
#1 - SNMP presents itself as good choice for a remote
administration protocol
#2 - We provide a native configuation storage system for apps/scripts that
desire. This is instead of the usual technique we have been using of
having a script as a conffile or having the script source another
file.
#3 - Using a heigharchical structure gives each package alot of freedom
in what variables they can have
#4 - The package writer does not have to use the query program to generate
the prompts, they can come from within the script, an external
program, a GUI plugin, etc - there is alot of flexability in how the
information can be inserted into the database and how it can be taken
out of the database - this is nescessary as we move to a more plugin
rich environment (imagine how ecompassing a sendmail plugin could be)
Some things we should look at,
- Free ASN.1 parsers/libraries, I think one is around
- What fields can be borrowed from SNMP and what more do we need?
- Consider how we would model some of the existing setup scripts
in ASN.1 and verifiy that this -IS- an acceptable model.
[Anyone want to do these? I should be working on APT.]
dtxtdb is not at all the same solution as this - it models the backend
native database/cache, which does not deal with any of these issues.
As a final thought, it is my belief that implementing a 'simple' system
like Oliver started with would be suicide. We would lock ourselves into
something that is not extensible or realizable into a powerfull GUI.
Jason
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
--
I could prove God statistically. -- George Gallup
Manoj Srivastava <srivasta@acm.org> <http://www.datasync.com/%7Esrivasta/>
Key C7261095 fingerprint = CB D9 F4 12 68 07 E4 05 CC 2D 27 12 1D F5 E8 6E
--
To UNSUBSCRIBE, email to debian-devel-request@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
----- End forwarded message -----
--
==============================================================================
This combination of bytes forms a message written to you by Wichert Akkerman.
E-Mail: wakkerma@cs.leidenuniv.nl
WWW: http://www.wi.leidenuniv.nl/~wichert/
Reply to: