[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Admintool/config tool

Note: the following is a too-brief summary of *my* view of the admintool
design.  I am sure that some would disagree with it.

On Fri, 3 Oct 1997, Jason Gunthorpe wrote:

> I'm looking for any information about what the AdminTool/ConfigTool group
> is doing. I'm especially interested in any configuration file format or
> standard that was decided on.. (Ie I need a configuration file(s) for
> deity)

the admintool project seems to have temporarily run out of steam.  we got
near to having a design spec amidst a few flamewars and then everyone just
took a rest and i don't think anybody has really had the energy to take up
the discussion again yet.

the basic idea which we had (almost) agreed upon was:

    - config files for any given program are generated:

      - simple generation schemes are inadequate because they lose
        too much information and lose flexibility and power for the
        competent sysadmin (e.g. Sun's Netra web based config system - 
        you can only do what the programmers have thought of already, if
        you need to do more than that you MUST disable their configtool
        otherwise your hand-crafted configs will get blown away)

      - solution is to generate config files from a "template".  the
        template is a text file containing the verbatim text of the
        output config file, with embedded tokens which get substituted
        with appropriate values. the values are extracted from the
        configuration database The templates can also contain simple
        language constructs (if/then, for loops, etc) for generating
        complex sections of a config file.

      - m4, cpp, perl (actually eperl or Text::Templates module) are
        possible languages for the template.  Alternatively, write our
        own much simpler language.

      - any changes made directly to the real configuration file will be
        blown away when the config generator is run. therefore permanent
        changes have to be made to the template.  Since the template
        *is* the config file + markup language, the full power and
        flexibility of the original config file is retained.

      - sendmail.cf being generated from sendmail.mc is an existing example
        of this template-based config file generation idea. the .mc file
        is the "template".  It uses m4 as the "template language".

    - the config database is, at the moment, based on dcfgtool. a simple
      text file database.  advantage of this is that it can be manipulated
      with command line tools, from gui front-ends, and from vi or other
      text editor.

    - the system MUST work in both text mode and graphics mode.  A sysadmin
      MUST be able to configure the system from the command line, bypassing
      all the pretty but slow graphics (e.g. telnet session to a system at
      the other end of a flooded 28.8Kbps modem link).

there was a lot more, but i'd have to dig up the details from my
archives of the admintool project.  If you can't find it archived
anywhere i can email you a mail file of the lot or selected articles.
I'll email you a few articles later today...if you want the rest, just

there was much discussion about parsers to a) parse data out of
existing config files and into the database and b) attempt to generate
a template automatically. general consensus (with some vehement
disagreements) was that parsers are useful as a conversion tool but are
too unreliable/limited for the admintool to rely on them heavily, and
that they multiplied the amount of work required enormously.

little discussion was made about the user interfaces, we hadn't really
got up to nailing that down yet. we did know that the user interfaces
were supposed to be interchangeably modular - you could use command line
tools, ncurses text mode program, web forms, X GUI programs, etc to
manipulate the config database.

i'm mostly interested in the config gen side of the project because (a)
i do a lot of config generating anyway for my own needs and (b) i've had
to work with several systems which have got it wrong and want it to work

After a long rest from the admintool project, i started working on some
of it yesterday morning.  I've done a little work on implementing a
config generator.  I've even got one working (using eperl) which can
generate SIMPLE config files.  The generator system was supposed to be
modular so that most simple config files could be created by a generic
config generator script. other, more complex config files would need
additional custom generators.

given these entries in the dcfgtool database:

---cut here--- /etc/config/gpm --- cut here ---
append="-R -3"
---cut here--- /etc/config/gpm --- cut here ---

and this template:

---cut here--- /etc/templates/gpm.conf --- cut here ---
#include "eperl.header"
#  /etc/gpm.conf - configuration file for gpm(1)
#  Written by Martin Schulze <joey@debian.org>
#  Mostly self-explanatory.  If you mouse seems to be to slow, try
#  to set responsiveness=15, append contains some random arguments appended
#  to the commandline.
---cut here--- /etc/templates/gpm.conf --- cut here ---

my simple config generator produces the following config file:

---cut here--- /etc/gpm.conf --- cut here ---
#  /etc/gpm.conf - configuration file for gpm(1)
#  Written by Martin Schulze <joey@debian.org>
#  Mostly self-explanatory.  If you mouse seems to be to slow, try
#  to set responsiveness=15, append contains some random arguments appended
#  to the commandline.
append="-R -3"
---cut here--- /etc/gpm.conf --- cut here ---

Now this doesn't seem so exciting.  it looks like it could be done by just
concatenating /etc/config/gpm onto the end of a text file containing
comments.  That's because this is an extraordinarily simple example.  The
template could contain a lot more, including more replacement tokens, eperl
code to create tables from the db, and comments interspersed anywhere.  

The key thing to remember about the templates is that ANYTHING which is
NOT interpreted as part of the markup language gets output VERBATIM.

This is crucial.  The template not only defines WHAT values get
interpolated from the database into the output config file, but also

This adds a great deal *without* taking away the sysadmin's ability to
write/tweak/comment their own config files.  Effectively, it enhances
all existing text config files with an extension language.

btw, as i said, what i have done so far is trivially simple. here's what
i have.

first of all, i have a shell script wrapper which preprocess the
template using sed to convert "<<__TOKEN__>>" to eperl 
"<: print $config{__TOKEN__} :>". 

simple token substitution will be the most common operation, and
this conversion stage allows the template creator to do that with a
minimum of typing. the whole lot then gets piped into eperl for further

it can take a "-d" command line option which pipes it into 'cat -n' for

---cut here--- gentemp.sh --- cut here ---
#! /bin/sh

prog='eperl -P -I/etc/templates -'
if [ "$2" == "-d" ] ; then prog='cat -n' ; fi

sed -e 's/\(<<\)\(.*\)\(>>\)/<:=$config{\2}:>/g' $1 | $prog
---cut here--- gentemp.sh --- cut here ---

and here's some eperl code which is #included into each template as
needed. it reads the config db and creates a hashed array $config{} to
store the config values.  This is only a fraction of what is needed, but
it does do the most common operation which is substitution of tokens
with values from the database.

---cut here--- /etc/templates/eperl.header --- cut here ---
	open(CONFIG,"dcfgtool --export|") || die "can't open dcfgtool pipe: $?" ;

	while (<CONFIG>) {
		chomp ;
		next if /^$/ ;
		next if /^#/ ;
		($var,$val) = split("=",$_) ;
		$var =~ s:/:__:g ;         # can't use "/", convert to "__"
		$var = "__" . $var . "__" ;
		$val =~ s/(")(.*)(")/$2/g ;
		$config{$var} = $val ;
#		print "$var --> $config{$var}\n" ;
---cut here--- /etc/templates/eperl.header --- cut here ---

from here, i need to figure out a non-clumsy way of generating tables
(like multiple cache_host entries in /etc/squid.conf or lines in
/etc/fstab) from the database.  A lot of this got thrashed out on the
admintool list so i'll have to spend some time reading through it again
to remember what we worked out :-).

conditional generation of output is already supported (eperl *is* perl,
after all). just put some code like:

    <: if $config{foo} { print "foo=$config{foo}\n" }  :>

into the template. for loops (for table generation) are already
supported too. the problem is not in the language syntax but in figuring
out the best way to store and extract the data elements for the tables.

there are two choices that i can see at the moment:

    1.  dumb database and smart generator.  this can be done with the
        existing text file dcfgtool database, just by writing eperl
        code to construct the table from the flat file db.  disadvantage is
        that it requires knowledge of the exact structure of the db, which 
        means that it would be difficult to change the structure without
        breaking existing templates.

    2.  smart database and not-so-smart generator.  e.g. SQL.  table format
        is implicit in the database, and can be queried by the code prior to
        using it.  a lot more flexible.  disadvantage is that it loses the
        simplicity (and vi editability) of the database.

i'm not really happy with either choice yet. i'll have to do some more
thinking about the problem.


craig sanders
networking consultant                  Available for casual or contract
temporary autonomous zone              system administration tasks.

TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to
debian-devel-request@lists.debian.org . 
Trouble?  e-mail to templin@bucknell.edu .

Reply to: