My take on the whole approach (was: Re: Old stuff)

[Aaron Van Couwenberghe <vanco@sonic.net>]

On Tue, Dec 22, 1998 at 01:48:06PM +1000, Anthony Towns wrote:
I don't agree with lots of this. for the specific reasons:

	We shouldn't really need to force package maintainers to use our
scripting language to be able to use these facilities.
	We should really use a powerful internal language like perl for the
internal handling code, and provide a flexible interface.
	We really should put in place a system that won't disrupt life for
anybody, if anything at all is going to be put into place.

	Conceptually, all of these issues are fairly easy to address when
they are taken out of context from one another; given the best solution to
each, a course of action can be planned that will bring the best results.

for issue #1, (maints being able to do things like they did before), we
could simply tell maintainers that their configuration scripts *must* take
the --nointeractive option and be able to take all of their variables on the
commandline, in the order that they are declared in their debian/vars file
(explained later)
	for issue 2, DBI is ideal because if anyone *wanted* to set up a
three-tier system where a database engine is not used but the options file
is mastered and transferred to each host, a simple perl module can be
written to drop-in (polymorphism is neat) and be *completely* transparent
even to debian-admintool (ie the internal code).
	for issue 3, the answer is to make everything following the current
standards-version to not break because of debian-admintool's existence.
Hopefully it can be implied that statement is retroactive as well but that
would be more difficult.

	The way to handle this is to *change nothing* in the standards
version but two things:

1) each package following standards version XXX must have a file included
named debian/vars. the contents might look something like this, for a berlin
display server:

ENCRYPTION	yes (or 1 -- yes/no might become boolean synonyms)
ORB_ENC_TYPE	kerberos
GROUP		berlin
MIGRATE_WIDGETS	yes (or 1)

2) each package following standards version XXX must be able to take the
'nointeractive' option, and a list of variables (guaranteed to be in the
order they are from debian/vars), for a postinst format such as this:
postinst --nointeractive ENCRYPTION=yes ORB_ENC_TYPE=kerberos GROUP=berlin
MIGRATE_WIDGETS=yes .... ad infinitum.

To simplify this, small modules (and/or libs) could be guaranteed to be
present that perform functions like, in C, splitting the entire list into a
flat array of type 'char *', omitting the keys. In perl, one call could
separate the commandline into an associative array. Similar behaviors in
bash.

one more thing to address. How do we force postinsts to communicate their
configuration info to debian-admintool? we could require the package print a
list of the same form as debian/vars, for every variable that is not
included in debian/vars or for which the default has been overridden.

> 
> > 6. Communication language
> > This communication between the frontend and the configmodule should be as
> > simple as possible. Since most IO implementations default to line-buffered O,
> > so we use a simple language where each command is exactly one line. A
> > prelimary list of commands is:
> 
> Are there any example scripts available?
> 
> I'm a little disinclined to trust a language that's never had a program
> written in it; and in particular, it sounds like the confmodule has to
> have a fair degree of intelligence about it. (am i interactive? then
> ask these questions? oh, the user answered no? then skip these. ask
> these too...)
> 
How about this:

	For the computer doing the key-value pair exportation, any kind of a
database accessible via Perl's DBI/DBD modules would suffice. there would be
a single database, with one of two schemes being used to hold the setup
variables:

	1) one table per package, one record per variable; accessible via
	SELECT 'var, data' FROM <packagename> (bad syntax? rusty with SQL)
	then package configuration info could be selected on a per-package
	basis.

	2) one huge table, with two fields. one being package name and one
	being a blob containing an internal structure that would be decoded
	on each end machine. This is better for type flexibility (with
	scenario one you limit each record to using the same type of var);
	it would also be faster. 

	the weakness of #2 is that it limits the maximum amount of total
config data any package may contain. However, maintainers are free from
conventional constraints (besides trying to use characters that are commonly
listed in the asyncmap ppp option); this also forces each local machine to
split the blob upon transmission.

The two ways the data could be internally represented:

1) data could be selected from the database on a per-package basis and
stored in a reused, variable length hash. This is slow, but better for
memory management. storage scenarios 1 and 2 provide well for this.

3) data could be selected from the database all at once, or in chunks. For
storage scenario one, this depends on the client machine knowing the names
of the packages it's looking for config info on. for scenario two it could
be easily done without the client having this knowledge.
	this would be represented internally as a hash containing
references to hashes, of the form $data{'packagename'}->{'key'}
Now, the use of a database engine, perl, and DBI require two things:

1) the first machine on the network have a database engine installed.
2) all other machines' capability to interface with the engine will depend
	on them having perl and DBI installed on them.

> Is there any possibility of using the work that's been put into the
> Linux kernel configuration here? Are there things we want to do that
> they haven't already done?
> 
> Cheers,
> aj

The linux kernel configuration is completely different. It just stores
defines in a configuration file that is then reloaded when make is called.
This is something that won't serve us at all.
	As I have expressed earlier, I think trying to simplify the
internals of the system is bad; we *really* want to use a high level,
flexible language like perl here. All that is needed on top of this is an
abstract interface so that all package maintainers need to do is supply a
list of variables that their package setup scripts will need access to.

In addition, this *needs* to coexist with the dpkg --nointeractive mode that
will eventually be implemented, simply because it needs to be. For this, the
scheme could be modified slightly:

dpkg --nointeractive -i <list of packages> # installs and configures
	# packages noninteractively
dpkg --pending --configure # runs the queue of interactive scripts that 
	# were registered by dpkg --nointeractive
# hypothetical: JUST dpkg -i runs the noninteractive postinsts and then the
	# interactive queue that was registered by it.
#subprocess
berlin-configure /*whatever interactive command berlin has registered*/
#	stdout:
#		ENCRYPT=no
#		ENC_TYPE=
#		GROUPS=adm root console <-- this would require the blob
#						storage scenario
# all other interactive commands -- output dumped to a file
if dpkg-admintool installed and configured:
dpkg-admintool --store /var/log/dpkg/interactive.log --tag profilename \
	--revision x
# those options could be set interactively as well
# IF dpkg-admintool is configured to actually store anything:
/usr/sbin/store-admintool-profile /var/log/dpkg/interactive.log
# decodes textfile in form:
# <packagename>
# ^<KEY1>\s.*<DATA>$
# ^<KEY2>\s.*<DATA>$
# 	and saves it to the database in table profilename-x. Registers
# a profile in the table "profiles", format:
#	CHAR(20) NAME, INT revision

# computer on other end calls:
admintool-pull-config $HOSTNAME

# variables are pulled into local memory and arranged. .debs are migrated
# via apt. the dpkg --nointeractive portion is run locally.
apt-get --nointeractive dselect-upgrade
# spawns:
dpkg --nointeractive -i <list of debs>
# after apt-get is finished, a call to dpkg --interactive --auto is made
dpkg --pending --configure --interactive --auto
# just runs the interactive queue in silent mode eg taking variables
# from somewhere else.
berlin-config --nointeractive ENCRYPTION=1 ENC_TYPE=kerberos ......
# perhaps a better name for this is berlin-config --silent
# and dpkg --pending --configure --silent
# which have no output since they are in nointeractive mode; therefore they
# won't be asking for any input and won't need to output anything.

so, the only *real* differences to the packaging system itself would be:

IF maintainers have an interactive portion of postinst:
	they must include debian/vars with reasonable defaults
	debian/tmp/<path-to-config-prog> must take the options
		{--silent|--nointeractive}, and accompanying that, a list
		to be used for config info instead of user input.

IF maintainers have a noninteractive postinst:
	maintainers don't have to do anything differently whatsoever.

I ackgnowledge that my ideas are not the end-all, and someone probably will
come up with better ones. However, these are the issues that
debian-admintool needs to be able to successfully address. Any complete
proposal *must* contain all of these aspects.
	Personally I think it should take a route similar to this. I see
many weaknesses in this scheme however, and would definately reconsider some
of these methods.

-- 
..Aaron Van Couwenberghe... ..vanco@sonic.net.. ..aaronv@debian.org....
	Berlin:			http://www.berlin-consortium.org
	Debian GNU/Linux:	http://www.debian.org
----
Nullum magnum ingenium sine mixtura dementiae fuit.
        [There is no great genius without some touch of madness.]
                -- Seneca