Re: Configuration file handling (Re: [desktop] Unix configuration nightmare)
On Mon, Oct 28, 2002 at 11:54:54PM -0500, Branden Robinson wrote:
> On Wed, Oct 23, 2002 at 08:10:43PM -0400, Matt Zimmerman wrote:
> > I don't think that existing .config handling necessarily needs to change
> > at this point, unless we want to provide a standard way to suppress all
> > attempts at automatic configuration for a particular config file.
>
> Well, Joey Hess is of the opinion that "manage this config file with
> debconf"-style questions are Evil and Wrong.
>
> So, assuming I want to get with the Debconf orthodoxy, I would be changing
> my config scripts to eliminate this sort of prompting.
I can't speak for joeyh, but I think his beef is that maintainers use this
kind of question to be lazy about preserving changes. I'm guilty of this
myself, because I do not want to parse some arbitrary config format using
shell commands to seed debconf with the appropriate responses.
We're all about preserving changes here, so the light of the One True Way
should guide us to salvation. I was thinking of things more similar to
--force-confnew/--force-confold than to "manage this config file with
debconf" questions.
> > In other maintainer scripts, we need to be able to say "I have generated a
> > configuration file /tmp/blah as a possible replacement for /etc/foo/bar." At
> > that point, the maintainer script is done with the file, and passes control
> > to us.
>
> Here we run into a problem:
>
> * For most packages, the "other maintainer script" is going to be the
> postinst. In fact it's difficult for me to imagine a scenario when
> anything but the postinst would be generating a config file.[1]
>From my perspective, the tool doesn't care from which maintainer script it's
called. postinst would seem to be the only useful place to call it, I
agree.
> * The postinst, by definition, runs during the configuration phase.
> Your proposal is going to pull us farther away from the utopia of
> being able to handle all interactivity before packages are even
> unpacked, a la dpkg-preconfigure. While dpkg's conffile
> prompts already violate this, they *can* be replaced with pre-unpack
> prompting, because dpkg-preconfigure can suck new conffiles out of a
> package just as well as it can config and templates files.
>
> Non-conffile config files cannot enjoy this luxury, because they don't
> exist within the package.
Right. As we discussed on IRC, there seems to be a fundamental conflict
between preconfiguration and generated configuration files because the
package is by definition not configured yet (for the new version) at
preconfiguration time.
Packages with simple requirements could theoretically generate a
configuration in .config, but that is not the right place for it, and
packages with simple requirements can use conffiles.
I had not considered that one day dpkg could prompt about conffiles ahead of
time. That weakens my "no worse than conffiles" argument considerably.
Preconfiguration should remain unchanged in (IMO) the two most important
cases: initial installation, and novice user. In the former case, we will
never need to prompt in postinst, and in the latter, we should have a
reasonable default for them.
> * On the other hand, if we're doing an upgrade instead of an install,
> the tool(s) we use to generate the config file may already be on the
> system at "pre-configure time". However, if those tools change, and
> a package's .config script needs to be able to use the
> config-generation tool that's in the corresponding version of the
> package, you'd need to have a way of declaring this requirement so
> that config file generation could be deferred to package
> configuration. Not to mention the fact that the runtime dependencies
> of the tools used to generate the configuration files would need to be
> present at pre-configure time. Oy vey.
Yes, this would be a mess. Upgrades from one Debian release to the next
would be fragile and difficult to maintain and test effectively.
> > We check again whether the file has been modified since last time by
> > comparing it to a saved copy or checksum (the copy is optional, but gives
> > much more flexibility than storing only a checksum).
>
> Why not just a checksum? Do you have a specific application in mind, or
> do you think the copy is a good idea for the sake of people cleverer
> than we? :)
The copy makes it possible to calculate the merge. Since I am pretty
confident that we will want merges, I think we should go ahead and use the
copy for the comparison as well. It might be marginally more efficient to
cache checksums for the comparison, but I doubt it will be worth worrying
about.
> > [prompts for various cases]
> >
> > In the common cases, this should be possible with a single prompt, though it
> > could be split into two phases or selected from either a "simple" or
> > "advanced" method, or even suppressed entirely for novice users if a sane
> > default action sequence could be decided (always preserve? merge, and if
> > that fails, preserve? warn?).
>
> As you said, this is really two different questions:
>
> 1) generated file only / existing file only / merge
> 2) if merge:
> prompt for review on problems / always prompt for review
>
> I think the latter is a prime candidate for a shared template that is used
> system-wide.
Agreed. I can also envision global preferences like "always mail me diffs"
which could be implemented down the road.
> If someone wants to implement object-based debconf templates with
> inheritance, then we might want to consider making it package specific.
> :)
I think that db_register and db_subst give us pretty much everything that we
would need to implement package-specific prompting. We would need to
reserve our own namespace under the package's template namespace
(package/configtool_blahblah), but I don't see many problems beyond that.
Now that I think about it, since we're prompting in the postinst anyway, we
could probably even share the templates by depending on a common package
which provides them. It should be possible to ensure that they are in the
debconf database that way.
> [generalizing the tools]
>
> However, for trial-run purposes in XFree86, I'll probably just encapsulate
> all of this inside my own package. If the concept flies, it will be
> broken out into its own package.
This seems reasonable to me.
> > - What should happen at preconfiguration time to minimize interaction for
> > novice users?
> >
> > - What sort of preferences would be useful, either at a global scope or a
> > per-package scope? "always leave my foobar config alone" "always merge
> > my changes if there are no merge conflicts"
>
> As debconf is itself configured using debconf questions (default
> frontend and question priority), so too should this config file handler
> be configured when it is installed, giving the user the chance to set a
> global policy for configuration file handling.
Hmm...how will this fit with including the infrastructure in the xfree86
package? Will we have a separate binary package for it, or include it in
each binary package?
> Since package maintainers will not be expected to have to manage this
> sort of thing on their own -- we won't have to trust them to write their
> if statements correctly, in other words :) -- I propose we have shared
> templates like this.
>
> * Template: shared/mdz-config/merge-policy
> Type: select
> Choices: always keep existing config, \
> always use generated config, \
> always merge existing and generate configs, \
> always prompt
>
> The last choice effectively means "handle on a per-package (actually
> per-config-file) basis.
Choosing a default here will be difficult, and this strikes me as the kind
of question that should not be shown to inexperienced users. Of course,
they're not supposed to edit config files, so they shouldn't need a policy
like this, but it has been established that they do anyway, for better or
for worse. The first two choices are never interactive, the third might be
interactive sometimes, and the fourth always interactive. Certainly a
non-interactive choice should be the default if it can be made to do the
right thing often enough.
The user who pastes in a config snippet someone gave them, without knowing
what it does, will certainly not be able to handle a merge conflict. If we
try to merge silently, we need to handle the failure silently as well. It
should work in most cases to keep the existing config and send a note,
except where, for example, the configuration has become incompatible, or the
maintainer's changes are needed in order for a package to continue to work
correctly with other packages.
I have not thought about whether it would be feasible to have a mechanism
for tracking this kind of incompatibility. It seems excessivevly ambitious
at this point.
> * Template: shared/mdz-config/diff-viewing-policy
> Type: select
> Choices: always view diffs between existing and generated config, \
> only view diffs when merging
>
> I'm not sure we should have an option for "never view diffs even if there
> is a problem merging", as that could cause the system to break.
>
> I'm also not sure it's worthwhile having an "always prompt" choice as we
> do with the merge-policy. That just seems excessively tedious.
Since we're saving config files anyway, it'd be easy to provide some simple
tools for the admin to use to pull up the diffs later anyway, so we only
need a couple of convenient options here.
I had imagined that the interactive merge process (intended exclusively for
people who can understand the 'diff' and 'merge' concepts) would display the
diff immediately, and then present the options for how to proceed. Ideally,
these would be on the same screen, but I don't think that this is possible
with current debconf frontends, so we would need something like:
Choices: proceed with the proposed merge, \
discard the proposed merge and keep the existing configuration, \
discard the proposed merge and use the newly-generated configuration, \
view the diff again
So on a per-config file basis, I see these interfaces:
shared/mdz-config/show-diff
shared/mdz-config/merge
shared/mdz-config/merge-conflict-warning
shared/mdz-config/merge-conflict-resolution
The "expert" process would go something like this:
- if conflicts, do merge-conflict-warning
- show-diff (which would implicitly highlight conflicts)
- if conflicts, merge-conflict-resolution, else merge
"merge" presents options like those above, while merge-conflict-resolution
does not even offer the possibility of using the broken configuration, and
instead provides options like:
Choices: let me resolve the conflicts interactively,
discard the proposed merge and use the existing configuration,
discard the proposed merge and use the generated configuration
The first option would, say, write out the config with conflict markers to a
temporary file, display a confirmation question which is displayed while the
admin edits in another terminal (throw away the merge attempt, or install
it). Alternatively, we could just dump the attempted merge somewhere and
let them edit and install it later. Telling them to go edit it somewhere
else is awkward.
The more I see it, the more I think we need clearer terminology when talking
about the various config files involved (the existing, generated, merged,
merged-with-conflict-markers, etc.). Also, all of these templates should
have soothing words about how nothing bad is going to happen and all of the
configuration files involved will be saved for later review.
> > Implementation issues:
> >
> > - How to store the saved config files, so that it is intuitively obvious to
> > which package they belong, and their installed location, so that they are
> > conveniently available to the admin? This should be a public interface.
>
> Well, there are two ways you could do it:
>
> 1) Make it package specific: /var/lib/mdz-config/packagename, I guess.
> This gets chancy if there are many config files and they are in a
> weird hierarchy in /etc, though, and especially tricky if there are
> multiple config files in the same package with the same basename.
>
> 2) Make it mirror /etc. I.e., /var/lib/mdz-config/X11 would "mirror"
> /etc/X11. This solves the problems in 1) but IMO it's ugly.
>
> There may not be enough packages going hog-wild with generate config
> files for 1) to be a problem, though.
The requirements for configuration files are at least:
- intuitive mapping to installed config files
- intuitive versioning: it should be clear where each file came from, and
what it is (generated, backup of locally edited version, etc.)
I had thought a little about the two options that you propose (and share
your concerns in both cases), and also a third option with its own
tradeoffs: storing the alternate versions next to the installed config files
as is done with conffiles. My biggest quarrel with that is that if we want
intuitive versioning, we end up with big ugly filename extensions like
foo.conf.new-1.2.4-1 and foo.conf.local-1.2.3-2 that will infuriate people
like me who like to have at least three columns of ls(1) output.
I am concerned that if we choose 1), there will definitely one day be a
naming conflict. When that does happen, the convention will not have a
logical way to resolve the conflict while preserving the mapping. This
would suck.
> [adding templates to the package]
> Oh, wait, I know what you mean. We can't just blindly append the
> mdz-config templates to each package's templates file every time the
> package is built. Hmm, yup. Going to need Joey Hess's assistance on
> that problem, but it's not a problem for prototyping this in
> xserver-xfree86.
Right, we'll just be including the templates in the usual way for this
experiment.
--
- mdz
Reply to: