More on migration (was Re: dh_config_model_upgrade: package upgrade with Config::Model)
On Tuesday 08 December 2009 08:58:59 Stefano Zacchiroli wrote:
> Fair enough. So, in your terminology, a model is the schema. How do you
> call instances?
Well, I tend to use 3 entities in my doc:
- the model (perl data structures loaded in Config::Model object).
- Configuration instance (derived from a model). This instance contains a ref
to the root node of the confguration tree. This Config::Model::Instance also
holds some instance attributes (like whether to tolerate or not erroneous
values). Several instances can be attached to one model.
- Configuration tree (attached to one instance by the "root" node). The tree
is a structure of nodes (Config::Model::Node) with leaves
(Config::Model::Value). Let skip other details like hash or list elements.
> I'll use "instances" in the rest of this mail :-) Now
> that that is clear, which kind of migrations of configuration file can
> you provide on top of Config::Model.
All depends on the model and the related read and write methods. From that,
sevaral kind of migrations can be done:
- complete change of syntax (i.e. from INI file to YAML)
- Semantic changes (parameter name, default value ...). The more complex the
change, the more complex the model will become (the model must understand tghe
old structure, the new structure and provide the way to migrate from old to
new structures.)
If you want more details on migration, please provide a small use case, and
I'll be happy to provide a small example of a model with migration features.
> Given that models contain default values, I believe it would be pretty
> easy to provide change of default values, it would be enough to change
> them in m_{i+1}, right?
With one caveat: to preserve idempotency, there's must be no ambiguity on the
content of a parameter: so changing the semantic of a parameter without
changing its name will be difficult.
For instance if old model specifies:
timeout: value in minutes
and the new model specifies:
timeout: value in seconds
When loading the files, there's no way to know if the value is in minutes or
in seconds.
In this case, the new model must provide a new parameter name like:
timeout_s: value in seconds
In this case , the model can specify (in "pseudo-code"):
timeout_s = timeout * 60
> Still, my recurring question has left unanswered, I swear this is the
> last time I pose it. Does Config::Model permit to *programmatically*
> apply more complex modification at upgrade time?
Hmm, I may have badly understood your question before. Here's another approach
to handle upgrades.
> Let's say I want to, at
> the same time during an upgrade:
>
> - change a default value from "yes" to "no"
Heh, "default value" is a loaded notion ;-) There are in fact 2 cases to
cover:
- the default value is encoded by the application (what is called
"upstream_version" in Config::Model)
- the default value must be written in the config file (e.g. Debian's default
for ssh's PermitRootLogin).
The latter case is tricky because we can't know whether this value is provided
by a script (or Config::Model) or by a sysadmin. Debian policy specify to keep
sysadmin value.
That said, here's the migration scenario for case one:
PermitRootLogin => { type => 'leaf', value_type => boolean,
upstream_default => 0 }
New model will contain:
PermitRootLogin => { type => 'leaf', value_type => boolean,
upstream_default => 1 }
For case 2 where the default value must be written in the file (specify
"default" instead of "upstream_default"):
PermitRootLogin => { type => 'leaf', value_type => boolean,
default => 1 }
> - add 1 to a given set of integer values wrt what the user configured by
> hand (e.g. because there was an off-by-one sort of semantics in past
> upstream configuration, which has been changed now)
This one is more difficult because of the ambiguity mentioned above: what if
the user re-installs the package that tries to upgrade from old semantic to
new semantic ? The danger would be to get new = old +2 (or more...) thus
resulting to a bad value.
The only possibility would be to store somewhere the fact that the upgrade was
already done. If you factor in the fact that users do sometime downgrade a
package,this can lead to tricky situations.
In this case, I'd recommend changing the parameter name and use this kind of
model that include a migration:
old_param => {type => 'leaf', value_type => integer, status => 'deprecated'}
new_param => {type => 'leaf', value_type => integer,
migrate_from => { formula => '$old+1',
variables => { old => '- old_param' } ,
},
> The second case is a bit more tricky, because you can easily fix default
> values, but you really want to fix also user-customized values.
Problem is to distinguish between user-customized values and script-customized
values...
> To achieve that, Config::Model would need to provide me (as the package
> maintainer) the ability to write a small perl snippet where I've access
> to the currently installed configuration file (which conforms to the old
> version of the model, let's say m_i), where I can migrate it to the new
> version of the model (let's say m_{i+1}),
You'd do this by loading old values with the new model. This would also ensure
the migration since migration "instructions" are also features of the new
model.
> and also manipulate it as a tree to change the values I want.
You can already do migration + modification in a single command:
config-edit -model FooBar -ui none -save foo=42
Note that this migration is idempotent. You can safely run it several times.
If needed, this can also be done in Perl.
> This of course is just an example, my general question was whether I can
> only migrate from m_i to m_{i+1} or also pipe in between some custom
> programmed logics.
Absolutely. We just need to work out the details using real use cases.
All the best
Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Reply to: