Discussion:
dh_config_model_upgrade: package upgrade with Config::Model
(too old to reply)
Dominique Dumont
2009-12-04 08:08:03 UTC
Permalink
Hello

The last version of libconfig-model-perl (640-3) is now shipped with
dh_config_model_upgrade.

The aim of dh_config_model_upgrade is to provide an easy way for a
debian developer to add better configuration upgrade to the packages
they maintain.

Let's assume that Joe, debian developer, wants to add smooth upgrade
capability to his foobar package.

What will the end user see ?

The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.

What does Joe, debian developer, has to do ?

- ensure that a proper configuration model for the application
contained in foobar exists.
- ensure that foobar.postinst and foobar.config have a #DEBHELPER#
line (if these files exist)
- ensure that foobar.control has a dependency on ${misc:Depends}
- ensure that call dh_config_model_upgrade during package build.
Either:
- run "dh --with config_model"
- directly call dh_config_model_upgrade. See
dh_config_model_upgrade man page for more details

(Simple instructions for CDBS should be provided. Help is welcome there)

How does this work ?

When called, dh_config_model_upgrade will setup foobar package to ask
whether to upgrade the configuration with Config::Model or not.

This requires to:
- add a dependency on libconfig-model-perl in foobar control file
- add or update foobar.config file so that the question is raised by
debconf. This file is generated from config-config-model delivered
by libconfig-model-perl package
- add or update foobar.postinst file so that the configuration is
upgraded if the user answered yes to the question above This file is
generated from postinst-config-model delivered by libconfig-model-perl
package
- last but not least, both foobar.config and foobar.postinst will use
a template delivered by libconfig-model-perl
(libconfig-model-perl.templates) to ask the question to user. This
template contains a ${pkg_name} substitution that will be replaced
with 'foobar' by foobar.config and foobar.postinst (See SUBST
debconf command in debconf-devel(7))

All these operations are handled by dh_config_model_upgrade.

For more informations, see http://wiki.debian.org/PackageConfigUpgrade

Config::Model home page is http://config-model.wiki.sourceforge.net/
User mailing list: http://sourceforge.net/mail/?group_id=155650

Comments are most welcome :-)

All the best
Stefano Zacchiroli
2009-12-04 09:57:51 UTC
Permalink
Post by Dominique Dumont
The last version of libconfig-model-perl (640-3) is now shipped with
dh_config_model_upgrade.
Cool, thanks for the update.
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.
I wonder why the detail about how the conf file is being upgraded should
be relevant to the final user at all. As you observe on the wiki page,
most lusers barely know of the existence of /etc, why should they care
about Config::Model being used? Ideally, whether to use it or not is a
choice of the package maintainer.

Also, assuming that the available configuration on disk matches the old
model assumed by the package, why bother with a question at all? After
all dpkg upgrade "untouched" config files automatically, hopefully with
Config::Model we just introduce a more flexible definition for
"untouched"
Post by Dominique Dumont
- ensure that a proper configuration model for the application
contained in foobar exists.
Exists where? It is not clear to me whether in Config::Model you have
the distinction between meta-models (that live in the Perl lib somehow)
and models that live in the package.

On the same line, it is not clear to me whether the maintainer can
programmatically define updates other than "from model m_1 go to m_2".


Many thanks for your stubborn work on this! :-)
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Neil Williams
2009-12-04 11:35:30 UTC
Permalink
On Fri, 4 Dec 2009 10:57:51 +0100
Post by Stefano Zacchiroli
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.
I wonder why the detail about how the conf file is being upgraded should
be relevant to the final user at all. As you observe on the wiki page,
most lusers barely know of the existence of /etc, why should they care
about Config::Model being used? Ideally, whether to use it or not is a
choice of the package maintainer.
I also don't understand why Config::Model seeks to replace debconf
questions from the package when Policy is that only programs compatible
with debconf-2.0 are allowed to prompt users during package
installation. (Policy 3.9.1). Config::Model is not debconf-2.0
compatible but tries to mangle compatibility by changing or replacing
the question that debconf would raise. The letter of Policy might be
being followed but the spirit of Policy is being broken IMHO.

Either Config::Model should not ask any questions, ever, or it should
ask one question based on the tasksel settings and determine a set of
answers for each package to suit that configuration model - system-wide.

i.e. a bit like pre-seeding in D-I whereby a set of answers to the
debconf questions can be pre-ordained as "suitable for ModelA" and then
fed to debconf to allow packages to not have to ask any questions and
yet still get a final configuration that matches ModelA. ModelA would
then apply system-wide.

For this to work it would have to encompass ALL packages that use
debconf and therefore it would need to be debconf-2.0 compatible and
fully implementable using cdebconf alone. That doesn't need perl, it
just needs a set of answers to the debconf questions and a POSIX shell
snippet to populate the debconf database. Each Model could have a
different range of answers.

This doesn't seem to be what Config::Model is trying to do, so there is
work to do to explain just what Config::Model *is* meant to do and
precisely which packages are expected to use it. The solution is being
described without the problem being explained.
Post by Stefano Zacchiroli
Also, assuming that the available configuration on disk matches the old
model assumed by the package, why bother with a question at all? After
all dpkg upgrade "untouched" config files automatically, hopefully with
Config::Model we just introduce a more flexible definition for
"untouched"
Does Config::Model obey DEBCONF_NONINTERACTIVE_SEEN=true ?

http://wiki.debian.org/Multistrap#Environment
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/
Stefano Zacchiroli
2009-12-04 12:01:39 UTC
Permalink
Post by Neil Williams
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.
I also don't understand why Config::Model seeks to replace debconf
questions from the package when Policy is that only programs compatible
with debconf-2.0 are allowed to prompt users during package
My reading of the above quoted test, is that Config::Model is *not*
seeking to replace debconf; it is going to _use_ it as other packages do
for interacting with the user. Or, to be more precise, my understanding
is that the package helper is going to prepare debconf questions that
will be shipped in the packages which use that helper.

(But again, I'm of the opinion that it should not ask any question in
the first place; see my former message.)

Let's see wait for the proposer to clarify these doubts :-)
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Dominique Dumont
2009-12-04 20:08:18 UTC
Permalink
My reading of the above quoted test, is that Config::Model is not
seeking to replace debconf; it is going to use it as other packages do
for interacting with the user. Or, to be more precise, my understanding
is that the package helper is going to prepare debconf questions that
will be shipped in the packages which use that helper.
Correct.

One possible extension to Config::Model is to use Debconf as a more generic
interface. This could be used to generate Debconf question to fill missing
mandatory values (as GUI or curses interface are generated at run time). I'm
working on this, but it seems quite difficult to generate debconf question
dynamically. Not to mention translation problems...

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Dominique Dumont
2009-12-16 15:51:23 UTC
Permalink
Post by Neil Williams
Does Config::Model obey DEBCONF_NONINTERACTIVE_SEEN=true ?
http://wiki.debian.org/Multistrap#Environment
Once the debconf question is removed, Config::Model will be used in non-
interactive mode (-ui none option) in postinst.

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Dominique Dumont
2009-12-04 17:54:56 UTC
Permalink
Post by Stefano Zacchiroli
I wonder why the detail about how the conf file is being upgraded should
be relevant to the final user at all. As you observe on the wiki page,
most lusers barely know of the existence of /etc, why should they care
about Config::Model being used? Ideally, whether to use it or not is a
choice of the package maintainer.
The idea was to offer the user a possibility to bail out since config-model is
still experimental. But I'm beginning to wonder if this is a good idea...
Post by Stefano Zacchiroli
Post by Dominique Dumont
- ensure that a proper configuration model for the application
contained in foobar exists.
Exists where?
Hmm, good point.

I've added these explanation in libconfig-model-perl README.debian:

- ensure that a proper configuration model for the application
contained in foobar exists. Either:
- find the package containing the relevant model in Debian
(something like libconfig-model-foobar-perl)
- find the package in CPAN and ask debian-perl team to package it
- write the model. In this case, you may want to subscribe to
config-model users list (see below)
Post by Stefano Zacchiroli
It is not clear to me whether in Config::Model you have
the distinction between meta-models (that live in the Perl lib somehow)
and models that live in the package.
The model for any application can be delivered either in a separate perl
package (like libconfig-model-approx) or directly within the package. In the
ebd, the model will be installed in the same place. There's no difference in
the model itself.
Post by Stefano Zacchiroli
On the same line, it is not clear to me whether the maintainer can
programmatically define updates other than "from model m_1 go to m_2".
Most of the time, there won't be limitations on the number of model version
you can jump. There will be limitations if the model evolves rapidly (i.e. the
configuration semantics also evolve rapidly). This kind of fast evolution is
generally avoided except for new projects.
Post by Stefano Zacchiroli
Many thanks for your stubborn work on this! :-)
:-}

All the best

Dominique (from home)
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Stefano Zacchiroli
2009-12-04 18:46:04 UTC
Permalink
Post by Dominique Dumont
The idea was to offer the user a possibility to bail out since
config-model is still experimental. But I'm beginning to wonder if
this is a good idea...
IMO it is not, in the sense that it is quite pointless to put this
choice on the final users. The choice about whether to use it or not
should be of the final *package maintainer*. If its use of config-model
works, fine, otherwise _his_ package will be considered RC buggy as any
other package erroneously fiddling with user conffile.
Post by Dominique Dumont
Hmm, good point.
- ensure that a proper configuration model for the application
- find the package containing the relevant model in Debian
(something like libconfig-model-foobar-perl)
- find the package in CPAN and ask debian-perl team to package it
- write the model. In this case, you may want to subscribe to
config-model users list (see below)
Please detail more the "write the model" case. I want that possibility,
but it is not clear to me whether the custom model can be shipped within
the package needing it at configuration time. (Yes, that of course
means that no-fiddling would be possible in preinst, but I got the
impression that it was already the case.)
Post by Dominique Dumont
Post by Stefano Zacchiroli
On the same line, it is not clear to me whether the maintainer can
programmatically define updates other than "from model m_1 go to m_2".
Most of the time, there won't be limitations on the number of model version
you can jump. There will be limitations if the model evolves rapidly (i.e. the
configuration semantics also evolve rapidly). This kind of fast evolution is
generally avoided except for new projects.
No, that's was not my question, sorry if it was not clear; let me try
again. config-model will have a model for the previous maintainer
version of some conffile, let's call it m_1; then config-model can have
several other models m_2, ... m_n and it can jump up to m_n.

What I want to know is whether config-model allows me not to write a
model m_i I'd like to arrive to in the package, but rather write a small
program snippet which takes m_1 and modify it in some way. That's
basically the different between "here is the model you should arrive to"
(which I understand is supported) and "here is a recipe/program to
obtain the model you should arrive to". Oh, well, I should probably
just study Config::Model!

Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
gregor herrmann
2009-12-04 19:08:57 UTC
Permalink
Post by Stefano Zacchiroli
Oh, well, I should probably
just study Config::Model!
Since you speak French you might be interested in Dominique's
presentation at some French Perl meeting:
http://fpw2009.ubicast.eu/videos/free/64/

Cheers,
gregor
--
.''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
: :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
`. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
`- NP: Beatles: Getting better
Dominique Dumont
2009-12-07 21:34:18 UTC
Permalink
Post by gregor herrmann
Since you speak French you might be interested in Dominique's
http://fpw2009.ubicast.eu/videos/free/64/
Today, I've recorded today an English version of this presentation (with some
enhancements to the slide set). I should be able to make it available next
week (I need to compress the 12GB DV file into something more manageable ;-) )

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Dominique Dumont
2009-12-23 19:17:08 UTC
Permalink
Post by gregor herrmann
Since you speak French you might be interested in Dominique's
http://fpw2009.ubicast.eu/videos/free/64/
For non-french speakers, I've recorded a new session in English. In this
presentation, I've also talked about Augeas and package upgrades.

Here are the slides : http://config-model.sourceforge.net/config-model-en.pdf
And the video: http://config-model.sourceforge.net/config-model-2.avi (280MB)

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont

Dominique Dumont
2009-12-12 17:11:06 UTC
Permalink
Post by Stefano Zacchiroli
Post by Dominique Dumont
The idea was to offer the user a possibility to bail out since
config-model is still experimental. But I'm beginning to wonder if
this is a good idea...
IMO it is not, in the sense that it is quite pointless to put this
choice on the final users. The choice about whether to use it or not
should be of the final *package maintainer*. If its use of config-model
works, fine, otherwise _his_ package will be considered RC buggy as any
other package erroneously fiddling with user conffile.
Ok, I'll remove this question.
Post by Stefano Zacchiroli
Please detail more the "write the model" case. I want that possibility,
but it is not clear to me whether the custom model can be shipped within
the package needing it at configuration time. (Yes, that of course
means that no-fiddling would be possible in preinst, but I got the
impression that it was already the case.)
There are several possibilities:
- either the model can be used also to configure the application. In this
case, the model should contain help text (description, summary...)
- or the model is used only in batch mode for upgrade. Since there's no user
interaction, the help text can be skipped.

In the first case, it's better to ship the model in a separate package and
upload it to CPAN. Debian-perl team will then package it.

In the second case, the model can be shipped only in the debian package.
Post by Stefano Zacchiroli
No, that's was not my question, sorry if it was not clear; let me try
again. config-model will have a model for the previous maintainer
version of some conffile, let's call it m_1; then config-model can have
several other models m_2, ... m_n and it can jump up to m_n.
What I want to know is whether config-model allows me not to write a
model m_i I'd like to arrive to in the package, but rather write a small
program snippet which takes m_1 and modify it in some way. That's
basically the different between "here is the model you should arrive to"
(which I understand is supported) and "here is a recipe/program to
obtain the model you should arrive to".
I can easily add some code so that you can use config-model-edit (provided by
Config::Model::Itself) in such a way.

For instance if you want to change the default value of PermitRootLogin, you
would need to run this line while packaging libconfig-model-openssh-perl:

config-model-edit -model Sshd -save class:Sshd element:PermitRootLogin \
default=no

Does this reply to your question ?

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Stefano Zacchiroli
2009-12-13 16:48:13 UTC
Permalink
Post by Dominique Dumont
In the first case, it's better to ship the model in a separate package and
upload it to CPAN. Debian-perl team will then package it.
In the second case, the model can be shipped only in the debian package.
Well, I believe here you're ignoring the problem of Debian archive bloat
(which might not be a problem for CPAN, but is one for Debian). Still,
from that explanation is clear that in fact packages will be able to
ship their model within themselves, which is good.

As a (very) degenerate case, I can even imagine packages shipping the
model of their conf only for a specific, alebit very tricky,
upgrade. For sure we should not keep this as the motivating example, but
keeping it in mind would, I believe, help maintaining your tool more
easily exploitable.
Post by Dominique Dumont
I can easily add some code so that you can use config-model-edit (provided by
Config::Model::Itself) in such a way.
For instance if you want to change the default value of PermitRootLogin, you
config-model-edit -model Sshd -save class:Sshd element:PermitRootLogin \
default=no
Does this reply to your question ?
Partly yes, because I know now that I've an extra tool that I can
invoke. Can I write my own upgrade script in Perl and have it play with
the models involved in the upgrade (i.e., the old model and the new
model) in the same scripts to migrate from one to the other in some
ad-hoc way?.

Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Dominique Dumont
2009-12-13 18:47:25 UTC
Permalink
Post by Stefano Zacchiroli
Can I write my own upgrade script in Perl and have it play with
the models involved in the upgrade (i.e., the old model and the new
model) in the same scripts to migrate from one to the other in some
ad-hoc way?.
Yes. Using Config::Model::Itself and some (untested) code like (please fasten
your seat belt):


my $meta_model = Config::Model -> new();

my $old_meta_inst
= $meta_model->instance (root_class_name => 'Itself::Model' ,
instance_name => 'old model' ,
);
my $old_meta_root = $old_meta_inst -> config_root ;

# create object to load model from files
my $old_rw_obj = Config::Model::Itself -> new(model_object => $old_meta_root);

# now load old model from old model files
$old_rw_obj -> read_all( model_dir => $old_model_dir,
root_model => 'SomeModel',
) ;

# serialise old model
my $old_dump = $old_meta_root->dump_tree;

# create new meta instance to hold new model
my $new_meta_inst
= $meta_model->instance (root_class_name => 'Itself::Model' ,
instance_name => 'new model' ,
);
# still empty model
my $new_meta_root = $new_meta_inst -> config_root ;

# copy old model in new instance
$new_meta_root->load($old_dump);

# customize new model
$new_meta_root->load('class:SomeModel element:foobar default="new default");

#create another rw object
my $new_rw_obj = Config::Model::Itself -> new(model_object => $new_meta_root);

# write the modified model somewhere else
$new_rw_obj -> write_all(model_dir => $new_model_dir,
root_model => 'SomeModel',
) ;


Depending on what you want to achieve, you may also directly modify the old
model in the program with:

$old_meta_root->load('class:SomeModel element:foobar default="new default");

and write the customised model files somewhere else as shown above.

HTH

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Neil Williams
2009-12-04 10:18:46 UTC
Permalink
On Fri, 4 Dec 2009 09:08:03 +0100
Post by Dominique Dumont
The last version of libconfig-model-perl (640-3) is now shipped with
dh_config_model_upgrade.
The package version doesn't sound like a new project, who is using it
(and why?).

(FYI the upstream CPAN description doesn't answer any of my questions
below either. At no point is a Config Model explained or described or
is any attempt made to explain *why* either the maintainer or the user
would want this in use.)

Is this some kind of tasksel interface where the configuration is
modelled on a particular end-user implementation type (like desktop vs
server)?

Is this an upstream configuration attempt that is trying to work in a
downstream mode?

Is it meant to just be one model per package or a unified model across
all packages?

Just what is the point and what problem are you trying to solve?
Post by Dominique Dumont
The aim of dh_config_model_upgrade is to provide an easy way for a
debian developer to add better configuration upgrade to the packages
they maintain.
I read the announcement and the wiki page and I still don't understand
the aim of the project - you describe the problem but I don't see how
Config::Model solves anything and the existing "descriptions" don't
actually explain what the Model is or why it would be useful.
Post by Dominique Dumont
From an embedded perspective, we certainly don't want every
configurable package depending on perl at package installation /
upgrade time - that's why we have cdebconf.

This is especially important for packages with Priority higher than
optional. If any of those packages start to use this, *I* am going to
be the one who has to patch it out of them, so I need to know how to
replace your code with a static configuration (created by the root
filesystem management scripts) *and* disable the postinsts such that
they can run without perl. ("Essential" has no meaning in embedded
Debian - you might only have a reconfigured busybox, uclibc, a
kernel and {a rebuilt} cdebconf as what Debian currently means by
"Essential".)

It might be as simple as providing an empty shell script to replace
dh_config_model *if* the system is sufficiently fault tolerant.

#!/bin/sh
exit 0

(In essence, your models *must* support not being implemented at all
- bog standard, default configuration just as you get when telling
debconf never to ask any questions. This is how packages are installed
as build dependencies on the buildd's. For embedded purposes, the
packages must also support exactly this config without /usr/bin/perl
existing at all.)
Post by Dominique Dumont
Let's assume that Joe, debian developer, wants to add smooth upgrade
capability to his foobar package.
Why? What is the point?
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.
How is that any more understandable than upgrade foobar package with
max_wait ? It is *less* relevant to the specific package, it makes no
sense and encourages a "who knows? just click yes to get rid of it"
response so common in Windows land.
Post by Dominique Dumont
(Simple instructions for CDBS should be provided. Help is welcome there)
Provide a Makefile snippet that calls what you need, package that and
allow packages to add it to the CDBS debian/rules.
Post by Dominique Dumont
- add a dependency on libconfig-model-perl in foobar control file
For a package that does not currently use perl, that is an unwarranted
change.
Post by Dominique Dumont
- add or update foobar.config file so that the question is raised by
debconf. This file is generated from config-config-model delivered
by libconfig-model-perl package
How is that to be translated?
Post by Dominique Dumont
- add or update foobar.postinst file so that the configuration is
upgraded if the user answered yes to the question above This file is
generated from postinst-config-model delivered by libconfig-model-perl
package
Why hand this task to a module when the maintainer is responsible for
it working and might as well do it in the postinst with direct control
over exactly what questions are asked and how they are translated?
Post by Dominique Dumont
For more informations, see http://wiki.debian.org/PackageConfigUpgrade
Which describes the technical minutiae of the codebase without
explaining the "big picture" benefits of using the model or how the
model proposes to "fix" the (IMHO non-existent) problem that it has
invented.

The commercial world is full of products that exist merely to fix a
problem that the product designer has invented out of thin air. There
are always some who will buy into such nonsense; I don't see Debian
doing that. Please explain the problem you're trying to fix - I don't
see it.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/
gregor herrmann
2009-12-04 15:45:53 UTC
Permalink
Post by Neil Williams
Post by Dominique Dumont
For more informations, see http://wiki.debian.org/PackageConfigUpgrade
Which describes the technical minutiae of the codebase without
explaining the "big picture" benefits of using the model or how the
model proposes to "fix" the (IMHO non-existent) problem that it has
invented.
The problem and the aim are stated in the Introduction section of
this page.

Cheers,
gregor
--
.''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
: :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
`. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
`- NP: Phil Collins: Doesn't Anybody Stay Together
Neil Williams
2009-12-04 16:04:04 UTC
Permalink
On Fri, 4 Dec 2009 16:45:53 +0100
Post by gregor herrmann
Post by Neil Williams
Post by Dominique Dumont
For more informations, see
http://wiki.debian.org/PackageConfigUpgrade
Which describes the technical minutiae of the codebase without
explaining the "big picture" benefits of using the model or how the
model proposes to "fix" the (IMHO non-existent) problem that it has
invented.
The problem and the aim are stated in the Introduction section of
this page.
So it claims but that still doesn't make sense. Merely repeating the
statement without supporting the assertion doesn't help.

Where is the Model?

Who designs the Model?

Is the model package specific or system-wide?

Why bother in the first place?

Just what is the problem that this is trying to solve? Is it the
packages themselves or some "model" of what the system should be
configured to resemble?

If it's the packages, why not just get the packages to explain
themselves more clearly in their debconf questions?

How is the model to be translated? How are the strings used by the
model to be translated? Who is in control of when those strings are
changed?

Why would a model be any clearer than the package-specific messages?
(Unless it stays out of the way entirely and is just an overgrown
system of defaults.)

How is a model meant to make sense to a user when the detail does not?

Why add yet another layer?

'Model' seems to be a completely misleading use of terminology. Why was
it chosen?

debhelper is a build time package, why use it dh_ styles for an
installation based system?
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
Felipe Sateler
2009-12-04 16:29:28 UTC
Permalink
Post by Neil Williams
On Fri, 4 Dec 2009 16:45:53 +0100
Post by gregor herrmann
Post by Neil Williams
Post by Dominique Dumont
For more informations, see
http://wiki.debian.org/PackageConfigUpgrade
Which describes the technical minutiae of the codebase without
explaining the "big picture" benefits of using the model or how the
model proposes to "fix" the (IMHO non-existent) problem that it has
invented.
The problem and the aim are stated in the Introduction section of
this page.
So it claims but that still doesn't make sense. Merely repeating the
statement without supporting the assertion doesn't help.
Where is the Model?
Who designs the Model?
You (or hopefully someone else who had the same config file syntax).
Post by Neil Williams
Is the model package specific or system-wide?
File specific.
Post by Neil Williams
Why bother in the first place?
Because a diff is rather useless for people who don't know what a config
file is.
Post by Neil Williams
Just what is the problem that this is trying to solve?
That people, when faced with the standard question by dpkg of conffile
updates, will adopt the "hmm.... just click enter?" attitude "so common
in the Windows world".
Post by Neil Williams
Is it the
packages themselves or some "model" of what the system should be
configured to resemble?
The model is a description of the syntax of a configuration file. That
way, config::model can read both on-disk and new conffile, abstract out
any irrelevant changes (like whitespace), and then decide how to do the
merge.
Post by Neil Williams
If it's the packages, why not just get the packages to explain
themselves more clearly in their debconf questions?
It is the standard conffile upgrade question that is being avoided.
Post by Neil Williams
How is the model to be translated? How are the strings used by the
model to be translated? Who is in control of when those strings are
changed?
Why would a model be any clearer than the package-specific messages?
(Unless it stays out of the way entirely and is just an overgrown
system of defaults.)
How is a model meant to make sense to a user when the detail does not?
Why add yet another layer?
'Model' seems to be a completely misleading use of terminology. Why was
it chosen?
debhelper is a build time package, why use it dh_ styles for an
installation based system?
dh_ is used at build time. It sets up .config and postinst, like other
dh_ commands.
--
Saludos,
Felipe Sateler
Neil Williams
2009-12-04 16:56:30 UTC
Permalink
On Fri, 04 Dec 2009 13:29:28 -0300
Post by Felipe Sateler
Post by Neil Williams
Where is the Model?
Who designs the Model?
You (or hopefully someone else who had the same config file syntax).
Then it's a config file parser, not a modeller?
Post by Felipe Sateler
Post by Neil Williams
Is the model package specific or system-wide?
File specific.
So why not just fix the package to handle the config file intelligently?
Post by Felipe Sateler
Post by Neil Williams
Why bother in the first place?
Because a diff is rather useless for people who don't know what a
config file is.
That's an argument for fixing the package to allow intelligent upgrades
to it's own config files, not an argument for adding another layer to
the postinst.

My problem with this is that it's fixing the wrong problem.

1. packages shouldn't need the user to look at a diff unless that user
is likely to be a sysadmin who would understand it. i.e. a diff is only
used when the user has made such changes to the file that no other
parsing can fix things. If the user hasn't modified the file
themselves, no package changes should leave the conffile in a
sufficiently bad state to need an extra parser - that's a bug in the
package, not a reason to implement a whole new approach.

2. If this can't be done, just don't update the config file and get the
application to complain or fix things.

Adding more work to the postinst is just going to make life harder.
Post by Felipe Sateler
Post by Neil Williams
Just what is the problem that this is trying to solve?
That people, when faced with the standard question by dpkg of conffile
updates, will adopt the "hmm.... just click enter?" attitude "so
common in the Windows world".
Wrong place to fix that problem IMHO. Solve it before it gets to this
stage or delay until the app needs to use the config and then it can
complain or fix things directly.

No model understands the config file as well as the application itself.
Creating yet another interface is not a solution.
Post by Felipe Sateler
Post by Neil Williams
Is it the
packages themselves or some "model" of what the system should be
configured to resemble?
The model is a description of the syntax of a configuration file. That
way, config::model can read both on-disk and new conffile, abstract
out any irrelevant changes (like whitespace), and then decide how to
do the merge.
Sounds as if you're trying to implement a quilt or git interface onto
conffile management when the idea would be better framed to prevent the
need for such an interface in the first place.
Post by Felipe Sateler
Post by Neil Williams
If it's the packages, why not just get the packages to explain
themselves more clearly in their debconf questions?
It is the standard conffile upgrade question that is being avoided.
No, it's being wrapped. Avoiding it would mean that the conflict
wouldn't be left for dpkg to resolve at all.
Post by Felipe Sateler
Post by Neil Williams
How is the model to be translated? How are the strings used by the
model to be translated? Who is in control of when those strings are
changed?
Why would a model be any clearer than the package-specific messages?
(Unless it stays out of the way entirely and is just an overgrown
system of defaults.)
How is a model meant to make sense to a user when the detail does not?
Why add yet another layer?
'Model' seems to be a completely misleading use of terminology. Why
was it chosen?
Those questions need answers too, BTW.

*Especially* those related to i18n.
Post by Felipe Sateler
Post by Neil Williams
debhelper is a build time package, why use it dh_ styles for an
installation based system?
dh_ is used at build time. It sets up .config and postinst, like other
dh_ commands.
That didn't appear obvious from the initial messages.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
Felipe Sateler
2009-12-04 18:00:55 UTC
Permalink
Post by Neil Williams
On Fri, 04 Dec 2009 13:29:28 -0300
Post by Felipe Sateler
Post by Neil Williams
Where is the Model?
Who designs the Model?
You (or hopefully someone else who had the same config file syntax).
Then it's a config file parser, not a modeller?
Well, there is an abstract model of the config file. So yes, basically
it parses the file into an internal model. I believe the idea is to be
able to migrate automagically from one version to the next without user
intervention. If I'm wrong, please correct me. Because then I have the
same questions as you do :P.
--
Saludos,
Felipe Sateler
Dominique Dumont
2009-12-04 18:30:51 UTC
Permalink
Post by Felipe Sateler
Well, there is an abstract model of the config file. So yes, basically
it parses the file into an internal model. I believe the idea is to be
able to migrate automagically from one version to the next without user
intervention. If I'm wrong, please correct me. Because then I have the
same questions as you do :P.
You're right. (short answer because I must look after the kids :-p )

all the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Stefano Zacchiroli
2009-12-04 16:48:00 UTC
Permalink
[ reordering some quoted text ]
Post by Neil Williams
So it claims but that still doesn't make sense. Merely repeating the
statement without supporting the assertion doesn't help.
Well, reading your posts I understand there is in fact a
misunderstanding. The question mentioned in the reported wiki page has
nothing to do with a debconf question: is the question posed by dpkg
when there is a mismatch between an on-disk configuration file and the
old version of the maintainer configuration file. Try re-reading the
wiki page with that in mind to see if it makes a bit more sense (it does
to me at least).

The source of the confusion is probably the detailed debconf mention
(which was unrelated and IMO unnecessary) in the proposer mail at the
top of this thread.
Post by Neil Williams
Where is the Model?
Who designs the Model?
These are question I've posed my self already in the thread. Again, can
we please leave the proposer the time to reply to those? Merely
repeating the questions will not help :-)
Post by Neil Williams
'Model' seems to be a completely misleading use of terminology. Why was
it chosen?
I believe the author is using the model term in the same it is used in
model-driven engineering [1]. *If* it is the case (I don't actually know
if it is, but with that assumption in mind the terminology makes sense),
a model is essentially an "abstract syntax tree"-like representation of
a specific configuration file. Furthermore, classes of configuration
file have a grammar in common (or "meta-model", in MDE terminology).

[1] http://en.wikipedia.org/wiki/Model-driven_engineering
Post by Neil Williams
Is the model package specific or system-wide?
According to the above assumptions, a model is just an abstract version
of a specific config file, with some specific data in it. A meta-model
is specific of a whole class of configuration files (e.g. fstab, apache
conf-file, postfix map file, etc.).
Post by Neil Williams
Why bother in the first place?
<snip>
Post by Neil Williams
Just what is the problem that this is trying to solve? Is it the
That, on the contrary, I feel is quite clear.

Right now, dpkg only knows about byte to byte equivalence among
different versions of the same configuration file. Hence I can be adding
a completely useless space to a conffile and dpkg will bother me at the
next upgrade, because it cannot distinguish among "meaningless" and
"meaningful" changes.

If you have a model of the old and of the new conffile, then you can
perform a more structured comparison, which ignores all semantically
irrelevant details.

The proposal apparently goes even further, and attempt some kind of
"semantic" merge between changes performed with some sysadm tool and
maintainer changes. Whether this merge is dumb or programmable by the
maintainer is another question I've already raised in this thread.

Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Neil Williams
2009-12-04 17:41:14 UTC
Permalink
On Fri, 4 Dec 2009 17:48:00 +0100
Post by Stefano Zacchiroli
Post by Neil Williams
So it claims but that still doesn't make sense. Merely repeating the
statement without supporting the assertion doesn't help.
Well, reading your posts I understand there is in fact a
misunderstanding. The question mentioned in the reported wiki page has
nothing to do with a debconf question: is the question posed by dpkg
when there is a mismatch between an on-disk configuration file and the
old version of the maintainer configuration file. Try re-reading the
wiki page with that in mind to see if it makes a bit more sense (it
does to me at least).
It is beginning to make some sense but the problems do not outweigh the
(frankly wishlist) benefits so far outlined. If the package isn't
buggy, it won't need a model; I think ::Model is just confusing.

If it stays in leaf packages Priority: optional or extra, maybe. If it
starts adding a perl dependency to packages included by a typical
debootstrap, embedded systems are going to have problems. (Problems I
need to fix.)
Post by Stefano Zacchiroli
Post by Neil Williams
'Model' seems to be a completely misleading use of terminology. Why
was it chosen?
I believe the author is using the model term in the same it is used in
model-driven engineering [1]. *If* it is the case (I don't actually
know if it is, but with that assumption in mind the terminology makes
sense), a model is essentially an "abstract syntax tree"-like
representation of a specific configuration file. Furthermore, classes
of configuration file have a grammar in common (or "meta-model", in
MDE terminology).
Like Model:View:Controller - I understand that.

Still doesn't mean (to me at least) that any package needs this support
unless it is already buggy. It isn't fixing the bug, particularly if
the package is not already written in perl.
Post by Stefano Zacchiroli
Right now, dpkg only knows about byte to byte equivalence among
different versions of the same configuration file. Hence I can be
adding a completely useless space to a conffile and dpkg will bother
me at the next upgrade, because it cannot distinguish among
"meaningless" and "meaningful" changes.
True - but don't you think that by the time dpkg is handed this
situation, the bug has already occurred because the file has been
modified in an incompatible way and not by the user?
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
Stefano Zacchiroli
2009-12-06 11:22:04 UTC
Permalink
Post by Neil Williams
It is beginning to make some sense but the problems do not outweigh the
(frankly wishlist) benefits so far outlined. If the package isn't
buggy, it won't need a model
No, not really. The package can be perfectly fine and still benefit of a
model, for instance when some external program is used to customize the
configuration (e.g.: web or GUI interfaces for newbies). And actually
also power users can benefit from this kind of changes, for instance I
would love to have automatic merges of comment/space changes in most
configuration files I've under /etc.


I believe most of your other comments in this follow-up were based on
the assumption that this stuff is useful only in case of bug, which I
don't believe it is the case. Ping me if I missed some comment based on
different assumptions.

Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Dominique Dumont
2009-12-07 21:57:56 UTC
Permalink
Post by Stefano Zacchiroli
Well, reading your posts I understand there is in fact a
misunderstanding. The question mentioned in the reported wiki page has
nothing to do with a debconf question: is the question posed by dpkg
when there is a mismatch between an on-disk configuration file and the
old version of the maintainer configuration file. Try re-reading the
wiki page with that in mind to see if it makes a bit more sense (it does
to me at least).
Well, I may have not been very clear about debconf usage with Config::Mode in
the Wiki.

I see 2 ways for Config::Model to use Debconf:

1. As proposed in the beginning of this thread, use debconf to ask a global
question whether to use Config::Model or not. As mentioned by Stefano, this is
a good idea. I agree (now) so I'll remove this part

2. Use Debconf to query missing configuration data (i.e. a mandatory value
without default value). But we're not there yet.
Post by Stefano Zacchiroli
Post by Neil Williams
Where is the Model?
Who designs the Model?
These are question I've posed my self already in the thread. Again, can
we please leave the proposer the time to reply to those? Merely
repeating the questions will not help :-)
In this case, the Model can be written by the upstream project, or by the
packager. But each model is dedicated to an application. I grant you that's
quite an effort, but not much more than writing a lens for Augeas. An Augeas
has already quite of lot of lenses written by the community. So this kind of
effort is possible by the same kind of community. Now, it's up to me to
convince this community that providing configuration upgrade ( and
configuration GUI ) is worth the effort.
Post by Stefano Zacchiroli
Post by Neil Williams
'Model' seems to be a completely misleading use of terminology. Why was
it chosen?
I believe the author is using the model term in the same it is used in
model-driven engineering [1]. *If* it is the case (I don't actually know
if it is, but with that assumption in mind the terminology makes sense),
a model is essentially an "abstract syntax tree"-like representation of
a specific configuration file. Furthermore, classes of configuration
file have a grammar in common (or "meta-model", in MDE terminology).
A model for Config::Model is to a configuration file what a schema is to an
XML file: a description of the structure and constraint of the semantic data
of the file.

That's big talk. So here's a very simple example extracted from Sshd model.
This example describes the TCPKeepAlive parameter:

'TCPKeepAlive' =>
{
'value_type' => 'enum',
'help' => {
'yes' => 'Send TCP keepalive messages. The server will notice if
the network goes down or the client host crashes. This avoids infinitely
hanging sessions.',
'no' => 'disable TCP keepalive messages'
},
'upstream_default' => 'yes',
'type' => 'leaf',
'description' => 'Specifies whether the system should send TCP keepalive
messages to the other side. If they are sent, death of the connection or crash
of one of the machines will be properly noticed. However, this means that
connections will die if the route is down temporarily, and some people find it
annoying. On the other hand, if TCP keepalives are not sent, sessions may
hang indefinitely on the server, leaving "ghost" users and consuming server
resources. This option was formerly called KeepAlive.',
'choice' => ['no','yes']
},
Post by Stefano Zacchiroli
Post by Neil Williams
Is the model package specific or system-wide?
According to the above assumptions, a model is just an abstract version
of a specific config file, with some specific data in it.
A meta-model is specific of a whole class of configuration files (e.g.
fstab, apache conf-file, postfix map file, etc.).

A configuration model is like a set of Class definition in C++, and the
configuration tree is an instance of the root class in the model. There are
like a set of configuration objects arranged in a tree structure from the
model (the set of classes)

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Stefano Zacchiroli
2009-12-08 07:58:59 UTC
Permalink
Post by Dominique Dumont
A model for Config::Model is to a configuration file what a schema is to an
XML file: a description of the structure and constraint of the semantic data
of the file.
Fair enough. So, in your terminology, a model is the schema. How do you
call instances? I'll use "instances" in the rest of this mail :-) Now
that that is clear, which kind of migrations of configuration file can
you provide on top of Config::Model.

Given that models contain default values, I believe it would be pretty
easy to provide change of default values, it would be enough to change
them in m_{i+1}, right?

Still, my recurring question has left unanswered, I swear this is the
last time I pose it. Does Config::Model permit to *programmatically*
apply more complex modification at upgrade time? Let's say I want to, at
the same time during an upgrade:

- change a default value from "yes" to "no"
- add 1 to a given set of integer values wrt what the user configured by
hand (e.g. because there was an off-by-one sort of semantics in past
upstream configuration, which has been changed now)

The second case is a bit more tricky, because you can easily fix default
values, but you really want to fix also user-customized values.

To achieve that, Config::Model would need to provide me (as the package
maintainer) the ability to write a small perl snippet where I've access
to the currently installed configuration file (which conforms to the old
version of the model, let's say m_i), where I can migrate it to the new
version of the model (let's say m_{i+1}), and also manipulate it as a
tree to change the values I want.

This of course is just an example, my general question was whether I can
only migrate from m_i to m_{i+1} or also pipe in between some custom
programmed logics.

TIA.
Cheers.
--
Stefano Zacchiroli -o- PhD in Computer Science \ PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c'è ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu à tous ceux que j'aime
Dominique Dumont
2009-12-14 13:14:31 UTC
Permalink
Post by Stefano Zacchiroli
Fair enough. So, in your terminology, a model is the schema. How do you
call instances?
Well, I tend to use 3 entities in my doc:
- the model (perl data structures loaded in Config::Model object).
- Configuration instance (derived from a model). This instance contains a ref
to the root node of the confguration tree. This Config::Model::Instance also
holds some instance attributes (like whether to tolerate or not erroneous
values). Several instances can be attached to one model.
- Configuration tree (attached to one instance by the "root" node). The tree
is a structure of nodes (Config::Model::Node) with leaves
(Config::Model::Value). Let skip other details like hash or list elements.
Post by Stefano Zacchiroli
I'll use "instances" in the rest of this mail :-) Now
that that is clear, which kind of migrations of configuration file can
you provide on top of Config::Model.
All depends on the model and the related read and write methods. From that,
sevaral kind of migrations can be done:
- complete change of syntax (i.e. from INI file to YAML)
- Semantic changes (parameter name, default value ...). The more complex the
change, the more complex the model will become (the model must understand tghe
old structure, the new structure and provide the way to migrate from old to
new structures.)

If you want more details on migration, please provide a small use case, and
I'll be happy to provide a small example of a model with migration features.
Post by Stefano Zacchiroli
Given that models contain default values, I believe it would be pretty
easy to provide change of default values, it would be enough to change
them in m_{i+1}, right?
With one caveat: to preserve idempotency, there's must be no ambiguity on the
content of a parameter: so changing the semantic of a parameter without
changing its name will be difficult.

For instance if old model specifies:
timeout: value in minutes
and the new model specifies:
timeout: value in seconds

When loading the files, there's no way to know if the value is in minutes or
in seconds.

In this case, the new model must provide a new parameter name like:

timeout_s: value in seconds

In this case , the model can specify (in "pseudo-code"):
timeout_s = timeout * 60
Post by Stefano Zacchiroli
Still, my recurring question has left unanswered, I swear this is the
last time I pose it. Does Config::Model permit to *programmatically*
apply more complex modification at upgrade time?
Hmm, I may have badly understood your question before. Here's another approach
to handle upgrades.
Post by Stefano Zacchiroli
Let's say I want to, at
- change a default value from "yes" to "no"
Heh, "default value" is a loaded notion ;-) There are in fact 2 cases to
cover:
- the default value is encoded by the application (what is called
"upstream_version" in Config::Model)
- the default value must be written in the config file (e.g. Debian's default
for ssh's PermitRootLogin).

The latter case is tricky because we can't know whether this value is provided
by a script (or Config::Model) or by a sysadmin. Debian policy specify to keep
sysadmin value.

That said, here's the migration scenario for case one:

PermitRootLogin => { type => 'leaf', value_type => boolean,
upstream_default => 0 }

New model will contain:

PermitRootLogin => { type => 'leaf', value_type => boolean,
upstream_default => 1 }

For case 2 where the default value must be written in the file (specify
"default" instead of "upstream_default"):
PermitRootLogin => { type => 'leaf', value_type => boolean,
default => 1 }
Post by Stefano Zacchiroli
- add 1 to a given set of integer values wrt what the user configured by
hand (e.g. because there was an off-by-one sort of semantics in past
upstream configuration, which has been changed now)
This one is more difficult because of the ambiguity mentioned above: what if
the user re-installs the package that tries to upgrade from old semantic to
new semantic ? The danger would be to get new = old +2 (or more...) thus
resulting to a bad value.

The only possibility would be to store somewhere the fact that the upgrade was
already done. If you factor in the fact that users do sometime downgrade a
package,this can lead to tricky situations.

In this case, I'd recommend changing the parameter name and use this kind of
model that include a migration:

old_param => {type => 'leaf', value_type => integer, status => 'deprecated'}

new_param => {type => 'leaf', value_type => integer,
migrate_from => { formula => '$old+1',
variables => { old => '- old_param' } ,
},
Post by Stefano Zacchiroli
The second case is a bit more tricky, because you can easily fix default
values, but you really want to fix also user-customized values.
Problem is to distinguish between user-customized values and script-customized
values...
Post by Stefano Zacchiroli
To achieve that, Config::Model would need to provide me (as the package
maintainer) the ability to write a small perl snippet where I've access
to the currently installed configuration file (which conforms to the old
version of the model, let's say m_i), where I can migrate it to the new
version of the model (let's say m_{i+1}),
You'd do this by loading old values with the new model. This would also ensure
the migration since migration "instructions" are also features of the new
model.
Post by Stefano Zacchiroli
and also manipulate it as a tree to change the values I want.
You can already do migration + modification in a single command:

config-edit -model FooBar -ui none -save foo=42

Note that this migration is idempotent. You can safely run it several times.

If needed, this can also be done in Perl.
Post by Stefano Zacchiroli
This of course is just an example, my general question was whether I can
only migrate from m_i to m_{i+1} or also pipe in between some custom
programmed logics.
Absolutely. We just need to work out the details using real use cases.

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Dominique Dumont
2009-12-04 16:10:20 UTC
Permalink
Post by Neil Williams
The package version doesn't sound like a new project, who is using it
(and why?).
Sorry, that's a typo. The version is 0.640-3. The project is new.
Post by Neil Williams
(FYI the upstream CPAN description doesn't answer any of my questions
below either. At no point is a Config Model explained or described or
is any attempt made to explain *why* either the maintainer or the user
would want this in use.)
Fair enough. I've updated the introduction of Config::Model wiki on
SourceForge. [1] I hope that this will convince you of the value of
Config::Model .
Post by Neil Williams
Is this some kind of tasksel interface where the configuration is
modelled on a particular end-user implementation type (like desktop vs
server)?
There's no relation to apt tasksel.
Post by Neil Williams
Is this an upstream configuration attempt that is trying to work in a
downstream mode?
Err... Assuming I understand your question, this is an upstream project that
tries to help downstream packagers.
Post by Neil Williams
Is it meant to just be one model per package or a unified model across
all packages?
For now, it's one model per application that require a configuration. Package
without config must not use dh_config_model_upgrade

We can imagine to glue models together to provide system wide semantic
validation, (i.e. application A that must work with B is correctly configured
to work with B, and vice-versa), but we're not there yet.
Post by Neil Williams
Just what is the point and what problem are you trying to solve?
The main goal is to lower the cost (in terms of time) of providing
configuration tools to end user. So that developers (or packagers) will be
able to afford the creation and maintenance of more graphical configuration
tools (with wizard, integrated help ...).

If we can manage that, I think that most casual users (i.e. non-geek) will be
grateful.

Another goal is to avoid casual users seeing things like this:

Configuration file `/etc/sensors3.conf'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
What would you like to do about it ? Your options are:
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : background this process to examine the situation
The default action is to keep your current version.
*** sensors3.conf (Y/I/N/O/D/Z) [default=N] ?

If my mother-in-law sees this, I'm sure to get a phone call ;-)

This can of situation can be handled automatically.

Using config-model could also help in reducing the number of lline of openssh-
server postinst script (480 lines) by removing the code dedicated to upgrades.
Post by Neil Williams
Post by Dominique Dumont
The aim of dh_config_model_upgrade is to provide an easy way for a
debian developer to add better configuration upgrade to the packages
they maintain.
I read the announcement and the wiki page and I still don't understand
the aim of the project - you describe the problem but I don't see how
Config::Model solves anything and the existing "descriptions" don't
actually explain what the Model is or why it would be useful.
I hope [1] will help in explaining the goal.
Post by Neil Williams
From an embedded perspective, we certainly don't want every
configurable package depending on perl at package installation /
upgrade time - that's why we have cdebconf.
Embedded world has other constraint. Does the user often store configuration
data in embedded device ?
Post by Neil Williams
This is especially important for packages with Priority higher than
optional. If any of those packages start to use this, *I* am going to
be the one who has to patch it out of them, so I need to know how to
replace your code with a static configuration (created by the root
filesystem management scripts) *and* disable the postinsts such that
they can run without perl. ("Essential" has no meaning in embedded
Debian - you might only have a reconfigured busybox, uclibc, a
kernel and {a rebuilt} cdebconf as what Debian currently means by
"Essential".)
dh_config_model is currently involved only with configuration upgrades.

It could also be used to generate at build time a generic configuration file.
But that's not the case right now. Would that help you ?
Post by Neil Williams
It might be as simple as providing an empty shell script to replace
dh_config_model *if* the system is sufficiently fault tolerant.
In this case, configuration files will simply be left as is. (which is also
the case if debconf variable libconfig-model-perl/automatic-upgrade is false)
Post by Neil Williams
(In essence, your models *must* support not being implemented at all
- bog standard, default configuration just as you get when telling
debconf never to ask any questions. This is how packages are installed
as build dependencies on the buildd's. For embedded purposes, the
packages must also support exactly this config without /usr/bin/perl
existing at all.)
I'm still not clear on how configuration data is handled in embedded
devices...
Post by Neil Williams
Post by Dominique Dumont
Let's assume that Joe, debian developer, wants to add smooth upgrade
capability to his foobar package.
Why? What is the point?
Avoid asking users configuration questions during packages upgrades. This
could make dist-upgrades between Debian releases easier.
Post by Neil Williams
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking him
whether to upgrade foobar package with Config::Model. Hopefully,
that's all the end user will see.
How is that any more understandable than upgrade foobar package with
max_wait ? It is *less* relevant to the specific package, it makes no
sense and encourages a "who knows? just click yes to get rid of it"
response so common in Windows land.
This question has a 'yes' default value. Currently, config-model is
experimental, and this provides the user a way to skip upgrade with config-
model is something goes wrong.

If the consensus is to remove this debconf question, I will.
Post by Neil Williams
Post by Dominique Dumont
(Simple instructions for CDBS should be provided. Help is welcome there)
Provide a Makefile snippet that calls what you need, package that and
allow packages to add it to the CDBS debian/rules.
ok. I'll dig this further.
Post by Neil Williams
Post by Dominique Dumont
- add a dependency on libconfig-model-perl in foobar control file
For a package that does not currently use perl, that is an unwarranted
change.
That's depend on how painful is the configuration upgrade for user. For
instance, if you have saved cups config file through the web interface, you
can no longer upgrade cupsd without answering a ucf question. That's not cool.
Post by Neil Williams
Post by Dominique Dumont
- add or update foobar.config file so that the question is raised by
debconf. This file is generated from config-config-model delivered
by libconfig-model-perl package
How is that to be translated?
In libconfig-model-perl package.
Post by Neil Williams
Post by Dominique Dumont
- add or update foobar.postinst file so that the configuration is
upgraded if the user answered yes to the question above This file is
generated from postinst-config-model delivered by libconfig-model-perl
package
Why hand this task to a module when the maintainer is responsible for
it working and might as well do it in the postinst with direct control
over exactly what questions are asked and how they are translated?
Less code to maintain. But that's the maintainer choice whether to control it
directly or use a tool set.
Post by Neil Williams
Post by Dominique Dumont
For more informations, see http://wiki.debian.org/PackageConfigUpgrade
Which describes the technical minutiae of the codebase without
explaining the "big picture" benefits of using the model or how the
model proposes to "fix" the (IMHO non-existent) problem that it has
invented.
Okay, I'll re-work the introduction part of this page.
Post by Neil Williams
The commercial world is full of products that exist merely to fix a
problem that the product designer has invented out of thin air. There
are always some who will buy into such nonsense; I don't see Debian
doing that. Please explain the problem you're trying to fix - I don't
see it.
I hope I provided enough explanations.

All the best

Dominique

[1] http://config-model.wiki.sourceforge.net/

--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Neil Williams
2009-12-04 17:32:13 UTC
Permalink
On Fri, 4 Dec 2009 17:10:20 +0100
Post by Dominique Dumont
Post by Neil Williams
(FYI the upstream CPAN description doesn't answer any of my
questions below either. At no point is a Config Model explained or
described or is any attempt made to explain *why* either the
maintainer or the user would want this in use.)
Fair enough. I've updated the introduction of Config::Model wiki on
SourceForge. [1] I hope that this will convince you of the value of
Config::Model .
It could do with some level of examples of just what kind of packages
are sensible candidates. Packages like exim4 and postfix are too
complex for some external parser to create a new interface and the kind
of changes needed are always beyond casual users. OTOH minor packages
shouldn't need this kind of support in the first place - it's a bug in
the package.
Post by Dominique Dumont
Post by Neil Williams
Is this some kind of tasksel interface where the configuration is
modelled on a particular end-user implementation type (like desktop
vs server)?
There's no relation to apt tasksel.
Then why it is a model not yet-another-parser?

It should be Config::Buggy::Parser

i.e. a parser for buggy configs that upstream can't be bothered to fix
properly.
Post by Dominique Dumont
Post by Neil Williams
Is this an upstream configuration attempt that is trying to work in
a downstream mode?
Err... Assuming I understand your question, this is an upstream
project that tries to help downstream packagers.
No, I meant why is this an upstream package that is trying to do the
job that packages themselves should do themselves and which Debian
(downstream) has to fix when those packages get it wrong.
Post by Dominique Dumont
Post by Neil Williams
Just what is the point and what problem are you trying to solve?
The main goal is to lower the cost (in terms of time) of providing
configuration tools to end user. So that developers (or packagers)
will be able to afford the creation and maintenance of more graphical
configuration tools (with wizard, integrated help ...).
If we can manage that, I think that most casual users (i.e. non-geek)
will be grateful.
Better to fix the packages themselves to have more intelligent config
file handling internally - *without* adding another dependency.

*IF* the package already needs perl, *maybe* it could use this. Adding
it to packages that do not currently need perl is unwarranted,
especially when perl interferes with package installation.
Post by Dominique Dumont
Configuration file `/etc/sensors3.conf'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : background this process to examine the situation
The default action is to keep your current version.
*** sensors3.conf (Y/I/N/O/D/Z) [default=N] ?
That is a bug in the package, as described in my other reply. If the
user hasn't changed that file, the package should be able to cope with
the changes without prompting. Whichever package caused the file to be
modified in such a way that it cannot be upgraded cleanly is BUGGY.

Trying to fix that bug by adding perl *as a runtime dependency* (or
worse, as a pre-depends) to a non-perl package is just not helpful. Do
what you like with perl at build time, I care about not adding perl to
runtime situations that haven't needed it before.

(Don't get me wrong, I like perl and I use it a lot.)
Post by Dominique Dumont
Using config-model could also help in reducing the number of lline of
openssh- server postinst script (480 lines) by removing the code
dedicated to upgrades.
but then adding an entire dependency. :-(
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
The aim of dh_config_model_upgrade is to provide an easy way for a
debian developer to add better configuration upgrade to the
packages they maintain.
I read the announcement and the wiki page and I still don't
understand the aim of the project - you describe the problem but I
don't see how Config::Model solves anything and the existing
"descriptions" don't actually explain what the Model is or why it
would be useful.
I hope [1] will help in explaining the goal.
Only slightly. I still don't see a real use for it.
Post by Dominique Dumont
Post by Neil Williams
From an embedded perspective, we certainly don't want every
configurable package depending on perl at package installation /
upgrade time - that's why we have cdebconf.
Embedded world has other constraint. Does the user often store
configuration data in embedded device ?
Yes. Think iPAQ. Specific changes are often needed
in /etc/fstab, /etc/inittab and other areas - the rest just need to
upgrade cleanly.

What would be better with embedded stuff is a way to never allow these
dpkg conflicts to arise, or failing that some way to just assume 'Y'.
Post by Dominique Dumont
Post by Neil Williams
This is especially important for packages with Priority higher than
optional. If any of those packages start to use this, *I* am going
to be the one who has to patch it out of them, so I need to know
how to replace your code with a static configuration (created by
the root filesystem management scripts) *and* disable the postinsts
such that they can run without perl. ("Essential" has no meaning in
embedded Debian - you might only have a reconfigured busybox,
uclibc, a kernel and {a rebuilt} cdebconf as what Debian currently
means by "Essential".)
dh_config_model is currently involved only with configuration
upgrades.
It could also be used to generate at build time a generic
configuration file. But that's not the case right now. Would that
help you ?
No, because it's perl. It needs to be without perl - i.e. within the
package and using the support within the package.
Post by Dominique Dumont
Post by Neil Williams
(In essence, your models *must* support not being implemented at all
- bog standard, default configuration just as you get when telling
debconf never to ask any questions. This is how packages are
installed as build dependencies on the buildd's. For embedded
purposes, the packages must also support exactly this config
without /usr/bin/perl existing at all.)
I'm still not clear on how configuration data is handled in embedded
devices...
1. Customised installation / root filesystem creation tools create
specialised configuration files for this specific type of device at
installation.

2. Package updates must not break these configurations *or* halt the
package upgrade process.

Other than that, it's normal conffile handling.

(Some embedded systems do have perl, it's just that perl cannot be
assumed to exist.)
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
Let's assume that Joe, debian developer, wants to add smooth
upgrade capability to his foobar package.
Why? What is the point?
Avoid asking users configuration questions during packages upgrades.
This could make dist-upgrades between Debian releases easier.
No, it hides the problem inside another dependency. The package should
handle it's own config files *and* the bugs that result.

If that package is already perl, maybe an extra perl module isn't a big
deal. When that package is not perl and is a Priority: important or
higher, it will be a problem.
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
The end user may have to answer a medium debconf question asking
him whether to upgrade foobar package with Config::Model.
Hopefully, that's all the end user will see.
How is that any more understandable than upgrade foobar package with
max_wait ? It is *less* relevant to the specific package, it makes
no sense and encourages a "who knows? just click yes to get rid of
it" response so common in Windows land.
This question has a 'yes' default value. Currently, config-model is
experimental, and this provides the user a way to skip upgrade with
config- model is something goes wrong.
That doesn't answer my question. Encouraging blind clicks to 'yes' isn't
helpful.
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
- add a dependency on libconfig-model-perl in foobar control file
For a package that does not currently use perl, that is an
unwarranted change.
That's depend on how painful is the configuration upgrade for user.
No, it is unwarranted. Fixing the wrong bug.
Post by Dominique Dumont
For instance, if you have saved cups config file through the web
interface, you can no longer upgrade cupsd without answering a ucf
question. That's not cool.
That's a bug in the cups package and the web interface, no need to
require perl. It's bad enough with defoma (which is to be going away at
some point).
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
- add or update foobar.config file so that the question is raised
by debconf. This file is generated from config-config-model
delivered by libconfig-model-perl package
How is that to be translated?
In libconfig-model-perl package.
How is that to be package-specific and actually make sense to the user?
Post by Dominique Dumont
Post by Neil Williams
Post by Dominique Dumont
- add or update foobar.postinst file so that the configuration is
upgraded if the user answered yes to the question above This
file is generated from postinst-config-model delivered by
libconfig-model-perl package
Why hand this task to a module when the maintainer is responsible
for it working and might as well do it in the postinst with direct
control over exactly what questions are asked and how they are
translated?
Less code to maintain. But that's the maintainer choice whether to
control it directly or use a tool set.
I still think this is being over-played and has a very low usefulness
count compared to actually fixing the packages directly.

By the time the package needs Config::Model it's already too late, the
bug has occurred. Hence Config::Bug::Parser.

There shouldn't be a need for any 'model' if the package isn't buggy.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
gregor herrmann
2009-12-04 18:25:54 UTC
Permalink
Post by Neil Williams
Post by Dominique Dumont
Configuration file `/etc/sensors3.conf'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : background this process to examine the situation
The default action is to keep your current version.
*** sensors3.conf (Y/I/N/O/D/Z) [default=N] ?
That is a bug in the package, as described in my other reply. If the
user hasn't changed that file, the package should be able to cope with
the changes without prompting. Whichever package caused the file to be
modified in such a way that it cannot be upgraded cleanly is BUGGY.
If the file is unmodified, then there's indeed something wrong. But
the interesting case is when the admin has changed something. The
common case that is getting on my nerves currently is:

* foo 1.0 ships /etc/foo.conf with
featureA=yes
featureB=no
* I change featureB to yes
* foo 2.0 ships /etc/foo.conf with
featureA=yes
featureB=no
featureC=7
* Now dpkg ask if I want the new version or my modified one. But what
I actually want is
- keep my change of the featureB variable and
- add the new featureC variable
* Or more generally: I want to preserve my locally modified values
and at the same time update the rest which I haven't touched.

And that's what Config::Model allows, as far as I understood it.
Post by Neil Williams
Post by Dominique Dumont
Avoid asking users configuration questions during packages upgrades.
This could make dist-upgrades between Debian releases easier.
No, it hides the problem inside another dependency. The package should
handle it's own config files *and* the bugs that result.
I don't consider writing parsers in each and every .postinst very
efficient.

Cheers,
gregor
--
.''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
: :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
`. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
`- NP: Bob Dylan: Like A Rolling Stone
Neil Williams
2009-12-04 18:38:19 UTC
Permalink
On Fri, 4 Dec 2009 19:25:54 +0100
Post by gregor herrmann
Post by Neil Williams
Post by Dominique Dumont
Configuration file `/etc/sensors3.conf'
==> File on system created by you or by a script.
==> File also in package provided by package maintainer.
Y or I : install the package maintainer's version
N or O : keep your currently-installed version
D : show the differences between the versions
Z : background this process to examine the situation
The default action is to keep your current version.
*** sensors3.conf (Y/I/N/O/D/Z) [default=N] ?
That is a bug in the package, as described in my other reply. If the
user hasn't changed that file, the package should be able to cope
with the changes without prompting. Whichever package caused the
file to be modified in such a way that it cannot be upgraded
cleanly is BUGGY.
If the file is unmodified, then there's indeed something wrong. But
the interesting case is when the admin has changed something. The
* foo 1.0 ships /etc/foo.conf with
featureA=yes
featureB=no
* I change featureB to yes
* foo 2.0 ships /etc/foo.conf with
featureA=yes
featureB=no
featureC=7
* Now dpkg ask if I want the new version or my modified one. But what
I actually want is
- keep my change of the featureB variable and
- add the new featureC variable
* Or more generally: I want to preserve my locally modified values
and at the same time update the rest which I haven't touched.
That's a user change, I thought the point of this was that changes *not
done by users* are causing problems. Different problem.
Post by gregor herrmann
And that's what Config::Model allows, as far as I understood it.
That's not how I understood the purposes of it, although it could be
used that way, it isn't sufficient justification IMHO.

If admins make changes, admins should expect to handle the dpkg
conflicts.

I thought the idea of this was to handle changes not made by users - I
think we agree that those instances are bugs.
Post by gregor herrmann
Post by Neil Williams
Post by Dominique Dumont
Avoid asking users configuration questions during packages
upgrades. This could make dist-upgrades between Debian releases
easier.
No, it hides the problem inside another dependency. The package
should handle it's own config files *and* the bugs that result.
I don't consider writing parsers in each and every .postinst very
efficient.
As is adding a perl dependency to packages that have no previous need
for perl.

The postinst doesn't have to parse the conffile, it just has to not
generate the dpkg result (by generating the conffile instead of
packaging it) and allow the app to transition from one to the next. If
the existing file is different to what the package thinks it can
upgrade, the package should not require dpkg to generate the error and
the consequent halt of the package installation.

What we see from dpkg should (IMHO) *only* appear as a direct result of
a bug that then needs to be fixed in the package. If the package can
anticipate that dpkg will have problems, the package should handle that
in advance. By the time it's left to dpkg it's too late.

Doesn't justify a separate parser.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
gregor herrmann
2009-12-04 18:51:21 UTC
Permalink
Post by Neil Williams
Post by gregor herrmann
* Or more generally: I want to preserve my locally modified values
and at the same time update the rest which I haven't touched.
That's a user change, I thought the point of this was that changes *not
done by users* are causing problems. Different problem.
Right, that's a different problem, and AFAIK Config::Model handles
both.
Post by Neil Williams
Post by gregor herrmann
And that's what Config::Model allows, as far as I understood it.
That's not how I understood the purposes of it, although it could be
used that way, it isn't sufficient justification IMHO.
If admins make changes, admins should expect to handle the dpkg
conflicts.
Sure, just that I (as an admin) would like to be able to do it easier
than it works now.
Post by Neil Williams
I thought the idea of this was to handle changes not made by users - I
think we agree that those instances are bugs.
Ack.

Cheers,
gregor
--
.''`. http://info.comodo.priv.at/ -- GPG Key IDs: 0x00F3CFE4, 0x8649AA06
: :' : Debian GNU/Linux user, admin, & developer - http://www.debian.org/
`. `' Member of VIBE!AT, SPI Inc., fellow of FSFE | http://got.to/quote/
`- NP: Bob Marley/Jimmy Cliff/Erica Badu: No Woman No Cry
Dominique Dumont
2009-12-05 11:52:05 UTC
Permalink
Post by Neil Williams
That's a user change, I thought the point of this was that changes *not
done by users* are causing problems. Different problem.
Currently, debian package will detect correctly if a user (or a script) left
the configuration unmodified and will upgrade the conf with the new file
coming from the package. This is perfectly fine.

As explained by gregoa, problem arise when a package script needs to keep
user's modifications and merge them with configuration data coming from new
packages.

Currently the only way to address this is to write ad-hoc code in the postinst
script. This is costly and often hard to maintain.

Another possibility is to perform three-way merge with ucf. This will often
succeed but not always: 3-way merge works at syntactic level and not semantic
level so it can often fails. For instance leaving duplicated config
parameters. and error prone.

Config::Model tries to fix this by performing the merge at semantic level.
Post by Neil Williams
That's not how I understood the purposes of it, although it could be
used that way, it isn't sufficient justification IMHO.
If admins make changes, admins should expect to handle the dpkg
conflicts.
This is where we disagree: admins are not the only users of Debian.
Config::Model tries to bring a solution for non-admin users. The kind of users
which are often comfortable on Ubuntu or Windows (yikes) because Ubuntu or
Windows tries to make thing easy for users (whether they succeed or not is
troll food and I will not express an opinion there :-p )

Config::Model tries to address 2 problems:
- configuration upgrade (which should be silent)
- application configuration (with the generated user interfaces)

I agree with you that experienced admins will not see a value in the GUI
("real men don't click") but the GUI will be welcome by casual users. IMHO,
Config::Model provides a way to generate these interfaces while no costing too
much to specify *and* maintain.
Post by Neil Williams
I thought the idea of this was to handle changes not made by users - I
think we agree that those instances are bugs.
To ease the user's life, Config::Model keeps users changes but also propagate
new data required by application evolutions.
Post by Neil Williams
The postinst doesn't have to parse the conffile, it just has to not
generate the dpkg result (by generating the conffile instead of
packaging it) and allow the app to transition from one to the next.
This is not always the case. Hence some upgrade tools were written by Debian
(like ucf)
Post by Neil Williams
If the existing file is different to what the package thinks it can
upgrade, the package should not require dpkg to generate the error and
the consequent halt of the package installation.
That a possibility which was choosen by RedHat. It's fine for servers, less so
for Desktops.

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Tollef Fog Heen
2009-12-06 11:28:31 UTC
Permalink
]] Neil Williams

| > * foo 1.0 ships /etc/foo.conf with
| > featureA=yes
| > featureB=no
| > * I change featureB to yes
| > * foo 2.0 ships /etc/foo.conf with
| > featureA=yes
| > featureB=no
| > featureC=7
| > * Now dpkg ask if I want the new version or my modified one. But what
| > I actually want is
| > - keep my change of the featureB variable and
| > - add the new featureC variable
| > * Or more generally: I want to preserve my locally modified values
| > and at the same time update the rest which I haven't touched.
|
| That's a user change, I thought the point of this was that changes *not
| done by users* are causing problems. Different problem.

You can change a configuration file without using a text editor. Like,
enabling or disabling the «look for other printers on the network»
checkbox in CUPS' config actually changes the setting on disk and when
then the package is upgraded, the user might get a confusing question
from ucf about merging a configuration file instead of the postinst just
being bright enough to see that the user changed the browse setting and
automatically merging in any new settings.

[...]

| As is adding a perl dependency to packages that have no previous need
| for perl.

*shrug*, unless you're talking about embedded systems, which are of
course free to patch their way out of this some other way, most systems
will have perl anyway. You well know that perl-base is Essential: yes
and not taking advantage of the tools we have is silly.

| The postinst doesn't have to parse the conffile, it just has to not
| generate the dpkg result (by generating the conffile instead of
| packaging it) and allow the app to transition from one to the next. If
| the existing file is different to what the package thinks it can
| upgrade, the package should not require dpkg to generate the error and
| the consequent halt of the package installation.

You seem to be quite confused as to the difference between configuration
file and conffile. In many cases, it's non-trivial to properly edit a
configuration file properly preserving all user changes. I believe this
module helps with that.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Paul Wise
2009-12-07 03:19:34 UTC
Permalink
Post by Tollef Fog Heen
| As is adding a perl dependency to packages that have no previous need
| for perl.
*shrug*, unless you're talking about embedded systems, which are of
course free to patch their way out of this some other way, most systems
will have perl anyway.  You well know that perl-base is Essential: yes
and not taking advantage of the tools we have is silly.
It would still be nice if the postinst snippets didn't have to be
patched for EmDebian. I imagine this would simply mean checking for
perl in the postinst and only performing configuration upgrades when
it is available.
--
bye,
pabs

http://wiki.debian.org/PaulWise
Dominique Dumont
2009-12-16 15:25:35 UTC
Permalink
Post by Paul Wise
It would still be nice if the postinst snippets didn't have to be
patched for EmDebian. I imagine this would simply mean checking for
perl in the postinst and only performing configuration upgrades when
it is available.
Paul, during package build, removing the call to dh_config_model_upgrade from
rules will remove dependencies on libconfig-model-perl and the config-model
snippet from postinst.

Is this enough for EmDebian ?

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Neil Williams
2009-12-16 16:40:55 UTC
Permalink
On Wed, 16 Dec 2009 16:25:35 +0100
Post by Dominique Dumont
Post by Paul Wise
It would still be nice if the postinst snippets didn't have to be
patched for EmDebian. I imagine this would simply mean checking for
perl in the postinst and only performing configuration upgrades when
it is available.
Paul, during package build, removing the call to
dh_config_model_upgrade from rules will remove dependencies on
libconfig-model-perl and the config-model snippet from postinst.
Is this enough for EmDebian ?
No. The package should simply exit cleanly with a successful return
value if perl does not exist, letting everything else proceed as before.
The postinst itself needs to check - that way, Emdebian doesn't have to
patch every package using dh_config_model.

Patching debian/rules is problematic because you cannot use
debian/rules to operate the patch mechanism. It is always better to
have a conditional either in debian/rules or in the maintainer
scripts, just as is used in the standard makefile snippets that
determine if it is a cross-build or not. [0]

An alternative is to make dh_config_model* into a no-op if a build
environment variable is set. DEB_BUILD_OPTIONS="noconfigmodel" or
something. With this set, it has to be entirely equivalent to
dh_config_model not appearing in debian/rules at all. It's not ideal
because it is yet another hack when it would be better to assume that
perl just might not be there. However, it also has the advantage that
the build option can be set even if perl does exist but the maintainer
wants to test something within the package without config_model getting
in the way. For this to work, dh_config_model must do absolutely
nothing at all if the build option is set, neither should any other
part of the build use the presence of dh_config_model in debian/rules
(e.g. by testing with grep) to make or prevent any action that would
not occur if dh_config_model was not present in debian/rules.

It would be so much better if this whole implementation was in C - as
long as nothing in the config model tried to execute the cross-built
executable on the build system. Then the ability to disable it via a
build option would be truly orthogonal to the actual issue of whether
perl exists.

[0]
DEB_HOST_GNU_TYPE=$(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
DEB_BUILD_GNU_TYPE=$(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
ifneq ($(DEB_HOST_GNU_TYPE),$(DEB_BUILD_GNU_TYPE))
CROSS= --build $(DEB_BUILD_GNU_TYPE) --host $(DEB_HOST_GNU_TYPE)
else
CROSS= --build $(DEB_BUILD_GNU_TYPE)
endif
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
Dominique Dumont
2009-12-17 13:24:48 UTC
Permalink
Post by Neil Williams
No. The package should simply exit cleanly with a successful return
value if perl does not exist, letting everything else proceed as before.
The postinst itself needs to check - that way, Emdebian doesn't have to
patch every package using dh_config_model.
Ok. Here's the new postinst snippet injected by dh_config_model_upgrade:

# In case of error (error in configuration file or model bug), the
# configuration file is left as is.

# testing perl is required to avoid problem in embedded environments
if [[ -e /usr/bin/perl ]]
then
echo "config-edit is upgrading %PACKAGE% configuration with model
%MODEL%."
config-edit -model %MODEL% -ui none -save %OPTION% || \
echo "WARNING: upgrade with config-edit failed: Run 'config-edit -model
%MODEL% -force %OPTION%' and save the configuration."
fi

Does this fit the bill ?
Post by Neil Williams
An alternative is to make dh_config_model* into a no-op if a build
environment variable is set. DEB_BUILD_OPTIONS="noconfigmodel" or
something. With this set, it has to be entirely equivalent to
dh_config_model not appearing in debian/rules at all.
Ok. Unless somebeody complains, I will add this at the beginning of
dh_config_model_upgrade:

if ($ENV{DEB_BUILD_OPTIONS} =~ /noconfigmodel/) {
warn "dh_config_model_upgrade: DEB_BUILD_OPTIONS specifies 'noconfigmodel',
exiting ...\n";
exit;
}
Post by Neil Williams
It would be so much better if this whole implementation was in C - as
long as nothing in the config model tried to execute the cross-built
executable on the build system. Then the ability to disable it via a
build option would be truly orthogonal to the actual issue of whether
perl exists.
Well, a C implementation of the core part of Config::Model is possible, but it
would take a while to create. I don't have the bandwidth for this.

All the best


Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
James Vega
2009-12-17 14:38:00 UTC
Permalink
On Thu, Dec 17, 2009 at 8:24 AM, Dominique Dumont
Post by Dominique Dumont
Post by Neil Williams
No. The package should simply exit cleanly with a successful return
value if perl does not exist, letting everything else proceed as before.
The postinst itself needs to check - that way, Emdebian doesn't have to
patch every package using dh_config_model.
# In case of error (error in configuration file or model bug), the
# configuration file is left as is.
# testing perl is required to avoid problem in embedded environments
if [[ -e /usr/bin/perl ]]
'[[' for testing is a bashism. This should be

if [ -e /usr/bin/perl ]

or more accurately

if [ -x /usr/bin/perl ]
--
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <***@debian.org>
Dominique Dumont
2009-12-22 14:06:46 UTC
Permalink
Post by James Vega
'[[' for testing is a bashism. This should be
if [ -e /usr/bin/perl ]
or more accurately
if [ -x /usr/bin/perl ]
Done. Thanks

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Dominique Dumont
2009-12-17 14:14:58 UTC
Permalink
Post by Dominique Dumont
Unless somebeody complains, I will add this at the beginning of
if ($ENV{DEB_BUILD_OPTIONS} =~ /noconfigmodel/) {
warn "dh_config_model_upgrade: DEB_BUILD_OPTIONS specifies
'noconfigmodel', exiting ...\n";
exit;
}
After reading (again) debhelper(7), I think it's better to exit silently if
DH_NO_ACT is set.

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Carl Fürstenberg
2009-12-17 15:41:03 UTC
Permalink
Will there be premade modules for the usual suspects? for example
Apache, INI, perl-hash, JSON, basic shell source, debcontrol/rfc-2822,
xml etc... Or is it meant that each package should reinvent the
wheel/copy-paste?
Also I think there should be a "simple mode", there the model is only
defined for the general file format (as above), which can be activated
with only a commandline argument to dh_config_config_model, without
the need to write perl code/understand what a model is (not all devs
can this kind of thingis).
--
/Carl Fürstenberg <***@gmail.com>
Dominique Dumont
2009-12-22 14:22:08 UTC
Permalink
Post by Carl Fürstenberg
Will there be premade modules for the usual suspects? for example
Apache, INI, perl-hash, JSON, basic shell source, debcontrol/rfc-2822,
xml etc...
There's already a generic parser and writer for perl-hash (more accurately
perl data structure: nested hash and array) and INI.

YAML is underway. JSON would be fast to add.

With Augeas or libdebctrl (hi Jonathan), debcontrol would be quite easy to
add.

Apache config style is a tougher nut to crack as I must take care of included
files.
Post by Carl Fürstenberg
Or is it meant that each package should reinvent the
wheel/copy-paste?
No. Parsers and writers can be shared across several models.
Post by Carl Fürstenberg
Also I think there should be a "simple mode", there the model is only
defined for the general file format (as above), which can be activated
with only a commandline argument to dh_config_config_model, without
the need to write perl code/understand what a model is (not all devs
can this kind of thingis).
This holds true for syntax. But how can a file be validated (from a semantic
point of view) if there's no model, i.e. if there's no knowledge of what's
supposed to be in the file ? (witjhout taking syntax into account)

In fact, Augeas is much closer to what you are thinking about. But Augeas
cannot provide configuration upgrade because there's almost no semantic
knowledge in Augeas.

Hope this helps

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Neil Williams
2009-12-17 19:48:50 UTC
Permalink
On Thu, 17 Dec 2009 14:24:48 +0100
Post by Dominique Dumont
Post by Neil Williams
No. The package should simply exit cleanly with a successful return
value if perl does not exist, letting everything else proceed as
before. The postinst itself needs to check - that way, Emdebian
doesn't have to patch every package using dh_config_model.
Ok. Here's the new postinst snippet injected by
# In case of error (error in configuration file or model bug), the
# configuration file is left as is.
# testing perl is required to avoid problem in embedded environments
if [[ -e /usr/bin/perl ]]
then
echo "config-edit is upgrading %PACKAGE% configuration with model
%MODEL%."
config-edit -model %MODEL% -ui none -save %OPTION% || \
echo "WARNING: upgrade with config-edit failed: Run
'config-edit -model %MODEL% -force %OPTION%' and save the
configuration." fi
Does this fit the bill ?
With the later removal of the bashism, yes. Thanks.
Post by Dominique Dumont
Post by Neil Williams
An alternative is to make dh_config_model* into a no-op if a build
environment variable is set. DEB_BUILD_OPTIONS="noconfigmodel" or
something. With this set, it has to be entirely equivalent to
dh_config_model not appearing in debian/rules at all.
Unless somebeody complains, I will add this at the beginning of
if ($ENV{DEB_BUILD_OPTIONS} =~ /noconfigmodel/) {
warn "dh_config_model_upgrade: DEB_BUILD_OPTIONS specifies
'noconfigmodel', exiting ...\n";
exit;
}
After reading (again) debhelper(7), I think it's better to exit
silently if DH_NO_ACT is set.
DH_NO_ACT still needs debian/rules to be modified or else all debhelper
routines would be disabled; we need an option that is specific to
dh_config_model without having to edit debian/rules.

I think DEB_BUILD_OPTIONS is still useful, whether or not
dh_config_model outputs the warning is neither here nor there for my
purposes.
Post by Dominique Dumont
Post by Neil Williams
It would be so much better if this whole implementation was in C -
as long as nothing in the config model tried to execute the
cross-built executable on the build system. Then the ability to
disable it via a build option would be truly orthogonal to the
actual issue of whether perl exists.
Well, a C implementation of the core part of Config::Model is
possible, but it would take a while to create. I don't have the
bandwidth for this.
The debconf maintainers aren't the cdebconf maintainers; it's not
unusual for someone else to step up as long as the interface in the
perl version is suitable for porting to C. So the real issue is that
config::model tries hard to make an interface that can be implemented
in another language.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.linux.codehelp.co.uk/
http://e-mail.is-not-s.ms/
Dominique Dumont
2009-12-22 14:45:39 UTC
Permalink
Post by Neil Williams
DH_NO_ACT still needs debian/rules to be modified or else all debhelper
routines would be disabled; we need an option that is specific to
dh_config_model without having to edit debian/rules.
I think DEB_BUILD_OPTIONS is still useful, whether or not
dh_config_model outputs the warning is neither here nor there for my
purposes.
ok. dh_config_model_upgrade will react to both variables.
Post by Neil Williams
The debconf maintainers aren't the cdebconf maintainers; it's not
unusual for someone else to step up as long as the interface in the
perl version is suitable for porting to C. So the real issue is that
config::model tries hard to make an interface that can be implemented
in another language.
Hmm. The core part of Config::Model could be rewritten in C: I've avoided
using too many Perl magic.

The fact that the models are currently stored as Perl data structure is not a
big problem. This was done just as a convenience to parse quickly the models.

The models could as well be stored as YAML, JSON or XML and used as is in a C
implementation of Config::Model. The Perl implementation would just have to
translate JSON or YAML or XML back as Perl data structure to work as usual.

All the best

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Joey Hess
2009-12-04 19:36:19 UTC
Permalink
Post by Dominique Dumont
From an embedded perspective, we certainly don't want every
configurable package depending on perl at package installation /
upgrade time - that's why we have cdebconf.
And if that argument had been used when debconf was being written, we
might currently have neither debconf nor cdebconf.

It makes sense to have exactly one general-purpose scripting language in
standard. More are bloat, and with less, rapid prototyping is not
possible. We should not prevent others from using that language for what
it's there for.

It also makes sense to design interfaces to things prototyped in that
language so they can be replaced with a C (or other language) implementation.
Config::Model uses perl for the config file parser/writer in a way that
doesn't allow doing that.

But it does seem likely that packages using it could fall back to
current config file handling if Config::Model were not available
in an embedded system.
--
see shy jo
Dominique Dumont
2009-12-05 11:53:43 UTC
Permalink
Post by Joey Hess
But it does seem likely that packages using it could fall back to
current config file handling if Config::Model were not available
in an embedded system.
Agreed. That's a reasonable goal.

Dominique
--
http://config-model.wiki.sourceforge.net/ -o- http://search.cpan.org/~ddumont/
http://www.ohloh.net/accounts/ddumont
Continue reading on narkive:
Loading...