Re: [PATCH 4/6] Hook conffile database into dpkg unpack/configure/remove stages

[Jonathan Nieder <jrnieder@gmail.com>]

Hi Sean,

Thanks for the thoughts and clarifications.

Until I read how it gets used (i.e. the remaining patches), I don’t
think I’m in a position to say much definitive about how the conffile
db should work.  So feel free to skip this message if you lack time.
I’ll write more when I get there.

sean finney wrote:

> it's a pretty small window for failure, and i designed it to do the least
> amount of stuff possible during said window of non-atomicity (i.e. one
> directory rename).  but yes it's hypothetically possible that a power outage
> some other externality (unrelated memory errors, i/o errors, etc) could 
> trigger what you describe.  one remedy not requiring any significant changes
> would be to have a final "flag" file of some kind to indicate that it's
> been successful, which doesn't solve the atomicity problem but could solve
> the "how to detect when we previously screwed up" matter.

It’s a small window of time, but it makes me nervous.  dpkg generally
has very nice error handling, and it seems worthwhile to make sure
additions continue in the same vein.

The conffile db does not contain any precious data, but if we drop the
conffiles from status eventually (is that part of the plan?) then
removing a package’s conffile db would mean either pestering the user
about its every conffile installed for the next upgrade or blindly
using the old versions.

>>    One (crazy?) idea would be to overwrite entries in current/pkg
>>    until they all match new/.  That is, run the equivalent of
>> 
>> 	ln -f new/pkg/afd9834287dc... current/pkg/afd9834287dc...
>> 
>>    for each configuration file until their stat information matches.
[...]
> it's also quite a bit more complicated, and more stuff could go wrong
> during this time (i.e. running out of inodes, etc).

Yes, mv would be more appropriate.

> with regards to losing
> the old files, if those were similarly link-shuffled off to a tmp location
> it might be possible to keep them in case of error, though.

Anything in tmp is as good as gone, in my opinion, unless dpkg starts
rolling back previous failed transactions at the start of the next
command.  That sounds a bit scarily complicated.

> another idea:  what if we just abandoned the concept of "current"/etc entirely
> and used package versions instead, or maybe had them as symlinks?  a symlink
> can be updated atomically, and if there were some kind of "byversion" dir
> then they wouldn't need to cross paths on the filesystem at all.

Both methods sound sane.

I think I will need to read the next patch in the series to learn what
is most convenient here. :)

Until then, let me compare the analogous moment in the current
configure phase. (deferred_configure_conffile(), in configure.c)

	1. if .dpkg-new file is missing, skip to the next conffile
	2. compute .dpkg-new file hash
	3. prompt the user for action
	4. with user’s help, bring installed configuration file up to date:
		- if doing something complicated, let the user modify
		  his own version. We’re not responsible for this. :)
		- if keeping the user’s version, remove the .dpkg-new
		- if using the package’s version, rename the .dpkg-new
		  to replace the conffile
	5. atomically write the saved .dpkg-new file hash to
			/var/lib/dpkg/status

If dpkg is interrupted between steps 4 and 5, the .dpkg-new file hash
is gone, and the conffile hash saved in status is the old one, meaning
the user will be asked about the conffile in the next run.  I’m not
sure why the code is written this way.  Why not save the hash between
steps 3 and 4?

So as long as your commit is atomic, you’re at least not making things
worse.

Atomic here means from the point of view of dpkg; step 5 does not
require status to be written out in full each time.  (Either dpkg stops
and status is unchanged, or dpkg bets everything on finishing updating
status.  For example, if there is no room on disk for an updated
status file, dpkg will become a lame duck until someone makes room,
after which execing dpkg will make it write the status file and
proceed happily again.)

Step 1 is important to make this idempotent.

Hmm.

Thanks,
Jonathan