Re: Bug#584254: dpkg should support a --force-unsafe-io option or such
Hi!
On Sun, 2010-10-10 at 12:37:27 +0300, Modestas Vainius wrote:
> On sekmadienis 10 Spalis 2010 01:06:41 Otavio Salvador wrote:
> > On Fri, Oct 8, 2010 at 8:55 PM, Modestas Vainius <modax@debian.org> wrote:
> > > It does not make much sense for dpkg to be in this uber-paranoid mode at
> > > debian-installer time. If power fails, install process will probably have
> > > to be started from scratch anyway. What's more, obviously I have no
> > > choice to use libeatmydata or similar to fight this dpkg behaviour at
> > > debian installer time.
> > >
> > > In my opinion, dpkg should provide a way to turn off those offensive
> > > *sync() calls and debian installer should make use of it.
> >
> > I fully agree that d-i doesn't need and shouldn't use it by default.
> > In fact some embedded systems shouldn't use it either in production
> > mode if the media suffer of wearing.
> >
> > As previously said in this thread, d-i needs to be restarted in case
> > of power outage or something broken and fsync won't help on that
> > except make it taking longer.
>
> I feel rather strong about this issue because I keep running and running into
> it in different situations. I truly believe that it should be high priority
> for squeeze but I won't bump severity myself. To sum up:
I had planned to include changes for this for squeeze, but then the
sudden freeze happened, and after the initial reaction from the
release team on the first exception request I didn't feel like begging
for this and other changes.
> 1) current dpkg is bog slow on modern file systems (brtfs, ext4). What is
> more, due to frequent sync() calls, it is a very bad idea to run dpkg when
> other unrelated disk I/O (esp. on modern FSes) is in the background. It just
> slows everything down.
Right, for some time now I've considered the switch from fsync() to
sync() (on Linux) to be a mistake, as it will affect unrelated file
systems, which might be disruptive in case there's background work being
done.
But this was done because those "modern file systems" didn't perform
adequately with fsync(), and they are the ones which require the syncs
or they will actually lose data.
> 3) dpkg is pointlessly slow in such use cases as buildds where *sync() is not
> important at all.
Well, even if the buildd chroot supposedly should be able to be recreated
easily, if the zero-lenght file issues appear on it, then it might not
be obvious something is wrong, and might produce bogus packages.
> 2) current dpkg is arguably not suitable for flash media (i.e. embedded
> devices). This hurts the "universal" part of Debian OS.
> 4) while typical dpkg could be work arounded with libeatmydata, there is no
> cure for debian-installer.
Sure, I agree the user should be able to disable this, at their own
risk. Or on specific cases, like on d-i.
> And all this is to protect against power failure? Sure, the purpose is noble
> and maybe it's a good default but due to negative side effects I find it
> unacceptable that this behaviour is not configurable.
Not only power failures, any abrupt system crash, say the bus getting
locked up, or a user assisted reboot, can produce for example
zero-lenght files on at least ext4 file systems, I don't know about
btrfs. The worst though is that the performance issues affect the file
systems which really need those safety measures. Also take into account
these are not anecdotal cases, Ubuntu who has offered ext4 on installation
for some time now had *hundreds* of duped bug reports on broken systems
due to this.
Anyway, I actually had the changes around last July, and I'll run them
through the release team to see what they say. I've rebased them now,
and will polish them a bit, but they are essentially these:
<http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=87740373>
<http://git.hadrons.org/?p=debian/dpkg/dpkg.git;a=commitdiff;h=0484cd7d>
regards,
guillem
Reply to: