[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#593883: marked as done (linux-2.6: race reading /proc/mounts causes umount failure and potential for serious dataloss)

Your message dated Mon, 04 Oct 2010 00:14:58 +0100
with message-id <1286147698.3916.186.camel@localhost>
and subject line Re: race reading /proc/mounts causes umount failure and potential for serious dataloss
has caused the Debian Bug report #593883,
regarding linux-2.6: race reading /proc/mounts causes umount failure and potential for serious dataloss
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
593883: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=593883
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: schroot
Version: 1.3.2-1
Severity: normal

Hello,

schroot suffers from a race condition where if two schroot sessions
are ended concurrently, one of them may fail to clean up properly.  On
some intensely schroot-using build machines, I see this failure
regularly.  The symptom looks like this:

  E: 10mount: umount:
/var/lib/schroot/mount/somechroot-source-bd8cdb9f-611a-4991-869c-e479fa673ec7:
device is busy.

and then the 'schroot' command exits with failure.

The issue is that /proc/mounts is unreliable.  It turns out that when
you read it concurrently with a umount, you can end up missing records
for mounts that have nothing to do with the ones that were unmounted.
Arguably this is a kernel bug, but there is no simple fix, so
/proc/mounts will definitely remain unreliable at least for squeeze.

Consequently because schroot-listmounts relies on /proc/mounts, it too
is unreliable.  So when do_umount_all() in etc/setup.d/10mount does
what amounts to this (but with better reporting and error handling):

    "$LIBEXEC_DIR/schroot-listmounts" -m "$base" | xargs -rn1 umount

the schroot-listmounts may skip a record if another process is ending
another schroot session, or unmounting something for any other reason.
Then one of the internal mounts, like .../tmp or .../dev/pts, remains
mounted, and the attempt to unmount the base directory fails.

Currently I'm working around the issue with a flock around the body of
do_umount_all().  That's easy and is enough to address the problem
when no other umounts are happening on the system.  A real fix would
probably have to be for schroot to record for itself the list of
mounts it makes inside a session's tree, and rely on that list instead
of on /proc/mounts.

Cheers,
Greg

--- End Message ---
--- Begin Message --- 
-------- Forwarded Message --------
From: Ben Hutchings <ben@decadent.org.uk>
To: 593516-done@bugs.debian.org
Subject: Re: race reading /proc/mounts causes umount failure and potential for serious dataloss
Date: Sun, 22 Aug 2010 02:28:43 +0100

On Sat, 2010-08-21 at 21:46 +0100, Roger Leigh wrote:
[...]
> I'm not aware of the specifics of how /proc/mounts is implemented, but
> I think this could be made reliable if the contents were not changed
> once opened (i.e. each reader gets a unique immutable copy rather than
> a mutable global copy that's subject to spurious changes).
[...]

And it would be a lot more convenient if directories worked like that
too.  Neither of them do, because it would require arbitrarily large
memory allocations in the kernel.

If you can't read the whole file in one go, allocate a larger buffer and
retry.  Repeat until successful.

Ben.

-- 
Ben Hutchings
Once a job is fouled up, anything done to improve it makes it worse.

Attachment: signature.asc
Description: This is a digitally signed message part


--- End Message ---
Reply to: