Bug#599203: marked as done (os-prober: Causes Data Corruption on a SAN setup; Mounts SAN volumes that are already mounted on a different host.)
Your message dated Wed, 10 Nov 2010 12:02:07 +0000
with message-id <E1PG9NP-00008j-0W@franck.debian.org>
and subject line Bug#599203: fixed in os-prober 1.40
has caused the Debian Bug report #599203,
regarding os-prober: Causes Data Corruption on a SAN setup; Mounts SAN volumes that are already mounted on a different host.
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact email@example.com
Debian Bug Tracking System
Contact firstname.lastname@example.org with problems
--- Begin Message ---
- To: Debian Bug Tracking System <email@example.com>
- Subject: os-prober: Causes Data Corruption on a SAN setup; Mounts SAN volumes that are already mounted on a different host.
- From: Aleksandr Levchuk <firstname.lastname@example.org>
- Date: Tue, 05 Oct 2010 08:22:31 -0700
- Message-id: <email@example.com>
An upgrade (os-prober 1.35 -> 1.39) corrupted 3.3 TB of data on our SAN.
I was upgrading the host space1 and the data corruption occurred on space2. An install script of os-prober tried mounting as read-only a SAN volume which was already mounted on space2. That volume (on sapce2) was in production use so EXT3-fs (on space1) concluded that the journal was inconsistent, re-mounted as writable and performed a "recovery".
The mount on space2 became unavailable bringing the production host down. Re-mounting failed. After rebooting space2 fsck was required on the affected partition. It ran for many hours and found a huge number errors. Probably more than 10,000 errors. Then I was able to mount the volume and saw that our data was turned into gray goo: parts of system prel scripts were replaced by binary chunks, databases and web servers would not start. I had 30 containers in production. Some actually booted despite major sporadic data corruption in them.
My fellow system administrator from another department on campus said that their distribution (CentOS) does not run install scripts. As he worded it - Debian ended-up managing your SAN for you.
The reason why I got os-prober was the change in Debian's policy to install all recommended packages and os-prober was recommended by Grub. I am not sure why the data corruption did not happed when I upgraded to Squeeze a month ago (grub-common 1.96+20080724-16 -> 1.98-1).
I'm attaching the aptitiude log and syslog of space1.
The root of the problem is also described in bug #556739 (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=556739). The author predicts filesystem corruption and data loss back in 2009.
-- System Information:
Debian Release: squeeze/sid
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 126.96.36.199+openvz-budarin (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages os-prober depends on:
ii libc6 2.11.2-2 Embedded GNU C Library: Shared lib
os-prober recommends no packages.
os-prober suggests no packages.
-- no debconf information
--- End Message ---
--- Begin Message ---
We believe that the bug you reported is fixed in the latest version of
os-prober, which is due to be installed in the Debian FTP archive:
A summary of the changes between this version and the previous one is
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to firstname.lastname@example.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
Colin Watson <email@example.com> (supplier of updated os-prober package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing firstname.lastname@example.org)
-----BEGIN PGP SIGNED MESSAGE-----
Date: Wed, 10 Nov 2010 11:51:19 +0000
Binary: os-prober-udeb os-prober
Architecture: source i386
Maintainer: Debian Install System Team <email@example.com>
Changed-By: Colin Watson <firstname.lastname@example.org>
os-prober - utility to detect other OSes on a set of drives
os-prober-udeb - utility to detect other OSes on a set of drives (udeb)
Closes: 417407 556739 567953 589676 592924 599203
os-prober (1.40) unstable; urgency=low
[ Christian Perrier ]
* Fix Windows Vista and Windows Recovery Environment partitions
recognition. (Thanks, Bouke Bunnik)
Closes: #589676, LP: #476625
* Allow recognition of recent MINIX installations.
Thanks to Feiran Zheng
[ Colin Watson ]
* Improve error message when /sys/block is missing.
* os-prober doesn't know how to probe other OSes on non-Linux kernels.
For now, just exit quietly rather than confusing people (closes:
* Ignore active swap partitions (thanks, Alex Owen; see #417407).
* Refactor linux_mount_boot to look up labels and UUIDs using blkid or
/dev/disk/by-*/ rather than relying on mount being smart enough. This
removes some horrible code that executes mount from /target.
* Set partitions read-only before mounting them (based on a patch by Alex
Owen; closes: #417407, #556739, #599203).
045432945afe4bad0671e4ee5a98836e244271e5 1577 os-prober_1.40.dsc
cdeca296318e51b37233d63424c9b262f41b2db0 23470 os-prober_1.40.tar.gz
cd9b549f42ad7275579778a1709e93a2568b6904 13156 os-prober-udeb_1.40_i386.udeb
42649ede8d363bf5c28a5f52febb04c4bd9e966d 23670 os-prober_1.40_i386.deb
a29c19dd59e82f87d5f6063a0d9e1cc7926339bb0338f08b55c6a6dac625d77b 1577 os-prober_1.40.dsc
fa4e6dc51521c0f60b5b8157a2bd42475473b958ea93725669a90e17b7cd6a70 23470 os-prober_1.40.tar.gz
c6ae392745556a2bc8a2d474a83daacc07e3a9829fd3af9588bb9c9bac310b8b 13156 os-prober-udeb_1.40_i386.udeb
84f4e5b291b94dba9f678fcd8881693a1848ab5ee4ac123c336ad95c2bb1e395 23670 os-prober_1.40_i386.deb
d7a7d3963331e103d9e1c36d65e6aa61 1577 debian-installer optional os-prober_1.40.dsc
5f441d4962c2be2fa226cc44eee81ab9 23470 debian-installer optional os-prober_1.40.tar.gz
37ff87a79839ae6cfa9d47bb0f98fd19 13156 debian-installer optional os-prober-udeb_1.40_i386.udeb
4a82106efb183f2d8929a44bf62f77b2 23670 utils extra os-prober_1.40_i386.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Colin Watson <email@example.com> -- Debian developer
-----END PGP SIGNATURE-----
--- End Message ---