--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: linux-image-2.6.18-4-k7: EXT3 remounts read-only, possible data loss with message "ext3_free_blocks: Freeing blocks not in datazone"
- From: Tim Miller Dyck <editor@canadianmennonite.org>
- Date: Mon, 28 May 2007 11:11:18 -0400
- Message-id: <20070528151118.27820.32543.reportbug@menno.office.canadianmennonite.org>
Package: linux-image-2.6.18-4-k7
Version: 2.6.18.dfsg.1-12etch2
Severity: important
Our office server had an ext3 volume remount itself read-only after an
error in handing a disk write. This is the first time this error or
anything like it has occurred. Another ext3 file system on the same
physical volume and RAID1 mirror volume did not report any
errors.
Afterwards, e2fsck reported a journal block was multiply-claimed with a
data file block. I restored the file as a precaution from backup.
This is a etch stable system.
--
from /var/log/syslog
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1):
ext3_free_blocks: Freeing blocks not in datazone - block = 327786496,
count = 1
May 23 12:30:02 menno kernel: Aborting journal on device dm-1.
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_free_blocks_sb: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_truncate: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_orphan_del: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in
ext3_delete_inode: Journal has aborted
May 23 12:30:02 menno kernel: __journal_remove_journal_head: freeing
b_committed_data
May 23 12:30:02 menno last message repeated 5 times
May 23 12:30:02 menno kernel: ext3_abort called.
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1):
ext3_journal_start_sb: Detected aborted journal
May 23 12:30:02 menno kernel: Remounting filesystem read-only
--
There were some errors in the file system (though I'm not sure how many
were directly from the ext3 errors and how many might have been present
already). I needed to specify "-f" to e2fsck as when I first ran it
afer unmounting the file system, it reported the filesystem was clean
and didn't do a check.
# umount /data/public
# e2fsck -f /dev/vg_on_md0/public
e2fsck 1.40-WIP (14-Nov-2006)
Pass 1: Checking inodes, blocks, and sizes
Inode 3375515, i_size is 741, should be 12288. Fix<y>? yes
Deleted inode 4538519 has zero dtime. Fix<y>? yes
Inode 5832931 is in use, but has dtime set. Fix<y>? yes
Inode 5832931, i_blocks is 1148070603, should be 0. Fix<y>? yes
Inode 7799003 has illegal block(s). Clear<y>? yes
Illegal block #1 (1149919138) in inode 7799003. CLEARED.
Illegal block #2 (1139529199) in inode 7799003. CLEARED.
Illegal block #3 (1054177516) in inode 7799003. CLEARED.
Inode 7799003, i_size is 741, should be 24576. Fix<y>? yes
Inode 7799003, i_blocks is 8, should be 16. Fix<y>? yes
Inode 7801123 is in use, but has dtime set. Fix<y>? yes
Inode 7801123, i_blocks is 1139529203, should be 64. Fix<y>? yes
Inode 7817987 is in use, but has dtime set. Fix<y>? yes
Inode 7817987, i_blocks is 1054177850, should be 8. Fix<y>? yes
Running additional passes to resolve blocks claimed by more than one
inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 8: 5224
Multiply-claimed block(s) in inode 49346: 131070
Multiply-claimed block(s) in inode 7799003: 5224 131070
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 2 inodes containing multiply-claimed blocks.)
File <The journal inode> (inode #8, mod time Wed Jan 19 23:28:44 2005)
has 1 multiply-claimed block(s), shared with 1 file(s):
/Computer/hardware/Philips monitors/LCD monitor, model
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent
(inode #7799003, mod time Thu Feb 9 18:53:19 2006)
Clone multiply-claimed blocks<y>? yes
File /Pending Art and Copy/reg bibby/IMG_0740.JPG (inode #49346, mod
time Thu Feb 22 11:49:08 2007)
has 1 multiply-claimed block(s), shared with 1 file(s):
/Computer/hardware/Philips monitors/LCD monitor, model
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent
(inode #7799003, mod time Thu Feb 9 18:53:19 2006)
Clone multiply-claimed blocks<y>? yes
File /Computer/hardware/Philips monitors/LCD monitor, model
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent
(inode #7799003, mod time Thu Feb 9 18:53:19 2006)
has 2 multiply-claimed block(s), shared with 2 file(s):
/Pending Art and Copy/reg bibby/IMG_0740.JPG (inode #49346, mod
time Thu Feb 22 11:49:08 2007)
<The journal inode> (inode #8, mod time Wed Jan 19 23:28:44
2005)
Multiply-claimed blocks already reassigned or cloned.
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found. Create<y>? yes
Pass 4: Checking reference counts
Unattached zero-length inode 5832931. Clear<y>? yes
Inode 7801123 ref count is 17387, should be 1. Fix<y>? yes
Inode 7817987 ref count is 17387, should be 1. Fix<y>? yes
Pass 5: Checking group summary information
Block bitmap differences: -9100492 -15622524
Fix<y>? yes
Free blocks count wrong for group #0 (49, counted=47).
Fix<y>? yes
Free blocks count wrong for group #277 (5317, counted=5318).
Fix<y>? yes
Free blocks count wrong for group #476 (24271, counted=24272).
Fix<y>? yes
Inode bitmap differences: -4538519
Fix<y>? yes
Free inodes count wrong for group #277 (16314, counted=16315).
Fix<y>? yes
Free inodes count wrong for group #356 (15705, counted=15704).
Fix<y>? yes
public: ***** FILE SYSTEM WAS MODIFIED *****
115332 inodes used (1.47%)
4917 non-contiguous inodes (4.3%)
# of inodes with ind/dind/tind blocks: 16957/724/0
6593745 blocks used (41.92%)
0 bad blocks
0 large files
104607 regular files
10716 directories
0 character device files
0 block device files
0 fifos
0 links
0 symbolic links (0 fast symbolic links)
0 sockets
--------
115322 files
--
Here is the settings for the file system:
root@menno:/var/log# tune2fs -l /dev/vg_on_md0/public
tune2fs 1.40-WIP (14-Nov-2006)
Filesystem volume name: public
Last mounted on: <not available>
Filesystem UUID: a0abccbc-79a7-4fff-95d6-b9a5bd7a1d61
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal filetype sparse_super
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 7864320
Block count: 15728640
Reserved block count: 786432
Free blocks: 9134895
Free inodes: 7748988
First block: 0
Block size: 4096
Fragment size: 4096
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Filesystem created: Wed Jan 19 23:28:12 2005
Last mount time: Wed May 16 17:04:29 2007
Last write time: Wed May 23 14:39:10 2007
Mount count: 0
Maximum mount count: 35
Last checked: Wed May 23 14:39:10 2007
Check interval: 15552000 (6 months)
Next check after: Mon Nov 19 13:39:10 2007
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
Default directory hash: tea
Directory Hash Seed: f706d258-fa07-4d1b-b556-9d5d555c0021
Journal backup: inode blocks
--
The file system is on a LVM2 volume "vg_on_md0", which is on a RAID1
mirror on /dev/md0.
from /etc/fstab:
/dev/vg_on_md0/public /data/public ext3 acl 0
0
# vgdisplay
--- Volume group ---
VG Name vg_on_md0
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 72
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 2
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 186.30 GB
PE Size 4.00 MB
Total PE 47694
Alloc PE / Size 47693 / 186.30 GB
Free PE / Size 1 / 4.00 MB
VG UUID eSnn0u-HyNr-FJav-xiDJ-oGuG-2dPi-FqwFE5
# pvdisplay
--- Physical volume ---
PV Name /dev/md0
VG Name vg_on_md0
PV Size 186.30 GB / not usable 0
Allocatable yes
PE Size (KByte) 4096
Total PE 47694
Free PE 1
Allocated PE 47693
PV UUID M4KXnz-C0rq-IM84-WHYE-Z4wv-LEsF-4JHxux
# cat /proc/mdstat
Personalities : [raid1]
....
md0 : active raid1 sda1[0] sdb1[1]
195358336 blocks [2/2] [UU]
The hardware:
2 x Western Digital Caviar WD2000 Serial ATA (in a software RAID1 mirror)
# lspci
....
01:06.0 Mass storage controller: Silicon Image, Inc. SiI 3112
[SATALink/SATARaid] Serial ATA Controller (rev 02)
....
--
I'll provide whatever other information is requested.
Regards,
Tim
-- System Information:
Debian Release: 4.0
APT prefers stable
APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell: /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-4-k7
Locale: LANG=en_CA.utf8, LC_CTYPE=en_CA.utf8 (charmap=UTF-8)
Versions of packages linux-image-2.6.18-4-k7 depends on:
ii coreutils 5.97-5.3 The GNU core utilities
ii debconf [debconf-2.0] 1.5.11 Debian configuration management sy
ii initramfs-tools [linux-initra 0.85g tools for generating an initramfs
ii module-init-tools 3.3-pre4-2 tools for managing Linux kernel mo
Versions of packages linux-image-2.6.18-4-k7 recommends:
ii libc6-i686 2.3.6.ds1-13 GNU C Library: Shared libraries [i
-- debconf information:
shared/kernel-image/really-run-bootloader: true
* linux-image-2.6.18-4-k7/preinst/already-running-this-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/old-system-map-link-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/abort-install-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/bootloader-test-error-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/bootloader-error-2.6.18-4-k7:
linux-image-2.6.18-4-k7/preinst/abort-overwrite-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/depmod-error-initrd-2.6.18-4-k7: false
linux-image-2.6.18-4-k7/prerm/would-invalidate-boot-loader-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/bootloader-initrd-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/prerm/removing-running-kernel-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/overwriting-modules-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/postinst/depmod-error-2.6.18-4-k7: false
linux-image-2.6.18-4-k7/postinst/create-kimage-link-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/initrd-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/old-initrd-link-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/elilo-initrd-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/failed-to-move-modules-2.6.18-4-k7:
linux-image-2.6.18-4-k7/postinst/old-dir-initrd-link-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/postinst/kimage-is-a-directory:
linux-image-2.6.18-4-k7/preinst/lilo-initrd-2.6.18-4-k7: true
linux-image-2.6.18-4-k7/preinst/lilo-has-ramdisk:
--- End Message ---