[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#426409: linux-image-2.6.18-4-k7: EXT3 remounts read-only, possible data loss with message "ext3_free_blocks: Freeing blocks not in datazone"



Package: linux-image-2.6.18-4-k7
Version: 2.6.18.dfsg.1-12etch2
Severity: important

Our office server had an ext3 volume remount itself read-only after an 
error in handing a disk write. This is the first time this error or 
anything like it has occurred. Another ext3 file system on the same 
physical volume and RAID1 mirror volume did not report any 
errors.

Afterwards, e2fsck reported a journal block was multiply-claimed with a 
data file block. I restored the file as a precaution from backup.

This is a etch stable system.
 
--
 
from /var/log/syslog

May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1): 
ext3_free_blocks: Freeing blocks not in datazone - block = 327786496, 
count = 1
May 23 12:30:02 menno kernel: Aborting journal on device dm-1.
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_free_blocks_sb: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_truncate: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_orphan_del: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_reserve_inode_write: Journal has aborted
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1) in 
ext3_delete_inode: Journal has aborted
May 23 12:30:02 menno kernel: __journal_remove_journal_head: freeing 
b_committed_data
May 23 12:30:02 menno last message repeated 5 times
May 23 12:30:02 menno kernel: ext3_abort called.
May 23 12:30:02 menno kernel: EXT3-fs error (device dm-1): 
ext3_journal_start_sb: Detected aborted journal
May 23 12:30:02 menno kernel: Remounting filesystem read-only  

--

There were some errors in the file system (though I'm not sure how many 
were directly from the ext3 errors and how many might have been present 
already). I needed to specify "-f" to e2fsck as when I first ran it 
afer unmounting the file system, it reported the filesystem was clean 
and didn't do a check.

# umount /data/public
# e2fsck -f /dev/vg_on_md0/public

e2fsck 1.40-WIP (14-Nov-2006)
Pass 1: Checking inodes, blocks, and sizes
Inode 3375515, i_size is 741, should be 12288.  Fix<y>? yes

Deleted inode 4538519 has zero dtime.  Fix<y>? yes

Inode 5832931 is in use, but has dtime set.  Fix<y>? yes

Inode 5832931, i_blocks is 1148070603, should be 0.  Fix<y>? yes

Inode 7799003 has illegal block(s).  Clear<y>? yes

Illegal block #1 (1149919138) in inode 7799003.  CLEARED.
Illegal block #2 (1139529199) in inode 7799003.  CLEARED.
Illegal block #3 (1054177516) in inode 7799003.  CLEARED.
Inode 7799003, i_size is 741, should be 24576.  Fix<y>? yes

Inode 7799003, i_blocks is 8, should be 16.  Fix<y>? yes
                                                                                
Inode 7801123 is in use, but has dtime set.  Fix<y>? yes

Inode 7801123, i_blocks is 1139529203, should be 64.  Fix<y>? yes

Inode 7817987 is in use, but has dtime set.  Fix<y>? yes

Inode 7817987, i_blocks is 1054177850, should be 8.  Fix<y>? yes


Running additional passes to resolve blocks claimed by more than one 
inode...
Pass 1B: Rescanning for multiply-claimed blocks
Multiply-claimed block(s) in inode 8: 5224
Multiply-claimed block(s) in inode 49346: 131070                                
Multiply-claimed block(s) in inode 7799003: 5224 131070
Pass 1C: Scanning directories for inodes with multiply-claimed blocks
Pass 1D: Reconciling multiply-claimed blocks
(There are 2 inodes containing multiply-claimed blocks.)

File <The journal inode> (inode #8, mod time Wed Jan 19 23:28:44 2005) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
        /Computer/hardware/Philips monitors/LCD monitor, model 
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent 
(inode #7799003, mod time Thu Feb  9 18:53:19 2006)
Clone multiply-claimed blocks<y>? yes

File /Pending Art and Copy/reg bibby/IMG_0740.JPG (inode #49346, mod 
time Thu Feb 22 11:49:08 2007) 
  has 1 multiply-claimed block(s), shared with 1 file(s):
        /Computer/hardware/Philips monitors/LCD monitor, model 
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent 
(inode #7799003, mod time Thu Feb  9 18:53:19 2006)
Clone multiply-claimed blocks<y>? yes

File /Computer/hardware/Philips monitors/LCD monitor, model 
170S5FB/CD-ROM/lcd/manual/ESPANOL/170P5/GLOSSARY/IMAGES/.AppleDouble/.Parent 
(inode #7799003, mod time Thu Feb  9 18:53:19 2006)                                                    
  has 2 multiply-claimed block(s), shared with 2 file(s):
        /Pending Art and Copy/reg bibby/IMG_0740.JPG (inode #49346, mod 
time Thu Feb 22 11:49:08 2007)
        <The journal inode> (inode #8, mod time Wed Jan 19 23:28:44 
2005)
Multiply-claimed blocks already reassigned or cloned.

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
/lost+found not found.  Create<y>? yes

Pass 4: Checking reference counts
Unattached zero-length inode 5832931.  Clear<y>? yes

Inode 7801123 ref count is 17387, should be 1.  Fix<y>? yes

Inode 7817987 ref count is 17387, should be 1.  Fix<y>? yes

Pass 5: Checking group summary information
Block bitmap differences:  -9100492 -15622524
Fix<y>? yes

Free blocks count wrong for group #0 (49, counted=47).
Fix<y>? yes

Free blocks count wrong for group #277 (5317, counted=5318).
Fix<y>? yes

Free blocks count wrong for group #476 (24271, counted=24272).
Fix<y>? yes

Inode bitmap differences:  -4538519
Fix<y>? yes                                                                     

Free inodes count wrong for group #277 (16314, counted=16315).
Fix<y>? yes

Free inodes count wrong for group #356 (15705, counted=15704).
Fix<y>? yes


public: ***** FILE SYSTEM WAS MODIFIED *****

  115332 inodes used (1.47%)
    4917 non-contiguous inodes (4.3%)
         # of inodes with ind/dind/tind blocks: 16957/724/0
 6593745 blocks used (41.92%)
       0 bad blocks
       0 large files

  104607 regular files
   10716 directories
       0 character device files
       0 block device files
       0 fifos
       0 links
       0 symbolic links (0 fast symbolic links)
       0 sockets
--------
  115322 files                  

--

Here is the settings for the file system:

root@menno:/var/log# tune2fs -l /dev/vg_on_md0/public 
tune2fs 1.40-WIP (14-Nov-2006)
Filesystem volume name:   public
Last mounted on:          <not available>
Filesystem UUID:          a0abccbc-79a7-4fff-95d6-b9a5bd7a1d61
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype sparse_super
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7864320
Block count:              15728640
Reserved block count:     786432
Free blocks:              9134895
Free inodes:              7748988
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Filesystem created:       Wed Jan 19 23:28:12 2005
Last mount time:          Wed May 16 17:04:29 2007
Last write time:          Wed May 23 14:39:10 2007
Mount count:              0
Maximum mount count:      35
Last checked:             Wed May 23 14:39:10 2007
Check interval:           15552000 (6 months)
Next check after:         Mon Nov 19 13:39:10 2007
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      f706d258-fa07-4d1b-b556-9d5d555c0021
Journal backup:           inode blocks 

--

The file system is on a LVM2 volume "vg_on_md0", which is on a RAID1 
mirror on /dev/md0.

from /etc/fstab:
/dev/vg_on_md0/public   /data/public    ext3    acl             0       
0

# vgdisplay

 --- Volume group ---
  VG Name               vg_on_md0
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  72
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                2
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               186.30 GB
  PE Size               4.00 MB
  Total PE              47694
  Alloc PE / Size       47693 / 186.30 GB
  Free  PE / Size       1 / 4.00 MB
  VG UUID               eSnn0u-HyNr-FJav-xiDJ-oGuG-2dPi-FqwFE5                  

# pvdisplay

--- Physical volume ---
  PV Name               /dev/md0
  VG Name               vg_on_md0
  PV Size               186.30 GB / not usable 0   
  Allocatable           yes 
  PE Size (KByte)       4096
  Total PE              47694
  Free PE               1
  Allocated PE          47693
  PV UUID               M4KXnz-C0rq-IM84-WHYE-Z4wv-LEsF-4JHxux
                    
# cat /proc/mdstat 
Personalities : [raid1] 
....      
md0 : active raid1 sda1[0] sdb1[1]
      195358336 blocks [2/2] [UU] 

The hardware:
2 x Western Digital Caviar WD2000 Serial ATA (in a software RAID1 mirror)
 
# lspci
....
01:06.0 Mass storage controller: Silicon Image, Inc. SiI 3112 
[SATALink/SATARaid] Serial ATA Controller (rev 02)
....

--

I'll provide whatever other information is requested.

Regards,
Tim



-- System Information:
Debian Release: 4.0
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-4-k7
Locale: LANG=en_CA.utf8, LC_CTYPE=en_CA.utf8 (charmap=UTF-8)

Versions of packages linux-image-2.6.18-4-k7 depends on:
ii  coreutils                     5.97-5.3   The GNU core utilities
ii  debconf [debconf-2.0]         1.5.11     Debian configuration management sy
ii  initramfs-tools [linux-initra 0.85g      tools for generating an initramfs
ii  module-init-tools             3.3-pre4-2 tools for managing Linux kernel mo

Versions of packages linux-image-2.6.18-4-k7 recommends:
ii  libc6-i686                  2.3.6.ds1-13 GNU C Library: Shared libraries [i

-- debconf information:
  shared/kernel-image/really-run-bootloader: true
* linux-image-2.6.18-4-k7/preinst/already-running-this-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/old-system-map-link-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/abort-install-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/bootloader-test-error-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/bootloader-error-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/preinst/abort-overwrite-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/depmod-error-initrd-2.6.18-4-k7: false
  linux-image-2.6.18-4-k7/prerm/would-invalidate-boot-loader-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/bootloader-initrd-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/prerm/removing-running-kernel-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/overwriting-modules-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/postinst/depmod-error-2.6.18-4-k7: false
  linux-image-2.6.18-4-k7/postinst/create-kimage-link-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/initrd-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/old-initrd-link-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/elilo-initrd-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/failed-to-move-modules-2.6.18-4-k7:
  linux-image-2.6.18-4-k7/postinst/old-dir-initrd-link-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/postinst/kimage-is-a-directory:
  linux-image-2.6.18-4-k7/preinst/lilo-initrd-2.6.18-4-k7: true
  linux-image-2.6.18-4-k7/preinst/lilo-has-ramdisk:



Reply to: