Bug#788295: open-iscsi+multipath+LVM breakage caused by incorrect size reporting
On further investigation, it seems that something (multipath-tools? the
kernel?) doesn't read the size of the underlying devices correctly. I
initially thought this might be a problem with the GPT on the LUN I was
using, but I was able to reproduce this problem on a new LUN.
I did the following:
1: I published a new LUN 22268000155ea65e2 to the server. The system saw
it and multipath started correctly.
2: I partitioned /dev/mapper/22268000155ea65e2 with gdisk.
3: I created a new PV on /dev/mapper/22268000155ea65e2-part1, pvmoved
everything off of the old PV (which I thought was on a disk with an
invalid GPT), removed the old LV from the volume group and rebooted.
4: The same LVM symptoms I described initially occurred on reboot, and
the kernel logged this:
[ 44.081263] device-mapper: table: 254:1: dm-0 too small for target: start=2048, len=12638472159, dev_size=12638472159
[ 44.170954] device-mapper: table: 254:1: dm-0 too small for target: start=2048, len=12638472159, dev_size=12638472159
5: On the underlying volumes (e.g. /dev/sdi), gdisk reports no
problems. On the multipath device, this happens:
$ sudo gdisk /dev/mapper/22268000155ea65e2
GPT fdisk (gdisk) version 0.8.10
Warning! Disk size is smaller than the main header indicates! Loading
secondary header from the last sector of the disk! You should use 'v' to
verify disk integrity, and perhaps options on the experts' menu to repair
the disk.
Caution: invalid backup GPT header, but valid main header; regenerating
backup header from main header.
Warning! One or more CRCs don't match. You should repair the disk!
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: damaged
****************************************************************************
Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk
verification and recovery are STRONGLY recommended.
****************************************************************************
Command (? for help): v
Caution: The CRC for the backup partition table is invalid. This table may
be corrupt. This program will automatically create a new backup partition
table when you save your partitions.
Problem: The secondary header's self-pointer indicates that it doesn't reside
at the end of the disk. If you've added a disk to a RAID array, use the 'e'
option on the experts' menu to adjust the secondary header's and partition
table's locations.
Problem: Disk is too small to hold all the data!
(Disk size is 12638472159 sectors, needs to be 12638474240 sectors.)
The 'e' option on the experts' menu may fix this problem.
Problem: GPT claims the disk is larger than it is! (Claimed last usable
sector is 12638474206, but backup header is at
12638474239 and disk size is 12638472159 sectors.
The 'e' option on the experts' menu will probably fix this problem
Problem: partition 1 is too big for the disk.
Identified 5 problems!
Command (? for help): q
I expect that I'll be able to produce a simple test case without LVM,
but I'm still not sure what's actually at fault.
Reply to: