Re: SoftRAID with network iSCSI multipath
Hi,
Thanks all for the suggestions:
On 26/11/13 07:54, Ritesh Raj Sarraf wrote:
Mounting the filesystem is not the problem.
It's just an ordering problem, of the init scripts.
As I have replied in the proxmox-user mailing list :
You should force multipath to run (or re-run) after open-iscsi is
started.
I would try to add a boot dependandcy in /etc/init.d/multipath-tools,
to start after open-iscsi.
Same thing with /etc/init.d/mdadm-raid, to start after multipath-tools.
Yes. And it already does that.
#! /bin/sh
### BEGIN INIT INFO
# Provides: multipath-tools
# Required-Start: udev $local_fs $remote_fs $syslog
# Required-Stop: udev $local_fs $remote_fs $syslog
# Should-Start: iscsi
# Should-Stop: iscsi
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: multipath daemon
# Description:
### END INIT INFO
It doesn't seem to work, md starts before network is up. From dmesg (snips):
[...]
md: md0 stopped.
md: bind<sdb>
md: raid1 personality registered for level 1
bio: create slab <bio-1> at 1
md/raid1:md0: active with 1 out of 2 mirrors
created bitmap (2 pages) for device md0
md0: bitmap initialized from disk: read 1 pages, set 7 of 3800 bits
md0: detected capacity change from 0 to 254998806528
md0: unknown partition table
[...]
device-mapper: multipath: version 1.5.0 loaded
device-mapper: multipath round-robin: version 1.0.0 loaded
device-mapper: table: 253:1: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:1: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:1: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:1: multipath: error getting device
device-mapper: ioctl: error adding target to table
[...]
EXT4-fs (md0): mounted filesystem with ordered data mode. Opts:
alloc irq_desc for 60 on node -1
alloc kstat_irqs on node -1
alloc irq_2_iommu on node -1
[...]
bnx2 0000:0b:00.0: irq 83 for MSI/MSI-X
bnx2 0000:0b:00.0: eth0: using MSIX
NET: Registered protocol family 10
ADDRCONF(NETDEV_UP): eth0: link is not ready
ADDRCONF(NETDEV_UP): eth1: link is not ready
ADDRCONF(NETDEV_UP): eth2: link is not ready
ADDRCONF(NETDEV_UP): eth3: link is not ready
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
[...]
bnx2 0000:0b:00.1: eth1: NIC Copper Link is Up, 1000 Mbps full duplex
ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
Loading iSCSI transport class v2.0-870.
iscsi: registered transport (tcp)
iscsi: registered transport (iser)
bnx2 0000:10:00.1: eth3: NIC Copper Link is Up, 1000 Mbps full duplex
ADDRCONF(NETDEV_CHANGE): eth3: link becomes ready
bnx2 0000:10:00.0: eth2: NIC Copper Link is Up, 1000 Mbps full duplex
ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
bnx2 0000:0b:00.0: eth0: NIC Copper Link is Up, 1000 Mbps full duplex
vmbr0: port 1(eth0) entering forwarding state
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
scsi5 : iSCSI Initiator over TCP/IP
scsi6 : iSCSI Initiator over TCP/IP
scsi7 : iSCSI Initiator over TCP/IP
scsi8 : iSCSI Initiator over TCP/IP
scsi 6:0:0:0: Direct-Access DGC RAID 5 0223 PQ: 0 ANSI: 4
scsi 8:0:0:0: Direct-Access DGC RAID 5 0223 PQ: 0 ANSI: 4
sd 6:0:0:0: Attached scsi generic sg3 type 0
sd 8:0:0:0: Attached scsi generic sg4 type 0
scsi 5:0:0:0: Direct-Access DGC RAID 5 0223 PQ: 0 ANSI: 4
scsi 7:0:0:0: Direct-Access DGC RAID 5 0223 PQ: 0 ANSI: 4
sd 5:0:0:0: Attached scsi generic sg5 type 0
sd 7:0:0:0: Attached scsi generic sg6 type 0
sd 6:0:0:0: [sdc] 503316480 512-byte logical blocks: (257 GB/240 GiB)
sd 8:0:0:0: [sdd] 503316480 512-byte logical blocks: (257 GB/240 GiB)
sd 6:0:0:0: [sdc] Test WP failed, assume Write Enabled
sd 8:0:0:0: [sdd] Test WP failed, assume Write Enabled
sd 6:0:0:0: [sdc] Asking for cache data failed
sd 6:0:0:0: [sdc] Assuming drive cache: write through
sd 8:0:0:0: [sdd] Asking for cache data failed
sd 8:0:0:0: [sdd] Assuming drive cache: write through
sd 5:0:0:0: [sde] 503316480 512-byte logical blocks: (257 GB/240 GiB)
sd 7:0:0:0: [sdf] 503316480 512-byte logical blocks: (257 GB/240 GiB)
sd 5:0:0:0: [sde] Write Protect is off
sd 5:0:0:0: [sde] Mode Sense: 7d 00 00 08
sd 7:0:0:0: [sdf] Write Protect is off
sd 7:0:0:0: [sdf] Mode Sense: 7d 00 00 08
sd 5:0:0:0: [sde] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
sd 7:0:0:0: [sdf] Write cache: disabled, read cache: enabled, doesn't
support DPO or FUA
sde:
sd 8:0:0:0: [sdd] Test WP failed, assume Write Enabled
sd 8:0:0:0: [sdd] Asking for cache data failed
sd 8:0:0:0: [sdd] Assuming drive cache: write through
sd 6:0:0:0: [sdc] Test WP failed, assume Write Enabled
sd 8:0:0:0: [sdd] Device not ready
sd 8:0:0:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 8:0:0:0: [sdd] Sense Key : Not Ready [current]
sd 8:0:0:0: [sdd] Add. Sense: Logical unit not ready, manual
intervention required
sd 8:0:0:0: [sdd] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
Buffer I/O error on device sdd, logical block 0
sd 6:0:0:0: [sdc] Asking for cache data failed
sd 6:0:0:0: [sdc] Assuming drive cache: write through
sdc:
sd 6:0:0:0: [sdc] Device not ready
sdf:
sd 6:0:0:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 6:0:0:0: [sdc] Sense Key : Not Ready [current]
sd 6:0:0:0: [sdc] Add. Sense: Logical unit not ready, manual
intervention required
sd 6:0:0:0: [sdc] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
Buffer I/O error on device sdc, logical block 0
[...continue some device not ready repeated messages...]
end_request: I/O error, dev sdd, sector 0
end_request: I/O error, dev sdd, sector 0
Dev sdd: unable to read RDB block 0
end_request: I/O error, dev sdc, sector 0
end_request: I/O error, dev sdd, sector 0
end_request: I/O error, dev sdd, sector 0
unable to read partition table
end_request: I/O error, dev sdc, sector 0
Dev sdc: unable to read RDB block 0
unknown partition table
unknown partition table
end_request: I/O error, dev sdc, sector 0
end_request: I/O error, dev sdc, sector 0
unable to read partition table
sd 7:0:0:0: [sdf] Attached SCSI disk
sd 5:0:0:0: [sde] Attached SCSI disk
sd 8:0:0:0: [sdd] Test WP failed, assume Write Enabled
sd 8:0:0:0: [sdd] Asking for cache data failed
sd 8:0:0:0: [sdd] Assuming drive cache: write through
sd 8:0:0:0: [sdd] Attached SCSI disk
sd 6:0:0:0: [sdc] Test WP failed, assume Write Enabled
sd 6:0:0:0: [sdc] Asking for cache data failed
sd 6:0:0:0: [sdc] Assuming drive cache: write through
sd 6:0:0:0: [sdc] Attached SCSI disk
end_request: I/O error, dev sdc, sector 503316352
end_request: I/O error, dev sdc, sector 503316352
end_request: I/O error, dev sdd, sector 503316352
end_request: I/O error, dev sdc, sector 503316464
end_request: I/O error, dev sdd, sector 503316352
end_request: I/O error, dev sdd, sector 503316464
end_request: I/O error, dev sdc, sector 503316464
end_request: I/O error, dev sdd, sector 503316464
end_request: I/O error, dev sdc, sector 0
end_request: I/O error, dev sdd, sector 0
[...about 200 similar error lines....]
[...]
sd 6:0:0:0: emc: detected Clariion AX4-5i, flags 0
sd 6:0:0:0: emc: connected to SP B Port 0 (bound, default SP A)
sd 8:0:0:0: emc: detected Clariion AX4-5i, flags 0
sd 8:0:0:0: emc: connected to SP B Port 1 (bound, default SP A)
sd 5:0:0:0: emc: detected Clariion AX4-5i, flags 0
sd 5:0:0:0: emc: connected to SP A Port 0 (owned, default SP A)
sd 7:0:0:0: emc: detected Clariion AX4-5i, flags 0
sd 7:0:0:0: emc: connected to SP A Port 1 (owned, default SP A)
emc: device handler registered
sd 5:0:0:0: emc: at SP A Port 0 (owned, default SP A)
sd 7:0:0:0: emc: at SP A Port 1 (owned, default SP A)
fuse init (API version 7.13)
device-mapper: table: 253:4: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:4: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:4: multipath: error getting device
device-mapper: ioctl: error adding target to table
device-mapper: table: 253:4: multipath: error getting device
device-mapper: ioctl: error adding target to table
sd 5:0:0:0: emc: at SP A Port 0 (owned, default SP A)
sd 7:0:0:0: emc: at SP A Port 1 (owned, default SP A)
DLM (built Oct 14 2013 08:10:28) installed
vmbr0: no IPv6 routers present
eth3: no IPv6 routers present
eth2: no IPv6 routers present
eth1: no IPv6 routers present
eth0: no IPv6 routers present
[...]
Current working multipath -l is:
36006016088012600e47c6e08e846e311 dm-3 DGC,RAID 5
size=240G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='round-robin 0' prio=0 status=active
| |- 5:0:0:0 sde 8:64 active undef running
| `- 7:0:0:0 sdf 8:80 active undef running
`-+- policy='round-robin 0' prio=0 status=enabled
|- 8:0:0:0 sdd 8:48 active undef running
`- 6:0:0:0 sdc 8:32 active undef running
Thanks a lot
Eneko
--
Zuzendari Teknikoa / Director Técnico
Binovo IT Human Project, S.L.
Telf. 943575997
943493611
Astigarraga bidea 2, planta 6 dcha., ofi. 3-2; 20180 Oiartzun (Gipuzkoa)
www.binovo.es
Reply to: