Unable to Access a Foreign Volume Group
Hello,
I have a buster/sid system with two volume groups that have made no
issue for years.
vg00 has not been actively used for a longer time, I migrated off of it,
but keep it up and around.
vg01 is holding most of the data for the system and works with no
issues.
Both are on two separate raid1's (see details at the end).
The server has no lvm system id set (never had) and both VGs worked
flawlessly for years. I've never concerned myself with setting a system
id for the standalone server, and am assuming neither VG ever had one
set.
During a recent package update apparently vg00 got a system ID set (see
below) and lvm now is ignoring it.
I am struggling to get access back to this VG, going by the
documentation:
# lvm systemid
system ID:
# vgs -o systemid vg00
Cannot access VG vg00 with system ID zaphod1105820973 with unknown local system ID.
Cannot access VG vg00 with system ID zaphod1105820973 with unknown local system ID.
# vgs --foreign -o +systemid
VG #PV #LV #SN Attr VSize VFree System ID
vg00 1 8 0 wz--n- 930.55g 20.00g zaphod1105820973
vg01 1 9 0 wz--n- <1.50t 534.37g
# vgchange --config 'local/extra_system_ids=["zaphod1105820973"]' --systemid "" vg00
Cannot access VG vg00 with system ID zaphod1105820973 with unknown local system ID.
Cannot access VG vg00 with system ID zaphod1105820973 with unknown local system ID.
Any help is appreciated to get back acess to the VG, vgchange (and
vgexport) seem to be ignoring it despite providing the extra system id.
Here are more details on how I think the situation arose, and the system
setup.
During an update via aptitude (2019-01-06 12:08:38 to 2019-01-06 12:15:32)
# grep liblvm2cmd2 /var/log/apt/term.log
Selecting previously unselected package liblvm2cmd2.03:amd64.
Preparing to unpack .../liblvm2cmd2.03_2.03.02-1_amd64.deb ...
Unpacking liblvm2cmd2.03:amd64 (2.03.02-1) ...
Removing liblvm2cmd2.02:amd64 (2.02.176-4.1) ...
Setting up liblvm2cmd2.03:amd64 (2.03.02-1) ...
the following was logged in syslog, the first time the "foreign system
ID" is showing up in my logs:
Jan 6 12:14:39 localhost systemd[1]: lvm2-lvmetad.socket: Socket unit configuration has changed whi
le unit has been running, no open socket file descriptor left. The socket unit is not functional unt
il restarted.
Jan 6 12:14:39 localhost systemd[1]: Stopping Monitoring of LVM2 mirrors, snapshots etc. using dmev
entd or progress polling...
Jan 6 12:14:40 localhost lvm[32256]: 9 logical volume(s) in volume group "vg01" unmonitored
Jan 6 12:14:40 localhost lvm[32256]: WARNING: Found LVs active in VG vg00 with foreign system ID zaphod1105820973. Possible data corruption.
Jan 6 12:14:40 localhost systemd[1]: lvm2-monitor.service: Succeeded.
Jan 6 12:14:40 localhost systemd[1]: Stopped Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Jan 6 12:14:40 localhost systemd[1]: dm-event.socket: Succeeded.
Jan 6 12:14:40 localhost systemd[1]: Closed Device-mapper event daemon FIFOs.
Jan 6 12:14:40 localhost systemd[1]: Stopping Device-mapper event daemon FIFOs.
Jan 6 12:14:40 localhost systemd[1]: Listening on Device-mapper event daemon FIFOs.
Jan 6 12:14:40 localhost systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
Jan 6 12:14:40 localhost lvm[32257]: 9 logical volume(s) in volume group "vg01" monitored
Jan 6 12:14:40 localhost lvm[32257]: WARNING: Found LVs active in VG vg00 with foreign system ID zaphod1105820973. Possible data corruption.
Jan 6 12:14:40 localhost systemd[1]: Started Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling.
Apparently lvm2 was already updated at that point to a newer version
earlier that day (2019-01-04 08:28:56 to 2019-01-04 08:32:37), not
sure how/why this happened:
# grep lvm2 /var/log/apt/term.log
Preparing to unpack .../lvm2_2.03.02-1_amd64.deb ...
Unpacking lvm2 (2.03.02-1) over (2.02.176-4.1) ...
Removing liblvm2app2.2:amd64 (2.02.176-4.1) ...
Setting up lvm2 (2.03.02-1) ...
# apt-cache policy lvm2 liblvm2cmd2.03
lvm2:
Installed: 2.03.02-1
Candidate: 2.03.02-1
Version table:
*** 2.03.02-1 500
500 http://ftp.us.debian.org/debian testing/main amd64 Packages
-100 http://ftp.us.debian.org/debian unstable/main amd64 Packages
100 /var/lib/dpkg/status
liblvm2cmd2.03:
Installed: 2.03.02-1
Candidate: 2.03.02-1
Version table:
*** 2.03.02-1 500
500 http://ftp.us.debian.org/debian testing/main amd64 Packages
-100 http://ftp.us.debian.org/debian unstable/main amd64 Packages
100 /var/lib/dpkg/status
# uname -a
Linux zaphod 4.20.1-20190110.zaphod #1 SMP Thu Jan 10 11:38:55 EST 2019 x86_64 GNU/Linux
Relevant output from lsblk only, vg00 is on md1, vg01 is on md2:
# lsblk -i
sdb 8:16 0 931.5G 0 disk
|-sdb1 8:17 0 979M 0 part
`-sdb2 8:18 0 930.6G 0 part
`-md1 9:1 0 930.6G 0 raid1
sdc 8:32 0 1.8T 0 disk
|-sdc1 8:33 0 512M 0 part
|-sdc2 8:34 0 1M 0 part
|-sdc3 8:35 0 512M 0 part
|-sdc4 8:36 0 1.5T 0 part
| `-md2 9:2 0 1.5T 0 raid1
...
sdd 8:48 0 1.8T 0 disk
|-sdd1 8:49 0 512M 0 part
|-sdd2 8:50 0 1M 0 part
|-sdd3 8:51 0 512M 0 part
|-sdd4 8:52 0 1.5T 0 part
| `-md2 9:2 0 1.5T 0 raid1
...
sde 8:64 0 931.5G 0 disk
|-sde1 8:65 0 979M 0 part
`-sde2 8:66 0 930.6G 0 part
`-md1 9:1 0 930.6G 0 raid1
# vgdisplay --foreign vg00
--- Volume group ---
VG Name vg00
System ID zaphod1105820973
Format lvm2
Metadata Areas 1
Metadata Sequence No 147
VG Access read/write
VG Status resizable
MAX LV 256
Cur LV 8
Open LV 0
Max PV 256
Cur PV 1
Act PV 1
VG Size 930.55 GiB
PE Size 4.00 MiB
Total PE 238221
Alloc PE / Size 233101 / 910.55 GiB
Free PE / Size 5120 / 20.00 GiB
# cat /proc/mdstat
Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md2 : active raid1 sdc4[0] sdd4[1]
1610481664 blocks super 1.2 [2/2] [UU]
bitmap: 4/12 pages [16KB], 65536KB chunk
md1 : active (auto-read-only) raid1 sdb2[2] sde2[4]
975754810 blocks super 1.2 [2/2] [UU]
bitmap: 0/8 pages [0KB], 65536KB chunk
Thanks,
Jens
Reply to: