[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#929486: marked as done (unblock: debootstick/2.3)

Your message dated Tue, 4 Jun 2019 20:24:10 +0200
with message-id <9fdf493e-9903-d67b-b213-4f8bd205cb5f@debian.org>
and subject line Re: Bug#929486: unblock: debootstick/2.3
has caused the Debian Bug report #929486,
regarding unblock: debootstick/2.3
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
929486: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=929486
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems
--- Begin Message --- 
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock

Please unblock package debootstick

Package 2.3 of debootstick was just uploaded to experimental by my sponsor (Vincent Danjean).
If you accept this unblock request, he will upload it to unstable.

This release compiles three bug fixes I hope buster users can get:
1- Bug#928080 - A fix about migration mode (long timeouts on LVM commands, was mostly unusable)
2- Bug#929355 - A fix about sporadic failures with LVM commands
3- Bug#929356 - A fix for debootstick to work with a chroot based on Ubuntu >= 18.04

Severity of bugs 1 and 2 is "important".
Bug 3 was reported with severity "normal". However, fixing it just meant updating two
constant values (see attachment "fix-3.diff"), and it improves significantly the usability of
the software. So I hope this fix can be included.
If not, I can rebuild the package.

The debdiff is attached.
For clarity, I also attached a separate diff file showing code changes related to each of the 3 fixes.

Thanks.

unblock debootstick/2.3

-- System Information:
Debian Release: 10.0
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.19.0-5-amd64 (SMP w/1 CPU core)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

diff -Nru debootstick-2.2/debian/changelog debootstick-2.3/debian/changelog
--- debootstick-2.2/debian/changelog	2019-03-08 12:48:40.000000000 +0000
+++ debootstick-2.3/debian/changelog	2019-05-22 12:03:47.000000000 +0000
@@ -1,3 +1,11 @@
+debootstick (2.3) unstable; urgency=medium
+
+  * Fix new LVM versions hanging in migration procedure (closes: Bug#928080).
+  * Fix migration script sometimes failing (closes: Bug#929355).
+  * Fix handling of chroots based on Ubuntu >= 18.04 (closes: Bug#929356).
+
+ -- Etienne Dublé <etienne.duble@gmail.com>  Wed, 22 May 2019 13:56:42 +0000
+
 debootstick (2.2) unstable; urgency=medium
 
   * Fix new LVM versions hanging in chroot (closes: Bug#923852).
diff -Nru debootstick-2.2/debootstick debootstick-2.3/debootstick
--- debootstick-2.2/debootstick	2019-03-08 12:40:46.000000000 +0000
+++ debootstick-2.3/debootstick	2019-05-22 12:03:47.000000000 +0000
@@ -5,7 +5,7 @@
 IMAGE_SIZE_MARGIN_KB=0  # fs size estimation is enough pessimistic
 MKSQUASHFS_OPTS="-b 1M -comp xz"
 ONE_GIGABYTE_KB=$((1024*1024))
-MAX_WORK_IMAGE_OVERHEAD_KB=$ONE_GIGABYTE_KB
+MAX_WORK_IMAGE_OVERHEAD_KB=$((2*ONE_GIGABYTE_KB))
 DEBUG=0
 DD="dd status=none"
 DBSTCK_DIR="/usr/share/debootstick"
@@ -290,8 +290,8 @@
 echo done
 release_image draft     # not needed anymore
 
-# add the dbstck.conf file
-cat > $final_rootfs_mountpoint/dbstck.conf << EOF
+# complete the dbstck.conf file
+cat >> $final_rootfs_mountpoint/dbstck.conf << EOF
 STICK_OS_ID=$STICK_OS_ID
 USE_LVM=$(target_use_lvm)
 SYSTEM_TYPE=$system_type
diff -Nru debootstick-2.2/scripts/create-image/common/finalize debootstick-2.3/scripts/create-image/common/finalize
--- debootstick-2.2/scripts/create-image/common/finalize	2018-03-15 08:56:17.000000000 +0000
+++ debootstick-2.3/scripts/create-image/common/finalize	2019-05-22 11:50:39.000000000 +0000
@@ -9,8 +9,9 @@
     rm -rf proc/* sys/* dev/* tmp/* \
             $(find run -type f) var/cache/* var/lock
 
-    # move the existing init
-    mv sbin/init sbin/init.orig
-    cd sbin
-    ln -s /opt/debootstick/live/init/first-init.sh init
+    # install debootstick init hook on getty command
+    getty_command="$(realpath --relative-to . "$(readlink -f sbin/getty)")"
+    mv "$getty_command" "${getty_command}.orig"
+    ln -s /opt/debootstick/live/init/getty-hook.sh "$getty_command"
+    echo "GETTY_COMMAND=$getty_command" >> dbstck.conf
 }
diff -Nru debootstick-2.2/scripts/create-image/target/pc/packages debootstick-2.3/scripts/create-image/target/pc/packages
--- debootstick-2.2/scripts/create-image/target/pc/packages	2018-03-15 08:56:17.000000000 +0000
+++ debootstick-2.3/scripts/create-image/target/pc/packages	2019-05-22 12:03:47.000000000 +0000
@@ -2,7 +2,7 @@
 
 custom_packages()
 {
-    echo grub-pc
+    echo grub-pc initramfs-tools
 }
 
 kernel_default_package()
diff -Nru debootstick-2.2/scripts/live/init/first-init.sh debootstick-2.3/scripts/live/init/first-init.sh
--- debootstick-2.2/scripts/live/init/first-init.sh	2018-03-15 08:56:17.000000000 +0000
+++ debootstick-2.3/scripts/live/init/first-init.sh	2019-05-22 11:50:39.000000000 +0000
@@ -8,17 +8,8 @@
 # if error, run a shell
 trap '[ "$?" -eq 0 ] || fallback_sh' EXIT
 
-# we will need /proc and /sys
-[ -e /proc/self ] || mount_proc=1
-[ -e /sys/class ] || mount_sys=1
-[ "$mount_proc" = "1" ] && mount -t proc none /proc
-[ "$mount_sys" = "1" ] && mount -t sysfs none /sys
-
-# remount / read-write
-mount -o remount,rw /
-
-# lvm may need this directory to run properly
-mkdir -p /run/lock
+# this script is now called at the end of the OS bootup procedure,
+# (getty hook), no need to mount/umount filesystems
 
 # ask and set the root password if needed
 if [ "$ASK_ROOT_PASSWORD_ON_FIRST_BOOT" = "1" ]
@@ -37,13 +28,3 @@
 # restore the lvm config as it was in the
 # initial chroot environment
 restore_lvm_conf
-
-# cleanup
-[ "$mount_proc" = "1" ] && umount /proc
-[ "$mount_sys" = "1" ] && umount /sys
-
-# restore and start the usual init
-rm /sbin/init
-mv /sbin/init.orig /sbin/init
-exec /sbin/init $*
-
diff -Nru debootstick-2.2/scripts/live/init/getty-hook.sh debootstick-2.3/scripts/live/init/getty-hook.sh
--- debootstick-2.2/scripts/live/init/getty-hook.sh	1970-01-01 00:00:00.000000000 +0000
+++ debootstick-2.3/scripts/live/init/getty-hook.sh	2019-05-22 11:50:39.000000000 +0000
@@ -0,0 +1,28 @@
+#!/bin/bash
+. /dbstck.conf  # get GETTY_COMMAND
+
+first_init()
+{
+    # - we will talk to the console
+    # - since we were called as a subprocess,
+    #   we can avoid leaking the lock fd
+    exec 0</dev/console 1>/dev/console 2>&1 200>&-
+    # run debootstick init procedure
+    /opt/debootstick/live/init/first-init.sh
+}
+
+# several getty processes will be spawned concurrently,
+# we have to use a lock
+{
+    flock 200
+    if [ -f "${GETTY_COMMAND}.orig" ]
+    then
+        # original getty not restored yet
+        # => this means we are first, we will do the job.
+        (first_init)    # execute in a sub-shell
+        # restore original getty
+        mv "${GETTY_COMMAND}.orig" "$GETTY_COMMAND"
+    fi
+} 200>/var/lib/debootstick-init.lock
+
+exec "$GETTY_COMMAND" "$@"
diff -Nru debootstick-2.2/scripts/live/init/migrate-to-disk.sh debootstick-2.3/scripts/live/init/migrate-to-disk.sh
--- debootstick-2.2/scripts/live/init/migrate-to-disk.sh	2018-03-15 08:56:17.000000000 +0000
+++ debootstick-2.3/scripts/live/init/migrate-to-disk.sh	2019-05-22 12:03:29.000000000 +0000
@@ -70,7 +70,27 @@
 read -t 10 -n 1 && { echo "Aborted!"; exit 1; }
 echo "** Going on."
 
+enforce_lvm_cmd() {
+    udevadm settle; sync; sync
+    while true; do
+        # handle rare failures
+        "$@" 2>/dev/null && break || sleep 1
+    done
+}
+
 {
+    echo MSG making sure ${TARGET} is not used...
+    pvs --no-headings -o pv_name | while read pv_name
+    do
+        [ "$(part_to_disk $pv_name)" == "$TARGET" ] || continue
+        vg=$(vgs --select "pv_name = $pv_name" --noheadings | awk '{print $1}')
+        if [ -n "$vg" ]; then
+            enforce_lvm_cmd vgchange -an "$vg"
+            enforce_lvm_cmd vgremove -ff -y "$vg"
+        fi
+        enforce_lvm_cmd pvremove -ff -y $pv_name
+    done
+
     echo MSG copying the partition scheme...
     sgdisk -Z ${TARGET}
     sgdisk -R ${TARGET} $ORIGIN
@@ -95,14 +115,14 @@
     echo MSG moving the lvm volume content on ${TARGET}...
     part_origin=$(get_part_device ${ORIGIN} $pv_part_num)
     part_target=$(get_part_device ${TARGET} $pv_part_num)
-    yes | pvcreate -ff $part_target
-    vgextend $LVM_VG $part_target
+    enforce_lvm_cmd pvcreate -ff -y $part_target
+    enforce_lvm_cmd vgextend $LVM_VG $part_target
     pvchange -x n $part_origin
     pvmove -i 1 $part_origin | while read pv action percent
     do
         echo REFRESHING_MSG "$percent"
     done
-    vgreduce $LVM_VG $part_origin
+    enforce_lvm_cmd vgreduce $LVM_VG $part_origin
     echo REFRESHING_DONE
 
     echo MSG filling the space available...
@@ -113,9 +133,8 @@
     $BOOTLOADER_INSTALL ${TARGET}
 
     echo MSG making sure ${ORIGIN} is not used anymore...
-    pvremove $part_origin
-    sync; sync
-    partx -d ${ORIGIN}
+    enforce_lvm_cmd pvremove -ff -y $part_origin
+    enforce_lvm_cmd partx -d ${ORIGIN}
 
     echo RETURN 0
 } | filter_quiet

diff --git a/debootstick b/debootstick
index 2bfbf36..2a9bc9c 100755
--- a/debootstick
+++ b/debootstick
@@ -290,8 +290,8 @@ cp -a $draft_rootfs_mountpoint/* $final_rootfs_mountpoint/
 echo done
 release_image draft     # not needed anymore
 
-# add the dbstck.conf file
-cat > $final_rootfs_mountpoint/dbstck.conf << EOF
+# complete the dbstck.conf file
+cat >> $final_rootfs_mountpoint/dbstck.conf << EOF
 STICK_OS_ID=$STICK_OS_ID
 USE_LVM=$(target_use_lvm)
 SYSTEM_TYPE=$system_type
diff --git a/scripts/create-image/common/finalize b/scripts/create-image/common/finalize
index e2b50fb..c87c9fe 100644
--- a/scripts/create-image/common/finalize
+++ b/scripts/create-image/common/finalize
@@ -9,8 +9,9 @@ finalize_fs()
     rm -rf proc/* sys/* dev/* tmp/* \
             $(find run -type f) var/cache/* var/lock
 
-    # move the existing init
-    mv sbin/init sbin/init.orig
-    cd sbin
-    ln -s /opt/debootstick/live/init/first-init.sh init
+    # install debootstick init hook on getty command
+    getty_command="$(realpath --relative-to . "$(readlink -f sbin/getty)")"
+    mv "$getty_command" "${getty_command}.orig"
+    ln -s /opt/debootstick/live/init/getty-hook.sh "$getty_command"
+    echo "GETTY_COMMAND=$getty_command" >> dbstck.conf
 }
diff --git a/scripts/live/init/first-init.sh b/scripts/live/init/first-init.sh
index bbde19b..3a006db 100755
--- a/scripts/live/init/first-init.sh
+++ b/scripts/live/init/first-init.sh
@@ -8,17 +8,8 @@ INIT_SCRIPTS_DIR=/opt/debootstick/live/init
 # if error, run a shell
 trap '[ "$?" -eq 0 ] || fallback_sh' EXIT
 
-# we will need /proc and /sys
-[ -e /proc/self ] || mount_proc=1
-[ -e /sys/class ] || mount_sys=1
-[ "$mount_proc" = "1" ] && mount -t proc none /proc
-[ "$mount_sys" = "1" ] && mount -t sysfs none /sys
-
-# remount / read-write
-mount -o remount,rw /
-
-# lvm may need this directory to run properly
-mkdir -p /run/lock
+# this script is now called at the end of the OS bootup procedure,
+# (getty hook), no need to mount/umount filesystems
 
 # ask and set the root password if needed
 if [ "$ASK_ROOT_PASSWORD_ON_FIRST_BOOT" = "1" ]
@@ -37,13 +28,3 @@ fi
 # restore the lvm config as it was in the
 # initial chroot environment
 restore_lvm_conf
-
-# cleanup
-[ "$mount_proc" = "1" ] && umount /proc
-[ "$mount_sys" = "1" ] && umount /sys
-
-# restore and start the usual init
-rm /sbin/init
-mv /sbin/init.orig /sbin/init
-exec /sbin/init $*
-
diff --git a/scripts/live/init/getty-hook.sh b/scripts/live/init/getty-hook.sh
new file mode 100755
index 0000000..efcb0b9
--- /dev/null
+++ b/scripts/live/init/getty-hook.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+. /dbstck.conf  # get GETTY_COMMAND
+
+first_init()
+{
+    # - we will talk to the console
+    # - since we were called as a subprocess,
+    #   we can avoid leaking the lock fd
+    exec 0</dev/console 1>/dev/console 2>&1 200>&-
+    # run debootstick init procedure
+    /opt/debootstick/live/init/first-init.sh
+}
+
+# several getty processes will be spawned concurrently,
+# we have to use a lock
+{
+    flock 200
+    if [ -f "${GETTY_COMMAND}.orig" ]
+    then
+        # original getty not restored yet
+        # => this means we are first, we will do the job.
+        (first_init)    # execute in a sub-shell
+        # restore original getty
+        mv "${GETTY_COMMAND}.orig" "$GETTY_COMMAND"
+    fi
+} 200>/var/lib/debootstick-init.lock
+
+exec "$GETTY_COMMAND" "$@"

diff --git a/scripts/live/init/migrate-to-disk.sh b/scripts/live/init/migrate-to-disk.sh
index ec95b9d..013e1f1 100755
--- a/scripts/live/init/migrate-to-disk.sh
+++ b/scripts/live/init/migrate-to-disk.sh
@@ -70,7 +70,27 @@ echo "** WARNING: Press any key NOW to cancel this process."
 read -t 10 -n 1 && { echo "Aborted!"; exit 1; }
 echo "** Going on."
 
+enforce_lvm_cmd() {
+    udevadm settle; sync; sync
+    while true; do
+        # handle rare failures
+        "$@" 2>/dev/null && break || sleep 1
+    done
+}
+
 {
+    echo MSG making sure ${TARGET} is not used...
+    pvs --no-headings -o pv_name | while read pv_name
+    do
+        [ "$(part_to_disk $pv_name)" == "$TARGET" ] || continue
+        vg=$(vgs --select "pv_name = $pv_name" --noheadings | awk '{print $1}')
+        if [ -n "$vg" ]; then
+            enforce_lvm_cmd vgchange -an "$vg"
+            enforce_lvm_cmd vgremove -ff -y "$vg"
+        fi
+        enforce_lvm_cmd pvremove -ff -y $pv_name
+    done
+
     echo MSG copying the partition scheme...
     sgdisk -Z ${TARGET}
     sgdisk -R ${TARGET} $ORIGIN
@@ -95,14 +115,14 @@ echo "** Going on."
     echo MSG moving the lvm volume content on ${TARGET}...
     part_origin=$(get_part_device ${ORIGIN} $pv_part_num)
     part_target=$(get_part_device ${TARGET} $pv_part_num)
-    yes | pvcreate -ff $part_target
-    vgextend $LVM_VG $part_target
+    enforce_lvm_cmd pvcreate -ff -y $part_target
+    enforce_lvm_cmd vgextend $LVM_VG $part_target
     pvchange -x n $part_origin
     pvmove -i 1 $part_origin | while read pv action percent
     do
         echo REFRESHING_MSG "$percent"
     done
-    vgreduce $LVM_VG $part_origin
+    enforce_lvm_cmd vgreduce $LVM_VG $part_origin
     echo REFRESHING_DONE
 
     echo MSG filling the space available...
@@ -113,9 +133,8 @@ echo "** Going on."
     $BOOTLOADER_INSTALL ${TARGET}
 
     echo MSG making sure ${ORIGIN} is not used anymore...
-    pvremove $part_origin
-    sync; sync
-    partx -d ${ORIGIN}
+    enforce_lvm_cmd pvremove -ff -y $part_origin
+    enforce_lvm_cmd partx -d ${ORIGIN}
 
     echo RETURN 0
 } | filter_quiet

diff --git a/debootstick b/debootstick
index 2a9bc9c..e26c07e 100755
--- a/debootstick
+++ b/debootstick
@@ -5,7 +5,7 @@ VERSION='@VERSION@'     # updated at install time
 IMAGE_SIZE_MARGIN_KB=0  # fs size estimation is enough pessimistic
 MKSQUASHFS_OPTS="-b 1M -comp xz"
 ONE_GIGABYTE_KB=$((1024*1024))
-MAX_WORK_IMAGE_OVERHEAD_KB=$ONE_GIGABYTE_KB
+MAX_WORK_IMAGE_OVERHEAD_KB=$((2*ONE_GIGABYTE_KB))
 DEBUG=0
 DD="dd status=none"
 DBSTCK_DIR="/usr/share/debootstick"
diff --git a/scripts/create-image/target/pc/packages b/scripts/create-image/target/pc/packages
index 844dde6..ee05ce3 100644
--- a/scripts/create-image/target/pc/packages
+++ b/scripts/create-image/target/pc/packages
@@ -2,7 +2,7 @@
 
 custom_packages()
 {
-    echo grub-pc
+    echo grub-pc initramfs-tools
 }
 
 kernel_default_package()

--- End Message ---
--- Begin Message --- 
Hi,

On 03-06-2019 11:05, Etienne Dublé wrote:
> The package has been uploaded to unstable, with version increased to
> 2.4. It obviously has the same content as v2.3, except the appropriate
> changelog entry.

Unblocked, thanks.

Paul

Attachment: signature.asc
Description: OpenPGP digital signature


--- End Message ---
Reply to: