[PATCH] More resilient detachment of loopback devices
Hello,
Please accept this patch which makes the following changes to make detachment of loopback devices while building usb images more resilient:
- Consolidates code for detaching loopback device into common function called Lodetach.
- I've discovered that changes to loopdevices causes udev rules/hooks to execute which cause other parts of the stack to access the loopdevice leading to a race condition when we're trying to detach the loopdevice. Unfortunately, sleep 1 is not entirely sufficient to prevent this from occurring. In the Lodetach function, I perform a 'udevadm settle', 'sync', and 'sleep 1'.
- Although I was not able to reproduce the race condition again after this change, further investigation into this issue led me to determine that 'udevadm settle' is not a guarantee. For this reason, I've also added code so that live-build will attempt three times to remove the loopdevice instead of failing the entire build immediately.
Although I wish we could have a definitive solution to this that doesn't require retry code, its unfortunately not possible with the current linux stack. However, this is only a small detraction from my happiness of having a very high degree of confidence that I won't have builds fail anymore due to a busy or unavailable loopback device. ;)
Cheers,
--
Cody A.W. Somerville
Release Engineer
Foundations Team
Custom Engineering Solutions Group
Canonical OEM Services
Phone: +1 781 850 2087
Cell: +1 613 401 5141
Fax: +1 613 687 7368
Email: cody.somerville@canonical.com
=== modified file 'functions/losetup.sh'
--- functions/losetup.sh 2011-03-09 18:20:42 +0000
+++ functions/losetup.sh 2011-07-21 14:17:31 +0000
@@ -7,6 +7,29 @@
## This is free software, and you are welcome to redistribute it
## under certain conditions; see COPYING for details.
+Lodetach ()
+{
+ DEVICE="${1}"
+ ATTEMPT="${2:-1}"
+
+ if [ "${ATTEMPT}" -gt 3 ]
+ then
+ Echo_error "Failed to detach loop device '${DEVICE}'."
+ exit 1
+ fi
+
+ # Changes to block devices result in uevents which trigger rules which in
+ # turn access the loop device (ex. udisks-part-id, blkid) which can cause
+ # a race condition. We call 'udevadm settle' to help avoid this.
+ ${LB_ROOT_COMMAND} udevadm settle
+
+ # Loop back devices aren't the most reliable when it comes to writes.
+ # We sleep and sync for good measure - better than build failure.
+ sync
+ sleep 1
+
+ ${LB_ROOT_COMMAND} ${LB_LOSETUP} -d "${DEVICE}" || Lodetach "${DEVICE}" "$(expr ${ATTEMPT} + 1)"
+}
Losetup ()
{
@@ -14,9 +37,9 @@
FILE="${2}"
PARTITION="${3:-1}"
- ${LB_ROOT_COMMAND} ${LB_LOSETUP} "${DEVICE}" "${FILE}"
+ ${LB_ROOT_COMMAND} ${LB_LOSETUP} --read-only "${DEVICE}" "${FILE}"
FDISK_OUT="$(${LB_FDISK} -l -u ${DEVICE} 2>&1)"
- ${LB_ROOT_COMMAND} ${LB_LOSETUP} -d "${DEVICE}"
+ Lodetach "${DEVICE}"
LOOPDEVICE="$(echo ${DEVICE}p${PARTITION})"
=== modified file 'scripts/build/lb_binary_usb'
--- scripts/build/lb_binary_usb 2011-03-21 19:52:50 +0000
+++ scripts/build/lb_binary_usb 2011-07-21 14:17:31 +0000
@@ -164,8 +164,7 @@
;;
esac
-sleep 1
-${LB_LOSETUP} -d ${FREELO}
+Lodetach ${FREELO}
FREELO="$(${LB_LOSETUP} -f)"
Losetup $FREELO chroot/binary.img 1
@@ -260,8 +259,7 @@
esac
fi
-sleep 1
-${LB_LOSETUP} -d ${FREELO}
+Lodetach ${FREELO}
echo "!!! The above error/warning messages can be ignored !!!"
=== modified file 'scripts/build/lb_source_usb'
--- scripts/build/lb_source_usb 2011-03-09 18:20:42 +0000
+++ scripts/build/lb_source_usb 2011-07-21 14:17:31 +0000
@@ -94,8 +94,7 @@
Chroot chroot "parted -s ${FREELO} mklabel msdos" || true
Chroot chroot "parted -s ${FREELO} mkpart primary ${PARTITION_TYPE} 0.0 100%" || true
Chroot chroot "parted -s ${FREELO} set 1 lba off" || true
-sleep 1
-${LB_LOSETUP} -d ${FREELO}
+Lodetach ${FREELO}
Losetup $FREELO source.img 1
@@ -123,8 +122,7 @@
cp -r source/* source.tmp
${LB_ROOT_COMMAND} umount source.tmp
rmdir source.tmp
-sleep 1
-${LB_LOSETUP} -d ${FREELO}
+Lodetach ${FREELO}
Echo_warning "!!! The above error/warning messages can be ignored !!!"
if [ -n "${MAKEDEV}" ]
Reply to: