--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: live-build: Hdd image fails due to hard links in binary directory
- From: Matthijs Kooijman <matthijs@stdin.nl>
- Date: Tue, 29 Aug 2017 20:06:09 +0200
- Message-id: <150402996983.13854.5401752848041898038.reportbug@grubby.stderr.nl>
Package: live-build
Version: 1:20161202
Severity: normal
Tags: patch
Hi Raphaël,
building a hdd image currently fails. After fixing #865586 (mkfs
complaining about existing partitions) I see:
$ sudo lb config -b hdd --bootloader syslinux
$ sudo lb build
(...)
P: Copying binary contents into image...
cp: error writing 'chroot/binary.tmp/live/initrd.img': No space left on device
cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents.
To fix this I've attached a patch that, passes --count-links to du when
the target is FAT, to make the space estimation correct.
This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should handle
hardlinks more gracefully in any case.
I've tested with version 1:20170807 and 1:20170829, but marked this bug
as found in 1:20161202 since that is the first version that has the
above commit.
Gr.
Matthijs
>From 0c313e9ed3d0de7cfa6ef8e8e21c01a08b8e4a8c Mon Sep 17 00:00:00 2001
From: Matthijs Kooijman <matthijs@stdin.nl>
Date: Tue, 29 Aug 2017 14:50:46 +0200
Subject: [PATCH 4/4] Handle hardlinks in binary_hdd
To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents, breaking the build:
P: Copying binary contents into image...
cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
cp: error writing 'chroot/binary.tmp/efi/boot/bootx64.efi': No space left on device
cp: error writing 'chroot/binary.tmp/efi/boot/bootia32.efi': No space left on device
cp: cannot create directory 'chroot/binary.tmp/boot/grub': No space left on device
cp: cannot create directory 'chroot/binary.tmp/isolinux': No space left on device
To fix this, pass --count-links to du when the target is FAT, to make
the space estimation correct.
This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should at least
handle hardlinks more gracefully.
---
scripts/build/binary_hdd | 17 +++++++++++++++--
1 file changed, 15 insertions(+), 2 deletions(-)
diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index 400403c0a..c6b842e95 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -97,6 +97,19 @@ then
rm -f ${LIVE_iMAGE_NAME}.img
fi
+case "${LB_BINARY_FILESYSTEM}" in
+ fat*)
+ # If the target does not support hardlinks, tell du to
+ # count them double
+ DU_OPTIONS="--count-links"
+ ;;
+
+ *)
+ DU_OPTIONS=""
+ ;;
+esac
+
+
# Enforce fat32 if we find individual files bigger than 2GB
if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ -n "$(find binary -size +1999M)" ]
then
@@ -107,7 +120,7 @@ then
fi
# Enforce fat32 if we have images in total bigger than 2GB
-if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ "$(du -s binary | awk '{ print $1 }')" -gt "1900000" ]
+if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ "$(du ${DU_OPTIONS} -s binary | awk '{ print $1 }')" -gt "1900000" ]
then
Echo_warning "FAT16 doesn't support partitions larger than 2GB, automatically enforcing FAT32"
@@ -127,7 +140,7 @@ fi
# Everything which comes here needs to be cleaned up,
if [ "$LB_HDD_SIZE" = "auto" ];
then
- DU_DIM="$(du -ms binary | cut -f1)"
+ DU_DIM="$(du ${DU_OPTIONS} -ms binary | cut -f1)"
REAL_DIM="$(Calculate_partition_size ${DU_DIM} ${LB_BINARY_FILESYSTEM})"
else
REAL_DIM=$LB_HDD_SIZE
--
2.11.0
--- End Message ---
--- Begin Message ---
Source: live-build
Source-Version: 1:20170920
We believe that the bug you reported is fixed in the latest version of
live-build, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 873640@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Raphaël Hertzog <hertzog@debian.org> (supplier of updated live-build package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmaster@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Format: 1.8
Date: Wed, 20 Sep 2017 10:01:42 +0200
Source: live-build
Binary: live-build
Architecture: source
Version: 1:20170920
Distribution: unstable
Urgency: medium
Maintainer: Debian Live <debian-live@lists.debian.org>
Changed-By: Raphaël Hertzog <hertzog@debian.org>
Description:
live-build - Live System Build Components
Closes: 865586 873513 873640
Changes:
live-build (1:20170920) unstable; urgency=medium
.
* Auto-update version strings in manual pages.
* Update default value of PREPARER to correct the embedded URL.
* Pass --partscan to losetup to clean up partition devices lingering
from former operations. Closes: #865586
Thanks to Matthijs Kooijman <matthijs@stdin.nl> for the patch.
* Fix multiple issues with LB_BUILD_WITH_CHROOT=false. Closes: #873513
Thanks to Matthijs Kooijman <matthijs@stdin.nl> for the patch.
* Fix size calculation when we use FAT filesystems that do not support
hardlinks. Closes: #873640
Thanks to Matthijs Kooijman <matthijs@stdin.nl> for the patch.
* Replace "kirkwood" and "orion5x" armel flavors with the new
"marvell" (as done by the linux source package since its version 4.4).
Checksums-Sha1:
ca796fe66659ad0c88482d23d25f198f180d17b6 1384 live-build_20170920.dsc
9a53ae227e81f49da64d65b74a8b550cb0a762cf 354112 live-build_20170920.tar.xz
fac412dc0ec56fe242ba9521b304743420a59fdf 5227 live-build_20170920_source.buildinfo
Checksums-Sha256:
ad96beda68a715906bb12aa71bdd760476cd370d8ae6b56b91bc15f141dbe031 1384 live-build_20170920.dsc
b62a7df886c3efb3e5f647aeb33adc696349cdbfe96cad91421f628261effe51 354112 live-build_20170920.tar.xz
21f0f2e47e1c90813eeb9d05c205fef8ff0d2b8a5ebadf8fc6ecdced5335d759 5227 live-build_20170920_source.buildinfo
Files:
b20201aa6476c64e991806ce07882ef9 1384 misc optional live-build_20170920.dsc
904af9cd4123ddbab8086638cad69d08 354112 misc optional live-build_20170920.tar.xz
1451ad2c13bc0fe990a9a4668f402b04 5227 misc optional live-build_20170920_source.buildinfo
-----BEGIN PGP SIGNATURE-----
Comment: Signed by Raphael Hertzog
iQEzBAEBCgAdFiEE1823g1EQnhJ1LsbSA4gdq+vCmrkFAlnCIIwACgkQA4gdq+vC
mrlTdAgAkWjizKIfYLY4TjkXAN2ch/nDc1Zuz/Nw5NR7b9Z21mikY88B+Y62v3p1
rYDmDFHflWsHwBCBru4zr/ohXfa63hGHAu4UxAUgWufPvfZuEfNSmoVC/FPMpwuO
aLdBmqeQsBDZlFoc4i34+3lkItLkMv7XZjZkdHTqtlzde3KrICLj+5eFpamZI01Z
QJFOZPPjwl/RFiTBP16c2gEplb4yd0YXmFZx2qFkjYr8lS3VOSOPQVzLbBc+eDSP
XUO5XrGukM0dOKIb8LF4MUuaogNAMEKkCuXss/qzQFzxwMzdFQUDBsys2rirOlMj
pbT+zqb0KNiphVLce+3nzXwl+NgoQg==
=0nEx
-----END PGP SIGNATURE-----
--- End Message ---