Bug#873640: live-build: Hdd image fails due to hard links in binary directory

[Matthijs Kooijman <matthijs@stdin.nl>]

Package: live-build
Version: 1:20161202
Severity: normal
Tags: patch

Hi Raphaël,

building a hdd image currently fails. After fixing #865586 (mkfs
complaining about existing partitions) I see:

  $ sudo lb config -b hdd --bootloader syslinux
  $ sudo lb build
  (...)

  P: Copying binary contents into image...
  cp: error writing 'chroot/binary.tmp/live/initrd.img': No space left on device
  cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device

To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents.

To fix this I've attached a patch that, passes --count-links to du when
the target is FAT, to make the space estimation correct.

This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should handle
hardlinks more gracefully in any case.

I've tested with version 1:20170807 and 1:20170829, but marked this bug
as found in 1:20161202 since that is the first version that has the
above commit.

Gr.

Matthijs

>From 0c313e9ed3d0de7cfa6ef8e8e21c01a08b8e4a8c Mon Sep 17 00:00:00 2001
From: Matthijs Kooijman <matthijs@stdin.nl>
Date: Tue, 29 Aug 2017 14:50:46 +0200
Subject: [PATCH 4/4] Handle hardlinks in binary_hdd

To generate an hdd image, binary_hdd first estimates the needed size of
the image using du. By default, when du finds multiple hardlinked copies
of a file, it counts them only once. However, when the target filesystem
is FAT, which does not support hardlinks, these files will take up more
space when finally copying the contents, breaking the build:

	P: Copying binary contents into image...
	cp: error writing 'chroot/binary.tmp/live/initrd.img-4.9.0-3-amd64': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootx64.efi': No space left on device
	cp: error writing 'chroot/binary.tmp/efi/boot/bootia32.efi': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/boot/grub': No space left on device
	cp: cannot create directory 'chroot/binary.tmp/isolinux': No space left on device

To fix this, pass --count-links to du when the target is FAT, to make
the space estimation correct.

This problem is exposed by commit 9c974b26b (Instead of renaming kernel
for syslinux, create hardlinks), which might need to be separately fixed
(to not waste space on FAT targets), but binary_hdd should at least
handle hardlinks more gracefully.
---
 scripts/build/binary_hdd | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/scripts/build/binary_hdd b/scripts/build/binary_hdd
index 400403c0a..c6b842e95 100755
--- a/scripts/build/binary_hdd
+++ b/scripts/build/binary_hdd
@@ -97,6 +97,19 @@ then
 	rm -f ${LIVE_iMAGE_NAME}.img
 fi
 
+case "${LB_BINARY_FILESYSTEM}" in
+	fat*)
+		# If the target does not support hardlinks, tell du to
+		# count them double
+		DU_OPTIONS="--count-links"
+		;;
+
+	*)
+		DU_OPTIONS=""
+		;;
+esac
+
+
 # Enforce fat32 if we find individual files bigger than 2GB
 if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ -n "$(find binary -size +1999M)" ]
 then
@@ -107,7 +120,7 @@ then
 fi
 
 # Enforce fat32 if we have images in total bigger than 2GB
-if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ "$(du -s binary | awk '{ print $1 }')" -gt "1900000" ]
+if [ "${LB_BINARY_FILESYSTEM}" = "fat16" ] && [ "$(du ${DU_OPTIONS} -s binary | awk '{ print $1 }')" -gt "1900000" ]
 then
 	Echo_warning "FAT16 doesn't support partitions larger than 2GB, automatically enforcing FAT32"
 
@@ -127,7 +140,7 @@ fi
 # Everything which comes here needs to be cleaned up,
 if [ "$LB_HDD_SIZE" = "auto" ];
 then
-	DU_DIM="$(du -ms binary | cut -f1)"
+	DU_DIM="$(du ${DU_OPTIONS} -ms binary | cut -f1)"
 	REAL_DIM="$(Calculate_partition_size ${DU_DIM} ${LB_BINARY_FILESYSTEM})"
 else
 	REAL_DIM=$LB_HDD_SIZE
-- 
2.11.0