Bug#700633: Debootstrap is very slow. Please use eatmydata to fix this.
> Debootstrap is very, very slow. I watched a server install crawl by
> for something like an hour at a rate of about 1 package per second. A
> simple debootstrap chroot takes 10 minutes to setup, with a
> predownloaded tarball. This slowness is due to dpkg making excessive
> use of fsync(). Requests to fix dpkg in the past have been rejected
> and I was told to use eatmydata. After patching debootstrap to add
> eatmydata to the required list, and activate it during the second
> stage install, the time to construct the chroot dropped from 10m to
> 2m. This should also make installing new systems MUCH faster.
The following patch introduces an --include-early option which allows
to introduce eatmydata early enough:
--- /usr/sbin/debootstrap 2013-03-22 07:04:12.000000000 +0100
+++ tmp/debootstrap/debootstrap 2013-03-31 14:17:26.425276613 +0200
@@ -40,6 +40,7 @@
EXTRACTOR_OVERRIDE=""
UNPACK_TARBALL=""
ADDITIONAL=""
+ADDITIONAL_EARLY=""
EXCLUDE=""
VERBOSE=""
CERTIFICATE=""
@@ -94,6 +95,8 @@
[ --arch=powerpc ]
--include=A,B,C adds specified names to the list of base packages
+ --include-early=A,B,C adds specified names to the list of required packages
+ (will be unpacked before chrooting in for the first time)
--exclude=A,B,C removes specified packages from the list
--components=A,B,C use packages from the listed components of the
archive
@@ -259,6 +262,18 @@
fi
ADDITIONAL="$(echo "$ADDITIONAL" | tr , " ")"
;;
+ --include-early|--include-early=?*)
+ if [ "$1" = "--include-early" -a -n "$2" ]; then
+ ADDITIONAL_EARLY="$2"
+ shift 2
+ elif [ "$1" != "${1#--include-early=}" ]; then
+ ADDITIONAL_EARLY="${1#--include-early=}"
+ shift 1
+ else
+ error 1 NEEDARG "option requires an argument %s" "$1"
+ fi
+ ADDITIONAL_EARLY="$(echo "$ADDITIONAL_EARLY" | tr , " ")"
+ ;;
--exclude|--exclude=?*)
if [ "$1" = "--exclude" -a -n "$2" ]; then
EXCLUDE="$2"
@@ -561,6 +576,7 @@
work_out_debs
base=$(without "$base $ADDITIONAL" "$EXCLUDE")
+ required="$required $ADDITIONAL_EARLY"
if [ "$RESOLVE_DEPS" = true ]; then
requiredX=$(echo $(echo $required | tr ' ' '\n' | sort | uniq))
This also does not require all distribution scripts to be adapted.
Here is the man page patch:
--- debootstrap.8.orig 2013-03-31 21:40:29.694855662 +0200
+++ debootstrap.8 2013-03-31 21:41:19.114966193 +0200
@@ -54,6 +54,12 @@
Comma separated list of packages which will be added to download and extract
lists.
.IP
+.IP "\fB\-\-include-early=alpha,beta\fP"
+Comma separated list of packages which will be added to download and extract
+lists. These packages are already unpacked during the first stage
+which allows, for example, eatmydata to be used early enough in the
+process.
+.IP
.IP "\fB\-\-exclude=alpha,beta\fP"
Comma separated list of packages which will be removed from download and
extract lists.
With this patch, one can use
eatmydata debootstrap --include-early=eatmydata ...
to get eatmydata. With strace -esync,fsync, I verified that this
actually only issues about 20 fsync calls in the final stage of
debootstrap.
On a rotating hard disk, this reduces the time needed to set up a sid
bootstrap from 10 minutes to 4 minutes. On an SSD, the time difference
is negligible though.
Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Mannheim, Germany | lose things." Winona Ryder | Fon: *49 621 31958061
Nordisch by Nature | How to make an American Quilt | Fax: *49 621 31958062
Reply to: