Re: Shorten build process with live-build
On 12/08/2016 02:14 AM, Machu Chukov wrote:
By the way: We tested using libeatmydata to make the build process
faster, but it didn't help at all. Any ideas?
Using eatmydata actually helps, but unfortunately several number of
steps are needed for it to be effective. I'll try to summarize them:
1. Eatmydata is not used at all during debootstrap step (see #700633).
As a workaround I copy libeatmydata so file *before* debootstrap is ran.
Here is the snippet I use:
## install eatmydata early
if [ ! -f chroot/usr/lib/i386-linux-gnu/libeatmydata.so ]; then
apt-get download libeatmydata1
sudo dpkg-deb --extract libeatmydata1*deb chroot
fi
I run this in auto/build, *before* calling "lb build". The reason I
don't use chroot hooks is that they are ran very late in the build
process and we need the so file very early.
2. Then I install eatmydata package just to be sure using
--debootstrap-options "--include=eatmydata --variant=minbase"
option for "lb config" in auto/config.
3. I call "lb build" with eatmydata wrapper. So change "lb build ..."
line in auto/build with "eatmydata lb build ..."
Our question is, if there is a way to shorten the build process.
Here are few other tricks I use. Note that they'll affect the resulting
image so you may not want to use all of them:
1. Use local mirrors or a caching proxy like apt-cacher-ng for *all*
repositories, because live-build *loves* to run "apt-get update" dozens
of times even if you only have few basic repositories.
2. Make sure the proxy is used throughout the build process. There are
several stages of lb build and unfortunately every single one of them
use different configurations for their apt-get steps. Unfortunately I
don't have a good answer to how to configure this for all stages
correctly. I currently set "--mirror-bootstrap",
"--mirror-chroot-security", "--mirror-binary" and
"--mirror-binary-security" options to "${PROXY}/${REAL_REPO_URL}" and
don't mess with config/apt.conf or config/includes.chroot/et/apt... but
this way proxified repo address leaks into resulting image's
/etc/apt/sources.list.
3. Start from "minbase" debootstrap variant and only add what's needed.
This reduces the # of packages that are installed, thus the image size &
build time.
4. Exclude unnecessary files using dpkg.cfg.d. For example I exclude doc
files and man pages from the resulting image, and I don't simply delete
them using chroot hooks, but I exclude them using /etc/dpkg/dpkg.cfg.d/
so they don't even get installed in the first place (thus less I/O).
Note that even if you put the relevant configuration files before
debootstrap is run, they'll not be effective until the second stage of
debootstrap (since dpkg is not used in the first stage).
Hope this is useful!
Reply to: