[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Shorten build process with live-build



Hello,

I have to come back to that issue my friend opened a while ago. I finally found out why using libeatmydata does not seem to help (much) - the script functions/chroot.sh in live-build cleans the environment before entering the chroot and so also removes LD_PRELOAD from the environment.

There are two solutions to this:

1.) include something like 'LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libeatmydata.so' in config/environment.chroot in the config tree. That's what I did right now in order to not modify live-build. It works and reduces build time by about 35 minutes.

2.) modify chroot.sh to pass LD_PRELOAD along. That is probably the right thing to do, so I will open a bug report against live-build and suggest this will be changed.

Thank you again for your effort to maintain live-build.

Andreas

Am 08.12.2016 um 18:41 schrieb Mark Zorko:

On 12/08/2016 02:14 AM, Machu Chukov wrote:

> By the way: We tested using libeatmydata to make the build process
> faster, but it didn't help at all. Any ideas?
>

Using eatmydata actually helps, but unfortunately several number of
steps are needed for it to be effective. I'll try to summarize them:

1. Eatmydata is not used at all during debootstrap step (see #700633).
As a workaround I copy libeatmydata so file *before* debootstrap is ran.
Here is the snippet I use:

> ## install eatmydata early
> if [ ! -f chroot/usr/lib/i386-linux-gnu/libeatmydata.so ]; then
> 	apt-get download libeatmydata1
> 	sudo dpkg-deb --extract libeatmydata1*deb chroot
> fi

I run this in auto/build, *before* calling "lb build". The reason I
don't use chroot hooks is that they are ran very late in the build
process and we need the so file very early.


2. Then I install eatmydata package just to be sure using

> --debootstrap-options "--include=eatmydata --variant=minbase"

option for "lb config" in auto/config.

3. I call "lb build" with eatmydata wrapper. So change "lb build ..."
line in auto/build with "eatmydata lb build ..."



> Our question is, if there is a way to shorten the build process.

Here are few other tricks I use. Note that they'll affect the resulting
image so you may not want to use all of them:

1. Use local mirrors or a caching proxy like apt-cacher-ng for *all*
repositories, because live-build *loves* to run "apt-get update" dozens
of times even if you only have few basic repositories.

2. Make sure the proxy is used throughout the build process. There are
several stages of lb build and unfortunately every single one of them
use different configurations for their apt-get steps. Unfortunately I
don't have a good answer to how to configure this for all stages
correctly. I currently set "--mirror-bootstrap",
"--mirror-chroot-security", "--mirror-binary" and
"--mirror-binary-security" options to "${PROXY}/${REAL_REPO_URL}" and
don't mess with config/apt.conf or config/includes.chroot/et/apt... but
this way proxified repo address leaks into resulting image's
/etc/apt/sources.list.

3. Start from "minbase" debootstrap variant and only add what's needed.
This reduces the # of packages that are installed, thus the image size &
build time.

4. Exclude unnecessary files using dpkg.cfg.d. For example I exclude doc
files and man pages from the resulting image, and I don't simply delete
them using chroot hooks, but I exclude them using /etc/dpkg/dpkg.cfg.d/
so they don't even



Reply to: