So bazel has a fancy caching and dependency mechanism so that rebuilds
are quick. Very nice for developers, but less handy for debian
packaging where we want a full, clean, determinstic, reproducible,
build every time.
The existing (tensorflow) packaging pattern, (which I am just
re-using, assuming that it is a sensible plan, rather than fully
understanding and designing from scratch) works like this:
export HOME=/tmp
(with the comment # not $(CURDIR), that leads to recursive symlinks)
bazel build \
-k $(BAZELOPTS)\
--verbose_failures \
--repository_cache= \
--config=opt \
--distdir=./debian/dist \
--repo_env=TF_SYSTEM_LIBS=nsync,curl,double_conversion,snappy,gif,zlib,com_google_protobuf,com_github_grpc_grpc,jsoncpp_git,libjpeg_turbo,nasm \
--override_repository=bazel_skylib=$(CURDIR)/debian/mock_repos/bazel_skylib \
--override_repository=rules_cc=$(CURDIR)/debian/mock_repos/rules_cc \
--override_repository=rules_java=$(CURDIR)/debian/mock_repos/rules_java \
//tensorflow:tensorflow_cc
$(BASELOPTS) is there to enabled or disable -s for verbose logs,
according to DEB_BUILD_OPTIONS=terse or not, for example.
So the general concept is to use /tmp as the home dir, disable the
repository cache, set up a local set of repositories for basic cc and
java and skylib (what is that?) rules and a list of libraries to come
from system libs.
Perhaps we should have a --nohome_rc in there to stop local config
changing the build? But maybe that's convenient for maintainers, and
we can reply on it being empty in a clean sbuild, so no need for this.
This is all fine and works. (I've just realised that this may well be
why the cc stuff is building but not the c stuff - no set of base
rules)
However whilst this all works once, it does not work a second
time. Subsequent builds just find everything is in the cache and build
nothing. So we need the 'clean' rule to actually clean things up for a
clean, determinstic, build.
bazel clean
doesn't seem to do much, and is intended to leave the cache intact.
the docs say that
bazel clean --expunge
should remove the cache
However putting that in the clean rule:
override_dh_clean:
bazel clean --expunge
rm -rf /tmp/
Does not result in the cache actually being removed.
I have to do
rm -rf /tmp/.cache/bazel
so _really_ clean.
So what am I doing wrong? Am I misunderstanding or is there a mistake
preventing bazel clean --expunge from working?
And are we happy with this overall approach to doing bazel builds? Is
every package going to need a set of boilerplate language rules in a
local (CURDIR)/debian/mock_repos/ tree with corresponding
--override_repository config?
Perhaps that is specific to tensorflow? I am just using it as an
example here because I don't think we have any others yet (do we? I
should probably be looking at them if we do).
Also, at some point we are going to have to teach debhelper about a
--bazel mode, which may obviate the need for some of this. Perhaps
some of you have thought about this already?
Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/
Attachment:
signature.asc
Description: PGP signature