[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#1063337: navit FTCBFS: uses CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt

Source: navit
Version: 0.5.6+dfsg.1-2
Severity: normal
Tags: patch
User: debian-cross@lists.debian.org
Usertags: ftcbfs
X-Debbugs-Cc: debian-cross@lists.debian.org

Hi,

TLDR; navit fails to cross build from source because it sets
CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt in
debian/rules. Navit cross builds just fine after deleting it. Trivial
patch:

--- a/debian/rules
+++ b/debian/rules
@@ -14,7 +14,6 @@ CMAKEFLAGS = -DCMAKE_BUILD_TYPE=debian
 ifneq ($(DEB_HOST_MULTIARCH),$(DEB_BUILD_MULTIARCH))
 export CC=$(DEB_HOST_MULTIARCH)-gcc
 export CXX=$(DEB_HOST_MULTIARCH)-g++
-export CMAKE_TOOLCHAIN_FILE=/etc/dpkg-cross/cmake/CMakeCross.txt
 CMAKEFLAGS += -DCMAKE_LIBRARY_PATH="/usr/$(DEB_HOST_MULTIARCH)/lib" -DCMAKE_REQUIRED_INCLUDES="/usr/$(DEB_HOST_MULTIARCH)/include;/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)" \
        -DCMAKE_C_FLAGS_DEBIAN="$(shell dpkg-buildflags --get CFLAGS) $(shell dpkg-buildflags --get CPPFLAGS) -I/usr/$(DEB_HOST_MULTIARCH)/include -I/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)" \
        -DCMAKE_CXX_FLAGS_DEBIAN="$(shell dpkg-buildflags --get CXXFLAGS) $(shell dpkg-buildflags --get CPPFLAGS) -I/usr/$(DEB_HOST_MULTIARCH)/include -I/usr/$(DEB_HOST_MULTIARCH)/include/$(DEB_HOST_MULTIARCH)"


Long story:

Helmut Grohne discovered that navit FTCBFS:

http://crossqa.subdivi.de/build/navit_0.5.5%2Bdfsg.1-2_i386_20211022094850.log
http://crossqa.subdivi.de/build/navit_0.5.6%2Bdfsg.1-2_arm64_20231230114317.log

Symptoms:

-- Could NOT find ZLIB (missing: ZLIB_LIBRARY ZLIB_INCLUDE_DIR) 
-- Could NOT find Freetype (missing: FREETYPE_LIBRARY FREETYPE_INCLUDE_DIRS) 

But the respective packages are installed, so CMake should be able to find it.
Why does it not?

Here is a minimal reproducer of the problem. I'm on an arm64 machine (build
architecture) and am attempting to cross compile for amd64 (host architecture):

mkdir -p src/build
cat << END > "src/CMakeLists.txt"
cmake_minimum_required(VERSION 3.2)
find_package(ZLIB REQUIRED)
END
env --chdir=src/build \
	CMAKE_TOOLCHAIN_FILE='/etc/dpkg-cross/cmake/CMakeCross.txt' \
	CONFIG_SITE='/etc/dpkg-cross/cross-config.amd64' \
	CXX='x86_64-linux-gnu-g++' \
	DEB_HOST_GNU_TYPE='x86_64-linux-gnu' \
	CC='x86_64-linux-gnu-gcc' \
	cmake "-GUnix Makefiles" \
	-DCMAKE_C_COMPILER=x86_64-linux-gnu-gcc \
	-DCMAKE_CXX_COMPILER=x86_64-linux-gnu-g\+\+ \
	-DCMAKE_SYSTEM_PROCESSOR=x86_64 \
	-DPKG_CONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config \
	-DPKGCONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config \
	-DCMAKE_LIBRARY_PATH=/usr/x86_64-linux-gnu/lib \
	-DCMAKE_REQUIRED_INCLUDES=/usr/x86_64-linux-gnu/include\;/usr/x86_64-linux-gnu/include/x86_64-linux-gnu \
	..

I put this into a script and threw it at debbisect:

debbisect --arch=arm64,amd64 --depends=cmake,build-essential,crossbuild-essential-amd64,zlib1g-dev:amd64 --cache="$(pwd)/cache" 2020-01-01 2024-01-01 ./test.sh
[...]
bisection finished successfully
  last good timestamp: 20210915T092301Z
  first bad timestamp: 20210915T150047Z
the following packages differ between the last good and first bad timestamp:
  cmake 3.18.4-2 -> 3.21.2-1
  cmake-data 3.18.4-2 -> 3.21.2-1
  libjs-underscore 1.9.1~dfsg-3 -> 1.9.1~dfsg-4
test upgrading cmake 3.18.4-2 -> 3.21.2-1...
using cached result (was bad) from debbisect.20210915T092301Z.cmake.log.bad
  upgrading cmake triggered the problem
  additional packages that got upgraded/installed at the same time:
    cmake-data 3.18.4-2 -> 3.21.2-1
test upgrading cmake-data 3.18.4-2 -> 3.21.2-1...
using cached result (was bad) from debbisect.20210915T092301Z.cmake-data.log.bad
  upgrading cmake-data triggered the problem
  additional packages that got upgraded/installed at the same time:
    cmake 3.18.4-2 -> 3.21.2-1
test upgrading libjs-underscore 1.9.1~dfsg-3 -> 1.9.1~dfsg-4...
  upgrading libjs-underscore does not cause the problem

The upstream git diff between v3.18.4 and v3.21.2 is 364800 lines long, so lets
bisect cmake git. Luckily, the Debian packaging of these two upstream versions
is near identical, so we can just bisect upstream git, create a new source
tarball for every commit we test, build a Debian package from that and then
test that package inside the test environment from above. Both building the
package as well as running the test is done inside a Debian chroot from
snapshot timestamp 20210915T092301Z:

mmdebstrap --variant=buildd --aptopt='Acquire::Check-Valid-Until "false"' unstable \
	~/.cache/sbuild/unstable-bisect-arm64.tar "http://127.0.0.1:8000/archive/debian/20210915T092301Z";

The following script can be passed to "git bisect run" to find the offending
version:

git archive --format=tar --prefix=cmake/ HEAD | gzip > ../cmake_3.21.2.orig.tar.gz
cp -a /tmp/cmake_3.21.2-1+bisect.debian debian
DEB_BUILD_OPTIONS=nocheck sbuild --verbose -d unstable-bisect --no-run-lintian
mmdebstrap --variant=apt --arch=arm64,amd64 --aptopt='Acquire::Check-Valid-Until "false"' \
	--include=build-essential,crossbuild-essential-amd64,zlib1g-dev:amd64 \
	--customize-hook='copy-in /tmp/cmake_3.21.2-1+bisect_arm64.deb .' \
	--customize-hook='copy-in /tmp/cmake-data_3.21.2-1+bisect_all.deb /' \
	--chrooted-customize-hook="apt install --yes ./cmake_3.21.2-1+bisect_arm64.deb .//cmake-data_3.21.2-1+bisect_all.deb
		&& mkdir -p /src/build && printf 'cmake_minimum_required(VERSION 3.2)\nfind_package(ZLIB REQUIRED)\n'>/src/CMakeLists.txt
		&& env --chdir=/src/build CMAKE_TOOLCHAIN_FILE='/etc/dpkg-cross/cmake/CMakeCross.txt' CONFIG_SITE='/etc/dpkg-cross/cross-config.amd64' CXX='x86_64-linux-gnu-g++' DEB_HOST_GNU_TYPE='x86_64-linux-gnu' CC='x86_64-linux-gnu-gcc' cmake '-GUnix Makefiles' -DCMAKE_C_COMPILER=x86_64-linux-gnu-gcc -DCMAKE_CXX_COMPILER=x86_64-linux-gnu-g\+\+ -DCMAKE_SYSTEM_PROCESSOR=x86_64 -DPKG_CONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config -DPKGCONFIG_EXECUTABLE=/usr/bin/x86_64-linux-gnu-pkg-config -DCMAKE_LIBRARY_PATH=/usr/x86_64-linux-gnu/lib -DCMAKE_REQUIRED_INCLUDES=/usr/x86_64-linux-gnu/include\;/usr/x86_64-linux-gnu/include/x86_64-linux-gnu .." \
	unstable /dev/null "http://127.0.0.1:8000/archive/debian/20210915T092301Z";

The above depends on having a http server (I use python3 http.server) serving
packages from snapshot timestamp 20210915T092301Z. Using snapshot.d.o directly
makes all of this near impossible due to throttling and connection cuts.

We arrive at this upstream commit that breaks cross-building navit:

https://gitlab.kitware.com/cmake/cmake/-/commit/6c34ed9b879906d1eaadad80f37f518829017789

This commit introduces respecting the CMAKE_TOOLCHAIN_FILE environment variable
which navit sets in debian/rules. The offending commit from the navit packaging
git is this one:

commit 37f6cc73730b6271e4abb3878a683032a96dbdea (tag: debian/0.5.0_svn4776+dfsg.1-2)
Author: Gilles Filippini <pini@debian.org>
Date:   Sun Oct 23 18:58:30 2011 +0200

    Version 0.5.0~svn4776+dfsg.1-2
    
    Cross compilation support.

So essentially this boils down to the cross-building mechanisms from more than
a decade ago not working anymore today. Luckily, according to codesearch.d.n,
navit seems to be the only package that uses this toolchain file via
CMAKE_TOOLCHAIN_FILE.

This mail mostly serves as a cheat-sheet for me in case I want to do something
similar in the future. Don't ask me why I found this problem funny. :)

Thanks!

cheers, josch
Reply to: