On Mon, 2008-05-12 at 13:47 +0200, Raphael Hertzog wrote:
> On Mon, 12 May 2008, Neil Williams wrote:
> >
> > OK, I'm using quiet now so there should be no more CC'd reports to
> > -devel with the next batch of reports. Sorry for the noise.
>
> Why are you filing those against "general" and not the respective
> packages?
I hope I covered that earlier:
http://lists.debian.org/debian-devel/2008/05/msg00176.html
1. When a package, foo, has been crossbuilt successfully but cannot be
uploaded because a dependency, bar, is failing to cross build, file a
bug against buildd.emdebian.org showing foo as Blocked by #number in
bar. The bug report against bar would be a normal BTS report against
that package, tagged "crossbuilt". (I need easily available data on
which packages are causing the most trouble.)
e.g.:
#465248 bar: mass bug filing for cross building support.
Tags: crossbuilt.
...
# 465mumble : buildd.emdebian.org [foo] {arm} : new version available.
Blocked by #465248
Emdebian has custom scripts to detect these situations (using
edos-debcheck) because the normal method of relying on a pbuilder /
chroot build cannot pick up these cross-building dependency issues. This
bug type would not normally occur in Debian - the package would simply
FTBFS due to a missing dependency but still be installable.
2. When a package has had a crossbuilding fix applied but still needs
nodocs support or nochecks support implemented via debhelper etc., file
a bug against the pseudo-package, again, blocked by the debhelper or
cdbs bug. If the package installs docs with direct calls to 'install' or
'cp', file a bug against the Debian package instead. It's not the fault
of the foo maintainer using dh_installman that debhelper doesn't use the
DEB_BUILD_OPTIONS="nodocs" variable but Emdebian still needs to track
which packages are waiting for which bugs.
3. When a package has been crossbuilt and uploaded but the installed
package does not behave as the Debian package would behave - i.e. where
crossbuilding might have introduced a new bug by changing dependencies
or removing a file that should be included. Emdebian has changed the
build environment created by the Debian maintainer to meet our own
requirements (which may well conflict with Debian requirements), it's
not the fault of the maintainer that the change causes a breakage, nor
is it really up to that maintainer to fix the breakage.
4. When a package has unwanted dependencies in Emdebian - typically
because the Emdebian package needs to be built with --disable-foo
instead of --enable-foo. (Note that {4} may cause {3} - in which case a
functional package split or a new generated package may be needed to
have both build options available as separate packages).
5. One pseudo-package for all Emdebian architectures, just use the
[package]-{$ARCH} prefix in bug titles - including
[package]-{i386-linux-uclibc} where appropriate. I'll be starting on
prebuilt packages for armel and i386 soon and experimental uClibc
support is already available in emdebian-tools (>= 1.0.0).
6. The majority of these bug reports against buildd.emdebian.org would
be automatically generated. The existing embug script in emdebian-tools
already uses SOAP to get information on existing bugs, it can migrate to
storing bug information in the BTS under this pseudo-package instead of
only being able to create a local log file.
7. Where a crossbuilt package now fails the lintian test for cross-built
packages (lintian -ioC em) e.g. when the lintian check script provided
by emdebian-tools is updated. (Packages are not uploaded if the build
fails the cross-building lintian test.)
I have been tracking these with just local files but that isn't any use
for others who want to use Emdebian and need to find out why a package
isn't up to date.
The basic premise of using buildd.emdebian.org as a usertag and
eventually pseudo-package is that the problem does not necessarily lie
within the Debian package. The bug report is filed to indicate the
reason why a package cannot be uploaded - i.e. buildd.emdebian.org is
insta-buggy as soon as one of the ~220 source packages in the Emdebian
target repository are updated in Debian Sid. Once a cross-building
buildd is available, buildd.emdebian.org can try to update the package
itself before the bug is filed - in effect, becoming like an unofficial
Debian port for emdebian-arm, emdebian-armel and emdebian-i386 (which
isn't actually cross built, just uses Emdebian patches to reduce package
size and dependencies). Not all bugs against buildd.emdebian.org are
related to crossbuilding, some are down to differences between Debian
Policy and Emdebian Policy. Due to the difficulties of debugging
cross-built packages, Emdebian requires that every package complies with
Emdebian Policy before upload, so lintian errors from the Emdebian
checks are fatal build errors - a big difference to normal Debian
methods.
http://wiki.debian.org/EmdebianPolicy
Where Debian Policy conflicts with Emdebian Policy (e.g. manpages and
changelogs), I file bugs against the relevant build tools (like
debhelper #448615 or CDBS #450483) in Debian so that DEB_BUILD_OPTIONS
can be used to allow one package to build for both policies, tracking
the packages waiting for the bug fix via buildd.emdebian.org bugs.
> As a maintainer, I would have no problem receiving such reports
> if they are filed with a severity minor. Then you use some usertags to get
> a global view of all the open issues distribution-wise.
I do that too - although I generally wait until I have at least some
idea of how to fix the bug and then attach a patch to a wishlist bug.
Cross-building can still be awkward in Debian and I find that it is hard
enough getting cross-building patches applied without expecting
maintainers to set up an Emdebian toolchain, install the tools, obtain
the patches and use customised build scripts to prepare a test build -
all of which are unfamiliar to most maintainers. I'm getting closer to a
situation where this would not be so much work but right now, I'm not
sure that many maintainers would have the motivation to fix and test the
cross-building bugs themselves. The fixes in dpkg have been a
significant benefit and some packages do crossbuild without any changes
but more work is needed to get the rest to work in the same manner.
Other bugs, like support for DEB_BUILD_OPTIONS="nodocs" (and "nocheck"),
I will file against the relevant Debian packages because those fixes are
generally obvious to the maintainer and easy to test with standard
Debian build tools. I'll probably use a usertag of "emdebian-policy" for
such reports.
I use the "crossbuilt" usertag (which I have requested to be converted
into the "crossbuilding" BTS tag in #480511) for failures in the
crossbuild itself.
There are also other issues like the whole question of cache files that
need to be thought through before I can expect packages to cross-build
without any changes whatsoever. (cache files have a high chance of going
stale without breaking the build but changing all those m4 macros to not
fail if cross-compiling is going to take time.)
Current packages needing cache files:
apt, avahi, coreutils, dbus-glib, dbus, devmapper, dpkg, glib2.0, gnupg,
krb5, libidl, libopenobex, mktemp, ntp, orbit2, psmisc, shadow, sqlite,
startup-notification, tar, tslib, usbutils, util-linux, xorg-server,
xserver-xorg-input-mouse
(there are 26 in total - you can find the cache file values at
http://buildd.emdebian.org/svn/browser/current/target/trunk/ under
initial letter/, source package
name/trunk/emdebian-arm-linux-gnu.cache.patch, e.g.
http://buildd.emdebian.org/svn/browser/current/target/trunk/d/dpkg/trunk/emdebian-arm-linux-gnu.cache.patch?format=txt
(Yes, those need to be more accessible too.)
(dpkg cache file is just dpkg_cv_va_copy=yes - see around line 11728
of ./configure in the dpkg source.)
One way around such small cache files would be for the ./configure
script to support a --enable-foo or --with-foo option that can be
enabled with another DEB_BUILD_OPTION - fine for variables that are
actually just GNU|non-GNU tests but other variables are likely to be
change between architectures so that could get messy. Adding the cache
values to /etc/dpkg-cross/cross-config.arm is just as likely to result
in stale values as keeping the value in an Emdebian patch. The real fix
is for the relevant ./configure snippet to not fail if
cross_compiling=yes and come up with another way of determining the
value without trying to run a compiled executable during ./configure.
Most current cache files are less than 6 lines, most values are also
unique to each package, AFAICT none are common to more than two or three
different packages.
> In fact, I think that it makes more sense than using a pseudo-package.
Maybe what I need is a list of maintainers who would be willing to spend
the time preparing a cross building environment (emdebian-tools supports
pbuilder-type builds) and who are willing to have bug reports without
patches. Even with those, I would still need to track updated versions
and Emdebian Policy issues in a buildd.emdebian.org pseudo-package.
I still have a very small number of packages, overall, but it is more
than enough work for one person (what with continuing the development of
the tools at the same time). I'm hoping to use buildd.emdebian.org as a
tracker for Emdebian - probably with quite a high turnover of bugs.
I am also hoping that the activity in the BTS will help others start on
Emdebian development by clearly documenting what is working, where the
problems lie and what still needs to be done. Currently, most of the
knowledge on how to fix certain cross-building issues is buried in the
Emdebian patches - hopefully by putting more detail into the BTS, I can
assist others who want to learn how to cross-build their own packages -
especially with regard to Maemo, Qt and other environments that I simply
won't have time to build or debug.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/
Attachment:
signature.asc
Description: This is a digitally signed message part