[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Bug#599206: dpkg-cross should leave files in converted package

On Wed, 6 Oct 2010 19:27:57 +0200
Loïc Minier <loic.minier@linaro.org> wrote:

> On Wed, Oct 06, 2010, Neil Williams wrote:
> > No. pkg-config specifies architecture-independent metadata which is
> > static within any single source release, not architecture-dependent
> > variables deduced at build time.
>  it's not architecture independent at all (which is why it's mostly in
>  /usr/lib/pkgconfig rather than /usr/share/pkgconfig)

The data in a .pc file is used to generate the data needed for the
build, it does not circumvent the generation of this data. Also,
pkg-config itself understands how to ensure that this data is available
in a way which understands cross-building.
>  The list of libraries might differ, or some random var, or simply the
>  version of the file, options or anything.

.pc files don't contain metadata like -DHAVE_STRING_H=1
-DHAVE_MEMORY_H=1 -DHAVE_STRINGS and the like. That goes in files like
config.h and is *generated* during the build from tests on the actual
system because different systems of the same architecture will change
the results of the calculations.

It is wrong to rely on that data being included in any part of the -dev
package, stuff like endianness and the size of char needs to be
generated / calculated / retrieved from a standard location. Not
included into the -dev package.

That is *system-dependent* data as well as architecture-dependent and
it has no place in a -cross -dev package. dpkg-cross has a mechanism
for retrieving only those pieces of metadata which cannot be calculated
without compiling and then executing a test program. This mechanism
itself is sub-optimal and prone to breakage - it will not bear extending
to data which *can* (and therefore should) be calculated within each
build environment or where merely compiling a test program will
suffice to retrieve the meta data (e.g. whether a particular library
function exists). Shortcuts are no good for anyone. Ubuntu has already
patched dpkg-cross to extend what shortcuts are already available into
territory which is not supportable in Debian.


Embedding that in a -dev package is just wrong. It needs to be
calculated according to the libc config in the environment. That is not
architecture-dependent, that is system dependent - e.g. uClibc does not
necessarily include all of the struct_tm data.

>  First, there is a difference between what the architecture needs, and
>  what tcl was built with. 

So the cross build should calculate what the architecture needs, not
what is retained from the build environment of the dependency. Data
from the build environment of tcl has nothing to do with the data which
needs to be calculated from the build environment of the package using

Tcl is built on machine foo - the calculations made in that build have
nothing to do with the data needed by the build on machine bar. All
pkg-config lists are the library names, not the definitions of what
size structs are and the rest.

-DTCL_CFGVAL_ENCODING=\"iso8859-1\" -DMODULE_SCOPE=extern\ __attribute__
\(\(__visibility__\(\"hidden\"\)\)\) -DTCL_SHLIB_EXT=\".so\"

That stuff needs to be tested against the build environment, not
assumed to be present.

Quite apart from the rest, relying on scripts like this prevents
bootstrapping of new architectures. The build system should calculate
and determine these pieces of data, not inherit them from some other
random system which may or may not have the same configuration.

> In some cases, it might actually be
> important that you use the same values as tcl did. 

If those values do not actually match the build or runtime environment
of the package, you're welcome to triage the resulting bugs. I won't be
a party to that particular mess.

> But I think it's
> not the case here; I rather see this as some non-autoconf approach to
> spreading the right build settings.

Cross-building has never supported build systems other than autoconf
particularly well. Nothing dpkg-cross can do about that - outside the
scope of the script. 

> > The machine preparing the -dev package does not necessarily have the
> > same configuration as the target device of the cross-build - these
> > things need to be *calculated* not retrieved from some previous
> > build somewhere else.
>  You're saying just the same thing again; it might be entirely valid
> to use the exact same values as the ones which were used for the tcl
>  build;

It usually is not and removing the ability to determine this is not
helpful. "might be" is not adequate. It is either always valid or
it needs to be calculated.

> I don't think it's the case here though.  I think tcl is
> trying to provide non-autoconf apps with a way to get the right
> config.

It is wrong to have that data in the -dev package and rely upon it in a
different build. There is no safe way of dpkg-cross supporting such
mangling - it does quite enough of that itself without adding special
cases like this.

There are other packages "trying to provide non-autoconf apps with a
way to get the config" but none has so far come up with a *standard* or
even *generalised* way of doing that.

Do you think I'm rejecting this idea just to be awkward? No, it's
because what you request is horrendously fragile, unpredictable and
more likely to break builds than to assist builds. There is no safe way
for dpkg-cross to mangle these files or to determine precisely which of
these non-standard scripts can be parsed and which cannot. Special
cases do not help - dpkg-cross already has too many of those.

I'll document the behaviour but dpkg-cross cannot withstand yet more
special cases and still be usable for the existing packages which it
has to support. Every interpreted language, every GObject marshaller
and every ORM package is going to want specialised support and there is
simply no reliable way of implementing a bespoke solution for each one.
None of the upstream teams, nor most of the Debian maintainers, care
sufficiently about cross-building to maintain such package-specific

Not every package in Debian is ever going to be compatible with
cross-building and dpkg-cross cannot support everything. (I doubt that
Multiarch will be able to support things like this either - there will
always be exceptions.) Cross building already has sufficient
special-cases and exceptions - adding more does not help anyone, least
of all those who actually have to fix either the cross builds or the
cross-building tools.

It is not safe to leave files in the package nor is it safe to move
files into arbitrary locations when there is no reliable and
standardised way to determine the usefulness of a particular kind of
file. Package specific config scripts cannot be supported and even
build-system specific ones are not necessarily going to work without a
lot of ongoing maintenance.

Patch the original package to understand cross-building and how to
calculate the necessary meta data. If there is data which cannot be
calculated without executing a foreign architecture binary, then
dpkg-cross should be able to help and I'll look at how that can be
done but the rest of the data needs to be generated from within the
cross-building environment. Anything else just isn't sufficiently
robust or supportable.

Why cross-build tcl in the first place? What functionality is meant to
be changed or removed? Cross-building packages to end up with a binary
which is almost identical to the natively built package is a waste of
time. Seeking to be able to cross-build every package in Debian is a
pointless exercise and far beyond the scope of what dpkg-cross can


Neil Williams

Attachment: pgpVHXaCMPp2A.pgp
Description: PGP signature

Reply to: