Re: RFC: Rules for distro-friendly packages
* Russ Allbery <rra@debian.org> schrieb:
Hi,
> > With that I mean that you don't need to unpack multiple tarballs to
> > get a working tree (Xorg/Xf86 was such a case, several years ago).
>
> Right. And I'm saying now that our packaging tools can handle that case,
> it's not clear to me why Debian should be telling upstreams they must not
> do that.
That's probably fine for Debian, but my rules are meant to be generic,
not distro specific. And many other packaging systems do not support it.
Note that such cases add a lot of complexity to the whole QM and packaging
workflow, which should be avoided (that's one of the major goals of this
document: make the workflow as simple and defined as possible).
> >> You're basically saying that people aren't allowed to use the typical
> >> Autoconf semantics of honoring --with and --without flags and
> >> autodetecting an optional dependency if neither is given, but that's
> >> best practice.
>
> > They should use --enable-*/--disable-* flags for switching features.
>
> --enable/--disable is for internal features. --with/--without is for
> features that depend on external packages. See the Autoconf
> documentation.
This distinction is IMHO a wrong concept, that maybe gets more obvious
when internal features start to introduce dependencies.
And it's also the wrong methodology - it turns things upside down: from
a QM perspective you're interested in the function requirements or
features, and dependencies might be triggered by that (or might not to).
For example, an "--with-zlib" doesn't tell anything about what is really
done with zlib, what features are provided by that. It just tells that
zlib is *somehow* used. The right way would be some "--enable-compression"
which tells that our package can handle compression - the dependency to
zlib now is just an implication on that. (depencencies are implications,
not bijective relations).
> > Switching dependencies which silently enables/disables features is a
> > generally bad approach. Instead have feature switches and maybe tell
> > what these features depend on.
>
> Best practices for use of Autoconf is to force building the feature with
> --with, force omitting the feature with --without, and to auto-detect
> based on the presence of the external library if neither is given. This
> is what we should be recommending to upstream to use, IMO.
Yes, but this approach is dangerous. It relies on assumptions about the
target system, taken from the running/building system. I already had
cases where that caused a hell of trouble.
The purpose of my rules is to get rid of unexpected behaviour as much
as possible - this is one of the key points.
> > Building static libraries (additional to shared/dynamic ones) usally
> > is not a big task.
>
> Except when it is, such as whenever you want to use some sort of dynamic
> plugin architecture, which is increasingly common. See also PAM modules,
> NSS modules, etc.
PAM and NSS are inherently instable by design, just by the fact that code
it loaded at runtime which is not known at compile time (the toolchain can
serve no proper checks here, since the necessary information is simply not
available at this time) - very bad from a QM perspective.
And of course, that all cannot be used on systems where dynamic library
loading is not available at all.
> > And still many people need them.
>
> I'm dubious of this.
It's required by much of my daily work.
> >> one can not yet commit to a stable ABI, and there are
>
> > Why that exactly ?
>
> Why do upstreams not commit to a stable ABI? Usually because the software
> is still under rapid development and they're not keeping track of which
> changes change the public ABI. It's not uncommon for libraries in their
> very early stages of growth.
Okay, in that case they simply have to do proper library versioning
(and if API also changes incompatibly, version the includes).
If upstream can't or doesn't want to do that, a separate layer in
the middle (eg. OSS-QM) has to step in, essentially forming an kind
of overlay or filter.
> >> various situations (such as support for dynamic plugins) where
> >> supporting them is a huge pain.
>
> > In very most cases, that's bad design.
>
> Um. I'm not sure there's anything I can say to that other than "no, it's
> not."
We could play a little game: you tell me some cases where you think
that dynamic library loading is necessary, and I'll give you better
solutions ;-p
> >> I strongly disagree with requiring pkg-config.
>
> > Well, actually, I need it, eg. for clean sysroot'ed crosscompiling.
>
> I doubt this a lot. Details?
Well, the major problem is that in many cases - especially when
crosscompiling - you cannot assume all libraries in standard
locations (those the upstream expects), and you also need to know
the library's own dependencies (not all targets/binfmt's have proper
dependency information, especially not on static libraries).
So you'll need some lookup mechanism (unless you want to tweak each
single importing package). That's where pkg-config comes into play:
it provides a generic lookup mechanism, which can be easily intercepted.
For example, when pkg-config did not yet support sysroot, I've injected
a small wrapper, which calls pkg-config with proper search pathes
(so it operates from the sysroot exclusively) and rewrites it's output,
so all pathes are all properly prepended with the sysroot prefix.
(meanwhile it's not necessary anymore, as you simply have to pass it
the sysroot prefix and it does that all on its own).
Using such a generic lookup also allows other things, eg. telling the
packaging systems that certain things are missing, in an automatized,
machine readable way. That can help a great deal in QM and package
maintenance.
> >> and I don't see any point in using pkg-config if the way that one uses
> >> the shared library is to just add -l<library-name>.
>
> > Because that doesn't always suffice.
>
> Then use pkg-config or some other tool in the rare cases where it doesn't.
These cases aren't that rare as you might think. I've got a lot of
libraries on my systems which are not located directly in /lib or /usr/lib.
Of course, you might say that everything should be in strictly defined
locations. But then you'll need a way for do the necessary adaptions
on filesystem level (eg. per-process unions and binds). That would be
the Plan9 way, but only few platforms really support that.
> > It requires that the library is in the toolchain's standard search
> > path.
>
> That's what user-settable LDFLAGS in Autoconf is for, and that handles the
> search paths for all libraries, whether they're using pkg-config or not.
Fine. So you expect the user or the calling packaging-/buildsystems to
know about all the probably imported libraries and their locations ?
Well, this essentially means moving out responsibility for intra-package
issues to the outside, breaking boxing/modularization.
> > And what about cflags ? What about dependencies ?
>
> Most libraries do not need either of those things on any modern platform.
You shouldn't reduce your view on ELF shared objects. There're many
platforms out there which dont have the necessary dependency information
within the shared object. And: this information only gives runtime
linking information, not compile-time stuff.
> Are you really writing your guidelines to cater to HP-UX and IRIX? I
> agree that this sort of thing is useful there, but there are numerous
> other portability issues to platforms that can't handle shared library
> dependencies. Most software is not going to build on those platforms
> anyway, since doing so requires access to those platforms to do reasonable
> testing and realize what changes have to be made.
My rules are meant to be generic. Of course they can't handle everything
that might be needed, but at least it's a good start.
> But one of the things that makes software packages unnecessarily
> complex in my opinion is cargo-cult portability.
True. The better approach is to entirely set on standardized interfaces
(eg. expect an sane libc, etc). pkg-config is one brick for this:
it sets an generic interface for quering on what to do in order to
get some library linked in. I'm doing some research for other areas
of portable software engineering, but that will be scope of future papers.
> For example, it's always slightly amusing to look through a large
> collection of Autoconf-using open source software and see how many of
> those packages have calls to AC_FUNC_SETVBUF_REVERSED in their
> configure.ac files. That check is a probe for a problem that hasn't
> existed since 1987, the software packages almost never correctly handle
> the case where that macro returns the unexpected result, and in modern
> Autoconf (in recognition of that) it now just silently does nothing. Yet
> it sticks around in many software packages as cargo-cult portability,
> sometimes with some preprocessor goo that purports to handle the results
> (but usually doesn't).
That's why autoconf'ed packages should be built w/ recent autoconf
versions and obsolete stuff kicked out. (autoupdate should tell
you such things).
> > It could have been named "wurstbrot" or whatever, but fact is, most
> > packages name it that way and so it's an de-facto standard. And
> > standardization is one of the major aspects of these rules.
>
> The standardization of this particular interface (which is of only slight
> utility; people use all sorts of other conventions already) is not
> important enough to warrant uglified names, IMO.
Not important ? Depends on the perspective. A vast majority of the
packages I'm daily coping with are autoconf-based. Most of them
(only few exceptions) have it named autogen.sh. I could also assign
a completely new name, but that would require virtually any package
to be touched.
At this point it'd be probably better to add some kind of generic spec
file which tells much more information about the package (some kind
of .lsm record with additional machine readable information about
build procedures, dependencies, etc). If you like that idea, then
lets talk about this :)
cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/
phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------
Reply to: