[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

Re: Good communication with upstream is good idea

On Sun, 2008-07-20 at 13:43 -0700, Steve Langasek wrote:
> Hi Neil,
> On Sun, Jul 20, 2008 at 05:32:31PM +0100, Neil Williams wrote:
> > I ask because emdebian-tools isn't intended for Ubuntu either. See [0] -
> > emdebian-tools also depends on server resources provided only by Debian
> > (in this case, the package repositories containing compatible packages
> > which I can use to generate cross-dependencies).
> That doesn't seem particularly Debian-specific, though?  It's not out of the
> question that Ubuntu could have an armel port later, and that's the only
> thing I can think of that /should/ cause emdebian-tools to be incompatible
> with Ubuntu.

emdebian-tools supports all the architectures currently supported by
Debian and a few that are pending (like uclibc ones - once the
well-known problems in uClibc are sorted out and uclibc returns to
Debian after Lenny). To support emdebian-tools properly, Ubuntu would
have to support all the Debian architectures with appropriate buildds
and repositories. This is more than a little pointless.

The other problem is that emdebian-tools is working towards getting
cross-building support into Debian packages but currently needs patches
to cross-build all current packages (even those that have closed the
relevant cross-building support bugs) due to issues in CDBS and
debhelper etc. Those patches are constantly updated against the versions
of the packages in Debian Sid - e.g. I've just updated the patches for
pam, pcre3 and a few others that allow me to upload a usable cross-built
package in advance of a solution compatible with the Debian package
itself. With the newly created cross-building autobuilder for Emdebian,
[0] I will now have more time to file such bugs to get the next round of
cross-building support into packages. I was concentrating on getting the
basic support into at least some packages for Lenny and that is now
done. All the time, the tools themselves are developing and becoming
more powerful. As support improves, patches change. It's a lot of work
and I am not about to make those patches compatible with the versions in
Ubuntu (or any other derivative) - the only solution for non-Debian
usage is to wait until the changes are made in the appropriate Debian
packages that remove the need for the patches. That is a slow process
and in the meantime, Emdebian needs packages to test and those need the

Even when the patches are folded into Debian, the lack of suitable
architecture repositories will prevent emdebian-tools being useful on
Ubuntu, especially when compared with running the tools inside a Debian
chroot on Ubuntu.

> > "emdebian-tools is not intended for Ubuntu but I don't have a way of
> > encoding that in the package. emdebian-tools is tightly integrated into
> > Debian (and Debian unstable in particular) and is, naturally, a Debian
> > native package (it was written to support Embedded Debian after all, not
> > UbuntuMobile). It isn't intended to work on Ubuntu because Ubuntu does
> > not provide the foreign packages needed for linking when cross building,
> > those come exclusively from Debian.
> So if an armel port of Ubuntu becomes available, is there anything else that
> stops emdebian-tools from working with it?

mips, mipsel, ARM, uclibc-arm, uclibc-mips . . . .

Currently, emdebian-tools only has prebuilt binary packages for ARM (not
armel) but adding more is supported (although not particularly trivial
at this time).

For Ubuntu to support emdebian-tools, Ubuntu would have to become Debian
which would be pointless (and futile).

Those who want to do things with Emdebian on Ubuntu are simply advised
to create a Debian chroot - Lenny or better - and run things from there.

> > Same with apt-cross, it is exclusively designed for Debian, Debian mirrors
> > and Debian buildd configurations.
> How does apt-cross have anything to do with the Debian buildds, at all?
> Surely you're not using this as a build-dependency to force Debian
> cross-builds on the Debian buildds, are you?

The Emdebian autobuilder uses apt-cross, yes. There is no other way of
downloading ARM packages on amd64 and converting them to -arm-cross
packages for use in /usr/arm-linux-gnu/lib/ etc and reconciling all the
dependencies to be compatible with dpkg. emdebian-tools uses a
debootstrap wrapper to create a disposable chroot cross-building
environment with emdebian-tools installed and configured inside
(including a cross-building toolchain for the desired arch). Yes, this
is a larger .tgz than a standard pbuilder but it works. i.e. for
Emdebian build-essential, in effect, includes emdebian-tools (which in
turn depends on apt-cross).

Debian buildds don't support cross-building, I'm using emdebian-tools
autobuilder support. (We also have an autobuilder for the toolchains.)
The autobuilders grew organically from the basic scripts due to the
inevitable need to let packages cross-build unattended whilst I get on
with the rest of my life. The autobuilder works with the patches using
subversion and thus gives us a usable cross-buildd until such time as
the patches become compatible with Debian. Emdebian drops large amounts
of Debian Policy [1] and therefore the current patches cannot be
directly applied in the Debian package - other support is needed for
DEB_BUILD_OPTIONS and dpkg variants.

apt-cross - when used by emdebian-tools - needs to be able to understand
the Debian repositories and know how to handle primary mirrors that have
ARM alongside amd64 rather than partial mirrors that might not.

> Nor do I see how apt-cross would be affected by differences between a Debian
> vs. an Ubuntu mirror.  (Ubuntu main is smaller than Debian main, but is
> still self-contained, to be sure.)

Ubuntu mirrors do not contain the ARM, armel, mips, mipsel or any other
architecture binaries which are converted into cross-dependencies for
the cross-build. This is entirely in line with the aims of Ubuntu and is
not something that I'd expect to change.

> > How is emdebian-tools meant to cross-build for ARM on Ubuntu when Ubuntu
> > does not provide ARM packages and makes changes to the equivalent Debian
> > packages?
> Hrm, what changes are at issue here?  The Debian maintainers also make
> changes to Debian packages, all the time.  In what way do the Ubuntu changes
> differ that makes emdebian-tools incompatible with Ubuntu?

Emdebian keeps pace with the Debian packages - that is what the
autobuilder and the patches do. There is no such method for Ubuntu and
the Emdebian patches will not apply because Ubuntu has made their own
changes. Typically, the packages concerned are the standard debootstrap
packages, the very ones that Ubuntu changes to make Ubuntu what it is.

> > To me it seems highly unlikely that
> > cross versions of Debian packages would install over a Ubuntu base,
> > especially when those packages are the typical debootstrap selection
> > that have a variety of changes in Ubuntu. I don't run Ubuntu, I have no
> > inclination to test for Ubuntu and as no-one else has offered, I cannot
> > support Ubuntu."
> While the current absence of any official Ubuntu armel port seems like a
> pretty good reason to omit emdebian-tools from Ubuntu for the moment, the
> fact that the Debian package maintainer or upstream author doesn't support
> Ubuntu would not generally be a reason for Ubuntu not to include the
> package.  Debian also has any number of upstreams who don't "support"
> Debian, after all.

The lack of all other cross-building ports is also sufficient reason.
There is no possibility of Ubuntu supporting *all* the architectures
that Debian supports - that is why I wrote the tools for Emdebian, not

The lack of patches for Ubuntu packages would also prevent support. Once
more Debian packages cross-build without any patches at all, these
changes will feed into Ubuntu (and other derivatives) but some patches
may still be needed.

The basic problem is a bootstrapping one. It is possible to choose to
start from scratch and only use cross-built packages as the
cross-dependencies of other cross-built packages but the path through
the dependencies is very difficult to forecast and can change with each
iteration. This is why I have 245 source packages in the repository but
only use 145 on the actual device. I've had to cross-build some packages
that I now do not need because the Emdebian changes have dropped
optional components from the Debian builds (changing --enable-foo to
--disable-foo by patching debian/rules and debian/foo.install files) and
dropped the dependencies along the way. Hence, emdebian-tools takes the
simpler route of using the Debian packages to create the
cross-dependencies during each build, via apt-cross.

emdebian-tools then uses edos-debcheck to ensure the integrity of the
Emdebian unstable repository by checking the compatibility of each
potential upload *before* dput is called. edos-debcheck is also used
when identifying the packages that need to be updated in Emdebian -
checking the integrity of the entire repository in each of the three
current flavours (base, X and full GPE GUI). In this way, Emdebian can
ensure that root filesystems can be generated at any time to aid testing
and development.

apt-cross and dpkg-cross are fundamental to how emdebian-tools operates
- which is why I develop all three together for Emdebian and Debian.

> > How many packages could be in this situation? I don't expect it to be
> > many. Some form of filter on the Ubuntu side may be necessary.
> Yes, there is a blacklist in Ubuntu to prevent certain packages from being
> synced from Debian.  Scott Kitterman has already started the process now of
> getting emdebian-tools added to that list.

Scott has indeed offered to arrange this and I am very grateful for his
help. The removal bug has now been filed for ibex.

> BTW, in your cited blog post, I noticed that you wrote:
> > I really don't like Launchpad (I have quite enough web-logins thank you very
> > much) or the PTS link that shows Ubuntu bugs that I cannot close from
> > Debian.
> You can close Launchpad bugs in Ubuntu packages from Debian.  The "LP: ######"
> syntax lets bugs get autoclosed when your package is synced to Debian, or
> when it's merged by an Ubuntu developer.

Didn't know that, thanks. (Wouldn't have helped for that particular bug
which needed a Won'tFix closure, not a "Fixed in version foo" closure.)

[0] http://www.emdebian.org/buildd/
[1] http://wiki.debian.org/EmdebianPolicy


Neil Williams

Attachment: signature.asc
Description: This is a digitally signed message part

Reply to: