I think an announcement may be better once it actually hits the Debian
archive. Definitely one for DPN though.
Neil
On Wed, Feb 27, 2013 at 09:59:23AM +0200, Andrei POPESCU wrote:
> Definitely worth a news entry, but what about an announcement?
>
> Andrei
>
> ----- Forwarded message from Wookey <wookey@wookware.org> -----
>
> Date: Wed, 27 Feb 2013 02:10:40 +0000
> From: Wookey <wookey@wookware.org>
> To: cross-distro@lists.linaro.org, debian-arm@lists.debian.org, linaro-dev@lists.linaro.org, debian-devel@lists.debian.org,
> ubuntu-devel@lists.ubuntu.com
> Subject: arm64 Debian/Ubuntu port image available
> Mail-Followup-To: cross-distro@lists.linaro.org, debian-arm@lists.debian.org, linaro-dev@lists.linaro.org, debian-devel@lists.debian.org,
> ubuntu-devel@lists.ubuntu.com
> Organization: Wookware
> User-Agent: Mutt/1.5.20 (2009-06-14)
>
> State of the Debian/Ubuntu arm64 port
> =====================================
>
> *** Arm64 lives! ***
>
> Executive summary
> -----------------
>
> * There is now a bootable (raring) image to download and run
> * Everything has been rebuilt against glibc 2.17 so it works
> * A bit more work is needed to make the rootfs useable as a native buildd
> * Multiarch crossbuilding and the build-profile mechanism is mature enough to cross-build
> a port from scratch (this is a big deal IMHO)
> * All packages, sources and tools are in a public repo and this work should be reproducible.
> * This image is fully multiarched so co-installing armhf for a
> 64/32 mix should work nicely, as should multiarch crossbuilding to
> legacy x86 architectures. :-) (but I haven't tried that yet...)
>
>
> * Linaro wants 'the distros' to take this work forward from here so people interested in
> Debian and Ubuntu on 64-bit arm hardware need to step up and help out.
>
>
> Bootable images
> ---------------
>
> A milestone was reached this week: Enough packages were built for arm64 to debootstrap an
> image which booted to a prompt! After a bit of fettling (and switching to multistrap) I got
> an image with all the packages configured which boots with upstart to a login prompt (I
> admit, I did get quite excited about this, as it represents the coming together of nearly 3
> years work on multiarch, crossbuilding, bootstrapping, cyclic dependencies and arm64 :-)
>
> The images are available for download: http://wiki.debian.org/Arm64Port#Pre-built_Rootfs
> And there are destructions there for making your own.
>
> All these packages were cross-built on raring, untangling cyclic dependencies with build
> profiles (see wiki.debian.org/DebianBootstrap for how that works), making this the first
> (non x86) self-bootstrapped debian port ever (so far as I know). All (?) previous ports have
> been done using something else like OpenEmbedded (armel, armhf), RedHat/HardHat (arm, alpha,
> mips), something IBMy (s390) to get an initial linux rootfs on which debian packages are
> built.
>
> The new bootstrap process is (almost) just a list of sbuild commands. In practice there are
> still a few rough edges around cross- build-dependencies so of the 140 packages needed for
> the bootstrap, 9 had to be built manually with 'dpkg-buildpackage -aarm64 -d' (to skip
> build-dep checks) instead of 'sbuild --host arm64 <package>'.
>
> The current bootstrap packageset status is here:
> http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
>
> There is no armv8 (arm64/aarch64) hardware available yet, so this image can currently only
> be run in a model. ARM provide a free-beer prorietary 'Foundation model' so we do have
> someting to test with. It's sluggish but perfectly useable. Booting the images takes a
> couple of minutes on my fairly average machine.
>
> The images are using the Linaro OE release kernels which seem to work fine for this purpose.
> Thanks to Marcin for modified bootloader lines in .axf files.
>
>
>
> Image status
> ------------
>
> I was impressed that things basically 'just worked' on first boot. There is of course plenty
> of breakage, I'm sure, and I haven't looked very hard yet, but it's a lot better than I
> expected after months of just building stuff and testing nothing. (Things that are poorly:
> nano can't parse it's own syntax-coluring files for example, and multiarch perl has the
> wrong @INC path compiled in; I'm sure there is more). Consider this alpha-grade until it's
> been used a bit more.
>
> Things that are not yet built which would make the images a lot more useful are apt and a
> dhcp client. apt needs gnupg needs curl needs nss. The nss cross-build needs fixing to
> unbung that. A debian chroot without apt turns out to be disappointing quite quickly :-)
> Expect an updated image with more packages very soon.
>
>
> Multiarch crossbuilding
> -----------------------
>
> It's really nice to have building and crossbuilding using exactly the same mechanisms
> and tools, with all files having one canonical location, and dependency mechanisms that
> are reliable. The more I've used this, the more I've been impressed by it. There is
> still work to do to expand the set of cross-buildable stuff, but it's a solid base to
> work from.
>
> Getting this port working has been 'interesting' because it's attempting 4 new things all at
> once: multiarch (file layouts and dependencies), crossbuilding (tools and packaging support
> in a distro that historically was always natively built), arm64 (aarch64) support in
> packages that need it, and build-profiles to linearise the build-order.
>
> The arm64 part of this is a relatively small part as the heavy lifting has been done
> upstream (gcc, (e)glibc, binutils, kernel, libffi, autotools and a lot of minor fixes in
> various packages). Thanks are due to doko (Matthias Klose) for sterling work getting all
> that integrated into the debian and ubuntu toolchain packages, and infinity (Adam Conrad)
> for merging various eglibc branches. There were also hordes of very boring patches of the
> form 'update config.sub and guess before building'.
>
> Most of the work has been in making things cross-build (exactly the same fixes needed for
> armel/hf too so I've had plenty of help there from canonical types who want cross-building
> for arm to work nicely), and particular thinks to Neil Williams for taking on the perl
> cross-build challenge and creating the debian-perl-cross package to manage the
> cross-configury, whilst also working with upstream to make the whole thing a bit less 1996.
>
> Multiarchifying has been going on nicely in libraries and -dev packages, but things like
> perl and python needed significant work, along with a lot of boring bugs saying 'mark this
> package MA: foreign' and 'build-dep on python:any or perl-base:any'. Thanks are due to doko
> for the python multiarching and Niko Tyni for the perl multiarchification. Getting all 3
> 'aspects' of multiarch perl, cross-built perl and arm64 perl config to work at the same time
> was quite hard work, and there are still bugs there. Wider usage of multiarched perl would
> no doubt sort this out reasonably quickly. I started a wiki page to track the status of
> multiarched cross-buildable perl: http://wiki.debian.org/Multiarch/Perl . Help would be
> welcome.
>
> The build-profile work is described on the http://wiki.debian.org/DebianBootstrap page.
> Progress has been greatly helped by GSOC projects last year, with good work on the tools
> (crossbuild-essential packages, build-profile support) from P.J McDermott and an impressive
> contribution from Johannes Schauer on dependency analysis tools around libdose, and apt
> build-profile support.
>
>
> All of this apart from multiarch perl, crossbuildable perl and build-profile stuff (and
> a few pending patches) is already in raring.
>
>
> Building stuff yourself
> -----------------------
>
> Setting up an arm64 build environment is very simple. Use sbuild-createchroot or mk-sbuild
> and point at the bootstrap repo, with a bit of config and some updated tools packages from
> the repo (amd64 only supplied). Details are given on
> https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap
>
> Once you've created a tarball chroot builds are simply done with
> sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package.dsc> or
> sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package>_<version> (I'd love it
> if sbuild got smart enough to work out the version itself when given a distro - Roger
> says he's working on it)
>
> To deal with the chore of 'find version, run sbuild, sign result, upload to repo, import to
> repo, deal with reprepro bitching if you re-upload the same version of something' for every
> package build, I wrote 'dimstrap' which is a simple-minded tool to wrap that up and either do
> one-off builds or run through a list. It is part of the xbuilder package here:
> https://launchpad.net/~linaro-foundations/+archive/cross-build-tools/ It also includes the
> logfile-parsing script ('generate html') which generates the nice status pages:
> http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
>
>
> Image building
> --------------
>
> The config and instructions provided (in
> http://wiki.debian.org/Arm64Port#Building_your_own_rootfs_image ) is
> for multistrap. Debootstrap sort-of produces working images too but
> takes a lot longer to unpack/configure, and misses out various vital
> packages (like libperl5.14). I'm sure it could be kicked into
> submission. In theory multistrap (apt really) should have got all the
> arch all packages from the main repo, but in practice it refused to do
> that so I had to rebuild them or copy them over anyway (grumble).
>
> Any package that installs replaced conffiles seems to generate invalid
> dpkg status entries (ifupdown did this to me). I've not got to the
> bottom of that yet. Deleting the offending line gets you an image that
> works.
>
> Issues
> ------
>
> General:
>
> The build-profile patches for dpkg and apt need to be pushed into the distro to make
> that feature permanent. A thread on debian-devel is working on that
> (http://debian.2.n7.nabble.com/Bootstrappable-Debian-proposal-of-needed-changes-td2848200.html).
> The main issue is what syntax to use '<>' or '[]' and how to deal with multiple overlapping
> profiles. The patches to debian control cannot go in until at least the syntax is agreed and
> the tools will parse them without barfing. Johannes ands I will send an updated spec
> soonish.
>
> The missing piece of bootstrapping with regard to build-deps is packages that build-dep on
> gcc-4.6 or binutils. When cross-building this should be satisfied by <triplet>-gcc-4.6 or
> <triplet>-binutils. Nothing makes that happen currently. A scheme has been mooted but
> nothing is implemented yet.
>
> There is debate about whether cross-toolchains should build against multiarch libraries
> (libgcc, libstdc++) like everything else, or have their own internal copies. Doko and I
> disagree on this matter. That will need to be worked out at some point.
>
> We won't get that much further with fixing cross- object-introspection, which is a
> non-trivial job.
>
> Image-related:
>
> The images do essentially work but very little has been tested so far.
>
> Multiarch perl still needs work.
>
> nss needs cross-building in order to get apt cross-built
>
> I've not got networking working yet. Info is here:
> https://fedoraproject.org/wiki/Architectures/ARM/AArch64/FoundationModel_Networking
> lack of a dhcp client in the image hasn't helped there.
>
>
> More info
> ---------
>
> The canonical arm64 port info page is:
> http://wiki.debian.org/Arm64Port
>
> Full arm64 cross-build status (i.e everything that has been tried) is here:
> http://people.linaro.org/~wookey/buildd/raring-arm64/status.html
>
> All the patches generated so far are here:
> http://people.debian.org/~wookey/bootstrap/patches/
>
> (most that can, have been filed as bugs - there is a backlog of stuff
> filed in Launchpad but not yet forwarded to the Debian BTS - yes I am
> a bad boy - blame the fact that you can't use reportbug or bts from
> inside ARM due to their idiotic email policies).
>
>
> Future work
> -----------
>
> Firstly we should say thank you to Linaro for sponsoring this work in various ways over
> the last 3 years. We wouldn't be at this point now if it wasn't for that. However
> Linaro has a lot of things to do and is trying hard not to do distro's work for them,
> concentrating on upstream things. This makes sense for commercially-backed distros like
> Red Hat and Ubuntu, but rather less for Debian where we _are_ the distro just as much
> as anyone else is, and ultimately someone has to spend the time to get stuff working.
>
> Anyway, I was supposed to stop work on this some time back, but have largely failed to
> do so (cross-building is so moreish - there is always one more build to try before
> bedtime!) and appreciate being given enough slack to get this to a point of actual
> utility. However I expect to have much less time to spend on this from now on, except
> insofar as it still co-oncides with things Linaro wants doing. I'd love to hear from
> people who actually want to use this, to get more packages built, the Debian
> cross-toolchains sorted, build-profiles finalised, and a whole pile of stuff fixed once
> Wheezy is released. I'm pretty sure there are quite a lot of people who want multiarch
> Debian or Ubuntu on their arm64 machines (or models).
>
> I hear rumours that actual hardware may appear sometime around the middle of the year
> with some bagsied for Debian. Setting up the ports infrastructure for that would be
> good. I don't know if anyone is interested in building slowly on models in the
> meantime, or if we should just carry on crossing and see how far we get. This table
> shows that 471 packages in raring can be expected to cross-build already:
> http://people.canonical.com/~cjwatson/cross/armhf/raring/
>
> Todo:
>
> Fix up multiarch/cross perl
> Fix nss
> Build missing packages for apt
> Build missing packages for build-essential
> Build Debian cross-toolchain
> Get all this working in unstable as well as raring
> Setup buildds
> Build all the other packages
> Set up automated bootstraping runs (eventually)
>
>
> Current setup
> -------------
>
> Builds have all been run locally using the sbuild/chroot setup described above and on
> the Arm64Port page, which should be easy for anyone to reproduce. The main irritation
> is keeping up with raring: out of sync libraries are not MA-installable. Logs are
> uploaded to people.linaro.org (rsync). The reprepro repo is on people.debian.org
> (dupload). This stuff should probably move to ports.debian.org and ports.ubuntu.com,
> but neither of those are set up for cross-building so I'm not quite sure how this will
> work.
>
> I could go on at great length about the machinery of profiled bootstrap builds, and
> interactions between tools, but it's not very exciting, so will resist. Suffice it to
> say that whilst it's all pretty slick I'd still like better buildd tools.
>
>
> Build-profile changes
> ---------------------
>
> The build-profile patches are not yet upstreamable so are collecting in the repo.
> The patch set so far is here: http://people.debian.org/~wookey/bootstrap/patches/profiles/packages/
>
>
> Other thanks:
> Other people who have helped make this happen in various ways but not got a mention above:
> Colin Watson, Dmitry Ledkovs, Steve Langasek, Harry Leibel, Thibaut Girka, Roger Leigh,
> Marcus Shawcroft, James Morrisey, Jonathan Austin, Steve McIntyre, Peter Pearse, Aurelien
> Jarno, and whoever does sysadmin at people.{linaro,debian}.org
>
> I hope I didn't forget anyone, or any important information.
>
> Feedback from anyone attempting to get this working outside my computer is very
> welcome. I have almost certainly forgotten to write down some things, and upload
> correct versions of some other things.
>
>
> Wookey
> --
> Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
> http://wookware.org/
>
>
> --
> To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: http://lists.debian.org/20130227021039.GN5031@stoneboat.aleph1.co.uk
>
>
> ----- End forwarded message -----
--
Attachment:
signature.asc
Description: Digital signature