On Mon, 2008-09-01 at 10:18 +0200, Manuel Prinz wrote:
> Hello everyone,
>
> I've heared about Emdebian a while ago and got interested in the
> project, so I had a look at it during the weekend. I'm totally new to
> the topic (as in "embedded system") but interested in the mechanisms,
> efforts and workflows in this area. I do not have a concrete project
> planned (yet), so I used Emdebian for playing around and learning. I had
> some success and was very impressed by all the tools and how great they
> worked! Kudos to everyone involved in the project, you did (and do!)
> great work!
Bear in mind that we only have ARM binaries so far, people are working
on armel, there has been some work on i386 but no work, AFAICT, on MIPS.
See http://www.emdebian.org/emdebian/extending.html for the kind of work
needed to bring Emdebian to MIPS (or mipsel).
In particular, I need someone to verify the cache file values against
MIPS builds and check that the values we are using are actually
architecture-independent. To do that, you need to have built each
package and checked the native build logs from Debian against the cached
values.
http://wiki.debian.org/EmdebianGuide
The current range of cache keys and values is now part of dpkg-cross
2.3.1 - see /etc/dpkg-cross/cross-config.cache - in order to have a
single place where these values can be checked and processed. Once you
have built packages that need mips-linux-gnu.cache files, 'emcache' from
emdebian-tools 1.4.6 can help you identify differences (although it only
has explicit support for ARM and armel at this time).
Note that in the following reply, I'm not talking about machine:variant
customisation which *can* be implemented out-of-tree - this is a problem
of porting Emdebian to MIPS at which point any changes that MIPS needs
must be folded into the existing changes needed for ARM and armel.
I knew this point would come and I've had it in mind during development
but I do want to ensure that we can retain ARM and add MIPS without
breakage.
> While playing around a few quesions came up for which I did not find
> answers on the webpage or wiki and I hope you can answer them. I hope
> those are not too stupid; as said, I'm totally new to the topic. So here
> is what I did:
>
> I first installed a MIPS toolchain as I wanted to use something not yet
> supported in Emdebian. (As far as I know.)
Emdebian does have a MIPS toolchain - $ emsetup -s -a mips - hopefully,
you find that OK.
> packages needed for a rootfs. I used em_autobuild for that. I modified
> the php files[1] to match my directory structure, so I could follow the
> progress better. Of course em_automake failed at some point. The
> problematic package was ncurses. I fixed it and put the patch file in
> the location of the other patch files,
I need the details of that patch, respective to the current patch set.
Which file did you change?
> named emdebian-something.patch.
> The patch was not applied by emdebuild, though.
Those patches come from SVN.
> I did so manually and
> emdebuild successfully build the package. I ran the build again using
> "em_autobuild -l --package ncurses" in order to get a build log created
OK, autobuilders are meant to just do as they are told from clean
sources in a clean environment. What you wanted to do there was just
emdebuild or empdebuild to build in a chroot. Don't worry about build
logs at this stage (or possibly create them manually using 'tee').
Autobuilders are for when you have everything working - think of them as
ftp-master. emdebuild is a local builder, like debuild, which will do
whatever you want. To get modifications of patches into the SVN for the
autobuilder to use, you need SVN commit access and as your modification
could possibly break an existing build, that means sending the patch
modification to me or to the list.
OK, a few pointers:
1. Always start with emdebuild and empdebuild. Do not go straight for
em_autobuild - that is for when you have a full set of working packages.
2. Fix all the package builds before you start playing with the
autobuilder or emsandbox for rootfs generation.
3. Be careful using a repository too early - you may well find that you
are needlessly incrementing the em suffix just to be able to upload
locally.
4. Report changes that you need in the patches. emdebuild will update
the timestamp within the patches everytime you build, I don't need those
changes, just the substantive changes with clear explanations of *why*
that had to be changed for MIPS. Cache file support will improve - for
now, you could probably just copy the relevant section out
of /etc/dpkg-cross/cross-config.cache for the packages described in that
file, as mips-linux-gnu.cache in the top level of the source directory
of the package.
5. DOCUMENT THINGS. Add things to the wiki, email the list, file bug
reports against emdebian-tools - if you do nothing else, ensure that
someone coming after you has a clear trail of docs that cover not only
what you did right but what you did wrong and how to avoid doing the
same.
One big task in Emdebian right now is getting the docs into a more
usable state. I wrote nearly all of the current docs and they make
perfect sense to me (!) so someone else needs to work on the docs and
ask questions to make sure they make sense to others.
Another BIG task is to translate the emdebian-tools manpages -
consisting of 1,700 strings (which can change fairly often). If English
is not your first language, see the README for how to generate
po/emdebian-tools.pot and then create a PO file from that. All
translations gratefully received.
As of Lenny, only a few packages cross-build directly from the Debian
source. By Squeeze (==Lenny+1), *ALL* core packages in Emdebian (about
200) need to cross-build without needing patches. To achieve that, I
need help, I need people filing bugs against emdebian-tools, I need
people reporting issues, fixing the docs, eventually using embug
--prepare to file bugs themselves, running autobuilders for their own
architectures and a massive number of tweaks and changes in
emdebian-tools to get multiple architectures supported cleanly.
Emdebian 1.0 (based on Debian 5.0 "Lenny") {yes, that's the full title}
is likely to ship only with ARM - armel is a possibility. Emdebian 2.0
(based on Debian 6.0 "Squeeze") needs to ship with armel, i386, mips,
mipsel and possibly a few others like m68k. Whether we still have ARM by
Squeeze is uncertain. I'm only going to be autobuilding ARM/armel so the
others need people to step up and run autobuilders.
> 1. Why did the patch file disappear? Does emdebuild clean the package
> and only patches in the SVN survive? How can I keep them? Do I need a
> SVN account or do I need some SVK mirror hackery?
emdebuild works locally with whatever changes you make - it's your test
builder, the lowest link in the chain.
em_autobuild implicitly assumes that you've done all the testing,
everything is fine and it can make a blind, fully-automated, build. To
do that, it must fall back to the minimal set of patches. em_autobuild
will not listen to local changes.
I thought about making an architecture-map of emdebian-tools a while ago
but then I changed the structure. :-)
Rough structure:
emsandbox - top level rootfs script
emrecent - support script
em_autobuild - top level build script
emdebcheck
emtargetcmp support scripts
emcache
empdebuild - chroot builder
emsource - patch management utility
emdebuild - lowest level build script
You start from the bottom up. Now, until more patches are included into
Debian, emdebuild won't work without emsource but the theory is that
emsource will become more and more of an option as more and more bugs
are fixed in Debian. Once that happens, emdebuild can also become
optional and 'dpkg-buildpackage -a' can become the default.
You *must* always ensure that your package builds cleanly in empdebuild
before going for the autobuilder.
Before considering upload to an "official" repository like
buildd.emdebian.org, you *MUST* be using chroots for all builds - just
like Debian, if you fail to test a version in a chroot build, you will
likely get a FTBFS RC bug in Debian. Consider it the same for Emdebian -
no chroot build, no upload. End of.
Therefore, for uploads destined for Emdebian, you should consider the -p
option to em_autobuild as mandatory.
> 2. Why did the patch not apply? (I guess because it was removed before
> it could be applied.) Is it sufficient to name patches emdebian-* to get
> them applied?
No. emdebuild does all the patch generation for you. If you modify a
file in debian/ then emdebuild will create or update the patch file for
you. If you have SVN commit access, you can use emdebuild --svn-only to
commit those changes but as you are working on MIPS, I want to see the
changes before a commit so that I can ensure that MIPS changes can live
alongside the ARM builds and to work out why you need a different patch
in the first place.
> 3. Some packages build by em_autobuild are necessary for other packages
> to build, so I think I need to set up a local repository with
> successfully build packages in order to get all packages of the rootfs
> to build. Is em_autobuild able to "upload" packages into that
> repository? If not, what is the currently used way to achieve this?
reprepro - that probably needs documenting on:
http://wiki.debian.org/Embedded_Debian_packaging_infrastructure
Basically, all you need is to create a file conf/distributions where you
want the repository, using the manpage of reprepro. Then use 'reprepro
export' to create the rest of the meta-data and 'reprepro include
unstable' to add a .changes file to the repository.
Once you have built the packages and any changes are folded into the
existing Emdebian patch sets, then yes, emrecent could upload the
packages to the Emdebian target repository - we'll need to sort out
access and there will probably need to be changes in emdebian-tools
(unless the armel people get there first) such that the .changes files
you create for MIPS are done with the -B option so that reprepro doesn't
refuse the upload due to a duplicate .diff.gz or .dsc.
> Thanks in advance and sorry for this rather lengthy email! If you think
> Emdebian can benefit from what I did/do, then I'd be happy to support
> you. Anyhow, as said, I'm quite unexperienced with that and not sure
> whether anyone is interested in MIPS at all, so maybe it's nothing more
> than an experiment. Anyhow, I'd fun so far! :)
>
> Best regards
> Manuel
>
> [1] I think I used the testing version, all paths were hard-coded. If
> that's not fixed in the latest emdebian-tools, I could provide patches.
I am not sure to what you are referring here.
--
Neil Williams
=============
http://www.data-freedom.org/
http://www.nosoftwarepatents.com/
http://www.linux.codehelp.co.uk/
Attachment:
signature.asc
Description: This is a digitally signed message part