On Thu, Apr 21, 2016 at 6:22 AM Baurzhan Ismagulov <
ibr@radix50.net> wrote:
+ Yocto provides one-command, on-demand building.
** This is a bug, not a feature. **
Speaking from a professional developer's perspective and regarding production-destined embedded Linux environments (I make my living as a freelancer doing both), complete rebuilds from source code solve no problems AND create plenty of their own. It's a development, testing, and QA nightmare. Just run from the room, screaming optional.
Disclaimer: I've made a lot of money fixing failed Yocto experiments. Like, six-digits USD money. So part of me REALLY wants Yocto to continue to fail.
I used to run the CrossGCC FAQ, and as a result have a lot of first-hand experience with how two people can run the exact same script and end up with different results. It doesn't matter that it "shouldn't" happen, it still happens and it's difficult for the developer to detect it when it does.
That experience is precisely why I switched to using multistrap (and the various things that existed before it) to compose filesystems from pre-built, pre-tested binary packages.
I haven't built my own cross-compiler in more than five years, because there's absolutely no need to anymore AND there are plenty of reasons NOT to. Let today's experts in cross-compiler building and testing do their jobs, and I'll focus on my expertise USING those cross compilers well. We don't need to duplicate each other's work.
+ Yocto provides layering (multiple core / vendor / product layers possible).
I do this with multistrap using a single meta-package per product that describes the packages I want included in the final system. I have my own repositories for my kernel, bootloader, and other product-specific packages, and from time to time I modify upstream sources to fix bugs, change defaults, etc. and then host the debs myself. Most of the time, those modified debs are visible to all my projects but that's not required.
In other words, you don't need to rebuild from source code to manage different products.
You don't even want to: 99% of Debian's vendor/product/user/application/installation/preference/whatever differences are handled with config files anyway, so there's no need to rebuild anything.
When I want a package built with different gcc parameters, or some other change that makes rebuilding a package from source truly unavoidable, I host the resulting package files in my own repository so that they are treated just like the rest of the Debian universe AND the tools can see both packages. Using APT priorities lets me tell the tools what I want, they sort out how to get it.
- Yocto builds the whole project from scratch for every end product.
- Yocto builds cross-compilers from scratch for every end product.
Bug, and...run-from-the-room-screaming irresponsible bug. No, seriously.
A co-worker and I have tried to document what we're doing---which is simply using Debian well--- and we've separately communicated with Neil and Wookey about it. We've presented it at a couple of conferences, and have a crappy, content-challenged website. Our stuff is completely public (albeit poorly documented), and all are welcome:
We've got about a dozen or so targets supported, most of which aren't documented yet. After you install the basic SDK per the instructions in either of the above quick-start guides, apt-cache is your friend:
(ptux)bgat@saturn:~$ apt-cache search mkos
...
mkos-mlo-common - Composes filesystem images for MLO-based targets
mkos-nano - Composes filesystem images for Nano
mkos-amd64 - Make OS images for amd64
mkos-beagleboard - Composes filesystem images for Beagleboard C/D
mkos-beaglexm - Make filesystem images for Beagleboard XM
mkos-boneblack - Composes filesystem images for Beaglebone Black
mkos-bonewhite - Composes filesystem images for Beaglebone White
mkos-db8060 - Makes db8060 root fs
mkos-db8060a - Make filesystem images for db8060a
mkos-efi - Make an EFI-bootable OS on a disk
mkos-efi-bootloader - Make EFI-bootable bootloader image
mkos-ifc6410 - Make filesystem images for ifc6410
mkos-n7 - Make filesystem images for Nexus 7
mkos-overo - Compose filesystem images for Gumstix Overo
mkos-overostorm - Compose filesystem images for Gumstix Overo Storm
mkos-ptux - OS image building tools for Pragmatux
mkos-ptux-ts4700 - OS image building tools for Pragmatux
mkos-udoo - <insert up to 60 chars description>
The "mkos-<target>" tool is just a script that kicks multistrap, then formats the results into whatever format is required by the target device: fastboot images for the Nexus 7, uimage+nand for most MLO-based targets like Beagle*, and so on. I've even got a bit-rotting Android package for the N7 in there somewhere, that I used for a couple of training classes. (Wookey has seen that one first-hand.)
As you might imagine, I mirror Debian packages internally, and use reprepro and apt-cacher-ng a lot. Once deployed, my target devices also use apt-get over openvpn and/or USB (and/or whatever is available depending on the device, i.e. Iridium, DVD, etc.) for updates.
The term "SDK" as I've used it above is important. There's utterly NO point in having a tool that just builds filesystem images UNLESS you also have tools that set up development environments to target the contents of that image. Debian already knows how to do that, I'm just doing it their way.
And yes, I package client code as well. I host those packages on private repositories accessible over HTTPS. The required certificates are placed into the target devices during setup.
While there have been
efforts to bring Debian closer to the requirements in the embedded area [1],
it's very difficult to achieve that for the whole distro. Meta-debian does this
today, so I'm interested in trying meta-debian as an upstream. Does it generate
binary debs? One also needs an apt repo and debootstrap config.
I don't know much about meta-debian, but I can tell you from personal experience that ordinary Debian as an embedded Linux OS isn't difficult at all if you've got > 64MB of filesystem space to work with. So I dispute the "it's very difficult to achieve that for the whole distro" statement.
Most of my systems are headless, but my co-worker (hi, Ryan!) has deployed several systems that use a Java GUI for their HMI. At the end of the day, Debian doesn't really care.
> How do you achieve Products' modified Debian packages?
I do it by modifying the package sources, rebuilding, and then hosting the modified .debs on a non-Debian repository. Sometimes it's reprepro on my local workstation, other times it's a cloud server I've bought for such things.
Actually, this requirement doesn't come up nearly as often as you'd think it would. More often we're just applying bug-fixes that haven't made their way out of unstable/experimental yet, and by careful version/naming we can automatically roll over to those packages as they become visible in Debian proper.
We have two levels of customizations. At the company level, there is an
internal embedded Linux distro (like meta-debian, but binary). The distro team
creates source and binary debs and packs them into an apt archive, which is
versioned in a distro repo. Projects install them without re-building every
time.
At the project level, selected upstream packages are optionally customized as a
complete fork (separate package repo / branch with debian/). The project team
creates source and binary debs and packs them into the project's apt repo. The
project installs them without re-building every time.
This way of customization is rather simplistic; it duplicates the whole package
and requires repo / branch management. Now we would like to study your
experience in this area and try the debian/*.patch + bbappend way. Still, we
would like to keep source and binary deb pre-building by the respective team.
At the moment, apt archive generation isn't yet integrated into isar.
Git pretty thoroughly replaces debian/*.patch + bbappend.
Product source and binary debs are re-built for every new release.
Why? "Just to make sure"? Make sure of what?
Sorry to be the cranky old man in the room, but someone has to say it: Debian in its present form already works REALLY well for embedded Linux. Let's just use it, and improve it where it needs improving so that everyone benefits AND everyone converges on a consistent, productive, familiar, and utterly reliable way of doing things.
b.g.