Re: Debian-based product build system

On Thu, Apr 21, 2016 at 7:15 PM, Bill Gatliff <bgat@billgatliff.com> wrote:

Sorry in advance for the long rant...

On Thu, Apr 21, 2016 at 6:22 AM Baurzhan Ismagulov <ibr@radix50.net> wrote:

+ Yocto provides one-command, on-demand building.

** This is a bug, not a feature. **

It may look like a bug, but it actually has some use.

Speaking from a professional developer's perspective and regarding production-destined embedded Linux environments (I make my living as a freelancer doing both), complete rebuilds from source code solve no problems AND create plenty of their own. It's a development, testing, and QA nightmare. Just run from the room, screaming optional.

This is true, it is an extremely complex tool. But it is used to do extremely complex builds on huge code bases that are also modular. For that use case it is actually quite good. This is the area that needs to be compared with ISAR, or one of the areas.

- Yocto builds the whole project from scratch for every end product.
- Yocto builds cross-compilers from scratch for every end product.

Bug, and...run-from-the-room-screaming irresponsible bug. No, seriously.

Definitely not. In automotive use cases, you can have *routinely* 100 million lines of code. In some cases you get up to double that. You get a heterogeneous software stack that starts with an RTOS, moves to Linux, then to virtualization, then to a mix of GNU/Linux with foreign frameworks, and finally a diverse userland often in mutiple levels of containment. When there is a code change anywhere you *often* need to rebuild everything. In addition, there is a fair amount of invention, so continuous integration is key and then rebuilding from scratch every day is a huge benefit because it finds bugs early. In addition, the layering and recipes provided by OE / Yocto allow you to arbitrarily switch out silicon which in product development can save millions of dollars. Lastly, you need the modularity of recipes and layers so that you can have mutliple, distributed software development teams that each integrate with arbitrary git repos behind firewalls and out on the internet.

Yocto is a seriously good tool for this.

[snip]

Git pretty thoroughly replaces debian/*.patch + bbappend.

Product source and binary debs are re-built for every new release.

Why? "Just to make sure"? Make sure of what?

Sorry to be the cranky old man in the room, but someone has to say it: Debian in its present form already works REALLY well for embedded Linux. Let's just use it, and improve it where it needs improving so that everyone benefits AND everyone converges on a consistent, productive, familiar, and utterly reliable way of doing things.

I agree with you wholeheartedly. And I'm deeply impressed with your work with Pragmatux. But I have come to realize, reluctantly, that very large software projects with a lot of development and hard deadlines doesn't work well with Debian. There just aren't enough resources in Debian to handle the flow of code and having to wait for bug fixes, new package uploads, build runs, etc. is not feasible, sadly. I think it can work if you pull in something like ISAR behind your firewall, but with that I worry that changes made behind the corporate firewall are a low priority to push up to Debian. Nonetheless, ISAR seems to potentially be a smart compromise between the "boil the ocean" yocto approach and binary package modularity of Debian and I hope it succeeds. I think it is needed.

Cheers,

Jeremiah