Re: Configuring foreign packages without qemu in order to boot a system
On Sunday 02 September 2012 23:21:09 Neil Williams wrote:
> On Sun, 2 Sep 2012 21:26:09 +0200
>
> Paul Boddie <paul@boddie.org.uk> wrote:
> >
> > Right, I can see straight away that /sbin/init might have a hard time
> > doing anything useful because the symbolic links within /etc/rcS.d don't
> > exist. The problem I've been experiencing is getting a shell to run at
> > all, I think, making it difficult to run a configscript like the one
> > described.
>
> That's the role of the setupscript - to put files into the rootfs which
> are not package-based. Many files on a typical Debian system were put
> there only by Debian Installer, not by packages. You have to decide
> which of those you want, what goes into each and make that happen via
> setupscript.
In this case, the configuration of sysvinit will populate /etc/rcS.d, but we
obviously can't get that to happen on the build host, and so we're left to
either jump the gun on that configuration in a setupscript, as you say, or
somehow make the configuration happen on the target in advance of wanting to
run /sbin/init.
> > The latter part of this paragraph was particularly helpful in pushing me
> > towards trying busybox as a helper in getting configuration to occur.
>
> You may well find it more useful to create a full initrd image which
> has a kernel and a proper environment. i.e. make the step up from
> busybox to buildroot.
Yes, I have that already, but the plan was to try and do everything
Debian-style beyond the bootloader and kernel. That's where busybox came in
as a sort of helper. I guess I could try and figure out what kind of
configuration I can "fake up" on the build host, but that could take days of
experimenting. (This isn't my job, by the way.)
> > > Multistrap can easily take a rootfs all the way up to a full package
> > > set with a UI and customisations and give you a tarball with absolutely
> > > everything setup and ready to run. The cost of that is that the rather
> > > arbitrary division line with which people have become familiar from the
> > > debootstrap model simply doesn't exist. This has advantages in package
> > > selection but also has requirements about package configuration.
> >
> > I'm not quite sure I follow this last paragraph. I've tried to use
> > multistrap to deploy a rootfs that I want to be able to boot into, but it
> > isn't ready to run, even with a minimal configuration.
>
> It is ready to run, it just needs you to sort out enough of the init
> that the kernel can make the jump from itself to the init and on to the
> rootfs. You're missing a stage. A rootfs is just that - a root
> filesystem. It is not what the kernel boots into directly - that's init.
> Depending on the system, init can be fully prepared within the rootfs
> but that involves using setupscript to put the init inside the rootfs
> and probably some work by the init process to finalise the rootfs
> itself the first time it runs.
I guess I'm just used to system deployments where the root and init
filesystems are one and the same. I've also played with things like live CDs
where the distinction still exists, however, but I suppose I'm just trying to
find reasonable ways of merging the environments and consolidating the
technologies involved.
> With a buildroot rescue environment, multistrap can and does do just
> what I described in that paragraph. We rely on it at work for all our
> boards and use buildroot as the rescue shell, scripted via the
> bootloader and configscript.
>
> > It's a shame that there aren't any prominent recipes out there for this.
>
> www.balloonboard.org but I wouldn't class that as prominent or
> particularly easy to follow.
I stopped following that site quite some time ago, sadly.
> > What
> > you're suggesting (uClibc and busybox) is what the OpenWrt distribution
> > on my NanoNote provides, and that's how I managed to get Emdebian
> > bootable initially, but it would be preferable to be able to drop the
> > requirement on OpenWrt, buildroot or other systems.
>
> Once the system is configured, you can drop that stage but keep it as
> a fallback. It will still prove immensely useful to have that rescue
> environment accessible (maybe via a keypress during boot) for those
> times when something screws up your rootfs.
But that's never supposed to happen! ;-)
I can see the argument for having it, though.
> > > "just enough of a system to run init (and chroot)" sounds a lot like
> > > a description of buildroot.
> >
> > Yes, but it would be great to drop the need for buildroot. I think
> > deploying busybox-static might be sufficient for that, at least when
> > deploying a system. I also want to look into building Debian kernel
> > packages.
>
> Build your own kernel package, not the Debian kernel package - that
> includes a complete initramfs and is an order of magnitude larger than
> a simple kernel, even with modules. And remember that the default
> in-kernel Debian packaging is *native-only* and will not cross-build
> without patching.
This is very useful to know, thanks! It certainly saves me a lot of time
knowing this in advance of any investigation into Debian practices.
> Why drop buildroot? You know you'll need a rescue shell when something
> goes wrong later and it's only a few Mb. Our buildroot + kernel
> zImageInitrd is only 3Mb all-in - it has to be, it goes onto a 4Mb
> NOR chip.
I don't think you necessarily need to persuade me further about the benefits
of deploying a separate init filesystem: the practice certainly has its
merits, and I can see particular benefits in having a running system while
Debian configuration processes occur in the background, even if it merely
displays some kind of indication that the configuration is ongoing.
[...]
> > Well, I'd like to be able to write something up that neither involves
> > tens of manually invoked steps nor something that just points to a rootfs
> > that has been magically constructed according to a set of forgotten
> > instructions (or worse, trial and error), that also helps people
> > troubleshoot if the infrastructure changes or if they're trying to do
> > something similar but not identical.
>
> The entire process, from bare metal to running UI, can be scripted - we
> do it using a custom bootloader, a custom kernel Debian package,
> buildroot, multistrap and configscript. No commands are even necessary,
> if the rootfs is sufficiently damaged, the buildroot kernel gets called
> instead of the broken/missing rootfs kernel and it downloads a new
> rootfs from the network and then just sorts itself out. Yes, it takes
> an amount of time but it's better than leaving the device bricked.
I'm really talking about community documentation here, describing not just the
concrete steps that accomplish the task, but also the decisions around it,
potentially for newcomers to the field who aren't already familiar with the
different tools and who don't have a ready-to-use environment. I've put an
initial draft here:
http://en.qi-hardware.com/wiki/Emdebian
Incidentally, there is a project called DebWrt that combines OpenWrt and
Debian, although I'm still not completely sure how it relates to what you
have been describing or to what I have been trying to do.
I certainly appreciate your guidance on this matter, and I'd like to thank you
and everyone else for suggesting solutions to my problems.
Paul
Reply to: