Re: [Question] Integrating the debian installer in live-build - DVD, multi-arch, custom, etc.

[jnqnfe@gmail.com]

@michael I always quote (include) the message I am replying to in my
message when writing to multiple people or a mailing list, or otherwise
use "@person" to address a specific person. When quoting, the quote
always starts clearly with an "On [DATE, TIME] Michael wrote" type line
(at least in my email program). Sometimes I write my message above the
quoted text, sometimes below, sometimes broken up inbetween where bits
of my reply need to be associated with specific parts of what I am
replying to for clarity. Perhaps the issue lies with the set of email
addresses that this thread has accumulated in the 'to' and 'cc' fields,
rather than generically being targetted at the 
'debian-live@lists.debian.org' mailing list address, or perhaps you're
not used to me replying above quoted text rather than below?? I feel I
am being perfectly clear; I do not understand why there's any
confusion.

btw 'jnqnfe' is my handle, my first name is Lyndon, is this not
appearing at your end? I see your name as "Michael ." at my end. Also
it's typical to address someone in the form @keltoiboy or @michael
rather than just a handle/name in all caps (shouting) fwiw.

Yes I know how to create a multi-arch system, not that I've actually
tried it, but I'm very familiar with live-build being a significant
contributor to the codebase.

I've already discussed the details a little in this thread tonight, but
to repeat a little:
 1. You use -a i386 --linux-flavours "686 amd64", replacing "amd64"
there with "amd64:amd64" if using live-build version 20191219 or newer.
This gets you the correct set of kernels.
 2. Having just one single package name like 'firefox:amd64' in a
package list in your config is enough for live-build to know to run
`dpkg --add-architecture amd64` and thus get a combined i386 and amd64
image.

Whether or not you need to have both 'Foo' and 'Foo:amd64' in your
package lists for every single package you add to your package lists to
get them in both arch forms is a detail I do not know off the top of my
head. But just one in the form 'Foo:amd64' is enough to get the
additional architecture configured as just stated.

This gives you such a combined live environment, and if you use the
"live" version of the installer, you thus also have a means of
installing it. If you use the other installer types on the otherhand
then you'll only get the one for the base architecture (i386 here) and
thus only be able to install i386 (though you can manually add amd64
later of course).

Note that having a live environment containing both i386 and amd64
architectures like this is the only means of having multiple
architectures in a single image. You can **not** have an i386 live OS
and an amd64 live OS sat side by side in the same image, selectable
from different menu entries.

With regards to your config, note that there is a big difference
between appearing to work and actually working. Just because something
appears to work does not mean that there is not a problem.

Furthermore your script in fact is **not** working just fine, it IS
broken, as I've just confirmed by running it myself (not that I needed
the confirmation). A clear example of the problem is that when you run
it you will see a final output line that ends with "--linux-flavours:
not found". This is an error, caused by some of the problems I
previously stated, and is explained further below.

Another problem I spot with your config script that I did not notice
before due to the mess around the variable initialisation, especially
with the forced line-wrapping done by your email editor, is that for
some reason you're sticking backslashes on the ends of every line
setting a variable. Understand that ending a line with a backslash is
done to tell the shell that what is on that line up to the backslash is
not the complete string/command, that it continues on the next line. It
thus has no business being on the end of the variable initialisation
lines in your script. It is necessary for the `lb config` stuff so that
the shell knows that "lb config noauto" is not the complete command,
that it continues on the next line, then the next and so on. Also, the
lines with comments will not work if the leading '#' were removed, you
need to replace the '\' in them with a '#'.

i.e.
_D="Rugged" \

should be
_D="Rugged"

and
#_D="Rugged" \ Debian Derivative Distro e.g. Cobber or Rugged
should be
#_D="Rugged" # Debian Derivative Distro e.g. Cobber or Rugged
if you want to be able to uncomment it and have it work, i.e.:
_D="Rugged" # Debian Derivative Distro e.g. Cobber or Rugged

or better, you'd just avoid repeating the variable initialisation bit
and simplifying as:
# Debian Derivative Distro e.g. Cobber or Rugged
_D="Rugged"

This is extremely basic shell scripting stuff though, and the mistakes
you're making suggest that it might be worthwhile for you to re-read
the manual/book/tutorial/whatever-you-learned-from.

Btw, your variables do not need to all start with an underscore, and
creating variables for "true" and "false" makes things excessively
confusing.

You need to trust me and fix your script, since as pointed out about it
IS broken.

Consider for instance this line that I pointed out mistakes with in the
last email:
--keyring-packages "mx19-archive-keyring" "antix-archive-
keyring"/

1) "mx19-archive-keyring" is the only package that will be used,
"antix-archive-keyring" will be ignored because of the quoting issue.
2) The forward slash (which should be a space then a backslash) will be
taken to be a part of the string argument it follows (the fact that it
is not within the quotes is not a problem in this shell script
language), i.e. is the same as "antix-archive-keyring/", which is
obviously the wrong package name, not that this matters since the
quoting issue means this package name is ignored anyway.
3) Furthermore, the lack of a space-then-backslash on the end of that
line means that the shell thinks that the `lb config` command ends on
the end of that line. The `--linux-flavours` bit on the next line will
be taken by the shell as being the start of an entirely new command,
thus results in an "--linux-flavours: not found" error because "
--linux-flavours" of course is not a name of a real binary. So the
latter third of your config is actually being completely ignored!!!

Please, just trust me and fix the issues I've pointed out.

Lyndon

On Tue, 2020-04-21 at 14:41 +1000, Michael . wrote:
> JNQNFE I think you need to make it clear who you are replying to. At
> one stage I thought you were talking to me so I replied but looking
> back it is not clear if you were talking to me or someone else.
> 
> With regards to my config it "just works" as it is. With regards to
> the keyring slash I noticed it as I pasted it into the email but I
> didn't change it because guess what it still "just works".
> 
> If you know how to build a multi arch system please share exaclty how
> to do it. I ask this because it has been brought up a few times over
> the years and Subhani is the only one that has said he has it
> working.
> 
> Cheers.
> Michael.
> 
> 
> 
> On 21/04/2020, jnqnfe@gmail.com <jnqnfe@gmail.com> wrote:
> > I've not taken a detailed look at this, but some things stand out:
> > 
> > 1) if you're using live-build version 20191219 or newer, add an
> > architecture specifier to the secondary kernel flavour.
> > 
> > -  _FL="686-pae amd64" \
> > +  _FL="686-pae amd64:amd64" \
> > 
> > 2) with bootloaders you're mixing comma AND space separation. this
> > may
> > work, but it might not. i'd air on the side of caution and just use
> > comma separation.
> > 
> > -  --bootloaders "syslinux, grub-efi" \
> > +  --bootloaders "syslinux,grub-efi" \
> > 
> > 3) "config" in --bootappend-live needs to be replaced with
> > "components"
> > (for live-config 4.x+ compatibility)
> > 
> > -   --bootappend-live "boot=live config quiet splash keyboard-
> > layouts=us" \
> > +   --bootappend-live "boot=live components quiet splash keyboard-
> > layouts=us" \
> > 
> > 4) i would not recommand caching the chroot stage
> > 
> > -  --cache-stages "bootstrap chroot" \
> > +  --cache-stages "bootstrap" \
> > 
> > (or just remove it, thus leaving it as default)
> > 
> > 5) --fdisk "fdisk" is an old hack, you should just remove this
> > 
> > 6) --gzip-options "-6 --rsyncable" is the default, so just remove
> > it
> > 
> > 7) bad quoting in --iso-publisher and --iso-volume
> > 
> > -   --iso-publisher ""${_D}" GNU/Linux" \
> > +   --iso-publisher "${_D} GNU/Linux" \
> > 
> > -   --iso-volume ""${_D}" "${_DE}" "${_BIT}" $(date +%Y%m%d)" \
> > +   --iso-volume "${_D} ${_DE} ${_BIT} $(date +%Y%m%d)" \
> > 
> > 8) bad quoting for keyring packages and wrong slash at end of line
> > 
> > -   --keyring-packages "mx19-archive-keyring" "antix-archive-
> > keyring"/
> > +   --keyring-packages "mx19-archive-keyring antix-archive-keyring" 
> > \
> > 
> > 9) --tasksel is an obsolete option, remove it
> > 
> > there are other options using defaults, but no particular needs to
> > remove them.
> > 
> > 
> > On Tue, 2020-04-21 at 12:47 +1000, Michael . wrote:
> > > This is the config I have used when I had the error and
> > > subsequent
> > > failure. It is a modified version of what used to be use by Ozzie
> > > (STAR Linux). This configuration works for everything I have done
> > > until the 2nd flavour as per Subhani's suggestion is added and
> > > that
> > > breaks this configuration.
> > > 
> > > #!/bin/sh
> > > 
> > > set -e
> > > 
> > > #_A="amd64" \ CPU Architecture i386 = 32 BIT (686 and 686-pae)
> > > amd64
> > > =
> > > 64 BIT. It would be great if we could build ARM images for RPi.
> > > _A="i386" \
> > > #_AA="main contrib non-free" \ Archive Areas must have at least
> > > main,
> > > contrib and non-free are optional
> > > _AA="main contrib non-free" \
> > > #_BIT="x86-64" \ x86 = 32 BIT system x86-64 = 64 BIT system
> > > _BIT="x86" \
> > > #_DD="buster" \ Debian Distribution e.g. buster
> > > _DD="buster" \
> > > #_D="Rugged" \ Debian Derivative Distro e.g. Cobber or Rugged
> > > _D="Rugged" \
> > > #_F="false" \ When the answer to a question is false
> > > _F="false" \
> > > #_FL="amd64" \ Linux Flavour as in BIT. Options (currently) are
> > > 686
> > > 686-pae amd64. Flavor MUST match _A="" (CPU Architecture) to work
> > > _FL="686-pae amd64" \
> > > #_MB="http://deb.debian.org/debian/"; \ Repository Mirror to be
> > > used
> > > _MB="http://deb.debian.org/debian/"; \
> > > #_MS="http://security.debian.org/"; \ Security Repository Mirror,
> > > this
> > > is usually always http://security.debian.org/ unless you are
> > > building
> > > a non Debian system
> > > _MS="http://security.debian.org/"; \
> > > #_T="true" \ When the answer to a question is true
> > > _T="true" \
> > > #_DE="builder-choice" \ Desktop Environment or Window Manager
> > > used in
> > > the build. Choices are Cinnamon, Gnome, KDE, LXDE, LXQT, Mate,
> > > TDE
> > > (Trinity), XFCE, etc etc etc.
> > > _DE="Mate" \
> > > #_WM="builder-choice" \ Window Manager used in the build. Choices
> > > are,
> > > currently, Fluxbox.
> > > _WM="Fluxbox" \
> > > 
> > > lb config noauto \
> > > 	--apt "apt" \
> > > 	--apt-indices "${_F}" \
> > > 	--apt-options "--yes" \
> > > 	--apt-recommends "${_F}" \
> > > 	--apt-secure "${_T}" \
> > > 	--apt-source-archives "${_F}" \
> > > 	--aptitude-options "--assume-yes" \
> > > 	--architectures "${_A}" \
> > > 	--archive-areas "${_AA}" \
> > > 	--backports "${_F}" \
> > > 	--binary-filesystem "fat32" \
> > > 	--bootappend-live "boot=live config quiet splash keyboard-
> > > layouts=us" \
> > > 	--bootappend-live-failsafe "boot=live components memtest noapic
> > > noapm
> > > nodma nomce nolapic nomodeset nosmp nosplash vga=normal" \
> > > 	--bootloaders "syslinux, grub-efi" \
> > > 	--cache "${_T}" \
> > > 	--cache-indices "${_F}" \
> > > 	--cache-packages "${_T}" \
> > > 	--cache-stages "bootstrap chroot" \
> > > 	--checksums "md5 sha1 sha256" \
> > >     --chroot-filesystem "squashfs" \
> > >     --debian-installer "live" \
> > > 	--distribution "${_DD}" \
> > > 	--debian-installer-distribution "${_DD}" \
> > > 	--debian-installer-gui "${_T}" \
> > > 	--fdisk "fdisk" \
> > > 	--firmware-binary "${_F}" \
> > > 	--firmware-chroot "${_F}" \
> > > 	--gzip-options "-6 --rsyncable" \
> > > 	--hdd-label ""${_D}"Live" \
> > > 	--initramfs "live-boot" \
> > > 	--initramfs-compression "gzip" \
> > > 	--iso-application ""${_D}"-Live-"${_DE}"" \
> > > 	--iso-preparer "Michael" \
> > > 	--iso-publisher ""${_D}" GNU/Linux" \
> > > 	--iso-volume ""${_D}" "${_DE}" "${_BIT}" $(date +%Y%m%d)" \
> > >     --keyring-packages "mx19-archive-keyring" "antix-archive-
> > > keyring"/
> > > 	--linux-flavours "${_FL}" \
> > >     --linux-packages "linux-image" \
> > > 	--mirror-binary "${_MB}" \
> > > 	--mirror-bootstrap "${_MB}" \
> > > 	--mirror-debian-installer "${_MB}" \
> > > 	--parent-mirror-binary-security "${_MS}" \
> > > 	--parent-mirror-bootstrap "${_MB}" \
> > > 	--parent-mirror-chroot-security "${_MS}" \
> > > 	--quiet "${_F}" \
> > > 	--security "${_T}" \
> > > 	--source "${_F}" \
> > > 	--source-images "${_F}" \
> > > 	--system "live" \
> > > 	--tasksel "apt" \
> > > 	--uefi-secure-boot "enable" \
> > > 	--updates "${_T}" \
> > > 	--verbose "${_T}" \
> > > 	--win32-loader "${_F}" \
> > > 	--zsync "${_T}" \
> > > "${@}"
> > > 
> > > After reading Subhani's latest reply, and Subhani please don't
> > > call
> > > anything MichaelOS, there is much more to it that just a regular
> > > live
> > > build.
> > > Cheers.
> > > Michael.
> > > 
> > > On 21/04/2020, jnqnfe@gmail.com <jnqnfe@gmail.com> wrote:
> > > > I've not really been following this thread, I've just
> > > > previously
> > > > given
> > > > it a brief read.
> > > > 
> > > > I noticed this in the example config given:
> > > > 
> > > > > --linux-flavours 686 amd64
> > > > 
> > > > This is obviously wrong, there needs to be quoting here! Could
> > > > that
> > > > be
> > > > the source of your problem?
> > > > 
> > > > Also, I'm not entirely familiar with using the multi-flavour
> > > > feature,
> > > > but I recall a mention of "686 amd64:amd64".
> > > > 
> > > > On Tue, 2020-04-21 at 10:07 +1000, Michael . wrote:
> > > > > I just tried Subhani's tip and it failed with
> > > > > E: Error reading the CPU table
> > > > > P: Begin unmounting filesystems...
> > > > > P: Saving caches...
> > > > > E: Error reading the CPU table
> > > > > 
> > > > > ...
> > > > > 
> > > > > > > > > > > lb config --binary-images iso-hybrid --mode
> > > > > > > > > > > debian
> > > > > > > > --architectures i386
> > > > > > > > > > > --linux-flavours 686 amd64 --distribution buster
> > > > > > > > > > > --
> > > > > > > > > > > archive-areas
> > > > > > > > "main
> > > > > > > > > > > contrib non-free" --updates true --security true
> > > > > > > > > > > --
> > > > > > > > > > > cache
> > > > > > > > > > > true
> > > > > > > > > > > --apt-recommends true --debian-installer live
> > > > > > > > --debian-installer-gui
> > > > > > > > > > > true
> > > > > > > > > > > --win32-loader true --iso-application PakOS10.3
> > > > > > > > > > > --
> > > > > > > > > > > iso-
> > > > > > > > > > > preparer
> > > > > > > > > > > subhaniminhas-
> > > > > > > > > > > https://sourceforge.net/projects/pakos/
> > > > > > > > --iso-publisher
> > > > > > > > > > > subhaniminhas-
> > > > > > > > > > > https://sourceforge.net/projects/pakos/
> > > > > > > > --iso-volume
> > > > > > > > > > > PakOS10.3