Re: initrd sizes mushroomed several months ago
On Fri 03 Feb 2023 at 01:45:33 (-0500), Felix Miata wrote:
> David Wright composed on 2023-02-01 22:39 (UTC-0600):
> > On Wed 01 Feb 2023 at 00:50:07 (-0500), Felix Miata wrote:
>
> >> I think 6.0's is smaller because that upgrade cycle is when I installed zstd
> >> before the newer kernel. It's specified by default in initramfs.conf, but the
> >> upgrades from 11 to 12 here have only been including libzstd1, which apparently
> >> is not used for initrd construction.
>
> > That's relatively easy to confirm with something like:
>
> > $ dd if=/boot/initrd.img-6.0.0-6-amd64 skip=NNNN | file -
>
> > where NNNN is given by the last line from:
>
> > $ cpio -t < /boot/initrd.img-6.0.0-6-amd64
>
> Is that a typo? I copied & pasted that and the screen loaded binary gibberish.
I just didn't allow for there to be no 'early' cpio archive containing
microcode corrections for the processor, which most people seem to have.
> > I haven't seen any response from you to The Wanderer's post about
> > the initramfs-tools upgrade. (Note that there are two packages,
> > including the -core.) That's why, having replied to that post and
> > yours, I didn't post any of my further researches, including
> > those I've repeated and reported here.
>
> At that time, I found no apparent reason to reply to that post.
OK, I had thought that comparing the output from lsinitramfs from
older and newer initrds would have shown up what is now revealed
as all this extra firmware.
> > You could check your /var/log/apt/history* files on both your
> > systems and see if the initramfs-tools were upgraded between
> > 12 Sep and 2 Nov (amd64) and 8 May and 2 Aug (686).
>
> All I know is all 19 installations are currently on 0.142. That old history
> is history.
Yes, sorry. I've made a habit of setting rotate -1 for the
log rotation of apt. Together with /var/log/installer/, those
logs for a valuable archive of the machine's history without
expending any effort. Others might not.
> #
> # du -sh /usr/lib/firmware/amdgpu
> 64M /usr/lib/firmware/amdgpu
> # du -sh /usr/lib/firmware/radeon
> 7.1M /usr/lib/firmware/radeon
> # ls -1 /usr/lib/firmware/amdgpu | wc -l
> 490
> # ls -1 /usr/lib/firmware/radeon | wc -l
> 247
>
I don't know what these sizes are to do with the initrd. They just
look like files installed into the rootfs for the OS. On bullseye,
I get 41M, 7.4M, 381, 247, but nothing in my initrds.
> > FTR, I reinstalled 5.10.0-21-amd64 on another machine with MODULES=dep
> > and for comparison (initrd only):
>
> > $ unmkinitramfs /boot/initrd.img-5.10.0-21-amd64 /tmp/unpacked10-21
> > cpio: etc/console-setup/null: Cannot mknod: Operation not permitted
> > $ du -sh /tmp/unpacked10-21/main/lib/modules/5.10.0-21-amd64/kernel/ # modules
> > 5.6M /tmp/unpacked10-21/main/lib/modules/5.10.0-21-amd64/kernel/
> > $ du -sh /tmp/unpacked10-21/main/lib/firmware
> > du: cannot access '/tmp/unpacked10-21/main/lib/firmware': No such file or directory
> > $
>
> > … so zero firmware is required to boot this machine (and the
> > same is true for my production machine).
>
> That's how it is here on PCs with Intel or NVidia graphics, but apparently
> not so AMD/ATI, going by what's now being packed into initrds.
I don't disagree, but haven't seen the evidence here.
> > If you find more modules being included in the newer /initrd/ file
> > along with the 654 extra firmware files, then my suspicion would
> > fall on the "Support network boot via USB Ethernet adapters" line
> > in the new version's changelog, rather than changes in compression.
> > What sort of modules are involved would obviously lend support to
> > or dispel that suspicion.
>
> # lsinitramfs -l /boot/initrd.img-5.17.0-1-amd64 | grep net
> 2 root root 0 May 15 2022 usr/lib/systemd/network
> 1 root root 44 Mar 14 2022 usr/lib/systemd/network/73-usb-net-by-mac.link
> 1 root root 499 Mar 11 2022 usr/lib/systemd/network/99-default.link
> 1 root root 796 May 7 2018 usr/lib/udev/rules.d/70-persistent-net.rules
> 1 root root 969 Mar 14 2022 usr/lib/udev/rules.d/73-special-net-names.rules
> 1 root root 452 Mar 11 2022 usr/lib/udev/rules.d/75-net-description.rules
> 1 root root 295 Mar 11 2022 usr/lib/udev/rules.d/80-net-setup-link.rules
> 247 root root 0 May 1 2022 usr/bin/telnet
> 247 root root 0 May 1 2022 usr/bin/netstat
> # lsinitramfs -l /boot/initrd.img-5.18.0-1-amd64 | grep net
> 2 root root 0 Jun 20 2022 usr/lib/systemd/network
> 1 root root 44 Jun 10 2022 usr/lib/systemd/network/73-usb-net-by-mac.link
> 1 root root 789 Jun 2 2022 usr/lib/systemd/network/99-default.link
> 1 root root 796 May 7 2018 usr/lib/udev/rules.d/70-persistent-net.rules
> 1 root root 969 Jun 10 2022 usr/lib/udev/rules.d/73-special-net-names.rules
> 1 root root 452 Jun 2 2022 usr/lib/udev/rules.d/75-net-description.rules
> 1 root root 295 Jun 2 2022 usr/lib/udev/rules.d/80-net-setup-link.rules
> 253 root root 0 Jun 6 2022 usr/bin/telnet
> 253 root root 0 Jun 6 2022 usr/bin/netstat
"net" is not a string I'd use. As you can see, you haven't got any
matches for "firmware" here, because …/firmware/ has but a single
directory level beneath it, with few exceptions, and those directory
names are mainly a mixture of manufacturers and devices, as in
{3com,advansys,cis,cxgb3,cxgb4,e100,ene-ub6250,intel,isci,rtl_nic,tehuti,tigon}
> I collected some data on all 19 64 bit Bookworms here:
> https://paste.debian.net/hidden/16afc442/
> https://paste.debian.net/plainh/16afc442
?
> It consists of three groups, each sorted by either PC, date or size. From it
> jumped out the main increase is limited to AMD/ATI graphics installations,
> sometime between 14 May and 21 June:
>
> Size Date Version Host LinesModsz Graphics/extras
> # ls -Gg initrd*4 # gx78b PCIe aHD6450 0-nvme-ssd 1-ata-ssd
> 7649297 May 15 2022 initrd.img-5.17.0-1-amd64 gx78b 445 434M Gati
> 34289103 Jun 20 2022 initrd.img-5.18.0-1-amd64 gx78b 1132 459M Gati
Yes, I can see the massive increase.
> # lsinitramfs -l /boot/initrd.img-5.17.0-1-amd64 | grep busy
> -rwxr-xr-x 247 root root 0 May 1 2022 usr/bin/busybox
> # lsinitramfs -l /boot/initrd.img-5.18.0-1-amd64 | grep busy
> -rwxr-xr-x 253 root root 0 Jun 6 2022 usr/bin/busybox
I'm not sure what that's about. We'd already decided that there
weren't ~250 binaries causing the bloat.
> Neither initrd contains the string radeon, nor string dgpu.
I don't know why they should or shouldn't.
> 5.18 contains 1132 lines of lsinitramfs output vs 445 in 5.17: +687. There's
> nothing older than October in /var/log/ on that installation, except in
> installer's 2018. 5.17 has one firmware line:
>
> usr/lib/udev/rules.d/50-firmware.rules
>
> # dpkg-query -S /usr/lib/udev/rules.d/50-firmware.rules
Yes, that matches my experience with MODULES=dep, reported earlier, viz:
$ du -sh /tmp/unpacked10-21/main/lib/firmware
du: cannot access '/tmp/unpacked10-21/main/lib/firmware': No such file or directory
$
> dpkg-query: no path found matching pattern /usr/lib/udev/rules.d/50-firmware.rules
> # cat /usr/lib/udev/rules.d/50-firmware.rules
> # stub for immediately telling the kernel that userspace firmware loading
> # failed; necessary to avoid long timeouts with CONFIG_FW_LOADER_USER_HELPER=y
> SUBSYSTEM=="firmware", ACTION=="add", ATTR{loading}="-1"
>
> I have no idea what the rule above does, or if it's something that changed or
> was new last spring. I don't really understand any of udev for that matter.
I think you were caught out by UsrMerge:
$ apt-file find 50-firmware
udev: /lib/udev/rules.d/50-firmware.rules
$
I think this rule is designed to do nothing when the firmware loading
attribute returns -1.
* 1: Start a load, discarding any previous partial load.
* 0: Conclude the load and hand the data to the driver code.
* -1: Conclude the load with an error and discard any written data.
Presumably it doesn't want to trigger otherwise, as there
may be more loading requests pending.
(See comments in drivers/base/firmware_loader/fallback.c)
> 5.18 has 655, so 33-34 that are not firmware. I messed up somehow accounting
> for most of the others:
> # diff -u1 ini517-0fw-0mods.txt ini518-0fw-0mods.txt
> --- ini517-0fw-0mods.txt 2023-02-02 22:54:09.000000000 -0500
> +++ ini518-0fw-0mods.txt 2023-02-02 22:54:17.000000000 -0500
> @@ -165,2 +165,3 @@
> usr/sbin/mkdosfs
> +usr/sbin/mim
> usr/sbin/mdev
> @@ -179,2 +180,3 @@
> usr/sbin/ifconfig
> +usr/sbin/i2ctransfer
> usr/sbin/i2cset
> @@ -229,2 +231,3 @@
> usr/bin/tty
> +usr/bin/ts
> usr/bin/truncate
> @@ -364,2 +367,3 @@
> usr/bin/cttyhack
> +usr/bin/crc32
> usr/bin/cpio
> @@ -381,4 +385,6 @@
> usr/bin/basename
> +usr/bin/base64
> usr/bin/awk
> usr/bin/ash
> +usr/bin/ascii
> usr/bin/arch
> @@ -388,2 +394,2 @@
> usr/sbin/watchdog
> -1
> +655
I don't know what this is meant to show. All I see is a few binaries
(or busybox hardlinks).
> Main thing now is, why put 100% of amd graphics firmware in existence
> in the initrd (beginning last spring)? Or any? And why only for AMD/ATI?
> The only mwar string in Intel graphics initrds is that very same
> /usr/lib/udev/rules.d/50-firmware.rules.
No idea why. I've just turned on two desktops that I haven't used
in a while. Both these do require video firmware from their initrds,
one Intel i915 (24 bins), and the other Radeon Redwood/Cypress
(232 bins). But in both cases, only the _one_ relevant directory from
the firmware packages is included in the initrd: i915 from
firmware-misc-nonfree, and radeon from firmware-amd-graphics.
But I don't yet know which directories are being included in yours.
Cheers,
David.
Reply to: