Bug#1029962: check-missing-firmware fails to find firmware file or package on partitioned removable media
Control: tags -1 patch
On 29/01/2023 at 17:27, I wrote:
Observed behaviour:
- When looking for a loose firmware file, check-missing-firmware calls
"mountmedia" which returns after mounting the first device which can be
mounted.
- When looking for a firmware package, check-missing-firmware calls
"mountmedia driver" which returns after mounting the first device which
can be mounted and contains {,/firmware}/*.{deb,udeb,ude} files.
mountmedia tries to mount whole drives first, then partitions, in
alphanumerical order.
In either case, the lookup fails if the mounted device is not the one
which contains the requested firmware, next devices are not tried.
IMO the correct behaviour is:
for each drive or partition
if device not mounted
try to mount device
if mount succeeds
look for missing loose firmware files
look for missing firmware in deb package files
But I do not see how this can be achieved by still using mountmedia...
Please find attached a patch for mountmedia 0.27 and two patches for
hw-detect 1.155 implementing a possible solution.
The first one adds subcommands into mountmedia to list and mount any media.
The second one uses the subcommands in check-missing-firmware to
implement the above desired behaviour.
The third one is an optimization to reduce the search time: it updates
the missing file list after installing a loose file or package and stops
scanning remaining media and packages if the missing file list is empty.
I chose to add subcommands to mountmedia in order to reuse existing code
and keep media management in one place (mountmedia) instead of
duplicating some code in hw-detect.
Any feedback (comments and tests) appreciated.
From 2d6463e03e4d388b6b3893b2761fea8aa1921098 Mon Sep 17 00:00:00 2001
From: Pascal Hambourg <pascal@plouf.fr.eu.org>
Date: Sat, 1 Apr 2023 00:07:47 +0200
Subject: [PATCH 1/2] check-missing-firmware: improve firmware file lookup
The current algorithm mounts only one device or partition. If it
mounts the wrong one, firmware files are not found.
This patch uses new subcommands in mountmedia in order to search
firmware files or packages on all available disks and partitions.
---
check-missing-firmware.sh | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/check-missing-firmware.sh b/check-missing-firmware.sh
index 0c71c243..041e7e17 100755
--- a/check-missing-firmware.sh
+++ b/check-missing-firmware.sh
@@ -456,19 +456,22 @@ while check_missing && ask_load_firmware; do
# Whether we should keep both mountmedia calls, and whether mountmedia
# is doing a good job is discussed in #1029543:
if [ "$loop" -gt 1 ]; then
- # second, look for loose firmware files on the media device.
- if mountmedia; then
- for file in $files; do
- try_copy "$file"
- done
- umount /media || true
- fi
-
- # last, look for firmware debs on the media device
- if mountmedia driver; then
- check_for_firmware /media /media/firmware
- umount /media || true
+ devices="$(mountmedia list)"
+ if mountmedia mounted; then
+ # use currently mounted device first
+ devices="mounted $devices"
fi
+ for dev in $devices; do
+ if [ $dev = mounted ] || mountmedia mount $dev; then
+ # second, look for loose firmware files on the media device.
+ for file in $files; do
+ try_copy "$file"
+ done
+ # last, look for firmware debs on the media device
+ check_for_firmware /media /media/firmware
+ umount /media || true
+ fi
+ done
fi
# remove and reload modules so they see the new firmware
--
2.30.2
From 7c931fbb82b27e9d288a75e66198c24e06e33c08 Mon Sep 17 00:00:00 2001
From: Pascal Hambourg <pascal@plouf.fr.eu.org>
Date: Sat, 1 Apr 2023 14:07:10 +0200
Subject: [PATCH 2/2] check-missing-firmware: stop scanning if all missing
files are present
Update the list of missing firmware files after installing loose files
or packages, and stop scanning media and packages if the list is empty.
---
check-missing-firmware.sh | 33 +++++++++++++++++++++++++++++++--
1 file changed, 31 insertions(+), 2 deletions(-)
diff --git a/check-missing-firmware.sh b/check-missing-firmware.sh
index 041e7e17..a5be8d3b 100755
--- a/check-missing-firmware.sh
+++ b/check-missing-firmware.sh
@@ -385,6 +385,19 @@ install_firmware_pkg () {
fi
}
+# update missing firmware files list after installing package or loose files
+update_missing_firmware_files () {
+ local new_files file
+
+ new_files=
+ for file in $files; do
+ if [ ! -f /lib/firmware/$file ]; then
+ new_files="${new_files:+$new_files }$file"
+ fi
+ files="$new_files"
+ done
+}
+
# Try to load debs that contain the missing firmware.
# This does not use anna because debs can have arbitrary
# dependencies, which anna might try to install.
@@ -403,6 +416,11 @@ check_for_firmware() {
if check_deb_arch "$dir/$fw_pkg_file"; then
log "installing firmware package $dir/$fw_pkg_file ($component)"
install_firmware_pkg "$dir/$fw_pkg_file" "$component" || true
+ update_missing_firmware_files
+ if [ -z "$files" ]; then
+ break 2
+ fi
+ echo "$files" | sed -e 's/ /\n/g' >/tmp/grepfor
echo "$fw_pkg_file" >> /tmp/pkginstalled
fi
done
@@ -416,6 +434,11 @@ check_for_firmware() {
if check_deb_arch "$filename" && list_deb_firmware "$filename" | grep -qf /tmp/grepfor; then
log "installing firmware package $filename"
install_firmware_pkg "$filename" $(get_deb_component "$filename") || true
+ update_missing_firmware_files
+ if [ -z "$files" ]; then
+ break 2
+ fi
+ echo "$files" | sed -e 's/ /\n/g' >/tmp/grepfor
fi
fi
done
@@ -467,9 +490,15 @@ while check_missing && ask_load_firmware; do
for file in $files; do
try_copy "$file"
done
- # last, look for firmware debs on the media device
- check_for_firmware /media /media/firmware
+ update_missing_firmware_files
+ if [ -n "$files" ]; then
+ # last, look for firmware debs on the media device
+ check_for_firmware /media /media/firmware
+ fi
umount /media || true
+ if [ -z "$files" ]; then
+ break
+ fi
fi
done
fi
--
2.30.2
From c2fed7a85d1495ee98b7a6ac53b9c9afa2f0c932 Mon Sep 17 00:00:00 2001
From: Pascal Hambourg <pascal@plouf.fr.eu.org>
Date: Fri, 31 Mar 2023 19:45:28 +0200
Subject: [PATCH] mountmedia: add subcommands to improve loose firmware file
search
The current algorithm mounts only the first "suitable" disk or partition
when searching for loose firmware files or packages.
If it mounts the wrong devices, firmware files are not found.
This patch adds the following subcommands to mountmedia:
mountmedia list [floppy]: list devices
mountmedia mounted: test if /media is mounted
mountmedia mount <device|dir>: mount device (ro) or directory on /media
The behaviour when no subcommand is passed is unchanged.
The "mount" subcommand loads the ext4 module to be able to mount media
with ext* filesystems in addition to vfat. It may be useful in cases
such as bug#1033498 when some loose firmware files are symlinks, which
are not supported by vfat. However it may increase the search time if
there are many ext* partitions.
Additionally, the "list" subcommand adds /hd-media to the device list
if it exists for compatibility with the current behaviour.
These subcommands are intended to be used by check-missing-firmware
from package hw-media in order to search firmware files or packages
on all available media.
---
mountmedia | 55 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 53 insertions(+), 2 deletions(-)
diff --git a/mountmedia b/mountmedia
index 6013579..7181ac5 100755
--- a/mountmedia
+++ b/mountmedia
@@ -1,6 +1,18 @@
#!/bin/sh
set -e
+# parse sub-commands
+if [ "$1" = list ]; then
+ CMD_LIST=1
+ # shift to parse optional parameter "floppy"
+ shift
+elif [ "$1" = mounted ]; then
+ CMD_MOUNTED=1
+elif [ "$1" = mount ]; then
+ CMD_MOUNT=1
+ dev="$2"
+fi
+
if [ "$1" = driver ]; then
WANTDRIVER=1
elif [ "$1" = floppy ]; then
@@ -59,10 +71,49 @@ checkcontents() {
fi
}
+# execute sub-commands
+
+if [ "$CMD_LIST" ]; then
+ if [ -d /hd-media ]; then
+ # for compatibility with default behaviour
+ echo /hd-media
+ fi
+ devlist
+ exit 0
+fi
+
+if [ "$CMD_MOUNTED" ]; then
+ media_mounted
+ exit
+fi
+
+if [ "$CMD_MOUNT" ]; then
+ if [ -d "$dev" ]; then # directory -> bind mount
+ mount --bind $dev $MNT
+ exit
+ elif [ -b "$dev" ]; then # block device -> normal mount
+ modprobe -q vfat || true
+ # allow to mount ext2/3/4 too, may be useful in some cases (see #1033498)
+ modprobe -q ext4 || true
+ if [ $dev = /dev/fd0 ]; then
+ # for compatibility with default behaviour
+ # useless if floppy support is dropped
+ log-output -t mountmedia modprobe -q floppy || true
+ log-output -t mountmedia modprobe -q ide-floppy || true
+ update-dev --settle
+ fi
+ mount -r -t auto $dev $MNT
+ exit
+ fi
+ exit 1
+fi
+
+# default operation when no subcommand is passed
+
if ! ( media_mounted && checkcontents $MNT ); then
# Special case for an already mounted /hd-media.
if [ -d /hd-media ] && checkcontents /hd-media; then
- mount --bind /hd-media /media
+ mount --bind /hd-media $MNT
exit 0
fi
@@ -79,7 +130,7 @@ if ! ( media_mounted && checkcontents $MNT ); then
fi
done
- if [ "$i" = 1 ]; then
+ if [ "$i" = 1 ]; then
# Give USB time to settle, make sure all devices are
# seen next time though.
sleep 5
--
2.30.2
Reply to: