[Fwd: Re: isorecorder]
-------- Forwarded Message --------
From: Lou Poppler <LouPoppler@cableone.net>
To: Pete Batard <pete@akeo.ie>
Subject: Re: isorecorder
Date: Sun, 10 Jan 2021 18:20:55 -0700
On Sun, 2021-01-10 at 23:57 +0000, Pete Batard wrote:
> On 2021.01.10 22:23, Lou Poppler wrote:
> > Let me emphasize that the unmodified debian installer images, when copied to a
> > USB stick via (unix) dd or cp, boot just fine in both UEFI mode and BIOS mode.
> > They also work just fine when copied unmodified to a CD or DVD.
> >
>
> I'm afraid I have to disagree with this, from direct empirical evidence.
>
> This is a test I conducted a few moments ago, using the latest AMD64
> netinst testing image (dated 2021.01.09). I didn't use Rufus to create
> the bootable media but instead did it on Linux (up to date Armbian on an
> RK3399 platform) with the exact set of commands below:
You download a testing .iso
then you make a fresh MS vfat partition on sda1
then you rsync [some of] the .iso onto this partition.
Then you have problems booting from this USB.
-----
This is radically different from what I suggested above.
Here is my experiment:
lwp@william:~/Downloads$ wget https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
lwp@william:~/Downloads$ umount /dev/sdb1
lwp@william:~/Downloads$ sudo bash
root@william:/home/lwp/Downloads# cp debian-testing-amd64-netinst.iso /dev/sdb
root@william:/home/lwp/Downloads# sync
At this point I moved the USB stick to a Toshiba amd64 laptop,
and booted with no problems into the installer.
Notice especially that I did *not* copy the installer .iso onto a partition of the USB stick,
I copied the image to the bare USB i.e. /dev/sdb _NOT_ /dev/sdb1
Here is what that USB stick looks like now:
root@william:/home/lwp/Downloads# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.29.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): p
Disk /dev/sdb: 14.3 GiB, 15376318464 bytes, 30031872 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5c546723
Device Boot Start End Sectors Size Id Type
/dev/sdb1 * 0 745471 745472 364M 0 Empty
/dev/sdb2 3944 9863 5920 2.9M ef EFI (FAT-12/16/32)
Command (m for help): q
root@william:/home/lwp/Downloads#
[below is the remainder of your procedure]:
> ---------------------------------------------------------------------
>
> root@nano:/mnt/ssd# wget
> https://cdimage.debian.org/cdimage/weekly-builds/amd64/iso-cd/debian-testing-amd64-netinst.iso
> root@nano:/mnt/ssd# mount -o loop debian-testing-amd64-netinst.iso /mnt/iso
> mount: /mnt/iso: WARNING: device write-protected, mounted read-only.
> root@nano:/mnt/ssd# dd if=/dev/zero of=/dev/sda bs=512 count=34
> 34+0 records in
> 34+0 records out
> 17408 bytes (17 kB, 17 KiB) copied, 0.0113974 s, 1.5 MB/s
> root@nano:/mnt/ssd# dd if=/dev/zero of=/dev/sda bs=512 count=34
> seek=$((`blockdev --getsz /dev/sda` - 34))
> 34+0 records in
> 34+0 records out
> 17408 bytes (17 kB, 17 KiB) copied, 0.0144371 s, 1.2 MB/s
> root@nano:/mnt/ssd# gdisk /dev/sda
> GPT fdisk (gdisk) version 1.0.3
>
> Partition table scan:
> MBR: not present
> BSD: not present
> APM: not present
> GPT: not present
>
> Creating new GPT entries.
>
> Command (? for help): n
> Partition number (1-128, default 1): 1
> First sector (34-234441614, default = 2048) or {+-}size{KMGTP}:
> Last sector (2048-234441614, default = 234441614) or {+-}size{KMGTP}:
> Current type is 'Linux filesystem'
> Hex code or GUID (L to show codes, Enter = 8300): 0700
> Changed type of partition to 'Microsoft basic data'
>
> Command (? for help): w
>
> Final checks complete. About to write GPT data. THIS WILL OVERWRITE
> EXISTING PARTITIONS!!
>
> Do you want to proceed? (Y/N): Y
> OK; writing new GUID partition table (GPT) to /dev/sda.
> The operation has completed successfully.
> root@nano:/mnt/ssd# mkfs.vfat /dev/sda1
> mkfs.fat 4.1 (2017-01-24)
> root@nano:/mnt/ssd# mount /dev/sda1 /mnt/hd
> root@nano:/mnt/ssd# rsync -av /mnt/iso/ /mnt/hd
> sending incremental file list
> rsync: symlink "/mnt/hd/debian" -> "." failed: Operation not permitted (1)
> ./
> README.html
> README.mirrors.html
> README.mirrors.txt
> README.source
> README.txt
> autorun.inf
> g2ldr
> g2ldr.mbr
> md5sum.txt
> setup.exe
> win32-loader.ini
> rsync: symlink "/mnt/hd/dists/testing" -> "bullseye" failed: Operation
> not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/basic-defs.html" ->
> "basic-defs.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/choosing.html" ->
> "choosing.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/compatibility.html" ->
> "compatibility.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/contributing.html" ->
> "contributing.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/customizing.html" ->
> "customizing.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/faqinfo.html" -> "faqinfo.en.html"
> failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/ftparchives.html" ->
> "ftparchives.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/getting-debian.html" ->
> "getting-debian.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/index.html" -> "index.en.html"
> failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/kernel.html" -> "kernel.en.html"
> failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/nextrelease.html" ->
> "nextrelease.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/pkg-basics.html" ->
> "pkg-basics.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/pkgtools.html" ->
> "pkgtools.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/redistributing.html" ->
> "redistributing.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/software.html" ->
> "software.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/support.html" -> "support.en.html"
> failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/doc/FAQ/html/uptodate.html" ->
> "uptodate.en.html" failed: Operation not permitted (1)
> rsync: symlink "/mnt/hd/firmware/firmware-linux-free_20200122-1_all.deb"
> -> "../pool/main/f/firmware-free/firmware-linux-free_20200122-1_all.deb"
> failed: Operation not permitted (1)
> .disk/
> .disk/base_components
> .disk/base_installable
> .disk/cd_type
> .disk/info
> .disk/mkisofs
> .disk/udeb_include
> EFI/
> EFI/boot/
> EFI/boot/bootx64.efi
>
> (...)
>
> pool/main/x/xauth/xauth_1.0.10-1_amd64.deb
> pool/main/x/xfsprogs/
> pool/main/x/xfsprogs/xfsprogs-udeb_5.6.0-1+b2_amd64.udeb
> pool/main/x/xfsprogs/xfsprogs_5.6.0-1+b2_amd64.deb
> pool/main/x/xkeyboard-config/
> pool/main/x/xkeyboard-config/xkb-data_2.29-2_all.deb
> pool/main/x/xserver-xorg-input-libinput/
> pool/main/x/xserver-xorg-input-libinput/xserver-xorg-input-libinput-udeb_0.30.0-1_amd64.udeb
> pool/main/x/xxhash/
> pool/main/x/xxhash/libxxhash0_0.8.0-1_amd64.deb
> pool/main/x/xz-utils/
> pool/main/x/xz-utils/liblzma5_5.2.5-1.0_amd64.deb
> pool/main/x/xz-utils/xz-utils_5.2.5-1.0_amd64.deb
> pool/main/z/
> pool/main/z/zlib/
> pool/main/z/zlib/zlib1g-udeb_1.2.11.dfsg-2_amd64.udeb
> pool/main/z/zlib/zlib1g_1.2.11.dfsg-2_amd64.deb
> tools/
> tools/loadlin.exe
> tools/loadlin.txt
>
> sent 434,994,400 bytes received 26,184 bytes 17,755,942.20 bytes/sec
> total size is 434,798,648 speedup is 1.00
> rsync error: some files/attrs were not transferred (see previous errors)
> (code 23) at main.c(1207) [sender=3.1.3]
> root@nano:/mnt/ssd# umount /mnt/hd
> root@nano:/mnt/ssd#
>
> ---------------------------------------------------------------------
>
> Then when booting the media on an intel NUC x86_64 PC (in UEFI mode) and
> after selecting the language, you *will* run into the following error:
>
> ---------------------------------------------------------------------
>
> [!!] Detect and mount installation media
>
> No device for installation media was detected.
>
> You may need to load additional drivers from removable media, such as
> a driver floppy or a USB stick. If you have these available now,
> insert the media, and continue. Otherwise, you will be given the
> option to manually select some modules.
>
> Load drivers from removable media?
>
> <Yes> <No>
>
> ---------------------------------------------------------------------
>
> Again, this is something I have been able to replicate using the latest,
> using all the steps highlighted above just a few minutes ago.
>
>
> Now, your report that you are not seeing this kind of issue (which we
> have additional confirmation of, at least for Raspberry Pi 4 Debian
> installation) made me dig a little further as to whether this issue is
> as universal as I initially was led to believe, or if there might be an
> environmental component to it. And after further testing, I have found
> that this seems to only affect UAS devices. For instance, I can validate
> that the following UAS device (Mushkin Ventura Ultra USB 3.0 120 GB)
> always produce the installation media detection issue:
>
> ---------------------------------------------------------------------
>
> root@nano:/mnt/ssd# dmesg
> (...)
> [549114.637578] usb 8-1: new SuperSpeed Gen 1 USB device number 19 using
> xhci-hcd
> [549114.658815] usb 8-1: New USB device found, idVendor=174c,
> idProduct=55aa, bcdDevice= 1.00
> [549114.658837] usb 8-1: New USB device strings: Mfr=2, Product=3,
> SerialNumber=1
> [549114.658852] usb 8-1: Product: Ventura Ultra
> [549114.658867] usb 8-1: Manufacturer: Mushkin
> [549114.658881] usb 8-1: SerialNumber: 000000000025
> [549114.666050] scsi host0: uas
> [549114.668168] scsi 0:0:0:0: Direct-Access Mushkin Ventura Ultra
> 0 PQ: 0 ANSI: 6
> [549114.670930] sd 0:0:0:0: Attached scsi generic sg0 type 0
> [549145.389481] sd 0:0:0:0: tag#14 uas_eh_abort_handler 0 uas-tag 1
> inflight: CMD IN
> [549145.389487] sd 0:0:0:0: tag#14 CDB: opcode=0x12 12 01 00 00 40 00
> [549145.409499] scsi host0: uas_eh_device_reset_handler start
> [549145.537655] usb 8-1: reset SuperSpeed Gen 1 USB device number 19
> using xhci-hcd
> [549145.559210] scsi host0: uas_eh_device_reset_handler success
> [549145.565782] sd 0:0:0:0: [sda] 234441648 512-byte logical blocks:
> (120 GB/112 GiB)
> [549145.565871] sd 0:0:0:0: [sda] Write Protect is off
> [549145.565874] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
> [549145.566029] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
> enabled, doesn't support DPO or FUA
> [549145.566228] sd 0:0:0:0: [sda] Optimal transfer size 33553920 bytes
> [549145.608066] sda: sda1 sda2 sda3
> [549145.610265] sd 0:0:0:0: [sda] Attached SCSI disk
>
> ---------------------------------------------------------------------
>
> However, when testing with another UAS USB device I have (a SanDisk
> Extreme 3.0 32 GB), the media detection script appears to behave as
> expected.
>
> So at least, this gives us a better idea of where the issue is likely to
> lie (most likely the current media detection scripts treat UAS USB
> devices differently from non UAS ones).
>
> But if you do get your hands on a UAS device, you should find that the
> statement that "unmodified debian installer images, when copied to a USB
> stick via (unix) dd or cp, boot just fine in UEFI mode" is not entirely
> accurate...
>
> Regards,
>
> /Pete
Reply to: