Hi,on the wishlist page for trixie I added the following entry, which rightly was deleted (since is actually for forky):
```Create a list of scenarios for the installer that contain the paths through the installer that must be followed. That could then become part of the extended test suite, so we will know in advance (way before release time/freeze time) if some feature gets broken.
``` Let me elaborate a bit on this:First, I'm aware that testing on openQA (which uses QEMU) will not find issues related to hardware support, but it will find issues due to the logic in the installer.
Second, the amount of possible paths though the installer is enormous. Doing (and finding) all of them is unfeasible, so the focus should be on the major paths.
The purpose of this proposal is to make it easier to see where and when the installation breaks (for whatever reason). It will allow for shorter freeze periods (given that many things have been tested before) and easier decision making about stuff that goes in or out.
That given, there already exist several tests for d-i (amd64):* Install with locale en_GB and British keyboard, default priority, empty initial HD, 'Guided - use entire disk', 'All in one partition' (times) each of the 8 desktop environments + no DE selected (times) UEFI (without Secure Boot) or BIOS boot
* The above for GNOME with additionally (times) graphics or text installer* Select each of the languages in the graphical installer (no automated evaluation of missing/wrong glyphs yet)
* Enter rescue mode (based on an older XFCE installation)* Select all entries in the isolinux/GRUB boot menu and proceed until the first question * As the first test, with 'Guided - use entire disk and set up encrypted LVM' instead (this test is not running regularly)
* After installation, check whether the machine still boots* For some DEs: check whether the default applications are able to start and stop
For arm64, a subset of the above is running. No other architectures are being tested in openQA at the moment.
For the live images similar tests related to d-i are running (however, the live installer variant copies all files from the live image instead of installing from scratch).
From my point of view, we are missing the following scenarios with the default debconf priority: * Co-installation with a pre-existing Debian installation (e.g. trixie next to bookworm) * Co-installation with a pre-existing Microsoft Windows (a typical user migration scheme, which will also support the EndOf10 use cases https://endof10.org/)
* Install each of the blends* Each desktop environment should have a start-stop test to verify that the default applications are basically working
Possibly also:* Upgrade/Replacement installation with a pre-existing Debian installation (e.g. trixie over bookworm)
* Upgrade a system from BIOS boot to UEFI boot with Secure Boot enabledWhen lowering the debconf priority, a lot more options are presented for partitioning (I'm thinking of #1102604, #1092977 and others) and several boot loaders are possible. I'm unsure which of these scenarios need to be tested.
What are your thoughts? I know we are still preparing trixie, so feel free to answer later. With kind regards, Roland Clobus
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature