Hello Andrew, lists, On 10/01/2026 17:59, Andrew M.A. Cater wrote:
Apologies for the multiple lists but this needs to be widespread:
For follow-up replies, I suggest to use only debian-cd (I'm subscribed to all of the mentioned mailing lists)
Currently testing CD/BD and so on images for the Trixie and Bookworm release. The question has come up: The images team do not consider that it is worthwhile generating live images for oldstable and testing live images for oldstable with each new point release. The likelihood of *needing* live images for oldstable is small and the amount of testing is large. For the expected use case, does the testing effort outweigh the actual expected use?
I understand that the testing effort (8 live images for amd64) is huge. Many of the tests are really slow to perform (i.e. they take more than 10 minutes each, with lots of waiting in-between).
What do others think?
Given that releases of oldstable have been tested many times when oldstable still was called stable, I would assume (or hope) that the basic functionality is OK and that the issues associated with such images are largely known.
Instead of doing many manual tests, it would be possible to lower the amount of testing to only the automated tests running at openqa.debian.net (and to announce this basic level of support in the corresponding release notes). Or just a handful of random tests on real hardware could be performed instead of all of them. I am aware that the virtual environment used by openQA will not catch hardware-related issues (especially hardware requiring kernel modules and/or firmware).
Whether it would make sense to release new live images or not, largely depends on how they are used:
* If the user does not use persistency (the default setting), a newer live image would contain all security fixes and would be the preferred live image for such user.
* If the user does use persistency (a tweak required to be done by the user), there would no need for updated live images, because all updated content would be stored in the persistency-layer and just replacing the image (while preserving the persistent data) is not trivial.
I assume that the largest group would be the first group. (But I know of no metrics)
Also, for bookworm the point release is number 13. Would it be possible for the release team to do the full manual tests for the first few point releases instead of for all of them?
Another point to note however, should it be decided to rely on openQA for automated tests, is that the amount of automated tests on openQA does not match the amount of manual tests. Even though we have 2 participants of Outreachy helping out, adding and maintaining tests still needs more volunteer time.
For your background: openQA simulates the user interaction via keyboard and mouse and tests the whole process from booting to shutting down. These tests are integration tests and benefit from the already existing autopkgtests of the individual packages.
You do not need to write code [1] in order to help. The major hurdle for adding tests is to know which tests would be meaningful. It is sufficient to write such tests in human readable recipes and the computer code can be written by other people.
Who would like to help? The documentation is located on the big buttons on the main screen of openqa.debian.net.
With kind regards, Roland Clobus Co-maintainer of the live images Co-maintainer of openqa.debian.net [1] https://openqa.debian.net/evolution_of_test_scenario.html
Attachment:
OpenPGP_signature.asc
Description: OpenPGP digital signature