RFC: Upload of our utilities to the main archive
Hi all,
A recent discussion on debian-science highlighted how having some of the
functionality we use within the team could also benefit others in the
project.
Specifically, this was about how to create autopkgtests for some other
package making use of ROCm. This requires some checks (GPU availability)
and benefits from logging (dmesg, firmware info).
Our 'upstream-binaries' script does this, but we just copy-paste it
everywhere. It's time to factor out the functionality, and to ship it in
the official Archive, so that others can also make use of it.
There are three things that I'd appreciate feedback on before I proceed
with an upload:
1. Scope
2. Naming/namespace
3. Interface of the autopkgtest helper
0. Preface
==========
In our own private archive [1], we currently have
src:rocm-dev-tools [2] with binary packages
- bin:rocm-qemu-support, shipping utilities:
$ rocm-qemu-setup
$ rocm-qemu-create
$ rocm-qemu-run
+ an autopkgtest-virt-qemu+rocm backend
- bin:rocm-podman-support, shipping utilities:
$ rocm-podman-setup
$ rocm-podman-create
$ rocm-podman-run
+ an autopkgtest-virt-podman+rocm backend
- bin:debci-scheduler
$ debci-scheduler
The setup/create/run utilities are helpers for setting up QEMU resp.
podman images for AMD GPU use. The autopkgtest backends bridge
autopkgtest to these images.
1. Scope
========
The must-have utility in the first upload would be a helper utility that
simplifies the definition (in d/test/control) of autopkgtests. So
something like
- bin:rocm-autopkgtest-support
$ rocm-autopkgtest-helper
I think the QEMU/podman support utilities are mature enough to ship, but
they are not must-haves, especially not in the first upload. We could
ship these at a later point.
The debci-scheduler utility will be factored out into its own source
package. This will happen regardless, as it has clearly outgrown
src:rocm-dev-tools.
2. Naming/namespace
===================
The rocm- prefix is misleading, I think, as it could easily be confused
with something that upstream provides.
To address this, I think we have two options: either we
(A) We drop the prefix, and use some private namespace like
/usr/libexec/rocm, so eg: /usr/libexec/rocm/qemu-create
(B) Install to /usr/bin but rename the prefix to rocm-team-, so
eg: rocm-team-qemu-create.
I'm tending towards (B) if only because the utilities are for actual
user use, so belong in $PATH.
I guess that src:rocm-dev-tools should also be renamed to
src:rocm-team-tools.
The Debian KDE Team calls its package pkg-kde-tools, so pkg-rocm-tools
would also be an option. Though I think they used pkg- as a prefix
because their tooling seems to be more about packaging.
3. Interface of the autopkgtest helper
============
Simply put, imagine the following:
d/tests/control
---------------
Test-Command: rocm-autokpgtest-helper /path/to/test(dir) [...]
Depends: rocm-autopkgtest-helper
That's it. The utility would execute the tests provided as one or more
arguments (if any argument is a directory, execute all tests in there).
It would also take care of pre/post-test stuff.
I imagine this utility supporting two environment variables:
ROCM_AUTOPKGTEST_HELPER_WITH_DMSG
ROCM_AUTOPKGTEST_HELPER_WITH_FIRMWARE
Using these would create artifacts that would be exported as part of the
autopkgtest result. You can WITH_DMESG enables dumping of dmesg before
and after a test, whereas WITH_FIRMWARE dumps firmware info before a test.
The idea is that our own ci.rocm.debian.net workers would set these
variables in applicable testbeds, as eg: for our QEMU workers, the risk
of an information leak is zero. But a user running this autopkgtest
would not have their dmesg exposed by default.
I'd really appreciate your thoughts on this.
Best,
Christian
[1]: https://apt.rocm.debian.net/
[2]: https://salsa.debian.org/rocm-team/rocm-dev-tools
Reply to: