Hi Christian,
Thank you for your thoughts on the benchmark clients.
On 2024-07-18 00:58, Cordell Bloor wrote:If anyone wants to bikeshed the package name, I'm open to that. I picked rocblas-bench to match the executable name, but I could see arguments for something like librocblas-benchmarks or librocblas0-benchmarks. When we update rocBLAS to ROCm 6.1, there will also be a `rocblas-gemm-tune` executable that we will need to decide how to handle [1].I think -bench as a package name suffix is fine. The argument for librocblas0-bench over rocblas-bench is that when using /usr/libexec, the former would allow an eventual librocblas1-bench to be co-installable, which could be a nice thing to have for comparison and regression testing. That was my rationale behind librocrand1-tests, etc.
After reviewing more of the benchmark utilities available in other ROCm packages, I've come to agree that librocblas0-bench may be preferable. While libraries like rocblas, rocfft, rocsparse and rocsolver have a single benchmark utility that might make sense for installing to /usr/bin, other libraries have collections of benchmark tools that are less general-purpose.
It will therefore be a more consistent convention if the benchmark tools are installed to /usr/libexec and if the package name is not tied to the name of a specific binary. If we feel a need, we can always create a package that symlinks a specific binary into /usr/bin and installs a manpage.
I may also need guidance on the appropriate section for these utilities. I've installed rocblas-bench into /usr/bin, but lintian warns against this because that the package is in the libdevel section. I tried reviewing the packages of other benchmark utilities in Debian, but was still left uncertain of how to categorize rocblas-bench. Perhaps the util section would be more appropriate?sysbench is in misc, stress-ng is in devel, so not all too helpful data points. misc is a catch-all. The description for util is "Utilities for file/disk manipulation, backup and archive tools, system monitoring, input systems, etc."; for devel its "Development utilities, compilers, development environments, libraries, etc.". I think it depends on who we eventually most see using this: developers or non-developers (rocblas as a dependency of other packages). But I'm really /really/ bikeshedding here.
If we install to /usr/libexec like the tests, the warning is no longer applicable. I suppose that would be problem solved.
Or maybe the rocblas-bench client should be installed in /usr/libexec like the tests? I think /usr/bin/rocblas-bench makes sense, but I'm open to suggestions if anyone feels strongly otherwise.The distinction between /usr/bin and /usr/libexec [1] is basically: do we want this in $PATH or not. (Also, see next reply below). For tests, I didn't think so, as I thought the primary driver would be autopkgtest even on bare metal, hence /usr/libexec. For benchmarks, the argument could be made that some user might want to install and call this directly. We could "abuse" the autopkgtest machinery for this as well (which I'd actually like to do), but if the typical use case is "we want users to be able to quickly determine whether their performance is on par", then direct calling would probably be it.
It's difficult for me to judge. As developer, I would run these utilities directly with specific arguments to check things. However, that is also true of the test clients. I am not representative of a typical user.
This upload will probably define the convention that will be used for rocfft, rocsolver, rocsparse, etc. when their benchmark utilities are packaged. As such, it would be particularly valuable to review this one carefully.I think a conclusive answer depends on how they all build stuff. rocrand [2] and rocsparse [3], to name two examples, build dozens of untypically named individual test binaries which we all but certainly don't want in /usr/bin even if we wanted users to call them directly, so the choice where to put tests was easy. I haven't looked but if some package builds dozens of benchmark binaries, then I think those should go into /usr/libexec, and we should provide the user with the means to easily run them all with one command, eg: a wrapper script similar to the 'upstream-binaries' we provided for autopkgtests.
You mean rocrand and rocprim, but yes. The benchmark utilities are much the same as the test clients. The libraries that build a single test client are generally the libraries that build a single benchmark client, and the libraries that build many test binaries are the libraries that build many benchmark binaries.
I suppose the most sensible thing to do is to handle the benchmark clients in the same way as the test clients. I will rework this package to follow those conventions, and submit a new RFS.
Sincerely, Cory Bloor
[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.html#usrlibexec [2]: https://packages.debian.org/sid/amd64/librocrand1-tests/filelist [3]: https://packages.debian.org/sid/amd64/librocprim-tests/filelist