OpenHPC on Debian
Hey folks,
Reasonably often in the OpenHPC group we get questions about Debian
support, and the answer is always, "yeah, why not?". But we never got
around looking properly into it. Few weeks ago I commented with
someone that pointed me at this group, so, here I go.
For those who don't know, OpenHPC is a Linux Foundation project [1]
that aims to *validate* cluster setups, not enforce specific
requirements. We have an OBS that builds RPM packages for both CentOS
and Suse and we use those packages to provide an overlay on top of
those OSs with pre-made packages (and dependencies, and LMod, and
easyBuild/Spack, etc).
The packages that we build are a super-set of the ones you build [2],
and we build for x86_64 and Arm64. We have recipes (scripts and PDF
documents) for many combinations of solutions, ex. { slurm, pbspro } x
{ warewulf, xcat } x { x86_64, arm } x ...
Fujitsu & Linaro are working on an Ansible replacement for the bash
recipes [3], but that's a shameless plug, so forget about it for now.
:)
We then do full validation of all those combinations into a set of
clusters at TACC (upstream validation) and Linaro (local Arm64
validation with different uarches). There are probably other local
efforts that I don't know of, but you get the idea.
I'm not a Debian guy, but I'm also not a CentOS guy (I ran Arch), but
Linaro is very much a Debian shop, so there's also internal push to
get something HPC-y going on Debian. There's also a lot of Debian-like
deployments in China, which is also strong within Linaro.
But instead of getting Deb packages building in our OBS, I thought I
should talk to you guys and see how we could maybe join efforts to
standardise HPC deployments, no matter the distro, no matter the
architecture.
To be clear, I don't want to merge groups, I just want to reduce
effort. So, if I can at least get you guys testing some of our stuff
and us testing some of yours, I'd be extremely happy. However, I think
de-duplicating the build would be an immense gain, better still if we
build Deb packages in the Debian way, as I don't have the know-how to
make sure they're even remotely correct.
Knowing the Debian community a bit, a few things I *think* would worry you:
1. We do have non-free packages. Ex. PBSPro, Intel and Arm compilers,
etc. The PBS we use is the community edition, but that doesn't help
with the Debian ethos. I think it would be perfectly reasonable if we
didn't build them on Debian, and only used the open source / free
ones.
2. Build resources are always scarce. I don't want to increase the
burden of your builders, but I'm sure we could work out getting new
builders if that were the case. I believe Linaro has done that before
for Debian (certainly has for other projects), so not a big deal.
3. Diluting your current efforts with a seemingly "company driven"
effort. We do have companies on OpenHPC, but it is driven by Jeff
(Linux Foundation) and Karl (TACC) who are not themselves affiliated
[4]. Also, OpenHPC is a small banner that is helping me unify multiple
Arm vendors into one "standard", and getting Debian at least close to
it would be a major win for standardise HPC deployments. Linaro, as
probably some of you know, is an "upstream first" effort, so we won't
keep anything downstream ourselves.
So, does any of that interest any of you to make a collaborative effort?
Did I forget anything? Or got it completely wrong?
I'm all ears, let me know.
thanks!
--renato
[1] http://www.openhpc.community/
[2] https://qa.debian.org/developer.php?email=debian-hpc%40lists.debian.org
[3] https://github.com/Linaro/ansible-playbook-for-ohpc
[4] Though, Karl did work at Intel when he started.
Reply to: