Hi, Renato
Thanks for your message!
على الثلاثاء 10 تـمـوز 2018 14:23، كتب Renato Golin:
> Hey folks,
>
> Reasonably often in the OpenHPC group we get questions about Debian
> support, and the answer is always, "yeah, why not?". But we never got
> around looking properly into it. Few weeks ago I commented with
> someone that pointed me at this group, so, here I go.
>
> For those who don't know, OpenHPC is a Linux Foundation project [1]
> that aims to *validate* cluster setups, not enforce specific
> requirements. We have an OBS that builds RPM packages for both CentOS
> and Suse and we use those packages to provide an overlay on top of
> those OSs with pre-made packages (and dependencies, and LMod, and
> easyBuild/Spack, etc).
>
> The packages that we build are a super-set of the ones you build [2],
> and we build for x86_64 and Arm64. We have recipes (scripts and PDF
> documents) for many combinations of solutions, ex. { slurm, pbspro } x
> { warewulf, xcat } x { x86_64, arm } x ...
>
> Fujitsu & Linaro are working on an Ansible replacement for the bash
> recipes [3], but that's a shameless plug, so forget about it for now.
> :)
>
> We then do full validation of all those combinations into a set of
> clusters at TACC (upstream validation) and Linaro (local Arm64
> validation with different uarches). There are probably other local
> efforts that I don't know of, but you get the idea.
>
> I'm not a Debian guy, but I'm also not a CentOS guy (I ran Arch), but
> Linaro is very much a Debian shop, so there's also internal push to
> get something HPC-y going on Debian. There's also a lot of Debian-like
> deployments in China, which is also strong within Linaro.
>
> But instead of getting Deb packages building in our OBS, I thought I
> should talk to you guys and see how we could maybe join efforts to
> standardise HPC deployments, no matter the distro, no matter the
> architecture.
>
> To be clear, I don't want to merge groups, I just want to reduce
> effort. So, if I can at least get you guys testing some of our stuff
> and us testing some of yours, I'd be extremely happy. However, I think
> de-duplicating the build would be an immense gain, better still if we
> build Deb packages in the Debian way, as I don't have the know-how to
> make sure they're even remotely correct.
If I understand correctly, you mean doing some of your work within
Debian officially? Everyone is welcome to contribute, of course, and
packages need to be sponsored by a Debian Developer, which entails a
review for policy compliance.
>
> Knowing the Debian community a bit, a few things I *think* would worry you:
>
> 1. We do have non-free packages. Ex. PBSPro, Intel and Arm compilers,
> etc. The PBS we use is the community edition, but that doesn't help
> with the Debian ethos. I think it would be perfectly reasonable if we
> didn't build them on Debian, and only used the open source / free
> ones.
The hard line is that we can include packages in the non-free section so
long as the licenses permit redistribution. However, those packages are
second-class citizens in that they are not autobuilt (if there's even
source code) or tested with the archive QA tools.
>
> 2. Build resources are always scarce. I don't want to increase the
> burden of your builders, but I'm sure we could work out getting new
> builders if that were the case. I believe Linaro has done that before
> for Debian (certainly has for other projects), so not a big deal.
I don't see a problem here. Packages are being added to Debian all the time.
>
> 3. Diluting your current efforts with a seemingly "company driven"
> effort. We do have companies on OpenHPC, but it is driven by Jeff
> (Linux Foundation) and Karl (TACC) who are not themselves affiliated
> [4]. Also, OpenHPC is a small banner that is helping me unify multiple
> Arm vendors into one "standard", and getting Debian at least close to
> it would be a major win for standardise HPC deployments. Linaro, as
> probably some of you know, is an "upstream first" effort, so we won't
> keep anything downstream ourselves.
I don't see a problem here either so long as we have people here that
are willing to help out. I'm sure most of us here would agree that
integrating your work into official Debian packages is a great idea, but
volunteers tend to only package software if it's something they
personally use.
>
> So, does any of that interest any of you to make a collaborative effort?
>
I'd be willing to help people get started with making official Debian
packages. If you have someone who is already working on the debs, maybe
we can help them start with one that you want to integrate into Debian
and get a process going.
So what I'm offering is guidance, review, and sponsorship to get into
the Debian archive, not that I'll do the additional packaging work and
maintenance myself. Would that be useful to you?
Thanks and regards
Afif
--
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name