Hi Andreas,
Debian is/can be present on HPC natively (to deploy and manage such
cluster, many examples will be given) or via containerization
(increasingly more relevant), in particular with singularity. And
scientific computing is not just about HPC but also reproducibility
IMHO, here Debian has some to offer too. A few bullet points on
both
- there is https://wiki.debian.org/HighPerformanceComputing which lists
a number of packages which relate to HPC computing, although seems
missing
https://packages.debian.org/sid/environment-modules
which is pretty much "the" solution across many HPCs to provide
flexibility in environments people need/use
and give some semblance of "reproducibility" to them.
- there is https://lists.debian.org/debian-hpc/
- since advent of singularity (singularity-container package in debian
due to conflict), you might discover more of "Debian" being used on
HPC within containers providing specific computing environments and
setups
- that further improves flexibility and reproducibility of compute on the cluster
- re reproducibility, here is an example from another fella DD
Michael Hanke et al
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8917149/
FAIRly big: A framework for computationally reproducible processing of large-scale data
- their cluster at https://www.fz-juelich.de/en/inm/inm-7 is running
Debian
- with reproducible builds, https://snapshot.debian.org/ (and our
http://snapshot-neuro.debian.net/) and with/without
https://packages.debian.org/search?keywords=neurodebian-freeze
it becomes easy to establish reproducible containers -- so you have
some guarantee that your container still builds in the future without
divergence.
Cheers,
On Sun, 19 May 2024, Andreas Tille wrote:
> Hi,
> I have an invitation to have some talk with the title
> Debian GNU/Linux for Scientific Research
> Abstract:
> Over the past decade, Enterprise Linux has dominated large-scale
> research computing infrastructure. However, recent developments have
> sparked increased interest in community-led alternatives. Debian
> GNU/Linux, a long-standing choice among researchers for supporting
> scientific work, is experiencing a renewed interest for High-Throughput
> Computing (HTC) and High-Performance Computing (HPC) applications. This
> presentation will provide an overview of how Debian is being utilized to
> support scientific research and will include a case study showcasing the
> migration of HTC operations from Enterprise Linux 7 (EL7) to Debian.
> While I could talk about Debian Science and Debian Med in general it
> would be cool to reference to some real life examples where Debian is
> used in Science and what might be the reason to use Debian.
> I personally would like to stress the "we package what we use" aspect
> and the "we mentor upstream to merge competence of the program with
> packaging skills" idea. Any input would be welcome to cover more ideas.
> Kind regards
> Andreas.
--
Yaroslav O. Halchenko
Center for Open Neuroscience http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
WWW: http://www.linkedin.com/in/yarik
Attachment:
signature.asc
Description: PGP signature