Speaker
Description
Current and upcoming astronomical surveys (e.g., SKA, LSST, Euclid, or even ALMA) present a significant data processing challenge, with data volumes that overwhelm traditional, single-node analysis workflows. Many of our community's essential analysis tools are built within the Python ecosystem, but they often struggle to scale to the high-performance computing (HPC) resources required for these future datasets.
Heat (GitHub) is a Python library designed to bridge this gap. As a cross-disciplinary tool developed within Germany's Helmholtz Association, it is already employed in applications ranging from climate modeling and neuroscience to aerospace engineering. Heat provides a distributed, NumPy-like array library that enables scientists to scale existing data analysis codes with minimal modification. This allows a seamless transition from a laptop to thousands of CPU cores or GPUs on HPC clusters. By using a data-parallel architecture, MPI for communication, and PyTorch as a backend, Heat is optimized for efficient execution on heterogeneous hardware.
In this talk, we will introduce the core concepts of Heat, show its potential as an HPC backend for the Python array ecosystem, and demonstrate the first few applications in astronomy.
| Affiliation of the submitter | Forschungszentrum Jülich, Jülich Supercomputing Centre |
|---|---|
| Attendance | in-person |
