2nd International Workshop on Readiness of HPC Extreme-scaling Applications

https://www.pop-coe.eu/news/events#RHEAworkshops 2nd International Workshop on Readiness of HPC Extreme-scaling Applications Marta García-Gasulla1[0000−0003−3682−9905] and Brian J.N. Wylie2[0000−0003−2770−2443] with contributions from Linda Gesenhues (EuroHPC JU, Luxembourg) Guntram Berti (scapos AG, Germany; CASTIEL2 CSA) Laura Bellantani (CINECA, Italy; MaX CoE) Erwan Raffin (Eviden, France; ESiWACE CoE) Martin Kronbichler (Ruhr University Bochum, Germany; dealii-X CoE) Francesco Salvadore (CINECA, Italy; ExCELLERAT CoE) Helena Vela Beltran (Do IT Now!, Spain; multiXscale CoE) João Barbosa ((IT4I@VŠB-TUO, Czechia; SPACE CoE) Guy Lonsdale (scapos AG, Germany; CASTIEL2 CSA) 1Barcelona Supercomputing Center, Barcelona, Spain [email protected] 2Jülich Supercomputing Centre, Forschungszentrum Jülich, Germany [email protected] Abstract. The workshop series on Readiness of HPC Extreme-scaling Applications provides an international forum to discuss common challenges, ideas, solutions, and opportunities from the point of view of HPC applications developers preparing for exa-scale. The second edition continued and expanded discussion from the first in 2024. Contributions from six currently active EuroHPC applications Centres of Excellence (CoEs) could be included this year in the half-day workshop in Hamburg, Germany on Friday 13 June, 2025. In short presentations and concluding moderated panel discussion each summarised the status of one or more applications from their CoE and associated challenges, particularly with respect to supercomputers solicited by the European High Performance Computing Joint Undertaking. Additional impulse presentations set the scene with an overview of the EuroHPC CoEs and their application portfolio, and an outline of EuroHPC plans to support application development in Europe. Keywords: Extreme-scaling applications ·exa-scale ·state of readiness. 2 M. García-Gasulla and B. J. N. Wylie 1 Introduction The Top500 list [4] of June 2022 had the first supercomputer with ExaFLOPS HPC performance, after many years of international community pursuit of this goal, and several more systems have followed or are expected within the next year. One is the European supercomputer JUPITER, hosted by Jülich Supercomputing Centre and half funded by EuroHPC Joint Undertaking, guaranteeing access to exa-scale primarily for projects led by institutions in Europe. It is therefore timely for HPC application software to demonstrate its readiness for extreme-scale computer systems composed from large assemblies of a heterogeneous variety of CPU processors and GPU accelerators. Europe has been preparing HPC applications for this challenge for the last decade through its Centres of Excellence (CoEs) [5]. They aim at greatly extending the scalability of a large variety of HPC codes and improving their execution efficiency and performance. Performance Optimisation and Productivity (POP) CoE [15], which is coordinated by both workshop organisers, is dedicated to providing free performance assessments to HPC application developers and in particular supports the domain-specific CoEs (as well as the wider HPC community of academic and industry). Insights gathered by POP showed that, although they serve different science fields, many of the challenges that these applications face are common and also the solutions adopted. For this reason, we have organised mini-symposia at two PASC conferences addressing the question: “Are HPC codes ready for exascale? An EU HPC Centre of Excellence Point of View.” Representatives from different CoEs shared their experience, via presentations and panels, and very fruitful discussions resulted. To broaden this activity to a larger community, last year the first edition of this ISC workshop [2] provided a forum to discuss common challenges, ideas, solutions, and opportunities from the point of view of HPC applications developers preparing for exa-scale. An inspiring keynote presentation by Lois Curfman McInnes (ANL, USA) on “Lessons learned by the DOE Exascale Computing Project” was complemented by CEEC, ChEESE, ESiWACE, HiDALGO2, MaX and multiXscale Centres of Excellence reviewing their progress. This year’s workshop continued and expanded the discussion with additional CoEs and updates for EuroHPC supercomputers, and plans to support application development. 2 Featured presentations 2.1 European HPC application ecosystem Guntram Berti (scapos AG, Germany) presented an overview of the about 60 HPC codes [6] of the current set of 14 HPC CoEs which are currently being ported to the EuroHPC JU supercomputer systems. In addition, some highlights from the Innovation Studies funded by the Inno4scale project [7] were discussed. 2nd Workshop on Readiness of HPC Extreme-scaling Applications 3 2.2 EuroHPC concept for applications Linda Gesenhues (EuroHPC JU/Luxemburg) presented the EuroHPC Joint Undertaking Research & Innovation concept for European HPC applications. Specifically, transversal Centres of Excellence are tasked to provide expertise and services to re-focused application community Centres of Excellence around distinct scientific domains and professional software development projects for HPC lighthouse codes, as detailed in a new call announced and opened during ISC. Additional funding for projects developing application frameworks, software development kits and libraries distributed as single packages, and new algorithms for HPC applications are expected to follow. Collaboration between CoEs and related projects (examples being EVITA, EPICURE & MINERVA) was emphasised, for consultancy, training and other services. 3 CoE presentations From contributions solicited from all of the active EuroHPC applications Centres of Excellence, the three hours allocated to this year’s workshop only allowed us to include six for short presentations in the workshop programme. Our selection favoured those CoEs who we were unable to accommodate in last year’s workshop. 3.1 MaXimizing portability and performance of material modelling Laura Bellantani (CINECA, Italy) introduced the MaX (Materials design at Exascale) CoE [13] which brings together computational scientists and core scientific communities with the goal of preparing key materials science codes for the future exascale computing clusters. The MaX portfolio includes several flagship codes — QUANTUM ESPRESSO, YAMBO, SIESTA, FLEUR and BIGDFT — selected for their global user base, open-source nature, and complementary approaches to quantum materials modeling. During earlier phases of the project, these code underwent substantial refactoring following the principle of separation of concerns: this strategy separates high-level components of the codes, such as property calculators and quantum engines primarily developed by domain scientists, from low-level libraries, typically maintained by technologists and optimised for performance on the diverse computing architectures. It was discussed how this modular design has proven effective in enhancing code portability and performance across a wide range of HPC platforms, while also ensuring long-term maintainability which is an essential requirement for community codes. The main challenges and milestones in enabling MaX applications to NVIDIA, AMD and Intel GPUs with ‘OpenX’ directive-based programming models were presented, along with discussion of runtime optimizations with Multi-Process-Service, and investigation of communication backends beyond OpenMPI (specifically HPCX-MPI and NCCL) to improve the efficiency of memory-demanding workloads in multi-GPU scaling. Their benchmarking and 4 M. García-Gasulla and B. J. N. Wylie profiling campaign on EuroHPC clusters has been streamlined by integrating applications, tools and platforms into an ecosystem of interoperable JUBE scripts ensuring consistent and reproducible data acquisition, and results are distributed to application users via an HTML-based visualization tool to help them identify the most suitable configuration for production runs. 3.2 Services and benchmark suite for weather and climate Erwan Raffin (Eviden, France) presented the CoE for Simulation of Weather and Climate in Europe, ESiWACE [10]. Among its activities, ESiWACE is delivering a range of services to support the weather and climate modelling community on the path to exascale computing on current and future European systems, including tailored help with model performance and scaling optimization. To illustrate this effort were two success stories: one dealing with GPU porting of the OGSTM ocean model and another on the optimization of the GLOBO atmospheric model improving its scalability. ESiWACE is also developing the High Performance Climate and Weather benchmark suite composed of flagship models and kernels. An overview of this domain specific benchmark was presented and its contribution to foster co-design especially for future European technology. 3.3 Preparing generic PDE solvers for exascale supercomputers Martin Kronbichler (Ruhr-University Bochum, Germany) presented activities in the EuroHPC CoE dealii-X [9], a project aiming to develop efficient simulation software for biomedical applications in the human body. One of the core activities is the development of highly efficient building blocks for solving partial differential equations with the deal.II finite element library. A comprehensive set of linear and nonlinear solvers for the mathematical models of fluid dynamics or (poro-)elasticity as well as new models for processes in cells are being worked on. Algorithmic advances to use large GPU-based supercomputers were presented with a report on experience with performance portability running their codes on hardware of different vendors. 3.4 Portability, performance, maintainability. Can they coexist? Francesco Salvadore (CINECA, Italy) presented the latest version of the compressible flows solver STREAmS that has been completely redesigned for HPC by the ExCELLERAT CoE [11] for engineering applications. The current version includes support for several programming paradigms oriented to the most popular HPC systems: starting from the version with pure MPI for CPU architectures, there is the OpenMP version to add the thread layer to CPU runs, the CUDA Fortran version to use NVIDIA GPUs, the HIP version to use AMD GPUs and APUs, and the OpenMP-offload version, potentially portable and today used mainly to exploit Intel GPUs. 2nd Workshop on Readiness of HPC Extreme-scaling Applications 5 Efficient algorithms and implementation enable STREAmS to address open problems in basic fluid dynamics research such as boundary layer or shockboundary layer interaction, achieving resolutions which can approach conditions normally achievable only by experiments. It has been tested on a considerable variety of clusters and architectures showing very good single-node and scalability performance. The case of an airfoil was considered which, thanks to the curvilinear grids, can be studied up to Reynolds and Mach values close to those of real flights allowing Direct Numerical Simulation to approach the area of industrial use from which it is historically far. The usability and maintainability of a cross-platform code such as STREAmS was investigated. The goal of the solver’s latest developments has been to broaden its usability without, however, distorting its origins and support from the community of scientific experts guiding its evolution. Code development takes place uniquely in CUDA Fortran (since Fortran is the historical language of CFD) and following a rather simple set of programming policies. From the developed source code, their performance portability library ‘sutils’ is able to translate the code and generate all the backend-dependent parts, even writing the C layer if necessary. This means that a normal PhD student is able to operate in the code without deep parallel programming or HPC skills and this gives chances for a successful maintainability of the solver. 3.5 Supporting cutting edge development of LAMMPS with EESSI Helena Vela (Do IT Now!, Spain) introduced the multiXscale CoE [14] which supports exascale-oriented application co-design and delivery for multiscale simulations. It is a collaborative project between members of the CECAM network and the EESSI community to allow domain scientists to take advantage of the computational resources offered by EuroHPC JU. The presentation focused on one of the lighthouse codes within multiXScale, LAMMPS, which is used by a large number of computational scientists. She discussed how the developers of new plugins for LAMMPS are testing on a wide range of systems with the help of the EESSI repositories. The software.eessi.io repository allows developers to share pre-releases of their software so they can test it on systems where EESSI is available, including the EuroHPC systems Vega,KAR0L1NA and Deucalion. For example, on Vega, development codes of LAMMPS are already available for using and testing through dev.eessi.io, and it was shown how continuous integration (CI) infrastructure provides pre-released versions of software using the plugins under development in multiXscale, with LAMMPS as an example. 3.6 Improving energy efficiency of SPACE CoE codes João Barbosa (IT4Innovations, Czechia) presented activities to enhance the energy efficiency of applications of the SPACE CoE [16]. He examined and measured their energy consumption and efficiency by modifying specific hardware power settings. The method involved static frequency tuning, where a single 6 M. García-Gasulla and B. J. N. Wylie hardware configuration (CPU or GPU frequency) was set at the beginning of each application run and remained unchanged. He explained how he monitored and assessed execution time, energy consumption, and FLOPS per Watt. He compared the energy efficiency of three hardware platforms: (i) NVidia Ampere A100 GPU installed in the IT4Innovations KAR0L1NA supercomputer, (ii) Intel Sapphire Rapids processor with DDR and HBM memory, and (iii) NVidia’s ARM-based Grace CPU. These platforms were chosen based on their relevance to SPACE CoE’s work, with a focus on GPU accelerators and co-design platforms. The methodology is not usable only for the astrophysics and cosmology community but, in general, for other parallel codes running on supercomputers globally. Considering that a machine consumes 20MW, energy savings of 5–10% are in orders in Megawatts, which significantly reduces operational costs. Energy efficiency analysis of the parallel codes was done by the open-source MERIC tool. He highlit that for the static tuning approach, there is no need to change the code to the optimal settings, and it can be applied job-wide using a job scheduler. 4 Panel discussion Guy Lonsdale (scapos AG/Germany) from CASTIEL2 CSA [8] moderated the panel which closed the workshop, comprising all of the workshop presenters. In keeping with the mid-week ISC conference keynote by Bjorn Stevens (Max-Planck-Institute for Meteorology), panelist introductions considered their favourite ‘barrels of beer.’ They then talked about their favourite HPC computing infrastructure and application code(s)/suite. These covered several personalised medical interventions, materials modelling, weather and climate, astrophysics and cosmoslogy, and numerous important discoveries from higher resolution simulations enabled with codes optimised to exploit the latest supercomputers. Challenges were also discussed, including access and use of supercomputers to be unified via the forthcoming EuroHPC federation platform, (big) data management issues, and engagement and exploitation from industry. While the new EuroHPC call tries to address these aspects, and encourage adaptation and modernisation of HPC applications, determining an effective mix of community and transversal CoEs and lighthouse codes for HPC will be tough. Individual lighthouse codes can be considered pathfinders providing inspiration for entire communities. Critically, the call should avoid a funding gap after the end of the current CoEs and start of those from the new funding round. Some CoEs reported currently being focused much more on addressing their science rather than improving their code scalability or GPU efficiency on heterogeneous supercomputers, while other applications have already done the preparatory reengineering of their codes to now address performance issues. Synergetic benefits of international collaborations were emphasised, to be balanced with IT technological sovereignty. Absence from this year’s workshop of representives of several EuroHPC CoEs was lamented, but overall a lot of achievements and impact were reported. 2nd Workshop on Readiness of HPC Extreme-scaling Applications 7 Fig. 1. Workshop organizers, presenters, moderator & panelists in Hamburg Acknowledgments. The organisers of this workshop were funded by grant agreement No. 101143931 (POP3) from the European High-Performance Computing Joint Undertaking (EuroHPC JU) which receives support from the European Union’s Horizon Europe research and innovation programme and Spain, Germany, Czechia, France and Portugal. Disclosure of Interests. The authors have no competing interests to declare that are relevant to the content of this article. References 1. International Workshops on Readiness of HPC Extreme-scaling Applications, pop-coe.eu/news/events#RHEAworkshops 2. ISC 2024 workshop, pop-coe.eu/news/events/readiness-of-hpc-extreme-scale-applications 3. ISC 2025 workshop, pop-coe.eu/news/events/readiness-of-hpc-extreme-scale-applications-2nd-edition 4. Top500 supercomputers, top500.org 5. HPC in Europe Centres of Excellence, hpc-portal.eu/coes 6. HPC in Europe Codes, hpc-portal.eu/codes-and-competences/codes 7. Innovative Algorithms for Applications on European Exascale Supercomputers, inno4scale.eu 8. CSA for National Competence Centres and Centres of Excellence on a European Level (CASTIEL2), hlrs.de/projects/detail/castiel-2 9. CoE for a Framework for Digital Twins of the Human Body (dealii-X), dealii-x.eu 10. CoE for Simulation of Weather and Climate in Europe (ESiWACE), esiwace.eu 11. CoE for Engineering Applications (ExCELLERAT), excellerat.eu 12. CoE for HPC and Big Data Technologies for Global Systems (HiDALGO2), hidalgo2.eu 13. CoE for Materials Design at the Exascale (MaX), max-centre.eu 14. CoE for Advancing Multiscale Simulation and Exascale Computing (multiXscale), multixscale.eu 15. CoE for Performance Optimisation and Productivity (POP), pop-coe.eu 16. CoE for Astrophysical and Cosmological Applications (SPACE), space-coe.eu