HORIZON JU Research and Innovation Actions HORIZON-EUROHPC-JU-2021-COE-01-01 European High-Performance Computing Joint Undertaking Plasma Exascale-Performance Simulations CoE 101093261 D1.9 First Release – Plasma Simulation Codes and Documentation WP1: Plasma Simulations - Codes and Grand Challenges Date of preparation (latest version): 31/12/2024 Copyright©2023 – 2027 The Plasma-PEPSC Consortium
D1.9: First Release – Plasma Simulation Codes and Documentation 2 DOCUMENT INFORMATION Deliverable Number D1.9 Deliverable Name First Release – Plasma Simulation Codes and Documentation Due Date 31/12/2024 Deliverable lead MPG Authors David Tskhakaya (IPP CR), Tilman Dannert (MPG), Michael Bussmann (HZDR), Urs Ganse (UoH), and Jeremy Williams (KTH) Responsible Author Tilman Dannert (MPG) E-mail: [email protected] Keywords Software Release WP/Task WP1/Task D1.9 Nature R Dissemination Level PU Final Version Date 31/12/2024 Reviewed by Leon Kos (UL) & Filippo Mantovani (BSC)
D1.9: First Release – Plasma Simulation Codes and Documentation 3 DOCUMENT HISTORY Partner Date Comment Version KTH 03/03/2024 Skeleton version of the deliverable 0.1 MPG 07/11/20024 Skeleton adapted 0.2 MPG 02/12/2024 First draft 0.3 KTH 03/12/2024 Final draft updated for internal review 0.4 MPG 12/12/2024 Revised draft after internal review 0.5 KTH 20/12/2024 Final cleanup for submission 1.0
D1.9: First Release – Plasma Simulation Codes and Documentation 4 Executive Summary In this deliverable, we publicly release current versions of the codes within the project and summarize the contributions to the codes made since the beginning of Plasma-PEPSC two years ago. In short, BIT1 development was split into four parts: optimization (vectorization) of a number of functions, implementation of the parallel I/O via openPMD-API and ADIOS-2 libraries, load balancing and GPU porting. The first two tasks were (almost) completed, while the others are under development. The development work for GENE and GENEX was mainly put into the GPU porting of the GENEX code, which will take most of the envisaged runtime of the target run. The code was fully ported with an OpenMP directives approach, and now the time loop completely runs on NVIDIA GPUs. The extension to AMD GPUs (like in LUMI-G) is in development. The development of PIConGPU is focused on realizing the exascale science case, with proposals for LUMI and JUPITER currently under consideration. The project includes the implementation of a high-precision laser model and associated solvers, which are expected to reach completion by 2025. Full-scale simulation runs have already been planned to take advantage of these advances. To enhance the performance of PIConGPU, significant updates have been made to its parallelism infrastructure, which is based on the alpaka library, including improvements to vectorization support. The on-device memory allocation scheme has been updated using MallocMC. To address the I/O and analysis bottleneck, work was done to enable PIConGPU, integrated with openPMD, to function as part of an in situ machine learning (ML) workflow, and this was successfully demonstrated in a full-scale run. PIConGPU has also already demonstrated its scalability and capability through full-scale runs on LUMI this year. Vlasiator’s development focussed on performance portability onto both vector-heavy and GPU-heavy target platforms, with a major restructuring of solver data structures and flow control behavior to be both flexible and future-proof. In parallel, algorithmic improvements of both the Vlasov and field solvers substantially improved the HPC scaling behavior of the code. New, more dynamic adaptive mesh refinement methods allow the simulations’ resolution to track areas of physical interest more effectively. New in situ data reducers for magnetic topology identification and non-Maxwellianity measures allow for reduced output file footprints. The adoption of more structured development practices and more end-user outreach has greatly improved documentation and usability.
D1.9: First Release – Plasma Simulation Codes and Documentation 5 Contents 1 Introduction 6 2 Code Releases 6 2.1 BIT ...................................... 6 2.2 GENE/GENEX................................ 6 2.3 PIConGPU .................................. 7 2.4 Vlasiator.................................... 7 3 Conclusion 7
D1.9: First Release – Plasma Simulation Codes and Documentation 6 1 Introduction The four codes of the project have been developed during the first two years of the project and the current versions are released with this deliverable. The main purpose is to make available the first public version for review. Some of the codes already had a regular release cycle with many predecessor public releases (Vlasiator, PIConGPU, GENE), and some codes are for the first time released into the public (GENEX and BIT). Together with the codes, user documentation has been published in the respective repositories. 2 Code Releases All four codes have been released and can be accessed publicly or by reviewers on request. 2.1 BIT BIT1 is released through its official main development repository at https://repo.tok.ipp.cas.cz/tskhakaya/bit1 and mirrored to https://codehub.hlrs.de/coes/plasma-pepsc/ippcas/bit1 under the customized license https://repo.tok.ipp.cas.cz/tskhakaya/bit1/-/wikis/home We decided to keep this strategy until the final version of the code is developed. The latest release is “BIT1-9”; the user manual can be found on the same site. 2.2 GENE/GENEX GENE has been released with a customized Open Source license under the official URL, https://gitlab.mpcdf.mpg.de/GENE/gene.git which is accessible only to registered users. Registration is open to everyone and can be requested at www.genecode.org. The current release is in branch release-3.1. Release notes (file CHANGELOG.md) can be found in the code repository of the release with a summary on what progress the code made during the project, which is mainly related to AMD GPU and LUMI support. The repository is also mirrored to https://codehub.hlrs.de/coes/plasma-pepsc/mpg/gene.git GENEX has been released for the very first time under https://gitlab.mpcdf.mpg.de/phoenix-public/genex with the release tag 1.0.0. GENEX uses the ParallaX library, which in turn depends on the PAccX library for GPU simulations. In addition, both use ConfiX for common configuration tasks. All of these repositories are also released publicly under the same gitlab group phoenix-public. The access to the group will be granted on request (please send an email to
[email protected]), and a full publication with an open source license will happen in 2025 in the EuroFusion context.
D1.9: First Release – Plasma Simulation Codes and Documentation 7 2.3 PIConGPU PIConGPU is developed as an open-source project under the GPLv3+ license, with an open development model. The project is hosted at https://github.com/ComputationalRadiationPhysics/picongpu and mirrored to https://codehub.hlrs.de/coes/plasma-pepsc/hzdr/picongpu We use public GitHub issues so that users and developers can report issues, suggest improvements, and participate in discussions. Similarly all contributions to the code are managed via public pull requests, enabling transparency and community engagement in the development process. All current and future releases are distributed under the GPLv3+ license, ensuring freedom to use, modify, and share the software. Regular releases are accompanied by changelogs, detailing new features, improvements, and bug fixes; these releases can be accessed at https://github.com/ComputationalRadiationPhysics/picongpu/releases The latest release is version 0.7.0. and the current state of development is always visible publicly in the ”dev” branch. The next release, 0.8.0, is in the final stages of review, and will be released by mid-December 2024. 2.4 Vlasiator Vlasiator is released through its official main development repository at https://github.com/fmihpc/vlasiator and mirrored to https://codehub.hlrs.de/coes/plasma-pepsc/uoh/vlasiator under the GPL-2 open-source license. The most recent release version at the point of this document release is the tag v5.3.1. Future development and release versions will continue to be available from the same repository under an open-source license. An unsupported pre-release of the GPU port of Vlasiator has been made available under the tag GPU v1 at the same repository. 3 Conclusion All codes have been released with a well-defined version in a public or reviewer-accessible repository. Release notes with a description of the work on the codes during the first two years of the project have been added directly to the repositories in files called ”RELEASE NOTES.md” or similar in the respective base directory.
