D5.3 - Report on testing provided software

Repo on es ing p o ided so wa e Mul iXscale Deli e able 5.3 Deli e able Type: Repo Deli e ed in June, 2025 Mul iXscale Eu oHPC Cen e o Excellence o Mul iscale Modelling Acknowledgemen Funded by he Eu opean Union. This wo k has ecei ed unding om he Eu opean High Pe o mance Compu ing Join Unde aking (JU) unde g an ag eemen No 101093169. Disclaime Funded by he Eu opean Union. Views and opinions exp essed a e howe e hose o he au ho (s) only and do no necessa ily e lec hose o he Eu opean Union o he Eu opean High Pe o mance Compu ing Join Unde aking (JU). Nei he he Eu opean Union no he g an ing au ho i y can be held esponsible o hem. Mul iXscale Deli e able 5.3 Page ii P ojec and Deli e able In o ma ion P ojec Ti le Mul iXscale: Eu oHPC Cen e o Excellence o Mul iscale Modelling P ojec Re . G an Ag eemen 101093169 P ojec Websi e h ps://www.mul ixscale.eu Eu oHPC P ojec O ice D . Ma eo Mascagni Deli e able ID D5.3 Deli e able Na u e Repo Dissemina ion Le el Public Con ac ual Da e o Deli e y P ojec Mon h 30 (30 h June, 2025) Ac ual Da e o Deli e y 27 h June, 2025 Desc ip ion o Deli e able Repo on he moni o ing and es ing o he cen al sha ed so wa e s ack ac oss he ange o suppo ed pla o ms. Dashboa d o p esen cu en suppo s a us o cen al so wa e s ack on cu en sys em a chi ec u es. Documen Con ol In o ma ion Documen Ti le: Repo on es ing p o ided so wa e ID: D5.3 Ve sion: As o June, 2025 S a us: Accep ed by S ee ing Commi ee A ailable a : h ps://www.mul ixscale.eu/deli e ables Documen his o y: In e nal P ojec Managemen Link Re iew Re iew S a us: Re iewed Au ho ship W i en by: Sa ish Kama h (SURF) and Maksim Mas e o (SURF) Con ibu o s: Caspa an Leeuwen (SURF), Caspe an Leeuwen (SURF), Paul Melis (SURF), Kenne h Hos e (UGen ), La a Pee e s (UGen ), Alan Ó Cais (UB), Thomas Röbli z (UiB) Re iewed by: Caspa an Leeuwen (SURF), Kenne h Hos e (UGen ) App o ed by: Alan O’Cais (UB) Documen Keywo ds Keywo ds: Mul iXscale, High Pe o mance Compu ing (HPC), so wa e, applica ions, in as uc u e 27 h June, 2025 Disclaime : This deli e able has been p epa ed by he esponsible Wo k Package o he P ojec in acco dance wi h he Conso ium Ag eemen and he G an Ag eemen . I solely e lec s he opinion o he pa ies o such ag eemen s on a collec i e basis in he con ex o he P ojec and o he ex en o eseen in such ag eemen s. Copy igh no ices: This deli e able was co-o dina ed by Sa ish Kama h1(SURF) and Maksim Mas e o 2(SURF) on be- hal o he Mul iXscale conso ium wi h con ibu ions om Caspa an Leeuwen (SURF), Caspe an Leeuwen (SURF), Paul Melis (SURF), Kenne h Hos e (UGen ), La a Pee e s (UGen ), Alan Ó Cais (UB), Thomas Röbli z (UiB) . This wo k is licensed unde he C ea i e Commons A ibu ion 4.0 In e na ional License. To iew a copy o his license, isi : h p://c ea i ecommons.o g/licenses/by/4.0 cb 1[email p o ec ed] 2maksim.mas e o[email p o ec ed] Mul iXscale Deli e able 5.3 Page iii Con en s Execu i e Summa y 1 1 In oduc ion 2 1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Ta ge audience o his deli e able . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Deli e able ou line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.4 Pa ne con ibu ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2 Deploymen s 3 2.1 Running he es sui e as pa o he EESSI deploymen pipeline . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Pe iodic uns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2.1 Tes ing se up .................................................. 4 3 Dashboa d 5 3.1 O e iew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.2 S uc u e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.3 Da a s o age . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.4 Secu i y .......................................................... 7 3.5 Dashboa ds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.5.1 Pe o mance as ime se ies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.5.2 Pe o mance as beeswa m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.5.3 Iden i y ma ix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.6 Connec ed sys ems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4 Analysis 11 4.1 Func ional e i ica ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.2 Time se ies analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.2.1 Pe o mance baseline and a iance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.2.2 Pe o mance pa e ns and implica ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.3 Ha dwa e based compa ison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 5 Conclusion and ou look 17 Re e ences 18 Lis o Figu es 1 Dashboa d ecosys em. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2 Dashboa d ne wo k and secu i y es ic ions. The g een line indica es ull (un es ic ed) access, he yel- low lines indica e es ic ed access g an ed o sys ems based on hei IP add esses, he ed line indica es es ic ed access o he gene ic public. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3 An example iew on he dashboa d depic ing pe o mance ime se ies o he Tenso Flow es execu ed on Snellius “genoa” pa i ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 4 Fil e ing panel wi h highligh ed sec ions (le ) and de ailed in o ma ion abou a es poin ( igh ). . . . . 8 5 An example o a beeswa m plo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 6 Iden i y ma ix. An o e iew o es names o e sys ems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 7 Iden i y ma ix ool ip wi h de ailed in o ma ion on a es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 8 Iden i y ma ix. De ailed iew on he module names o e sys ems. . . . . . . . . . . . . . . . . . . . . . . . 10 9 Pe o mance ( ime s eps pe second, highe is be e ) s ime o LJ es o La ge-scale A omic/Molecula Massi ely Pa allel Simula o (LAMMPS) applica ion on Vega CPU pa i ion (Rome 7H12) execu ed on a scale o 2 nodes. Time spans om 01-01-2024 ill 01-05-2025. The blue dashed line indica es he mean pe o mance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 10 Pe o mance (ns pe day, highe is be e ) s ime o GROMACS applica ion benchma k on Snellius CPU pa i ion (Genoa 9654) execu ed on a scale o 1 node. Time spans om 01-07-2024 ill 09-05-2025. . . . . 12 11 Bandwid h (MB pe second, highe is be e ) s ime o poin o poin es o OSU Mic obenchma ks applica ion on Snellius CPU pa i ion (Genoa 9654) execu ed wi hin a node. Time spans om 01-01- 2024 ill 01-03-2025. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 12 Bandwid h (MB pe second, highe is be e ) s ime o poin o poin es o OSU Mic obenchma ks applica ion on Snellius CPU pa i ion (Genoa 9654) execu ed ac oss 2 nodes. Time spans om 01-01- 2024 ill 01-03-2025. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Mul iXscale Deli e able 5.3 Page i 13 Pe o mance (seconds pe s ep, lowe is be e ) s ime o he P3M es o ESPResSo applica ion on Vega CPU pa i ion (Rome 7H12) execu ed on 1 node. Time spans om 01-01-2024 ill 04-01-2025. . . . 15 14 Pe o mance (seconds pe s ep, lowe is be e ) s ime o he LJ es o ESPResSo applica ion on Vega CPU pa i ion (Rome 7H12) execu ed on 1 node. Time spans om 01-01-2024 ill 04-01-2025. . . . . . . 15 15 Pe o mance (images pe second, highe is be e ) s ime o Tenso Flow applica ion on a ious CPU pa i ions on he cloud (AWS), Eu oHPC sys ems (Vega and Ka olina), Snellius (Du ch Tie -1 sys em) execu ed on 1 node. Time spans om 01-01-2025 ill 19-05-2025. . . . . . . . . . . . . . . . . . . . . . . . 16 16 Pe o mance (images pe second, highe is be e ) s ime o Tenso Flow applica ion on a ious CPU pa i ions on he cloud (AWS), Eu oHPC sys ems (Vega and Ka olina), Snellius (Du ch Tie -1 sys em) execu ed on 2 nodes. Time spans om 01-01-2025 ill 19-05-2025. . . . . . . . . . . . . . . . . . . . . . . . 16 Lis o Tables 1 Lis o HPC si es and clouds p esen ed in he dashboa d. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Mul iXscale Deli e able 5.3 Page 1 Execu i e Summa y The Eu opean En i onmen o Scien i ic So wa e Ins alla ions (EESSI) es sui e desc ibed in Deli e able 1.5 is de- ployed on a ious pla o ms in wo phases, namely (i) he p e-deploymen phase and (ii) he pos -deploymen phase. These pla o ms include he cloud such as Amazon Web Se ices (AWS), Eu oHPC sys ems such as Vega and Ka olina, Na ional Tie 1 sys ems such as Snellius (The Ne he lands), Ho ense (Belgium) and Be zy (No way) and Na ional Tie 2 sys ems such as F am (No way). P e-deploymen es ing is pe o med be o e he applica ion is inges ed in o EESSI so wa e s ack and pos -deploymen es ing is pe o med in a pe iodic manne on a ious sys ems lis ed in his deli e able. The pu pose o he p e-deploymen es s ep is no only o es he so wa e i sel bu also o es ha he unde lying lib a ies ha a e used by he so wa e a e also wo king in a sane manne on a gi en ha dwa e a chi ec u e. A mapping is implemen ed be ween he so wa e o be inges ed and he es s o be execu ed. O cou se, his es ing is designed o consume minimal esou ces while es ing he so wa e, some o i s componen s (i applicable), and ano he appli- ca ion om he s ack o check i he ins alla ion a ec s hem. The pos -deploymen es s a e pe iodically un on a ious sys ems lis ed in his deli e able. The esul s o hese es s a e au oma ically inges ed in o a da abase and can be isualized using a dashboa d. The echnical de ails o he da abase inges ion pipeline, he design o he dashboa d, i s iews and o he aspec s such as s o age, secu i y e c. a e discussed in de ail wi hin his deli e able. The esul s o he es sui e show he indica i e pe o mance o a pa icula so wa e ha can be achie ed on a gi en ha dwa e. The dashboa d p o ides a pla o m o analyze his indica i e pe o mance on a ious ha dwa e as a ime se ies and pe o m a compa a i e analysis ac oss di e en sys ems. This is no only use ul o he end-use s, bu also o he sys em main aine s and adminis a o s. To summa ize: he de eloped es sui e is deployed ac oss a ious sys ems o collec indica i e pe o mance da a and a dashboa d is de eloped o he analysis and con inuous moni o ing o he EESSI so wa e s ack. Mul iXscale Deli e able 5.3 Page 2 1 In oduc ion 1.1 Scope This deli e able aims o desc ibe he pe iodic es s ha a e unning ia he de eloped po able es sui e desc ibed in Deli e able 1.5, unning on se e al Eu opean High Pe o mance Compu ing Join Unde aking (Eu oHPC) sys ems as well as in ou Con inuous In eg a ion (CI) bo . The p ima y pu pose o unning pe iodic es s is o check he pe o mance o he EESSI so wa e s ack in a ime se ies and p o ide he esul s o he use s so ha pe o mance expec a ions o a ious so wa e can be se on di e en ha dwa e o he use s on di e en sys ems. To do his in an e icien manne , a es sui e dashboa d (also desc ibed in his documen ) is de eloped whe e he pe o mance o he EESSI so wa e s ack can be checked on a ious sys ems and compa ed. 1.2 Ta ge audience o his deli e able This deli e able is a ge ed a eade s wi h expe ience in High Pe o mance Compu ing (HPC) sys em suppo and HPC end-use s, pa icula ly in he con ex o main aining and es ing so wa e s acks (o speci ic applica ions) o HPC in as uc u e. 1.3 Deli e able ou line Sec ion 2desc ibes how he EESSI es sui e is deployed as a es s ep in he so wa e deploymen pipeline and as pe i- odic uns whe e he ins alled so wa e a e es ed on a ious ha dwa e pla o ms. The da a p oduced by hese pe iodic uns a e collec ed in a da abase and can be isualized using a dashboa d whose de elopmen and echnical de ails a e desc ibed in Sec ion 3. Fu he mo e, Sec ion 4desc ibes he po en ial usage o he deployed es sui e and he da a collec ed om pe iodic uns, h ough he dashboa d, by end use s, sys em main aine s, and adminis a o s. 1.4 Pa ne con ibu ions SURF and UGen con ibu ed as planned o he wo k in his deli e able. No e ha Task 5.3 had o he pa ne s (RIJK- SUNIGRON, Uib, Ub), who con ibu ed o he de elopmen o he p e-deploymen pipeline and o he deploymen o he es sui e (as pe iodic uns) in a ious sys ems bo h a he na ional and Eu oHPC le el. Mul iXscale Deli e able 5.3 Page 3 2 Deploymen s Due o he sepa a ion o sys em-speci ic and es -speci ic in o ma ion, he EESSI es sui e (desc ibed in deli e able 1.5) can be deployed on any sys em, while only equi ing ha one w i e a ReF ame con igu a ion ile ha desc ibes he speci ics o ha sys em. The e a e wo ways in which he es sui e is used wi hin he Mul iXscale p ojec : 1. In he es s ep o he deploymen pipeline o add new so wa e o EESSI. 2. Regula uns on a la ge a ie y o HPC sys ems p o iding EESSI. 2.1 Running he es sui e as pa o he EESSI deploymen pipeline All EESSI so wa e is buil by he EESSI build bo s (see [1]). Bo ins ances a e unning on a a ie y o sys ems: om Magic Cas le clus e s in AWS and Azu e, o HPC clus e s om Mul iXscale pa ne s (e.g. a SURF and UGen ), o Eu oHPC clus e s (e.g. on Deucalion, and in p epa a ion o JUPITER). The bo con igu a ion desc ibes which pa i ions a bo ins ance can submi o. Fo example, he EESSI build bo unning on he AWS Magic Cas le clus e is con igu ed wi h: 1a ch_ a ge _map = { 2"linux /x86_64/gene ic" : "−−pa i ion x86−64−gene ic−node" , 3"linux /x86_64/ in el /haswell" : "−−pa i ion x86−64−in el −haswell−node" , 4"linux /x86_64/ in el /sapphi e apids" : "−−pa i ion x86−64−in el −s apids−node" , 5"linux /x86_64/ in el /skylake_a x512" : "−−pa i ion x86−64−in el −skylake−node" , 6"linux /x86_64/ in el /cascadelake ": "−−pa i ion x86−64−in el −caslake −node" , 7"linux /x86_64/ in el /icelake ": "−−pa i ion x86−64−in el −icelake −node" , 8" linux /x86_64/amd/zen2 " : "−− pa i ion x86−64−amd−zen2−node" , 9"linux /x86_64/amd/zen3" : "−−pa i ion x86−64−amd−zen3−node" , 10 "linux /aa ch64/gene ic" : "−−pa i ion aa ch64−gene ic−node" , 11 "linux /aa ch64/neo e se_n1" : "−−pa i ion aa ch64−neo e se−n1−node" , 12 "linux /aa ch64/neo e se_ 1" : "−−pa i ion aa ch64−neo e se− 1−node" } The build pipeline ha he bo uns has h ee s ages: a build s age, a es s age, and a deploy s age. To be able o un he es s ep, a ReF ame con igu a ion ile is equi ed ha ma ches he pa i ions o which he bo is con igu ed. Fo example, he sec ion o he In el Skylake nodes on his clus e would look like his in he ReF ame con igu a ion ile: 1{ 2’name’: ’x86_64_in el_skylake_a x512 ’ , 3’schedule ’ : ’ local ’ , 4’launche ’ : ’mpi un ’ , 5’ access ’ : [’−−nodes=1 ’ , ’−−n asks−pe −node=16 ’ , ’−− pa i ion x86−64−in el −skylake −node’ ] , 6’ en i ons ’ : [ ’ de aul ’ ] , 7’ ea u es ’ : [ 8FEATURES.CPU 9] + l i s (SCALES. keys () ) , 10 ’ esou ces ’ : [ 11 { 12 ’name’ : ’memo y’ , 13 ’op ions ’ : [’−−mem={ size } ’] , 14 } 15 ] , 16 ’ ex as ’ : { 17 # Make su e o ound down, o he wise a job migh ask o mo e mem han is a ailable 18 # pe node 19 EXTRAS.MEM_PER_NODE: 31342, 20 } , 21 ’max_jobs ’ : 1 22 } , We don’ un he ull es sui e in he es s ep: i he e a e es s in he EESSI es sui e o he so wa e ha is being added, we un hose. In addi ion, we un a small numbe o es s o make su e he ins alla ion does no inad e en ly b eak o he componen s o he so wa e s ack. A mapping is done be ween which so wa e is being ins alled, and he co esponding se o es s om he EESSI es sui e ha should be un o his so wa e. 2.2 Pe iodic uns Pe iodic es ing o applica ions wi hin he EESSI so wa e s ack is se up on a ious Eu oHPC sys ems as well as local Tie 1 sys ems, and also in he cloud. In any clus e , he pe o mance o a ce ain so wa e depends on pe o mance Mul iXscale Deli e able 5.3 Page 4 o a ious sys em le el componen s such as he OS-le el s ack, he compile s/ oolchains, he in e media e ma h li- b a ies and inally applica ion le el so wa e and i s dependencies. The p ima y pu pose o he es ing is o check he pe o mance o he so wa e p o ided by he EESSI s ack which mainly co e s e e y hing abo e he OS-le el s ack. Fu he mo e, since he so wa e s ack is being es ed on a ious di e en sys ems, obse a ions ega ding he sys ems hemsel es can be d awn om he da a which is collec ed and displayed in he dashboa d. These will be co e ed in sec ion 4. 2.2.1 Tes ing se up These es s a e se up using he c on jobs on he login nodes o he sys ems and hen he es esul s a e pushed o he dashboa d s o age ia an inges ion sc ip . Based on he esou ce a ailabili y, he equency and scale o he es s a e de ined. The daily es s a e limi ed o a maximum scale o 2 nodes and he weekly es s can scale up o 16 nodes which is he cu en maximum chosen wi hin he es sui e. Fo mo e in o ma ion ega ding possible scales, please e e o Deli e able 1.5. Fo each o he pe iodic uns, a ReF ame con igu a ion ile desc ibing he sys em is also equi ed. Cu en ly, pe iodic uns a e pe o med on he ollowing clus e s: • Vega (IZUM) • Ka olina (IT4inno a ions) • Snellius (SURF) • Doduo, Donphan, Gallade, Ho ense, Shinx, Ski y (UGen ) • BETZY, FRAM, SAGA (Sigma2) •AWS Magic Cas le All o hese clus e s un he EESSI es sui e on he EESSI so wa e en i onmen , and push hei da a o he dashboa d (discussed mo e ex ensi ely in sec ion 3). Mul iXscale Deli e able 5.3 Page 5 3 Dashboa d 3.1 O e iew The de eloped online dashboa d o e s an in ui i e and s aigh o wa d way o isualize es esul s gene a ed by he EESSI ReF ame es sui e. I p esen s da a ei he as a se ies o pe o mance me ics (e.g. ime se ies) o as an iden i y ma ix showing es pass a es. The o me can be accessed by isi ing h ps://dashboa d.eessi.io, whe eas he la e is in eg a ed in o he documen a ion page on h ps://eessi.io/docs/ es sui e/dashboa d/. These isual ep esen a ions o he es esul s help sys em adminis a o s and EESSI de elope s o quickly iden i y and add ess inconsis encies in he pe o mance o scien i ic applica ions in eg a ed in o he EESSI sha ed so wa e s ack. 3.2 S uc u e Figu e 1: Dashboa d ecosys em. Figu e 1illus a es he gene ic s uc u e o he dashboa d ecosys em. Each e ical laye ep esen s a sepa a e hos ing si e dedica ed o a speci ic ask: •HPC si e – an HPC sys em whe e he ReF ame es s a e execu ed and es epo s a e gene a ed. • Da abase Vi ual Machine (Da abase VM) – a cloud i ual machine ha hos s he Elas icsea ch da abase whe e he es da a is s o ed. • Dashboa d Vi ual Machine (Dashboa d VM) – a cloud i ual machine ha uns he on -end applica ion wi h a dashboa d. A he HPC si e, es epo s om he EESSI ReF ame sui e a e c ea ed and sa ed in Ja aSc ip Objec No a ion (JSON) o ma . These JSON iles a e hen pa sed by a Py hon inges ion sc ip , which pushes he pa sed da a in o he Elas icsea ch (ES)da abase on he Da abase VM. On he Dashboa d VM, an Elas ic p oxy handles que ies o he da abase, e ie ing ele an da a and making i accessible o he on -end o isualiza ion. On all pa icipa ing HPC si es, he inges ion sc ip is scheduled o un daily ia a c on job. Upon execu ion, he sc ip scans he designa ed di ec o y whe e ReF ame s o es i s epo iles, pa ses each ile o ex ac es me ada a, and compu es a unique hash alue o e e y es en y. This hash alue is gene a ed using he ollowing ields om each es epo : • Times amp o execu ion • Tes name • Hos name • Sys em name • ReF ame hash alue (unique pe un) • Schedule assigned job ID (e.g., om SLURM) • Tes elapsed ime • Command line used o un he es A combina ion o hese ields is hashed using MD5 o c ea e a uly unique iden i ie o each es ha is used by he inges ion sc ip o de-duplica ion. The sc ip hen que ies he Elas icsea ch (ES)da abase o check o he exis ence Mul iXscale Deli e able 5.3 Page 12 4.2 Time se ies analysis 4.2.1 Pe o mance baseline and a iance Es ablishing a baseline pe o mance o a gi en es is gene ally di icul . Fo ( e y) basic syn he ic es s (e.g. a band- wid h es ), he baseline is ypically based on he ha dwa e speci ica ions and i mwa e se ings. Howe e , o applica- ion es s, i is nea -impossible o de e mine a heo e ical baseline pe o mance. A p ac ical esul o he pe iodic es - ing is ha i p o ides a clea baseline, which hen allows de ec ion o any changes compa ed o ha baseline. Figu e 9shows he LAMMPS pe o mance o e ime on Vega’s Rome CPU pa i ion. Apa om he mean, which is he baseline pe o mance achie ed (in ime s eps pe second), he a iance can p o ide in o ma ion on he s abili y o pe o mance o a sys em. Some es s inhe en ly show a highe a iabili y han o he s - his is no eason o conce n. Howe e , i a es shows subs an ially highe a iabili y on one sys em han on o he s, his may be a eason o he sys em adminis a o s o in es iga e, as i may indica e issues wi h a subse o he nodes, o an o e load on sha ed esou ces (e.g. pa allel ilesys em, ne wo k conges ion e c.). 4.2.2 Pe o mance pa e ns and implica ions Figu e 10: Pe o mance (ns pe day, highe is be e ) s ime o GROMACS applica ion benchma k on Snellius CPU pa i ion (Genoa 9654) execu ed on a scale o 1 node. Time spans om 01-07-2024 ill 09-05-2025. A conc e e example o whe e he dashboa d clea ly indica ed a sys ema ic p oblem on a sys em (in his case: Snellius) can be seen in Figu e 10. The single node and wo node GROMACS pe o mance had d opped du ing he ime pe iod Augus o Sep embe 2024 on he Snellius’ genoa CPU pa i ion. This d op in pe o mance was caused by: • A secu i y mi iga ion ha was applied a he i mwa e le el and which included an upg ade o he ope a ing sys em om RHEL 8 o RHEL 9. The secu i y mi iga ion was only applicable on Zen4 ha dwa e, and hus only implemen ed he e. • Fi mwa e ela ed issues which equi ed a ull sys em eboo . These p oblems we e iden i ied in he o de hey a e men ioned abo e. The secu i y mi iga ion ela ed p oblem can also be seen in Figu e 11 whe e he pe o mance d op and he eco e y a e he ix in Janua y 2025 can be clea ly iden i ied. I can also be seen om Figu e 12 ha he d op inpe o mance was no ne wo k ela ed since he in e node pe o mance didn’ show any deg ada ion. I is o be no ed ha some da a o he OSU es s is missing he e, mos likely because he es s we e no unning. The i mwa e ela ed p oblem was ixed in Feb ua y 2025 and once he sys em was eboo ed, he GROMACS pe o mance e u ned o no mal. A ac o ha also has an e ec on pe o mance is he binding o Message Passing In e ace (MPI) anks and OpenMP h eads. In he elease o he es sui e du ing July 2024, p ope binding was en o ced wi hin he es sui e o he Mul iXscale Deli e able 5.3 Page 13 Figu e 11: Bandwid h (MB pe second, highe is be e ) s ime o poin o poin es o OSU Mic obenchma ks applica ion on Snellius CPU pa i ion (Genoa 9654) execu ed wi hin a node. Time spans om 01-01-2024 ill 01-03- 2025. sys ems by dis inguishing sys ems whe e hype - h eading is enabled and whe e i is no . I hype - h eading is enabled, hen he es s can be w i en such ha one can pin a ask on each ha dwa e h ead o on each physical co e based on he op ion ha is chosen in he es . This change was adop ed in he Vega sys em la e a ound Augus 2024 and a s a k imp o emen in Ex ensible Simula ion Package o Resea ch on So Ma e Sys ems (ESPResSo) pe o mance (lowe is be e ) can be seen in Figu es 13 and 14. In he ESPResSo es , one ask pe physical co e is chosen due o which he numbe o asks pe node was hal ed. The pinning also esul ed in a mo e consis en pe o mance (less a iance), which can be clea ly seen in he igu es. The educ ion in a iance can also be a ibu ed o less OS ji e and cache misses compa ed o he si ua ion whe e all hype - h eads we e occupied by he mig a ing MPI p ocesses. 4.3 Ha dwa e based compa ison An impo an u ili y o his sys em is o compa e he pe o mance ac oss a ious sys ems, including he cloud. I is impo an o no e ha he es sui e is no ine- uned o achie e maximum pe o mance on a gi en sys em (i is no a benchma k sui e), bu he es s a e designed o show indica i e pe o mance on all sys ems since he same es s a e un on each sys em. The easons we call i indica i e a e he ollowing: • The so wa e ins alled wi hin he EESSI so wa e s ack is op imized o ha ha dwa e. • The pinning o he MPI asks can be con olled wi hin indi idual es s ia he unc ions p esen in he Mixin class which in u n con ols he schedule op ions using en i onmen a iables. This can be a he h ead le el (assuming hype - h eading is enabled), physical CPU le el o he socke le el. This ich o e iew gi es he use an indica ion as o which ha dwa e hei applica ion pe o ms he bes . Se e al ie 0, 1 and 2 sys ems unning on simila ha dwa e should also ge an indica ion i he applica ions wi hin hei sys ems pe o m op imally on a gi en se o ha dwa e. I no , hen hey can app oach he espec i e sys em main aine s o ad- minis a o s o exchange in o ma ion ega ding he se ings ha each o hem applies o achie e his, which p omo es u he collabo a ion. As an example, he Tenso low applica ion om he es sui e execu ed on a ious CPU pa i ions is shown in Figu es 15 and 16. He e we also compa e pe o mance on a ious ARM a chi ec u es a ailable on AWS clus e . E en wi h E he ne based in e connec , he pe o mance almos doubles om one o wo nodes, which shows ha he es i sel is no ne wo k dependen bu is mo e memo y dependen wi h highes pe o mance epo ed on Snellius-genoa pa - i ion which has a AMD Zen4 based a chi ec u e. Ano he in e es ing poin is ha he CPU pa i ions Ka olina-qcpu, Vega and Snellius- ome ha e he same CPU ha dwa e, namely AMD 7H12 bu in e es ingly he a e age pe o mance is di e en , Vega pe o ming 15 pe cen be e some imes on a single node bu on wo nodes he Snellius- ome pe - Mul iXscale Deli e able 5.3 Page 14 Figu e 12: Bandwid h (MB pe second, highe is be e ) s ime o poin o poin es o OSU Mic obenchma ks applica ion on Snellius CPU pa i ion (Genoa 9654) execu ed ac oss 2 nodes. Time spans om 01-01-2024 ill 01-03- 2025. o mance seems o be he bes (apa om a ew blips due o sys em ela ed issues). This could be due o di e ences in clock equencies se by he sys em admins and may also be due o di e ences in ne wo k ha dwa e employed by he sys ems. Mul iXscale Deli e able 5.3 Page 15 Figu e 13: Pe o mance (seconds pe s ep, lowe is be e ) s ime o he P3M es o ESPResSo applica ion on Vega CPU pa i ion (Rome 7H12) execu ed on 1 node. Time spans om 01-01-2024 ill 04-01-2025. Figu e 14: Pe o mance (seconds pe s ep, lowe is be e ) s ime o he LJ es o ESPResSo applica ion on Vega CPU pa i ion (Rome 7H12) execu ed on 1 node. Time spans om 01-01-2024 ill 04-01-2025. Mul iXscale Deli e able 5.3 Page 16 Figu e 15: Pe o mance (images pe second, highe is be e ) s ime o Tenso Flow applica ion on a ious CPU pa i ions on he cloud (AWS), Eu oHPC sys ems (Vega and Ka olina), Snellius (Du ch Tie -1 sys em) execu ed on 1 node. Time spans om 01-01-2025 ill 19-05-2025. Figu e 16: Pe o mance (images pe second, highe is be e ) s ime o Tenso Flow applica ion on a ious CPU pa i ions on he cloud (AWS), Eu oHPC sys ems (Vega and Ka olina), Snellius (Du ch Tie -1 sys em) execu ed on 2 nodes. Time spans om 01-01-2025 ill 19-05-2025. Mul iXscale Deli e able 5.3 Page 17 5 Conclusion and ou look A unc ional es se up and pipeline is de eloped, deploying he es sui e de eloped wi hin Task 1.3 in a pe iodic manne , ac oss a ious Tie 0, Tie 1 and Tie 2 sys ems. The pipeline in ol es pushing he esul s o he pe o med es in o a public dashboa d. The es esul s in his dashboa d p o ide alue no only o main aine s o he EESSI sha ed so wa e s ack, bu also o sys em adminis a o s and end-use s. As illus a ed by he examples in his deli e able, he dashboa d can be used o moni o p oblems wi hin he EESSI so wa e s ack, bu also iden i y issues in he unde lying sys ems hemsel es, which can hen be esol ed by he espec i e sys em adminis a o s. Mul iXscale Deli e able 5.3 Page 18 Re e ences Ac onyms used AWS Amazon Web Se ices CI Con inuous In eg a ion EESSI Eu opean En i onmen o Scien i ic So wa e Ins alla ions HPC High Pe o mance Compu ing MPI Message Passing In e ace JSON Ja aSc ip Objec No a ion Eu oHPC Eu opean High Pe o mance Compu ing Join Unde aking VM Vi ual Machine SIMD Single Ins uc ion Mul iple Da a Da abase VM Da abase Vi ual Machine Dashboa d VM Dashboa d Vi ual Machine ES Elas icsea ch SRC SURF Resea ch Cloud API Applica ion P og amming In e ace So wa e men ioned ESPResSo Ex ensible Simula ion Package o Resea ch on So Ma e Sys ems GROMACS GROningen MAChine o Chemical Simula ion LAMMPS La ge-scale A omic/Molecula Massi ely Pa allel Simula o URLs e e enced Page ii h ps://www.mul ixscale.eu ... h ps://www.mul ixscale.eu h ps://www.mul ixscale.eu/deli e ables ... h ps://www.mul ixscale.eu/deli e ables In e nal P ojec Managemen Link ... h ps://gi hub.com/mul ixscale/planning/issues/40 [email p o ec ed] ... mail o:[email p o ec ed] maksim.mas e o @su .nl ... mail o:[email p o ec ed] h p://c ea i ecommons.o g/licenses/by/4.0 ... h p://c ea i ecommons.o g/licenses/by/4.0 Page 5 h ps://dashboa d.eessi.io ... h ps://dashboa d.eessi.io h ps://eessi.io/docs/ es sui e/dashboa d/ ... h ps://eessi.io/docs/ es sui e/dashboa d inges ion sc ip ... h ps://gi hub.com/EESSI/dashboa d-inges ion Page 6 Vue ... h ps:// uejs.o g Nginx ... h ps://nginx.o g/en/docs/beginne s_guide.h ml Ja aSc ip ... h ps://de elope .mozilla.o g/en-US/docs/Web/Ja aSc ip TypeSc ip ... h ps://www. ypesc ip lang.o g D3 ... h ps://d3js.o g Elas ic ... h ps://www.elas ic.co/licensing/elas ic-license dis ibu ion ... h ps://gi hub.com/EESSI/dashboa d-inges ion/blob/main/docs/se up_es_on_cloud. md Page 7 Nginx ... h ps://nginx.o g/en/docs/beginne s_guide.h ml Page 11 GROMACS issue ... h ps://gi lab.com/g omacs/g omacs/-/issues/5057 Ci a ions [1] T. Röbli z and K. Hos e, “D5.1 - communi y con ibu ion policy and gi hub app,” Jan. 2024. [Online]. A ailable: h ps://doi.o g/10.5281/zenodo.10451793