Po able es sui e o sha ed so wa e s ack
Mul iXscale Deli e able 1.5
Deli e able Type: Repo
Deli e ed in June, 2025
Mul iXscale
Eu oHPC Cen e o Excellence o
Mul iscale Modelling
Acknowledgemen
Funded by he Eu opean Union. This wo k has ecei ed unding om he Eu opean High Pe o mance Compu ing Join
Unde aking (JU) unde g an ag eemen No 101093169.
Disclaime
Funded by he Eu opean Union. Views and opinions exp essed a e howe e hose o he au ho (s) only and do no necessa ily
e lec hose o he Eu opean Union o he Eu opean High Pe o mance Compu ing Join Unde aking (JU). Nei he he Eu opean
Union no he g an ing au ho i y can be held esponsible o hem.
Mul iXscale Deli e able 1.5 Page ii
P ojec and Deli e able In o ma ion
P ojec Ti le Mul iXscale: Eu oHPC Cen e o Excellence o Mul iscale Modelling
P ojec Re . G an Ag eemen 101093169
P ojec Websi e h ps://www.mul ixscale.eu
Eu oHPC P ojec O ice D . Ma eo Mascagni
Deli e able ID D1.5
Deli e able Na u e Repo
Dissemina ion Le el Public
Con ac ual Da e o Deli e y P ojec Mon h 30 (30 h June, 2025)
Ac ual Da e o Deli e y 27 h June, 2025
Desc ip ion o Deli e able Final epo on he po able es sui e o he sha ed so wa e s ack
Documen Con ol In o ma ion
Documen
Ti le: Po able es sui e o sha ed so wa e s ack
ID: D1.5
Ve sion: As o June, 2025
S a us: Accep ed by S ee ing Commi ee
A ailable a : h ps://www.mul ixscale.eu/deli e ables
Documen his o y: In e nal P ojec Managemen Link
Re iew Re iew S a us: Re iewed
Au ho ship
W i en by: Caspa an Leeuwen (SURF)
Con ibu o s: Kenne h Hos e (UGen ), La a Pee e s (UGen ), Sa ish Kama h (SURF)
Re iewed by: Kenne h Hos e (UGen ), Sa ish Kama h (SURF)
App o ed by: Alan O’Cais (UB)
Documen Keywo ds
Keywo ds: Mul iXscale, HPC, so wa e, applica ions , es ing
27 h June, 2025
Disclaime : This deli e able has been p epa ed by he esponsible Wo k Package o he P ojec in acco dance wi h he
Conso ium Ag eemen and he G an Ag eemen . I solely e lec s he opinion o he pa ies o such ag eemen s on a
collec i e basis in he con ex o he P ojec and o he ex en o eseen in such ag eemen s.
Copy igh no ices: This deli e able was co-o dina ed by Caspa an Leeuwen1(SURF) on behal o he Mul iXscale
conso ium wi h con ibu ions om Kenne h Hos e (UGen ), La a Pee e s (UGen ), Sa ish Kama h (SURF) . This wo k
is licensed unde he C ea i e Commons A ibu ion 4.0 In e na ional License. To iew a copy o his license, isi :
h p://c ea i ecommons.o g/licenses/by/4.0
cb
1caspa [email p o ec ed]
Mul iXscale Deli e able 1.5 Page iii
Con en s
Execu i e Summa y 1
1 In oduc ion 2
1.1 Scope o he deli e able ................................................ 2
1.2 Ta ge audience o he deli e able . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Deli e able ou line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Pa ne con ibu ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2 S uc u e o he es sui e 3
2.1 ReF ame es classes .................................................. 3
2.2 EESSI es sui e logic .................................................. 3
2.3 Re ame con igu a ion iles .............................................. 3
2.4 Facili a ing pe iodic uns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 Po abili y 4
3.1 O e coming he po abili y challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2 Con igu ing he EESSI es sui e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 EESSI Mixin class .................................................... 5
4 O he no able ea u es 7
4.1 Decla ing es memo y equi emen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.2 Memo y usage epo ing ................................................ 7
4.3 P ocess binding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.4 Imp o ed logging .................................................... 7
4.5 Flexible ini ializa ion .................................................. 8
4.6 Mo e e icien ly handling o s aged iles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
4.7 Imp o ed suppo o hype h eading sys ems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
5 Suppo ed applica ions 9
6 Communi y building and sus ainabili y 10
7 Conclusion and ou look 11
A The EESSI es sui e GROMACS es 12
B The u o ial es : mpi4py 13
Re e ences 17
Mul iXscale Deli e able 1.5 Page 1
Execu i e Summa y
The Eu opean En i onmen o Scien i ic So wa e Ins alla ions (EESSI) es sui e consis s o h ee main pa s: EESSI
es sui e logic, he es classes hemsel es, and some auxila y iles (con igu a ion, sc ip s easily enabling pe iodic uns
o he es sui e, e c).
One o he main ocus poin s in de eloping he EESSI es sui e was o ensu e po abili y o es s. ReF ame es s
adi ionally con ain a la ge amoun o sys em-speci ic in o ma ion. The pa adigm ollowed by he EESSI es sui e
is ha all sys em speci ic in o ma ion is limi ed o he ReF ame con igu a ion iles ha desc ibe he sys em, so ha
he es classes hemsel es only con ain gene alizable in o ma ion. As an example: ins ead o speci ying a ha dcoded
numbe o asks o un a es wi h ( he adi ional ReF ame app oach), es s in he EESSI mixin class may decla e ha
hey ’ un on a ull node, wi h one ask pe co e’. The EESSI Mixin class hen eads he con igu a ion o he sys em i is
unning on, and ansla es his o a conc e e ask coun .
While he implemen a ion o he EESSI Mixin class was one o he key changes since he publica ion o D1.2, se e al
o he ea u es we e added as well ha acili a e es de elopmen (e.g. memo y usage epo ing), imp o e es pe -
o mance (e.g. p ocess binding, imp o ed suppo o hype h eading sys ems) o imp o e usabili y (lowe chance
o ou -o -memo y e o s since es s decla e hei memo y equi emen , imp o ed logging, being able o un he es
sui e on a local module s ack, mo e e icien s aging).
A o al o 10 applica ions a e now suppo ed in he es sui e. To ge be e co e age, communi y con ibu ions a e
essen ial. Thus, subs an ial a en ion has been paid o aspec s ha may inc ease communi y up ake. This includes
ying o make es de elopmen simple , simpli y unning he es sui e on a local so wa e s ack, and ou each in he
o m o a hands-on session.
In summa y, he EESSI es sui e p o ides a po able es sui e wi h key applica ions ha a e impo an o Mul iXscale.
I has sol ed he po abili y challenge h ough he implemen a ion o he EESSI Mixin class, and po abili y was p o en
by unning ac oss a wide ange o sys ems.
Mul iXscale Deli e able 1.5 Page 2
1 In oduc ion
1.1 Scope o he deli e able
This deli e able desc ibes he de elopmen o he EESSI es sui e: a po able es sui e c ea ed o es he so wa e
in he sha ed so wa e s ack. I also discusses how he es sui e is employed in p ac ice, and ou e o s o a ac
communi y con ibu ions. The wo k p esen ed he e was execu ed in he con ex o Task 1.3 - "Design and c ea ion o
a so wa e es sui e and acili a ing Con inuous In eg a ion (CI) o so wa e de elope s".
1.2 Ta ge audience o he deli e able
This deli e able is a ge ed a eade s wi h expe ience in High Pe o mance Compu ing (HPC) sys em suppo , pa ic-
ula ly in he con ex o main aining and es ing so wa e s acks (o speci ic applica ions) o HPC in as uc u e.
1.3 Deli e able ou line
Sec ion 2desc ibes he (code) s uc u e o he EESSI es sui e. Sec ion 3desc ibes how we achie e po abili y, and dis-
cuss wo impo an componen s ha p o ide po abili y ( he ReF ame con igu a ion ile, and he EESSI Mixin class).
Sec ion 4discusses o he key ea u es o he EESSI es sui e. In sec ion 5we lis he applica ions ha a e cu en ly
suppo in he EESSI es sui e. Sec ion 6discusses wha has been done o s imula e up ake o he EESSI es sui e by
he communi y in o de o os e sus ainabili y.
1.4 Pa ne con ibu ions
SURF and UGen con ibu ed as planned o he wo k in his deli e able. No e ha Task 1.3 had o he pa ne s (RIJK-
SUNIGRON, Bsc), bu hose we e ne e en isioned o con ibu e o he EESSI es sui e i sel - hey de eloped o he
componen s wi hin Task 1.3.

Mul iXscale Deli e able 1.5 Page 3
2 S uc u e o he es sui e
In his sec ion, we discuss he s uc u e o he es sui e. Unde s anding he s uc u e o he es sui e is help ul in
unde s anding how he es sui e wo ks.
The key di ec o y and ile s uc u e o he es sui e eposi o y [1] is as ollows:
1CI/< si e >_<sys em_name>/ci_con ig . sh
2CI/ un_ e ame . sh
3CI/ un_ e ame_w appe . sh
4con ig/< si e >_<sys em_name>.py
5eessi / es s u i e / e s s /
6ees si / es s ui e /common_con ig . py
7eessi / es s ui e / cons an s . py
8eessi / es s ui e / eessi_mixin . py
9eessi / es s ui e /hooks . py
10 ees si / es su i e / u i l s . py
2.1 ReF ame es classes
The co e pa o he es sui e a e he es de ini ions and esou ces unde eessi/ es sui e/ es s/. Each es is essen ially a
class de ini ion ha inhe i s om a ReF ame es class. Some es s come wi h esou ces, e.g. inpu iles, which a e also
s o ed in his subdi ec o y. Typically, es s o a single applica ion a e de ined in a single py hon ile, bu mul iple es
classes may be de ined o a single applica ion. Fo example, he ile eessi/ es sui e/ es s/apps/osu.py includes es s o he
OSU Mic obenchma ks bo h o poin - o-poin and collec i e communica ion, each o which is de ined in a sepa a e
es class.
2.2 EESSI es sui e logic
The EESSI es sui e is no a s anda d ReF ame es sui e: i implemen s addi ional logic and con igu a ion in o -
de o achie e po abili y. This is achie ed h ough eessi/ es sui e/common_con ig.py,eessi/ es sui e/cons an s.py,eessi/ es sui e/
eessi_mixin.py,eessi/ es sui e/hooks.py and eessi/ es sui e/u ils .py.
2.3 Re ame con igu a ion iles
The con ig di ec o y con ains a numbe o con igu a ion iles o sys ems on which he EESSI es sui e has been de-
ployed in he con ex o he Mul iXscale p ojec . These se e bo h as examples o o he use s, and a he same ime
allows us o ha e e sion-con olled con igu a ion iles.
2.4 Facili a ing pe iodic uns
E e y hing unde he CI di ec o y is in ended o make i easy o se up pe iodic (e.g. daily o weekly) uns on a sys-
em. The CI/<si e>_<sys em_name>/ci_con ig.sh sc ip allows one o se a con igu a ion o hese pe iodic uns. Fo example,
hese con igu a ion i ems can be used o de e mine which e sion o he es sui e o un, which e sion o ReF ame
o use, which a gumen s o pass o ReF ame (e.g. o unning a subse o he es s), e c. The CI/ un_ e ame.sh and
un_ e ame_w appe .sh sc ip s a e he main d i e s o hese pe iodic uns. Based on he con igu a ion, hey c ea e in-
s alla ions o ReF ame, he es sui e, and unning e e y hing wi h he desi ed a gumen s. I also se s de aul s o all
con igu a ion i ems no explici ly de ined in ci_con ig.sh.
Wi h his, se ing up a pe iodic un as e.g. a c onjob can be as simple as:
1. Cloning he CI subdi ec o y o he eposi o y;
2. C ea ing a ci_con ig.sh con ig ile o he sys em (i none is p esen ye in he es -sui e eposi o y);
3. Adding a line like
0 0 ***EESSI_CI_SYSTEM_NAME=su _snellius $HOME/ es −sui e/CI/ un_ e ame_w appe .sh
o you c on ab ile.
Mul iXscale Deli e able 1.5 Page 4
3 Po abili y
3.1 O e coming he po abili y challenge
In o de o c ea e a po able es sui e, i is o pa amoun impo ance ha sys em-speci ic in o ma ion and es -speci ic
in o ma ion is clea ly sepa a ed. In he EESSI es sui e, ha is achie ed by making su e ha all sys em-speci ic in o -
ma ion is p o ided h ough he ReF ame con igu a ion ile, while all he es -speci ic in o ma ion is con ained wi hin
he (ReF ame) es classes.
This means ha es s should be de eloped in such a way ha hey can ake he in o ma ion p o ided in he ReF ame
con igu a ion ile, and do some hing sensible based on ha . Fo example, i a es equi es 200 GB o memo y o
un, bu he ReF ame con igu a ion ile s a es ha a gi en pa i ion only p o ides 100 GB, he es should be skipped.
Ano he example would be a p og am ha uses pu e MPI pa allelism, whe e one ypically wan s o launch one ask
pe a ailable physical co e. The es should hen ead he numbe o a ailable co es om he ReF ame con igu a ion
ile, and de e mine he numbe o asks o be launched o he es acco dingly.
No e ha his app oach o achie ing po abili y is su icien because his is a es sui e, no a benchma k sui e. A
es sui e is mean o allow iden i ica ion o (pe o mance) eg essions, whe eas a benchma k is mean o show he
absolu e-bes pe o mance one can achie e o a gi en applica ion on a gi en sys em. The la e ypically equi es
manual uning. Fo example, a ask binding s a egy ha is op imal on sys em A is no necessa ily op imal on sys em
B. Fo he es sui e, we jus se abinding s a egy ha is expec ed o gi e easonable pe o mance on any sys em -
bu mo e impo an ly ha is expec ed o esul in ep oducible pe o mance: he mo e s able he pe o mance, he
smalle he pe o mance eg essions ha one can iden i y.
3.2 Con igu ing he EESSI es sui e
While he EESSI es sui e uses s anda d ReF ame con igu a ion iles, i does impose addi ional cons ain s. ReF ame
allows ce ain ields o be de ined as key- alue pai s, i.e. ee ex . The EESSI es sui e s anda dizes hese key alue
pai s h ough cons an s de ined in eessi/ es sui e/cons an s.py. This allows us o assign meaning o hose ee ex la-
bels.
Typically, he ollowing sec ions ha a e speci ic o he EESSI es sui e should be se in he ReF ame con igu a ion
ile:
1’sys ems ’ : [
2. . .
3’ pa i ions ’ : [
4{
5’p epa e_cmds ’ : [ common_eessi_ini ( ) ] ,
6’ esou ces ’ : [
7{
8’name’ : ’memo y’ ,
9’ op ions ’ : [ ’− −mem={ si ze } ’ ]
10 }
11 ’ ea u es ’ : [
12 FEATURES.CPU,
13 ] + l i s (SCALES. keys ( ) ) ,
14 ’ ex as ’ : {
15 EXTRAS.MEM_PER_NODE: 229376 # in MiB
16 } ,
17 . . .
18 ]
19 } ,
20 ]
21 ] ,
22 ’ logging ’ : common_logging_con ig ( e ame_p e ix ) ,
23 ’ gene al ’ : [
24 {
25 ’ emo e_de ec ’ : T ue ,
26 **common_gene al_con ig( e ame_p e ix )
27 }
28 ] ,
• The common_eessi_ini () is a common se o ini ializa ion commands ha checks wha EESSI en i onmen is ini-
ialized on he node unning he e ame command ( ypically a login node), and makes su e he same EESSI
en i onmen is ini ialized on he ba ch nodes.
• The memo y esou ce de ines he lag ha should be passed o he gi en esou ce schedule o ask o a ce ain
amoun o memo y pe node. This can hen be used by es classes o y and eques su icien memo y o un
a es .
Mul iXscale Deli e able 1.5 Page 5
• The ea u es lis includes e.g. i a pa i ion should un CPU es s (FEATURES.CPU), GPU es s (FEATURES.GPU) o bo h,
and a wha scales. A se o de aul es scales is de ined ha anges om a single co e o 16 ull nodes, bu on
small sys ems one can pass a subse o his lis o limi he maximum size o es s being un.
• The EXTRAS.MEM_PER_NODE i em de ines he maximum amoun o memo y pe node ha a job can ask o on
ha pa i ion. The es sui e uses his in o ma ion o skip es s ha equi e mo e memo y han ha .
• The emo e_de ec : T ue i em ells ReF ame o au oma ically de ec he CPU opology on he pa i ion. ReF ame
s o es his in o ma ion in a sepa a e ile, which hen desc ibes hings like he numbe o co es pe node, numbe
o co es pe socke , numbe o o numa domains, e c.
All o he common_* unc ions a e de ined in eessi/ es sui e/common_con ig.py and make su e ha he gene al beha io o he
sys em on di e en sys ems is simila , hus p o iding p edic abili y in e.g. wha i ems a e logged, whe e, e c.
Wi h hese ew key i ems in he ReF ame con igu a ion ile, we can achie e es po abili y: we can ensu e ha es s
only un i he equi ed esou ces (GPUs, su icien memo y) a e p esen , and we can ensu e ha a sensible numbe
o asks / h ead o ha sys em a e launched (by using he de ec ed CPU opology).
No e ha addi ional ea u es and ex as may be added in u u e es sui e eleases, as needed. E.g. o now, he sha ed
so wa e s ack only has suppo o NVIDIA GPUs, and hus a gene al FEATURES.GPU is su icien ly speci ic. In he u u e,
mo e speci ic ea u es may be de ined o p o ide mo e ine-g ained con ol, e.g. ‘FEATURES.NVIDIA_GPU‘, ‘FEATURES.
AMD_GPU‘, e c.
3.3 EESSI Mixin class
In Deli e able D 1.2, a p oo o concep o a po able es o GROningen MAChine o Chemical Simula ion (GROMACS)
was p esen ed. Since hen, es s o se e al o he applica ions whe e added (Sec ion 5). As mo e es s we e added, i
became clea he e was a ai amoun o duplica ion o logic be ween hese es s. Thus, he code was e ac o ed and
he common logic be ween hese es s classes ha is needed o make he es s po able was sepa a ed in o a so-called
Mixin class. The esul is ha he es classes became mo e ligh weigh , and ocussed on con igu a ion a he han
logic. E.g. a es can decla e h ough a simple a iable assignmen ha i equi es a GPU. The logic in he EESSI Mixin
class hen makes su e his es is il e ed ou on pa i ions ha do no o e any GPUs. Fo example, he implemen a-
ion o he GROMACS es class in Appendix Ais now inhe i ing om he EESSI mixin class, and is subs an ially mo e
compac ha i was be o e (see Deli e able 1.2, Appendix A).
An addi ional ad an age o he EESSI Mixin class is ha i makes w i ing he po able es s easie , o wo easons:
1. W i ing a es class is now (mos ly) done by jus de ining he equi ed class p ope ies (i.e. con igu a ion).
2. The EESSI Mixin class has in e nal checks o make su e he inhe i ing class de ines he co ec p ope ies a
he co ec s age o he ReF ame pipeline, and ha he alues assigned o hose p ope ies a e alid. I gi es
e bose eedback i a ce ain keywo d hasn’ been assigned in a imely ashion o has an in alid alue. Thus,
e en wi hou eading he documen a ion, a es de elope is guided h ough he de elopmen p ocess.
Fo example, he EESSI Mixin class de ines he ollowing ReF ame pipeline hook ha is un a e he ’ini ’ s age o he
ReF ame pipeline:
1@ un_a e ( ’ ini ’ )
2de EESSI_mixin_ alida e_ini ( sel ) :
3"""Check ha a l l a iabl es ha ha e o be se o subsequen hooks in he i n i phase ha e been se """
4# L is which a iables we wi l l need/use in he un_a e ( ’ ini ’ ) hooks
5 a _ l i s = [ ’ de ice_ ype ’ , ’ scale ’ , ’module_name’ , ’measu e_memo y_usage ’ ]
6 o a in a _ l i s :
7i no hasa ( sel , a ) :
8msg = "The a iable ’%s ’ should be de ined in any es class ha inhe i s " % a
9msg += " om EESSI_Mixin be o e ( o in ) he i n i phase , bu i wasn’ "
10 aise Re ameFa alE o (msg)
11
12 # Check ha he alue o hese a iables i s alid ,
13 # i . e . exis s in hei espec i e dic om eessi . es sui e . cons an s
14 s e l . EESSI_mixin_ alida e_i em_in_lis ( ’ de ice_ ype ’ , DEVICE_TYPES [ : ] )
15 s e l . EESSI_mixin_ alida e_i em_in_lis ( ’ scale ’ , SCALES . keys ( ) )
16 s e l . EESSI_mixin_ alida e_i em_in_lis ( ’ alid_sys ems ’ , [ [ ’ *’ ] ] )
17 s e l . EESSI_mixin_ alida e_i em_in_lis ( ’ alid_p og_en i ons ’ , [ [ ’ de aul ’ ] ] )
Essen ially, his hook checks ha o any es inhe i ing om EESSI mixin, he class a ibu es de ice_ ype,scale, alid_sys ems
and alid_p og_en i ons ha e been de ined, and hei alue is alid. I a es de elope ails o assign one o hese key-
wo ds, o assigns an in alid alue, he EESSI Mixin class will aise a clea e o .
Subsequen ly, he EESSI Mixin class calls he ollowing ReF ame pipeline hook:
Mul iXscale Deli e able 1.5 Page 6
1@ un_a e ( ’ ini ’ )
2de EESSI_mixin_ un_a e _ini ( s e l ) :
3"""Hooks o un a e i n i phase """
4
5# F i l e on which sca les a e suppo ed by he p a i ion s de ined in he ReF ame con igu a ion
6hooks . i l e _s up po ed _s ca le s ( s e l )
7
8hooks . il e _ alid_sys ems_by_de ice_ ype ( s el , equi ed_de ice_ ype= s e l . de ice_ ype )
9
10 hooks . se _modules ( s e l )
11
12 # Se scales as ags
13 hooks . se _ ag_ sca le ( s e l )
Each o he unc ions om he hooks namespace can now sa ely assume ha he ou a o emen ioned class a ibu es
a e se .
To show how his simpli ies he es de elopmen , conside he mpi4py u o ial es ha is in he EESSI documen a ion
( he inal es om his u o ial is included in Appendix B). The only hing his class now needs o de ine o he ini
pipeline hook is:
1de ice_ ype = DEVICE_TYPES.CPU
I does no need o call any o he hooks unc ions i sel , no does i need o de ine he o he h ee p ope ies (scale,
alid_sys ems and alid_p og_en i ons): he EESSI Mixin class p o ides de aul s - o e w i ing hese is op ional.
O e all, he u o ial es is a clea example o how he EESSI Mixin class simpli ies he es : he o iginal es (be o e in-
he i ing om he EESSI Mixin class) was 43 lines o code ini ially, and is educed o only 21 lines o code (see Appendix
B) a e using he EESSI Mixin class.
Mul iXscale Deli e able 1.5 Page 13
B The u o ial es : mpi4py
He e, we p o ide he Py hon code o he EESSI mpi4py es , which se es as a u o ial in he documen a ion. Below,
we show wo e sions: i s , he e sion wi hou using he EESSI Mixin class, hen, he e sion ha inhe i s om
he EESSI Mixin class. This code is included he e o illus a e he ela i e simplici y o a es using he EESSI Mixin
class.
1impo e ame as m
2impo e ame . u i l i y . sani y as sn
3
4# added only o make he l i n e happy
5 om e ame . co e . buil ins impo a iable , pa ame e , un_a e , pe o mance_ unc ion , sani y_ unc ion
6
7 om eessi . e s sui e impo hooks
8 om eessi . e s s u i e . cons an s impo SCALES, COMPUTE_UNITS
9 om ees si . es sui e . u i l s impo ind_modules
10
11
12 # This py hon deco a o indica es o ReF ame ha his class de ines a es
13 # Ou class inhe i s om m . RunOnlyReg essionTes , since his es does no ha e a compila ion s age
14 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline .
RunOnlyReg essionTes
15 @ m. simple_ es
16 class EESSI_MPI4PY( m . RunOnlyReg essionTes ) :
17 # P og amming en i onmen s a e only ele an o e s s ha compile some hing
18 # Since we a e es ing exis ing modules , we ypically don’ compile any hing and simply de ine
19 # ’ de aul ’ as he alid p og amming en i onmen
20 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
alid_p og_en i ons
21 alid_p og_en i ons = [ ’ de aul ’ ]
22
23 # Typically , we l i s he e he name o ou clus e as i is speci ied in ou ReF ame con igu a ion i l e
24 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
alid_sys ems
25 alid_sys ems = [ ’ *’ ]
26
27 # ReF ame w i l l gene a e a es o each module
28 # NOTE: each pa ame e adds a new dimension o he pa ame iza ion space .
29 # (EG 4 pa ame e s wi h (3 ,3 ,2 ,2) possible alues wi l l esul in 36 es s ) .
30 # Be mind ul o how many pa ame e s you add o a oid he numbe o e s s gene a ed being excessi e .
31 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins . pa ame e
32 module_name = pa ame e ( ind_modules ( ’ mpi4py ’ ) )
33
34 # ReF ame w i l l gene a e a es o each scale
35 scale = pa ame e (SCALES . keys ( ) )
36
37 # Ou sc ip has wo a gumen s , −−n_i e and −−n_wa mup. By de ining hese as ReF ame a iables , we can
38 # enable he end−use o o e w i e hei alue on he command line when in oking ReF ame.
39 # No e ha we don’ ypically expose ALL a iables , especially i a sc ip has many − we expose
40 # only hose ha we hink an end−use migh wan o o e w i e
41 # Numbe o i e a ion s o un (mo e i e a ions akes longe , bu es ul s in mo e accu a e iming )
42 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins . a iable
43 n_i e a ions = a iable ( in , alue=1000)
44
45 # Simila o he numbe o wa mup i e a io ns
46 n_wa mup = a iable ( in , alue=100)
47
48 # De ine which execu able o un
49 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
execu able
50 execu able = ’py hon3 ’
51
52 # De ine which op ions o pass o he execu able
53 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
execu able_op s
54 execu able_op s = [ ’ mpi4py_ educe . py ’ , ’−− n_i e ’ , ’ { n_i e a ions } ’ , ’−−n_wa mup’ , ’ {n_wa mup} ’]
55
56 # Tempo a ily de ine pos un_cmds o make i easy o ind ou memo y usage
57 pos un_cmds = [
58 # o cg oups 1
59 ’MAX_MEM_IN_BYTES=$( </ sys / s /cg oup/memo y/$( </p oc/ s el /cpuse ) / . . /memo y. max_usage_in_by es ) ’ ,
60 # o cg oups 2
61 # ’MAX_MEM_IN_BYTES=$(</ sys / s /cg oup/$( </p oc/ se l /cpuse ) / . . / . . / . . / memo y. peak ) ’ ,
62 ’echo "MAX_MEM_IN_BYTES=$MAX_MEM_IN_BYTES" ’ ,
63 ’echo "MAX_MEM_IN_MIB=$ ( ($MAX_MEM_IN_BYTES/1048576) ) " ’
64 ]

Mul iXscale Deli e able 1.5 Page 14
65
66 # De ine a ime limi o he schedule unning his es
67 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
ime_limi
68 ime_limi = ’5m00s’
69
70 @ un_a e ( ’ ini ’ )
71 de se _modules ( s e l ) :
72 hooks . se _modules ( s e l )
73
74 # Using hi s deco a o , we e l l ReF ame o un hi s AFTER he i n i s ep o he es
75 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins . un_a e
76 # See h ps : // e ame−hpc . ead hedocs . io /en/ s able / pipeline . h ml o a l l s eps in he pipeline
77 # ha e ame uses o execu e es s .
78 @ un_a e ( ’ ini ’ )
79 de u n_a e _i ni ( s e l ) :
80 hooks . se _ ag_ sca le ( s e l )
81
82 @ un_a e ( ’ se up ’ )
83 de se _num_ asks_pe _node( sel ) :
84 """ Se ing numbe o asks pe node and cpus pe ask in his unc ion . This unc ion se s
85 num_ asks , num_ asks_pe _node , num_cpus_pe _ ask , and num_gpus_pe _node , based on he cu en scale
86 and he cu en pa i ion ’ s num_cpus, max_a ail_gpus_pe _node and num_nodes"""
87 hooks . assign_ asks_pe _compu e_uni ( sel , COMPUTE_UNITS.CPU)
88
89 # This es scales almos i nde ini ely
90 # Fo es s ha ha e limi ed scaling , make su e ha e s ins ances exceeding
91 # a p ede ined maximum ask coun a e skipped using :
92 # max_ asks = 300
93 # s e l . s k ip _i ( s e l . num_ asks > max_ asks ,
94 # ’ Skipping e s : mo e han { max_ asks } asks a e eques ed ( { s e l . num_ asks } ) ’ )
95
96 # Make su e we eques s u icien memo y om he schedule
97 @ un_a e ( ’ se up ’ )
98 de eques _mem ( s e l ) :
99 mem_ equi ed = s e l . num_ asks_pe _node *256 # eques 256 MB pe ask pe node
100 hooks . eq_memo y_pe _node( sel , app_mem_ eq=mem_ equi ed)
101
102 # Se binding s a egy
103 @ un_a e ( ’ se up ’ )
104 de se _binding ( s e l ) :
105 hooks . se _compac _p ocess_binding ( s e l )
106
107 # Now, we check i he pa e n ’Sum o a l l anks : X’ wi h X he co ec sum o he amoun o anks i s ound
108 # in he s anda d ou pu :
109 # h ps :// e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . b uil ins . sani y_ unc ion
110 @sani y_ unc ion
111 de al id a e ( s e l ) :
112 # Sum o 0 , . . . , N−1 is (N *(N−1) / 2)
113 sum_o _ anks = ound ( s e l . num_ asks *( ( s e l . num_ asks − 1) / 2) )
114 # h ps :// e ame−hpc . ead hedocs . io /en/ s able / de e able_ unc ions_ e e ence . h ml# e ame . u i l i y . sani y .
asse _ ound
115 e u n sn . asse _ ound ( ’Sum o a l l anks : %s ’ % sum_o _ anks , s e l . s dou )
116
117 # Now, we de ine a pa e n o ex ac a numbe ha e l ec s he pe o mance o his es
118 # h ps :// e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . b uil ins .
pe o mance_ unc ion
119 @pe o mance_ unc ion ( ’ s ’ )
120 de ime ( s e l ) :
121 # h ps :// e ame−hpc . ead hedocs . io /en/ s able / de e able_ unc ions_ e e ence . h ml# e ame . u i l i y . sani y .
ex ac single
122 e u n sn . e x ac s ing le ( ’^Time elapsed : s +(?P<pe > S+) ’ , s e l . s dou , ’ pe ’ , l o a )
123
124 @pe o mance_ unc ion ( ’MiB’ )
125 de max_mem_in_mib( s e l ) :
126 e u n sn . e x ac s ing le ( ’^MAX_MEM_IN_MIB=(?P<pe > S+) ’ , s e l . s dou , ’ pe ’ , i n )
1impo e ame as m
2impo e ame . u i l i y . sani y as sn
3
4# added only o make he l i n e happy
5 om e ame . co e . buil ins impo a iable , pa ame e , pe o mance_ unc ion , sani y_ unc ion
6
7# Impo he EESSI_Mixin class so ha we can inhe i om i
8 om eessi . e s s u i e . eessi_mixin impo EESSI_Mixin
9 om eessi . e s s u i e . cons an s impo COMPUTE_UNITS, DEVICE_TYPES
10 om ees si . es sui e . u i l s impo ind_modules
Mul iXscale Deli e able 1.5 Page 15
11
12
13 # This py hon deco a o indica es o ReF ame ha his class de ines a es
14 # Ou class inhe i s om m . RunOnlyReg essionTes , since his es does no ha e a compila ion s age
15 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline .
RunOnlyReg essionTes
16 @ m. simple_ es
17 class EESSI_MPI4PY( m . RunOnlyReg essionTes , EESSI_Mixin ) :
18
19 # The de ice ype makes su e his es only ge s execu ed on sys ems/ pa i ions ha can p o ide his de ice
20 de ice_ ype = DEVICE_TYPES.CPU
21
22 # One ask is launched pe compu e uni . In his case , one ask pe ( physical ) CPU co e
23 compu e_uni = COMPUTE_UNITS.CPU
24
25 # ReF ame w i l l gene a e a es o each module ha ma ches he egex ‘mpi4py‘
26 # This means we impli ci ly assume ha any module ma ching his name p o ides he equi ed unc ionali y
27 # o un his es
28 module_name = pa ame e ( ind_modules ( ’ mpi4py ’ ) )
29
30 # Ou sc ip has wo a gumen s , −−n_i e and −−n_wa mup. By de ining hese as ReF ame a iables , we can
31 # enable he end−use o o e w i e hei alue on he command line when in oking ReF ame.
32 # No e ha we don’ ypically expose ALL a iables , especially i a sc ip has many − we expose
33 # only hose ha we hink an end−use migh wan o o e w i e
34 # Numbe o i e a ion s o un (mo e i e a ions akes longe , bu es ul s in mo e accu a e iming )
35 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins . a iable
36 n_i e a ions = a iable ( in , alue=1000)
37
38 # Simila o he numbe o wa mup i e a io ns
39 n_wa mup = a iable ( in , alue=100)
40
41 # De ine which execu able o un
42 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
execu able
43 execu able = ’py hon3 ’
44
45 # De ine which op ions o pass o he execu able
46 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
execu able_op s
47 execu able_op s = [ ’ mpi4py_ educe . py ’ , ’−− n_i e ’ , ’ { n_i e a ions } ’ , ’−−n_wa mup’ , ’ {n_wa mup} ’]
48
49 # De ine a ime limi o he schedule unning his es
50 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . pipeline . Reg essionTes .
ime_limi
51 ime_limi = ’5m00s’
52
53 # De ine he benchma ks ha a e a ailable in he es .
54 # In his es ( ‘ EESSI_MPI4PY ‘ ) he e is only one benchma k . I he e a e mo e han one ,
55 # de ine hem using he ‘ pa ame e ( ) ‘ unc ion .
56 bench_name = ’mpi4pi ’
57
58 # Speci y he benchma k o be es ed in CI ( wil l be ma ked wi h a ‘CI ‘ ag ) .
59 bench_name_ci = ’mpi4pi ’
60
61 # De ine he i l e s and/o di s inside sou cesdi ( de aul =s c ) ha should be symlinked in o he s age di
62 eadonly_ iles = [ ’ mpi4py_ educe . py ’ ]
63
64 # De ine he class me hod ha e u ns he equi ed memo y pe node
65 de equi ed_mem_pe _node( sel ) :
66 e u n s e l . num_ asks_pe _node *100 + 250
67
68 # Now, we check i he pa e n ’Sum o a l l anks : X’ wi h X he co ec sum o he amoun o anks i s ound
69 # in he s anda d ou pu :
70 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins . sani y_ unc ion
71 @sani y_ unc ion
72 de al id a e ( s e l ) :
73 # Sum o 0 , . . . , N−1 is (N *(N−1) / 2)
74 sum_o _ anks = ound ( s e l . num_ asks *( ( s e l . num_ asks − 1) / 2) )
75 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / de e able_ unc ions_ e e ence . h ml# e ame . u i l i y . sani y .
asse _ ound
76 e u n sn . asse _ ound ( ’Sum o a l l anks : %s ’ % sum_o _ anks , s e l . s dou )
77
78 # Now, we de ine a pa e n o ex ac a numbe ha e l e c s he pe o mance o his es
79 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / eg ession_ es _api . h ml# e ame . co e . buil ins .
pe o mance_ unc ion
80 @pe o mance_ unc ion ( ’ s ’ )
Mul iXscale Deli e able 1.5 Page 16
81 de ime ( s e l ) :
82 # h ps :/ / e ame−hpc . ead hedocs . io /en/ s able / de e able_ unc ions_ e e ence . h ml# e ame . u i l i y . sani y .
ex ac single
83 e u n sn . e x ac s ing le ( ’^Time elapsed : s +(?P<pe > S+) ’ , s e l . s dou , ’ pe ’ , l oa )
Mul iXscale Deli e able 1.5 Page 17
Re e ences
Ac onyms used
CI Con inuous In eg a ion
Ce nVM-FS Ce nVM File Sys em
EESSI Eu opean En i onmen o Scien i ic So wa e Ins alla ions
HPC High Pe o mance Compu ing
Eu oHPC Eu opean High Pe o mance Compu ing Join Unde aking
So wa e men ioned
GROMACS GROningen MAChine o Chemical Simula ion
URLs e e enced
Page ii
h ps://www.mul ixscale.eu ... h ps://www.mul ixscale.eu
h ps://www.mul ixscale.eu/deli e ables .. . h ps://www.mul ixscale.eu/deli e ables
In e nal P ojec Managemen Link . .. h ps://gi hub.com/mul ixscale/planning/issues/105
caspa [email p o ec ed] ... mail o:[email p o ec ed]
h p://c ea i ecommons.o g/licenses/by/4.0 ... h p://c ea i ecommons.o g/licenses/by/4.0
Page 6
mpi4py u o ial es ha is in he EESSI documen a ion ... h ps://www.eessi.io/docs/ es -sui e/w i ing-po able- es s/
#s ep-by-s ep- u o ial- o -w i ing-a-po able- e ame- es
Ci a ions
[1] “EESSI es sui e,” h ps://gi hub.com/EESSI/ es -sui e.
[2] “EESSI es sui e: a ailable es s,” h ps://www.eessi.io/docs/ es -sui e/a ailable- es s/.
[3] “EESSI es sui e documen a ion,” h ps://www.eessi.io/docs/ es -sui e/.
[4] “10 h EasyBuild Use Mee ing,” h ps://easybuild.io/eum25/.