1
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
I/O Simula ion:
F om esou ces o
complex wo kloads
Sep embe 2, 2025 –Edinbu gh, Sco land
F ed Su e
REX-IO 2025
2
E olu ion o I/O and Da a Managemen
App. Disk Tape
POSIX
Files, aw scien i ic da a Files, aw scien i ic da a
Ou pu A chi al
La ge-scale
Dis ibu ed
High Pe .
I/O amewo k
Sel -desc ibed, Reduced,
Re ac o ed, Su oga ed
Pa allel & Dis ibu ed
File Sys em
S eaming
Sel -desc ibed, Reduced,
Re ac o ed, Su oga ed
Ou pu
Re ie al
Que y
Flash s. HDD DNA s o age
QoIs, RoIs, In e ence
App. Disk Tape
POSIX
Files, aw scien i ic da a Files, aw scien i ic da a
Inpu P eload
On a sha ed and
limi ed in as uc u e
3
O e a ching Ques ions in Tha Con ex
Can we
•Build be e I/O and da a managemen so wa e o accele a e science?
•Op imize esou ce u iliza ion and handle dynamic changes?
•Reduce manual in e en ions in complex da a managemen asks?
Yes, i we can ake he igh decisions and selec he igh le e s a he igh ime
Wha is needed o ha ?
•Pe o mance models ha a e as , scalable, dynamic / in e ac i e
•And can cap u e he en i e HW/SW s ack and ha e p edic i e alue
In o he wo ds, we need a comp ehensi e Digi al Twin
4
Why Simula e I/Os and S o age?
An impo an pe o mance d i e o unde s and
•Independen o scale and ype o he compu ing in as uc u e
•As much impo an as compu ing and ne wo king
Speci ics and conce ns o s o age subsys ems may a y
•Da a Cen e s Hie a chical (mass) s o age subsys ems Di e en ypes o media in ol ed
•Supe compu e s La ge-scale dedica ed s o age ne wo k High-speed ne wo k in e connec
•Clus e s Speci ic and uned ile sys em Reliable, scalable, and simple
•G ids and Clouds Se ices o e ed by mul iple da a cen e s Hidden unde lying in as uc u e
Ve sa ili y is key!
5
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
A B ie His o y o
6
Dis ibu ed Sys ems as Scien i ic Objec s o S udy
Clus e s, supe compu e s, pee - o-pee sys ems, g ids, clouds, . . .
How o s udy hese sys ems and hei applica ions on my lap op?
7
The SimG id Toolki
Open P ojec since 1998
•2,200+ ci a ions and 665+ usages
Key s eng hs
•Usabili y: Fas , Reliable, Use -o ien ed APIs
•Valida ed pe o mance models: Open Science P edic i e Powe
•Ve sa ili y: G id, P2P, HPC, Cloud, Fog, …
A scien i ic ins umen on you lap op h ps://simg id.o g
8
SimG id His o y
2014–2025: U ilisabili y and Ex ensibili y
1998 –2001: Fac o s uden code (DAG scheduling)
Casano a, H. Simg id: a oolki o he simula ion o applica ion scheduling
2001–2005: CSP and imp o ed ne wo k models
Leg and, A., Ma chal, L., Casano a H.
Scheduling dis ibu ed applica ions: he simg id simula ion amewo k
2005–2014: Ve sa ili y, Accu acy, Scalabili y
Casano a, H., Leg and, A., Quinson, M.
Simg id: A gene ic amewo k o la ge-scale dis ibu ed expe imen s
Casano a, H., Gie sch, A., Leg and, A., Quinson, Su e , F.
Ve sa ile, scalable, and accu a e simula ion o dis ibu ed applica ions and pla o ms
SG1 SG2 SG3 SG4
Casano a H., Gie sch A. Leg and, A., Quinson M, Su e , F.
Lowe ing En y Ba ie s o De eloping Cus om Simula o s
o Dis ibu ed Applica ions and Pla o ms wi h SimG id
9
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
SimG id in a Nu shell
16
Back in 2015 –G ound T u h Da a Acquisi ion
Tes bed
−G id'5000 expe imen al pla o m (h ps://www.g id5000. )
−Th ee ypes o disk: SATA-II, SAS, and SATA/SSD
Me hodology
−Randomized FIO benchma ks
−Synch onous, non-bu e ed I/O ope a ions
−Independen : F om 32kiB o 2GiB wi h a ixed block size o 32KiB
−Concu en : 1 o 15 ope a ions
−Fo 10, 50, 100, 500, 1024, and 2048 MiB iles
h p://dx.doi.o g/10.6084/m9.gsha e.1175156
17
De i ing Models om Expe imen al Da a
SSD HDD
•Linea w. . . bandwid h
•No la ency
•He e oscedas ic beha io
−Va iabili y p opo ional o size
Concu en accesses
•Modi y esou ce capaci y as concu ency inc eases
•Ree alua e each ime a ans e begins o ends
h ps://simg id.o g/doc/la es /Calib a ing_ he_models.h ml#i-o-calib a ion
18
File Sys em Plugin, I/O S eams, and JBOD
File Sys em Plugin (ca. 2017) – Be e sepa a ion o conce ns
−SimG id models: aw by e s eams on disk ( ead/w i e bandwi h)
−Plugin: concep o ile and s anda d ope a ions, Posix-like ope a ions on ile desc ip o s
I/O s eams – Speed up simula ions
−Model [ ead] – ans e – [w i e] om a hos o ano he as a luid ac i i y
−Disk o disk, disk o memo y, o memo y o disk
−Fluid? Wo ks as i doing s o e-and- o wa d a a e y ine g ain
−I/O and Comm ac i i ies p og ess oge he a he limi ing bandwid h speed
JBOD Plugin – Modeling RAID sys ems
−New concep o compound ac i i y
−Combines se e al ac i i ies and
wai o he comple ion o he las one
19
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
I/Os in SimG id –File sys em le el
20
W ench
•P ojec ini ia ed in 2016
–Wi h R. Fe ei a da Sil a (ORNL) and H.
Casono a (UH Manoa)
•Objec i es
–A i ual lab o s udy WMS
–Imp o e SimG id exp esi eness
•DSL-like app oach:
–High le el concep s
–Composable modules
–Di e en le els o APIs
h ps://w ench-p ojec .o g
21
W ench O e iew
22
Implemen a ion
FIVES: a Simula o o High-Pe o mance S o age Sys ems
Cou esy o F. Tessie and J. Monnio
23
FIVES’ Compound S o age Se ice:
Cou esy o F. Tessie and J. Monnio
●Gene ic model o a dis ibu ed ile sys em
●Suppo s spli ing a ile in pa s and dis ibu ing i on mul iple S o age Se ices
●In eg a ed as a se ice in o WRENCH
In e nally
●File Index →F ee MDS
●Alloca o →Use p o ided alloca ion policy
−FIVES comes wi h he Lus e ound- obin/weigh ed policy
24
File Sys em Module
Mo i a ions
•Fac o de elopmen o simila capabili ies be ween SimG id and WRENCH
•Replace he old and simplis ic ile sys em plugin
Objec i e
•Implemen a simula ed ile sys em on op o SimG id
•Suppo he no ion o pa i ions ha s o e di ec o ies ha s o e iles
•S anda d ope a ions: c ea e, mo e, unlink iles, unlink di ec o ies, check o exis ence)
•Suppo he no ion o a ile desc ip o wi h POSIX-like ope a ions (open, seek, ead, w i e, close).
App oach
•De eloped as a s andalone lib a y o be used in any SimG id-based simula o
Fu u e wo k
•In eg a e he de elopmen o a Lus e ile sys em made in FIVES
h ps://gi hub.com/simg id/ ile-sys em-module
25
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
DTLMod – Ve sa ile Simula ed Da a T anspo Laye
Mo e de ails on DTLMod
Thu sday Sep 4 h
Session 7 on Pe o mance
Modelling and Op imisa ion
in Pen land oom
32
G ound- u h execu ion da a om a ge sys em
Simula o o a ge sys em
Pa ame e anges o calib a e
−I/O bandwi h, RAM page cache, WAN bandwid h, …
Loss unc ion
Calib a ion algo i hm
−G id, Random, G adien
App oach: Au oma ed Calib a ion P ocedu e
Cou esy o J. McDonald
h ps://gi hub.com/w ench-p ojec /simcal
33
De e mining Le els o De ail o Simula o s
E alua ion o
Wo k low Scheduling s a egies
G ound u h
•5 applica ion wo k lows
•5 sizes (# asks)
•5 pe - ask CPU wo k amoun s
•4 da a oo p in s
34
ORNL IS MANAGED BY UT-BATTELLE LLC
FOR THE US DEPARTMENT OF ENERGY
I/O simula ion – The g ound u h challenge
35
The e can’ be any (use ul) simula ion model wi hou da a
Applica ion
Comm.Compu ing I/O
CPU Ne wo k S o ageGPU
Resou ce manage
MPI
un ime
cuDA, HIP,
Kokkos, …
ADIOS, HDF5,
Ne CDF, …
Lus e, DAOS,
Vec o DB
LCF HPSS, DNA
MGARD, SZ,
ZFP, …
Time
Resou ces
36
Finding Rele an Da ase s is Ha d and Requi es P ocessing!
Da shan logs om The a (ANL) →Yea 2022, ~18,000 jobs
Need o il e ing and clus e ing o moni o ed jobs:
●Remo ing jobs wi h no I/O ac i i y, classi ica ion based on bw pe o mance, e c
●Reducing he e ogenei y / add essing echnical limi a ions o simula o s
Cou esy o F. Tessie and J. Monnio
37
Challenges
Pe o mance da a managemen aces he same challenges as scien i ic da a managemen
Challenge 1: How can we cap u e and e icien ly expo and s o e pe o mance da a?
Design moni o ing ools along he same p inciples as o science da a
Challenge 2: How can dispa a e in o ma ion om mul iple sou ces ega ding da a
managemen ac i i ies be used in o use ul knowledge?
Build AI su oga es o DM wo kloads
Challenge 3: How can we ep oduce Da a Managemen beha io in a con olled ashion
o “wha i ” in es iga ions?
Design mul i-scale, mul i- ideli y pe o mance e alua ion ools
And some mo e …
Da a sha ing s. policies, in eg a ion o new ad anced s o age, coo dina ion beyond DM, …
38
Conclusion
Simula ion can be used o assess he pe o mance o I/Os and da a managemen
−F om he esou ces (a.k.a. disks)
−To applica ion wo k lows
−Th ough (dis ibu ed) ile sys ems
Models and ools exis
−SimG id, FSMod, FIVES, WRENCH, DTLMod … and many o he s ou side he SimG id ecosys em
−Con ibu ions welcomed!
Bu challenges emain
•G ound u h da a acquisi ion and accessibili y
•Calib a ion o he simula ion models
•Selec ion o he app op ia e le el o de ail
39
Ques ions?
Thank you o you a en ion
