LUMI AI Fac o y Se ice Cen e
Empowe ing Eu ope’s AI Ecosys em
D4.4 Con aine ized wo k lows
2
D4.4
Con aine ized wo k lows
D4.4 Con aine ized wo k lows
3
P ojec Ti le
LUMI AI Fac o y Se ice Cen e
P ojec Ac onym
LUMI-AIF
P ojec Numbe
101234208
Type o Ac ion
HORIZON-JU-RIA
Topic
HORIZON-JU-EUROHPC-2025-AI-01-IBA-01
S a ing Da e o P ojec
01.03.2025
Ending Da e o P ojec
29.02.2028
Du a ion o he P ojec
36 mon hs
Websi e
lumi-ai- ac o y.eu
Wo k Package
WP4
Task
T4.2. P o ision o ich selec ion o eady- o-use AI ools
Lead Au ho s
Ville Ahlg en (CSC)
Con ibu o s
Juha Hulkkonen (CSC)
Tomáš Ma ino ic (IT4I)
Pauliina Somekoski (CSC)
Pee Re iewe s
Ma kus Koskela (CSC)
Ve sion
1.0.
Due Da e
29.8.2025
Submission Da e
29.8.2025
Dissemina ion le el
x
PU: Public
SEN: Sensi i e – limi ed unde he condi ions o he G an Ag eemen
EU-RES. Classi ied In o ma ion: RESTREINT UE (Commission Decision 2005/444/EC)
EU-CON. Classi ied In o ma ion: CONFIDENTIEL UE (Commission Decision 2005/444/EC)
EU-SEC. Classi ied In o ma ion: SECRET UE (Commission Decision 2005/444/EC)
D4.4 Con aine ized wo k lows
4
Ve sion His o y
Re ision
Da e
Edi o s
Commen s
0.1
5.6.2025
Pauliina
Some koski
Ville Ahlg en
Ini ial e sion
0.2
24.7.2025
Ville Ahlg en
Added bulk o he documen 's con en and
s uc u e.
0.3
25.7.2025
Juha Hulkkonen
Added desc ip ions o e ms.
0.4
4.8.2025
Ville Ahlg en
O e all polish, inalize chap e s Nex S eps
and Wo k low Files, and add chap e AI
Models, So wa e, and Use.
0.5
5.8.2025
Ville Ahlg en
Mino g amma ical ixes and es uc u ing
he Wo k low Ca alog chap e .
0.6
12.8.2025
Ville Ahlg en
Added e e ences and imp o ed se e al
sec ions based on eedback.
0.7
13.8.2025
Tomáš Ma ino ic
Added cla i ying sen ence o sec ion
Wo k lows, Con aine ized and a couple o
examples o Vision-Language wo k low
desc ip ion.
0.8
18.8.2025
Ville Ahlg en
Mino upda es o mul iple chap e s based
on ecei ed eedback.
0.9
27.8.2025
Ville Ahlg en
Changed dissemina ion le el o public, inal
e sion eady o submission.
1.0
29.8.2025
Anna Luoma
Final quali y check pe o med by he PMO,
sen o o icial e iew.
D4.4 Con aine ized wo k lows
5
Glossa y o Te ms
I em
Desc ip ion
Ai a
AI In e ence se ice de eloped by CSC.
Ai low
Da a-enginee ing o ien ed wo k low engine.
AWQ
Ac i a ion Awa e Quan iza ion is an AI model comp ession echnique, designed
o enable e icien in e ence while main aining accu acy.
Docke
Docke is a pla o m o package so wa e en i onmen s in o uni s called
con aine s.
CI/CD
CI Con inuous In eg a ion and CD Con inuous Deli e y a e p ac ices in which
code changes a e au oma ically es ed, in eg a ed, and deli e ed.
Flash A en ion
An e icien algo i hm used in la ge language models o op imize memo y
access.
GPQA
G adua e-Le el Google-P oo Ques ions and Answe s is a benchma k designed
o e alua e he easoning and p oblem-sol ing abili ies o la ge language
models.
HPC-API
High-Pe o mance Compu ing - Applica ion P og amming In e ace.
Llama
A amily o la ge language models de eloped by Me a AI.
LLM
La ge Language Model.
LUST
LUMI Use Suppo Team, esponsible o he o icial LUMI so wa e s ack and
use suppo .
Lus e
Lus e is a high-pe o mance dis ibu ed ile sys em, commonly used in la ge-
scale compu ing en i onmen s, such as HPC clus e s.
MLOps
Machine Lea ning Ope a ions is a se o p ac ices ha aims o s eamline he
de elopmen , deploymen , and main enance o machine lea ning models
MI250x
AMD GPU designed o high-pe o mance compu ing and AI wo kloads, used in
sys ems like LUMI.
MMLU
Massi e Mul i ask Language Unde s anding is a benchma k o e alua ing he
mul i ask accu acy and gene al knowledge o la ge language models.
Mis al
Mis al is a amily o la ge language models de eloped by he company Mis al
AI.
Kube low
A popula wo k low sys em o machine lea ning ope a ions
Podman
An open-sou ce con aine managemen ool al e na i e o Docke .
PyTo ch
PyTo ch is an open-sou ce machine lea ning lib a y de eloped by Me a AI,
widely used o de eloping deep lea ning models.
Qwen
Qwen is a amily o la ge language models de eloped by Alibaba Cloud.
ROCm
AMD's open sou ce so wa e s ack designed o GPU-accele a ed high
pe o mance compu ing (HPC) and machine lea ning wo kloads.
Singula i y
A con aine pla o m a ge ed o high-pe o mance compu ing (HPC) use case.
STT
Speech- o-Tex (STT) e e s o echnologies o models ha con e spoken
language in o w i en ex , commonly used in oice ecogni ion sys ems.
Tenso Flow
Tenso Flow is an open-sou ce machine lea ning pla o m de eloped by Google,
used o de eloping, aining, and deploying machine lea ning models.
Ubun u
A popula and gene ic Linux dis ibu ion widely used wi h machine lea ning and
AI ela ed applica ions.
VLLM
VLLM is an open-sou ce Py hon lib a y and in e ence engine designed o
e icien and as in e ence and se ing o la ge language models.
D4.4 Con aine ized wo k lows
6
YAML
YAML is a human- eadable da a se ializa ion o ma o en used o con igu a ion
iles and da a exchange be ween languages wi h di e en da a s uc u es.
YOLO
You Only Look Once (YOLO) is a eal- ime objec de ec ion algo i hm in
compu e ision ha p edic s bounding boxes and class p obabili ies o objec s
in images in a single e alua ion.
Execu i e Summa y
One impo an pa o he LUMI AI Fac o y Se ice Cen e 's mission is o con ibu e o empowe ing
Eu ope's AI ecosys em by de eloping and making a ailable eady- o-use AI so wa e solu ions. Cen al
o his e o is he p o ision o Con aine ized Wo k lows, which simpli y he deploymen o AI
capabili ies ac oss a ious applica ions.
The Con aine ized Wo k lows consis s o p e-buil con aine images wi h ailo ed so wa e, example da a
se s and p e- es ed models o imp o e he accessibili y o AI usage and de elopmen . The de elopmen
o hese wo k lows is linked wi h he b oade LUMI AI Fac o y so wa e ecosys em, which includes High-
Pe o mance Compu ing API (HPC-API) and Machine Lea ning Ope a ions (MLOps) ools. Toge he ,
hese elemen s will o m an in eg a ed en i onmen whe e he de elopmen o di e en componen s
suppo s each o he . Fo ins ance, he con aine base images o a ious pa s o he ecosys em, and he
Con aine ized Wo k lows will sha e he same ounda ions.
The key objec i es o de eloping he Con aine ized Wo k lows concep include deli e ing he i s
comple e wo k lows ea ly on. Addi ionally, new wo k lows will be con inuously added h oughou he
li e ime o he LUMI AI Fac o y Se ice Cen e based on eme ging oppo uni ies and use needs. Ini ially,
he ocus is on de eloping wo k lows o La ge Language Models (LLMs), including ex p ocessing
wo k lows o asks such as da ase cu a ion, LLM ine- uning, and LLM e alua ion.
An impo an pa o his de elopmen e o in ol es es ablishing p ocesses, bes p ac ices, and ooling
o c ea ing hese wo k lows. The concep o Con aine ized Wo k lows aims o include e e y hing
necessa y o help new use s ge s a ed quickly. I also p o ides a p o en s a ing poin o expe use s
o build upon o mo e specialized asks. To suppo his, documen a ion wi h examples, openly licensed
models es ed wi h he wo k lows, and example da ase s will be a ailable di ec ly wi hin he
en i onmen .
This documen p o ides an upda e on he s a us o he Con aine ized Wo k lows de elopmen and
ou lines he nex s eps in his ongoing e o .
D4.4 Con aine ized wo k lows
7
Table o Con en s
1. In oduc ion ............................................................................................................... 8
2. Backg ound ................................................................................................................ 9
2.1 AI Models, So wa e, and Use Cases 9
2.2 Cu en Use o Con aine s 10
2.3 Wo k lows, Con aine ized 11
3. Design and Implemen a ion ...................................................................................... 13
3.1 Con aine Images 13
3.2 Wo k low Files 14
3.3 Addi ional A i ac s 14
3.4 Ins alla ion and Di ec o y S uc u e 15
3.5 Cu en S a us 15
4. Wo k low Ca alog ..................................................................................................... 16
4.1 LLM Tex P ocessing 16
4.2 Vision-Language Ba ch P ocessing 17
4.3 YOLO Image P ocessing Wo k low 18
4.4 Planned Fu u e Wo k lows 19
5. Nex S eps ................................................................................................................ 20
6. Re e ences ............................................................................................................... 21
D4.4 Con aine ized wo k lows
8
1. In oduc ion
In ecen yea s, he landscape o a i icial in elligence has unde gone signi ican ad ancemen s, wi h a
g owing numbe o AI ools and models becoming accessible and p ac ical o a wide ange o
applica ions [1]. These de elopmen s ha e con ibu ed o he democ a iza ion o AI, enabling
esea che s and p ac i ione s ac oss a ious domains o inco po a e ad anced echnologies in o hei
wo k [2]. As he use o AI con inues o expand, he need o s uc u ed, scalable app oaches o s eamline
deploymen and in eg a ion has become inc easingly e iden .
To add ess hese eme ging needs and oppo uni ies in AI adop ion, he LUMI AI Fac o y Se ice Cen e
is de eloping AI so wa e solu ions, including he Con aine ized Wo k lows concep , designed o
s eamline he execu ion o AI- ela ed asks. Wi h Con aine ized Wo k lows concep , his is achie ed by
o e ing p e-p epa ed so wa e, con aine images, da ase s, models, and documen a ion as packages,
es ed o wo k p ope ly oge he .
This e o is pa o he la ge LUMI AI Fac o y so wa e ecosys em de elopmen , which includes
de eloping ools and APIs such as he High-Pe o mance Compu ing API (HPC-API), Machine Lea ning
Ope a ions (MLOps) ools, and AI model in e ence pla o ms (Ai a). These componen s collec i ely
enhance he unc ionali y and ease o use o AI so wa e on LUMI and he u u e LUMI-AI
supe compu e s.
The Con aine ized Wo k lows wo k package is se o deli e a selec ion o unc ional wo k lows ea ly on,
wi h con inuous upda es based on use needs and echnological ad ancemen s h oughou he p ojec .
The ini ial ocus is on de eloping wo k lows o La ge Language Models (LLMs), including ex
p ocessing, ine- uning, and model e alua ion asks as well as wo k lows o di e en image p ocessing
asks.
The de elopmen p ocess o he Con aine ized Wo k lows in ol es es ablishing p ocesses, bes
p ac ices, and ools o c ea ing and main aining hese wo k lows and he ela ed a i ac s. Key
conside a ions include iden i ying use cases ha add ess speci ic esea ch needs, guiding he
de elopmen o ailo ed wo k lows. The goal is o p o ide comp ehensi e wo k low packages ha
enable bo h no ice and expe use s o quickly deploy and cus omize AI wo k lows, suppo ed by de ailed
documen a ion, example da ase s, and p e- es ed models wi hin he LUMI en i onmen .
D4.4 Con aine ized wo k lows
9
2. Backg ound
The ollowing sec ions p o ide an o e iew o key aspec s unde lying he demand o he Con aine ized
Wo k lows as an app oach o deli e ing AI capabili ies on high-pe o mance compu ing en i onmen s.
These include he g owing a ailabili y o open AI models and so wa e ools [1], he adop ion o con aine
echnologies in high-pe o mance compu ing en i onmen s [3], and he oppo uni y o s uc u ed,
ep oducible wo k lows o accele a e he adop ion o di e en AI ools in a ious domains and p ac ical
applica ions.
2.1 AI Models, So wa e, and Use Cases
In ecen yea s, he e has been a signi ican inc ease in he a ailabili y o open AI models and AI- ela ed
ools. Examples include gene al-pu pose language models such as LLaMA [4], Mis al [5], and Po o [6],
domain-speci ic models o a eas like p o ein s uc u e p edic ion (AlphaFold [7]), as well as widely used
so wa e amewo ks like PyTo ch [8], Hugging Face T ans o me s [9] and op imized in e ence engines
such as LLM [10]. A la ge selec ion o esou ces is now eely accessible, enabling esea che s and
de elope s o build on hese ad anced capabili ies wi hou elying on p op ie a y sys ems, o coming up
wi h cus om solu ions om sc a ch, o example when c ea ing au oma ed documen analysis pipelines,
aining clima e models, o de eloping compu e ision sys ems o scien i ic imaging. While he
a ailabili y o open models and ools o e s g ea oppo uni ies, i also in oduces signi ican complexi y,
especially o hose new o he ield.
Many high-quali y p e- ained models a e openly a ailable ac oss domains such as language, ision,
speech, and gene a i e asks. Examples include:
• Na u al Language P ocessing: Models used in asks like classi ica ion, summa iza ion, and
ansla ion, o example au oma ically ca ego izing housands o incoming cus ome suppo
emails by opic, summa izing leng hy esea ch epo s o quick e iew, o ansla ing echnical
documen a ion o mul iple languages o in e na ional eams.
• Compu e Vision: Models used in objec de ec ion, classi ica ion, and segmen a ion, o example
iden i ying de ec s on a manu ac u ing line in eal ime, classi ying sa elli e images o de ec land
use changes, o segmen ing mic oscopic images o medical diagnosis.
• Speech Recogni ion: Models used o ansc ip ion and o he audio-based asks, o example
con e ing mee ing eco dings in o sea chable ansc ip s, enabling oice-based commands in
indus ial equipmen , o gene a ing cap ions o li e b oadcas s.
• Gene a i e Models: Models used o image gene a ion om ex p omp s, audio syn hesis, and
ideo gene a ion, o example c ea ing pho o ealis ic ma ke ing isuals om a sho desc ip ion,
syn hesizing oiceo e s in mul iple languages o aining ideos, o p oducing sho anima ions
o educa ional con en .
These models a e o en eleased unde licenses ha pe mi euse and adap a ion, and a e ypically
accompanied by example code and documen a ion [11].
One impo an dis inc ion o make is be ween open-weigh models and ully open models. Open-weigh
models p o ide ained pa ame e s o use in in e ence o ine- uning bu may wi hhold aining da a,
code, o ull me hodology. Fully open models elease hese alongside he weigh s, enabling be e
D4.4 Con aine ized wo k lows
16
4. Wo k low Ca alog
This chap e lis s he cu en ly implemen ed wo k lows and in oduces ew new wo k lows planned o
be implemen ed in u u e.
4.1 LLM Tex P ocessing
S a us: Ini ial implemen a ion has been comple ed, and he wo k low is a ailable o LUMI use .
Desc ip ion:
The LLM Tex P ocessing wo k low is designed o acili a e e icien ex p ocessing using La ge
Language Models (LLMs). This wo k low is implemen ed as a eady- o-go con aine image and ecipe,
speci ically ailo ed o asks such as da ase cu a ion. I le e ages he LLM lib a y o p ocess inpu iles
based on use -p o ided ins uc ions and gene a e co esponding ou pu iles. The LLM engine
implemen s op imized enso -pa allel ba ched in e ence o high pe o mance, while he wo k low as a
whole p o ides a es ed, eady- o-use se up ha allows use s o ge s a ed quickly wi hou ex ensi e
con igu a ion o oubleshoo ing.
Key Fea u es:
• Inpu and Ou pu Handling: The wo k low p ocesses iles in a speci ied inpu di ec o y and
gene a es ou pu iles in a designa ed ou pu di ec o y. Each inpu ile is p ocessed acco ding o
he gi en ins uc ions, wi h he ou pu being ei he a single ile o one ou pu ile pe inpu ile.
• Task Con igu a ion: The wo k low allows o command-line o YAML con igu a ion ile inpu s,
p o iding lexibili y in de ining he p ocessing asks. Use s can speci y he model o be used,
inpu and ou pu ile pa hs, and ins uc ions o p ocessing.
• Cus om Sc ip s and Models: Use s can cus omize he wo k low sc ip and selec om a se o
eady- o-go models ha ha e been es ed o wo k e icien ly wi hin he wo k low.
• Scalabili y: The p ocessing speed scales well wi h he numbe o inpu iles. Op imal pe o mance
is achie ed wi h a la ge numbe o inpu iles, le e aging he e iciency o ba ched p ocessing
used in LLM.
Planned Imp o emen s:
• Allow mul i-s ep p ocessing wi hou eloading he language model.
Usage o e iew:
To u ilize he LLM Tex P ocessing wo k low, use s can begin by copying one o he p o ided example
di ec o ies om he wo k low's examples olde in o hei own wo king di ec o y. The examples se e as
empla es and include all necessa y iles and con igu a ions o un a ex p ocessing job.
Once he example di ec o y is copied, use s can submi he job o he compu ing clus e using a ba ch
sc ip included in he di ec o y. This sc ip equi es use s o speci y hei p ojec accoun o alloca e he
necessa y esou ces o job execu ion.
D4.4 Con aine ized wo k lows
17
Du ing job execu ion, use s can moni o he p og ess and s a us using s anda d Slu m commands. This
allows o eal- ime acking o he job's ou pu and o e all p og ess once i has s a ed.
Upon comple ion o he job, he wo k low gene a es ou pu iles in he speci ied ou pu di ec o y. These
iles con ain he p ocessed esul s based on he inpu iles and ins uc ions p o ided.
Fo de ailed speci ic examples, use s can e e o he example README iles loca ed wi hin he wo k low
example di ec o ies. These iles p o ide comp ehensi e ins uc ions, sample ask con igu a ions in
YAML o ma , and example ins uc ion iles ha guide use s h ough se ing up and cus omizing hei
ex p ocessing asks.
4.2 Vision-Language Ba ch P ocessing
S a us: Ini ial implemen a ion has been comple ed, and he wo k low is a ailable o LUMI use .
Desc ip ion: The Vision-Language Ba ch P ocessing wo k low is designed o acili a e e icien image
p ocessing, such as objec de ec ion, classi ica ion, segmen a ion, mo ion- acking, using ision-
language models. This wo k low le e ages con aine and wi h T ans o me s lib a y o p ocess inpu
images based on use -p o ided ins uc ions and gene a e co esponding ou pu iles wi h ex ual
answe s.
Key Fea u es:
• Inpu and Ou pu Handling: The wo k low p ocesses images in a speci ied inpu di ec o y and
gene a es ou pu ex iles in a designa ed ou pu di ec o y. Each inpu image is p ocessed
acco ding o he gi en ins uc ions, wi h he ou pu being one ex ile pe inpu image.
• Task Con igu a ion: The wo k low uses a YAML con igu a ion ile o de ine he p ocessing asks.
Use s can speci y he model o be used, inpu and ou pu ile pa hs, and ins uc ions o
p ocessing.
• Cus om Sc ip s and Models: Use s can cus omize he wo k low sc ip and selec om a se o
eady- o-go models ha ha e been es ed o wo k p ope ly wi hin he wo k low.
• Scalabili y: The p ocessing speed scales well wi h he numbe o inpu images, based he
e iciency o ba ched p ocessing p o ided in he T ans o me s lib a y.
Planned Imp o emen s:
• Enable using quan ized models like AWQ o imp o e loading imes and o e all pe o mance.
• Conduc an addi ional code e iew o ensu e he quali y and main ainabili y o he wo k low.
Usage O e iew: To u ilize he Hugging Face Vision-Language Ba ch P ocessing wo k low, use s can
begin by submi ing he p o ided SLURM sc ip o launch a demo job on LUMI. The sc ip eques s GPU
esou ces and se s sensible de aul s o in e ac i e es ing.
Once he job is submi ed, he con aine execu ion begins, and he ision-language pipeline is execu ed.
The pipeline eads a YAML ask speci ica ion, loads a Qwen2.5 ision-language model, loops o e he
inpu images, asks he p omp , and s o es he model's answe in he co esponding ou pu ex iles.
D4.4 Con aine ized wo k lows
18
Use s can c ea e hei own asks by p epa ing inpu images, w i ing a p omp , c ea ing a ask.yaml ile
ha poin s o he inpu s, ou pu s, and ins uc ions, and unning he job by adap ing he las line o he
un_con aine .sh sc ip o poin o hei YAML and e-submi ing wi h sba ch.
Fo de ailed speci ic examples, use s can e e o he example README iles loca ed wi hin he wo k low
example di ec o ies. These iles p o ide comp ehensi e ins uc ions, sample ask con igu a ions in
YAML o ma , and example ins uc ion iles ha guide use s h ough se ing up and cus omizing hei
image p ocessing asks.
4.3 YOLO Image P ocessing Wo k low
S a us: Ini ial implemen a ion has been comple ed, and he wo k low is a ailable o LUMI use .
Desc ip ion: The YOLO Image P ocessing Wo k low is designed o acili a e e icien image p ocessing
using YOLO (You Only Look Once) models. This wo k low is implemen ed as a eady- o-go con aine
image and a wo k low ecipe, sui ed o asks such as objec classi ica ion. I p ocesses inpu image iles
and gene a es co esponding ou pu iles wi h class p edic ions in a human- eadable o ma .
Key Fea u es:
• Inpu and Ou pu Handling: The wo k low p ocesses images in a speci ied inpu di ec o y and
gene a es ou pu ex iles in a designa ed ou pu di ec o y wi h one ex ile pe inpu image.
• Task Con igu a ion: The wo k low uses a YAML con igu a ion ile o de ine he p ocessing asks.
Use s can speci y he model di ec o y and ile, inpu and ou pu ile pa hs.
• Scalabili y: The p ocessing speed scales well wi h he numbe o inpu images, le e aging he
e iciency o ba ched p ocessing.
Planned Imp o emen s:
• None a he momen
Usage O e iew: To u ilize he YOLO Image P ocessing Wo k low, use s can begin by copying one o
he p o ided example di ec o ies om he wo k low's examples olde in o hei own wo king di ec o y.
The examples se e as empla es and include all necessa y iles and con igu a ions o un an image
p ocessing job.
Once he example di ec o y is copied, use s can submi he job o he compu ing clus e using a ba ch
sc ip included in he di ec o y. This sc ip equi es use s o speci y hei p ojec accoun o alloca e he
necessa y esou ces o job execu ion.
Du ing job execu ion, use s can moni o he p og ess and s a us using s anda d Slu m commands. This
allows o eal- ime acking o he job's ou pu and o e all p og ess once i has s a ed.
Upon comple ion o he job, he wo k low gene a es ou pu ex iles in he speci ied ou pu di ec o y.
These iles con ain he class p edic ions based on he inpu images.
D4.4 Con aine ized wo k lows
19
Fo de ailed speci ic examples, use s can e e o he example README iles loca ed wi hin he wo k low
example di ec o ies. These iles p o ide comp ehensi e ins uc ions, sample ask con igu a ions in
YAML o ma , and example ins uc ion iles ha guide use s h ough se ing up and cus omizing hei
image p ocessing asks.
4.4 Planned Fu u e Wo k lows
Looking o wa d o expanding he selec ion o a ailable wo k lows, we a e conside ing in oducing
se e al new wo k lows. These upcoming wo k lows should be bene icial o add essing se e al
oppo uni ies in esea ch and o he use cases. Below is ou cu en lis o planned and ones in idea s age.
LLM Fine- uning
S a us: Planned o be implemen ed
Desc ip ion: This wo k low will p o ide eady- o-go ecipes and con aine images o ine- uning p e-
ained and ins uc ion- uned ounda ion models using cus om da ase s. The wo k low will include a se
o es ed models eadily a ailable on LUMI, con aine images, example da ase s, and example job iles
o launching aining jobs. This will enable use s o adap exis ing models o speci ic asks o domains,
imp o ing hei pe o mance and ele ance. The wo k low will suppo a a ie y o p e- ained models
and model a chi ec u es.
LLM E alua ion
S a us: Planned o be implemen ed
Desc ip ion: This wo k low will o e a con aine ized solu ion o unning s anda d LLM e alua ion
benchma ks, such as GPQA (G adua e-Le el Google-P oo Q&A) and MMLU (Massi e Mul i ask
Language Unde s anding). S anda d benchma ks a e cen al o assessing he pe o mance and
capabili ies o language models, p o iding insigh s in o model s eng hs and weaknesses.
Speech o Tex
S a us: Unde conside a ion
Desc ip ion: This wo k low will enable ansc ibing audio iles o ex using Speech- o-Tex (STT)
models. This capabili y can be use ul mul iple esea ch and applica ion a eas. The wo k low will include
p e- ained models, con aine images, and example audio iles o demons a e he ansc ip ion p ocess.
Wo k low F amewo ks
S a us: Unde conside a ion
Desc ip ion: We a e e alua ing he use ulness o p o iding wo k lows enabling he use o wo k low
engines such as Nex low, OpenWDL, Snakemake and S eamFlow.
D4.4 Con aine ized wo k lows
20
5. Nex S eps
The nex phase o de elopmen o he Con aine ized Wo k lows includes inc easing he co e age o he
wo k lows o di e en asks, enhancing he unc ionali y, accessibili y, and use expe ience o he
exis ing wo k lows, and s eamlining he Con aine ized Wo k lows de elopmen p ocess as well as
inalizing a p ocess o use suppo eques s ela ed o he wo k lows.
Building on he ini ial se o p o ided wo k lows o LLM Tex P ocessing, Vision-Language Ba ch
P ocessing, and YOLO Image P ocessing Wo k lows, we will p oceed o expand he ca alog o a ailable
wo k lows. The nex phase is planned o include he de elopmen o wo k lows o LLM ine- uning and
LLM benchma k e alua ion. Addi ionally, we will explo e wo k lows o audio p ocessing, such as
Speech- o-Tex . We will also con inue acking he so wa e and AI model landscape o iden i y
oppo uni ies o in eg a ing new echnologies and ools o ensu e ha he Con aine ized Wo k lows
concep emain up- o-da e and e ec i ely mee he needs o he use communi y.
Main aining he c ea ed wo k lows is c ucial. This in ol es egula ly upda ing he con aine s images o
he la es so wa e e sions o imp o e pe o mance, ix bugs, and main ain he secu i y o he so wa e
componen s. Addi ionally, sys em upg ades on LUMI may b eak compa ibili y wi h he wo k lows, which
may equi e addi ional wo k o ensu e con inuous ope a ion and smoo h use expe ience. We aim o
p o ide a s able and e icien en i onmen o use s o con inuously accomplish hei AI- ela ed asks.
Con inuous imp o emen o he de elopmen p ocesses is needed o achie ing high s anda ds o he
Con aine ized Wo k lows concep . This in ol es adop ing bes p ac ices in so wa e de elopmen ,
con aine iza ion, and e icien collabo a ion be ween he con ibu ing pa ne o ganiza ions. By e ining
he p ocesses, we aim o ensu e he wo k lows a e obus , eliable, and bene icial o he use s. Code
e iews, au oma ed es ing, and CI/CD pipelines a e planned o be implemen ed o s eamline he
de elopmen p ocess.
Addi ionally, we a e looking o es ablish clea channels o use communica ion and eedback o enable
us o ga he insigh s and add ess any issues he use s may ace p omp ly. We a e looking o p o ide
egula upda es o keep he use s in o med abou imp o emen s and upcoming wo k lows.
In conclusion, he g oundwo k o he Con aine ized Wo k lows has been success ully laid wi h he ini ial
se o wo k lows and suppo ing in as uc u e now a ailable on LUMI. This ounda ion enables us o
mo e o wa d wi h expanding unc ionali y, imp o ing usabili y and imp o ing he de elopmen
p ocess. Mo ing o wa d, ou ocus is on collabo a ion, main ainabili y, and scalabili y o he concep .
Th ough ongoing echnical e inemen and use engagemen , we s i e o make su e ha he
Con aine ized Wo k lows concep becomes a sus ainable and impac ul oolse o AI de elopmen and
esea ch on LUMI, and a la e s age, on LUMI-AI.
D4.4 Con aine ized wo k lows
21
6. Re e ences
[1]
"Recen T ends in A i icial In elligence Technology: A Scoping Re iew [a Xi :2305.04532 3],"
2023.
[2]
"Explo ing he La es T ends in AI Technologies: A S udy on Cu en S a e, Applica ion and
Indi idual Impac s [10.4236/jcc.2024.128002]," 2024.
[3]
"Con aine isa ion o High Pe o mance Compu ing Sys ems: Su ey and P ospec s
[a Xi :2212.08717]," 2022.
[4]
"Hugging ace Llama-3.3-70B-Ins uc Model Ca d," [Online]. A ailable:
h ps://hugging ace.co/me a-llama/Llama-3.3-70B-Ins uc .
[5]
"Hugging ace Mis al-Small-3.2-24B-Ins uc -2506 Model Ca d," [Online]. A ailable:
h ps://hugging ace.co/mis alai/Mis al-Small-3.2-24B-Ins uc -2506.
[6]
"Hugging ace Po o-34B-cha Model Ca d," [Online]. A ailable:
h ps://hugging ace.co/LumiOpen/Po o-34B-cha .
[7]
"AlphaFold," [Online]. A ailable: h ps://deepmind.google/science/alpha old/.
[8]
"Py o ch," [Online]. A ailable: h ps://py o ch.o g .
[9]
"Hugging ace T ans o me s," [Online]. A ailable:
h ps://hugging ace.co/docs/ ans o me s/en/index.
[10]
" LLM," [Online]. A ailable: h ps://docs. llm.ai/en/la es /.
[11]
"An Empi ical S udy o P e-T ained Model Reuse in he Hugging Face Deep Lea ning Model
Regis y [a Xi :2303.02552]," 2023.
[12]
"Comp ehensi e Analysis o T anspa ency and Accessibili y o Cha GPT, DeepSeek, and o he
SoTA La ge Language Models [a Xi :2502.18505 1]," 2025.
[13]
"Tenso low," [Online]. A ailable: h ps://www. enso low.o g.
[14]
"Numpy," [Online]. A ailable: h ps://numpy.o g/.
[15]
"Pandas," [Online]. A ailable: h ps://pandas.pyda a.o g.
[16]
"LUMI Documen a ion, Con aine jobs," [Online]. A ailable: h ps://docs.lumi-
supe compu e .eu/ unjobs/scheduled-jobs/con aine -jobs/.
D4.4 Con aine ized wo k lows
22
[17]
"LUMI EasyBuild con aine s," [Online]. A ailable: h ps://gi hub.com/Lumi-supe compu e /LUMI-
EasyBuild-con aine s.
[18]
"Enabling Message Passing In e ace Con aine s on he LUMI Supe compu e [a
Xi :2407.19928]," 2024.
[19]
"LUMI Documen a ion - Singula i y," [Online]. A ailable: h ps://docs.lumi-
supe compu e .eu/so wa e/con aine s/singula i y/.
[20]
"Singula i y Documen a ion - Secu i y in Singula i y," [Online]. A ailable:
h ps://docs.sylabs.io/guides/3.5/use -guide/secu i y.h ml.
[21]
"LUMI Documen a ion, GPU nodes - LUMI-G," [Online]. A ailable: h ps://docs.lumi-
supe compu e .eu/ha dwa e/lumig/.
[22]
"LUMI Documen a ion, Ne wo k and in e connec ," [Online]. A ailable: h ps://docs.lumi-
supe compu e .eu/ha dwa e/ne wo k/.
[23]
"Radeon Open Compu e Documen a ion," [Online]. A ailable:
h ps:// ocm.docs.amd.com/en/la es /.
[24]
"NVidia Cuda," [Online]. A ailable: h ps://de elope .n idia.com/cuda- oolki .
[25]
"A Sys ema ic Li e a u e Re iew on G aphics P ocessing Uni Accele a ed Realm o High-
Pe o mance Compu ing [10.47941/ijce.1813]," [Online].
[26]
"LUMI-con aine Image Docke Recipes," [Online]. A ailable: h ps://gi hub.com/s an ao/LUMI-
con aine s/ ee/main/RecipesDocke .
[27]
"Podman," [Online]. A ailable: h ps://podman.io/.
[28]
"Docke documen a ion, Docke ile," [Online]. A ailable:
h ps://docs.docke .com/ e e ence/docke ile/.
[29]
"LUMI Use Documen a ion/LUMI AI Fac o y," [Online]. A ailable: h ps://docs.lumi-
supe compu e .eu/so wa e/local/lumi-ai .
[30]
"Is Open Sou ce he Fu u e o AI? A Da a-D i en App oach. [a Xi :2501.16403 1]," 2025.
[31]
"How Fa Behind A e Open Models?," [Online]. A ailable: h ps://epoch.ai/blog/open-models-
epo .
