LUMI AI Fac o y Se ice Cen e
Empowe ing Eu ope’s AI Ecosys em
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
2
D5.3
Tools o in eg a ing dynamic and eal- ime da a s eams
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
3
P ojec Ti le
LUMI AI Fac o y Se ice Cen e
P ojec Ac onym
LUMI-AIF
P ojec Numbe
101234208
Type o Ac ion
HORIZON-JU-RIA
Topic
HORIZON-JU-EUROHPC-2025-AI-01-IBA-01
S a ing Da e o P ojec
01.03.2025
Ending Da e o P ojec
29.02.2028
Du a ion o he P ojec
36 mon hs
Websi e
lumi-ai- ac o y.eu
Wo k Package
WP5 Da a access and in eg a ion
Task
Task 5.3: P o ision o ools o objec s o age
Lead Au ho s
Heidi Laine (CSC), Ja no Lai inen (CSC)
Con ibu o s
Abdul ahman Azab (Sigma2)
Pee Re iewe s
Pauliina Some koski (CSC), Dhanya Pushpadas (Sigma 2)
Ve sion
1.0
Due Da e
31.08.2025
Submission Da e
29.08.2025
Dissemina ion le el
X
PU: Public
SEN: Sensi i e – limi ed unde he condi ions o he G an Ag eemen
EU-RES. Classi ied In o ma ion: RESTREINT UE (Commission Decision 2005/444/EC)
EU-CON. Classi ied In o ma ion: CONFIDENTIEL UE (Commission Decision 2005/444/EC)
EU-SEC. Classi ied In o ma ion: SECRET UE (Commission Decision 2005/444/EC)
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
4
Ve sion His o y
Re ision
Da e
Edi o s
Commen s
0.1
11.08.2025
Heidi Laine
Fi s e sion o p ojec in e nal pee e iew
0.5
25.08.2025
Heidi Laine
Edi s based on pee e iewe commen s
0.9
29.08.2025
Heidi Laine
Edi s on case s udy desc ip ions, spell-
checking, inal o ma ing, addi ion o
glossa y
1.0
29.08.2025
Anna Luoma
Final quali y check pe o med by he PMO,
sen o e iew.
Decla a ion on he Use o AI Assis ance
This epo has been p epa ed wi h he suppo o GPT-5-enabled Mic oso Copilo , which was used o
assis in d a ing ex , checking language, and ga he ing backg ound in o ma ion. All con en has been
ho oughly e iewed, ac -checked, and edi ed by he au ho s o ensu e accu acy and alignmen wi h
he objec i es o he epo .
Glossa y o Te ms
I em
Desc ip ion
5G
Fi h-gene a ion mobile ne wo k echnology ha p o ides
low-la ency, high-bandwid h wi eless connec i i y sui able o
eal- ime IoT and edge scena ios.
Ac i e lea ning
Machine-lea ning app oach whe e a model eques s labels o he
mos in o ma i e samples o imp o e quickly wi h limi ed
anno a ion e o .
AI/ML in e ence
Running a ained model o gene a e p edic ions on new da a, o en
wi h low-la ency equi emen s o s eaming use cases.
Apache Ai low
Wo k low o ches a ion pla o m o scheduling and managing da a
pipelines (mainly ba ch).
Apache Flink
Dis ibu ed s eam-p ocessing engine o s a e ul compu a ions
o e unbounded and bounded da a s eams wi h low la ency.
Apache Ka ka
Dis ibu ed e en -s eaming pla o m p o iding du able, scalable
pub/sub messaging wi h opics and pa i ions.
App aine
(Singula i y)
Con aine un ime commonly used in HPC o package and un
applica ions ep oducibly wi hou ele a ed p i ileges.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
5
Aspe a / FASP
High-speed ile ans e p o ocol (UDP-based) op imized o
long-dis ance, high-la ency ne wo ks.
Backp essu e
Mechanism in s eaming sys ems o slow p oduce s o in e media e
s ages when consume s canno keep up, p e en ing da a loss.
Ba ch p ocessing
P ocessing da a in disc e e chunks a scheduled imes, as opposed
o con inuous eal- ime p ocessing.
CAN bus
Con olle A ea Ne wo k; ehicle bus s anda d used o enable
communica ion be ween au omo i e componen s.
cPou a / Rah i
CSC cloud se ices: cPou a (IaaS) and Rah i (Kube ne es PaaS) used
o con aine ized o VM-based wo kloads.
CSC
CSC – IT Cen e o Science (Finland).
CSV
Comma-Sepa a ed Values; simple ex o ma o abula da a
exchange.
Da a go e nance
Policies and p ocesses ensu ing da a quali y, secu i y, compliance,
and li ecycle managemen .
Da a lake
Cen alized s o age o aw, s uc u ed and uns uc u ed da a a any
scale o analy ics and ML.
Da a minimiza ion
GDPR p inciple: collec and p ocess only he da a necessa y o a
speci ic pu pose.
Da a p o enance
Documen ed lineage o da a, including o igin, ans o ma ions, and
owne ship, suppo ing ep oducibili y and FAIR.
Da a wa ehouse
Schema-o ien ed analy ical s o age op imized o s uc u ed,
que y-in ensi e wo kloads.
DPIA
Da a P o ec ion Impac Assessmen ; GDPR equi emen o assess
and mi iga e p i acy isks o a p ocessing ac i i y.
Edge compu ing
P ocessing pe o med close o da a sou ces (e.g., de ices,
ga eways) o educe la ency and bandwid h usage.
Elas ic scaling
Abili y o a sys em o scale esou ces up/down au oma ically based
on load.
E en -d i en
a chi ec u e
Design whe e e en s igge p ocessing and downs eam ac ions,
common in s eaming se ups.
Eu oHPC
Eu opean High-Pe o mance Compu ing Join Unde aking.
FAIR
Da a p inciples: Findable, Accessible, In e ope able, Reusable.
FASP
See Aspe a / FASP.
FlexE
Flexible E he ne ; echnology o agg ega e/slice E he ne links o
high-capaci y, de e minis ic anspo .
gRPC
High-pe o mance, bina y RPC amewo k using P o ocol Bu e s;
suppo s s eaming calls.
Globus
Resea ch da a ans e and sha ing pla o m ha p o ides secu e,
eliable, esumable high- h oughpu ans e s.
G a ana
Visualiza ion and dashboa ding ool o en pai ed wi h ime-se ies
da abases and P ome heus me ics.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
6
Helm cha
Kube ne es packaging o ma desc ibing a se o esou ces o
ins alling/con igu ing applica ions.
HPC
High-Pe o mance Compu ing; la ge-scale compu e sys ems
op imized o pa allel wo kloads.
HTTP/HTTPS
Web p o ocol (secu e a ian wi h TLS) used o APIs and da a
exchange.
ICEYE
Mic osa elli e (SAR) da a p o ide ; example in he documen o
nea - eal- ime en i onmen al moni o ing.
Idempo ency
P ope y whe e epea ing an ope a ion yields he same esul ;
impo an o a -leas -once p ocessing.
In luxDB
Time-se ies da abase op imized o high-inges eleme y and
me ics.
IoT
In e ne o Things; ne wo k o connec ed de ices and senso s
p oducing eleme y da a.
Jupy e Hub
Mul i-use en i onmen o unning Jupy e no ebooks on sha ed
in as uc u e.
Ka ka pa i ion
Subdi ision o a Ka ka opic enabling pa allelism and scalabili y
ac oss b oke s/consume s.
Kube ne es
Con aine o ches a ion pla o m o deploying, scaling, and
managing con aine ized applica ions.
LiDAR
Ligh De ec ion and Ranging; senso p oducing high- a e 3D poin
clouds, o en s eamed o e UDP.
LoRaWAN
Low-powe , long- ange wi eless p o ocol o IoT de ices wi h
modes da a a es.
Lus e
High-pe o mance pa allel ile sys em widely used in HPC
en i onmen s.
LUMI
Eu oHPC p e-exascale supe compu e hos ed in Kajaani, Finland.
LUMI AIF
LUMI AI Fac o y; se ices enabling AI/HPC wo k lows, including da a
access and in eg a ion.
LUMI-AI
Fo hcoming AI-op imized expansion o LUMI imp o ing aining
and eal- ime p ocessing capabili ies.
LUMI-G
GPU pa i ion o LUMI (AMD MI250X GPUs) o accele a ed AI/ML
and compu e wo kloads.
Me ada a
Desc ip i e in o ma ion abou da a (schema, p o enance,
iden i ie s) enabling disco e y and euse.
Mic o-ba ching
P ocessing incoming da a in e y small ba ches o balance la ency
and h oughpu .
MinIO
S3-compa ible objec s o age so wa e commonly used o
on-p emises da a lakes.
MQTT
Ligh weigh pub/sub messaging p o ocol o cons ained de ices
and un eliable ne wo ks.
NB-IoT
Na owband IoT; cellula LPWAN echnology o low-powe ,
low- h oughpu de ices.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
7
NIRD
No wegian esea ch da a in as uc u e used in he SeaBee case o
s o age and p ocessing.
NIS2
EU di ec i e es ablishing cybe secu i y isk-managemen and
epo ing obliga ions o essen ial/impo an en i ies.
OEE
O e all Equipmen E ec i eness; KPI combining a ailabili y,
pe o mance, and quali y in manu ac u ing.
ONNX Run ime
In e ence engine o models in ONNX o ma ; enables po able,
high-pe o mance in e ence.
ORC
Op imized Row Columna ; columna s o age o ma o e icien
analy ics.
Pa que
Columna s o age o ma op imized o analy ics and comp ession.
PCAP
Packe Cap u e; ile o ma o eco ding ne wo k a ic (e.g., UDP
LiDAR ames).
P edic i e
main enance
Using analy ics/ML on eleme y o p edic componen ailu e and
schedule main enance p oac i ely.
P ome heus
Moni o ing sys em and ime-se ies da abase ocused on me ics
collec ion and ale ing.
P o ocol ansla ion
Con e ing be ween p o ocols o payload o ma s (e.g.,
MQTT→Ka ka, bina y→JSON/Pa que ).
Pseudonymiza ion
Replacing iden i ie s wi h pseudonyms o educe p i acy isks while
enabling analysis.
RBAC
Role-Based Access Con ol; au ho iza ion model assigning
pe missions o oles a he han indi iduals.
Rclone
Command-line ool o syncing and copying da a ac oss s o age
sys ems (e.g., S3, POSIX, cloud objec s o es).
Real- ime analy ics
Con inuous analysis o s eaming da a wi h low la ency o d i e
immedia e decisions.
ROCm
AMD’s open compu e pla o m o GPU accele a ion (LUMI-G
GPUs).
RUL
Remaining Use ul Li e; p edic ed ime un il a componen ails o
equi es se icing.
SeaBee
Coas al moni o ing ini ia i e using d ones; case s udy
demons a ing hyb id s eaming and HPC p ocessing.
Slu m
HPC wo kload manage used on LUMI o schedule and un jobs.
Spa k S eaming
S eaming lib a y o Apache Spa k enabling mic o-ba ch and
con inuous p ocessing.
S eam p ocessing
Con inuous compu a ion o e da a s eams o de i e me ics, ale s,
and ea u es wi h minimal delay.
Teleg a
Agen o collec ing, p ocessing, and sending me ics/logs o
da abases such as In luxDB.
Tenso RT
NVIDIA in e ence op imize / un ime ( ele an o po abili y
compa isons; no used on AMD GPUs in LUMI-G).
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
8
Th eshold-based
ale ing
Gene a ing ale s when me ics c oss p ede ined alues (e.g.,
empe a u e spikes).
Time-se ies
da abase
Da abase op imized o ime-indexed da a ( eleme y, me ics),
e.g., In luxDB, TimescaleDB, P ome heus.
TLS/SSL
T anspo -laye enc yp ion ensu ing con iden iali y and in eg i y o
p o ocols like HTTPS, MQTT, Ka ka.
UDP
Use Da ag am P o ocol; low-la ency, connec ionless anspo
commonly used o high- a e senso s eams.
WebSocke s
Full-duplex communica ion channel o e a single TCP connec ion o
bidi ec ional eal- ime messaging.
Zigbee
Low-powe wi eless mesh p o ocol o sho - ange IoT ne wo ks.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
9
Execu i e Summa y
This deli e able p esen s he ini ial esul s o Task 5.3, P o ision o In e aces o Dynamic and Real-Time
Da a, wi hin he LUMI AI Fac o y Se ice Cen e . The epo ou lines he echnical, a chi ec u al, and
ope a ional equi emen s o in eg a ing dynamic and eal- ime da a s eams in o he LUMI
en i onmen , and assesses he cu en capabili ies o he in as uc u e in ela ion o hese needs.
Dynamic da a e e s o con inuously changing in o ma ion gene a ed by senso s, use in e ac ions, o
au oma ed sys ems. I s in eg a ion in o high-pe o mance compu ing (HPC) en i onmen s is essen ial
o enabling esponsi e, da a-d i en AI wo k lows. The epo iden i ies key challenges such as low-
la ency inges ion, p o ocol in e ope abili y, edge compu ing, and secu e da a handling.
The i s e sion o he LUMI AI Fac o y ools includes componen s o high-speed da a ans e , s eam
p ocessing, edge in eg a ion, and obse abili y. These ools a e designed o suppo he e ogeneous use
cases, including en i onmen al moni o ing, p edic i e main enance, and sma in as uc u e. A
compa a i e analysis highligh s a eas whe e LUMI’s cu en capabili ies align wi h equi emen s and
whe e u he de elopmen is needed.
Case s udies, such as he SeaBee coas al moni o ing pilo , demons a e p ac ical implemen a ion
scena ios and in o m u u e de elopmen p io i ies. The epo concludes wi h a oadmap o enhancing
eal- ime da a suppo , including inges ion middlewa e, p o ocol ansla ion se ices, and AI/ML
in e ence capabili ies.
I is likely ha s eaming da a use cases a e so he e ogeneous ha he e canno be one solu ion
a chi ec u e. The end-use s need o adap he solu ion o hei use cases. LUMI AI Fac o y can p o ide
open sou ce so wa e componen in con aine s and p o ide guidance o hem.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
16
• Au oma ed ale s a e sen o main enance eams when anomalies
a e de ec ed.
• P edic i e insigh s om HPC simula ions a e in eg a ed in o he
dashboa d o p oac i e decision-making.
Key Bene i s
Immedia e isibili y in o p oduc ion pe o mance.
• Reduced down ime h ough ea ly anomaly de ec ion.
• Op imized esou ce alloca ion ia p edic i e analy ics.
• Secu e and complian da a handling ac oss he pipeline.
2.3 P edic i e main enance
Scena io:
A la ge ene gy company ope a es wind a ms ac oss Eu ope. Each u bine is
equipped wi h IoT senso s ha moni o ib a ion, empe a u e, and powe
ou pu . The goal is o p edic componen ailu es be o e hey occu , educing
down ime and main enance cos s.
S ep 1: Con inuous
Da a Collec ion
Senso s on u bine componen s (gea boxes, bea ings, blades) gene a e
eleme y da a e e y second.
• Edge de ices p ep ocess he da a, comp essing and no malizing i
in o JSON o ma .
• Da a is ansmi ed secu ely ia MQTT o he inges ion laye .
S ep 2: Inges ion
and Bu e ing
An Apache Ka ka clus e ecei es he da a s eams and pa i ions hem by
u bine ID.
• Ka ka ensu es du abili y and p o ides a bu e o handle ne wo k
luc ua ions o HPC main enance windows.
S ep 3: Real-Time
Anomaly De ec ion
• A Flink-based s eam p ocessing job consumes da a om Ka ka.
• The job applies:
• Th eshold-based ale s o c i ical pa ame e s (e.g.,
empe a u e spikes).
• Ligh weigh ML models o ea ly anomaly de ec ion a he
edge o inges ion laye .
S ep 4: HPC
In eg a ion o
P edic i e Modeling
• Agg ega ed and en iched da a is s eamed o he LUMI HPC
en i onmen .
• HPC nodes un deep lea ning models ained on his o ical ailu e da a
o p edic Remaining Use ul Li e (RUL) o componen s.
• Models a e upda ed pe iodically using he la es s eaming da a o
imp o ed accu acy.
S ep 5:
Visualiza ion and
• P edic ions and ale s a e displayed on a G a ana dashboa d o
ope a ions eams.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
17
Main enance
Scheduling
• The sys em au oma ically gene a es main enance icke s o u bines
p edic ed o ail wi hin a de ined ime window.
• In eg a ion wi h ERP sys ems ensu es spa e pa s and echnicians a e
scheduled p oac i ely.
Key Bene i s
• Reduced unplanned down ime by p edic ing ailu es days o weeks in
ad ance.
• Op imized main enance cos s h ough condi ion-based se icing.
• Imp o ed sa e y by p e en ing ca as ophic ailu es.
• Scalable solu ion capable o handling housands o u bines and
millions o da a poin s pe day.
2.4 Sma in as uc u e and en i onmen s
Scena io:
A me opoli an ci y deploys a ne wo k o IoT senso s and connec ed de ices
o moni o a ic low, ai quali y, and ene gy consump ion in eal ime. The
objec i e is o op imize a ic managemen , educe emissions, and imp o e
u ban li ing condi ions h ough da a-d i en decisions.
S ep 1: Da a
Collec ion om
Dis ibu ed Sou ces
• IoT senso s embedded in a ic ligh s, oadways, and public
anspo ehicles collec da a on:
• Vehicle coun s and speeds
• Ai quali y me ics (CO₂, NO₂, PM2.5)
• Ene gy usage in sma s ee ligh s
• Edge ga eways agg ega e and no malize he da a in o JSON o ma
o ansmission.
S ep 2: Inges ion
and Rou ing
• Da a s eams a e ansmi ed ia MQTT and HTTP/HTTPS o he
ci y’s cen al inges ion laye .
• An Apache Ka ka clus e pa i ions he da a by ca ego y ( a ic, ai
quali y, ene gy) o e icien p ocessing.
S ep 3: Real-Time
Analy ics
• A Flink-based s eam p ocessing pipeline analyzes:
• T a ic conges ion pa e ns in eal ime.
• Ai quali y anomalies and pollu ion ho spo s.
• Ene gy consump ion ends o adap i e ligh ing con ol.
• E en -d i en igge s adjus a ic signals dynamically and ac i a e
low-emission zones when pollu ion exceeds h esholds.
S ep 4: HPC
In eg a ion o
P edic i e Modeling
• Agg ega ed da a is s eamed o he LUMI HPC en i onmen o :
• T a ic low simula ions o p edic conges ion unde di e en
scena ios.
• Ai quali y o ecas ing using AI-d i en models.
• Ene gy op imiza ion models o sma g id in eg a ion.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
18
S ep 5:
Visualiza ion and
Ci izen
Engagemen
• Real- ime dashboa ds display a ic and ai quali y da a o ci y
ope a o s.
• Ci izens access li e upda es ia a mobile app, including al e na i e
ou e sugges ions and pollu ion ale s.
• P edic i e insigh s in o m long- e m u ban planning decisions.
Key Bene i s
• Reduced conges ion h ough adap i e a ic con ol.
• Imp o ed ai quali y ia p oac i e in e en ions.
• Ene gy sa ings h ough sma ligh ing and g id op imiza ion.
• Enhanced ci izen expe ience wi h eal- ime in o ma ion and sa e
en i onmen s.
3. Requi emen s o dynamic and eal ime da a
s eams
De eloping suppo o dynamic and eal- ime da a s eams in he LUMI AI Fac o y en i onmen equi es
a ca e ul analysis o mul iple ypes o equi emen s. These include echnical equi emen s, such as he
abili y o inges and p ocess high- eloci y da a s eams wi h low la ency, ensu e da a in eg i y, and scale
o handle la ge olumes. A chi ec u al equi emen s mus also be conside ed, including e en -d i en
p ocessing, bu e ing mechanisms, and in eg a ion wi h HPC wo k lows. In addi ion, ope a ional
equi emen s, such as esilience du ing main enance, in e ope abili y wi h exis ing ools, and suppo o
di e se p o ocols, a e essen ial o eliabili y. Finally, go e nance and compliance equi emen s,
including secu i y, access con ol, and adhe ence o FAIR p inciples, mus be add essed o ensu e us
and sus ainabili y. This chap e ou lines hese equi emen s and explains why hey a e c i ical o
enabling eal- ime and dynamic da a capabili ies in esea ch and inno a ion wo k lows.
3.1 Technical, a chi ec u al, and ope a ional equi emen s
S eaming da a om IoT de ices o a da a analysis en i onmen in ol es se e al key equi emen s ac oss
ha dwa e, so wa e, and ne wo k laye s. Na u ally he use case de ines he needed unc ions. LUMI AIF
en i onmen s eaming da a capabili ies will be con inuously de eloped based on expe iences om pilo
case s udies, i s wo o which a e p esen ed in chap e 7. Case S udies in his epo . In his chap e we
p esen on possible classi ica ion o he essen ial componen s. The sui abili y o he classi ica ion will be
examined in he a o e men ioned case s udies.
De ice-Le el Requi emen s
• Senso s and Ac ua o s: De ices mus be equipped wi h senso s o collec da a (e.g.,
empe a u e, mo ion, humidi y).
• Connec i i y Modules: Suppo o communica ion p o ocols like Wi-Fi, Blue oo h, Zigbee,
LoRaWAN, NB-IoT, o 5G.
Communica ion and Ne wo king
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
19
• Low-La ency, High-Bandwid h Ne wo ks: Especially impo an o eal- ime applica ions
(e.g., 5G o E he ne ).
• Reliable P o ocols:
• MQTT: Ligh weigh , ideal o cons ained de ices.
• CoAP: Op imized o low-powe de ices.
• HTTP/HTTPS: Common o REST ul APIs.
• WebSocke s: Fo bi-di ec ional, eal- ime communica ion.
• Secu i y: TLS/SSL enc yp ion, VPNs, and secu e au hen ica ion (e.g., OAu h2, oken-based
access).
Da a Inges ion and S eaming Pla o ms
• Message B oke s: Tools like Apache Ka ka, Rabbi MQ, o MQTT b oke s o bu e and ou e
da a.
• Log collec ion and shipping ools (e.g., Logs ash and , Filebea )
• S eaming Wo k low Engines o compu a ion and analy ics (e.g. Apache Ai low, Tempo al,
A go )
• S eam P ocessing Engines o o ches a ion.
• Scalabili y: Abili y o handle high- h oughpu and bu s y da a loads.
Da a S o age and Managemen
• Time-Se ies Da abases: In luxDB, TimescaleDB, o P ome heus o s o ing senso da a.
• Da a Lakes/Wa ehouses: Fo long- e m s o age and ba ch analy ics (e.g., AWS S3, Azu e Da a
Lake, Google BigQue y).
• Me ada a Managemen : Tagging and indexing o e icien que ying.
Da a Analysis En i onmen
• Real- ime s eam p ocessing engine ools like Apache Flink
• Real-Time Dashboa ds: Tools like G a ana, Kibana, o Powe BI o isualiza ion.
• Machine Lea ning Pipelines: In eg a ion wi h pla o ms like Tenso Flow, PyTo ch, o cloud ML
se ices.
• Ale ing and Au oma ion: T igge ing ac ions based on h esholds o anomalies. Fo his could
be con igu ed o example Apache Spa k S eaming o and Apache Flink
1
.
Some o he ools migh ha e so called Helm Cha o launch and con igu e he se up in he use 's name
space. AI algo i hms can be called also om ools like Flink, and Ka ka ( ia API). Some so wa e may be
a ailable in he clus e o be consumed by all use s, which make i easie o he use s. Also, some
so wa e may equi e special p i ileges o ins all hose, which is no possible ia Helm Cha based
me hod. In he wo s case, no all so wa e can be ins alled on he Kube ne es.
1
h ps://doi.o g/10.1109/ISDFS58141.2023.10131800
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
20
Fo log managemen he e a e specialized ools. One needs o be awa e o he possible limi a ions o he
pla o m such as he con aine s canno use oo use accoun . The so wa e can be made eadily a ailable
in o m o con aine s and helm cha s.
3.2 In eg a ion capabili ies
In eg a ing dynamic and eal- ime da a s eams in o a High-Pe o mance Compu ing (HPC) en i onmen
such as LUMI equi es specialized capabili ies ha b idge he gap be ween adi ional ba ch-o ien ed
HPC wo k lows and he con inuous, e en -d i en na u e o s eaming da a. These capabili ies ensu e
ha da a lows seamlessly om dis ibu ed sou ces o HPC esou ces o p ocessing, analy ics, and AI
wo kloads.
1. Mul i-P o ocol Da a Inges ion
• Requi emen : Abili y o inges da a om he e ogeneous sou ces using p o ocols such as
MQTT, Ka ka, HTTP/HTTPS, and WebSocke s.
• Ra ionale: IoT de ices, APIs, and ex e nal sys ems o en use di e en communica ion
s anda ds. A lexible inges ion laye ensu es in e ope abili y and minimizes in eg a ion
o e head.
2. Real-Time Bu e ing and Backp essu e Managemen
• Requi emen : Middlewa e ha can bu e incoming s eams and manage backp essu e o
p e en da a loss du ing HPC job scheduling o main enance windows.
• Ra ionale: HPC sys ems a e op imized o ba ch jobs, which may no align wi h con inuous
da a a i al. Bu e ing decouples p oduce s om consume s, ensu ing eliabili y.
3. S eam P ocessing F amewo k In eg a ion
• Requi emen : Suppo o amewo ks like Apache Flink o Spa k S eaming, deployed in
con aine ized en i onmen s o ia o ches a ion pla o ms (e.g., Kube ne es).
• Ra ionale: Enables eal- ime analy ics, e en -d i en igge s, and p e-p ocessing be o e da a
eaches HPC compu e nodes.
4. Hyb id Edge-HPC In eg a ion
• Requi emen : Edge agen s capable o il e ing, agg ega ing, and ansla ing da a be o e
ansmission o he HPC cen e .
• Ra ionale: Reduces bandwid h usage, minimizes la ency, and ensu es only ele an da a is sen
o he HPC sys em.
5. P o ocol and Fo ma T ansla ion Se ices
• Requi emen : Se ices o con e IoT-na i e p o ocols and o ma s (e.g., MQTT, CoAP, bina y
payloads) in o HPC-compa ible o ma s (e.g., JSON, Pa que ).
• Ra ionale: Ensu es seamless in eg a ion wi h HPC s o age and analy ics pipelines.
6. Secu e and Complian Da a T ans e
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
21
• Requi emen : End- o-end enc yp ion (TLS/SSL), quan um-sa e anspo , and ole-based
access con ol.
• Ra ionale: P o ec s sensi i e da a and ensu es compliance wi h egula ions such as GDPR.
7. Obse abili y and Moni o ing
• Requi emen : Real- ime dashboa ds and ale ing sys ems (e.g., G a ana, P ome heus) o
moni o ing da a lows, la ency, and sys em heal h.
• Ra ionale: P o ides isibili y in o s eaming pipelines and enables p oac i e issue esolu ion.
8. Scalable S o age and Re ie al
• Requi emen : In eg a ion wi h high-pe o mance pa allel ile sys ems (e.g., Lus e) and ime-
se ies da abases o sho - e m analy ics.
• Ra ionale: Suppo s bo h eal- ime and his o ical analysis wi hou comp omising pe o mance.
9. AI/ML Pipeline In eg a ion
• Requi emen : Abili y o eed s eaming da a in o AI in e ence se ices o online lea ning
pipelines.
• Ra ionale: Enables adap i e AI models and eal- ime decision-making.
3.3. Secu i y and compliance
Gene al echnical secu i y equi emen s
• Secu e Da a T ansmission
• Use TLS/SSL enc yp ion o all s eaming p o ocols (MQTT, HTTP, Ka ka).
• Implemen quan um-sa e enc yp ion o long- e m con iden iali y (as es ed in LUMI’s
op ical link).
• Au hen ica ion and Au ho iza ion
• En o ce s ong au hen ica ion (OAu h2, oken-based access).
• Apply ole-based access con ol (RBAC) o HPC esou ces and da a pipelines.
• Ne wo k Segmen a ion and Fi ewalls
• Isola e inges ion se ices om co e HPC compu e nodes.
• Use i ewalls and VPNs o ex e nal connec ions.
• Da a Go e nance and Compliance Laye
• Real- ime da a alida ion and anonymiza ion o sensi i e in o ma ion.
• Implemen policy-d i en access con ol aligned wi h GDPR and o he egula ions.
• Moni o ing and Audi Logging
• Con inuous moni o ing o da a lows using ools like P ome heus and G a ana.
• Main ain immu able audi logs o aceabili y and compliance epo ing.
• Resilience and Faul Tole ance
• Bu e ing and backp essu e managemen (e.g., Ka ka) o p e en da a loss du ing
ou ages.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
22
• Redundan inges ion nodes o high a ailabili y.
• Secu e Con aine O ches a ion
• Deploy inges ion and p ocessing se ices in Kube ne es namespaces wi h s ic secu i y
policies.
• A oid oo p i ileges in con aine s; en o ce Pod Secu i y S anda ds.
Secu i y and Compliance Measu es in EU Con ex
Measu e
Desc ip ion
Rele an EU Regula ion
TLS/SSL & Quan um-Sa e
Enc yp ion
Secu e da a ansmission o
s eaming p o ocols
GDPR A . 32 (Secu i y o
P ocessing), NIS2
(Cybe secu i y)
S ong Au hen ica ion & MFA
OAu h2, ce i ica es, mul i-
ac o au hen ica ion o access
con ol
GDPR A . 32, NIS2
Role-Based Access Con ol
(RBAC)
Res ic access o HPC
esou ces and da a pipelines
based on oles
GDPR A . 25 (Da a P o ec ion
by Design), NIS2
Da a Minimiza ion &
Anonymiza ion
Remo e o pseudonymize
pe sonal da a be o e inges ion
GDPR A . 5 (Da a
Minimiza ion), A . 32
Ne wo k Segmen a ion &
Fi ewalls
Isola e inges ion se ices om
HPC compu e nodes
NIS2 (Ne wo k and In o ma ion
Secu i y)
Immu able Audi Logs
Main ain logs o accoun abili y
and inciden epo ing
GDPR A . 30 (Reco ds o
P ocessing), NIS2
Inciden Response Plan
P ocedu es o b each
no i ica ion wi hin 24–72 hou s
GDPR A . 33 (B each
No i ica ion), NIS2
Da a P o ec ion Impac
Assessmen
Assess isks o s eaming
p ojec s
GDPR A . 35 (DPIA)
Vendo Risk Managemen
Secu i y checks and con ac ual
clauses o hi d-pa y da a
p o ide s
GDPR A . 28 (P ocesso
Obliga ions), NIS2
T aining & Awa eness
GDPR and cybe secu i y
aining o s a
GDPR A . 39 (DPO Tasks),
NIS2
Compliance Audi s & Pen
Tes ing
Regula audi s and ulne abili y
assessmen s
GDPR A . 32, NIS2
O ganiza ional Measu es
• Da a P o ec ion Policies
• De ine clea policies o da a classi ica ion, e en ion, and dele ion.
• Ensu e compliance wi h GDPR and sec o -speci ic egula ions.
• Access Managemen P ocedu es
• Regula ly e iew and upda e use pe missions.
• Implemen leas p i ilege p inciple o all oles.
• Inciden Response Plan
• Es ablish a documen ed p ocess o secu i y b eaches o da a leaks.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
23
• Conduc egula inciden esponse d ills.
• Vendo and Thi d-Pa y Risk Managemen
• Assess secu i y pos u e o ex e nal da a p o ide s.
• Include da a p o ec ion clauses in con ac s.
• T aining and Awa eness
• P o ide secu i y aining o de elope s, da a enginee s, and HPC ope a o s.
• P omo e awa eness o phishing and social enginee ing isks.
• Compliance Audi s
• Schedule egula in e nal and ex e nal audi s.
• Main ain documen a ion o egula o y inspec ions.
4. Fi s e sion o LUMI AIF ools o dynamic and
eal ime da a s eams
As o mid-2025, he LUMI supe compu e is being p epa ed o ad anced eal- ime and dynamic da a
s eaming capabili ies, pa icula ly in suppo o AI and high-pe o mance compu ing (HPC) wo kloads.
He e a e he key de elopmen s:
• High-Speed Da a T ans e In as uc u e: A majo miles one was achie ed in June 2025 when
CSC, SURF, and Nokia success ully es ed a 1.2 e abi pe second (Tbi /s) quan um-sa e ibe -
op ic connec ion be ween Ams e dam and Kajaani, Finland—whe e LUMI is hos ed. This
in as uc u e is designed o suppo massi e, con inuous da a lows ("elephan lows") and is
c ucial o eal- ime da a s eaming in HPC and AI applica ions
• Flexible E he ne and Op ical Ne wo king: The es u ilized Flexible E he ne (FlexE) and
high-capaci y op ical anspo echnologies, enabling LUMI o handle la ge-scale, eal- ime
da a ans e s ac oss long dis ances. This is essen ial o applica ions like aining la ge AI
models o s eaming senso da a om IoT de ices
• Real-Time En i onmen al Moni o ing: LUMI is al eady being used o eal- ime analysis o
ada da a om ICEYE’s mic osa elli e sys em. This allows o nea -ins an aneous gene a ion o
e ain images o de ec en i onmen al e en s like loods o i es, ega dless o wea he
condi ions
• Suppo o AI Fac o ies and LUMI-AI: The in as uc u e upg ades a e also in p epa a ion o
LUMI-AI, a nex -gene a ion AI-op imized supe compu e . This sys em will u he enhance eal-
ime da a p ocessing capabili ies, suppo ing AI ac o ies and o he da a-in ensi e applica ions
LUMI Capabili ies s. Real-Time & Dynamic Da a S eaming Requi emen s
Ca ego y
Gene al Requi emen s o
Real-Time & Dynamic Da a
LUMI Capabili ies (Based on
Docs & Public In o)
Da a Inges ion
Con inuous, high- h oughpu
inges ion om dis ibu ed
sou ces (e.g., IoT, senso s, APIs)
Suppo s high-speed da a
ans e ia 1.2 Tbi /s op ical
link; in eg a ion wi h sa elli e
da a (e.g., ICEYE)
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
24
La ency
Low-la ency p ocessing and
esponse imes o eal- ime
decision-making
Designed o high-pe o mance
compu ing; la ency depends on
wo kload and a chi ec u e
(ba ch-o ien ed by de aul )
S eaming P o ocols
Suppo o MQTT, Ka ka,
WebSocke s, HTTP/2, e c.
No na i ely documen ed;
in eg a ion likely equi es
cus om middlewa e o edge
p ep ocessing
Edge In eg a ion
Abili y o p ocess da a a he
edge be o e sending o cen al
sys ems
No na i e edge compu ing
laye ; elies on ex e nal sys ems
o p ep ocess be o e inges ion
Scalabili y
Elas ic scaling o handle bu s y
o g owing da a s eams
Ex emely scalable compu e
and s o age in as uc u e ( op-
ie HPC sys em)
Real-Time Analy ics
S eam p ocessing engines
(e.g., Flink, Spa k S eaming)
o on- he- ly analy ics
No explici ly documen ed;
use s can deploy cus om
analy ics pipelines using
a ailable compu e nodes
Da a S o age
Time-se ies da abases, da a
lakes, and as -access s o age
o s eaming da a
High-pe o mance pa allel ile
sys ems (e.g., Lus e); sui able
o la ge-scale da a s o age and
e ie al
Secu i y & Compliance
End- o-end enc yp ion, access
con ol, GDPR/indus y
compliance
Quan um-sa e da a ans e
es ed; secu e in as uc u e
managed by CSC and Eu oHPC
AI/ML In eg a ion
Real- ime model in e ence and
online lea ning capabili ies
Suppo s la ge-scale AI aining;
LUMI-AI expansion will enhance
eal- ime AI capabili ies
Moni o ing & Obse abili y
Real- ime dashboa ds, logs,
and me ics o da a low
isibili y
Moni o ing ools a ailable o
HPC jobs; eal- ime
obse abili y o s eaming no
na i ely documen ed
LUMI o e s excep ional compu e and da a ans e capabili ies, making i well-sui ed o high- olume,
high-pe o mance wo kloads. Howe e , o eal- ime and dynamic da a s eaming, i ypically equi es
cus om in eg a ion laye s o ex e nal ools o mee he esponsi eness and p o ocol lexibili y expec ed
in IoT and s eaming en i onmen s.
5. A chi ec u e o he Tools Package
5.1 Co e componen s and design p inciples
The i s e sion o he LUMI AI Fac o y (LUMI AIF) ools o dynamic and eal- ime da a s eams builds
upon he high-pe o mance capabili ies o he LUMI supe compu e while add essing he unique
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
25
equi emen s o s eaming and IoT-d i en wo kloads. The design emphasizes modula i y, scalabili y, and
in e ope abili y o accommoda e di e se use cases and e ol ing echnologies.
Co e Componen s
High-Speed Da a T ans e Laye
• U ilizes he ecen ly es ed 1.2 Tbi /s quan um-sa e op ical link be ween Kajaani and
Ams e dam o suppo massi e, con inuous da a lows.
• Inco po a es Flexible E he ne (FlexE) and ad anced op ical ne wo king o low-
la ency, high- h oughpu connec i i y.
Da a Inges ion and Bu e ing Se ices
• Middlewa e laye o inges ing eal- ime s eams om IoT de ices, APIs, and ex e nal
sys ems.
• Suppo s in eg a ion wi h message b oke s (e.g., Apache Ka ka, MQTT) and p o ides
bu e ing o decouple da a p oduce s om consume s.
S eam P ocessing F amewo k
• Con aine ized deploymen o Apache Flink o Spa k S eaming o eal- ime analy ics.
• Enables e en -d i en p ocessing and in eg a ion wi h AI/ML pipelines.
Edge In eg a ion Laye
• Ligh weigh agen s o p ep ocessing and il e ing da a a he edge, educing
bandwid h and la ency.
• Suppo s p o ocol ansla ion (e.g., MQTT → JSON/Pa que ) be o e o wa ding o
LUMI.
S o age and Pe sis ence
• High-pe o mance pa allel ile sys ems (e.g., Lus e) o la ge-scale s o age.
• Op ional in eg a ion wi h ime-se ies da abases o sho - e m e en ion and analy ics.
Moni o ing and Obse abili y
• Dashboa ds and ale ing sys ems (e.g., G a ana, P ome heus) o eal- ime isibili y
in o da a lows and sys em heal h.
Secu i y and Compliance Laye
• End- o-end enc yp ion, quan um-sa e da a ans e , and ole-based access con ol.
• Compliance wi h GDPR and o he ele an egula ions.
Design P inciples
• Modula i y: Each componen is loosely coupled, enabling independen scaling and upg ades.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
32
Con ex
Uni e si y o Oulu ITEE Facili y
3
has Toyo a RAV4 ehicles o ehicula esea ch. The ehicle is
equipped wi h se e al senso s such as lida , wo he mal came as and s e eocame a. They p oduce
da a a a high a e, a leas 10gb/min, which amoun s o hund eds i no e aby es o da a pe d i ing
session. The da a will be sen o he cloud o he supe compu e , whe e i will be p ocessed and
machine lea ning algo i hms will be un on i . In he u u e, he plan is o ha e he p ocessed da a and
possibly commands passed back on o he ehicle o enable e.g. sel -d i ing ca unc ionali ies. Ideal
case would be a ull digi al win o he ca , which could be un au oma ically.
Handling la ge amoun s o da a wi h minimal delay in eal- ime is c ucial o his use case.
Cu en Challenge
Poo connec ion, high da a a e and he need o minimizing delay a e all cu en challenges. The
cu en se up is based on a MQTT connec ion o Rah i, bu MQTT connec ion does no sui well o
sending hea y ideo ma e ial h ough i . Also, ou u u e wo k includes se ing up ML algo i hms un on
CSC se ices om he da a go en om he ehicle. Cu ene ly we ha e no ied o un any ML
algo i hms on e.g. Mah i om he da a om he Rah i and ehicle in eal ime, as Mah i has he SLURM
ba ch jobs and queuing.
Da a Managemen Requi emen s o S eaming
• Real- ime p ocessing
• S eaming la ge amoun o da a om di e en senso ypes
• S eaming also ideo à need o e.g. UDP connec ion (o he s possible as well, bu his seems o
be a sui able candida e o ideo s eaming)
• Relaying da a back o he ca
• Minimal delay / nea - eal- ime unc ioning o he whole se up
Fu u e Vision
In he u u e ideally he ca could elay all he senso da a i ecei es o he CSC cloud and
supe compu e s, he algo i hms could be un on CSC se ices, and hen hey would be passed back
on o he ehicles. Ideally his would enable us o ha e a eal- ime digi al win o he ehicle, ha could
be ei he au oma ically d i en o d i en emo ely.
Pilo S udy Design: Sma ca –LUMI AIF In eg a ion
Objec i es
• Replace Rah i/cPou a backend wi h LUMI o p ep ocessing and HPC
wo kloads.
• Add LiDAR (UDP→PCAP) alongside CAN and p o e end- o-end inges
wi h mic o-ba ching.
Scope
• Use exis ing Je son CAN-bus in eg a ion and ex end i wi h LiDAR UDP
s eaming.
3
h ps://www.oulu. i/en/uni e si y/ acul ies-and-uni s/ acul y-in o ma ion- echnology-and-elec ical-enginee ing
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
33
• Mig a e he cu en MQTT → Teleg a → In luxDB → G a ana pipeline
o LUMI.
• In eg a e he g aphical managemen UI in o he Je son, enabling local
con ol o e :
• Enabled senso s.
• Ta ge In luxDB bucke s.
• UDP/MQTT endpoin con igu a ion.
• Demons a e end- o-end eal- ime mul i-senso da a s eaming
wi h moni o ing dashboa ds in G a ana.
E alua ion me ics
• Pe o mance: Measu ed la ency and h oughpu o CAN + LiDAR
s eams.
• Scalabili y: How he sys em handles addi ional senso s o highe da a
a es.
• Reliabili y: LonPacke loss, synch oniza ion accu acy, and sys em
up ime.
Technical se up
Componen
LUMI AIF Adap a ion
Compu e
LUMI VMs unning Docke Compose o In luxDB, Teleg a , G a ana, Mosqui o
o example
LiDAR
UDP-based PCAP s eaming a high h oughpu
Je son
en i onmen
Py hon-based CAN + LiDAR collec ion, REST API backend + g aphical UI
Da a ans e
Globus o bulk ( esumable, scheduled); Aspe a/FASP (UDP) o accele a ed
nea ‑ eal‑ ime hops when WAN condi ions equi e i
Ne wo king
MQTT o CAN da a; UDP o LiDAR
Da a wo k lows o es
• Baseline CAN s eaming: Ve i y exis ing MQTT → In luxDB pipeline wo ks a e mig a ion.
• LiDAR UDP in eg a ion: Benchma k packe inges ion a es and packe loss ole ance.
• Combined inges ion: S ess es simul aneous CAN + LiDAR da a low in o In luxDB.
Da a managemen equi emen s
• Real- ime inges ion: Handle simul aneous CAN, LiDAR, and op ional ideo s eams.
• High- h oughpu UDP handling: Op imize ne wo k bu e sizes and Teleg a con igu a ion.
• Dynamic da a ou ing: Enable he Je son UI o econ igu e a ge s wi hou backend
edeploymen .
• Visualiza ion and moni o ing: G a ana dashboa ds mus expose la ency, h oughpu , and
packe s a is ics.
• Secu i y:
o Token-based access o In luxDB and G a ana.
o Secu i y g oups o UDP, MQTT, and HTTPS.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
34
o Op ional VPN unneling o senso da a.
7. Fu u e de elopmen s
This chap e ou lines he key de elopmen oppo uni ies iden i ied du ing he cu en phase o wo k. I
does no ep esen a ixed implemen a ion plan bu a he highligh s a eas whe e enhancemen s could
signi ican ly imp o e unc ionali y and pe o mance. A dynamic de elopmen oadmap, e lec ing
p io i ies and imelines, will be main ained and egula ly upda ed on he LUMI AI Fac o y websi e ( o be
published du ing Sep embe 2025). This ensu es anspa ency and p o ides s akeholde s wi h an up- o-
da e iew o planned and ongoing imp o emen s.
To suppo eal- ime and dynamic da a in eg a ion, o ganiza ions a e adop ing echnologies such as:
• S eam p ocessing engines (e.g., Apache Ka ka, Flink, Spa k S eaming)
• E en -d i en a chi ec u es ha igge ac ions based on da a changes
• Edge compu ing o p ocess da a close o i s sou ce
• Cloud-na i e pla o ms ha scale dynamically wi h da a olume
These echnologies allow o seamless in eg a ion o da a om IoT de ices, mobile apps, en e p ise
sys ems, and ex e nal APIs in o a cohesi e analy ical en i onmen .
Based on he compa ison be ween LUMI’s cu en capabili ies and he gene al equi emen s o dynamic
and eal- ime da a s eaming, he ollowing key se ice de elopmen needs eme ge:
1. Real-Time Da a Inges ion Laye
• Need: A middlewa e o ga eway se ice o inges and bu e eal- ime da a s eams om IoT
de ices o ex e nal APIs.
• Why: LUMI does no na i ely suppo s eaming p o ocols like MQTT o Ka ka.
• Solu ion Di ec ion: De elop o in eg a e a scalable inges ion laye ha can p ep ocess and ou e
da a o LUMI’s compu e nodes.
2. S eam P ocessing F amewo k In eg a ion
• Need: Suppo o eal- ime analy ics engines such as Apache Flink, Spa k S eaming, o cloud-
na i e equi alen s.
• Why: LUMI is op imized o ba ch HPC wo kloads; eal- ime s eam p ocessing is no
documen ed.
• Solu ion Di ec ion: Con aine ized o o ches a ed deploymen o s eam p ocessing
amewo ks on LUMI o in hyb id cloud-edge se ups.
3. Edge Compu ing Enablemen
• Need: P ep ocessing and il e ing da a a he edge be o e ansmission o LUMI.
• Why: Reduces la ency, bandwid h usage, and o loads non-c i ical p ocessing.
• Solu ion Di ec ion: De elop edge agen s o mic ose ices ha handle local compu a ion and
s eam only ele an da a o LUMI.
D5.3 Tools o in eg a ing dynamic and eal- ime da a s eams
35
4. P o ocol and Fo ma T ansla ion Se ices
• Need: Se ices o ansla e IoT-na i e p o ocols (e.g., MQTT, CoAP) in o o ma s LUMI can
p ocess (e.g., JSON, CSV, Pa que ).
• Why: LUMI lacks na i e suppo o ligh weigh IoT p o ocols.
• Solu ion Di ec ion: Build p o ocol adap e s o use exis ing open-sou ce b idges.
5. Real-Time Moni o ing and Obse abili y Tools
• Need: Dashboa ds and ale ing sys ems o moni o ing da a lows and sys em heal h in eal
ime.
• Why: HPC sys ems like LUMI a e no ypically equipped wi h eal- ime obse abili y ools.
• Solu ion Di ec ion: In eg a e ools like G a ana, P ome heus, o cus om dashboa ds ailo ed o
s eaming wo kloads.
6. AI/ML In e ence Se ices
• Need: Real- ime model in e ence capabili ies o s eaming da a.
• Why: LUMI is op imized o aining la ge models, bu eal- ime in e ence is no documen ed.
• Solu ion Di ec ion: Deploy ligh weigh in e ence se ices (e.g., ONNX Run ime, Tenso RT) on
edge o cloud nodes, wi h LUMI used o model aining and upda es.
7. Da a Go e nance and Compliance Laye
• Need: Real- ime da a alida ion, anonymiza ion, and access con ol mechanisms.
• Why: S eaming da a o en includes sensi i e o egula ed in o ma ion.
• Solu ion Di ec ion: Implemen policy-d i en da a go e nance se ices ha ope a e inline wi h
da a s eams.
8. Conclusion
Real- ime and dynamic da a in eg a ion is a c i ical enable o ad anced AI and HPC applica ions. This
epo has iden i ied he co e equi emen s o suppo ing such capabili ies wi hin he LUMI AI Fac o y
en i onmen and p esen ed he i s e sion o ools designed o mee hese needs.
While LUMI o e s excep ional compu e and da a ans e in as uc u e, addi ional componen s a e
equi ed o ully suppo s eaming da a wo k lows. These include inges ion laye s, s eam p ocessing
amewo ks, edge compu ing agen s, and obse abili y ools. Secu i y and compliance measu es mus
also be in eg a ed o ensu e us wo hy and sus ainable da a handling.
The case s udies p esen ed alida e he ele ance o hese equi emen s and demons a e he po en ial
o scalable, esponsi e AI wo k lows. Fu u e de elopmen will ocus on modula , use -cen ic solu ions
ha enable elas ic scaling, p o ocol in e ope abili y, and eal- ime decision suppo .
By add essing hese p io i ies, he LUMI AI Fac o y will s eng hen i s ole as a Eu opean hub o AI
inno a ion, suppo ing esea ch and de elopmen ac oss domains ha ely on dynamic da a— om
en i onmen al science o indus ial au oma ion.
