Recei ed July 7, 2021, accep ed July 23, 2021, da e o publica ion July 28, 2021, da e o cu en e sion Augus 6, 2021.
Digi al Objec Iden i ie 10.1109/ACCESS.2021.3100706
Towa ds Au onomous De ense o SDN Ne wo ks
Using MuZe o Based In elligen Agen s
JON GABIRONDO-LÓPEZ 1,2, JON EGAÑA 1, JOSE MIGUEL-ALONSO 3, (Membe , IEEE),
AND RAUL ORDUNA URRUTIA1
1Vicom ech Founda ion, Basque Resea ch and Technology Alliance (BRTA), 20009 Donos ia-San Sebas ián, Spain
2Facul y o In o ma ics, Uni e si y o he Basque Coun y (UPV/EHU), 20018 Donos ia-San Sebas ián, Spain
3Depa men o Compu e A chi ec u e and Technology, Uni e si y o he Basque Coun y (UPV/EHU), 20018 Donos ia-San Sebas ián, Spain
Co esponding au ho : Jon Gabi ondo-López ([email p o ec ed])
This wo k was suppo ed in pa by he Spanish Cen e o he De elopmen o Indus ial Technology (CDTI) h ough he P ojec
ÉGIDA—RED DE EXCELENCIA EN TECNOLOGIAS DE SEGURIDAD Y PRIVACIDAD unde G an CER20191012, in pa by he
Spanish Minis y o Science and Inno a ion unde G an PID2019-104966GB-I00, in pa by he Basque Business De elopmen Agency
(SPRI)-Basque Coun y Go e nmen ELKARTEK P og am h ough he p ojec s TRUSTIND unde G an KK-2020/00054 and 3KIA
unde G an KK-2020/00049, and in pa by he Basque Coun y P og am o G an s o Resea ch G oups unde G an IT-1244-19.
ABSTRACT The So wa e De ined Ne wo king (SDN) pa adigm enables he de elopmen o sys ems ha
cen ally moni o and manage ne wo k a ic, p o iding suppo o he deploymen o machine lea ning-
based sys ems ha au oma ically de ec and mi iga e ne wo k in usions. This pape p esen s an in elligen
sys em capable o deciding which coun e measu es o ake in o de o mi iga e an in usion in a so wa e
de ined ne wo k. The in e ac ion be ween he in ude and he de ende is posed as a Ma ko game and
MuZe o algo i hm is used o ain he model h ough sel -play. Once ained, he model is in eg a ed wi h
an SDN con olle , so ha i is able o apply he coun e measu es o he game in a eal ne wo k. To measu e
he pe o mance o he model, a acke s and de ende s wi h di e en aining s eps ha e been con on ed
and he sco es ob ained by each o hem, he du a ion o he games and he a io o games won ha e been
collec ed. The esul s show ha he de ende is capable o deciding which measu es minimize he impac o
he in usion, isola ing he a acke and p e en ing i om comp omising key machines in he ne wo k.
INDEX TERMS Au oma ed esponse, cybe secu i y, in elligen agen s, Ma ko games, MuZe o, ne wo k
secu i y, OpenFlow, so wa e de ined ne wo king.
I. INTRODUCTION
The numbe o In e ne use s has g own conside ably in
ecen yea s and mo e and mo e se ices— om e-comme ce
o banking—a e p o ided o e he In e ne . Consequen ly,
no only he numbe , bu also he se e i y o cybe a acks
on o ganiza ions and businesses has been inc easing, causing
millions o dolla s in losses [1].
En e p ise secu i y sys ems ha e adi ionally been
designed and implemen ed manually by expe pe sonnel
and he esponse o a acks o in usions has also been
ca ied ou by hose echnicians. The new ne wo k in as-
uc u e pa adigm in oduced by So wa e De ined Ne wo k-
ing (SDN) has enabled he de elopmen o au oma ic a ack
de ec ion and mi iga ion sys ems based on machine lea n-
ing echniques [2]–[6]. Compa ed o con en ional de ec ion
The associa e edi o coo dina ing he e iew o his manusc ip and
app o ing i o publica ion was Nabil Benama .
sys ems o human-d i en esponse s a egies, hese sys ems
can de ec a acks as e and mo e accu a ely, and e en imple-
men coun e measu es au onomously and au oma ically, min-
imizing he eac ion and esponse ime o an a ack and hus
educing he damage caused in he ne wo k.
Addi ionally, he de elopmen o ein o cemen lea n-
ing algo i hms capable o ou pe o ming human ma ks in
boa d games—such as chess o Go—has os e ed he idea
o app oaching ne wo k in usion as i i we e a Ma ko
game, using hese algo i hms o ain in elligen agen s ha
can au onomously ind secu i y s a egies o mi iga e in u-
sions [7].
This pape p esen s an in elligen sys em which can mini-
mize he impac caused by an in usion in an SDN ne wo k by
au onomously choosing and execu ing adequa e coun e mea-
su es. The p oblem is posed as a pa ially obse ed Ma ko
game in which he a acke ( he in ude ) ies o comp omise
a c i ical machine and he de ende (an au oma ic secu i y
107184
This wo k is licensed unde a C ea i e Commons A ibu ion-NonComme cial-NoDe i a i es 4.0 License.
Fo mo e in o ma ion, see h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/ VOLUME 9, 2021
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo k
agen ha manages he in as uc u e) ies o educe he
impac o he a ack. The game has been designed aking
in o accoun ha i mus be able o ep esen he s a e o he
nodes o a eal ne wo k and ha he ac ions o he de ende
in he game mus be implemen able by an SDN con olle .
The model has been ained using he MuZe o model-based
ein o cemen lea ning algo i hm [8].
The majo con ibu ions o his pape a e:
•The design o a Ma ko game sui able o ep esen a ack
a emp s o a compu e ne wo k in which nodes ha e
ulne abili ies.
•The implemen a ion o he necessa y coun e measu es
by means o an SDN con olle and OpenFlow-enabled
swi ches.
•The implemen a ion o a i ual en i onmen in which
he coun e measu es chosen by he de ende a e ca ied
ou au onomously in an emula ed SDN ne wo k.
The es o his pape is o ganized as ollows: Sec ion II
in oduces he con ex o SDN and ein o cemen lea ning
and p esen s he s a e o he a in bo h ields. Sec ion III
p esen s he p oposal made in his wo k, explaining how
he game wo ks and i s in eg a ion wi h an SDN ne wo k.
Sec ion IV shows he in o ma ion ega ding he aining o he
model and he esul s ob ained. Finally, Sec ion Ve alua es
hese esul s and highligh s some possible lines o u u e
wo k.
II. BACKGROUND AND STATE OF THE ART
A. BACKGROUND
1) SOFTWARE DEFINED NETWORKING
In ecen yea s, he applica ions and se ices o e ed on he
In e ne ha e become inc easingly complex and demanding.
This has highligh ed he need o a pa adigm shi in he
wo ld o ne wo k in as uc u es, as con en ional ne wo ks
lack he dynamism and adap abili y equi ed by he new
pla o ms [9].
Con en ional ne wo ks a e composed o elemen s (such
as ou e s and swi ches) usually ea ed as black boxes ha
ely on limi ed o manu ac u e -speci ic con ol in e aces.
These de ices a e ypically con igu ed indi idually, and a -
ic managemen decisions a e made di ec ly a he de ice
le el, in e mingling he con ol plane esponsible o making
such decisions wi h he da a plane composed o he ne -
wo k elemen s [10], [11]. The lack o independence be ween
hese wo planes makes i ex emely di icul o dynamically
adap he ne wo k o he needs o he applica ions deployed,
o o cope wi h ce ain ype o e en s. This p oblem is one
o he main causes o he ‘‘ossi ica ion o he In e ne ’’:
he de elopmen o new p o ocols and in as uc u es has
been se e ely hampe ed by he e y a chi ec u e o ne wo k
elemen s [10], [12].
So wa e de ined ne wo king p oposes sys ems in which
he con ol plane and he da a plane a e comple ely decou-
pled, allowing cen alized con igu a ion o he in as uc-
u es, as shown in Fig. 1. Al hough he e a e di e en
FIGURE 1. S uc u al di e ences be ween (a) a con en ional ne wo k and
(b) a SDN.
SDN a chi ec u es, his pape only conside s hose based on
he OpenFlow p o ocol [12] in he communica ion be ween
he con ol plane and he da a plane, as i is a widely
used p o ocol endo sed by he Open Ne wo king Founda-
ion (ONF)— he non-p o i o ganiza ion dedica ed o he
de elopmen and s anda diza ion o SDN ne wo ks [11].
The elemen s ha cons i u e he ne wo k (known as Open-
Flow swi ches) a e only esponsible o o wa ding a ic,
whe eas decisions a e made by he ne wo k con olle . The
con olle ins alls low ables in he swi ches using he Open-
Flow p o ocol. The ables consis o lows ha de e mine
how packe s ha mee ce ain c i e ia a e p ocessed and
o wa ded.
A low is composed o di e en elemen s: he Ma ch ields
a e he condi ions ha he incoming packe has o mee (such
as he sou ce IP add ess o he incoming po , o example)
o execu e he ins uc ions de ined in he Ins uc ions ield.
These ins uc ions may include blocking all a ic coming
om some po , o wa ding he ma ching a ic h ough
a speci ic ou pu po o o wa ding some packe s o he
con olle , o example. When an incoming packe ma ches
wi h mo e han one low, he one wi h he highes alue in
he P io i y ield is execu ed. The Coun e s ield collec s he
numbe o packe s p ocessed by ha low, he Timeou ield
de ines how much ime mus pass wi hou inpu s o a low
o be d opped and he Cookie ield is jus an iden i ie se by
he con olle [13]. When a packe ha does no ma ch any
low is ecei ed, he ins uc ions de ined in he special able-
miss low a e execu ed, which may include ac ions such as
d opping he packe o sending i o he con olle o u he
analysis.
VOLUME 9, 2021 107185
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo ks
FIGURE 2. Diag am o he componen s o a SDN and he in e aces
be ween hem.
Fig. 2shows he s uc u e o his ype o SDN ne wo ks,
which consis s o h ee main g oups o componen s (planes)
and wo in e aces ha allow communica ion be ween hem.
The e a e mo e complex s uc u es ha include se e al con-
olle s and mo e in e aces, bu in his pape , o he sake o
simplici y, we only conside he case o a single con olle .
The con olle uses he Sou hbound In e ace (SBI) o com-
munica e wi h he ne wo k de ices. In his way, he con-
olle manages all packe p ocessing by ins alling lows
in he swi ches. The con olle can also collec in o ma-
ion and s a is ics abou he da a plane. The SBI is endo -
independen and he mos widely used is he a o emen ioned
OpenFlow [14].
The con olle also communica es wi h he applica ion
plane h ough a No hbound In e ace (NBI). The applica ion
plane consis s o se e al SDN applica ions ha implemen
a ic con ol and managemen s a egies, such as load bal-
ancing, packe il e ing ( i ewalling), a ic moni o ing, e c.
The e o e, he NBI allows applica ions o p ocess he da a
plane in o ma ion ecei ed by he con olle , pe o m ac ions
a a high le el and ha e he con olle execu e hem on
he in as uc u e. The e a e no s anda dized NBIs, hey a e
con olle -speci ic. Howe e , hey a e ypically buil using
REST APIs [15].
An impo an p ope y o a SDN is i s abili y o eac o di -
e en e en s. The con olle can pe o m an ini ial, p oac i e
con igu a ion o he ne wo k. Howe e , his con igu a ion
can be dynamically modi ied (by adding o emo ing lows),
adap ing i o he a ic obse ed in he ne wo k, whe he
ha mless o dange ous. SDN ne wo ks ha e he e o e been
used on many occasions o design sys ems capable o
de ec ing and esponding dynamically o cybe a acks [6],
[16], [17].
2) REINFORCEMENT LEARNING AND MARKOV GAMES
Rein o cemen Lea ning (RL) is an a ea o machine lea ning
ha s udies how an agen lea ns o make decisions by ollow-
ing a ial-and-e o s a egy [18], [19]. The main elemen s o
he RL models a e shown in Fig. 3. The agen is he subjec o
he aining and he one who has o lea n o pe o m decisions.
The en i onmen ep esen s he wo ld wi h which he agen
in e ac s [20]. In each in e ac ion , he agen ecei es an
obse a ion o ∈Oo he s a e o he en i onmen s ∈S,
FIGURE 3. Summa y o he main elemen s o ein o cemen lea ning
p oblems and he in e ac ions among hem.
whe e Ois he se o obse a ions o he ensemble o
possible s a es S. Based on he obse a ion i decides which
ac ion a ∈A(s ) o ake, whe e A(s ) is he se o possible
ac ions in s a e s . Then, he en i onmen changes in esponse
o he agen ’s ac ion o independen ly. Once he ac ion is exe-
cu ed, he agen ecei es a ewa d +1and a new obse a ion
o he en i onmen , which allows i o e alua e he e ec
o he ac ion jus pe o med, as well as o pe o m a new
s ep.
I is wo h no ing he di e ence be ween he s a e and
an obse a ion o he en i onmen . The s a e ep esen s all
he in o ma ion o he en i onmen and hus de ines i com-
ple ely. An obse a ion, howe e , is a iew ha usually con-
ains only a pa o he in o ma ion abou he s a e. In he
pa icula case whe e he obse a ion includes all he in o -
ma ion abou he s a e, he en i onmen is ully obse ed;
o he wise, he en i onmen is pa ially obse ed. In many
cases, he e m ‘‘s a e’’ is used o ac ually e e o ‘‘obse a-
ion’’, and he symbol s is used ins ead o o . In his pape we
ha e chosen o use an explici exp ession o he obse a ion,
o wo main easons: he i s is ha a main cha ac e is ic
o he model we ha e implemen ed is ha i is based on a
pa ially obse ed en i onmen ; he second is ha he au ho s
o he algo i hm used o ain he model use his no a ion in
hei a icle [8]. A main piece o a RL model is he policy,
i.e., he s a egy ha de e mines wha ac ion he agen should
ake gi en an obse a ion. This unc ion is upda ed as he
model is ained. In gene al, he policy is he p obabili y ha
he agen will ake an ac ion a based on he obse a ion o .
The policy πusually depends on a se o pa ame e s θ, so i s
comple e ep esen a ion would be πθ(a |o ).
A succession o s a es isi ed and ac ions aken, τ=
(s0,a0,s1,a1, . . .), is o en e e ed o as an episode. The
agen ’s main goal is o maximize he sum o all ewa ds
ob ained in an episode. This sum o ewa ds is known as
e u n. A s a egy called discoun ed e u n is o en used,
which makes ewa ds ea ned se e al s eps back wo h less
han ewa ds ea ned close in ime. This e u n is de ined as
R(τ)= +1+γ +2+γ2 +3+. . . =
∞
X
k=0
γk +k+1(1)
whe e γis a pa ame e , 0 ≤γ≤1, known as discoun a e.
Beyond he in ui ion es ablished so a , hose p ocesses
s udied in RL a e Ma ko Decision P ocesses (MDP) which
107186 VOLUME 9, 2021
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo k
a e de ined by a six-componen uple as shown in (2) [21].
M= hS,A,Pss0,Ra
ss0, γ, ρ0i(2)
being
Pa
ss0=P s +1=s0|s =s,a =a(3)
Ra
ss0=E +1|a =a,s =s,s +1=s0(4)
whe e Pa
ss0is he p obabili y o eaching s a e s0by applying
an ac ion ain s a e s, and Ra
ss0is he expec ed ewa d a e
ha ansi ion. The e m ρ0:S7→ [0,1]is he dis ibu ion
o he ini ial s a e [7].
The MDPs a e so named because he ansi ion om s a e s
o s a e s +1sa is ies he Ma ko p ope y (5): i only depends
on he men ioned s a es and no on he es o he p e ious
s a es [20].
P [s +1|s ]=P [s +1|s1,...,s ](5)
As men ioned in he in oduc ion, he model we desc ibe
in his pape is a Ma ko game based on a Pa ially Obse ed
Ma ko Decision P ocess (POMDP), in which he agen
makes decisions based on i s (pa ial) obse a ion o he sys-
em and no di ec ly on he s a e. A a s ep , he obse a ion
o ∈Ois ela ed o he s a e s ∈Sby he unc ion Z, so a
POMDP is de ined as ollows:
MP= hS,A,Pss0,Ra
ss0, γ, ρ0,O,Zi(6)
whe e Ois he se o obse a ions o he sys em and Z=
P [o|s]is he p obabili y o obse ing oin s a e s.
B inging he wo ields oge he , he p oblem posed in RL
can be unde s ood as he sea ch o he op imal policy π∗ ha
maximizes he expec ed alue Eo he sum o ewa ds o an
MDP o a maximum o Ts eps:
π∗=a g max
π
E"T
X
=0
γ +1#(7)
In RL algo i hms, alue unc ions a e de ined, which cal-
cula e he expec ed e u n o ac ions aken om an ini ial s a e
and ollowing a ce ain policy. Fo mally, in he case o MDPs,
he unc ion Vπ(s) called s a e- alue unc ion o policy πis
de ined as
Vπ(s)=Eπ[R|s =s]
=Eπ"∞
X
k=0
γk +k+1|s =s#(8)
and ep esen s he expec ed e u n when he agen ollows a
policy πs a ing om a s a e s.
Analogously, he alue o aking an ac ion ain a s a e s,
he ac ion- alue unc ion o policy π, is also de ined as
Qπ(s,a)=Eπ[R|s =s,a =a]
=Eπ"∞
X
k=0
γk +k+1|s =s,a =a#(9)
which ep esen s he expec ed e u n ob ained by ollowing a
policy πa e pe o ming ac ion ain s a e s.
The Bellman equa ions se ou in (10) and (11) show
ha he alue o a s a e o s a e-ac ion pai is he expec ed
e u n ob ained by ollowing he op imal policy π∗de ined
in (7) [22].
V∗(s)=max
a
E +1+γV∗(s +1)|s =s,a =a(10)
Q∗(s,a)=E +1+γmax
a0Q∗(s +1,a0)|s =s,a =a(11)
Recall ha he ul ima e goal o a RL algo i hm is o ind
he op imal policy π∗. Fo ini e MDPs, (10) has a unique
solu ion, so once V∗has been calcula ed i is ela i ely simple
o ob ain he op imal policy, since i his unc ion is used o
e alua e he sho - e m ac ions (e alua ing he s a e sa i ed
a a e pe o ming a) he policy ollowed when aking he
bes op ion a each s ep is he op imal long- e m policy [20].
Simila ly, using Q∗, he agen no longe has o sea ch o
he new s a e s ha maximizes V(s), bu only has o sea ch o
he ac ion a ha esul s in a highe Q(a). This allows he agen
o make decisions wi hou ha ing o know all he consecu i e
s a es, ha is, wi hou ha ing o know he dynamics o he
en i onmen wi h which i in e ac s.
Es ima ing hese alue unc ions is a undamen al pa
o model- ee algo i hms, which seek o ind he op imal
policy wi hou ha ing a model o he en i onmen , as he
Q-lea ning [23] algo i hm does, o example. On he o he
hand, model-based algo i hms also use such unc ions o
cons uc models ha ha e he same alue unc ions as he
o iginal en i onmen and a e he e o e equi alen . The algo-
i hm MuZe o [8] is one o he mos ecen examples o
model-based algo i hms.
Finally, i only emains o men ion ha decision p o-
cesses in ol ing mul iple agen s a e s udied wi hin game
heo y [24], [25]. These agen s in e ac wi h he en i onmen
simul aneously o in u n and each one ob ains a co espond-
ing payo , hus e ining hei policies. When designing a
game, he ollowing ac o s mus be de e mined, which also
se e o classi y hem [26]:
•Ze o-sum: Whe he he sum o all playe s’ ewa ds is
0 o no . In wo-playe games, ewa ds sum o 0 i bo h
playe s a e s ic ly compe ing agains each o he .
•In o ma ion: Whe he he game s a e is ully o pa -
ially obse able by he playe s.
•De e minism: Whe he he ou come o he game
depends o some ex en on luck.
•Sequen ial: Whe he he agen s in e ac sequen ially o
simul aneously.
•Disc e e: Whe he ac ions a e implemen ed in eal ime
o no .
Speci ically, Ma ko games a e he heo e ical con ex o
mul i-agen RL algo i hms [27]. A Ma ko game MGwi h
Nagen s is de ined by a uple simila o he one gi en in (5):
MG= hS,A1,...,AN,T,R1,...,RN, γ, ρ0i.(12)
VOLUME 9, 2021 107187
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo ks
Compa ed o an MDP, he Ma ko game de ined by (12)
p esen s a lis o se s o possible ac ions o be pe o med by
he Nagen s in which o an agen i∈[1,N]co esponds i s
se Aiins ead o he unique se A. The ansi ion unc ion
Tope a es on all possible combina ions be ween he se o
s a es and he combined ac ion space o all agen s: T:S×
A1×A2. . .×AN7→ S. Simila ly, he unc ions Ride ine he
ewa ds ob ained by he agen s: Ri:S×Ai7→ R. In he case
o a game based on a pa ially obse ed p ocess, he de ini ion
(12) also includes he lis s o se s o obse a ions O1,...,ON
and o obse a ion unc ions Z1,...,ZN[7].
B. STATE OF THE ART
1) INTRUSION DETECTION AND PREVENTION SYSTEMS
In usion de ec ion sys ems (IDS) a emp o iden i y mali-
cious ac i i y occu ing on a ne wo k (ne wo k IDS) o on
a compu e (hos IDS) by cap u ing and analyzing di e en
sou ces o in o ma ion. A ne wo k IDS cap u es and analyses
he packe s a e sing he ne wo k (bo h p o ocol heade s
and con en ), as well as agg ega e measu emen s on a ic:
sessions, IP add esses in ol ed, a ic olumes, e c. They a e
usually guided by a se o ules agains which he cap u ed
da a is compa ed. I he e is a ma ch, an ale is issued:
he IDS has de ec ed suspicious ac i i y. In usion de ec ion
and p e en ion sys ems (IDPS) go one s ep u he : hey
a e no only capable o de ec ing an in usion a emp , bu
also o eac ing o educe o comple ely p e en he mali-
cious e ec s by execu ing some coun e measu es. To his
end, an IDPS is able o pe o m ne wo k modi ica ions, o
example by au oma ically adding a ule in he co po a e
i ewall [28].
SDNs a e pa icula ly sui able o IDPS deploymen . They
can be deployed as con ol applica ions, capable o collec ing
in o ma ion om he ne wo k wi h he help o he con olle .
The de ec ion subsys em will hen sea ch he collec ed da a
o signs o in usion. I dange ous ac i i ies a e de ec ed,
he IDPS in elligence will choose he bes cou se o ac ion,
consis ing o a se ies o coun e measu es ha a ec he low
ables o he ne wo k de ices. The necessa y changes will be
implemen ed h ough eques s o he con olle .
The wo k p esen ed he e is in ended o be only one piece
o a comple e IDPS. We assume ha he e is a wo king IDS
ha ells ou in elligen agen wha ype o a ack has been
de ec ed. Ou agen will choose he bes coun e measu es and
ask he con olle o apply hem. This is he pa we ocus
on. The design and implemen a ion o a good ne wo k IDS
goes beyond he scope o ou wo k. The in e es ed eade
can ind ele an and up- o-da e in o ma ion on ne wo k IDS
in [29]–[32].
2) AUTONOMOUS INTRUSION MITIGATION SYSTEMS IN
SDNs
As s a ed be o e, he abs ac ion, lexibili y and p og amma-
bili y o in as uc u es based on he SDN pa adigm ha e
enabled he de elopmen o a ack de ec ion and au oma ic
coun e measu e deploymen sys ems [5], [33]. Focusing on
he de ense ac i i ies, se e al in elligen sys ems capable
o deciding which coun e measu es o ake ha e been p o-
posed. One o he mos comple e sys ems is NICE (Ne -
wo k In usion de ec ion and Coun e measu e sElec ion in
i ual ne wo k sys ems) [6], which in eg a es he de ec ion o
in ec ed i ual machines in cloud compu ing en i onmen s
wi h he au oma ic deploymen o op imal coun e measu es,
which include pa ching he so wa e o an a acked machine
o qua an ining a suspicious node. The en i e in as uc u e
p oposed in NICE is based on SDN elemen s using he
OpenFlow p o ocol. Each machine in he ne wo k has a
ulne abili y egis e ed in he Common Vulne abili ies and
Exposu es (CVE) lis [34], and he a ack is ep esen ed by
an a ack g aph whe e each node ep esen s he p e ious
s a e o consequence o he exploi on one o he machines.
The selec ion o he esponse o an a ack is made aking
in o accoun he in usi eness and cos o he coun e mea-
su e, so as o minimize he impac o he esponse i sel .
The Sno Flow [17] sys em ollows he guidelines se by
NICE, bu all p oposed coun e measu es a e based solely on
ac ions aken on he ne wo k i sel (such as edi ec ing a ic
o blocking a po ) and a e implemen ed ia an OpenFlow
con olle .
Apa om p oposals ocusing on a p ac ical aspec o
coun e measu e implemen a ion, wo ks such as [7] p opose
o use mul i-agen Ma ko games o sea ch o de ense s a e-
gies agains cybe a acks. Speci ically, he game is se up
as a pa ially obse ed game and he au ho s s udy di e en
scena ios in which he abili ies o he a acke and de ende
a y. In hei game, he a acke mus mo e h ough a g aph o
each a a ge machine, simula ing a p oblem simila o hose
posed in he Cybe Ranges [35]. In ha wo k, he model- ee
algo i hms PPO [36] and REINFORCE [37] a e used o ain
he agen .
3) REINFORCEMENT LEARNING ALGORITHMS AND THE
MuZe o ALGORITHM
A classic way o e alua ing ein o cemen lea ning algo-
i hms is o pi hem agains games such as chess o Go
in an a emp o bea human sco es and ob ain supe human
esul s. One o he i s miles ones in his ield, apa om
specialized compu e s o winning a chess o shogi [38],
was he AlphaGo algo i hm de eloped by he company Deep-
Mind, which managed o bea a p o essional Go playe o
he i s ime [39]. A modi ica ion o ha algo i hm called
AlphaGo Ze o [40] achie ed supe human esul s playing
Go and led o i s successo AlphaZe o, which managed o
bea wo ld champions in chess, shogi and Go wi h only
24 hou s o aining in each case [41]. AlphaZe o was
ained using 5000 i s -gene a ion TPUs o gene a e sel -
play games and 64 second-gene a ion TPUs o ain he neu al
ne wo ks.
Model-based algo i hms ha e epea edly ou pe o med
humans in classic games such as checke s, chess, Go o
poke , as hey a e able o de elop a long- e m s a egy.
107188 VOLUME 9, 2021
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo k
FIGURE 4. Diag am o he p oposal made in his wo k, in which he
ained model, he ne wo k con olle and he simula ion en i onmen
a e comple ely independen .
Howe e , hose algo i hms ha e ended o pe o m poo ly
when con on ed wi h complex en i onmen s like he A a i
2600 compu e games. Model- ee algo i hms ob ain be e
esul s in hese en i onmen s [42]–[44], bu pe o m much
wo se han model-based algo i hms in games such as hose
men ioned abo e, which equi e p ecise and sophis ica ed
lookahead [8].
The MuZe o algo i hm in oduced by DeepMind in 2020
(a model-based algo i hm) comple ely changed he s a e o
he a . I ob ains pe o mances compa able o hose o
model- ee algo i hms in isually complex en i onmen s,
while main aining supe human esul s in p ecision planning
asks such as classic boa d games, as p e ious model-based
algo i hms [8].
III. PROPOSED ARCHITECTURE
In his wo k we p opose o use he MuZe o algo i hm o
ain a model ha is able o decide which coun e measu es
o implemen when an a acke in udes an SDN ne wo k,
wi h he in en ion o mi iga ing he a ack and minimiz-
ing he numbe o comp omised machines. The in e ac ion
be ween he in ude and he de ende has been modeled
as a s ochas ic, pa ially obse ed, ze o-sum Ma ko game,
in which bo h agen s pe o m ac ions sequen ially. Fu -
he mo e, he model has been in eg a ed in o a ealis ic
SDN ne wo k emula ion en i onmen based on Minine [45].
The ( i ual) swi ches deployed wi hin he Minine en i on-
men suppo he OpenFlow speci ica ion and accep con ol
om an ex e nal con olle —we ha e used he Ryu con-
olle [46]. Fig. 4summa izes he h ee blocks de eloped o
his wo k.
Below we p esen he p og ams and lib a ies used o he
implemen a ion o ou p oposal, he decisions ela ed o he
game design and he i ual ne wo k en i onmen imple-
men ed. The ha dwa e used o ain and es he model is
desc ibed in Sec ion IV-C.
A. DESIGN OF THE GAME
This sec ion explains he di e en elemen s designed o
ob ain a pa ially obse ed Ma ko game ep esen ing he
in e ac ion be ween he a acke and he de ende .
1) AGENTS AND THE ENVIRONMENT
The game is se up as a ma ch be ween wo playe s,
he a acke and he de ende , in which he a acke ies o
comp omise key a ge s in a ne wo k and he de ende ies
o mi iga e he in usion.
Al hough he MuZe o algo i hm was designed o deal wi h
boa d games o ideo games, in his case he en i onmen o
be simula ed is a ne wo k o in e connec ed compu e s, so we
ha e chosen o use a g aph o ep esen i . Fo p ac ical pu -
poses, MuZe o can be used o ain agen s in any en i onmen
ep esen ed by a ec o o a ma ix. The ansla ion om he
ne wo k g aph o a ma ix sui able o MuZe o is explained
in Sec ion III-A2.
In his game, he a acked ne wo k is composed o N
ulne able nodes, o which ma e pa o a honeyne . A hon-
eyne is an isola ed pa o he ne wo k whe e machines
(called honeypo s) a e used as aps o a acke s [47]. These
machines do no p o ide ac ual se ices, bu a e ulne able
o a acks, causing a acke s o was e ime and esou ces
explo ing while allowing de ende s o ob ain in ude - ela ed
in o ma ion [48]. This s uc u e is simila o he one p oposed
by he Science DMZ [49].
Inspi ed by he NICE p ojec , each node in he ne -
wo k (also known as hos ) is assigned a Base Sco e (BS),
an Exploi abili y, an Impac , and a Scope. The Na ional Vul-
ne abili y Da abase (NVD) [50] uses hose ac o s o quan-
i a i ely assess a ulne abili y, which ep esen s a weak-
ness in so wa e and ha dwa e componen s o a sys em
ha , when exploi ed, nega i ely a ec s i s con iden iali y,
in eg i y, o a ailabili y. In a eal se up whe e he IDS would
de ec he in usion, hose ac o s would be compu ed using
he equa ions de ined by he e sion 3.1 o he Common
Vulne abili y Sco ing Sys em (CVSS) [51]. Acco ding o ha
s anda d, he alue o he BS depends on he Impac sub-
sco e (ISS), Exploi abili y, and Impac . The impac sub-sco e
is de ined as
ISS =1−[(1 −C)×(1 −I)×(1 −A)](13)
whe e Cis he Con iden iali y Impac , I he In eg i y Impac
and A he A ailabili y Impac . Those me ics measu e he
impac o he con iden iali y, in eg i y and a ailabili y o
he in o ma ion esou ces caused by he exploi a ion o he
ulne abili y o a so wa e componen .
The Exploi abili y o a hos depends on he A ack Vec o
(AV ), he A ack Complexi y (AC), P i ileges Requi ed (PR)
and Use In e ac ion (UI) acco ding o (14). Those me ics
ep esen he p ope ies o he ulne abili y ha lead o a suc-
cess ul a ack. The A ack Vec o inc eases wi h he physical
and logical dis ance be ween he a acke and a success ully
comp omised componen . The A ack Complexi y ep esen s
he condi ions ha a e beyond he a acke ’s con ol and
ha a e necessa y o exploi he componen . The P i ileges
Requi ed me ic desc ibes he p i ileges ha he a acke
mus ha e be o e exploi ing he ulne able componen and
he Use In e ac ion ep esen s i he ac i i y o ano he use
o he han he a acke is equi ed o success ully comp omise
he ulne abili y.
Exploi abili y =8.22 ×AV ×AC ×PR ×UI.(14)
The Impac o he ulne abili y depends on i s Scope (S).
This ac o de e mines whe he he a ec ed hos is he
one wi h he ulne abili y (Unchanged) o a di e en one
VOLUME 9, 2021 107189
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo ks
(Changed). In he case whe e he Scope is Changed,
he Impac is compu ed as
Impac =6.42 ×ISS (15)
whe eas i i is Unchanged,
Impac =7.52×(ISS −0.029)−3.25×(ISS−0.02)15.(16)
Finally, BS is calcula ed di e en ly depending on he
Impac and Scope. I Impac =0, BS =0. I Impac >0,
BS is compu ed as
oundup min 1.08×(Impac +Exploi abili y),10 (17)
i Scope is Changed, and as
oundup min (Impac +Exploi abili y),10 (18)
i Scope is Unchanged. The oundup unc ion e u ns he
smalles numbe , speci ied o one decimal place, ha is equal
o o g ea e han i s inpu .
In e ms o he game, he Scope o a ulne abili y de e -
mines wha ac ions he a acke can ake i he succeeds
in exploi ing i : i he Scope is Unchanged, he a acke
could only ead iles om he a ge machine, whe eas
i he Scope is Changed, he a acke could scan and
exploi he machines o which he comp omised hos is con-
nec ed. The design decisions ela ed wi h he Base Sco e,
he Exploi abili y and he Impac will be discussed in
Sec ion III-A3.
The ini ial ne wo k he e o e consis s o N−mhos s
connec ed o each o he o ming a ully connec ed logi-
cal ne wo k and mhos s isola ed om he main ne wo k
bu connec ed o each o he (see Table 1and Fig. 6in
Sec ion IV-B). Each node is assigned a ulne abili y and i
can be in di e en s a es—no mal (s a e 0), scanned (s a e
1) and a acked (s a e 2)—allowing only he ansi ions 0 →
1→2. Machines canno e e o a p e ious s a e, so ha
i is gua an eed ha he e is a educed numbe o possible
ac ions. A he s a o he game one o he hos s on he
main ne wo k is se o lag (s a e -1) and he a acke ’s main
objec i e will be o each ha hos ; ano he hos wi h Scope
‘‘Changed’’ is se o s a e 2, allowing he a ack o s a om
he e.
In o de o illus a e he explana ions ha ollow, in he
es o his pape we will ep esen he s a e o he ne wo k
using g aphs, since hey a e also used o show he s a e o
he game. Fo mally, a g aph G=(V,E) is an o de ed pai
whe e Vis a se o nodes o e ices and Eis a se o edges
such ha E⊆ {{x,y} | x,y∈V∧x6= y}. In ou
ep esen a ion o he sys em, he nodes in he g aph ep esen
he hos s, and he edges show ha a ic be ween wo hos s is
no blocked (i.e. he e is connec i i y be ween hem). As hos s
can be in di e en s a es, a colo coding has been chosen
o ep esen hem in he igu es (see Fig. 5). In his coding,
hose machines in i s ini ial s a e a e ep esen ed by whi e
backg ound nodes, he explo ed ones by yellow nodes and
he a acked ones by he pu ple nodes. A node wi h a dashed-
do ed bo de ep esen s he lag. The label placed inside each
FIGURE 5. Encoding used o ep esen he s a es in which he nodes can
be.
TABLE 1. Example o a game en i onmen . The ed colo ed ow
ep esen s he a acked node and he g een colo ed one is he lag.
FIGURE 6. G aphical ep esen a ion o he en i onmen desc ibed by
Table 1.
node is jus an iden i ie , and has no ele ance om he poin
o iew o he game.
As an example, Table 1de ines a possible en i onmen
consis ing o N=12 nodes (wi h m=4 nodes o m-
ing a honeyne ), and Fig. 6shows he g aph ep esen a ion
o he ne wo k. I should be no ed ha hese a e eacha-
bili y g aphs ha do no ep esen he physical opology
o he ne wo k (consis ing o swi ches, hos s and E he ne
links), since an edge be ween wo nodes does no ep esen
a physical link, bu means ha hey can exchange pack-
e s. No e also ha swi ches a e no ep esen ed in hese
g aphs.
2) THE OBSERVATIONS
The e olu ion o he in usion and i s mi iga ion is modeled
as a pa ially obse ed decision p ocess, since nei he he
a acke no he de ende has ull in o ma ion o he s a e
o he ne wo k. To ep esen his ac we ha e used h ee
di e en g aphs: he ac ual o gene al g aph GG=(VG,EG)
wi h he comple e s a e o he ne wo k, he a acke ’s g aph
GA=(VA,EA) wi h he pa ial iew ha he a acke has,
and he de ende ’s g aph GD=(VD,ED) ha is also a pa ial
iew.
107190 VOLUME 9, 2021
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo k
FIGURE 7. Calcula ion o he ma ix M ep esen ing he obse ed s a e o
he g aph G.
Remembe ha we need a ma ix ep esen a ion o a game
boa d, so we need o ansla e he abo e g aphs in o equi a-
len ma ices. Al hough he adjacency ma ix ep esen a ion
o a g aph makes i possible o e lec he exis ing (logical)
connec ions in he ne wo k, i is no su icien o desc ibe i
in i s en i e y, since i does no include he di e en s a es
in which nodes can be. Taking ad an age o he ac ha he
main diagonal o he adjacency ma ix is null, i we ep esen
he s a e ( eal o obse ed) o all nodes acco ding o he
coding desc ibed in Fig. 5as a diagonal ma ix, a comple e
boa d can be ep esen ed by adding he wo ma ices. The e-
o e, he g aphs GG,GAand GDa e ully ep esen ed by he
ma ices MG,MAand MD, espec i ely. Fig. 7summa izes
he de i a ion o a boa d ma ix o a simple g aph wi h h ee
nodes.
The h ee ne wo k g aphs ep esen ing he s a e o he game
a e di e en om each o he ; an ac ion aken by a playe
(explained in Sec ion III-A3) only has an e ec on he gene al
g aph and on he g aph o ha playe . Consequen ly, each
playe only has a pa ial obse a ion o he game. Fig. 8shows
h ee g aphs/ma ices compu ed o he same s a e. The ac ual
s a e can be desc ibed as ollows: s a ing om he ini ial
si ua ion de e mined by Table 1, he a acke has a acked
machine 4 and has explo ed machine 2. In eali y, machine
1 is isola ed om he es (as seen in he g aphs GGand GD)
bu he a acke has no ye no iced he change in connec i i y
be ween he nodes o he ne wo k. Simila ly, he de ende
has de ec ed ha machine 1 has been a acked (and we can
assume ha i has aken some o he ac ions explained in
Sec ion III-A3) bu does no no ice ha node 2 is in s a e 1 and
node 4 is in s a e 2.
3) THE ACTIONS
The ollowing condi ions ha e been conside ed in o de o
design he ac ions ha each playe can execu e:
•Only de ensi e ac ions ha can be implemen ed using
an SDN con olle a e conside ed, lea ing ou o he
s udy op ions such as so wa e upg ades o changes o
he unde lying physical opology—such as adding o
emo ing a swi ch o changing he swi ch/po o which
a hos is connec ed.
•De ensi e ac ions can only be applied agains machines
ha ha e al eady been a acked, bu ne wo k moni o ing
is allowed.
•O ensi e ac ions can be only ca ied ou agains he
hos s: bo h swi ches and he con olle a e in ulne able.
FIGURE 8. Example o he gene al s a e o a game and he obse a ions
seen by he de ende and he a acke .
•The MuZe o algo i hm equi es he boa d size o emain
cons an h oughou he game, so he numbe o nodes
canno a y: new machines canno be deployed o
emo ed comple ely.
I should be no ed ha an agen ’s decision making is
based on i s obse a ion o he game, no on he ac ual s a e.
The e o e, legal ac ions ha an agen can ake heo e ically
may no be easible in p ac ice o may no ha e he expec ed
e ec due o di e ences be ween he agen ’s obse a ion and
he ac ual s a e. Mo eo e , as men ioned abo e, his is a
s ochas ic game, so he e a e ac ions ha ha e a p obabilis ic
ac o . Fo example, whe he o no he a acke succeeds in
exploi ing a ulne abili y in a machine depends di ec ly on
he cha ac e is ics o ha ulne abili y, so he agen will no
always ge he same esul .
Ac ions a ailable o he de ende a e:
•Check s a us
The de ende e i ies whe he he s a e o a node co -
esponds o ha o i s obse a ion. I a node has been
a acked, he p obabili y o de ec ing i is p opo ional
o he Base Sco e o he ulne abili y o he a acked
machine.
P [oD(ni)=2|s(ni)=2,a=check]=BS/(10N)
(19)
VOLUME 9, 2021 107191
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo ks
FIGURE 9. Main ne wo k and honeyne s a us be o e edi ec ing all
a ic om node1 o honeyne nodes (up) and a e (bo om).
whe e oD(ni) and s(ni) a e he de ende ’s obse a ion
and he eal s a e o he a ec ed machine niand Nis
he o al numbe o nodes in he ne wo k.
•Isola e node
De e minis ic ac ion ha comple ely isola es an
exploi ed machine.
•Send node o honeyne
De e minis ic ac ion ha blocks all a ic be ween he
a ec ed machine and he main ne wo k and allows
he node o communica e wi h he es o he nodes in
he honeyne (see Fig. 9). In ou game, he goal o his
ac ion is o make he de ende gain ime o check he
s a e o he es o he ne wo k while he a acke scans
and a acks he honeypo s.
•Mo e he lag
This ac ion simula es he mig a ion o c i ical se ices
om one machine o ano he . The de ende can y o
pe o m his ac ion i he objec i e hos is no comp o-
mised acco ding o i s obse a ion. E en hough i is a
de e minis ic ac ion, i he machine has been a acked
( he ac ual s a e o he node is 2 ins ead o he obse ed
s a e 0), he ac ion canno be pe o med, bu he de ende
can de ec ha he machine has been a acked wi h a
p obabili y equal o ha used in he ac ion Check s a us
(19). Mig a ion o se ices o mul iple machines ( lag
spli ing) is no conside ed.
Each coun e measu e has been assigned a alue ha ep-
esen s he o al cos o ca ying ou ha ac ion. The coun-
e measu es p oposed in NICE [6] and hose p oposed in his
pape a e equi alen , so each ac ion has been assigned he cos
and in usi eness o i s equi alen measu e. The ac ion Check
s a us has no equi alen in NICE, as i is no a coun e measu e
bu a moni o ing ask, so i has been assigned a low cos
and ze o in usi eness. The o al cos o each measu e, which
akes in o accoun he e ec i has on he ne wo k and he
TABLE 2. Cos s and in usi eness o p oposed coun e measu es.
esou ces needed o implemen i , is ob ained by adding i s
cos and in usi eness, see Table 2. These cos s ha e been
de ined in o de o make he de ende mi iga e a acks by
ollowing an op imal s a egy, hus minimizing he esou ces
used.
Fo he a acke , he ollowing ac ions a e a ailable:
•Explo e he opology
The a acke agen ob ains he lis o machines connec ed
o hose al eady a acked wi h Scope ‘‘Changed’’, hus
upda ing i s obse a ion.
•Scan o ulne abili ies
This ac ion scans he ulne abili y o a machine om
ano he machine wi h Scope ‘‘Changed’’. I he ac ion
is iable, he p obabili y o de ec ing a ulne abil-
i y is i s Exploi abili y/10, which is in he ange
o 0 o 1. Once scanned, node s a us changes om
0 o 1.
•A ack ulne abili y
A acke a emp s o exploi one o he ulne able
machines om ano he wi h Scope ‘‘Changed’’. The
p obabili y o comp omising an explo ed machine is,
again, p opo ional o he Exploi abili y o he ulne -
abili y o ha hos : Exploi abili y/10. Once a acked,
node s a us changes om 1 o 2.
Ha ing p esen ed all he ac ions, i is pe inen o e u n
o he ollowing idea: an a acke may decide o pe o m an
ac ion, bu ha does no mean ha he succeeds in pe o ming
i . The pseudo-code p esen ed in Algo i hm 1shows how an
a acke ac ion is execu ed. When ca ying ou an explo a ion
o an a ack, he in ude akes he se o neighbo ing nodes
o he a ge machine nO∈VA om he g aph GA. F om
ha se i andomly chooses one o he nodes nA∈VAwi h
Scope ‘‘Changed’’ and ies o pe o m he ac ion. I he edge
be ween nOand nAdoes no exis in GG, he a acke canno
use ha connec ion o a ack, so i emo es he edge om i s
g aph and selec s ano he node. I he edge exis s, a andom
numbe be ween 0 and 1 is gene a ed and he Exploi abili y o
he objec i e machine is di ided by 10 o scale i o ha ange.
Thus, he ac ion succeeds i he andom numbe is smalle
han he scaled Exploi abili y, leading o a success p obabili y
o Exploi abili y/10.
4) THE OBJECTIVES
The a acke ’s main objec i e is o a ack he lag machine
and he de ende ’s main objec i e is o p e en his om
happening. To do his, bo h playe s ake ac ions sequen ially
un il he a acke eaches he lag ( he a acke wins) o canno
ake any mo e ac ions ( he de ende wins).
107192 VOLUME 9, 2021
J. Gabi ondo-lópez e al.: Towa ds Au onomous De ense o SDN Ne wo k
[51] Common Vulne abili y Sco ing Sys em Ve sion 3.1: Speci ica ion Docu-
men , Fo um o Inciden Response and Secu i y Teams, Ca y, NC, USA,
Jun. 2019.
[52] G. Van Rossum and F. L. D ake, Py hon 3 Re e ence Manual. Sco s Valley,
CA, USA: C ea eSpace, 2009.
[53] A. H. W. Du aud. (2019). MuZe o Gene al: Open Reimplemen a ion o
MuZe o. [Online]. A ailable: h ps://gi hub.com/we ne -du aud/muze o-
gene al
[54] RYU P ojec Team. (Feb. 2014). RYU SDN F amewo k. [Online]. A ail-
able: h ps://book. yu-sdn.o g/en/Ryubook.pd
[55] S. Bha , P. K. Manadha a, and L. Zomlo , ‘‘The ope a ional ole o secu i y
in o ma ion and e en managemen sys ems,’’ IEEE Secu i y P i acy,
ol. 12, no. 5, pp. 35–41, Sep. 2014.
[56] A. B. Asi , M. Im an, N. Shah, M. A zal, and H. Khu shid, ‘‘ROCA:
Au o- esol ing o e lapping and con lic s in access con ol lis policies
o so wa e de ined ne wo king,’’ In . J. Commun. Sys ., ol. 34, no. 9,
p. e4815, Jun. 2021.
[57] D. P. Kingma and J. Ba, ‘‘Adam: A me hod o s ochas ic op imiza ion,’’
p esen ed a he 3 d In . Con . Lea n. Rep esen ., San Diego, CA, USA,
2015.
JON GABIRONDO-LÓPEZ ecei ed he deg ee
in physics, he deg ee in elec onical enginee -
ing, and he deg ee (ad anced) in compu a ional
enginee ing and in elligen sys ems om he
Uni e si y o he Basque Coun y (UPV/EHU),
in 2020 and 2021, espec i ely.
Du ing he 2018–2019 academic yea , he
ob ained an IKASIKER Fellowship o wo k a he
Physics Depa men , UPV/EHU, whe e he ca ied
ou a esea ch p ojec aimed a he compu a ional
s udy o op ical p ope ies in me als. Du ing he 2020–2021 academic yea ,
he was a Resea che on ein o cemen lea ning and cybe secu i y wi h
he Depa men o Digi al Secu i y, Vicom ech. He is cu en ly pa o
he Physics Depa men , UPV/EHU. He de elops web applica ions o he
Bilbao C ys allog aphic Se e .
JON EGAÑA ecei ed he Telecommunica ion
Enginee ing deg ee and he ad anced deg ee in
elecommunica ion enginee ing om UPV/EHU,
in 2014 and 2016, espec i ely.
He is cu en ly wi h he Depa men o
Digi al Secu i y, Vicom ech, whe e he wo ks as
a Resea che in he ield o da a analy ics o
cybe secu i y. He is an Ac i e Membe o he
5GPPP Secu i y Wo k G oup.
JOSE MIGUEL-ALONSO (Membe , IEEE) g ad-
ua ed in compu e science om he Uni e si y o
he Basque Coun y (UPV/EHU), Spain, in 1989,
and ecei ed he Ph.D. deg ee om UPV/EHU,
in 1996.
He is cu en ly a Full P o esso wi h he Depa -
men o Compu e A chi ec u e and Technology,
UPV/EHU. He is a membe o he In elligen Sys-
ems G oup, UPV/EHU. He ca ies ou esea ch
ela ed o ne wo ks and pa allel-dis ibu ed sys-
ems, in a eas such as cybe secu i y, pe o mance modeling (wi h ocus
on he in e connec ion ne wo k), esou ce managemen in supe compu e s,
cloud in as uc u es and high-pe o mance scien i ic, and echnical appli-
ca ions. He has published 2 books, 33 jou nal a icles, and 30 pape s in
in e na ional con e ences. He is a membe o he IEEE Compu e Socie y and
he HiPEAC Ne wo k o Excellence on High Pe o mance and Embedded
A chi ec u e and Compila ion.
RAUL ORDUNA URRUTIA ecei ed he deg ee
in compu e enginee ing om he Uni e si y o
he Basque Coun y (UPV/EHU), in 1999, and
he Ph.D. deg ee om he Public Uni e si y o
Na a e (UPNA), in 2010.
F om 2001 o 2018, he has wo ked in
p i a e companies as S21se, Panda Secu i y,
o T acasa in cybe secu i y and inno a ion posi-
ions. He is cu en ly he Digi al Secu i y Di ec o
a Vicom ech. He has aken pa o led p ojec s
ela ed wi h e hical hacking, o ensic analysis, malwa e analysis, access
con ol, and c yp og aphy. The cu en esea ch lines a e ocused on anomaly
de ec ion in in o ma ion sys ems and communica ion ne wo ks, au oma ic
esponses, iden i y managemen using biome ics and ede a ed sys ems, and
secu e aceabili y sys ems.
VOLUME 9, 2021 107199
