Mic o-Id-Gym: a Flexible Tool o Pen es ing
Iden i y Managemen P o ocols in he Wild and
in he Labo a o y?
And ea Bisegna1,2[0000−0002−8055−5262], Robe o Ca bone1[0000−0003−2853−4269],
Giulio Pellizza i3[0000−0001−6455−780X], and Sil io Ranise1,4[0000−0001−7269−9285]
1Secu i y & T us , Fondazione B uno Kessle , T en o (I aly)
{a.bisegna, ca bone, anise}@ bk.eu
2DIBRIS, Uni e si y o Geno a, Geno a (I aly)
3DISI, Uni e si y o T en o, T en o (I aly)
[email p o ec ed]
4Depa men o Ma hema ics, Uni e si y o T en o, T en o (I aly)
Abs ac . Iden i y Managemen (IdM) solu ions a e inc easingly im-
po an o digi al in as uc u es o bo h en e p ises and public admin-
is a ions. Thei secu i y is a manda o y p e- equisi e o building us
in cu en and u u e digi al ecosys ems. Un o una ely, no only hei
secu e deploymen bu e en hei usage a e non- i ial ac i i ies ha e-
qui e a good le el o secu i y awa eness. In o de o es whe he known
exploi s can be ep oduced in di e en en i onmen s, be e unde s and
hei e ec s and acili a e he disco e y o new ulne abili ies, we need
o ha e a eliable es bed. Fo his, we p esen Mic o-Id-Gym which ab-
s ac ly suppo s wo main ac i i ies: he c ea ion o sandboxes wi h an
IdM p o ocol deploymen and he pen es ing o IdM p o ocol deploy-
men s in he wild o in he labo a o y (on he c ea ed sandboxes).
Keywo ds: secu i y p o ocols ·pene a ion es ing ·OAu h
1 In oduc ion
Iden i y Managemen (IdM) solu ions a e inc easingly impo an o building
us in cu en and u u e digi al ecosys ems. The design and implemen a ion
o he IdM p o ocols unde lying he mos widely adop ed IdM solu ions is no-
o iously e o -p one, as wi nessed by se e al ulne abili ies epo ed in he las
ew yea s [3].
The ac i i ies o a pen es e , especially in digi al iden i y scena ios, equi e
a deep knowledge o he p o ocols and he ela ed implemen a ion aspec s. The
lack o compliance wi h he IdM p o ocols de ined in he s anda ds o a miss-
ing check o he alue o an HTTP messages can lead also o majo secu i y
?Pa ially suppo ed by he inno a ion ac i i y 19183 “Teˆıchos” o he Digi al Finance
ac ion line o he EIT Digi al, and by he join labo a o y “DigiMa Lab” be ween
FBK and he I alian Na ional Min and P in ing House (IPZS).
2 Bisegna e al.
p oblems. Then conside ing also he p oblem o compliance wi h he Paymen
Se ices Di ec i e 2 (PSD2) [1] egula ions, he ac i i ies o a pen es e inc ease
signi ican ly. Thus i is non- i ial o a pen es e o ha e he skills and echni-
cal knowledge o pe o m all he pen es ing ac i i ies equi ed o ensu e p ope
secu i y pos u e o IdM p o ocols.
Se e al ools o au oma ic pen es ing exis , bu hey usually a ge speci ic
ulne abili ies, and ew o hem a e able o spo all he ele an ulne abili ies
o IdM p o ocols. In addi ion, in case a ulne abili y has been de ec ed, he
bu den o inding adequa e mi iga ion measu es is comple ely le on he es e
who mus also collec in o ma ion abou he iden i ied p oblem and ela ed ixes.
Typically, such in o ma ion is dis ibu ed in se e al sou ces anging om he o i-
cial s anda ds and ela ed secu i y conside a ions o scien i ic pape s add essing
speci ic no el ulne abili ies.
Ano he issue wi h IdM p o ocols is he ac ha in many cases i is no
possible o pe o m a pen es ing in he wild in he p oduc ion en i onmen due
o se e al a acks wi h high impac like DoS o iden i y he wi h se ious le-
gal implica ions. The e o e i is desi able o ep oduce he p oduc ion solu ion
in a con olled en i onmen . Un o una ely, he s eps o ep oduce he p oduc-
ion en i onmen a e complica ed and i is no always possible o c ea e he
igh condi ions o be able o spo he same ulne abili ies as in he p oduc ion
en i onmen .
Fo his, we p opose Mic o-Id-Gym, o e ing on he one hand (in he labo a-
o y) an easy way o con igu e he p oduc ion en i onmen in a sandbox whe e
pen es e s can de elop hands-on expe iences on how IdM solu ions wo k, pe -
o ming a acks wi h high impac s and be e unde s and he unde lying secu i y
issues. On he o he hand (in he wild) a se o pen es ing ools o he au oma ic
secu i y analysis o IdM p o ocols a e p o ided. In [4] we ha e p o ided a high
le el o e iew o he main idea behind Mic o-Id-Gym whe e he main ocus was
on he educa ional pu poses. In his pape we a e ocusing on eal en i onmen s
desc ibing he a chi ec u e and he main echnical de ails o Mic o-Id-Gym. We
make he ollowing ou main con ibu ions:
–We ha e p o ided a lexible en i onmen o pen es ing IdM p o ocols, which
p o ides a se o deploymen s (by using con aine -based mic o-se ices) and
exploi s he possibili y o ede a e and se -up a local ne wo k among hem.
–We ha e p o ided pen es ing ools o he au oma ic secu i y analysis o IdM
p o ocols. To ease his phase, he pene a ion es ing ools communica es
wi h an applica ion (MSC D awe ) ha anima es a message sequence cha
(MSC) o he IdM p o ocols unde conside a ion.
–We ha e assessed he pene a ion es ing ools by analyzing a deployed se -
ice o PSD2 in he wild: he MSC D awe has been ex emely help ul o
quickly show he di e ences be ween he expec ed MSC and he one ob-
ained by analyzing he se ice. Then, we pe o med a ine -g ained secu i y
analysis by epo ing o he ele an issues.
–We ha e assessed he en i onmen o pen es ing and he MSC D awe in he
labo a o y expe ience: we used Mic o-Id-Gym o c ea e he p ope expe ience,
Mic o-Id-Gym 3
Fig. 1. O e iew o Mic o-Id-Gym.
by conside ing ulne able scena ios, and assessed he e ec i eness o MSC
D awe o help in inding ulne abili ies.
Plan o he pape . In Sec ion 2 we gi e an o e iew o Mic o-Id-Gym. Then,
we p o ide mo e de ails o Mic o-Id-Gym componen s in Sec ion 3. To e alua e
he e ec i eness o Mic o-Id-Gym, Sec ion 4 epo s he use o Mic o-Id-Gym in
he wild. Sec ion 5 p esen s he esul s o a use s udy in ol ing bachelo and
mas e deg ee s uden s in he labo a o y. We conclude and o e iew u u e wo k
in Sec ion 6.
2 O e iew o Mic o-Id-Gym
To assis sys em adminis a o s and es e s in he deploymen and pen es ing o
IdM p o ocol ins ances we p opose Mic o-Id-Gym. In his sec ion we p o ide an
o e iew o he ool be o e gi ing he de ails o he a ious componen s in he
es o he pape .
The IdM p o ocols a e designed speci ically o he ans e o au hen ica ion
in o ma ion and consis o a se ies o messages in a p ese sequence designed
o p o ec da a as i a els h ough ne wo ks o be ween se e s. All he IdM
p o ocols p o ide s anda ds o secu i y o simpli y access managemen , help in
compliance, and c ea e a uni o m sys em o handling in e ac ions be ween use s
and sys ems. Fo IdM p o ocols we e e o he web p o ocols whe e a Clien
(C) elies on a us ed hi d-pa y se e called Iden i y P o ide (IdP) o use
au hen ica ion. Secu i y Asse ion Ma kup Language Single Sign-On 2.0 (he e-
a e SAML) [8] and OAu h 2.0 (OAu h) [6] / OpenID Connec (OIDC) [10] a e
wo o he mos known p o ocols p o iding his au hen ica ion pa e n despi e
he ac ha di e en names may be used o e e o he a o emen ioned en i ies.
In he case o SAML, C akes he name o Se ice P o ide (SP).
Abs ac ly, Mic o-Id-Gym suppo s wo main ac i i ies: pen es ing o IdM
p o ocol deploymen s and c ea ing sandboxes wi h an IdM p o ocol deploymen .
4 Bisegna e al.
The i s ac i i y can be ca ied ou on a deploymen in he wild o one in a
sandbox—say, in he labo a o y—ob ained by he second ac i i y. We obse e
ha he capabili y o c ea ing sandboxes is use ul o pe o m a acks wi h high
impac like DoS o iden i y he , he o me dange ous o he se ice i sel while
he la e o legal and compliance issues.
Fig. 1 shows a high le el iew o he a chi ec u e o Mic o-Id-Gym composed o
wo main componen s, namely Mic o-Id-Gym F on end ( ha suppo s pen es ing)
and he Mic o-Id-Gym Backend ( ha suppo s he c ea ion o a sandbox). In
Fig. 1, i is also depic ed he IdM p o ocol deploymen ha is supposed o be
es ed o secu i y p oblems (called Sys em Unde Tes , SUT). Below, we p o ide
an o e iew o he wo main componen s.
2.1 Mic o-Id-Gym Backend
The Mic o-Id-Gym Backend is used o ec ea e locally a sandbox as an ins ance
o an IdP and a C and i can be done by uploading he own p op ie a y sandbox
o by composing a new sandbox choosing he ins ances o IdPs and Cs p o ided
by he ool as depic ed in Fig. 1 in he IdP and C eposi o ies. All he p o ided
ins ances ha e been collec ed so a du ing ou expe ience o using Mic o-Id-Gym
and hey sa is y he equi emen s o be compa ible wi h Mic o-Id-Gym namely
o ha e an ins ance o he IdP, a leas one o he C and o use SAML o
OAu h/OIDC as IdM p o ocols. The backend is also in cha ge o ins an ia e he
selec ed ins ances, o ede a e each o he , o exchange he equi ed me ada a, o
pe o m he deploymen o he sandbox and o se up he local ne wo k.
The p ocess o c ea ing a sandbox is s aigh o wa d: om he Dashboa d
he use picks Cs and an IdP om a se o a ailable IdM p o ocol ins ances (Cs
and IdPs eposi o ies in Fig. 1) and se s he URLs and po s. Once he sandbox
has been selec ed, he ool au oma ically connec s he chosen ins ances and un
hem in he SUT which includes he en i ies o IdP and Cs p ope ly ins an ia ed,
ede a ed and deployed.
The Mic o-Id-Gym Backend p o ides also Cybe Th ea In elligence (CTI)
in o ma ion use ul o assessing ulne abili ies and h ea s ela ed o he cho-
sen ins ances. These da a ollow he S uc u ed Th ea In o ma ion Exp ession
(STIX) o ma p oposed by OASIS CTI TC.5This in o ma ion is e y use ul
since i immedia ely makes he pen es e awa e o possible speci ic a acks o
ha p o ocol known in he li e a u e.
2.2 Mic o-Id-Gym F on end
The Mic o-Id-Gym F on end consis s o ools o suppo use pen es ing ac i i ies
on he SUT, namely a P oxy, a se o Pen es ing Tools, and wo ools called
MSC D awe and MSC STIX Visualize . As al eady men ioned, he SUT can be
a sandbox o any IdM p o ocol a ailable on In e ne .
5h ps://www.oasis-open.o g/commi ees/c i/
Mic o-Id-Gym 5
P oxy.I is a web p oxy ool used o in e cep he a ic be ween he use
agen (i.e. a web b owse ) and he SUT. I p o ides a se o APIs used by he
pen es ing ools o inspec , modi y, eplay and d op he in e cep ed messages.
Pen es ing Tool.I suppo s a use o pe o m pen es ing o an IdM p o o-
col deploymen , by p o iding ins umen s o au oma ically de ec secu i y is-
sues. The ools pe o m bo h passi e and ac i e es s. Passi e es s analyze he
HTTP messages exchanged be ween he b owse and he se e s, while ac i e
es s also in e cep and modi y hose messages. An example o a passi e es
can be o check whe he he o ma o he exchanged messages is complian o
wha p esc ibed by he s anda d. Ins ead, an example o an ac i e es can be
he modi ica ion a un ime o a pa ame e e e ed as equi ed by he s an-
da d. Wi h hese es s, i is possible o de ec ulne abili ies speci ically due
o an inco ec implemen a ion o he IdM p o ocols. Fo ins ance, in case o a
SAML implemen a ion, he Pen es ing Tool can iden i y a ulne abili y leading o
man-in- he-middle a ack due o an inco ec implemen a ion o he RelayS a e
pa ame e , an iden i ie o he esou ce a he SP ha he IdP will edi ec he
use o a e success ul login. All he secu i y issues iden i ied by he ools a e
epo ed in a able, including he sugges ions o mi iga e hem.
MSC D awe .The messages in e cep ed by he P oxy a e hen passed o he
MSC D awe which ep esen s hei low as a MSC. The MSC D awe p o ides
a g aphical o e iew o he au hen ica ion low and allows easie inspec ion o
he exchanged messages. Fo each HTTP message, he pen es e can di e in o
heade s, pa ame e s, and body. Usually he s anda ds o IdM p o ocols p e-
sc ibe which a e he manda o y/op ional messages and hei o ma , and he
endpoin s o in oke. S ill, hey usually do no p esc ibe any hing abou wha
happens be ween wo subsequen eques s o an endpoin . The messages o he
s anda d can be hus in e lea ed wi h o he “spu ious” messages. Fo spu ious
messages we mean any HTTP a ic be ween wo subsequen in oca ions p e-
sc ibed by he s anda d (e.g., ad e isemen s and images). Thus, being able o
ex ac in o ma ion abou he s anda d is an ex emely ime consuming ask
o a pen es e . A ailable s a e-o - he-a web p oxy ools p o ide sea ching ea-
u es, bu i is s ill di icul o g asp he main messages, by selec ing he ele an
in o ma ion e e ing o he s anda d, among he spu ious messages.
MSC STIX Visualize .I p o ides a g aph o CTI in o ma ion aken om he
STIX ulne abili y eposi o y ela ed o he in e cep ed au hen ica ion low,
cu en ly only o SAML. Using he MSC D awe UI he pen es e can choose
he g anula i y o CTI in o ma ion he wan s o look o . Fo ins ance, in case
o a SUT using he SAML p o ocol, he pen es e can look o CTI in o ma ion
ega ding he RelayS a e pa ame e o , mo e gene ally, o CTI in o ma ion
ela ed o a SAML IdP. The combina ion o hese ea u es wi h he pen es ing
ools makes he p ocess o ulne abili y iden i ica ion and cybe isk assessmen
easie .
6 Bisegna e al.
Fig. 2. Mic o-Id-Gym F on end.
Usage o he Mic o-Id-Gym F on end.The ools o he Mic o-Id-Gym F on end
a e used o he analysis o he HTTP messages gene a ed du ing he au hen i-
ca ion low. The i s s ep owa ds his p ocess is o pe o m he au hen ica ion
on he SUT. The messages exchanged du ing his p ocess a e displayed in he
MSC D awe . This is use ul because a i s glance he pen es e can ecognize
whe he he SUT ollows he expec ed low o no . The second s ep is o execu e
he au oma ed es s p o ided by he Pen es ing Tools. These au oma ed es s
e i y i he SUT su e s he ulne abili ies es ed by he ools. I a es was
success ul, he esul will epo he disco e ed ulne abili y, o he wise, no ale
will be epo ed.
Thanks o he ulne abili y esul s, he pen es e can iden i y whe e i is
exposed, hus he knows whe e i has o be pa ched. Fu he mo e, using he CTI
in o ma ion p o ided by MSC STIX Visualize , he pen es e can assess how he
ulne abili y can be exploi ed and how se e e i is.
3 The Componen s o Mic o-Id-Gym
In his sec ion we p o ide mo e de ails abou componen s o he Mic o-Id-Gym
F on end and he Mic o-Id-Gym Backend.
3.1 The Componen s o Mic o-Id-Gym F on end
As depic ed in Fig. 2, he Mic o-Id-Gym F on end is composed by Pen es ing Tool,
MSC D awe (consis ing o MSC Logge and MSC WebApp), MSC STIX Visualize
and a P oxy.
P oxy.I is a web p oxy ool ha in e cep s he HTTP a ic be ween a b owse
and he se e s o he SUT. I o e s unc ionali ies o inspec ing, collec ing, and
modi ying he HTTP messages, which a e le e aged by he Pen es ing Tool and
MSC D awe .
MSC D awe .I is a ool o d awing MSC and allows he pen es e s o quickly
selec he ele an messages, being able o spo a po en ial gap w. . . wha p e-
sc ibed by he s anda d and i is ex emely help ul o sa e ime, being mo e
e ec i e. IdM p o ocols a e o en exp essed as a MSC wi h he ad an age o im-
media ely de ec any inco ec ness in he messages o he MSC and consequen ly
Mic o-Id-Gym 7
Fig. 3. MSC D awe componen s.
iden i y any laws. We e alua ed some o hese claims by pe o ming a class oom
expe ience, desc ibed in Sec ion 5.
The Fig. 3 depic s he main componen s o he MSC D awe . The MSC Logge
is a P oxy’s plugin eponsible o cap u e a selec ion o he HTTP messages, o
il e and pa se hem acco ding o he speci ic con igu a ion, and send hem
h ough API o he MSC WebApp, a web applica ion exposing a se o Res ul
API and esponsible o d aw a MSC.
As depic ed in Fig. 3, he MSC Logge has he ollowing unc ionali ies:
–Con igu a ion: i allows o se a passwo d no allowing o o e w i e a MSC
al eady d awn and se up he URL o he MSC WebApp ( he con igu a ion
in o ma ion can be also p o ided h ough a con igu a ion ile, gene a ed
h ough he Dashboa d o he Mic o-Id-Gym Backend);
–In e cep o : i collec s all he in e cep ed HTTP messages;
–Fil e : i allows o add il e s in e ms o keywo ds o he HTTP messages
ha will be collec ed and epo ed in he MSC. The keywo ds can e e
o Hos ,Reques Heade s,Reques Pa ame e s,Response Heade s and
Response Body; and
–Pa se : i allows o add ules o mapping keywo ds o sequence o keywo ds
in new e ms in o de o u he imp o e he eadabili y o he MSC. Fo
ins ance, in a aining con ex , i is possible o p o ide a MSC close o he
abs ac iew o he p o ocol unde es (e.g., by mapping he ac ual URL
o a Clien wi h he label C).
P e-con igu ed il e ing and pa sing ules a e cu en ly a ailable o SAML
and OIDC/OAu h. In addi ion, he pens es e can add cus om ules.
To allow he MSC D awe o c ea e he MSC, he pen es e has o na iga e
wi h he b owse ollowing he s eps o he p o ocol and he co esponding MSC
is dinamically gene a ed in ano he b owse a he URL se in he Dashboa d.
Di ec ly in he MSC, he pen es e can di e in o he messages, and hanks o
he in e ac ion wi h he MSC STIX Visualize he can check he a ailable CTI
in o ma ion ela ed o he pa ame e s o he HTTP message.
8 Bisegna e al.
Pen es ing Tool.I p o ides a se o ools o pe o m au oma ic pen es ing
o IdM p o ocol. Ou idea is on he one hand o in eg a e in ou ool (mos
o ) he exis ing open sou ce ools o au oma ic pen es ing o IdM p o ocol, on
he o he hand o complemen hem wi h he pen es ing echniques p oposed
in his pape . In addi ion, o e e y de ec ed ulne abili y, ou ool e u ns he
HTTP messages ha may cause he law and sugges s mi iga ions so o allow
use s o unde s and how o (manually) ix he issue. A e an in-dep h analysis
o he s a e-o - he-a , we elici ed hose p ominen and ecen es s bo h o
OAu h/OIDC and SAML, no ye co e ed by o he plugins and added hem in
he Pen es ing Tool. We de eloped wo kinds o es s, he so-called passi e and
ac i e es s, namely:
–Passi e es s: es s done analyzing s a ically he in e cep ed HTTP mes-
sages wi hou any in e ac ion/modi ica ion o he HTTP messages du ing
execu ion o he IdM p o ocol.
–Ac i e es s: es s ha need an in e ac ion du ing he execu ion o he IdM
p o ocol. A e he ini ial execu ion, he use ac ions a e s o ed and au oma -
ically e-execu ed while in e cep ing/changing he con en o he messages
be o e sending hem o he C and IdP.
By using bo h ac i e and passi e es s, Pen es ing Tool pe o ms he ollowing
es ca ego ies: (i) Compliance wi h a gi en s anda d in e ms o o ma o
he messages, and manda o y ields, (ii) Gene al secu i y checks, pe o med on
any collec ed HTTP message, and (iii) Speci ic es s o C and IdP oles. We
decided o de ail each es abou OAu h/OIDC in Sec ion 4 and no p o ide
de ails ega ds he es o SAML because he scena io ha we analyzed in his
pape is based on OAu h/OIDC. The lis o he SAML es s is a ailable a he
complemen a y ma e ial page.6
Table 1. Collec ion o Secu i y Tes s a ge ing Any ole.
Secu i y Tes P o . P/A Desc ip ion Mi iga ion
Use o HTTPS - P Check i all HTTP messages
communica es o e a secu e
channel.
Change con . o he web se e
and o wa d unsecu ed HTTP
messages o HTTPS.
Clickjaking P e en ion - P Check i all he HTTP
messages has he heade
X-FRAME-OPTIONS se as
DENY o SAME-ORIGIN.
Se DENY o SAME-ORIGIN
he heade X-FRAME-
OPTIONS.
The se o pen es ing es s is a om being comple e. We s a ed om ha ,
bu we would like o include in ou ool (mos o ) he exis ing open sou ce ools
o pen es ing IdM p o ocol. Ou goal is o co e as many ulne abili ies/a acks
as possible.
6h ps://s bk.gi hub.io/complemen a y/ETAA2020
Mic o-Id-Gym 9
We p o ide an exce p o all he es s o OAu h/OIDC p o ocol we a e
cu en ly suppo ing, and in de ail he Table 2 epo s he es s o IdP, and
Table 1 o bo h IdP and C. Fo each es we se : (i) a name o iden i y he es
(e.g., Use o HTTPS), (ii) he IdM p o ocol implemen ed in he en i onmen
whe e he es will be execu ed (“O” s ands o OIDC), (iii) he ype o es
Passi e (P) o Ac i e (A), (i ) he desc ip ion o he secu i y es , and ( ) he
desc ip ion o he mi iga ion.
3.2 The Componen s o Mic o-Id-Gym Backend
The goal o he Mic o-Id-Gym Backend is by cons uc ion o p o ide a es en-
i onmen gene a o ailo ed o IdM p o ocols and deploy he en i onmen in
he SUT. Gi en a se o a ailable IdM p o ocol implemen a ions collec ed while
using he ool o hi d pa ies, he SUT au oma ically se s-up a wo king en i-
onmen in a local ne wo k. The main eason is o allow sys em adminis a o s
o ec ea e locally in he labo a o y hei p oduc ion en i onmen s, being able
o pen es hem in sandboxes. As depic ed in Fig. 4, he Mic o-Id-Gym Back-
end is composed by a se o IdPs and Cs ins ances bo h o OAu h/OIDC and
SAML, a STIX no es eposi o y and a Dashboa d. The se o a ailable ins ances
is indeed a wo k in p og ess, and can be easily ex ended/upda ed o e he ime.
By design, he a chi ec u e allows con inuous in eg a ion o newe and di e en
implemen a ions.
The componen edi o (i.e. he pe son in cha ge o con igu e he SUT) h ough
aDashboa d can selec he IdM p o ocols (cu en ly ei he SAML o OAu-
Table 2. Collec ion o Secu i y Tes s a ge ing IdP ole.
Secu i y Tes P o . P/A Desc ip ion Mi iga ion
CSRF P e en ion O P Checks whe he s a e pa ame e is used. In oduce he s a e pa ame e in he
low.
Check Compliance wi h
S anda d
O P Checks whe he all he pa ame e s epo ed as RE-
QUIRED in he S anda d a e in he HTTP messages
o he conside ed low.
In oduce he missing pa ame e s in he
low.
Adop ed PKCE O P Checks whe he he implemen a ion used he pa am-
e e P oo Key o Code Exchange (PKCE). In oduce he PKCE pa ame e in he
low.
Al e a ion s a e pa ame-
e
O A Changes in he Au ho iza ion Reques he alue o
he s a e pa ame e .
Sani ize he alue o s a e pa ame e .
Dele ion s a e pa ame e O A Dele es he alue o he s a e pa ame e when sen
o he AS wi h he Au ho iza ion Reques .
Sani ize he alue o s a e pa ame e .
Al e a ion
code challenge pa-
ame e
O A Changes he alue o he code challenge pa ame e
when sen o he AS wi h he Au ho iza ion Reques .
Sani ize he alue o code challenge
pa ame e .
Dele ion code challenge
pa ame e O A Dele es he alue o he code challenge pa ame e
when sen o he AS wi h he Au ho iza ion Reques . Sani ize he alue o code challenge
pa ame e .
Al e a ion
code challenge me hod
pa ame e
O A Changes he alue o he code challenge me hod
pa ame e when sen o he AS wi h he Au ho iza-
ion Reques .
Sani ize he alue o
code challenge me hod pa ame e .
Dele ion
code challenge me hod
pa ame e
O A Dele es he alue o he code challenge me hod pa-
ame e when sen o he AS wi h he Au ho iza ion
Reques .
Sani ize he alue o
code challenge me hod pa ame e .
Legenda: “P”: Passi e Tes , “A”: Ac i e Tes . “-”: any p o ocol
16 Bisegna e al.
Sys ems. The sys ems used o conduc he expe imen a e wo di e en
deploymen s o he OIDC p o ocol:
– S1a de ec i e OIDC implemen a ion ulne able o a missing sani iza ion o
he edi ec u i pa ame e ; and
– S2a de ec i e OIDC implemen a ion ulne able o a no adequa e p o ec ion
o he s a e pa ame e .
The selec ed sys ems a e compa able in e ms o complexi y o he ope a ions
equi ed o de ec he p oblem. I is impo an o no e ha hese sys ems a e
ep esen a i e o ealis ic OIDC implemen a ions and bo h ulne abili ies and
hei ela ed a acks a e desc ibed in Sec ion 4. To i he ime cons ain o ou
expe imen , only one ulne abili y is p esen in each sys em.
5.2 Va iables Selec ion
To measu e he suppo o ulne abili y de ec ion and o conduc a co ec i e
main enance on OIDC implemen a ions, we iden i ied as he main ac o o he
expe imen — ha ac s as an independen a iable— he p esence o he MSC du -
ing he execu ion o he ask. In ou expe imen , he base ea men case TRzap
consis s o using OWASP ZAP; and TRmig consis s o using MSC D awe , ha
includes no only he lis o he HTTP in e cep ed messages, bu also gene a es
a MSC o he in e cep ed a ic. Mo eo e , we ins umen ed he expe imen al
se ings o measu e Co ec ness o each co ec i e asks pe o med by pa ici-
pan s.
5.3 Expe imen Design and P ocedu e
We adop a coun e -balanced expe imen al design in ended o i wo lab ses-
sions. Pa icipan s a e andomly assigned o ou g oups (despi e hey wo k
alone), each one wo king in wo labs on di e en sys ems wi h di e en ea -
men s. The design allows o conside ing di e en combina ions o Sys ems and
T ea men s in di e en o de ac oss Labs (see Table 4).
Be o e ou expe imen , pa icipan s we e p ope ly ained wi h lec u es and
exe cises on OIDC p o ocol, on MSC D awe and on OWASP ZAP, o p o ide/ e-
call he equi ed backg ound. The pu pose o aining is o make pa icipan s
con iden abou he kind o asks hey a e going o pe o m and he en i on-
men hey will ha e a ailable. The expe imen was ca ied ou acco ding o he
ollowing p ocedu e. Pa icipan s had o (i) comple e a p e-expe imen p o iling
Table 4. Expe imen al design.
G oup A G oup B G oup C G oup D
Lab 1 S1wi h TRmig S2wi h TRzap S2wi h TRmig S1wi h TRzap
Lab 2 S2wi h TRzap S1wi h TRmig S1wi h TRzap S2wi h TRmig
Mic o-Id-Gym 17
su ey ques ionnai e, (ii) pe o m he de ec ion ask o he i s lab, (iii) pe -
o m he de ec ion ask o he second lab, and (i ) comple e a pos -expe imen
su ey ques ionnai e.
The p e-expe imen p o iling su ey collec s backg ound knowledge abou he
pa icipan s, such as hei p e ious expe ience wi h P oxy ool and hei knowl-
edge o OIDC p o ocol. Pos -expe imen su ey ques ionnai e ( epo ed in Ap-
pendix A) deals wi h he cla i y o he asks, cogni i e e ec s o he ea men s
on he beha io o he pa icipan s and pe cei ed use ulness o MSC D awe .
5.4 Summa y o Findings
We collec ed all he esul s and no iced ha : (i) he dis ibu ions o co ec-
/w ong answe s when using MSC D awe is espec i ely 262 and 80, (ii) he
dis ibu ions o co ec /w ong answe s when using OWASP ZAP is espec i ely
222 and 120, (iii) 8 pa icipan s o e 38 we e able o de ec co ec ly and com-
ple ely he ulne abili ies when hey we e using MSC D awe , and (i ) only
3 pa icipan s we e able o de ec co ec ly and comple ely he ulne abili ies
when using OWASP ZAP.
The eedback ques ionnai e was posi i e and om he pos -expe imen su ey
we can lea n ha 87.5% o he s uden s conside s TRmig mo e use ul and 84.4%
assessed ha TRzap is mo e complex o unde s and. In addi ion, all he s uden s
posi i ely ecommend ou ool. He e we epo some commen s:“Ve y clea in
he p esen a ion o in o ma ion.”, “Simple o isualize he HTTP messages wi h
di ec ions.”, “I is easie and as e o ead all he in o ma ion.”.
Finally, looking a hei subjec i e eedback, i seems ha pa icipan s ag ee
wi h ou claim ha MSC D awe is highly bene icial in de ec ing ulne abili ies
in OIDC implemen a ions and i will help o inc ease he secu i y awa eness.
6 Conclusions and Fu u e Wo k
We ha e desc ibed Mic o-Id-Gym a lexible ool o pen es ing IdM p o ocols easy
o con igu e and in which use s can de elop hands-on expe iences on how IdM
p o ocols wo k, pe o ming a acks wi h high impac s and be e unde s and he
unde lying secu i y issues. Fo ease o con igu a ion and deploymen , Mic o-Id-
Gym uses con aine -based mic o-se ices and s a e-o - he-a pene a ion es ing
ools. We ha e imp o ed Mic o-Id-Gym by suppo ing new IdM p o ocols and a
ca alog o ealis ic scena ios in which di e en ulne abili ies and a acks can
be e-c ea ed, analyzed, and mi iga ed. Seconda y, we ha e analyzed a eal use-
case scena io in ol ing a PSD2 se ice p o ided by an impo an I alian iden i y
p o ide . Finally, we ha e alida ed he use expe ience and secu i y awa eness
p o ided by ou amewo k by using he esul s o a use -s udy expe imen a ion
in ol ing s uden s om uni e si y.
As u u e wo k, we plan o ex end Mic o-Id-Gym by (i) in eg a ing new pen-
es ing ools, (ii) suppo ing o he mul i-pa y web applica ions, and (iii) sup-
po ing STIX also o OAu h/OIDC.
18 Bisegna e al.
Re e ences
1. PSD2 (accessed june 23, 2020). h ps://eu -lex.eu opa.eu/legal-con en /EN/
TXT/?u i=CELEX%3A32015L2366
2. Secu i y Conside a ions OAu h (accessed june 23, 2020). h ps:// ools.ie .
o g/id/d a -b adley-oau h-jw -encoded-s a e-08.h ml# c.sec ion.6
3. A mando, A., Ca bone, R., Compagna, L., Cuella , J., Toba a, L.: Fo mal analysis
o SAML 2.0 web b owse single sign-on: b eaking he SAML-based single sign-on
o google apps. In: P oceedings o he 6 h ACM wo kshop on Fo mal me hods in
secu i y enginee ing. pp. 1–10 (2008)
4. Bisegna, A., Ca bone, R., Ma ini, I., Odo izzi, V., Pellizza i, G., Ranise, S.: Mic o-
Id-Gym: Iden i y Managemen Wo kou s wi h Con aine -Based Mic ose ices. In:
In e na ional Jou nal o In o ma ion Secu i y and Cybe c ime 8(1). pp. 45–50
(2019)
5. C. Wohlin, P. Runeson, M.H.M.O.B.R.A.W.: Expe imen a ion in so wa e engi-
nee ing. So w. Tes ., Ve i . Reliab. (2001). h ps://doi.o g/10.1002/s .230
6. Ha d , D.: The OAu h 2.0 Au ho iza ion F amewo k (RFC6749). In e ne Engi-
nee ing Task Fo ce (IETF) (2012)
7. H¨os , M., Regnell, B., Wohlin, C.: Using s uden s as subjec s—a compa a i e s udy
o s uden s and p o essionals in lead- ime impac assessmen . Empi ical So wa e
Enginee ing 5(3), 201–214 (2000)
8. Hughes, J., Male , E.: Secu i y asse ion ma kup language (saml) 2.0 echni-
cal o e iew. OASIS SSTC Wo king D a ss c-saml- ech-o e iew-2.0-d a -08 pp.
29–38 (2005)
9. Li, W., Mi chell, C.J.: Secu i y issues in oau h 2.0 sso implemen a ions. In: In e -
na ional Con e ence on In o ma ion Secu i y. pp. 529–541. Sp inge (2014)
10. Sakimu a, N., B adley, J., Jones, M., De Medei os, B., Mo imo e, C.: OpenID
Connec Co e 1.0 inco po a ing e a a se 1. The OpenID Founda ion, speci ica ion
335 (2014), h ps://openid.ne /specs/openid-connec -co e-1_0.h ml
11. Salman, I., Misi li, A.T., Ju is o, N.: A e s uden s ep esen a i es o p o essionals
in so wa e enginee ing expe imen s? In: 2015 IEEE/ACM 37 h IEEE In e na ional
Con e ence on So wa e Enginee ing. ol. 1, pp. 666–676. IEEE (2015)
12. S ahnbe g, M., Au um, A., Wohlin, C.: Redi ec u i a ack. In: P oceedings o he
Second ACM-IEEE in e na ional symposium on Empi ical so wa e enginee ing
and measu emen . pp. 288–290 (2008)
13. S ahnbe g, M., Au um, A., Wohlin, C.: Using s uden s as subjec s-an empi ical
e alua ion. In: P oceedings o he Second ACM-IEEE in e na ional symposium on
Empi ical so wa e enginee ing and measu emen . pp. 288–290 (2008)
Mic o-Id-Gym 19
A Pos -ques ionnai e
The ollowing able shows he con en o he pos -expe imen su ey ques ion-
nai e men ioned in Sec ion 5. I deals wi h objec cla i y o he asks, cogni i e
e ec s o he ea men s on he beha io o he subjec s and pe cei ed use ulness
o MSC D awe . The i s se o ques ions (Q1-Q6) needs o be answe ed wice
(one answe o each pe o med lab) while he emaining se only needs o be
answe ed once as i e e s o he o e all session.
Table 5. Pos -expe imen su ey ques ionnai e.
ID Applies o Ques ion
Q1 Each lab I had enough ime o pe o m he asks. (1-5).
Q2 Each lab I expe ienced no di icul y in de ec ing he ulne abili y. (1-5).
Q3 Each lab Which ope a ions (e.g., mouse o e s eps, open ab, sea ch, . . .) did
you pe o m o unde s and whe he he p o ocol was ulne able o
he men ioned ulne abili y?
Q4 Each lab Did you consul in e ne o ind help o answe he ques ionnai e? I
yes, which online que ies did you sea ch(e.g., keywo ds used)? Which
con en was help ul?
Q5 O e all Which ool did you ind mo e use ul o answe he ques ions? (Repo
o Lab 1-2).
Q6 O e all Which ool did you ind mo e in ui i e? Fo which ool was mo e
di icul o ind he p ope in o ma ion abou he p o ocol in o de o
answe he ques ions? (Repo o Lab 1-2).
Q7 O e all Which ool would you use o you wo k? Mo i a e you answe ( o
he p e ious ques ion). (open ques ion).
Q8 O e all Do you know any ool ha pe o ms simila asks? (open ques ion).
Q9 O e all Do you ha e any sugges ion ela ed o he ool usage? (open ques ion).
Q10 O e all Wha do you hink is he main ad an age using MSCD awe ? Would
you add mo e in o ma ion o he MSCD awe ? (open ques ion).
