A Me hodology o Rep oducible and Po able Expe imen Wo k lows
Henning S ubbe1, Sebas ian Gallenmülle 1,∗
, Geo g Ca le1
TUM School o Compu a ion, In o ma ion and Technology,
Technical Uni e si y o Munich, Bol zmanns . 3, 85748, Ga ching nea Munich, Ge many
Abs ac
Tes beds allow he c ea ion o esea ch p o o ypes o es new ideas h ough p ac ical expe imen s. This cen al ole
in alida ing ideas makes hem i eplaceable ools o da a-d i en esea ch in compu e science. Va ious es beds we e
c ea ed o p o ide es beds o he scien i ic communi y. To simpli y es bed usage, amewo ks help o au hen ica e
use s, alloca e esou ces, and un expe imen s. Each es bed ypically implemen s i s own amewo k using a speci ic API
o ealize expe imen s. Such an expe imen design impedes he po abili y o expe imen s be ween di e en es beds. In
his pape , we p esen a solu ion whe e we po he pos expe imen con olle o he Chameleon and CloudLab es bed.
The well-s uc u ed pos expe imen wo k low allows he c ea ion o inhe en ly ep oducible expe imen s. P e iously, he
expe imen s using he pos wo k low we e only possible in dedica ed es beds. By in oducing he po abili y ea u e,
hese expe imen s can un on Chameleon and CloudLab. We demons a e ha expe imen s can be execu ed on any o he
men ioned pla o ms wi hou changing he expe imen de ini ion. Based on hese esul s, we discuss how he po abili y
ea u e will be used in he upcoming SLICES-RI es beds o c ea e ep oducible and easily-sha eable expe imen s.
Keywo ds: Repea abili y, Rep oducibili y, Ne wo k Expe imen , Po abili y, plain o ches a ing se ice (pos)
This publica ion is an ex ension o he pape „The
pos Expe imen Con olle : Rep oducible & Po able Ne -
wo k Expe imen s“ o iginally published a he 19 h Wi e-
less On-demand Ne wo k sys ems and Se ices Con e ence
(WONS’24) [1].
1. In oduc ion
Compu e ne wo ks and connec i i y o he In e ne
ha e ound hei way in o ou e e yday li es, whe e hey
ha e become a commodi y o many p oduc s and se ices.
Indus ial manu ac u ing, he heal h sec o , and anspo
sys ems ha e de eloped in o in e connec ed sys ems e-
lying on he implici a ailabili y o ne wo k connec i i y.
Such se ices expec eal- ime o nea eal- ime communi-
ca ion o a smoo h and eliable ope a ion. A he same
ime, ne wo ked sys ems ha e eached a le el o complexi y
ha equi es a ho ough unde s anding o ensu e ha ne -
wo ks p o ide connec i i y wi h he equi ed quali y. The
complexi y o ne wo k sys ems a ises om a combina ion
o mul iple laye s o ( i ualized) so wa e in end hos s
and p og ammable ne wo k componen s such as swi ches
o sma ne wo k in e ace ca ds (NICs) o complex con-
nec ed ha dwa e componen s.
∗Co esponding au ho
Email add esses: [email p o ec ed] (Henning S ubbe),
[email p o ec ed] (Sebas ian Gallenmülle ),
[email p o ec ed] (Geo g Ca le)
Vi ualiza ion, simula ion, and emula ion a e ools o
ec ea e eal, complex sys ems as a a ge pla o m o e-
sea ch in a cos -e icien way. Howe e , ec ea ing such
complex en i onmen s is a highly challenging ask, espe-
cially he in e play o all he di e en ha dwa e and so -
wa e ne wo k componen s. Mino di e ences be ween he
o iginal and he ec ea ed sys em may impac he mea-
su emen esul s, especially in he eal- ime domain whe e
ac ions o a second ma e . Ha dwa e es beds help sol e
his p oblem, allowing he ec ea ion o sys ems based on
ba e-me al ha dwa e. Resea ch based on such a es bed
can use ba e-me al ha dwa e and he same so wa e s ack
as he o iginal sys ems, ensu ing a ealis ic delay beha -
io . Emula ed, simula ed, o i ualized sys ems can be-
ha e di e en ly compa ed o he o iginal sys ems [2]. The
eason o he di e en beha io a e e ec s ha may be
hidden, in en ionally o unin en ionally, om he expe i-
men e s, such as he delay o ne wo k connec ions o un-
documen ed ea u es / bugs o ha dwa e componen s. A
ha dwa e-based es bed will be subjec ed o he same e -
ec s as he o iginal sys ems, inc easing he con idence in
he esul s gained.
I e ec s a e well unde s ood, simula ion and emula ion-
based esea ch pla o ms can be powe ul ools o c ea -
ing expe imen s wi h a high numbe o pa icipa ing en-
i ies cos -e icien ly. To gain his unde s anding, mul i-
ple es beds we e c ea ed ha o e ba e-me al ha dwa e
access [3, 4, 5, 6]. Despi e o e ing simila unc ional-
i y, all es beds ypically equi e expe imen e s o use a
Managemen Node
Expe imen
Con olle
Tes bed
O ches a o
Expe imen
Node
Expe imen
Node
Expe imen
Node
(a) S ep 1: Tes bed o ches a o
p epa ing expe imen nodes
Managemen Node
Expe imen
Con olle
Tes bed
O ches a o
Expe imen
Node
Expe imen
Node
Expe imen
Node
(b) S ep 2: Expe imen con olle
pe o ming expe imen
Figu e 1: Two componen s o he managemen node
es bed-speci ic API o hei expe imen s. The di e en
APIs limi he execu ion o expe imen s o a speci ic es -
bed, p e en ing simple po abili y o expe imen s be ween
es beds. Consequen ly, c ea ing po able expe imen s e-
qui es addi ional e o om he expe imen e s o make
hei expe imen s compa ible wi h he API o o he es -
beds. This lack o na i e po abili y se e ely limi s he
ep oduc ion o expe imen s ac oss es beds.
In his wo k, we wan o demons a e a no el app oach
o make expe imen s po able. Ou app oach uses pos [3],
a amewo k wi h a ocus on he c ea ion o ep oducible
expe imen s. The pos amewo k implemen s he API ha
is used by expe imen s. To make he expe imen s po able,
his API mus be suppo ed by o he es beds. We achie e
his by po ing he pos amewo k o o he es beds. To
es he easibili y o ou app oach, we demons a e ha
an unmodi ied expe imen wo k low can be execu ed on
o he es beds such as CloudLab [4] and Chameleon [5].
The pape is s uc u ed as ollows. In Sec ion 2, di -
e en es beds a e p esen ed wi h a ocus on hei design
and usabili y o expe imen e s. Sec ion 3 p esen s he
high-le el design o ou app oach. In Sec ion 6, we discuss
how he p esen ed app oach can be used in u u e es -
beds. The a i ac s published in addi ion o he pape a e
explained in Sec ion 7. Sec ion 8 concludes he pape .
2. Backg ound and Rela ed Wo k
In his sec ion, we in es iga e di e en aspec s o cu -
en es beds, ocusing on po abili y and ep oducibili y.
Tes bed o ches a ion s. expe imen con ol. We iden i-
ied wo componen s necessa y o success ully ope a e a
es bed: The i s componen , he es bed o ches a o ,
handles low-le el ope a ions o manage o o ches a e he
esou ces o a es bed. This es bed o ches a o p epa es
and con igu es he en i onmen o un he expe imen s.
The o ches a o ypically o e s an in en o y o a ailable
esou ces ha helps esea che s iden i y ele an esou ces.
A common ea u e in mul i-use en i onmen s is he ese -
a ion o es bed esou ces, e.g., ia a calenda . Based
on he a ailable esou ces and hei ese a ion, he o -
ches a o ini ializes he speci ied esou ces and p epa es
hem o become eady o expe imen a ion. The second
componen , he expe imen con olle , is esponsible o
high-le el asks o con ol expe imen s. The main ask o
his componen is he execu ion o he expe imen wo k-
low in a p epa ed en i onmen . Typically, he expe imen
con olle expec s a pa icula s uc u e, o e en a domain-
speci ic language (DSL), o de ine expe imen s. The expe -
imen con olle can u ilize esou ces p epa ed by he p e-
ious o ches a ion s eps o pe o m expe imen s. Based
on he de ini ion o he expe imen wo k low, expe imen s
can in ol e he c ea ion o measu emen esul s, e alua-
ions, o plo s. The sepa a ion be ween es bed o ches a-
o and expe imen con olle is shown in Figu e 1.
To make expe imen s ep oducible, he o ches a o
needs o c ea e he s a e o he expe imen en i onmen
ep oducibly. Po abili y is ensu ed i an o ches a o c e-
a es he en i onmen o he expe imen con olle ac oss
di e en pla o ms and he expe imen con olle pe o ms
he expe imen acco ding o a de ined wo k low. The ol-
lowing o e iew uses he p e iously in oduced concep s
o ca ego ize exis ing app oaches o es bed o ches a ion
and expe imen con ol.
Rep oducibili y. The Associa ion o Compu ing Machin-
e y (ACM) de ines ep oducibili y as a h ee-s age p o-
cess [7]: The i s s age, epea abili y, is eached i he
same eam ob ains esul s on he same expe imen al se up;
he second s age, ep oducibili y, is achie ed i a di e en
eam ec ea es an expe imen using he same expe imen-
al se up; in he hi d s age, eplicabili y, a di e en eam
uses a di e en expe imen al se up o ec ea e an expe -
imen . Tes beds mainly a ge he expe imen al se up.
The e o e, s ages one and wo can be achie ed h ough
echnical measu es [3]. Expe imen s become epea able
o esea che s i he es bed ensu es ha expe imen s a e
execu ed ully au oma ed and all expe imen s s a om
a well-de ined s a e. By sha ing access o a common ex-
pe imen se up, e.g., a es bed, o he esea che s can e-
p oduce expe imen s. Technical measu es canno ensu e
he c ea ion o an independen expe imen se up h ough
a di e en esea ch g oup. The e o e, a es bed canno
ensu e eaching hi d-s age eplicabili y. Howe e , by c e-
a ing and publishing well-documen ed expe imen al a i-
ac s, expe imen e s can suppo po en ial eplica ions.
2.1. Tes bed O e iew
Tes beds come in a wide a ie y, e lec ing he di e en
equi emen s o esea ch domains. A he same ime, es -
bed ope a o s a e conscious o he exis ence o o he es -
beds and, hence, s i e o join o ces o a common goal.
These join wo ks esul ed in success ul coope a ion, such
as GENI [8], whose successo FABRIC [9] ecen ly ook
o e and has enabled access o expe imen esou ces ac oss
a numbe o es beds, including Chameleon and CloudLab.
Simila ly, on he Eu opean side, he e is ongoing wo k
owa ds p o iding no only indi idual es beds bu also
opening access o di e en es beds o esea che s. One
2
mani es a ion o his wo k was Fed4FIRE [10]. A ede a-
ion o mul iple Eu opean es beds. A mo e ecen p ojec
ha plans o es ablish a mo e igh ly in eg a ed Eu opean
es bed is SLICES-RI [11].
Connec ing es beds. A common heme be ween Cloud-
Lab, Chameleon, and Fed4FIRE [10] es beds is hei spa-
ial dis ibu ion. The CloudLab and Chameleon es beds
a e dis ibu ed ac oss di e en si es. Typically, o ganiza-
ions ha con ibu e o he ope a ion o a es bed ope a e
hei own on-si e es beds, leading o he dis ibu ed s uc-
u e. F om he expe imen e ’s poin o iew, all si es can
be managed ia he same API. The common API equi es
li le o no wo k om he expe imen e when mo ing he
expe imen be ween di e en si es. The Fed4FIRE es -
beds ollow a mo e open app oach o collabo a ion be-
ween di e en es beds. Fed4FIRE o e ed h ee le els:
(1) associa ion, (2) ligh ede a ion, and (3) ad anced ed-
e a ion. Associa ed es beds a e no equi ed o p o ide
any sha ed API o ooling. The wo mo e ad anced le els
use a sha ed solu ion o au hen ica e use s. The highes
le el o ede a ion p o ides a common se o basic se ices
o expe imen e s ac oss es beds. Depending on he asso-
cia ion/ ede a ion le el, expe imen e s ha e o use di e -
en APIs. This di e si y o APIs inc eases he complexi y
o po ing expe imen s be ween di e en es beds.
Exis ing app oaches o es bed o ches a ion. The geni-
lib [12] is a lib a y de eloped by he GENI ini ia i e ha
was c ea ed o p o ide an in e ace o o ches a e di e en
es beds, such as CloudLab [4]. Tes bed use s use he API
o his lib a y o o ches a e he es bed. The Chameleon
es bed [5] uses OpenS ack [13] o manage he es bed e-
sou ces. Use s can use he OpenS ack APIs o o ches a e
his es bed. pos [3] is a es bed amewo k designed o
c ea e ep oducible expe imen s. Using li e Linux images
ensu es ha any machine s a e is e ased on eboo . In ad-
di ion, pos use s need o ully au oma e he o ches a ion
p ocess o a ep oducible se up. O ches a ion is handled
ia a command line in e ace (CLI), i.e., any applica ion
ha can u ilize he CLI will be able o use he pos o ches-
a o .
Exis ing app oaches o expe imen con ol. Along wi h
he con inuous need o expe imen s comes he commu-
ni y’s s i ing o ools o simpli y hei handling. One
ep esen a i e o his is he Common Wo k low Language
(CWL) [14]. This p og amming language p o ides scaleable
ooling o au oma ed execu ion o expe imen s, imple-
men ing wo s anda ds: 1. de ails on a ailable ools wi h
hei in- and ou pu s and 2. desc ip ion o composi ion
ules o known ools. CWL has se e al implemen a ions
using a ious pla o ms such as SSH-accessible sys ems,
Kube ne es, AWS, o Azu e o un expe imen s. I is
a popula choice o p ocessing da a-d i en expe imen s
in li e sciences [15]. CWL is well-sui ed o un pu ely
so wa e-based expe imen s ha do no equi e special-
ized ha dwa e o ne wo k opologies. While also ocusing
on he execu ion o expe imen s, he cOn ol and Manage-
men F amewo k (OMF) [16] app oached his opic di e -
en ly. Rako oa i elo e al. de eloped a cen al con olle
accep ing jobs om expe imen s o execu e on he es -
bed’s ha dwa e. Using a Ruby-based DSL o desc ibe ex-
pe imen s, OMF p oduces expe imen ou pu s a ailable
ia SQL o HTTP in e aces. NEPI [17] is a pla o m
ha p o ides di e en backends o execu e expe imen s:
a physical es bed, a ne wo k emula o , and a ne wo k
simula o . The NEPI amewo k p o ides an abs ac ion
laye on op o i s suppo ed backends u ilized by he ex-
pe imen s. Expe imen s use a Py hon API o con ol he
expe imen al wo k low. In ecen yea s, Chameleon in e-
g a ed se ices o expe imen con ol [5]. The p e e ed
in e ace o expe imen speci ica ion a e Jupy e no e-
books and Chameleon’s Jupy e hub; o s o e hese expe -
imen a i ac s, esea che s a e encou aged o use T o i.
Jupy e no ebooks agg ega e all s eps o he expe imen
wo k low. Resea che s a e ee o s uc u e he expe imen
wo k low acco ding o hei pe sonal p e e ences. The pos
amewo k [3] also in eg a es a componen o expe imen
con ol. I elies no on a speci ic API o lib a y o de ine
expe imen s. The use w i es sc ip s execu ed on he al-
loca ed expe imen nodes, and he sc ip s can be s a ed
ia he pos CLI om he managemen node.
Bene i s o he pos app oach. In con as o he p e iously
men ioned amewo ks, pos does no equi e use s o lea n
and apply APIs o DSLs o o ches a e es beds o desc ibe
expe imen s. Use s can exp ess he s eps o o ches a ion
and expe imen al con ol using any p og amming language
o so wa e. In e ac ion wi h pos is kep minimal and han-
dled ia a CLI ha can be u ilized by expe imen e -de ined
sc ip s. This ensu es a high deg ee o eedom o i s use s
while pos’ p ope ies ensu e a ep oducible execu ion o
expe imen s.
2.2. In es iga ed Tes beds
Fo his pape , we wan o ocus on a speci ic ype o
es bed, a ge ing a simila domain and o e ing o - he-
shel se e ha dwa e: Chameleon, CloudLab, and pos.
All h ee o e low-le el access o esou ces, he e o e, a
high deg ee o eedom o he expe imen e .
2.2.1. Chameleon
Ope a ing since 2015, he Chameleon es bed [5] p o-
ides an expe imen pla o m o esea che s. Designed o
a b oad se o expe imen domains, his es bed p o ides
ne wo ked compu e on a he e ogeneous de ice se . The
a ailable ha dwa e is lis ed on he es bed’s webpage. The
desi ed esou ces, e.g., ne wo ks and machines, can be e-
se ed o a gi en in e al, assuming su icien a ailabili y.
Ano he limi ing ac o when ese ing is ha esou ce al-
loca ion equi es a i ual cu ency; each p ojec accep ed
on he es bed has a limi ed amoun o said cu ency o
spend. Du ing he ese ed in e al, esea che s can con-
igu e hei esou ces, e.g., he disk image o be used o
3
how nodes a e connec ed, and conduc hei expe imen s.
A ecen addi ion o Chameleon is T o i, a se ice hos ing
Jupy e -no ebook-based expe imen s and a i ac s. The
mo i a ion o T o i and Jupy e no ebooks is he a emp
o p o ide a eposi o y o collec ing, publishing, and sha -
ing di e en expe imen s. O he esea che s can easily ac-
cess T o i o ep oduce exis ing expe imen s o o de i e
hei own expe imen s.
2.2.2. CloudLab
CloudLab [4] is buil on Emulab [18], a pu pose-buil
es bed managemen so wa e. Designed as a ede a ed
es bed, CloudLab spans ac oss mul iple si es in he USA.
While si es a y in he kind o o e ed ha dwa e, he e is
a endency owa ds comme cial o - he-shel se e s. Each
ype o ha dwa e is gene ally a ailable se e al imes, hus
enabling expe imen s among he same kind o ha dwa e.
Simila o Chameleon, connec ions be ween machines a e
implemen ed ia es bed-con igu able da a cen e swi ches.
As wi h se e s, he kind o swi ch di e s be ween si es.
Expe imen se ups in CloudLab can be con igu ed ei-
he ia an XML desc ip ion o ia a Py hon sc ip , which,
based on p o ided lib a ies, allows he gene a ion o his
XML desc ip ion. This se up desc ip ion includes, e.g., he
numbe o machines equi ed, hei ype and disk image,
o he in a-expe imen links. Addi ionally, op ional in o -
ma ion, such as expe imen documen a ion o commands
o execu e on boo , can be con igu ed his way. The e-
sul ing expe imen desc ip ion can be sha ed wi h o he
esea che s and simpli ies ec ea ing he same expe imen
se up as needed.
To conduc an expe imen , esea che s can ei he sea ch
o a sui able ime slo by en e ing hei equi emen s o
y o immedia ely begin expe imen ing. In he la e case,
hough, mo e e ol ed ese a ions a e mo e likely o be in-
easible, as many esou ce ypes a e well-u ilized. Pa o
he expe imen con igu a ion associa ed wi h a ese a ion
is, e.g., he disk image o use on he ese ed nodes. Once
alloca ed o he esea che , machines will be con igu ed
as ins uc ed; his con igu a ion commonly includes asks
such as deploying SSH keys o moun ing a pe sis en NFS
sha e. Be o e he alloca ion e mina es, he esea che
may, p o ided su icien a ailabili y, eques an ex ension
o hei expe imen . A e he expe imen , nodes a e eim-
aged wi h a de aul con igu a ion and made inaccessible.
2.2.3. pos: Dedica ed Deploymen
The pos amewo k consis s o wo componen s: a es -
bed o ches a o and an expe imen con olle [3]. Du ing
he pas yea s, pos-managed es beds we e used o e-
sea ch and eaching a he Technical Uni e si y o Mu-
nich. Only a i ualized es bed has been publicly a ail-
able, wi h a limi ed amoun o esou ces.
To o ganize access o es bed esou ces be ween mul-
iple esea che s, he pos amewo k suppo s a calenda -
based esou ce ese a ion and alloca ion. Resea che s can
no e hei in en ion o use esou ces du ing any in e al. In
hei ese ed in e al, expe imen s can be conduc ed by
applying he pos me hodology. Tha is, esea che s may
alloca e hei ese ed nodes, con igu e a bi a y images
o be li e-boo ed, and make use o o he ea u es o he
pos amewo k o implemen hei expe imen .
2.2.4. pos: Vi ualized Deploymen
The i ualized deploymen o pos conside ed in his
wo k is simila o he dedica ed deploymen (c . Sec-
ion 2.2.3). The only di e ence be ween i ualized and
dedica ed deploymen is he ype o he expe imen nodes:
he dedica ed deploymen uses ba e-me al se e s; in con-
as , in he i ualized deploymen , i ual machines (VMs)
a e used. Each physical machine p esen in he dedica ed
deploymen is ep esen ed by a VM unning; all VMs un
on a single physical hos . One bene i o his app oach is
he educed numbe o physical machines equi ed o hos
he es bed and pe o m expe imen s [3].
3. Analysis & Design
In his sec ion, we wan o b ie ly discuss he d i e s
ha shaped oday’s es bed landscape. Based on hese
d i e s, we deduc he equi emen s o po able and e-
p oducible expe imen s. Finally, we sugges a high-le el
design o mee ou equi emen s.
3.1. Requi emen s Analysis
Ini ially, we analyze he equi emen s o es beds wi h
a ocus on he aspec s o po abili y and ep oducibili y.
Requi emen (1): Low-e o ep oducibili y. To os e he
way owa d mo e ep oducible esea ch, he ACM in o-
duced badges ha can be awa ded o pape s o p o id-
ing a i ac s o hei pape s [19]. A i ac E alua ion
(AE) Commi ees e alua e he a i ac s and awa d di e -
en badges acco ding o he quali y o he a i ac s and
hei documen a ion. A su ey among AE pa icipan s,
pape au ho s, and a i ac e alua o s ound he e alu-
a ion p ocess use ul bu ime-consuming [19]. Tes beds,
such as Chameleon and CloudLab, we e p oposed as e-
sou ce p o ide s o a i ac e alua o s [20, 21]. The c e-
a ion o po able expe imen s can help lowe he e o o
he e alua ion p ocess. Au ho s and e iewe s ha e alu-
a e po able expe imen s can use di e en es beds o un
expe imen s, and ha ing a sha ed pla o m o un expe i-
men s simpli ies debugging. Tes beds such as Chameleon
and CloudLab ha e al eady shown hei commi men o
he long- e m a ailabili y o he pla o ms. This ensu es
ha a i ac s can be ep oduced o a longe e m, whe eas
indi idual esea che s o esea ch g oups may no be able
o p o ide his long- e m commi men .
4
Requi emen (2): Low-e o po abili y. Po ing expe i-
men s be ween es beds is highly a ac i e. Tes beds o -
e di e en ha dwa e pla o ms. By suppo ing a po able
expe imen o ma , esea che s can p o i om his ha d-
wa e di e si y by unning hei expe imen s on mul iple
pla o ms. The cu en es bed landscape also allows his.
Howe e , he e o is highe i he expe imen has o be
adap ed o he di e en APIs o he di e en es beds.
Requi emen (3): Po abili y o in e na ional collabo a-
ion. Sha ing es bed esou ces in e na ionally can be chal-
lenging i esea ch is unded by di e en coun ies. US-
based es beds, unded by US axpaye s, na u ally p e e
US-based esea ch. The same is ue o Eu opean e-
sea ch. Though c oss-coun y access o es beds is pos-
sible, we used i ou sel es o his pape , he amoun o
esou ces o e ed o in e na ional gues s may be limi ed.
Po able expe imen s can help sol e his issue. Resea che s
can exchange hei expe imen s and execu e hem on es -
beds ha i hei unding egimes.
Requi emen (4): Expe imen API. Tes bed use s ely on
he s abili y and a ailabili y o APIs o hei expe imen s.
Remo ing he adi ional APIs will aliena e long- e m use s
amilia wi h he es bed-speci ic APIs. A he same ime,
emo ing adi ional APIs will b eak exis ing expe imen s
ha can no longe be execu ed on hese es beds. Con-
e ing es beds o a new API will lead o he same p ob-
lems. The e o e, emo ing APIs o con e ing o a new
API a e unlikely o happen o es ablished es beds. The
only solu ion ha emains is he addi ion o new APIs
while e aining backwa d compa ibili y o olde expe i-
men s. Howe e , adding new APIs ypically equi es he
suppo o he es bed ope a o , who needs o implemen
and deploy so wa e ha o e s he new APIs. This ad-
di ional e o o he ope a o can be a eason why new
APIs a e a ely added.
3.2. Requi emen s s. exis ing es beds
Designed wi h ep oducibili y in mind, pos al eady in-
eg a es ea u es ha mee Requi emen (1). To ensu e
ep oducibili y, he pos expe imen wo k low includes all
sc ip s necessa y o un and documen an expe imen . The
pos wo k low equi es he expe imen e o in eg a e code
ha au oma es expe imen con igu a ion, execu ion, and
e alua ion. This ensu es comple eness o he expe imen
desc ip ion, which is a necessa y p econdi ion o ensu e
po abili y, i.e., Requi emen (2). Requi emen s (3) and
(4) a e no ye add essed by pos in i s cu en o m. In
he ollowing, we sugges and implemen an app oach ha
sol es hese issues. The e o e, we ex end pos o deploy
he pos API o o he es beds (c . Requi emen (4)) wi h-
ou c ea ing he deploymen o e head o es bed ope a-
o s. As a esul o his wo k, he pos API p oposed
by Gallenmülle e al. [3] becomes a ailable ou side o a-
di ional pos-powe ed es beds. In ou app oach, we a oid
any o e head o es bed ope a o s due o he execu ion
o pos-based expe imen s. To add ess Requi emen (3),
in e na ional collabo a ion, we selec ed he US-based es -
beds CloudLab and Chameleon as a ge pla o ms o un
he po able pos expe imen s.
CloudLab and Chameleon also p o ide he means o
c ea e ep oducible expe imen s. Howe e , he es beds
can be used o c ea e ep oducible expe imen s, bu hey
do no ensu e ep oducibili y [22]. In pos, ese ing he
sys ems be o e each expe imen and he equi emen o
au oma e all s eps in he expe imen al wo k low en o ce
he c ea ion o ep oducible expe imen s—a p ope y ha
we call ep oducibili y-by-design. Whe eas CloudLab and
Chameleon expe imen s can be c ea ed in a way ha ul-
ills Requi emen (1), pos inhe en ly suppo s i , making
pos an ad an ageous pla o m o implemen he o he e-
qui emen s.
3.3. High-le el Design
The high-le el a chi ec u e o pos ollows a wo-laye
design, consis ing o he es bed o ches a o as he lowe
laye and he expe imen con olle unning on op (c .
Figu e 1). The es bed o ches a o con ains all es bed-
speci ic unc ionali y; he expe imen con olle only u i-
lizes he in e ace p o ided by he es bed o ches a o . To
achie e po abili y, only he es bed o ches a o needs o
be modi ied when po ing he pos amewo k o o he es -
beds; he expe imen con olle emains unchanged. Wi h
he expe imen con olle le un ouched, expe imen s us-
ing he pos expe imen con olle do no need o be mod-
i ied. P ese ing he expe imen sc ip s educes he wo k-
load o expe imen e s, as hey do no need o adap hei
expe imen code o new es beds. A he same ime, he
ep oducibili y-by-design is conse ed, as es bed o ches-
a ion and expe imen con olle ensu e ha all p ope -
ies o he pos me hodology a e main ained.
Also, apa om po abili y, ano he bene i o ou ap-
p oach comes o mind: expe imen me ada a. To au o-
ma ically en ich expe imen esul s wi h in o ma ion abou
he expe imen en i onmen , e.g., ne wo k opology and
ha dwa e in ol ed, pos cap u es and s o es said in o ma-
ion o each expe imen conduc ed. While pe o med au-
oma ically o scena ios whe e pos ac s as es bed o ches-
a o and expe imen con olle , his capabili y is ex en-
sible o o he es beds, p o iding he equi ed in o ma ion
in a machine- eadable ashion.
Looking a commonali ies be ween scien i ic es beds,
we ind ha such se ings ea u e in pa icula : 1. a se o
en i ies a ailable o expe imen s, 2. a managemen acil-
i y o con ol en i ies a ailable o expe imen s, and 3. op-
ional addi ional se ices p o iding supplemen ing unc-
ionali y o use s, e.g., da a s o age o expe imen esul s.
Based on he p emise o limi ed assump ions, only he o -
me wo can be conside ed when designing he pos ex-
pe imen con olle . Howe e , he use o pos should no
p e en he use o ea u es p esen in he hos ing es bed.
A high-le el o e iew o he a chi ec u e o he pos
amewo k is depic ed in Figu e 2.
5
pos Manage-
men Node
Expe imen
Node
Expe imen
Node
Expe imen
Node
Hos ed Tes bed
Expe imen Expe imen
Managemen
Node
Addi ional
Se ice
Addi ional
Se ice
Addi ional
Se ice
Hos ing Tes bed
Figu e 2: Deploymen o pos con olle inside o he es beds
The e, a di e en ia ion be ween he hos ing and he
hos ed es bed is made. In his wo k, hos ing es bed e e s
o he es bed ha p o ides he physical in as uc u e and
means o o ches a e i . Mo eo e , he hos ing es bed
may, as men ioned, ea u e addi ional se ices su passing
he baseline equi emen s imposed by ou app oach. A
ypical example o a hos ing es bed is a widely used es -
bed, such as Chameleon o CloudLab. Compa ed wi h he
hos ing es bed, he hos ed es bed in Figu e 2 e e s o
he es bed whose ea u es a e used o pe o m he ac ual
expe imen . Fo his ask, we p opose he use o ou pos
expe imen con olle . In his case, he hos ed es bed
is an ins ance o pos ha exis s inside ano he es bed,
i.e., he hos ing es bed. Subsequen ly, we will demon-
s a e ha examples o hos ing es beds include Chame-
leon and CloudLab. Embedded in he hos ing es bed,
he pos amewo k p o ides a subse o he hos ing es -
bed’s a ailable unc ionali y, ul ima ely p o iding an ab-
s ac ion o i . Inside his hos ed es bed’s abs ac ion,
he e is, again, a di e en ia ion o esou ces. On he one
hand, he pos expe imen con olle i sel is unning on
a managed expe imen node. On he o he hand, one o
mul iple expe imen nodes, p o ided by he hos ing es -
bed, a e managed h ough he pos expe imen con olle .
No e ha bo h he managing and he managed expe i-
men nodes may appea indis inguishable in hei ype o
he hos ing es bed’s managemen node. Addi ionally, de-
pending on he hos ing es bed’s design, nodes used by he
hos ed es bed may s em om one o mul iple expe imen s
o he hos ing es bed. Fo example, he pos expe imen
con olle and expe imen nodes managed by i may be
pa o di e en expe imen s on he hos ing es bed, c .
Figu e 2.
Running he pos amewo k in a es bed elies on he
a ailabili y o ce ain unc ionali y o be exposed ia an
API, namely: 1. he abili y o con igu e he powe s a e o
an expe imen node, i.e., u ning i o and on again, and
2. he means o con ol he boo p ocess o expe imen
nodes, e.g., by allowing ne wo k boo o cus omizing he
disk image o boo . Wi h hese equi emen s me , he pos
amewo k can be po ed o a es bed. P esuming a suc-
cess ul po , he pos amewo k can hen expose i s expe -
imen API. This API p o ides expe imen s wi h abili ies
such as: 1. de ining he s a e o hei expe imen nodes,
e.g., powe s a e o unning ope a ing sys em, 2. schedul-
ing execu ion o expe imen sc ip s on expe imen nodes,
3. synch onizing be ween expe imen nodes, and 4. expo -
ing expe imen a i ac s o he pos managemen node.
To summa ize, o hos he pos expe imen con olle
inside a a ie y o es beds, a minimal se o unc ions,
akin o all obse ed es beds and equi ed o allow hos -
ing ou app oach, was de e mined. Building on his, he
pos amewo k was ex ended o enable in e acing wi h
dedica ed as well as ep esen a i e exis ing es beds. De-
ails on he implemen a ions will be p o ided subsequen ly.
The a chi ec u e o he pos expe imen con olle is sum-
ma ized in Figu e 2 and p ominen ly isola es he hos ed
om he hos ing es bed.
4. Implemen a ion
As indica ed p e iously, we implemen ed ou app oach
o a selec ion o es beds. This selec ion is based on he
expe iences epo ed by Nussbaum [22]; he su eyed a ail-
able es beds wi h a ocus on Chameleon, CloudLab, and
G id’5000 [6]. A mo e ecen su ey by Gomez e al. [23]
con i ms his lis o a ailable medium- o la ge-scale cloud-
compu ing o gene al pu pose es beds. While he au ho s
in oduce u he es beds wi h ma ching pa ame e s, e.g.,
P4Campus [24], hese es beds’ mission highligh hem as
designed o a speci ic goal. Thus, we did no conside
hem o he i s se o suppo ed es beds. Howe e , he
gene ali y o he pos me hodology makes i s applica ion
also o in e es o o he domains, e.g., IoT. Gi en ha
CloudLab and G id’5000 suppo GENI [8], we base ou
implemen a ion on his API, hus achie ing compa ibili y
wi h bo h es beds. Suppo o Chameleon is achie ed by
adding suppo o OpenS ack’s API o pos. Consequen ly,
he p esen ed app oach is no limi ed o he chosen es -
beds.
While he pos me hodology is impa ial o he language
and ools used o conduc expe imen s, esea che s a e no .
In e ac i e and isual ools such as Jupy e [25] enjoy g ea
popula i y. In a p e ious wo k, Demchenko e al. [26] in-
es iga ed ep oducible esea ch and ools sui able o his
ask. They show how he pos me hodology and Jupy e
no ebooks can be combined o his e ec . Building on
his, his sec ion shows how Jupy e no ebooks can be in-
eg a ed wi h he p oposed pos expe imen con olle .
This sec ion con inues wi h in o ma ion abou imple-
men a ion conside a ions o he espec i e es beds. As
each implemen a ion ollows he o e a ching a chi ec u e,
his desc ip ion is ocused on pa icula i ies ela ed o he
indi idual es beds. An o e iew o he di e en imple-
men a ion app oaches is gi en in Figu e 3.
4.1. Ex ending pos: So wa e A chi ec u e Pe spec i e
Ex ending pos, i.e., pos’ es bed o ches a o , o sup-
po an addi ional hos ing es bed co esponds o p o id-
ing a ansla ion be ween he ac ion o he wo pa ies. Fo
6
pos Manage-
men Node
Expe imen
Node
Expe imen
Node
Expe imen
Node
Expe imen
(a) Dedica ed pos deploymen
pos Manage-
men Node
Expe imen
Node (VM)
Expe imen
Node (VM)
Expe imen
Node (VM)
Expe imen
(b) Vi ualized pos deploymen
pos Manage-
men Node
Expe imen
Node
Expe imen
Node
Expe imen
Node
Hos ed Tes bed
Expe imen
Tes bed
Si e T o i
(c) Chameleon
pos Manage-
men Node
Expe imen
Node
Expe imen
Node
Expe imen
Node
Hos ed Tes bed
Expe imen Expe imen
Clea ing
House
Agg ega e
Manage
Agg ega e
Manage
Agg ega e
Manage
(d) CloudLab
Figu e 3: Implemen a ion de ails depend on he hos ing es bed
example, a esea che ’s command o eboo an expe imen
node, issued ia pos, mus be ansla ed by pos o in e -
ac ions wi h he hos ing es bed. The hos ing es bed’s
esponse o hese in e ac ions mus be ansla ed back and
in e p e ed by pos, o p o ide he esea che wi h consis-
en beha io . To acili a e he ideally anspa en ans-
la ion be ween he hos ing es bed and pos, pos de ines an
in e nal abs ac ion ha mus be implemen ed o enable
he es bed o ches a o o in e ac wi h a hos ing es -
bed. This in e nal abs ac ion enables pos and i s es bed
o ches a o o execu e basic ope a ions, such as s a ing
o s opping an expe imen node. Consequen ly, he in e -
nal abs ac is he ounda ion o pos’ es bed o ches a o
implemen a ion. Nex o managing he powe s a e o an
expe imen node, ensu ing unique access o an expe imen
node o he du a ion o an expe imen de ines he second
pilla o pos’ in e nal abs ac ion. The selec ion o he
p ope ansla ion laye o in e ac ions be ween pos and
i s hos ing es bed is pa o pos’ con igu a ion. The e,
each a ailable expe imen node is associa ed wi h p ope -
ies, such as i s hos name; hese p ope ies include a e -
e ence o he app op ia e ansla ion laye o he hos ing
es bed.
4.2. pos: Dedica ed Deploymen
In he dedica ed deploymen , c . Figu e 3a, he hos ing
es bed is non-exis en and he pos expe imen con olle
is solely esponsible o managing he in as uc u e. This
implies inc eased con ol o he expe imen in as uc u e.
A he same ime, i also imposes a conside able bu den
on he expe imen e . As an example, he hos ing es beds
may p o ide addi ional eco e y ea u es o in eg a ion o
long- e m da a s o age o use au hen ica ion. Such ea-
u es a e no p o ided by he pos expe imen con olle .
Howe e , as indica ed, he bene i o accep ing his bu -
den is a igh e con ol o he in as uc u e. Fo exam-
ple, some hos ing es beds may only p o ide i ualized
esou ces, hus limi ing he expe imen e ’s con ol o e e-
sou ces and po en ially subjec ing hei expe imen s o he
ac ions o o he s.
While a pos es bed ypically elies on a CLI o un
expe imen s, Jupy e no ebook-based in e ac ion is also
easible. To un a Jupy e no ebook-based expe imen on
he dedica ed pos es bed, he expe imen e i s needs o
p o ision Jupy e in a Py hon i ual en i onmen on he
es bed’s managemen se e . Once he Jupy e no ebook
is ins alled and unning, he expe imen e needs o make
his ins ance accessible, e.g., by o wa ding he espec i e
po ia SSH. A e his se up phase, expe imen e s can
in e ac wi h pos ei he ia he CLI o an op ional Py hon
lib a y. Based on es ablished Py hon lib a ies [27], his
cus omized lib a y w aps pos’ in e nal HTTP API. Fo
expe imen e s seeking o desc ibe hei expe imen s in a
p og amming language, his lib a y exposes bo h high-
and low-le el unc ions o desc ibe a pos expe imen . CLI
and Py hon lib a y a e unc ionally equi alen , i.e., ex-
pe imen e s can choose hei pe e ed app oach wi hou
sac i icing unc ionali y.
4.3. CloudLab
Nei he CloudLab no pos a e designed wi h a speci ic
expe imen sc ip o ma in mind. On he con a y, bo h
es bed and me hodology a e delibe a ely open in e ms
o op ions a ailable o hei use s. Thus, gene ally, any
app oach o deploy he pos expe imen con olle would
be easible. Fo consis en deploymen ac oss es beds, we
decided o implemen a Jupy e no ebook-based deploy-
men .
As highligh ed in Sec ion 2.2.2, o s a an expe imen
on CloudLab o he deploymen o he pos expe imen
con olle , a desc ip ion o he expe imen is equi ed. I.e.,
among o he s, he numbe o nodes and hei ype, hei
connec ions, and disk image. Fo he pos expe imen con-
olle , a dedica ed expe imen was c ea ed. Using a De-
bian bullseye cloud image [28], we use Debian’s cloud-
ini [29] capabili ies o deploy Jupy e and se up sc ip s
o ins all he pos expe imen con olle o he expe imen
node. Once CloudLab p o isioned he expe imen node
and cloud-ini ’s con igu a ion has concluded, a Jupy e
ins ance is a ailable. To se up he pos expe imen con-
olle , he expe imen e execu es cells o one o he p o i-
sioned se up sc ip s, a Jupy e no ebook. Since he pos ex-
pe imen con olle needs o impe sona e he expe imen e
when in e ac ing wi h CloudLab, he expe imen e will be
asked o p o ide means o au hen ica e wi h CloudLab.
Subsequen s eps o his no ebook will deploy pos using
Ansible. Finally, he deploymen ins an ia es a second
CloudLab expe imen . Unless con igu ed o he wise, wo
nodes a e ins an ia ed and in eg a ed in o pos. Wi h pos
con igu ed and he sample opology in place, he second
deployed Jupy e no ebook con aining he sample expe i-
men can be execu ed. Figu e 3d summa izes his imple-
men a ion app oach. The e, “Clea ing House” e e s o
CloudLab’s cen al en i y o esou ce managemen , while
he “Agg ega e Manage ” p o ides an en i y ha p o ides
an in e ace o a ailable esou ces [8].
7
4.4. pos: Vi ualized Deploymen
The deploymen o pos in he i ualized es bed di e s
om he dedica ed deploymen desc ibed in Sec ion 4.2.
Fo ins ance, access o he es bed is g an ed ia a web-
shell. Despi e ha , he implemen a ion o he i ualized
and he dedica ed deploymen do no di e as he ea-
u es o pos used a e unchanged; his simila i y is shown
in Figu e 3b. Howe e , due o he change in access o
he es bed, he wo k low changes sligh ly: ins ead o un-
ning a Jupy e se e on he managemen node, we sugges
con e ing he in ol ed no ebooks o Py hon sc ip s and
subsequen ly execu ing hose.
4.5. Chameleon
Jupy e no ebooks [25] a e a he cen e o Chame-
leon’s wo k low. While he use o no ebooks is no e-
qui ed, hei use is encou aged h ough he a ailabili y o
APIs, examples, and documen a ion on hei in eg a ion.
Gi en ha a co e concep o he pos expe imen con olle
is i s embed-abili y in o di e en es bed con ex s, i s im-
plemen a ion akes his in o accoun . Speci ically, he pos
expe imen con olle p o ides a Py hon API o con ol ex-
pe imen s. As a esul , he in e ac ion wi h he con olle
is well-sui ed o Jupy e no ebooks. The mos p ominen
execu ion engine o Jupy e no ebooks, also called ke nel,
uses Py hon as a p og amming language.
To un an expe imen on Chameleon, a Jupy e no e-
book should be p o ided on Chameleon’s expe imen sc ip -
sha ing and a chi ing pla o m T o i. Thus, ou imple-
men a ion, as shown in Figu e 3c, ollows his app oach
and consis s o a Jupy e no ebook o in e ac wi h Cha-
meleon. The p o ided Jupy e no ebook a emp s o al-
loca e all esou ces, equi ed o la e ope a ion, in Cha-
meleon. I.e., one node ea u ing he pos expe imen con-
olle , he pos managemen node, and o he nodes used
o conduc expe imen s wi h. E.g., o he expe imen dis-
cussed la e , apa om he pos managemen node, wo
expe imen nodes a e eques ed o unc ion as de ice un-
de es and load gene a o .
A e he equi ed esou ces ha e been p o ided by
Chameleon, he implemen a ion is o he wise unable o p o-
ceed, managemen and expe imen nodes a e con igu ed.
In pa icula , he nodes’ disk images a e se . While he
managemen node is con igu ed o un on Debian bullseye,
he expe imen nodes a e con igu ed o boo iPXE [30].
We selec ed he o me due o i s p o en epu a ion
as a s able and well- es ed pla o m. Besides, he selec-
ion o a PXE boo ing image is needed o apply he pos
me hodology. As men ioned, his me hodology includes
li e-boo ing he expe imen node’s ope a ing sys em. To
his end, PXE is used o se e he desi ed ope a ing sys em
o he expe imen nodes. E en hough nume ous de ices
suppo PXE boo ing, i s use o en equi es BIOS econ-
igu a ion. BIOS econ igu a ion is no suppo ed on all
hos ed es beds we encoun e ed and may su e om pa -
ial o laky PXE implemen a ions. The e o e, expe imen
pos Manage-
men Node
DuT LoadGen E alua o
Figu e 4: Expe imen se up
nodes boo he iPXE PXE implemen a ion and, hus, mi -
iga e hese issues.
Wi h he managemen node boo ed, he p o ided Ju-
py e no ebook will con inue o se up a pos expe imen
con olle on he same. This se up s ep elies on Ansi-
ble [31] and, apa om ins alling he so wa e i sel , en-
su es p econdi ions o conduc ing expe imen s, such as
he a ailabili y o boo able images, a e me .
Once he se up sc ip concludes, a b ie unc ionali y
es is pe o med o alida e he success o he deploymen .
Wi h his es succeeding, he desi ed expe imen may be
pe o med.
The p o ided Jupy e no ebook concludes wi h ins uc-
ions o ea down he se up, once all wo k has been pe -
o med. We encou age eleasing expe imen esou ces in-
s ead o elying on au oma ic expe imen e mina ion.
The specializa ion o he high-le el a chi ec u e de-
pic ed in Figu e 2 o he implemen a ion in Chameleon
is shown in Figu e 3c. O pa icula no e is he in e ac-
ion wi h T o i as an addi ional se ice. T o i hos s he
Jupy e no ebook desc ibing he expe imen . Ano he spe-
cializa ion is he way each es bed alloca es esou ces and
how o map he pos expe imen s on o he hos ing es bed:
In Chameleon, o in eg a ion o he pos expe imen con-
olle , bo h managemen and expe imen nodes can be
pa o a single expe imen . On CloudLab, we use wo
sepa a e expe imen s o he managemen node and he
expe imen nodes.
5. E alua ion
To show he usabili y o he de eloped pos expe imen
con olle , we e isi a p e iously conduc ed pos expe -
imen [3, 32]. I.e., we execu e he p e iously desc ibed
expe imen , main aining he p e ious con igu a ion bu
expanding he conside ed es beds. We e isi he ex-
pe imen on he ou suppo ed es beds: (1) ou lo-
cal ba e-me al pos deploymen , (2) a i ualized pos de-
ploymen , (3) he Chameleon es bed, and (4) he Cloud-
Lab es bed. The conside ed expe imen in es iga es he
achie able h oughpu o a de ice unde es (DuT), when
subjec ed o cons an bi a e a ic by a load gene a o
(LoadGen). E alua ion o he expe imen obse a ions is
done a e ha by an e alua o sc ip . Managemen o he
expe imen is done by he pos expe imen con olle . Bo h
DuT and LoadGen un on a Debian bus e li e sys em.
8
Figu e 4 depic s he o e all expe imen se up. The e,
he es a ic is gene a ed by MoonGen [33] on he load
gene a o .
The na u e o he a ic is a ied be ween di e en ex-
pe imen ounds depending on wo pa ame e s. The i s
pa ame e is he packe size, he e, ei he 64 B o 1450 B.
Ranging om 10 kpps o 3000 kpps, he second pa ame e
is he eques ed packe a e. The packe size is chosen o
e lec bo h he smalles and he la ges possible packe
size ansmi able wi hou agmen a ion on any o he
in es iga ed es beds. In pa icula , on Chameleon, he
usual MTU o 1500 B is no a ailable, possibly due o he
use o VXLAN as a unneling p o ocol be ween he un-
ning i ual machines. The p esen ed in es iga ions and
p e ious s udies [32] ha e shown a pe ec linea scaling
o he Linux ou e wi h he numbe o p ocessed pack-
e s. The e o e, we in es iga ed only he minimum and
maximum packe size o he in es iga ed pla o m as ad-
di ional measu emen s wi h o he packe sizes p o ide no
addi ional insigh s.
F om he load gene a o , he gene a ed a ic is sen o
he DuT. The la e is con igu ed o ac as a Linux-based
o wa de , i.e., a ic ecei ed on he ing ess in e ace is
emi ed unchanged on he eg ess in e ace. Packe s om
he DuT’s eg ess hen a i e again a he load gene a o .
Expe imen s a e desc ibed ia and we e conduc ed om
Jupy e no ebooks [25]. While he pos amewo k and,
hus, he pos expe imen con olle is agnos ic o he lan-
guage used o con ain expe imen ins uc ions, he e is a
no iceable p e e ence o his o ma by, e.g., he Chame-
leon communi y. In ou expe imen Jupy e no ebooks,
we sepa a ed he expe imen se up om i s execu ion. As
a esul , he se up o he hos ing es beds, e.g., Chameleon
o CloudLab, is independen o he execu ion o he pos
expe imen . Thus, he pos expe imen Jupy e no ebook
emains he same. This aligns wi h he p oposed design o
ha ing a hos ed es bed p o iding an API independen o
i s hos ing en i onmen .
Figu es 5 and 6 summa ize he measu emen esul s
ob ained om execu ing he same expe imen sc ip s on
mul iple es beds. The indi idual esul s a e discussed in
mo e de ail he ea e .
5.1. pos: Dedica ed Deploymen
Fo he dedica ed deploymen , he expe imen was con-
duc ed on wo dedica ed machines. A DuT ea u ing an
In el Xeon E5-2640 2 unning a 2.0 GHz wi h 32 GB o
memo y. As load gene a o , we used an In el Xeon E5-
2640 2 wi h 16 GB RAM. Bo h DuT and LoadGen we e
equipped wi h a 10 Gbi /s In el X540-AT2. Resul s in Fig-
u e 5a indica e a linea ela ionship be ween eques ed and
ecei ed a ic o gene a ed packe a es below 0.5 Mpps.
The e, o e e y in es iga ed combina ion o he pa am-
e e , he DuT was able o o wa d he a ic such ha
he load gene a o ecei ed any packe s sen . Fo highe
packe a es, howe e , esul s a e mo e di e se. He e, he
0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
2
2.5
3
pk a e [Mpps]
A e age Packe Ra e [Mpps]
64 B TX 64 B RX 1450 B TX 1450 B RX
(a) Dedica ed pos deploymen
0 0.5 1 1.5 2 2.5 3
0
0.5
1
1.5
2
2.5
3
pk a e [Mpps]
A e age Packe Ra e [Mpps]
64 B TX 64 B RX 1450 B TX 1450 B RX
(b) CloudLab
Figu e 5: Measu emen esul compa ison: ba e-me al expe imen
esul s
load gene a o ’s ansmi ed a e con inues o g ow lin-
ea ly un il app ox. 2 Mpps and 1 Mpps o packe s o size
64 B and 1450 B, espec i ely. A e wa d, despi e eques -
ing highe packe a es, he load gene a o is unable o
comply. In con as o ha , he packe ansmi ed back
om he DuT, independen o he packe size, ne e su -
passes a a e o jus below 0.5 Mpps.
5.2. CloudLab
The CloudLab expe imen used wo machines o ype
c220g2 o he LoadGen and DuT. These nodes ea u e
an In el Xeon E5-2660 3, unning a 2.2 GHz, as well as
160 GB RAM each. Mo eo e , his node ype has dual-
po In el X520 NICs, which we e used as expe imen in-
e aces. Simila o he dedica ed deploymen , he esul s
in Figu e 5b show a linea ela ionship be ween eques ed
and ecei ed a ic. I.e., wi h he gi en expe imen , no
de ia ion o he ecei ed a ic om he eques ed a ic
is no iceable.
5.3. pos: Vi ualized Deploymen
In he i ualized se ing, he VMs a e hos ed on a
sys em equipped wi h wo In el Xeon Sil e 4214 12-co e
CPUs unning a 2.2 GHz and wi h 384 GB RAM. Apa
om he pos managemen node p o iding bo h o ches a-
o and con olle , wo expe imen nodes, p o ided by i -
ualized machines, a e p o isioned on his hos sys em.
Each expe imen node is assigned ou co es and 7.4 GB
o RAM. The i ualized machines a e connec ed ia wo
i ual links p o ided by he KVM-based hype iso .
Fo bo h in es iga ed packe sizes, he expe imen e-
sul s, as depic ed in Figu e 6a, look alike. The load gene -
a o is, o all in es iga ed packe a es, able o p o ide he
9
