Techniques to Protect Confidentiality and Integrity of Persistent and In-Memory Data
Abstract
Today computers store and analyze valuable and sensitive data. As a result we need to protect this data against confidentiality and integrity violations that can result in the illicit release, loss, or modification of a user’s and an organization’s sensitive data such as personal media content or client records. Existing techniques protecting confidentiality and integrity lack either efficiency or are vulnerable to malicious attacks. In this thesis we suggest techniques, Guardat and ERIM, to efficiently and robustly protect persistent and in-memory data. To protect the confidentiality and integrity of persistent data, clients specify per-file policies to Guardat declaratively, concisely and separately from code. Guardat enforces policies by mediating I/O in the storage layer. In contrast to prior techniques, we protect against accidental or malicious circumvention of higher software layers. We present the design and prototype implementation, and demonstrate that Guardat efficiently enforces example policies in a web server. To protect the confidentiality and integrity of in-memory data, ERIM isolates sensitive data using Intel Memory Protection Keys (MPK), a recent x86 extension to partition the address space. However, MPK does not protect against malicious attacks by itself. We prevent malicious attacks by combining MPK with call gates to trusted entry points and ahead-of-time binary inspection. In contrast to existing techniques, ERIM efficiently protects frequently-used session keys of web servers, an in-memory reference monitor’s private state, and managed runtimes from native libraries. These use cases result in high switch rates of the order of 10 5 –10 6 switches/s. Our experiments demonstrate less then 1% runtime overhead per 100,000 switches/s, thus outperforming existing techniques.
Full text
Techniques o P o ec
Con iden iali y and In eg i y o
Pe sis en and In-Memo y Da a
A disse a ion submi ed owa ds he deg ee
Doc o o Enginee ing
o he Facul y o Ma hema ics and Compu e Science
o Saa land Uni e si y
by
Anjo Lucas Vahldiek-Obe wagne
Saa b ücken
Oc obe , 2018
Da e o Colloquium: Feb ua y 5 h, 2019
Dean o Facul y: P o . D . Sebas ian Hack
Chai o he Commi ee: P o . D . Ge Smolka
Repo e s
Fi s Re iewe : P o . Pe e D uschel, Ph.D.
Second Re iewe : Deepak Ga g, Ph.D.
Thi d Re iewe : S e an Sa oiu, Ph.D.
Academic Assis an : Engel Le aucheux, Ph.D.
II
III
Abs ac
Today compu e s s o e and analyze aluable and sensi i e da a. As a esul we need
o p o ec his da a agains con iden iali y and in eg i y iola ions ha can esul
in he illici elease, loss, o modi ica ion o a use ’s and an o ganiza ion’s sensi i e
da a such as pe sonal media con en o clien eco ds. Exis ing echniques p o ec ing
con iden iali y and in eg i y lack ei he e iciency o a e ulne able o malicious
a acks. In his hesis we sugges echniques, Gua da and ERIM, o e icien ly and
obus ly p o ec pe sis en and in-memo y da a.
To p o ec he con iden iali y and in eg i y o pe sis en da a, clien s speci y
pe - ile
policies o Gua da decla a i ely, concisely and sepa a ely om code. Gua da
en o ces policies by media ing I/O in he s o age laye . In con as o p io echniques,
we p o ec agains acciden al o malicious ci cum en ion o highe so wa e laye s.
We p esen he design and p o o ype implemen a ion, and demons a e ha Gua da
e icien ly en o ces example policies in a web se e .
To p o ec he con iden iali y and in eg i y o in-memo y da a, ERIM isola es
sensi i e da a using In el Memo y P o ec ion Keys (MPK), a ecen x86 ex ension
o pa i ion he add ess space. Howe e , MPK does no p o ec agains malicious
a acks by i sel . We p e en malicious a acks by combining MPK wi h call ga es
o us ed en y poin s and ahead-o - ime bina y inspec ion. In con as o exis ing
echniques, ERIM e icien ly p o ec s equen ly-used session keys o web se e s,
an in-memo y e e ence moni o ’s p i a e s a e, and managed un imes om na i e
lib a ies. These use cases esul in high swi ch a es o he o de o 10
5
–10
6
swi ches/s.
Ou expe imen s demons a e less hen 1% un ime o e head pe 100,000 swi ches/s,
hus ou pe o ming exis ing echniques.
IV
Ku zda s ellung
Compu e speiche n und analysie en we olle und sensi i e Da en. Das ha zu Folge,
dass wi diese Da en gegen Ve aulichkei s- und In eg i ä s e le zungen schü zen
müssen. Ande n alls d oh die une laub e F eigabe, de Ve lus ode die Modi ika ion
de Da en. Exis ie ende Me hoden schü zen die Ve aulichkei und In eg i ä
unzu eichend, da sie ine izien und an ällig ü mu willige Ang i e sind. In diese
Dok o a bei s ellen wi zwei Me hoden, Gua da und ERIM, o , die pe sis en e
Da en und Da en im A bei sspeiche e izien und wide s ands ähig beschü zen.
Um die Ve aulichkei und In eg i ä pe sis en e Da en zu schü zen, e knüp en
Nu ze ü jede Da ei Rich linien in Gua da . Gua da übe p ü diese Rich linien ü
jeden Zug i und se z diese im Speiche medium du ch. Im Gegensa z zu exis ie en-
den Me hoden, beschü z Gua da o mu willigem Umgehen. Wi besch eiben die
Me hode, eine Implemen ie ung und e aluie en die E izienz on Beispiel ich linien.
Um die Ve aulichkei und In eg i ä on Da en im A bei sspeiche zu schü zen,
isolie ERIM sensi i e Da en mi Hil e on In el Memo y P o ec ion Keys (MPK),
eine neue x86 E wei e ung, um den A bei sspeiche au zu eilen. Da MPK alle dings
nich gegen mu willige Ang i e schü z , e hinde ERIM diese, indem es MPK mi
wide s ands ähigen Wechseln de Speiche be eiche und eine Binä codeübe p ü ung
kombinie . Im Gegensa z zu exis ie enden Me hoden, beschü z ERIM e izien
häu ig genu z e Si zungsschlüssel, Zus ands a iablen eines Re e enzmoni o s und
e wal e e Lau zei umgebungen on na i en Biblio heken. Unse e Expe imen e
zeigen, dass wenige als 1% Lau zei meh au wand je 100.000 Wechselope a ionen p o
Sekunde no wendig sind.
V
Publica ions
Pa s o his hesis ha e appea ed in he ollowing publica ions.
•
“Gua da : En o cing da a policies a he s o age laye ”. Anjo Vahldiek-
Obe wagne , Eslam Elnike y, Aas ha Meh a, Deepak Ga g, Pe e D uschel,
Ansley Pos , Rod igo Rod igues, Johannes Geh ke. In P oceedings o he
Eu opean Con e ence on Compu e Sys ems (Eu oSys), 2015.
•
“ERIM: Secu e and E icien In-p ocess Isola ion”, Anjo Vahldiek-Obe wagne ,
Eslam Elnike y, Nuno O. Dua e, Deepak Ga g, Pe e D uschel. Unde e iew
and echnical epo (a Xi :1801.06822), 2018.
Addi ional publica ions no included in his hesis.
•
“P o ec ing Da a In eg i y wi h S o age Leases”, Anjo Vahldiek, Eslam Elnike y,
Ansley Pos , Pe e D uschel, Rod igo Rod igues. MPI-SWS Technical Repo
2011-008.
•
“Tho h: Comp ehensi e Policy Compliance in Da a Re ie al Sys ems”, Eslam
Elnike y, Aas ha Meh a, Anjo Vahldiek-Obe wagne , Deepak Ga g, Pe e
D uschel. In P oceedings o he USENIX Secu i y Symposium, 2016.
•
“Ligh -Weigh Con ex s: An OS Abs ac ion o Sa e y and Pe o mance”,
James Li on, Anjo Vahldiek-Obe wagne , Eslam Elnike y, Deepak Ga g, Bobby
Bha acha jee, Pe e D uschel. In P oceedings o he USENIX Symposium on
Ope a ing Sys ems Design and Implemen a ion (OSDI), 2016.
VI
•
“Pesos: Policy Enhanced Secu e Objec S o e”, Robe K ahn, Bohdan T ach,
Anjo Vahldiek-Obe wagne , Thomas Knau h, P amod Bha o ia, Ch is o Fe ze .
In P oceedings o he Eu opean Con e ence on Compu e Sys ems (Eu oSys),
2018.
VII
Fynn, Julius, Timon and Ke s in.
VIII
Acknowledgmen s
I would like o ex end my hanks o many people. They so gene ously con ibu ed
o he wo k p esen ed in his hesis and helped me du ing he ups and downs o
g adua e li e.
Fi s ly, I would like o exp ess my since e g a i ude o my ad ise s Pe e D uschel
and Deepak Ga g o hei con inuous suppo . Thei guidance helped me in esea ch
and w i ing o his hesis. I’m e y g a e ul o hem o gi ing me he ime o aise
my amily and be wi h my child en.
I would like o hank he es o my hesis commi ee: Paul F ancis, and S e an
Sa oiu, o hei insigh ul commen s and encou agemen , bu also o he ha d
ques ions which helped me o u he imp o e my esea ch om a ious pe spec i es.
I ha e been e y o una e o collabo a e wi h an as ic ellow PhD s uden s,
Eslam Elnike y and Aas ha Me ha. Wi h hem, he coun less and i ing deadlines
u ned in o ad en u e ips. I canno imagine going h ough g adua e li e wi hou
hei iendship and suppo .
Besides Eslam and Aas ha, I had he g ea pleasu e o collabo a e wi h Bobby
Bha acha jee, Nuno Dua e, Johannes Geh ke, James Li on, Rod igo Rod igues,
and Ansley Pos .
Fu he mo e, I would like o hank ellow s uden s and pos docs a MPI-SWS
o c ea ing a g ea place o wo k, in pa icula A pan, Bimal, Bilal, Cheng, Ekin,
Ezgi, Felipe, Filip, Geo g, Jan-Oli e , James, Juhi, Manoha , Na acha, Nancy, Nuno,
Oana, Paa ijaa , Ped o, P amod, Reinha d, Sco , and Vik o .
IX
Lis o Figu es
2.1 Re e ence moni o implemen a ion scena ios . . . . . . . . . . . . . . 8
3.1 Gua da implemen a ion in a SAN se e . . . . . . . . . . . . . . . . 40
3.2 Absolu e Gua da la ency o e head . . . . . . . . . . . . . . . . . . . 47
3.3 La ency wi h an SSD, ela i e o iSCSI . . . . . . . . . . . . . . . . . 48
3.4 Absolu e la ency wi h SSD . . . . . . . . . . . . . . . . . . . . . . . . 49
3.5 Absolu e la ency wi h HDD . . . . . . . . . . . . . . . . . . . . . . . 49
3.6 SSDI/O h oughpu ........................... 50
3.7 FS benchma ks ead and w i e ( /w) pe o mance . . . . . . . . . . . 53
3.8 Web se e h oughpu . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.9 La ency wi h MAL, olun a y and no logging . . . . . . . . . . . . . 57
4.1
SPEC CPU o e head o CPI/CPS wi h ERIM and an emu-
la ion o WRPKRU (EMUL-CPI/CPS), ela i e o no p o ec ion . . . 90
4.2
SPEC CPU o e head o CPI/CPS and ERIM-CPI/CPS,
ela i e o no p o ec ion. . . . . . . . . . . . . . . . . . . . . . . . . . 95
4.3 Nginx h oughpu wi h one wo ke wi h and wi hou ERIM . . . . . 98
4.4 Nginx h oughpu wi h one wo ke wi h emula ed ERIM and lwCs . . 102
5.1 S eps owa ds an isola ed c yp og aphic lib a y in se e applica ions 116
XVI
CHAPTER 1
In oduc ion
Today compu e s assis people in mos daily ac i i ies such as social in e ac-
ions, lea ning, and in o ma ion sha ing. People en us compu e sys ems wi h
hei aluable da a, e.g., pe sonal media con en , inancial and heal h eco ds, and
c yp og aphic keys. Compu e sys ems ough o p o ec he con iden iali y and
in eg i y o such da a. Con iden iali y gua an ees ha only au ho ized eads o he
da a succeed. In eg i y p e en s unau ho ized upda es o he da a.
Viola ing he con iden iali y and in eg i y o sensi i e da a can esul in i s
leak, loss o modi ica ion. As a esul clien s and o ganiza ions may ace he loss o
highly sen imen al da a and epu a ional o inancial loss. Common causes o da a
con iden iali y and in eg i y iola ions [
15
] include so wa e bugs, secu i y ulne -
abili ies, miscon igu a ion and ope a o e o . Fi s , a bug, e.g., in an applica ion
may o e w i e exis ing iles, iola ing in eg i y. Second, secu i y ulne abili ies in
online se ices may be used by malicious a acke s o ex ac sensi i e da a such as
c yp og aphic keys, iola ing con iden iali y. Thi d, miscon igu a ions may lead o
acciden al da a eads, iola ing con iden iali y. Fou h, an adminis a i e ope a o
o a sys em may acciden ally dele e da a om he sys em, iola ing in eg i y.
To p e en hese iola ions, many echniques o p o ec da a con iden iali y
and in eg i y ha e been p oposed. Model checking, language-based s a ic analysis,
es ing and e e ence moni o s a e b oad classes o such echniques. Model checking
ensu es ha an applica ion ollows a gi en speci ica ion, and hus p o ides he
1
s onges gua an ees compa ed o he emaining gua an ees. Howe e , model checking
o e e yday so wa e (e.g., ope a ing sys ems o web b owse s) is complex and
consequen ly di icul and expensi e, which limi s he use o model checking o
speci ic componen s in high- isk applica ions. Language-based s a ic analysis en o ces
speci ic p og am in a ian s o e sou ce code and ma ks in a ian iola ions such
as bugs and ulne abili ies du ing de elopmen . Due o he app oxima ion o
un ime alues, analysis ools su e in p ac ice om high alse posi i es a es. In
addi ion, limi ed suppo o mul i-language so wa e sys ems hinde s hei b oad
adop ion [
104
]. Tes ing, on he o he hand, p o ides a easonable and bes -e o
co e age o iola ions o e a subse o applica ion inpu s. Al hough widely used,
es ing-based app oaches do no p o ide o mal gua an ees, since es ing e e y
possible inpu is usually un easible, especially when conside ing malicious a acks.
Thus, none o hese echniques p o ides he abili y o en o ce con iden iali y and
in eg i y sys ema ically ac oss applica ions and independen o he applica ion
implemen a ion.
In con as , e e ence moni o s [
6
] en o ce con iden iali y and in eg i y o da a by
obse ing applica ions a un ime, media ing ele an e en s (such as I/O o memo y
accesses) and denying accesses which iola e con iden iali y o in eg i y. T ea ing
he applica ion as a black box allows e e ence moni o s o en o ce con iden iali y
and in eg i y independen o he applica ion implemen a ion and applica ion size,
and sys ema ically ac oss applica ions. This allows adop ion ac oss a wide ange o
use cases including legacy applica ions wi h no access o sou ce code. Compa ed
o model checking o s a ic analysis, which p o e co ec ness o an applica ion,
e e ence moni o s educe he p oo o co ec ness o a smalle and simple piece o
code, namely he e e ence moni o . In con as o he o he echniques, e e ence
moni o s induce un ime o e head on he p oduc ion sys ems o which hey a e
applied. Reducing his o e head is an impo an design conside a ion.
2
Re e ence moni o s ha e been applied o a ious use cases. When p o ec ing da a
con iden iali y and in eg i y, we di e en ia e be ween p o ec ing in- ansi ,pe sis en ,
o in-memo y da a.In- ansi da a is ypically p o ec ed using c yp og aphic me hods
which enc yp he con en s p o iding con iden iali y, and sign he con en s p o iding
in eg i y. Fo secu e communica ion oday’s compu e sys ems ely on he SSL/TLS
s anda d wi h eadily a ailable implemen a ions in se e al c yp og aphic lib a ies
such as OpenSSL [
96
]. Howe e , o bo h, pe sis en and in-memo y da a, exis ing
echniques do no e icien ly and comp ehensi ely p o ec da a con iden iali y and
in eg i y [134, 45, 110, 124, 48, 3, 74, 59, 29, 68, 75, 113, 58, 38, 29].
To p o ec he con iden iali y and in eg i y o pe sis en da a, exis ing moni o ing
echniques [
48
,
134
,
45
,
118
,
139
,
19
,
74
,
5
,
124
,
110
] media e applica ion I/O by in
a lib a y, ile sys em, ope a ing sys em, hype iso , o in he s o age laye . Howe e ,
media ion in a laye o he han he s o age laye can be easily bypassed, and none
o he s o age laye echniques suppo gene al con iden iali y and in eg i y policies.
Hence, a s ong and gene al policy en o cemen echnique o pe sis en da a is
cu en ly missing.
To p o ec he con iden iali y and in eg i y o in-memo y da a om accesses by
an un us ed applica ion, p io wo k elies on memo y isola ion h ough language and
un ime [
38
,
72
,
129
,
143
,
68
], p ocess-based [
74
,
59
,
29
,
20
], o andomiza ion-based
echniques [
113
,
58
]. Fi s , language and un ime echniques isola e by inse ing
checks in o he applica ion bina y o p o ec agains a bi a y da a accesses. Al hough
obus agains malicious a acks, hese echniques su e om un ime o e heads
o pe o m he checks. Second, p ocess-based isola ion spli s he execu ion o an
applica ion in o sepa a e ha dwa e-p o ec ed p ocesses. Simila o language and
un ime isola ion, p ocess-based isola ion is obus agains malicious a acks. The
e iciency depends on he cos o con ex swi ches be ween applica ion p ocesses,
which is usually high. Thi d, andomiza ion-based isola ion uses he huge add ess
space o hide sensi i e da a a a andom loca ion. While andomiza ion-based
3
echniques a e e icien (no checks o high swi ch cos s), malicious a acke s can
ind he sec e loca ion and b eak gua an ees [
113
,
60
,
39
,
49
,
94
]. No exis ing
echnique e icien ly isola es memo y wi h low un ime o e head, and o e s s ong
p o ec ion agains malicious a acke s. As a esul , no exis ing isola ion echnique is
su icien o se e al impo an use cases such as p o ec ing c yp og aphic keys o
na i e lib a ies in managed un imes.
Con ibu ions:
This disse a ion con ibu es Gua da , which e icien ly en o ces ex-
p essi e con iden iali y and in eg i y policies a he s o age laye p o ec ing pe sis en
da a, and ERIM, which s ongly and e icien ly isola es in-memo y da a.
Gua da :
In con as o p e ious echniques [
48
,
134
,
45
,
118
,
139
,
19
,
74
] ha
in e cep a he applica ion o a sys em so wa e laye , Gua da p o ec s he con-
iden iali y and in eg i y o pe sis en da a a he s o age laye . Thus, Gua da
minimizes he size and a ack su ace o he us ed compu ing base (TCB) elied
upon o en o cemen .
P o ec ing a he s o age laye limi s a ailable access in o ma ion o block
add esses. As a esul , exis ing s o age laye echniques echniques [
5
,
124
,
110
] do
no en o ce gene ic con iden iali y and in eg i y policies. To o e come he lack o
clien in o ma ion such as ile names and access c eden ials, clien s communica e wi h
Gua da h ough secu e channels, unneling h ough un us ed sys em laye s like he
ope a ing sys em. Clien s use his communica ion o send addi ional in o ma ion
such as ile names o access c eden ials. Using his in o ma ion, Gua da en o ces
con iden iali y and in eg i y o e e y da a access while elying only on i s own
en o cemen logic.
Wi h Gua da , con iden iali y and in eg i y equi emen s a e speci ied as pe - ile
policies by use s, de elope s, o adminis a o s. Policies speci y he condi ions unde
which a ile may be ead, upda ed, o ha e i s policy changed. These condi ions,
w i en in a decla a i e language, may depend on clien au hen ica ion, he ini ial
4
and inal s a es o he ile (size and con en ) in an upda e ansac ion, o signed
s a emen s by ex e nal us ed componen s (ce i ying, o ins ance, he cu en
wall-clock ime). Gua da s o es he policy as pa o i s own me ada a and ensu es
ha each access o he ile complies wi h he policy.
Fo example, use s can ely on Gua da o mi iga e se ious h ea s: To p e en
he inse ion o malicious code in execu ables, a policy can p o ec execu able iles
by allowing only upda es signed by a us ed pa y; o p e en sys em logs om
co up ion and ampe ing, a policy can p o ec log iles by making hem append-only;
o p e en he illici elease o a use ’s p i a e da a, a policy can p o ec he use ’s
da a by equi ing an au hen ica ed secu e session o ead da a; o p e en a bi a y
ile accesses and allow audi ing o accesses, a policy can p o ec iles by equi ing a
manda o y log en y be o e accessing a ile.
We e alua e he e iciency o Gua da using he policy examples desc ibed
abo e. We show ha Gua da en o ces con iden iali y and in eg i y policies wi h low
o e head. When p o ec ing a web se e ’s con en om accesses by unau ho ized
use s, and bina ies om unau ho ized upda es, he h oughpu o e head is less han
1% compa ed o no p o ec ion.
ERIM:
ERIM is a amewo k o s ong, e icien isola ion o in-memo y da a. I
allows pa i ioning an applica ion in o a us ed and an un us ed componen wi hin
a single add ess space. Fo his, ERIM elies on Memo y P o ec ion Keys (MPK) [
64
],
a ecen x86 ex ension o pa i ion he add ess space in o up o 16 disjoin memo y
domains. Wi h ERIM he us ed and he un us ed componen ’s da a eside in
di e en domains and ERIM con ols access o each domain. A new use -mode CPU
ins uc ion (WRPKRU) swi ches access pe missions o domains e icien ly (abou 60
cycles pe swi ch), wi hou ke nel in e en ion. Al hough e icien , his ins uc ions
allows malicious a acke s o escala e hei access pe missions. Hence, by i sel MPK
5
is no su icien o gua an ee secu i y agains malicious o comp omised un us ed
componen s.
ERIM’s con ibu ion is o build secu e memo y isola ion using MPK by (1)
p o iding call ga es o secu ely ans e con ol o he us ed componen a p ede ined
en y poin s wi hou ke nel in e en ion, and (2) use bina y inspec ion o emo e
exploi able bina y code ensu ing ha he swi ch ins uc ion canno be exploi ed.
As a esul , o gain access o sec e da a, an un us ed componen has o in oke
a call ga e ans e ing con ol o he us ed componen . In con as o p io
echniques, ERIM’s memo y isola ion signi ican ly educes he swi ch cos be ween
he us ed and un us ed componen , does no slow down un us ed componen like
language-based echniques, and p o ec s agains malicious a acke s.
We apply ERIM’s design o challenging and p e iously high-o e head use
cases [
74
,
29
,
72
]. Fi s , we isola e equen ly used OpenSSL session keys o a
web se e (nginx) and show scalabili y. Second, we isola e he sa e egion in an
implemen a ion o code-poin e in eg i y (CPI) [
72
]. Thi d, we isola e a managed
un ime (node.js) om an un us ed na i e lib a y (SQLi e). Ou esul s show ha
ERIM p o ides obus memo y isola ion wi h a low o e head o less han 1% o
100,000 swi ches pe second.
O e iew:
In he emainde o his hesis we u he desc ibe he backg ound and
ela ed wo k (Chap e 2), ollowed by de ailed desc ip ion o he design, implemen a-
ion, and e alua ion o Gua da (Chap e 3) and ERIM (Chap e 4). Finally, we
conclude and desc ibe u u e wo k (Chap e 5).
6
CHAPTER 2
Backg ound
In his chap e we p o ide an o e iew o he backg ound wo k on e e ence
moni o s and b ie ly desc ibe exis ing echniques o p o ec con iden iali y and
in eg i y o pe sis en and in-memo y da a. This chap e is only mean o se e
as a backg ound ma e ial o unde s anding he hesis. A de ailed compa ison o
exis ing wo k is p o ided in Sec ions 3.8 and 4.6.
Re e ence moni o ing en o ces secu i y policies a un ime wi hou insis ing ha
he applica ion be bug- ee. Re e ence moni o s in e cep all ele an ope a ions,
e alua e each ope a ion agains he equi ed policy and deny ope a ions when
iola ions a e imminen . We summa ize s a e-o - he-a echniques o e e ence
moni o ing.
Figu e 2.1 depic s possible implemen a ion scena ios o e e ence moni o s.
Re e ence moni o s ha e been implemen ed a di e en abs ac ion laye s wi hin
he so wa e and ha dwa e s ack (see Figu e 2.1a). Each abs ac ion laye gua ds
access o he esou ces p o ided o highe laye s. Re e ence moni o s in highe laye s
(e.g., applica ion, da abase o ile sys em) ely on p o ec ion gua an ees p o ided by
lowe laye s, inc easing he TCB and isk o ci cum en ion o he e e ence moni o .
While moni o ing a an abs ac ion laye , e e ence moni o s can be implemen ed
by isola ing so wa e componen s in us ed execu ion en i onmen s (TEE) [
82
] (see
Figu e 2.1b) o a sepa a e applica ion p ocess [
22
] (see Figu e 2.1c), by sandboxing
7
Ope a ing Sys em
Applica ion
VMM
CPU
Remo e
Hos
Da abase RM
RM
RM
RM
RM
(a) Re e ence moni o a each le el o abs ac ion
CPU
Applica ion
TEE
T us ed
Componen
RM
(b)
Re e ence moni o in us ed execu-
ion en i onmen (TEE)
Ope a ing Sys em
Applica ion RM
(c)
Re e ence moni o in sepa a e appli-
ca ion
Ope a ing Sys em
RM
Applica ion
(d)
Sandboxing applica ion inside a e -
e ence moni o
Ope a ing Sys em
Applica ion
RM
(e) Inlined e e ence moni o
Figu e 2.1: Re e ence moni o implemen a ion scena ios
an applica ion [
143
] (see Figu e 2.1d), o by inlining moni o s in o he applica ion
i sel [1, 72, 29] (see Figu e 2.1e).
En o cemen echniques in non-applica ion laye s e icien ly media e all accesses
o ele an esou ces (e.g., memo y o iles). Usually lowe abs ac ion laye s, such
as he ope a ing sys em (OS) o i ual machine moni o (VMM), in e cep e en s
wi h coa se-g ain in o ma ion om he applica ion. Each laye abs ac s in o ma ion
wi h help om he applica ion. Fo ins ance, implemen ing a pe ile con iden iali y
policy is only possible wi hin he ile sys em laye o abo e. A hese laye s he
accessed ile and i s associa ed policy is s ill a ailable.
Moni o ing a he applica ion laye o e s he mos de ailed in o ma ion abou he
applica ion s a e and execu ion a he cos o a la ge TCB and isk o ci cum en ion.
Inlining he media ion and en o cemen in o he applica ion [
37
,
72
] o e s he abili y
o p o ec he in eg i y o he con ol low o an applica ion a he cos o addi ional
checks o e e y indi ec jump and e u n. En o cing such applica ion le el gua an ees
8
a a lowe laye (e.g., he OS) is in easible, since e e y check would incu high swi ch
cos s be ween he laye and he applica ion.
While nume ous e e ence moni o ing echniques ha e been sugges ed, his
disse a ion ocuses on p o ec ing he con iden iali y and in eg i y o pe sis en and
in-memo y da a. We desc ibe nex he s a e o he a in p o ec ing pe sis en and
in-memo y da a.
2.1 P o ec ing pe sis en da a
In he ollowing we desc ibe echniques o p o ec pe sis en da a om illici elease,
co up ion o dele ion due o bugs, miscon igu a ions, ope a o e o o malicious
a acks. We do no conside ha dwa e ailu es, since eplica ion (such as RAID [
98
])
o da a enc yp ion mi iga e hese h ea s easily.
In gene al, he da a con iden iali y and in eg i y gua an ees in oday’s compu e
sys ems depend on, and a e sp ead ac oss he applica ion, a da abase managemen
sys em, he OS (including he ile sys em) and i ual machine moni o s. Fo example,
each laye en o ces i s own use access con ol p o ec ing agains illici accesses and in
some cases also keeps da a hashes o p o ec he in eg i y. Compa ed o applica ion
laye p o ec ion, lowe laye s p o ide a s onge p o ec ion agains ci cum en ion, bu
ypically do no p o ide a gene ic policy en o cemen and ins ead ocus on speci ic
uses and policies.
Hype iso /OS da a p o ec ion
Nexus [
118
] and TAOS [
139
] a e wo OS-le el
echniques ha en o ce au ho iza ion policies on OS in e aces (e.g., iles, in e -
p ocess communica ion, memo y mappings o p ocess managemen ) p o ec ing da a
con iden iali y. Nexus op ionally main ains a Me kle hash ee o he ile sys em
o p o ide da a in eg i y. In con as o Nexus and TAOS which en o ce policies,
Dune [
19
] and lwC [
74
] a e amewo ks o build a e e ence moni o a he OS
abs ac ion laye media ing he sys em call in e ace and bo h show use cases o
9
CHAPTER 3
Gua da : En o cing da a policies a
he s o age laye
This chap e desc ibes Gua da , a sys em o en o ce con iden iali y and in eg i y
policies on pe sis en da a. B ie ly, he p oblem is ha compu e and s o age sys ems
inc ease in complexi y and so does he isk o da a con iden iali y and in eg i y
om so wa e bugs, secu i y ulne abili ies and human e o . In addi ion, da a
is inc easingly s o ed on hi d-pa y pla o ms, in oducing addi ional isks like
unau ho ized da a use by he hi d pa y. Da a s o ed in hi d-pa y pla o ms ely
on he us and eliabili y o he hi d-pa y p o ide . Today’s sys ems en o ce he
applicable secu i y policy o a ile implici ly in hei code. Fu he mo e, he policy
speci ica ion and en o cemen may sp ead o e di e en subsys ems, inc easing he
isk o ci cum en ion and miscon igu a ion.
Gua da in oduces a e e ence moni o a he s o age laye o ackle hese
challenges. I p o ides a single-poin o policy speci ica ion, con igu a ion and
en o cemen a he s o age laye elying only on i s own policy in e p e e , en o cemen
logic and explici policy dependencies, hus minimizing he TCB and a ack su ace.
The ollowing sec ions desc ibe Gua da ’s design and API, i s decla a i e policy
language, example use cases, an implemen a ion, ela ed wo k and an expe imen al
e alua ion o a p o o ype implemen a ion.
17
3.1 Design
Gua da ’s design was guided by ou p inciples:
1.
Gua da policies a e a ached o iles, sepa a e om code, and speci ied in a
cus om decla a i e policy language. The e o e, he policy o a ile’s da a can
be speci ied concisely in one place and audi ed easily.
2.
Gua da en o ces policies in he s o age laye o minimize he isk o policy
ci cum en ion. Ou implemen a ion o Gua da in a SAN se e , o ins ance,
allows a scalable con igu a ion whe e policies a e en o ced by block se e s
in a machine oom, while clien compu e s and he en e p ise ne wo k a e
un us ed.
3.
Gua da policies s a e me ely wha accesses a e allowed unde which condi ions,
lea ing i o un us ed code how o demons a e compliance wi h a policy. This
sepa a ion keeps he policy language small and policies concise, while shi ing
complexi y o un us ed so wa e and o e head o clien compu e s.
4.
Gua da elies on c yp og aphic ile a es a ions o b idge he seman ic gap
be ween pe - ile policies and block-le el en o cemen . By eques ing an a es a-
ion o a ile’s policy, name and con en hash, an applica ion can e i y ha
Gua da associa es da a and policy co ec ly, independen o he ilesys em o
i s me ada a.
En o cemen a he s o age laye is p e e able, since i minimizes he isk o ci -
cum en ion, and makes i easy o physically p o ec he us ed Gua da componen s
in a machine oom. A design en o cing a a highe laye (e.g., NAS ile se e , VMM
o clien OS laye ) would ex end us o addi ional, and likely mo e dis ibu ed,
componen s. Mo eo e , Gua da is able o b idge he seman ic gap be ween iles and
blocks as wi hou elying on he un us ed ilesys em and i s me ada a.
18
Da a s o ed in Gua da is o ganized in o iles. Fo each policy-p o ec ed ile,
Gua da main ains i s own shadow me ada a, consis ing o an o de ed lis o ex en s,
a unique nume ic iden i ie , a ex ual name s ing ( ypically used o s o e he ile’s
pa hname(s)—mul iple in he case o ha d links), and a e e ence o a policy in
e ec o he ile. The se o nume ic iden i ie s o m a la namespace, while he
se o names ypically encode a con en ional namespace hie a chy main ained by an
un us ed ilesys em. Each ile can ha e i s own policy bu , ypically, a collec ion o
iles sha e he same policy.
The policy o a ile consis s o ou ules, one o each o he pe missions
ead
,
upda e
,
des oy
and
se policy
. Each ule speci ies condi ions on he con ex and
en i onmen unde which he espec i e pe mission holds. Abs ac ly, he
ead
ule
ep esen s he ile’s con iden iali y policy; he
upda e
ule encodes he ile’s in eg i y
policy; he
des oy
ule go e ns when he ile’s iden i ie (name) can be ecycled;
and he
se policy
ule desc ibes when he policy can be changed. S o age commands
ha ead o upda e a ile o i s me ada a check condi ions o he co esponding
policy ules.
Gua da in eg a es wi h ilesys ems. The (un us ed) ilesys em as usual assigns
names and s o age blocks o a ile and ansla es ile eques s in o block eques s
using i s me ada a. Gua da uses i s own shadow me ada a o look up he ile and
policy associa ed wi h a block eques secu ely and e icien ly. Gua da also assigns
i s own unique ile iden i ie s, which can be eused only unde policy con ol.
File a es a ions ie he GDC’s iew o a ile as a sequence o ex en s o an
applica ion’s iew o a named ile, he eby emo ing he need o us he ilesys em
and i s me ada a. By eques ing an a es a ion a e a ile is w i en o ead, an
applica ion can e i y ha i s iew o he ile is iden ical o he GDC’s. Gua da has
suppo o spa se iles. The cu en design assumes ha a block is assigned o a
mos one ile; block sha ing o suppo de-duplica ion, o ins ance, could be added
easily.
19
Gua da ’s p og am logic, called he Gua da con olle o GDC, is in eg a ed
wi h a s o age block de ice and en o ces policies on e e y ead and w i e. The
GDC ex ends he s anda d block-de ice in e ace wi h a ile-le el in e ace, which
allows highe so wa e laye s o (a) c ea e, dele e, ead and upda e se s o ex en s
( iles) using simple ansac ions, (b) associa e policies wi h iles, (c) c yp og aphically
au hen ica e and es ablish secu e sessions, (d) p o ide c eden ials and o he e idence
o policy compliance, and (e) ob ain a es a ions on s o ed iles and hei policies. The
ile-le el in e ace can be used by a Gua da -awa e ilesys em, o by an applica ion
lib a y in combina ion wi h a legacy ilesys em ia IOCTL calls.
3.2 Th ea model
The GDC, me ada a and da a mus be physically p o ec ed om unau ho ized access
and unde ec ed ampe ing. In ou implemen a ion (see Sec ion 3.6.1), da a and
me ada a s o age de ices a e assumed o be physically p o ec ed, e.g., in a machine
oom wi h es ic ed access. Gua da policies a e en o ced, subjec o ex e nal policy
dependencies, ega dless o bugs, miscon igu a ions, o secu i y inciden s ou side he
s o age de ice, including inciden s on any numbe o clien machines.
We make s anda d assump ions abou policies: Co ec policies mus be ins alled
when da a is i s s o ed, and ex e nal dependencies o policies like ime se e s,
clien au hen ica ion keys, and admin au hen ica ion keys mus be us wo hy (in
pa icula , admin au hen ica ion keys can o en be s o ed o line and p o ec ed phys-
ically). Unde hese assump ions, Gua da de ends agains h ea s o con iden iali y
and in eg i y o s o ed da a. In addi ion, Gua da can p o ec he in eg i y and
con iden iali y o iles ans e ed be ween Gua da de ices, and be ween a Gua da
de ice and a clien de ice h ough a secu e channel. This includes h ea s due o
bugs and ulne abili ies in in e media e so wa e laye s including ope a ing sys ems,
ilesys ems, s o age se ices buil on op o Gua da , and ne wo ks, and h ea s due
20
o human negligence and oppo unis ic malice. Gua da is no conce ned wi h da a
a ailabili y. To mask he e ec s o a ha dwa e o media ailu e, loss, o des uc ion
o a Gua da de ice, da a mus be eplica ed on mul iple Gua da de ices wi h
independen ailu e modes.
3.3 In e ace
Gua da ex ends he s anda d block de ice in e ace wi h means o es ablish sessions,
c ea e, upda e and dele e iles, ins all policies, p o ide e idence o policy compliance,
and ob ain a es a ions. In he ollowing, we desc ibe he unc ionali y p o ided by
he in e ace. Table 3.1 shows all Gua da API calls.
3.3.1 Session in e ace
A use applica ion (also called a clien ) in e ac s wi h Gua da in a session. A
secu e, au hen ica ed session mus be used o access iles whose policy equi es clien
au hen ica ion. To access o he iles, no explici session is equi ed. Such use is
concep ually ea ed as pa o a de aul , un us ed session.
A session is es ablished wi h a s anda d handshake p o ocol in which he clien
and Gua da au hen ica e each o he using hei p i a e keys. As pa o he p o ocol,
new, session-speci ic keys a e c ea ed. These keys a e used o enc yp and/o
au hen ica e ( h ough message au hen ica ion codes) all subsequen communica ion
in he session. This p o ec s in- ansi da a and commands om snooping and
modi ica ion in in e media e laye s. Mo eo e , he clien ’s public key (which ac s as
a clien iden i ie ) becomes a ailable du ing e e y policy e alua ion in he session;
hence, Gua da can en o ce policies ha es ic access o a speci ic use . A he
end o he handshake, Gua da e u ns a unique session iden i ie (sId) ha links
la e commands o he session. In he desc ip ion o he emaining in e ace, we omi
21
Session API:
message,sId handshake1(message) Ini ia es he session es ablishmen .
in handshake2(sId,message) Finalizes he session es ablishmen .
in endsession(sId) Te mina es he session sId.
T ansac ion API:
Id openTx(sId,objname) S a s a ansac ion on ile named objname.
in endTx(sId, Id) Commi s a ansac ion.
in
se Policy
(sId, Id,
pId)
Se policy pId o objname.
in
euse
(sId, Id,o ,len,
o ’)
Takes con en o in e al [o ,o +len - 1] and inse s
con en a o ’.
in
esh
(sId, Id,b,len,
bu ,o [, cache lag])
W i e con en o block and add o objname a o se o .
bu
eadTx
(sId, Id,o ,
len [, cache lag])
Reads om objname a o se o .
File/Policy API:
pId c ea ePolicy(sId,policy) S o es policy and e u ns a unique iden i ie pId.
in des oy(sId,objname) Dele es objname’s me ada a and con en .
Con en Hashing API:
hId ini Hash( Id,cu O New)
C ea es hash iden i ie o cu en o new objname.
ce i ica e closeHash(hId)
Compu es hash, c ea es ce i ica e and s o es hash
in cache.
Ce i ica e API:
nonce ge Nonce(sId) Re u ns pseudo- andom nonce alue.
in se Ce i ica e(sId,ce i ica e) P o ide ce i ica e o Gua da .
ce i ica e a es (sId,objname,nonce)
C ea es a ce i ica e a es ing he s a e o obj-
name.
Replica ion/Mig a ion API:
bu pickle(sId,objname, a ge GdKey)
C ea es an enc yp ed bu e bu including
he con en and policy o ile objname which
can only be enc yp ed by a Gua da de ice
wi h a ge GdKey as public key.
in unpickle(sId,bu ,objname)
Dec yp s bu o ex ac con en cand policy
p; c ea es policy pand ile named objname
wi h con en c; associa es he p e iously c e-
a ed policy.
Table 3.1: Gua da In e ace Calls
22
he sId a gumen as i appea s in e e y call. Gua da can wo k wi h any clien -side
in as uc u e o c ea ing, managing and dis ibu ing public keys.
3.3.2 T ansac ion in e ace
Rich policies may equi e mo e han one ead o w i e ope a ion o ansi ion a
ile om one complian s a e o ano he . Fo ins ance, a ile’s in eg i y policy may
equi e ha each upda e inc emen s an embedded e sion coun e . Fo his pu pose,
Gua da suppo s ansac ions consis ing o a sequence o eads and upda es on a
single ile. T ansac ions a e a omic: ei he all he upda es a e pe sis ed o hey a e
all disca ded. Policies may e e o bo h he cu en and new con en o a ile in
a ansac ion, as well as he con en o o he iles. The policy is checked once a
he end o he ansac ion, which commi s i he policy check succeeds, and abo s
o he wise.
We ind his design use ul in encoding policy s a e machines and access-accoun ing
policies, as illus a ed in Sec ion 3.5. Howe e , he design comes wi h a ade-o :
To a oid bu e ing a po en ially unbounded numbe o upda es du ing a ansac ion,
Gua da o bids des uc i e upda es as pa o a ansac ion. Ins ead, new con en
mus be w i en o esh (no cu en ly alloca ed o a policy-p o ec ed ile) ex en s
on disk. This choice mi o s mode n ilesys em designs wi h copy-on-w i e block
alloca ion, e.g., in WAFL, ZFS, and B s [
56
,
125
,
23
]. Ou side a ansac ion,
des uc i e w i es succeed i allowed by he policy.
The ansac ion API adds 5 new commands: openTx, endTx, euse, esh and
eadTx, se Policy. The call
openTx
(sId,objname) s a s a new ansac ion on he ile
named objname. Gene a ing objname is up o he (un us ed) highe laye s, e.g., he
ilesys em. I objname does no exis , a new emp y ile is c ea ed and gi en his name
( his is he only way o c ea e a ile in Gua da ). The call e u ns a ansac ion id
( Id) ha links la e calls o he ansac ion and he session. A ile is upda ed by
23
eusing con en om i s cu en e sion and adding esh con en o c ea e a new
e sion. The call
euse
( Id,o ,len,o ’) akes con en in he logical ange [o ,o +len-1]
om he cu en e sion and inse s i a o se o ’ in he new e sion (inse ion is
pu ely a me ada a ope a ion). The call
esh
( Id,blk,len,bu ,o ) w i es len by es
om bu e bu o he ex en s a ing a by e numbe bon disk and adds he esul ing
ex en o he new e sion a logical o se o . Be o e w i ing he ex en , Gua da
checks ha i is no occupied by any ile (including he ile being modi ied). The
new e sion o he ile may be gi en a new policy wi h he call
se Policy
( Id,pId).
The call bu
eadTx
( Id,o ,len) eads len by es o he ile s a ing a logical o se o
in he ile and e u ns he esul o he bu e bu . The ead ule o he ile’s policy is
e alua ed be o e eading o bu ; i i denies access, he call ails. This en o ces da a
con iden iali y. No e ha we allow by e-le el add essing on iles, so policies can be
e y ine-g ained.
The upda es in a ansac ion a e commi ed wi h he call
endTx
( Id). Gua da
e alua es he upda e ule o he ile’s policy be o e commi ing he new e sion. This
en o ces da a in eg i y. The upda e ule has access o he cu en and new con en o
he ile, as well as ele an me ada a, e.g., he o se s and leng hs o eads and w i es
in he ansac ion. Addi ionally, i he policy has been upda ed, Gua da e alua es
he se policy ule o he ile’s policy; his p o ec s he policy i sel om unau ho ized
changes.
3.3.3 File/Policy in e ace
While ile c ea ions a e implemen ed as ansac ions, ile des uc ion and policy
c ea ion exis as addi ional calls. The
des oy
(objname) call emo es he con en
and me ada a o he ile named objname om Gua da a e success ully e alua ing
he des oy pe mission o he associa ed policy. To educe he equi ed me ada a
space, Gua da allows mul iple iles o be p o ec ed by he same policy. The e o e,
24
he
c ea ePolicy
call e u ns a policy Id (pId) which can be used mul iple imes in
se Policy calls du ing a ansac ion.
3.3.4 Con en cache in e ace
Gua da policies may be con ingen on he cu en con en o one o mo e iles and
he p oposed new con en o he upda ed ile in he con ex o a ansac ion. To
enable he e icien e alua ion o such policies, wo Gua da caches hold ile con en
o use in policy e alua ion. A pe -session cache con ains en ies ha e e o cu en
ile con en s, ei he as a sequence o by es a a gi en ile o se and leng h, o as
he hash o such a sequence. A pe - ansac ion cache con ains he same ypes o
en ies bu e e s o en a i e upda es o a ile. En ies a e added o he cache as a
side-e ec o ead, w i e, esh o eadTx commands wi h app op ia e lags (cache lag).
When a ansac ion commi s, any en ies in he ansac ion cache a e mo ed in o
he session cache, and any exis ing session cache en ies hey supe sede a e e ic ed.
When a ansac ion abo s, he en ies in he ansac ion cache a e disca ded. To
sa is y a policy ha e e s o cu en o pending ile con en , un us ed clien code
is expec ed o ill app op ia e cache en ies by issuing ead/w i e commands be o e
a emp ing a ansac ion commi .
In o de o i e a i ely build con en hashes, Gua da o e s he
ini Hash
call o
s a he hash compu a ion. I he e u ned iden i ie (hId) is speci ied as cache lag
du ing a ead, w i e, esh o eadTx call, hen he espec i e con en is added o
he hash compu a ion. A e a clien inishes he ead/w i e sequence, she closes
he hash ia he
closeHash
call which inalizes he hash compu a ion and s o es
he esul in he espec i e session o ansac ion cache o la e use du ing policy
e alua ion. In addi ion a c yp og aphically signed ce i ica e including he compu ed
hash, ile name and a hash o he associa ed policy is e u ned.
25
We belie e ha policies will be w i en mos ly by p i acy and secu i y expe s.
Fo any applica ion, he e will be a limi ed numbe o basic use ul policies, and
mos sys em adminis a o s, use s o de elope s will me ely selec om a lib a y o
policies, pe haps wi h mino cus omiza ion.
3.5 Policy examples
We illus a e Gua da ’s capabili ies by p esen ing example policies o p o ec exe-
cu ables, log iles and backups. I he
ead
o
upda e
ule o a policy is omi ed,
hen he pe mission is always allowed and i a
se policy
o
des oy
ule is omi ed,
hen ha pe mission is ne e allowed.
3.5.1 P o ec ed execu ables
Fo an execu able ile, i is desi able o p e en acciden al o malicious o e w i ing
o ollback o a p io e sion. A ep esen a i e Gua da policy o accomplish his
is shown below. The policy s a es ha he new con en o he execu able a e any
upda e mus be signed by he so wa e endo (called “Vendo ”) as being e sion 10
o la e . Mo eo e , any policy changes mus be ce i ied wi h he adminis a o ’s
key, kad.
upda e :- ile_name_is(F)∧new_leng h_is(L)∧
(0, L)willHa eHash Nh ∧key_is(K, “Vendo ”)∧
Ksigns ok_hash(F, N, Nh)∧(N≥10)
se policy :- ile_name_is(F)∧
new_pol_hash_is(Nph)∧
kad signs good_policy(F, Nph)
The i s ule allows an upda e o he ile only i he e is a public key
K
belonging
o “Vendo ” (condi ion
key_is
(
K, “Vendo ”
)), which signs ha he ile’s new con en
32
hash,
Nh
, is he
N
h e sion o he execu able (condi ion
Ksigns ok_hash
(
F, N, Nh
))
and
N≥
10. The p edica es
key_is
(
K, “Vendo ”
)and
Ksigns ok_hash
(
F, N,Nh
)a e
e i ied om clien -p o ided ce i ica es signed by a ce i ying au ho i y and he
endo , espec i ely. The second ule allows a change o he execu able’s policy only
i he hash o he new policy, called
Nph
, has been ce i ied by he adminis a o
(condi ion kad signs good_policy(F, Nph)).
P ope ies:
As long as he in eg i y o he endo ’s and admin’s keys is main ained,
iles p o ec ed by he policy canno be o e w i en excep wi h con en signed by he
endo and e sion
≥
10, e en i he en i e sys em is comp omised (w i e in eg i y).
A a ian o his policy can limi con en on he sys em’s boo sec o o endo -signed
boo images, hus p o ec ing he boo sequence om ojans and oo ki s.
3.5.2 Append-only logs
The ollowing policy speci ies an append-only ile ha may be ex ended by anyone
bu modi ied in-place (e.g., o a ed) only by an adminis a o iden i ied by he public
key
kad
. The policy p e en s acciden al o malicious manipula ion o sys em log iles.
upda e :- session_is(kad)∨
(old_leng h_is(Lo)∧new_leng h_is(Ln)∧(Ln ≥Lo)∧
upda ed_loca ions_a e(M)∧disjoin (M, [0, Lo]))
The policy allows an upda e i ei he he session is au hen ica ed by he adminis a o
(condi ion
session_is
(
kad
)) o he ile’s new leng h
Ln
exceeds i s cu en leng h
Lo
and he i s Lo by es o he ile a e no modi ied.
P ope ies:
As long as he in eg i y o he admin’s key is main ained, he policy is
en o ced e en i he sys em is comp omised.
33
3.5.3 P o ec ed backup
Backup iles can be p o ec ed om acciden al o malicious modi ica ion o a ixed
pe iod o ime using he ollowing policy.
upda e :- key_is(K, “TimeSe e ”)∧
Ksigns ime(T)a Ti∧
coun _is(Tj)∧(T+Tj−Ti>endT)
The policy allows modi ica ion o he ile only i he cu en ime exceeds a p e-
de e mined ime
endT
. To en o ce such policies, Gua da elies on signed ce i i-
ca es om ime se e s and a sho - ange in e nal iming coun e . In de ail, he
policy says ha he e should be a key
K
belonging o a ime se e (condi ion
key_is
(
K, “TimeSe e ”
)), which issued a ce i ica e ha he ime was
T
when he
Gua da in e nal coun e had alue
Ti
(condi ion
Ksigns ime
(
T
)
a Ti
), he cu en
in e nal coun e alue is
Tj
(condi ion
coun _is
(
Tj
)) and he cu en ime (calcula ed
as T+Tj−Ti) exceeds he backup end ime endT.
P ope ies:
As long as he in eg i y o he ime se e and i s signing key is
main ained, a ile wi h his policy canno be modi ied be o e he designa ed ime,
e en i he sys em, he admin’s and he ile owne ’s p i a e keys a e comp omised.
3.5.4 Manda o y access logging (MAL)
Legisla ion and o ganiza ional policies o en manda e ha all ead and w i e access
o sensi i e in o ma ion like medical eco ds be logged. Al hough applica ion-le el
solu ions o en o ce such manda o y access logging (MAL) exis , en o cing he policy
in Gua da is desi able because i would inc ease secu i y.
34
Fo his exposi ion, le
P
be he sensi i e ile which mus be p o ec ed by MAL
and le
L
be i s log ile. We assume ha he log ile is append-only, h ough he
policy desc ibed ea lie . The MAL equi emen is h ee- old:
Comple eness
Fo e e y ead on
P
, an en y in
L
should desc ibe who ead and
om whe e in
P
. Fo e e y w i e, a simila en y mus exis in
L
and i mus
addi ionally con ain a hash o he con en w i en.
Causali y
Gi en wo w i e en ies in
L
, he o de in which hey we e applied o
P
should be e iden and, simila ly o a ead and a w i e en y.
P ecision
Call a w i e en y in
L
dangling i i does no co espond o an ac ual
w i e on
P
. Then, ei he dangling en ies should no be allowed in
L
o hey
should be de ec able.
Dangling ead en ies a e usually no a p oblem, because i is in he clien ’s
in e es o es ablish ha i did no ead ce ain da a and, hence, no c ea e dangling
ead en ies. We also desc ibe la e how ead en ies can be made p ecise.
We s a wi h an ob ious s awman policy o
P
, which is comple e, bu does
no p o ide causali y and p ecision. We e ine he design la e . We de ine wo kinds
o en ies o
L
:
may_ ead
(
K, S
), which indica es ha he clien wi h public key
K
has po en ially ead he se
S
o (o ,len) anges om
P
; and
change
(
K, S, H
),
which s a es ha con en wi h hash
H
has been w i en o he anges in
S
. To o ce
logging o eads, we equi e in he
ead
ule o
P
’s policy ha i he ange
R
is ead
by clien
K
, hen an en y
may_ ead
(
K, S
)wi h
R⊆S
exis in
L
. Simila ly, w i e
logging could be o ced h ough P’s upda e ule.
This s awman policy o
P
can be exp essed in he Gua da policy language
because he se
R
o loca ions ead o upda ed is a ailable h ough con ex ual
p edica es in he policy language, he clien
K
is a ailable h ough he p edica e
is_session
(
K
)and
L
’s con en is a ailable h ough he session cache (p edica e
says
).
35
The policy can also be easily sa is ied by he clien : P io o eading o w i ing,
he clien could append an app op ia e en y o
L
and ha e i cached o
P
’s
subsequen policy e alua ion. E en hough his policy sa is ies he MAL equi emen
o comple eness, i does no sa is y causali y and p ecision. No hing in
L
’s policy
p e en s he clien om c ea ing en ies ha a e ne e used and such en ies canno
be dis inguished om o he s ( his iola es p ecision). Mo eo e , no hing in
P
’s
policy p e en s use o L’s en ies ou -o -o de , which iola es causali y.
To ob ain causali y and p ecision, we e ine his s awman design. We embed a
coun e in each en y in
L
and en o ce h ough
L
’s policy ha he coun e inc ease
by 1a each successi e
change
en y and emain he same a each
may_ ead
en y. We
en o ce h ough
P
’s policy ha he alue o he coun e in he las
change
en y ha
has al eady been applied o
P
be w i en a a designa ed locus in
P
. Fu he , he
en y used o jus i y a ead mus ha e a coun e numbe ha ma ches he cu en
coun e in
P
. We desc ibe below how we en o ce hese equi emen s. Assuming ha
hey ha e been en o ced, bo h causali y and p ecision a e sa is ied. Causali y holds
because he policies jus desc ibed o ce ha
change
en ies apply o
P
in inc easing
o de o hei coun e numbe s, and ha a ead co esponding o a
may_ ead
is used
a e all
change
en ies wi h smalle o equal coun e numbe s ha e been applied.
P ecision holds because a
change
en y is dangling i and only i i s coun e numbe
is highe han he coun e in P.
The log’s en ies a e e ised o include coun e numbe s. They ake he o ms
may_ ead
(
N, K, S
)and
change
(
N, K, S, H
), whe e
N
deno es a coun e . We ese e a ixed
locus in
P
o a coun e , called
C
. The log is ini ialized wi h a dummy en y wi h
N
= 0 and
P
is ini ialized wi h
C
= 0. We desc ibe ele an policies o
L
and
P
in
wo ds, omi ing symbolic ep esen a ions o cla i y. We ha e o mally ep esen ed
hese policies in ou p o o ype implemen a ion; expe imen al esul s a e p esen ed in
Sec ion 3.7.5.
36
L’s upda e policy:
Only appends a e allowed and only en ies o he wo designa ed
o ms may be added. I he added en y has he o m
may_ ead
(
N, . . .
), hen
N
mus be copied om he p e ious en y and i he added en y has he o m
change
(
N, . . .
), hen
N
mus be one mo e han he p e ious en y’s coun e .
These equi emen s can be ep esen ed in he Gua da policy language because
he p e ious en y and he new en y a e accessible h ough he session and
ansa ion caches, espec i ely, du ing e alua ion o he upda e ule.
P’s ead policy: L
mus con ain a
may_ ead
en y wi h he same coun e numbe
as
C
and ange se la ge han he ac ual ange ead.
L
’s ele an en y and
C
a e accessible h ough he session cache du ing
P
’s policy e alua ion. In
pa icula ,
C
can be e e enced because Gua da suppo s by e-le el add essing
on iles and he locus o
C
is ixed in ad ance. The clien is esponsible o
speci ying which en y o Lin he session cache sa is ies he policy.
P’s upda e policy: L
mus con ain an en y desc ibing he upda e p ecisely. The
coun e in he en y mus be one mo e han
C
. The upda e mus also inc emen
C
by 1. When e alua ing
P
’s policy,
L
’s ele an en y and he old alue o
C
a e accessible h ough he session cache. The new alue o
C
is accessible
h ough he ansac ion cache.
MAL clien :
The MAL clien mus pe o m some bookkeeping s eps o sa is y he
MAL policy. P io o each access on
P
, app op ia e log en ies mus be c ea ed and
commi ed o Gua da . When c ea ing log en ies, lags mus be se o bu e hem
in he con en cache o use in
P
’s policy e alua ion. A log en y’s cache eco d is
also necessa y o c ea e he nex log en y. Simila ly, when
C
is upda ed, lags mus
be se o cache i o use in u u e policy e alua ions. This app oach ollows om
ou design p inciple o placing he bu den and complexi y o how o sa is y a policy
on he un us ed code.
37
The o e head o c ea ing log en ies o upda es can be educed by commi ing
ansac ions less equen ly (and, hence, equi ing ewe
change
en ies). Simila ly,
he o e head o c ea ing log en ies o eads can be educed by clubbing se e al
an icipa ed eads in o a single
may_ ead
en y. The pe o mance bene i o hese
op imiza ions is subs an ial and we epo on i in Sec ion 3.7.5. Applica ions
ha canno accu a ely es ima e hei ead-se s ahead o ime can simply c ea e
blanke
may_ ead
en ies ha co e he en i e ile and pe iodically commi ead-only
ansac ions accompanied by special log en ies ha speci y p ecisely wha has been
ead in he ansac ion. The p ecise ead se is a ailable o Gua da du ing a commi
ansac ion, so he log en y’s accu acy can be e i ied. This mode o use equi es a
second coun e in log en ies and he sensi i e ile o coun ead-only ansac ions.
3.5.5 O he policy idioms
Many o he common policies can be exp essed in Gua da . Examples include:
(a) Role-based policies whe e access depends on he clien ’s ole in an o ganiza ion
(ce i ica es can ela e clien s o oles), (b) Blacklis (whi elis ) policies whe e access
is denied (allowed) i he clien ’s iden i y exis s in a so ed ile ( he ile’s so edness
can also be en o ced using Gua da policies), and (c) His o y-based policies whe e
access depends on pas e en s ha a e isible o Gua da . The la e can be en o ced
by eco ding e en s in a dedica ed log ile and allowing access o he da a ile only
when he log ile is in ce ain s a es. The MAL policy is a simple his o y-based
policy ha allows access only when he e en o c ea ing an app op ia e log en y
has occu ed.
3.5.6 Exp essi eness
As hese examples demons a e, he Gua da policy language is exp essi e. I
can exp ess con en -based policies like MAL ha p io wo k on decla a i e policy
38
languages canno . Howe e , he language has limi a ions. I disallows ecu si ely-
de ined p edica es and, hence, canno exp ess layou s de ined by i e a ion o ecu sion,
e.g., i canno exp ess ha he con en o a ile be well- o med XML. Such cons ain s
may be checked by a us ed ex e nal e i ie using ce i ica es o communica e
be ween he e i ie and Gua da , o by ex ending he language wi h ecu si e
p edica es.
3.6 Implemen a ion
This sec ion desc ibes he p o o ype implemen a ion o Gua da in a SAN se e
and p esen implemen a ion al e na i es o he Gua da design.
3.6.1 P o o ype
Ou p o o ype is based on he iSCSI En e p ise Ta ge (IET) SAN se e , which
implemen s he se e -side iSCSI p o ocol and p o ides SCSI block s o age access
ia E he ne . IET is in p oduc ion use and a ailable o many Linux dis ibu ions.
Figu e 3.1 depic s he componen le el design. The se e accesses an SSD o he
Gua da me ada a and one o mo e payload disks which a e ei he magne ic- o
lash-based. IET consis s o a ke nel module, which implemen s block accesses, and
a use -le el daemon p ocess, which implemen s iSCSI managemen unc ions. To
implemen Gua da , we ex ended he ke nel module and added a second use -le el
daemon, which implemen s he Gua da in e ace and e alua es policies. The ke nel
module pe o ms upcalls o de e mine i iSCSI block accesses should be allowed. The
se e is con igu ed wi h a small SSD o s o ing Gua da me ada a, as well as one
o mo e magne ic disks o SSDs o he payload da a.
The Gua da daemon main ains wo B- ee index s uc u es on he me ada a
SSD: a block- o- ile index o ind he ile and policy associa ed wi h a gi en block
numbe , and a name- o- ile index o e ie e he ile in o ma ion (se o ex en s,
39
Figu e 3.1: Gua da implemen a ion in a SAN se e
policy, e c.) gi en a ile id. Fo pe o mance, he Gua da daemon main ains a
w i e- h ough DRAM cache o B- ee nodes and policies, backed by he SSD. Upda es
a e pe sis ed on he SSD du ing a ansac ion commi .
When he ke nel module ecei es a block access eques , i passes he access ype
( ead/w i e) and loca ion (disk o se , leng h) o he mul i- h eaded Gua da daemon,
which consul s he block- o- ile index. I he block loca ion is no associa ed wi h
a policy-p o ec ed ile, he access is g an ed. O he wise, he daemon e alua es he
policy and e u ns he esul o he ke nel module. Fo ead eques s, he block ead
is scheduled while checking he pe mission o educe la ency. Du ing a w i e eques ,
he block w i e mus be de e ed un il he Gua da daemon g an s he pe mission.
To educe he numbe o upcalls and policy e alua ions, he ke nel module main-
ains a cache o p e ious policy e alua ion esul s o he o m
hex en , pe missionsi
.
To eed his cache, he Gua da daemon always e u ns he la ges ex en encompass-
ing he p esen ly eques ed block o which he same pe missions hold. The cache is
lushed when a policy changes. This op imiza ion a oids policy e-e alua ion and
sa es he communica ion cos be ween ke nel module and he Gua da daemon in
many cases.
Ou p o o ype’s a ack su ace consis s o he IET managemen in e ace, he
block-de ice in e ace, he Gua da in e ace ex ensions as well as he policy language.
40
Despi e he ela i ely la ge IET codebase, which includes a minimally con igu ed
Linux ke nel, he esul ing a ack su ace is likely o be signi ican ly smalle han
ha o he sys ems and applica ions buil on op o Gua da in mos cases. Ou
Gua da implemen a ion adds less han 20,000 LOC o he exis ing IET codebase,
plus he OpenSSL and glib lib a ies i elies on.
3.6.2 Implemen a ion al e na i es
Gua da can be implemen ed in di e en ways depending on he deploymen and
h ea model. The GDC can be implemen ed using he ollowing mechanisms:
(a)
In a SAN se e o use in a da a cen e , as desc ibed in he p e ious p o o ype
sec ion.
(b)
In eg a ed wi h he mic ocon olle o a hyb id disk o use in an indi idual
machine.
(c)
In a us le wi hin a i ual machine moni o o ope a ing sys em, isola ed
using us ed ha dwa e ea u es like In el SGX [63] o ARM T us Zone [12].
Table 3.3 lis s examples o deploymen scena ios, hei h ea models and us
assump ions. As desc ibed in he h ea model each implemen a ion mus p o ec
he GDC, me ada a and da a om unau ho ized physical access and unde ec ed
ampe ing
Implemen a ion (a) elies on physical p o ec ion, e.g., in a machine oom wi h
access only by us ed employees. A possible deploymen scena io a a Cloud p o ide
p o ec s use da a om bugs and miscon igu a ions in i s in as uc u e and om
oppo unis ic access by employees. The use mus us he Cloud p o ide o p e en
physical access o he SAN se e by all bu us ed employees.
In implemen a ion (b), he GDC is implemen ed as pa o a mic ocon olle
embedded in a hyb id disk. He e, he me ada a and da a a e enc yp ed and au-
41
0.5
1
1.5
2
2.5
3
3.5
4
RR LR SR RW LW SW
Rela i e o e head
Wo kload
iSCSI
Gua da emp y
Gua da ile
Gua da policy
Figu e 3.3: La ency wi h an SSD, ela i e o iSCSI
Figu e 3.3 shows he esul ing a e age access la ency wi h he SSD, ela i e o
he plain iSCSI. E en wi h he as SSD as a block s o e de ice, he Gua da la ency
o e head is gene ally low, bu signi ican o andom w i es (2- old inc ease). The
ac ha ou block s o e SSD pe o ms andom w i es much as e han andom
eads (153
µ
s e sus 233
µ
s), p esumably due o w i e bu e ing in i s in e nal DRAM,
combined wi h he ac ha he policy check canno be o e lapped wi h he access
du ing a w i e, con ibu es o his high ela i e o e head.
No e ha he andom access wo kload is ex eme: The SSD block s o e de ice
is e y as , we a e measu ing he la ency o iny accesses (512 by es) a andom
loca ions o e he en i e disk, and he e a e many iles and policies. Inc easing he
eques size educes he o e head. Fo example, wi h a 4K eques size, he o e heads
dec ease om 29.3% o
RR
and 101.6% o
RW
o 17.7% and 96.1%, espec i ely.
Wi h 128K eques s, he o e heads go u he down o 0.9% and 23.5%, espec i ely.
Mo eo e , as we show nex , e en unde his wo kload he SSD e ains much o i s
la ency ad an age o e he HDD wi h Gua da , and Gua da ’s h oughpu o e head
is e y low on bo h he SSD and he HDD.
Figu es 3.4 and 3.5 compa e he absolu e la encies achie ed on a HDD and SSD
wi h and wi hou Gua da . Despi e Gua da ’s la ge ela i e o e heads o pu ely
andom w i es, he SSD e ains i s owe ing la ency ad an age on such accesses o e
48
0
0.1
0.2
0.3
0.4
0.5
0.6
RR LR SR RW LW SW
Response ime (ms)
Wo kload
iSCSI SSD Gua da policy SSD
Figu e 3.4: Absolu e la ency wi h SSD
0.01
0.1
1
10
RR LR SR RW LW SW
Log. esponse ime (ms)
Wo kload
iSCSI HDD Gua da policy HDD
Figu e 3.5: Absolu e la ency wi h HDD
he HDD (no e ha he y-axis is di e en o SSD and HDD). Wi h he magne ic
HDD, he Gua da la ency o e heads o all con igu a ions a e negligible (below 1%).
Compa ed o a locally a ached SSD, he a e age la ency o a emo ely connec ed
iSCSI SSD inc eases by 0.051 ms, a li le mo e han one ne wo k ound ip (0.047
ms).
3.7.2.2 Read/w i e h oughpu
Nex we examine he ead/w i e h oughpu o he Gua da p o o ype, using he same
con igu a ions as he la ency expe imen . The es clien issues ou 128KB eques s
concu en ly, which is su icien o achie e maximal ead and w i e h oughpu in
49
0
100
200
300
400
500
600
RR LR SR RW LW SW
Th oughpu (MB/s)
Wo kload
iSCSI
Gua da emp y
Gua da ile
Gua da policy
Figu e 3.6: SSD I/O h oughpu
he baseline iSCSI in all cases. Fo each access pa e n in each con igu a ion, we un
he h oughpu es 5 imes; each un issues a o al o 20,000 accesses and s a s a
a andom block wi hin he disk.
Figu e 3.6 shows he absolu e h oughpu wi h he SSD. The esul s shown
a e he a e ages o 5 uns, whe e e o ba s indica e he s anda d de ia ion. The
Gua da o e head is below 2% o all access pa e ns wi h he SSD. Wi h he HDD,
he o e heads a e in he same ange.
The high la ency o e head on andom w i es does no signi ican ly a ec he
h oughpu because policy e alua ion o di e en eques s can be pe o med in
pa allel by he mul i- h eaded Gua da daemon, and o e lapped wi h disk and SSD
accesses o me ada a and blocks.
Mo eo e , compa ed o a locally a ached SSD, he h oughpu o e head is a
mos 3% o all iSCSI and Gua da con igu a ions and wo kloads.
3.7.2.3 I/O pe o mance summa y
While Gua da adds li le la ency o HDD accesses and SSD accesses wi h good
locali y, i has a no iceable la ency o e head on small, pu ely andom w i es o
an SSD. Howe e , his o e head diminishes quickly wi h la ge eques sizes and
50
Policy size Domain size
1 2 4 8 16
12.2 3.4 5.8 10.7 20.4
24.6 10.4 28.9 95.1 345.8
37.0 24.0 121.2 770.5 5,518.1
49.4 50.9 485.3 6,156.4 88,319.3
511.9 104.9 1,951.3 49,234.7 1,411,800.8
Table 3.4:
E alua ion la ency in
µ
s o a ying policy size (numbe o p edica es
and a iables in he policy) and domain size (maximum numbe o cache en ies)
mo e locali y, and can be o e lapped wi h concu en accesses, so ha he SSD’s
h oughpu is no a ec ed.
3.7.2.4 Policy e alua ion o e head
Consis en wi h Da alog, he heo e ical wo s -case e alua ion ime o a policy
ule is in
O
(
m·Dn
), whe e
m
is he size o he ule (numbe o p edica es),
D
is
he size o he domain (bounded by he size o he Gua da cache) and
n
is he
numbe o a iables in he ule. In Table 3.4, we show he measu ed policy e alua ion
ime o syn he ic policies designed o ex ica e wo s -case execu ion om ou policy
in e p e e .
D
a ies along columns o he able and
m
and
n
a y along ows (
m
=
n
in all expe imen s). The esul s ma ch he expec ed complexi y
O
(
m·Dn
). The
able indica es (co ec ly) ha policy e alua ion could be a subs an ial bo leneck o
some policies bu we do no obse e his bo leneck in p ac ice. The a e age policy
e alua ion la ency o he mos complex policy e alua ed, MAL (Sec ion 3.7.5) is only
27.7
µ
s, e en hough he policy has
m
= 4,
n
= 4 and
D
= 40. This is because o a
ca e ul implemen a ion o he policy in e p e e o conside mo e ecen cache en ies
i s . Ou o he
example
policies e alua e e en as e ; he a e age e alua ion ime o
he ime-based policy om he la ency expe imen con igu a ion
Gua da policy
is
only 3.7µs.
51
3.7.2.5 Space equi emen s o me ada a
We quan i y he me ada a s o age equi emen s. Because he me ada a size depends
on he s uc u e o he payload da a, we analyzed he me ada a space equi emen s
o 70,825 ilesys em snapsho s collec ed by Ag awal e al. [
2
]. The snapsho s we e
aken om Windows sys ems wi hin Mic oso co po a ion be ween 2000 and 2004,
and con ain be ween 30k and 90k iles each wi h an a e age ile size be ween 108KB
and 189KB. Fo e alua ion pu poses, we gi e each ile in each snapsho an in eg i y
policy ha disallows modi ica ion p io o a gi en da e. The snapsho s a e mo e
han 10 yea s old a he ime o his w i ing. Because he a e age ile size in
oday’s sys ems has likely inc eased, howe e , ou analysis o Gua da ’s me ada a
equi emen s ela i e o he size o he da a is conse a i e.
The equi ed me ada a can be accommoda ed in a solid s a e memo y o 0.8% o
he da a size o 99.89% o he snapsho s. As a poin o e e ence, e en comme cially
a ailable hyb id disks p o ide a leas 0.8% Flash [
112
] a he ime o his w i ing.
Newe combina ions o Flash/disk de ices achie e much highe Flash o disk capaci y
a ios and his end is p ojec ed o con inue gi en he p ice and space educ ion
a es o lash memo y. Fo example, Apple’s Fusion D i e [
10
] has a a io o 128GB
Flash o a 1TB HDD, which can easily accommoda e all he snapsho s. In all ou
expe imen s, which use o he da a se s, he me ada a i in o only 0.2% o he da a
size.
3.7.2.6 Flash memo y wea
Because Flash memo y can endu e only a limi ed numbe o e ase/p og am cycles,
we mus check ha he SSD used o s o e me ada a will no wea quickly. To be
conse a i e, we assume ha he Flash mus las a leas 10 yea s. The li e ime is
in luenced by he size o he me ada a, he a e o me ada a upda es, and he Flash
capaci y. A smalle capaci y causes he Flash log o w ap a ound as e and leads o
52
0
50
100
150
200
250
300
iozone( /w) bonnie++( /w)
Pe o mance (MB/s)
iSCSI SSD
Gua da policy SSD
iSCSI HDD
Gua da policy HDD
Figu e 3.7: FS benchma ks ead and w i e ( /w) pe o mance
highe u iliza ion, which in u n educes cleaning e iciency and equi es e en mo e
Flash w i es.
Unde he con igu a ion o
Gua da policy
used abo e, we keep ack o how
much wea he Flash expe iences while p esen ed wi h a se ies o me ada a upda es,
i.e., adding and emo ing ex en s o a con en ile picked a andom. En e p ise
en i onmen s ypically deploy single-le el cell (SLC) Flash memo y, which has a
nominal li e ime o 100,000 e ase/p og am cycles. Using only 4GB o such memo y
we can accommoda e up o 19.5M upda es pe day (225 pe second). This is an
ex ao dina ily high upda e a e ha can accommoda e e en he mos w i e-in ensi e
applica ions. Cheape mul iple-le el cell (MLC) and iple-le el cell (TLC) Flash
memo y wi h nominal li e imes o 10,000 and 1,000 e ase/p og am cycles would
suppo up o 1.95M and 195,000 me ada a upda es pe day, espec i ely.
3.7.3 Filesys em benchma ks
Nex , we measu e he pe o mance o he Gua da p o o ype using he s anda d
ilesys em benchma ks
iozone
3.429 and
Bonnie++
1.03. The block s o e was
o ma ed unde ex 4.
iozone
uses ou wo ke h eads o w i e 1GB sequen ially
53
o ou sepa a e iles.
1
La e , each wo ke pe o ms a sequen ial ead o he ile
hey p e iously w o e. Simila ly,
Bonnie++
w i es hen eads 1GB each o 16
iles. Figu e 3.7 shows he pe o mance o he baseline and Gua da unde he
Gua da policy
con igu a ion. The esul s shown a e he a e ages o 5 uns and
e o ba s indica e he s anda d de ia ion. The Gua da o e heads a e below 1.0%
o bo h benchma ks on bo h he HDD and he SSD. No e ha
Bonnie++
uses he
C lib a y unc ions ge c and pu c o pe o m ile eads and w i es, and is he e o e
unable o sa u a e he disks.
Simila o he h oughpu expe imen , he iSCSI SSD esul s a e close o hose
achie ed wi h a locally a ached SSD (a mos 3.5% lowe ).
3.7.4 Use case: Web se e
Nex , we conside he pe o mance o he Gua da p o o ype as pa o a modi ied
Apache Web se e . The se e holds a 220GB s a ic snapsho o English language
Wikipedia a icles om 2008 [
137
] and Wikimedia images om 2005 [
136
], con aining
15 million iles wi h an a e age ile size o 15KB and maximum ile size o
∼
500KB.
The HTTP clien asynch onously eques s HTML pages om he Web se e , using
a wo kload based on he ac ual access coun s o Wikipedia pages du ing one hou on
Ap il 1, 2012 [
138
]. Because ou snapsho is much olde and had ewe a icles a
he ime, we igno e accesses o non-exis ing pages. In o al, abou 350,000 di e en
pages we e accessed in he ace, o which 250,000 a e pa o he 2008 snapsho .
Since we do no ha e access o ime s amps, we dis ibu ed he indi idual accesses
e enly wi hin an hou , and eplayed he i s 100,000 page eques s.
We use he ollowing Gua da policies o p o ec he se e ’s pe sis en s a e:
Con en :
Requi e con en upda es signed by owne s. We andomly assign one o
40,000 owne s o each con en ile.
1We used he command iozone -i 0 -i 1 - 512k -I -c -e -T - 4 -s 1g -F iles
54
200
250
0 10 20 30 40 50 60 70 80 90
Th oughpu (Reques s/s)
Numbe o concu en HTTP eques s
iSCSI
Gua da
Figu e 3.8: Web se e h oughpu
Execu ables/Con ig:
Requi e ha upda es o execu able and con igu a ion iles
be signed by he adminis a o .
Log iles:
The Apache log iles can only be appended, excep wi h an adminis a o
key used o o a e he log.
To sa is y he log ile policy, we added a o al o 51 lines o code o Apache. This
ex a code issues Gua da commands o send con en hashes o Gua da and lush
applica ion and ilesys em caches ( lush & sync) be o e e e y log ile upda e. The
policies p o ec ing he con en , execu ables and con igu a ion iles do no equi e any
modi ica ions o Apache.
Figu e 3.8 shows he a e age h oughpu o h ee uns as a unc ion o he numbe
o concu en HTTP accesses, o plain iSCSI and Gua da (s anda d de ia ion is
below 0.5%). Each un loads 100,000 Wikipedia pages. The h oughpu o e head o
he Gua da con igu a ion o e he unmodi ied iSCSI se e is 1.95% a 60 concu en
eques s, whe e iSCSI eaches i s peak h oughpu , and always wi hin 2.7%. This
esul shows ha he Gua da o e heads mos ly o e lap wi h o he ac i i ies in he
Web se e . The 100,000 page eques s esul in app oxima ely 350,000 Gua da
eads, o an a e age o 3.5 eads pe page. This shows ha a subs an ial numbe
o eads each he Gua da de ice and a e no abso bed by he ilesys em bu e
55
cache. In addi ion, Apache w i es 2.7MB o log eco ds in 170 ansac ions unde he
append-only policy. The e a e no upda es o con en , execu ables and con igu a ion
iles, no log o a ions in he wo kload, bu policies mus s ill be checked du ing each
access.
In e ms o unc ionali y, Gua da p o ec s con en , logs, con igu a ion and
execu able iles om ampe ing by unau ho ized pa ies, which we con i med h ough
aul injec ion expe imen s.
3.7.5 Manda o y access logging
In ou inal expe imen , we pe o m accesses o a ile wi h ou manda o y access
logging (MAL) policy. The policy equi es an app op ia e en y in a sepa a e log
ile o an access o be allowed by Gua da . We use a 64MB p ima y ile wi h o
wi hou he MAL policy in place. The p ima y ile and he log ile eside on di e en
HDDs a ached o he same Gua da IET se e . The e sion coun e embedded in
he p ima y ile is s o ed in Flash memo y no used by Gua da . The clien connec s
o he Gua da de ice and accesses he p ima y ile in h ee di e en con igu a ions.
no log: File accessed wi hou any logging and en o cemen . (ho izon al lines)
log: Accesses logged wi hou policy en o cemen .
Gua da MAL: Accesses logged and policy en o ced by Gua da .
Figu e 3.9 shows he a e age access la ency o 100,000 sequen ial 4KB eads and
w i es o he p ima y ile, a ying he numbe o accesses pe eco ded log en y om
1 o 512. E o ba s indica e he s anda d de ia ion. In he case o a single access
pe log en y, en o cing he MAL policy inc eases he ead/w i e la ency by 11.5%
and 50.6%, espec i ely, o e olun a y logging. The highe cos o logged w i es
compa ed o eads e lec s he need o upda e he e sion numbe . Bo h cos s can
be educed by issuing e sion coun e upda es, log w i es, and p ima y ile accesses
56
0.2
0.4
0.6
0.8
1
1.2
1.4
1 2 4 8 32 128 512
Access la ency (ms)
Accesses pe log en y
Read log
Read Gua da MAL
W i e log
W i e Gua da MAL
ead no log
w i e no log
Figu e 3.9: La ency wi h MAL, olun a y and no logging
in pa allel. Mo eo e , as shown in he igu e, he cos o MAL can be amo ized by
logging se e al accesses in a single log en y, and app oaches he cos o comple ely
unlogged accesses o 512 accesses pe log en y.
3.8 Rela ed wo k
Policy languages based on Da alog.
Many decla a i e policy language a e based
on Da alog and esemble he Gua da policy language in syn ax and seman ics. Some
examples a e Sou ei [
100
], Binde [
33
] and SecPAL [
18
]. Whe eas hese languages
a e gene ic, he Gua da policy language is domain-speci ic and con ains cus om-
designed, s o age- ele an p edica es (Sec ion 3.4). Sou ei, Binde and SecPAL allow
in ensional ( ecu si e, ule-de ined) p edica es, which he Gua da policy language
omi s o keep he implemen a ion simple. These p edica es can be added o Gua da
wi hou any concep ual challenges. DKAL [
52
] ex ends Da alog wi h decla a i e
ules o exchanging au ho iza ion c eden ials in dis ibu ed sys ems. Such ules can
be added o Gua da as well.
TCG s o age wo k g oup speci ica ion.
Al hough de eloped independen ly, he
Gua da a chi ec u e bea s some esemblance o s o age wo k g oup s anda ds o
he us ed compu ing g oup (TCG) [
127
]. Simila o Gua da , he TCG s anda d
57
is easy o deploy and amenable o an e icien implemen a ion, as demons a ed by
ou expe imen al e alua ion.
64
CHAPTER 4
ERIM: Secu e and E icien
In-p ocess Isola ion
The p e ious chap e desc ibed he design, implemen a ion and e alua ion o
Gua da , a s o age laye e e ence moni o en o cing con iden iali y, in eg i y and
accoun ing policies o e pe sis en da a. I en o ces hese policies independen o
highe abs ac ion laye s p o iding a s ong h ea model elying on a small TCB
and a ack su ace.
Howe e , he se o en o ceable policies a e limi ed by he obse able e en s
a he s o age laye . Gua da has no con ol o e da a eleased o an applica ion
wi h su icien access c eden ials. As a esul s an applica ion may leak da a o e he
ne wo k, due o bugs, malicious a acks o miscon igu a ions.
In con as o Gua da , ERIM media es an un us ed applica ion’s execu ion,
in e cep ing ele an applica ion ope a ions like accesses o p i a e da a o ope a ing
sys em se ices. To media e un us ed applica ions, ERIM pa i ions sensi i e da a
and code in o an isola ed and us ed componen , he eby limi ing he e ec s o bugs
and ulne abili ies in he un us ed componen o da a accessible in he un us ed
applica ion only. Fo ins ance, isola ing c yp og aphic keys om he emaining
applica ion can hwa ulne abili ies like he OpenSSL Hea bleed bug [
90
]; isola ing
jump ables can p e en a acks on he in eg i y o an applica ion’s con ol low; and
isola ing a managed language’s un ime can p o ec i s secu i y in a ian s om bugs
and ulne abili ies in co-linked na i e lib a ies.
65
Isola ion o emos equi es memo y isola ion, which p e en s an un us ed com-
ponen om di ec ly accessing he p i a e memo y o o he componen s. B oadly
speaking, memo y isola ion can be en o ced using one o wo app oaches. Fi s ,
we may ins umen he code o un us ed componen s wi h bounds checks p io o
indi ec memo y accesses, ensu ing ha memo y o o he componen s is no accessed
di ec ly, as in SFI [
129
]. Howe e , his app oach imposes o e head on all execu ion o
un us ed componen s due o bounds checks, and i equi es an addi ional echnique
o p e en ci cum en ion o he bounds checks in he ace o con ol- low hijacks [
68
].
The o al o e head is commonly o he o de o ens o pe cen poin s.
The second app oach is o use ha dwa e suppo o memo y isola ion such as
OS o hype iso (ex ended) page ables [
20
,
29
,
74
,
19
]. He e, as access checks in
ha dwa e p e en a componen om accessing he memo y o o he componen s, bu
he e is an o e head on swi ches be ween componen s, since ha dwa e p i ileges mus
be changed, e.g., by swi ching page ables and possibly in alida ing TLB en ies.
Recen wo k on in-p ocess isola ion such as Wedge [
20
], Sh eds [
29
], and ligh -weigh
con ex s (lwCs) [
74
] has educed he cos o ha dwa e-based isola ion somewha .
None heless, swi ching s ill equi es a sys em call and i s cos is signi ican (a 1 us
pe swi ch [
74
] a conse a i e swi ch a e o 100,000 imes a seconds amoun s o 10%
o e head).1
Consequen ly, he e is need o an isola ion echnique ha does no impose
con inuous o e head while a componen execu es and ha also has e y low swi ching
cos on componen ansi ion. ERIM achie es his goal by building on a ecen x86
ISA ex ension called memo y p o ec ion keys o MPKs, also simply called p o ec ion
keys [
64
]. MPKs allow agging each page wi h one o 16 domains, hus pa i ioning
a p ocess’ add ess space in o disjoin domains. A special pe -co e egis e , PKRU,
1
Using x86 memo y segmen a ion ins ead o page ables, as in Na i e Clien [
143
], can educe
he swi ch cos . Howe e , suppo o segmen a ion wi h 64-bi add essing is limi ed and Na i e
Clien has been dep eca ed in a o o he memo y-sa e language WebAssembly [53].
66
de e mines which domains a e accessible. Swi ching pe missions equi es only w i ing
he PKRU egis e wi h a use -mode ins uc ion, which is a ela i ely quick ope a ion
(11–260 cycles on cu en In el CPUs in ou expe imen s).
Howe e , since he PKRU-upda e ins uc ion is use -mode, MPK by i sel
in insu icien o secu i y: Comp omised o malicious componen s can execu e
he ins uc ion o gain unau ho ized access o he memo y o o he componen s.
To p e en his, ERIM addi ionally elies on bina y inspec ion o ensu e ha all
occu ences o his ins uc ion (called WRPKRU) in he bina y a e sa e, i.e., hey
canno be exploi ed o gain unau ho ized access. By design, his p ope y holds e en
i he e is a con ol- low hijack in he un us ed componen . Hence, he e is no need
o complemen ERIM wi h con ol- low in eg i y, which would add o e head.
ERIM dis inguishes i sel om p io wo k on applica ions ha ha e e y high
swi ching a es (~10
5
/s o mo e) and ha addi ionally spend a non i ial amoun o
ime in un us ed componen s. The e a e many such applica ions. We e alua e ou
p o o ype o ERIM on h ee such applica ions. Fi s , in he web se e nginx, we
show ha ERIM can isola e session keys. P o ec ing session keys is a meaning ul
goal, since a acks a ge ed a indi idual use s’ p i acy only need o comp omise
session keys. Second, we show ha ERIM can e icien ly isola e he sa e egion in
code-poin e in eg i y [
72
]. Thi d, we show ha ERIM can be used o isola e a
managed language un ime om possibly buggy na i e lib a ies. In all cases, we
obse e swi ching a es o o de s a leas 10
5
imes/s pe co e. ERIM p o ides s ong
ha dwa e isola ion wi h o e heads less han 1% o e e y 100,000 swi ches/s, which
is conside ably lowe han ha o exis ing echniques.
The ollowing sec ions desc ibe he design, h ee use cases, a p o o ype imple-
men a ion, ela ed wo k and he e alua ion using h ee use cases.
67
4.1 Design
Like p io wo k, ERIM enables a us ed applica ion componen o isola e sensi i e
da a om he es o he un us ed applica ion. Unlike p io wo k, ERIM suppo s
such isola ion wi h low o e head e en a high swi ching a es be ween he un us ed
applica ion and he us ed componen , and wi hou elying on any o he (possibly
expensi e) p o ec ion mechanism, e.g., con ol- low in eg i y. By way o example,
he us ed componen may be a c yp o lib a y ha wan s o isola e c yp og aphic
keys, an inlined e e ence moni o ha wan s o isola e sensi i e me a da a (such as
a ain map o jump ables), o i may be a managed language un ime ha wan s
o isola e om a buggy na i e lib a y. We use he le e
T
o deno e he us ed
componen and U o deno e he emaining un us ed applica ion.
The main p imi i e ERIM p o ides is memo y isola ion—i ese es a egion o
he add ess space accessible exclusi ely om he us ed componen
T
. This ese ed
egion is deno ed
MT
and i can be used by
T
o s o e sensi i e da a. The es o
he add ess space, deno ed
MU
, holds he applica ion’s egula heap and s ack and
is accessible om bo h
U
and
T
. ERIM p e en s
U
om ha ing di ec access o
MT
;
access o
MT
is enabled a omically wi h a con ol ans e o designa ed en y poin s
in
T
, and disabled when
T
e u ns con ol o
U
. Mo e p ecisely, ERIM en o ces he
ollowing in a ian s:
(1) While con ol is in U, access o MT emains disabled.
(2)
Access o
MT
is enabled a omically wi h a con ol ans e o a designa ed en y
poin in T and disabled when T ans e s con ol back o U.
The i s in a ian p o ides isola ion o
MT
om
U
, while he second in a ian
p e en s
U
om con using
T
in o accessing
MT
imp ope ly by jumping in o he
middle o
MT
’s code. Due o he second in a ian , ERIM does no need suppo
om a solu ion o con ol- low in eg i y o secu i y.
68
Con ol ans e s om
U
o
T
and back, wi h he co esponding enabling and
disabling o access o
MT
a e acili a ed by special sequences o ERIM-p o ided code,
dubbed call ga es. Call ga es a e implemen ed in a manne ha p e en s exploi a ion
o hei bina y code o ele a ing p i ileges.
A call ga e enables access o
MT
, execu es a speci ied en y poin o
T
, hen
disables access o
MT
when ans e ing con ol o he us ed componen and
disables access on he way back. A call ga e ans e s con ol only o a designa ed
en y poin in he us ed componen . This en y poin may be a unc ion (i he
us ed componen is a lib a y) o a speci ic sequence o ins uc ions (i he us ed
componen is inlined in o he applica ion).
By design, ERIM imposes negligible o e head on he execu ion o code wi hin
he un us ed applica ion and wi hin he us ed componen , and i s call ga es a e
e y as . Addi ionally, ERIM’s isola ion is s ong—i is de i ed di ec ly om a
ha dwa e secu i y ea u e and i is absolu e, no p obabilis ic (unlike add ess space
layou andomiza ion (ASLR)). Bo h, as swi ching and he s ong isola ion, make
ERIM sui able o p o ec sensi i e da a in high-pe o mance applica ions, e en hose
ha swi ch be ween he applica ion and he lib a y e y equen ly.
4.1.1 Th ea model
ERIM makes no assump ions abou he un us ed componen (
U
) o an applica ion.
U
may beha e a bi a ily and may con ain memo y co up ion and con ol- low
hijack ulne abili ies ha may be exploi ed du ing i s execu ion.
Howe e , ERIM assumes ha he us ed componen
T
’s bina y does no ha e
such ulne abili ies and does no comp omise sensi i e da a by calling back in o
U
while access o
MT
is enabled, h ough in o ma ion leaks, o by mapping execu able
pages wi h unsa e/exploi able occu ences o he WRPKRU ins uc ion.
69
The ha dwa e, he OS ke nel, and a small lib a y added by ERIM o each p ocess
ha uses ERIM a e us ed o be secu e. We also assume ha he ke nel en o ces
s anda d DEP—an execu able page mus no be simul aneously mapped wi h w i e
pe missions. ERIM elies on a lis o legi ima e en y poin s in o
T
p o ided ei he
by he p og amme o he compile , and his lis is assumed o be co ec (see
Sec ion 4.1.5). The OS’s dynamic p og am loade /linke is us ed o in oke ERIM’s
ini ializa ion unc ion be o e any o he code in a new p ocess.
Side-channel and owhamme a acks, and mic oachi ec u al leaks, al hough
impo an , a e beyond he scope o his wo k. Howe e , ERIM is compa ible wi h
exis ing de enses.
4.1.2 In el Memo y P o ec ion Keys (MPK)
To ealize i s goals, ERIM uses he ecen MPK ex ension o he x86 ISA [
64
].
MPK allows associa ing one o 16 p o ec ion keys wi h each memo y page, hus
pa i ioning he add ess space in o up o 16 domains. A pe -co e egis e , called
PKRU, de e mines he cu en access pe missions ( ead, w i e, nei he o bo h) on
each domain o he code unning on ha co e. Access checks agains he PKRU a e
implemen ed in ha dwa e and impose no o e head on p og am execu ion.
Changing access p i ileges equi es w i ing new pe missions o he PKRU egis e
wi h a use -mode ins uc ion, WRPKRU. This ins uc ion is ela i ely as (11–260
cycles on cu en In el CPUs), does no equi e a syscall, changes o page ables, a
TLB lush, o in e -co e synch oniza ion.
Since WRPKRU can be execu ed in use -mode, un us ed code can execu e i
a any poin o ele a e p i ileges and MPK canno p o ide any memo y secu i y
agains un us ed applica ion code by i sel . To ge his p o ec ion, ERIM combines
MPK wi h addi ional bina y inspec ion o ensu e ha any WRPKRU occu ences
70
on execu able pages a e sa e, i.e., hey canno be exploi ed o imp ope ly ele a e
p i ilege.
The mains eam Linux ke nel ully suppo s page- able en ies agged wi h MPK
domains, syscalls o ag he en ies wi h speci ic domains and es o es PKRU egis e s
upon con ex swi ches. Since ha dwa e PKRU checks a e disabled in ke nel mode, he
ke nel has also been modi ied o check PKRU pe missions explici ly be o e accessing
any use space poin e . To elimina e he isk o signal handle s ele a ing p i ileges,
he ke nel upda es he PKRU egis e o i s ini ial se o p i ileges (only ead/w i e
access o domain 0) be o e h owing a signal o he use space.
4.1.3 High-le el o e iew o he design
ERIM can be con igu ed o p o ide ei he comple e isola ion o
MT
om
U
(con-
iden iali y and in eg i y), o only w i e p o ec ion (only in eg i y). Fo simplici y,
we desc ibe he design o comple e isola ion i s . Sec ion 4.1.7 desc ibes how o
con igu e ERIM sligh ly di e en ly o p o ide w i e p o ec ion only.
ERIM’s isola ion mechanism is concep ually simple: I maps
T
’s ese ed memo y,
MT
, and he applica ion’s gene al memo y,
MU
, o wo di e en MPK domains.
I manages MPK pe missions ( he pe -co e PKRU egis e s) o ensu e ha
MU
is
always accessible, while
MT
is ne e accessible when con ol is in
U
. I allows
U
o
secu ely ans e con ol o
T
and back ia call ga es. A call ga e enables access o
MT
using he WRPKRU ins uc ion and immedia ely ans e s con ol o a speci ied
en y poin o
T
, which may be an explici o inlined unc ion. When
T
is done
execu ing, he call ga e disables access o
MT
and e u ns con ol o
U
. This en o ces
ERIM’s wo in a ian s (1) and (2) om Sec ion 4.1. Call ga es ope a e en i ely in
use -mode ( hey don’ use syscalls) and a e desc ibed in Sec ion 4.1.4.
P e en ing WRPKRU exploi a ion
A key di icul y in ERIM’s design is p e-
en ing he un us ed
U
om exploi ing WRPKRU ins uc ions on execu able pages
71
in he add ess space o ele a e p i ileges, e.g. using con ol- low hijack o code-
injec ion a acks. To p e en such exploi s, ERIM elies on bina y inspec ion o
en o ce he in a ian ha only sa e WRPKRU occu ences appea on execu able
pages. A WRPKRU occu ence is sa e i i is immedia ely ollowed by one o he
ollowing:
(A) A p e-designa ed en y poin o T.
(B)
A speci ic sequence o ins uc ions ha checks ha he pe missions se by he
WRPKRU do no include access o
MT
and e mina es he p og am o he wise.
A sa e WRPKRU occu ence canno be exploi ed o execu e un us ed code wi h
access o
MT
. I he occu ence sa is ies (A), hen i does no gi e con ol o
U
a all;
ins ead, i en e s
T
a a designa ed en y poin . I he occu ence sa is ies (B), hen
i would e mina e he p og am immedia ely we e i used by a con ol- low hijack o
enable access o MT.
ERIM’s call ga es use only sa e WRPKRU occu ences and, he e o e, pass
ou bina y inspec ion. Ou modi ied ke nel inspec s any page p io o mapping
i in execu able mode, en o cing he in a ian ha all occu ences o WRPKRU
on execu able pages a e sa e. Sec ion 4.1.5 p o ides de ails o his ke nel bina y
inspec ion mechanism.
C ea ing sa e bina ies
An impo an ques ion is how o cons uc bina ies ha
do no ha e unsa e WRPKRUs. On x86, an inad e en o unin ended execu able
WRPKRU may a ise spanning he by es o wo adjacen ins uc ions o as a subse-
quence in a longe ins uc ion. To elimina e inad e en WRPKRUs, we de elop a
bina y ew i ing mechanism ha ew i es any sequence o ins uc ions con aining an
inad e en WRPKRU o a unc ionally equi alen sequence wi hou any WRPKRUs.
Simila ly, he mechanism also al e s delibe a e uses o WRPKRU which olun a ily
swi ch domains by inse ing p i ilege checks. The mechanism can be deployed as
72
1xo ecx , ecx
2xo edx , edx
3mo PKRU_ALLOW_TRUSTED, eax
4WRPKRU // cop ies eax o PKRU
6// Execu e us ed componen ’ s code
8xo ecx , ecx
9xo edx , edx
10mo PKRU_DISALLOW_TRUSTED, eax
11WRPKRU // cop ies eax o PKRU
12cmp PKRU_DISALLOW_TRUSTED, eax
13j e con inue
14s y s c a l l exi // e mina e p og am
15con inue :
16// co n ol e u ns o he un us ed a pplic a io n he e
Lis ing 4.1:
Call ga e implemen a ion in assembly. The code o he us ed
componen ’s en y poin may be inlined by he compile on line 6, o he e may be
an explici di ec call o i .
a compile pass, in eg a ed wi h ou bina y inspec ion, o by s a ically ew i ing
bina ies p io o hei use as explained in Sec ion 4.2
4.1.4 Call ga es
A call ga e ans e s con ol om
U
o
T
, enabling access o
MT
, hen uns code
om a designa ed en y poin o
T
, and la e e u ns con ol o
U
a e disabling
access o
MT
. This equi es wo WRPKRUs. The p ima y challenge in designing
he call ga e is ensu ing ha bo h hese WRPKRUs a e sa e in he sense explained
in Sec ion 4.1.3.
Lis ing 4.1 shows he assembly code o a call ga e. WRPKRU expec s he new
PKRU alue in he
eax
egis e and equi es ecx and edx o be 0. The call ga e
wo ks as ollows. Fi s , i se s PKRU o enable access o
MT
(lines 1–4). The mac o
PKRU_ALLOW_TRUSTED is a PKRU se ing ha allows access o
MT
. Nex ,
he call ga e passes con ol o he designa ed en y poin o
T
(line 6). The en y
poin ’s code may be in oked ei he by a di ec call, o i may be inlined he e.
73
he p esen alue o he PKRU is no PKRU_ALLOW_TRUSTED, indica ing ha
he syscall does no o igina e om
T
. To gain access o es ic ed esou ces,
U
has
o in oke T, which can ac as a e e ence moni o .
4.2 Rew i ing inad e en WRPKRUs
Fo secu i y, ou bina y inspec ion (see Sec ion 4.1.5) equi es bina ies o ha e only
sa e WRPKRU occu ences. WRPKRUs emi ed pu pose ully by a compile can
be made sa e by changing he compile sligh ly o inse he check on lines 12–15
o Figu e 4.1 a e e e y po en ially unsa e WRPKRU. Inad e en WRPKRUs—
hose ha occu unin en ionally as pa s o longe x86 ins uc ions o spanning wo
consecu i e x86 ins uc ions—a e mo e in e es ing. In his Sec ion, we desc ibe
a ew i e s a egy o elimina e such WRPKRUs. The s a egy is comple e: Any
sequence o x86 ins uc ions con aining an inad e en WRPKRU can be ew i en
o a unc ionally equi alen sequence wi hou any WRPKRUs.
4.2.1 Rew i e s a egy
WRPKRU is a 3 by e ins uc ion, 0x0F01EF. WRPKRU sequences ha span wo
o mo e ins uc ions can be “b oken” by inse ing a 1 by e nop like 0x90 be ween
any wo consecu i e ins uc ions. 0x90 does no coincide wi h any indi idual by e
o WRPKRU (0x0F, 0x01 and 0xEF), so his inse ion canno gene a e a new
WRPKRU.
A WRPKRU sequence ha lies en i ely wi hin a longe ins uc ion can be
elimina ed by inding an equi alen sequence o ins uc ions. Doing so sys ema ically
equi es unde s anding x86 ins uc ion coding. An x86 ins uc ion consis s o :
(i) An opcode ield possibly wi h p e ix.
80
(ii)
A MOD R/M ield ha de e mines he add essing mode and includes he code
o a egis e ope and.
(iii) An op ional SIB ield ha speci ies egis e s o indi ec memo y add essing.
(i )
Op ional displacemen and/o immedia e ields which speci y cons an o se s
o memo y ope a ions and o he cons an ope ands.
Ou s a egy o ew i ing an ins uc ion con aining WRPKRU as a subsequence
depends on he ields wi h which he WRPKRU subsequence o e laps. Table 4.1
summa izes ou s a egy. I he WRPKRU sequence lies en i ely in he opcode ield,
hen he ins uc ion is WRPKRU. As explained ea lie , his case is handled by
adding a check (B) a e he ins uc ion o make i sa e.
I he sequence o e laps wi h he MOD R/M ield, we change he egis e code in
he MOD R/M ield, which elimina es he WRPKRU sequence. This change equi es
a ee egis e . I one exis s, we use i , else we ew i e o push an exis ing egis e o
he s ack, use i in he ins uc ion, and pop i back. (Lines 2 and 3 in Table 4.1.)
I he sequence o e laps wi h he displacemen o he immedia e ield, we change
he mode o he ins uc ion o use a egis e ins ead o a cons an . The cons an
is compu ed in he egis e be o e he ins uc ion (lines 4 and 6). I a ee egis e
is una ailable, we push and pop one. Two ins uc ion-speci ic op imiza ions a e
possible. I he ins uc ion is jump-like, hen he jump a ge can be eloca ed in he
bina y; his changes he displacemen in he ins uc ion, elimina ing he need o a
ee egis e (line 5). I he ins uc ion is an associa i e ope a ion such as addi ion,
hen he ope a ion can be pe o med in wo inc emen s wi hou an ex a egis e
(line 7).
We ne e ew i e he SIB ield. This does no a ec he comple eness o ou
echnique since any WRPKRU mus o e lap wi h a leas one non-SIB ield ( he SIB
ield is 1 by e long while WRPKRU is 3 by es long).
81
O e lap wi h Cases Rew i e s a egy ID Example
Opcode Opcode = WRPKRU Inse p i ilege check a e WRPKRU 1
Mod R/M Mod R/M = 0x0F
Change o unused egis e + mo e com-
mand
2
add ecx, [ebx + 0x01EF0000]
→
mo eax, ebx; add ecx, [eax +
0x01EF0000];
Push/Pop used egis e + mo e com-
mand
3
add ecx, [ebx + 0x01EF0000]
→
push eax; mo eax, ebx; add ecx,
[eax + 0x01EF0000]; pop eax;
Displacemen Full/Pa ial sequence Change mode o use egis e 4
add eax, 0x0F01EF00
→
(push ebx;)
mo ebx, 0x0F010000; add ebx,
0x0000EA00; add eax, ebx; (pop
ebx;)
Jump-like ins uc ion
Mo e code segmen o al e cons an used
in add ess
5
call [ ip + 0x e 010 ]
→
call [ ip +
0x e 0100]
Immedia e Full/Pa ial sequence Change mode o use egis e 6
add eax, 0x0F01EF
→
(push ebx;)
mo ebx, 0x0F01EE00; add ebx,
0x00000100; add eax, ebx; (pop ebx;)
Associa i e opcode
Apply ins uc ion wice wi h di e en im-
media es o ge equi alen e ec
7
add ebx, 0x0F01EF00
→
add ebx,
0x0E01EF00; add ebx, 0x01000000
Table 4.1: Rew i e s a egy o in a-ins uc ion occu ences o WRPKRU
82
4.2.2 Implemen ing he ew i ing
Fo bina ies ha can be ( e)compiled om sou ce, ew i ing can be added o he
codegen phase o he compile , which con e s he in e media e ep esen a ion (IR)
o machine ins uc ions. Whene e codegen ou pu s an inad e en WRPKRU, he
su ounding ins uc ions in he IR can be eplaced wi h equi alen WRPKRU- ee
ins uc ions as desc ibed abo e, and codegen can be un again on he upda ed IR.
Fo bina ies ha canno be ecompiled, he ew i e s a egy can be in eg a ed
wi h ou bina y inspec ion handle (Sec ion 4.1.5). I he handle disco e s an unsa e
WRPKRU on an execu able page du ing i s scan, i can o e w i e he page wi h
1-by e ap ins uc ions, make i execu able, and s o e he o iginal page in ese e
wi hou enabling i o execu ion. Subsequen ly, i he e is a jump in o he execu able
page, a ap occu s and he ap handle disco e s an en y poin in o he page. I can
hen disassemble he ese ed page om ha en y poin on, ew i ing any disco e ed
WRPKRU occu ences, and copy he WRPKRU- ee ins uc ion sequences back o
he execu able page. To p e en o he h eads om execu ing pa ially o e w i en
ins uc ion sequences, we ac ually ew i e a esh copy o he execu able page wi h
he WRPKRU- ee sequences, and hen swap his ew i en copy o he execu able
page. This echnique is anspa en o he applica ion, has an o e head p opo ional
o he numbe o en y poin s in o o ending pages (we disassemble om e e y en y
poin only once) and main ains he in a ian ha only sa e WRPKRU sequences a e
execu able.
In con as o ew i ing a un ime, a bina y can be s a ically ew i en o emo e
all inad e en WRPKRUs. Compa ed o a compile o un ime app oach, s a ic
bina y ew i ing does no ely on sou ce code a ailabili y and does no imposes
addi ional un ime o e head. I s d awback is he dependence on a s a ic ew i e ool
which can success ully ew i e a bina y. In o de o success ully ew i e a bina y,
83
ools like Dynins [
34
] equi e a ull disassembly o he bina y. Recen ly Bauman e
al. [16] ha e p oposed a s a ic ew i e echnique emo ing his dependency.
Ou s a ic ew i e app oach, simila o he bina y inspec ion, pe o ms a simple
linea scan o he bina y o ind all hose inad e en occu ences o he 3-by e
WRPKRU sequence in execu able sec ions. Nex , using any bina y ew i ing ool, e.g.,
Dynins [
34
], we disassemble he bina y o he ex en possible, and ew i e ins uc ions
o elimina e hese inad e en occu ences. We use he p e iously desc ibed ew i ing
s a egy (see Sec ion 4.2.1 o able 4.1). Occu ences o WRPKRU in pa s ha we
canno disassemble a e handled by he bina y inspec ion and ew i ing a un ime as
desc ibed in he p e ious Sec ion.
We e alua e he e ec i eness o s a ically ew i ing bina ies in Sec ion 4.5.1.4.
4.3 Use Cases
ERIM di e s om p io wo k by p o iding e icien isola ion in applica ions whe e
swi ches be ween us ed and un us ed componen s a e e y equen , o he o de
o 10
5
o 10
6
imes a second. We desc ibe h ee such use-cases he e, and show in
Sec ion 4.5 ha ERIM’s o e head is low on all o hem.
4.3.1 Isola ing c yp og aphic keys in web se e s
Isola ing long- e m SSL keys o p o ec om web se e ulne abili ies such as he
Hea bleed bug [
90
] is well-s udied [
74
,
75
]. Howe e , long- e m keys a e accessed
ela i ely in equen ly (only a ew imes pe use session). Session keys ha a e
accessed a mo e equen ly (up o 10
6
imes a second pe co e in a high h oughpu
web se e like nginx) ha e no been isola ed so a . Isola ing sessions keys is also
ele an as hese keys p o ec he con iden iali y o indi idual use s. No exis ing
echnique can isola e session keys wi hou signi ican o e head.
84
Taking ERIM’s e icien isola ion in o accoun , an ERIM-p o ec ed componen
can isola e he c yp og aphic keys and c yp og aphic me hods. This esul s in a small
TCB and a ack su ace. OpenSSL does no implemen isola ion wi hin he lib a y,
hence we pa i ioned OpenSSL’s low-le el c yp o lib a y (libc yp o) o isola e he
session keys and basic c yp o ou ines, which un as
T
, om he es o he web se e ,
which uns as
U
. The ou e laye o OpenSSL p o ides he high-le el SSL/TLS
in e ace, whe eas he inne , isola ed laye secu ely s o es he c yp og aphic keys
and pe o ms c yp og aphic ope a ions. When using his ERIM-p o ec ed OpenSSL
wi hin a se e applica ion, a new SSL/TLS session c ea es a session key wi hin he
T
. Messages o his session can only be en-/dec yp ed wi hin he
T
. This e icien ly
p o ec s he c yp og aphic keys o se e applica ions om memo y ulne abili ies.
4.3.2 CPI/CPS
Code-poin e in eg i y (CPI) [
72
] is a compile ans o m ha p e en s con ol- low
hijacks by isola ing sensi i e objec s—code poin e s and objec s ha can lead o
code poin e s—in a sa e egion ha canno be w i en wi hou bounds checks. CPS
is a ligh e , less-secu e a ian o CPI ha isola es only code poin e s. Swi ching
a es o he sa e egion can be e y high in CPI, o he o de o 10
6
swi ches pe
second on s anda d benchma ks. A key ques ion in CPI/CPS is how o isola e he
sa e egion. The o iginal pape uses ASLR on x86-64 o i s e alua ion. ASLR
has almos no un ime o e head, bu i is now known o be ine ec i e o da a
isola ion [113, 60, 39, 49, 94].
We show ha ERIM can p o ide s ong isola ion o he sa e egion a low cos .
To do his, we o e ide he CPI/CPS-enabled compile ’s in insic unc ion o w i ing
he sensi i e egion o use a call ga e a ound an inlined sequence o
T
code ha
pe o ms a bounds check be o e he w i e. (MemSen y [
68
] also p oposes he use
85
o MPKs o isola ing he sa e egion, bu does no ac ually build o e alua e his
use-case.)
4.3.3 Na i e lib a ies in managed un imes
Applica ions unning on managed un imes such as a Ja a o Ja aSc ip VM o en
ely on hi d-pa y na i e code lib a ies. A ele an secu i y goal is o isola e he
managed un ime om bugs and ulne abili ies in he na i e lib a ies. ERIM can
be used o his pu pose by mapping he managed un ime o
T
and he na i e
lib a y(ies) o
U
. We es his by isola ing a na i e SQLi e plugin om Node.js.
SQLi e and Node.js a e, espec i ely, a s a e-o - he-a C da abase lib a y and a
s a e-o - he-a managed un ime o Ja aSc ip [121, 91].
4.4 Implemen a ion
We ha e implemen ed a p o o ype o ERIM on Linux. The p o o ype includes a 77
line Linux Secu i y Module (LSM) ha in e cep s all mmap and mp o ec calls o
p e en
U
om mapping pages in execu able mode, and p e en s
U
om o e iding
he bina y inspec ion handle . We also added 26 LoC in ke nel hooks needed o
his module. Ou implemen a ion also includes he ERIM un ime lib a y, which
p o ides a memo y alloca o o e
MT
, call ga es, he ERIM ini ializa ion code, and
bina y inspec ion. These comp ise 569 LoC.
Sepa a ely, we ha e implemen ed he ew i ing logic o elimina e inad e en
WRPKRU occu ences (abou 2250 LoC). While we ha e no ye in eg a ed he
logic in o ei he a compile o ou inspec ion handle , we ha e in eg a ed i in o a
s andalone bina y ew i ing ool ha uses Dynins [
34
] o disassemble bina ies. The
bina ies used in ou e alua ion do no ha e any unsa e WRPKRU occu ences and
do no load any lib a ies a un ime.
86
Call ype Cos (cycles)
Inlined call (no swi ch) 5
Di ec call (no swi ch) 8
Indi ec call (no swi ch) 19
Inlined call + swi ch 60
Di ec call + swi ch 69
Indi ec call + swi ch 99
ge pid sys em call 152
lwC swi ch [74] (Skylake CPU) 6050
Table 4.2: Cycle coun s o basic call and e u n
4.5 E alua ion
We e alua e ERIM on mic obenchma ks and on he h ee applica ions men ioned
in Sec ion 4.3. We pe o m ou expe imen s on Dell Powe Edge R640 machines
wi h 16-co e MPK-enabled In el Xeon Gold 6142 2.6GHz CPUs (wi h Tu bo Boos
and SpeedS ep disabled), 384GB memo y, 10Gbps E he ne links, unning Debian
8. Fo he CPI expe imen , we use he Le ee p o o ype 0.2 a ailable om
h p:
//dslab.ep l.ch/p oj/cpi/
and Clang 3.3.1 including i s CPI compile pass,
un ime lib a y ex ensions and link- ime op imiza ion. Fo he nginx expe imen , we
use nginx 1.12.1 and OpenSSL 1.1.1 and he ECDHE-RSA-AES128-GCM-SHA256
ciphe . Fo he managed language un ime expe imen , we use Node.js 9.11.1 and
SQLi e 3.22.0. Fo a compa ison base line we use SQLi e compiled o WebAssembly
ia emsc ip en 1.37.37’s WebAssembly backend [36].
4.5.1 Mic obenchma ks
4.5.1.1 Swi ch cos
We pe o med a mic obenchma k o measu e he o e head o in oking a unc ion
wi h and wi hou a swi ch o a us ed componen . The unc ion adds a cons an o
an in ege a gumen and e u ns he esul . Table 4.2 shows he cos o in oking
87
he unc ion, in cycles, as an inlined unc ion (I), as a di ec ly called unc ion (DC),
and as a unc ion called ia a unc ion poin e (FP). Fo e e ence, he able also
includes he cos o a simple syscall (ge pid) and he cos o a swi ch on lwCs, a
ecen in-p ocess isola ion mechanism based on s anda d page able p o ec ions [
74
].
In ou mic obenchma k, calls wi h an ERIM swi ch a e be ween 55 and 80 cycles
mo e expensi e han hei no-swi ch coun e pa s. The mos expensi e indi ec call
cos s less han he simples sys em call (ge pid). ERIM swi ches a e up o 100x
as e han lwC swi ches.
Because he CPU mus no eo de loads and s o es wi h espec o a WRPKRU
ins uc ion, he o e head o an ERIM swi ch depends on he CPU pipeline s a e
a he ime o he WRPKRUs in he swi ch. In expe imen s desc ibed la e in his
Sec ion, we obse ed a e age o e heads anging om 11 o 260 cycles pe swi ch. A
a clock a e o 2.6GHz, his co esponds o o e heads be ween 0.04% and 1.0% o
100,000 swi ches pe second, which is signi ican ly lowe han he o e head o any
bounds-check, ke nel- o hype iso -based isola ion.
4.5.1.2 Emula ing MPK’s swi ch cos
Following we desc ibe how o emula e he WRPKRU ins uc ion. This enables
us o compa e agains echniques who’s en i onmen does no suppo MPK. The
WRPKRU ins uc ion mo es he alue o he eax egis e o he PKRU egis e .
Howe e , since he ins uc ion impac s he alidi y o subsequen loads/s o es, he
ins uc ion canno be e-o de ed ela i e o su ounding load/s o e ins uc ions in
he execu ion pipeline. We emula e he cos o WRPKRU using a sequence o
xo ins uc ions ha ha e no ne unc ional e ec (excep consuming CPU cycles),
ollowed by RDTSCP, which causes a pipeline s all and p e en s ins uc ion e-
o de ing. The emula ion code is shown in Lis ing 4.2.
88
o ( i = 0; i < 5; i++) {
xo eax , ecx
xo ecx , eax
xo eax , ecx
}
d scp
Lis ing 4.2: WRPKRU emula ion using RDTSCP and Xo Swi ch
Benchma k Swi ches/sec CPI O e head (%)
ERIM EMUL
403.gcc 13,454,647 22.3 22.68
445.gobmk 1,055,994 1.77 1.76
447.dealII 1,270,582 0.56 0.17
450.soplex 408,192 0.6 2.56
464.h264 e 1,684,572 1.22 0.86
471.omne pp 36,578,718 144.02 142.26
482.sphinx 1,148,883 0.84 0.65
483.xalancbmk 21,448,977 52.22 51.74
Table 4.3:
Domain swi ch a es o selec ed SPEC CPU benchma ks and o e heads
o ERIM-CPI and EMUL-CPI, ela i e o s anda d CPI.
Valida ion
To alida e ha ou emula ion es ima es o e heads close o hose o he
ac ual WRPKRU ins uc ion, we e- un he CPI/CPS benchma ks o Sec ion 4.5.2
wi h WRPKRU emula ion in place o he ac ual WRPKRU ins uc ion. Figu e 4.1
ep oduces ERIM’s ela i e o e heads on a ious benchma ks om Figu e 4.2 bu
addi ionally lis s he ela i e o e heads using he WRPKRU emula ion (lines EMUL-
CPI and EMUL-CPS). Table 4.3 lis s he p ecise o e heads o CPI on benchma ks
ha ha e high swi ching a es. As can be seen, he o e heads o he emula ion a e
e y close o ac ual ERIM’s o e heads on all benchma ks.
No e om Table 4.3 ha ou emula ion is no pe ec , bu qui e close o he
ac ual in e ms o o e head. Emula ing he pe o mance o WRPKRU pe ec ly is
di icul since emula ion canno exac ly ep oduce he e ec s o WRPKRU on he
execu ion pipeline. (WRPKRU mus p e en he eo de ing o loads and s o es wi h
espec o i sel .) Depending on he speci ic benchma k, ou emula ion sligh ly o e -
89
Benchma k Swi ches/sec ERIM-CPI o e head
ela i e o o ig. CPI in %
403.gcc 16,454,595 22.30%
445.gobmk 1,074,716 1.77%
447.dealII 1,277,645 0.56%
450.soplex 410,649 0.60%
464.h264 e 1,705,131 1.22%
471.omne pp 89,260,024 144.02%
482.sphinx3 1,158,495 0.84%
483.xalancbmk 32,650,497 52.22%
Table 4.5:
Domain swi ch a es o selec ed SPEC CPU benchma ks and o e heads
o ERIM-CPI wi hou bina y inspec ion, ela i e o he o iginal CPI wi h ASLR.
Table 4.5 also shows he o e head o ERIM-CPI excluding bina y inspec ion,
ela i e o he o iginal CPI o e ASLR (no ela i e o an unp o ec ed baseline
as in Figu e 4.2). This ela i e o e head is exac ly he cos o ERIM’s swi ching.
Depending on he benchma k, i a ies om 0.03% o 0.16% o 100,000 swi ches
pe second o , equi alen ly, 7.8 o 41.6 cycles pe swi ch. These esul s indica e ha
ERIM can suppo inlined e e ence moni o s wi h swi ching a es o up o 10
6
imes
a second wi h low o e head. Beyond his a e, he o e head becomes no iceable.
4.5.2.2 CPS
The esul s o CPS a e simila o hose o CPI, bu he o e heads a e gene ally
lowe . Rela i e o anilla SPEC wi h no p o ec ion, he geome ic means o he
o e heads o he o iginal CPS and ERIM-CPS ac oss all benchma ks a e 1.1% and
2.4%, espec i ely. ERIM-CPS o e head ela i e o he o iginal CPS is wi hin 2.5%
on all benchma ks, excep excep pe lbench, omne pp and xalancbmk, whe e i
anges up o 17.9%.
4.5.3 P o ec ing session keys in nginx
Nex , we use ERIM o isola e SSL session keys in a high pe o mance web se e ,
nginx. We modi ied OpenSSL’s libc yp o o isola e he keys and he unc ions o
96
File
size
(KB)
Th oughpu Swi ches/s CPU
load
na i e
(%)
Na i e
( e-
q/s)
ERIM
el.
(%)
0
95,761
95.83 1,342,605 100.0
1
87,022
95.18 1,220,266 100.0
2
82,137
95.44 1,151,877 100.0
4
76,562
95.25 1,073,843 100.0
8
67,855
95.98 974,780 100.0
16
45,483
97.10 820,534 100.0
32
32,381
97.31 779,141 100.0
64
17,827
100.00 679,371 96.7
128 8,937 99.99 556,152 86.4
Table 4.6:
Nginx h oughpu wi h a single wo ke . The s anda d de ia ion is below
1.1% in all cases.
AES key alloca ion and enc yp ion/dec yp ion in o ERIM’s
T
and use ERIM call
ga es o in oke hese unc ions.
Ou goal is o measu e ERIM’s o e head on he peak h oughpu o nginx. To
s a , we con igu e nginx o un a single wo ke pinned o a CPU co e, and connec
o i emo ely om 4 concu en ApacheBench (
ab
) [
8
] ins ances o e HTTPS wi h
keep-ali e. Each ins ance simula es 75 concu en clien s. The clien s all eques
he same ile, whose size we a y om 0 o 128KB ac oss expe imen s. Figu e 4.3b
shows he h oughpu o ERIM-p o ec ed nginx ela i e o ou baseline (na i e
nginx wi hou any p o ec ion) o di e en eques sizes, measu ed a e an ini ial
wa m-up pe iod. Figu e 4.3a shows he absolu e h oughpu s in eques s/s in he
same expe imen . All numbe s a e a e ages o 10 uns.
ERIM-p o ec ed nginx p o ides a h oughpu wi hin 95.18% o he unp o ec ed
se e o all eques sizes. To explain he o e head u he , we lis he numbe o
ERIM swi ches pe second in he nginx wo ke and he wo ke ’s CPU u iliza ion
in Table 4.6 o eques sizes up o 128KB. The o e head shows a gene al end up
o eques s o size 32 KB: The wo ke ’s co e emains sa u a ed bu as he eques
size inc eases, he numbe o ERIM swi ches pe second dec ease, and so does
97
0
20000
40000
60000
80000
100000
0kb
1kb
2kb
4kb
8kb
16kb
32kb
64kb
128kb
Reques s/s
File size
Na i e
ERIM
(a) A e age numbe o eques s pe second o na i e and ERIM.
0
0.2
0.4
0.6
0.8
1
0kb
1kb
2kb
4kb
8kb
16kb
32kb
64kb
128kb
No malized Th oughpu
File size
Na i e ERIM
(b) No malized h oughpu o na i e (no p o ec ion).
Figu e 4.3:
Nginx h oughpu wi h one wo ke , wi h and wi hou ERIM p o ec ion,
wi h a ying eques sizes. S anda d de ia ions we e all below 1.1%.
98
ERIM’s ela i e o e head. The obse a ions a e consis en wi h an o e head o abou
0.31%–0.44% o 100,000 swi ches pe second. Fo eques sizes o 64KB and highe ,
he 10Gbps ne wo k ca d sa u a es and he wo ke does no u ilize i s CPU co e
comple ely in he baseline. The ee CPU cycles abso b ERIM’s CPU o e head, so
ERIM’s h oughpu ma ches ha o he baseline.
No e ha his is an ex eme es case o a web se e . He e, he web se e does
almos no hing and se es he same cached ile epea edly. To ge a mo e ealis ic
assessmen , we se up nginx o se e om a 571 MB co pus o 15,520 s a ic HTML
Wikipedia pages snapsho ed in 2006 [
137
]. File sizes a y om 417 by es o 522
KB (a e age size 37.7 KB). 75 keep-ali e clien s eques andom pages (selec ed
based on page iews on Wikipedia [
138
]). The a e age h oughpu wi h a single nginx
wo ke was 22,415 eques s/s in he base line and 21,802 eques s/s wi h ERIM (s d.
de s. below 0.6% in bo h cases). On a e age, he e we e 615,000 swi ches a second.
This co esponds o a o al o e head o 2.7%, o abou 0.43% o 100,000 swi ches a
second.
4.5.3.1 Scaling wi h mul iple wo ke s
To e i y ha ERIM scales wi h co e pa allelism, we e- an he i s expe imen abo e
wi h 3, 5 and 10 nginx wo ke s pinned o sepa a e co es, and su icien numbe s o
concu en clien s o sa u a e all he wo ke s. Table 4.7 shows he ela i e o e heads
wi h di e en numbe o wo ke s. Fo eques s la ge han hose shown in he able,
he ne wo k ca d sa u a es, and he spa e CPU cycles in he na i e base line abso b
ERIM’s o e head comple ely. Fo compa ison, he second and hi d columns o he
able epea he numbe s o he 1 wo ke con igu a ion o Table 4.6.
In he baseline, nginx’s h oughpu scales qui e well wi h he numbe o wo ke s.
Impo an ly, he ela i e o e head o ERIM’s p o ec ion does no inc ease wi h he
numbe o co es. Thus, ERIM scales wi h mul i-co e pa allelism indica ing ha
ERIM adds no addi ional synch oniza ion and scales pe ec ly wi h co e pa allelism.
99
File
size
(KB)
1 wo ke 3 wo ke s 5 wo ke s 10 wo ke s
Na i e
( e-
q/s)
ERIM
el.
(%)
Na i e
( e-
q/s)
ERIM
el.
(%)
Na i e
( e-
q/s)
ERIM
el.
(%)
Na i e
( e-
q/s)
ERIM
el.
(%)
0
95,761
95.83
276,736
96.05
466,419
95.67
823,471
96.40
1
87,022
95.18
250,565
94.50
421,656
96.08
746,278
95.47
2
82,137
95.44
235,820
95.12
388,926
96.60
497,778
100.00
4
76,562
95.25
217,602
94.91
263,719
100.00
8
67,855
95.98
142,680
100.00
Table 4.7:
Nginx h oughpu wi h mul iple wo ke s. The s anda d de ia ion is
below 1.5% in all cases.
This is unsu p ising gi en ha upda es o he PKRU o a co e a ec execu ion on
ha co e only.
4.5.3.2 Compa ison o ke nel-based isola ion
Using he single wo ke nginx expe imen , we compa e ERIM’s o e head o ha
o lwCs [
74
], a s a e-o - he-a sys em o in-p ocess isola ion based on s anda d
page- able p o ec ions. LwCs map each isola ed componen o a sepa a e add ess
space (in he same p ocess). A swi ch be ween componen s equi es ke nel media ion
o change page ables.
Since lwCs we e implemen ed only o F eeBSD, whose cu en ke nel suppo s
nei he MPKs no ou Xeon Gold machines, we un his compa ison expe imen
on an olde machine wi hou MPK suppo and use an emula ion o WRPKRU
o accoun o ERIM’s o e head as desc ibed in Sec ion 4.5.1.2. All expe imen s
desc ibed he e we e pe o med on Dell Op iPlex 7040 machines wi h 4-co e In el
Skylake i5-6500 CPUs clocked a 3.2GHz, 16GB memo y, 10 Gbps E he ne ca ds,
unning F eeBSD 11.
We use he exis ing single wo ke nginx expe imen (Sec ion 4.5.2) o compa e
he pe o mance o an ERIM-based isola ion o ha o lwC-based isola ion. As
opposed o ERIM swi ches, which ope a e en i ely in use space, lwC swi ches a e
syscalls. We c ea e a second ins ance o nginx ha uses an lwC (in place o an ERIM
100
componen ) o isola e session keys and basic c yp og aphic unc ions. We alloca e
da a bu e s in a memo y egion ha is sha ed be ween he p o ec ed lwC and he
web se e lwC o acili a e e icien da a sha ing. The o e head o WRPKRU is
emula ed as p e iously explained.
Figu e 4.4b depic s he o e head o he ERIM- and lwC-based a ian s ela i e
o he na i e baseline o an unmodi ied nginx, a e aged o e 20 uns. Nginx is
con igu ed o un one wo ke and se es 4 ApacheBench ins ances each simula ing
75 clien s accessing a s a ic ile ia HTTPS wi h keep-ali e.
The ERIM-based emula ion p o ides h oughpu wi hin 97.88% (wi hin 99%
o iles 64KB and la ge ) o he unp o ec ed na i e se e , whe eas he lwC-based
isola ion is limi ed o 50% o he na i e se e h oughpu o small iles and up o
80% o la ge (2MB) iles. The eason is he cos o lwC swi ch syscalls, which is oo
high gi en he a e o in oca ions o he enc yp ion unc ions.
Figu e 4.4a shows he absolu e numbe o se ed eques s pe second o he same
expe imen . The lwC-based nginx canno sus ain mo e han 26,500 eq/s, whe eas
ERIM pe o ms close o he na i e implemen a ion. A 64KB iles we sa u a e he
10Gbi ne wo k link esul ing in lowe eq/s o he na i e baseline and ERIM.
In summa y, we ind ha lwCs pe o m signi ican ly wo se han ERIM in his
expe imen : The h oughpu o nginx wi h lwC-based isola ion is ne e abo e 80%
o na i e nginx and, o small eques s, whe e he swi ch a e is highe , i is below
50% o na i e nginx. In con as , wi h ERIM’s isola ion, he h oughpu is wi hin
95% o na i e nginx in all con igu a ions. Hence, ERIM pe o ms signi ican ly be e
han ke nel-media ed isola ion a high swi ch a es.
4.5.4 Isola ing managed un imes
Nex , we es ERIM’s use o isola e a managed language un ime om an un us ed
na i e lib a y. Speci ically, we link he widely-used na i e da abase lib a y, SQLi e,
101
0
20000
40000
60000
80000
100000
0kb
1kb
2kb
4kb
8kb
16kb
32kb
64kb
128kb
256kb
512kb
1mb
2mb
Reques s/s
File size
Na i e
ERIM (emula ed)
LwC
(a) A e age numbe o eques s pe second, na i e, ERIM and lwC.
0
0.2
0.4
0.6
0.8
1
0kb
1kb
2kb
4kb
8kb
16kb
32kb
64kb
128kb
256kb
512kb
1mb
2mb
No malized Th oughpu
File size
Na i e
ERIM (emula ed)
LwC
(b) ERIM and lwC h oughpu no malized o na i e.
Figu e 4.4:
Nginx h oughpu wi h one wo ke , wi h emula ed ERIM p o ec ion
and wi h lwCs, wi h a ying eques sizes. S anda d de ia ions we e all below 1.1%.
102
o Node.js, a s a e-o - he-a Ja aSc ip un ime and use ERIM o isola e Node.js
om SQLi e by mapping Node.js’s un ime o
T
and he na i e lib a y o
U
. We
manually ins umen ed SQLi e’s en ypoin s o in oke call ga es. Addi ionally, since
we wan o isola e Node.js’s s ack om SQLi e, we un Node.js on a sepa a e s ack
in
MT
, and add code o swi ch o he s anda d s ack (in
MU
) p io o calling a
SQLi e unc ion. Finally, SQLi e uses he libc unc ion
memmo e
, which accesses libc
cons an s ha a e in
MT
, so we implemen ed a sepa a e
memmo e
o SQLi e. In
o al, we added 437 LoC.
We measu e un ime using he speed es 1 benchma k ha comes wi h SQLi e
and emula es a ypical da abase wo kload [
122
]. This benchma k pe o ms a o al o
32 sho es s ha s ess di e en da abase unc ions like selec s, joins, inse s and
dele es. We inc eased he i e a ions in each es by a ac o o ou o make he es s
longe . Ou base line o compa ison is anilla SQLi e linked o Node.js wi hou any
p o ec ion. We con igu e he benchma k o s o e he da abase in-memo y and epo
a e ages o 20 uns.
The geome ic mean o ERIM’s o e head on un ime ac oss all es s is 4.3%.
The o e head is below 6.7% on all es s excep hose wi h mo e han 10
6
swi ches
pe second. This sugges s ha ERIM can be used o isola ing na i e lib a ies om
managed language un imes wi h low o e heads up o a swi ching cos o he o de o
10
6
pe second. Beyond ha he o e head is no iceable. Table 4.8, columns 1–3, show
he ela i e o e heads o es s wi h swi ching a es o a leas 100,000/s. These a e
consis en wi h an a e age o e head be ween 0.07% and 0.41% o 100,000 swi ches/s.
The ac ual swi ch cos measu ed om di ec CPU cycle coun s a ies om 73 o 260
cycles ac oss all es s. The swi ch cos exceeds 100 cycles only on benchma ks whe e
he swi ch a e is e y low (less han 2,000 imes/s). We e i ied ha hese highe
cycle coun s a e due o ins uc ion cache misses—a e y low swi ch a es, he call
ga e is lushed ou o he ins uc ion cache be ween swi ches.
103
Tes
#Swi ches/s O e head (%)
ERIM WebAssembly
100 11,183,281 12.73% 132.48%
110 8,329,914 12.18% 135.44%
400 8,161,584 15.42% 156.04%
120 7,190,766 13.81% 145.19%
142 7,074,553 9.41% 165.88%
500 6,419,008 12.13% 119.15%
510 5,868,395 5.60% 113.76%
410 5,091,212 3.64% 122.77%
240 2,358,524 3.74% 126.63%
280 2,303,516 3.22% 100.05%
170 1,264,366 4.22% 104.87%
310 1,133,364 2.92% 81.71%
161 1,019,138 2.81% 138.64%
160 1,014,829 2.73% 136.27%
230 670,196 2.04% 193.42%
270 560,257 2.28% 92.78%
Table 4.8:
O e head ela i e o na i e execu ion o SQLi e speed es 1 es s wi h
mo e han 100,000 swi ches/s. S anda d de ia ions we e below 5.6% o na i e, and
ERIM and below 15.4% o WebAssembly.
4.5.4.1 Compa ison o isola ion wi h bounds checks (SFI)
We also use he abo e expe imen o compa e ERIM o isola ion based on bounds
checks. Fo his, we e-compile he SQLi e lib a y o na i e code indi ec ly h ough
WebAssembly, a new memo y-sa e, low-le el language designed speci ically o w i ing
sa e na i e plugins o Ja aSc ip en i onmen s [
53
]. The WebAssembly o na i e
code ansla ion inse s bounds checks p io o indi ec memo y accesses. Compila ion
ia WebAssembly is he cu en ly ecommended me hod o sa ely adding na i e
plugins o Google’s Ch ome web b owse ; mos majo web b owse s a e expec ed o
ecommend he same me hod in he nea u u e.
Ac oss all 32 es s, he geome ic mean o he ela i e o e head o WebAssembly-
based isola ion on un ime is 133.5%. The o e heads ange om 66.4% o 280.6%,
which is signi ican ly highe han ERIM’s o e heads. Howe e , WebAssembly’s
o e heads do no inc ease wi h he swi ching a e since i does no in e pose on
104
swi ches. Ins ead, i imposes a con inuous o e head while execu ion is in SQLi e.
O he wo k using bounds checks has ound simila ly high o e heads on pe o mance-
in ensi e benchma ks [95, 53].
4.6 Rela ed Wo k
Re e ence moni o s [
6
] media e p i ileged access by un us ed applica ions p o ec ing
da a con iden iali y and in eg i y, by sandboxing he applica ion o checks inse ed
in o he applica ions execu ion. All implemen a ions sha e he impo an p ope y
o p o ec ing he e e ence moni o ’s code and s a e om co up ion and deploy
isola ion echniques shielding he e e ence moni o . Leas -p i ilege and p i ilege
sepa a ion [
105
] de ine he basis o oday’s e e ence moni o mechanisms in ha dwa e
([
82
,
63
,
12
]), hype iso s ([
14
,
19
]), ope a ing sys ems ([
132
,
35
]) o applica ions
([
129
,
80
,
41
,
143
,
29
]). Two ecen su eys [
126
,
117
] show iable a acks and possible
coun e measu es o p o ec agains da a con iden iali y and in eg i y iola ions.
Fu he mo e hei in e cep ion g anula i y a ies om a sepa a e gues OS ([
14
]),
a single applica ion ([
19
,
20
,
132
]) o applica ion componen s ([
129
,
82
,
74
,
80
,
41
,
143
,
29
,
144
]). In e cep ing a ine g anula i y o e s isola ion ac oss applica ion
componen s, whe eas cou se-g ain in e cep ion isola es independen applica ions o
componen s. Due o i s equen in oca ions, ine-g ained in e cep ion solu ions
equi e isola ion echniques wi h low o e head. ERIM isola es applica ion componen s
and in e cep s ine-g ained secu i y ele an e en s wi hin he applica ion, simila ly
o ARMlock [
144
] o SFI-based isola ion [
129
,
80
,
41
,
143
]. This is in con as o
echniques using OS p ocess bounda ies ([74]) o CPU p i ilege le els ([19, 14]).
Koning e al. [
68
] su ey echniques o e icien da a encapsula ion wi hin a
p ocess, including SFI, dynamic enc yp ion o p i a e da a using he In el AES-NI
ISA ex ensions, app oaches ha use VT-x i ualiza ion ha dwa e, and hose ha
ely on he In el MPX and MPK ISA ex ensions. I hen p esen s a gene al isola ion
105
CHAPTER 5
Conclusion
Today compu e s s o e and analyze aluable and sensi i e da a such as pe sonal
mul imedia o clien eco ds. An impo an goal is o p o ec he con iden iali y
and in eg i y o such da a, minimizing he isk o illici elease, loss o modi ica ion.
Howe e , exis ing echniques o p o ec con iden iali y and in eg i y a e ulne able
o malicious a acks o a e ine icien . This hesis con ibu es wo new echniques,
Gua da and ERIM, p o iding con iden iali y and in eg i y o pe sis en and in-
memo y da a secu ely and e icien ly.
Gua da en o ces, a he s o age laye , ich pe - ile con iden iali y and in eg i y
policies wi h low o e head. The en o cemen a he s o age laye educes he a ack
su ace and he isk o ci cum en ion due o so wa e bugs, miscon igu a ions and
ope a o e o s in highe laye s. Gua da o e comes he gap be ween s o age laye
en o cemen and pe - ile policies by a es ing he s a e o iles and associa ed policies
h ough c yp og aphically-signed ce i ica es. To speci y policies, we de elop a
domain-speci ic language which allows da a accesses condi ioned on au hen ica ion,
us ed wall clock ime, and a ile’s s a e including he con en . We demons a e an
e icien implemen a ion o en o ce such policies in an iSCSI SAN se e and apply
Gua da o wo use cases p o ec ing he con en , execu able, and log iles o a web
se e , as well as en o cing manda o y access logging.
ERIM p o ides da a con iden iali y and in eg i y o in-memo y da a by isola ing
sensi i e da a om accesses by un us ed componen s. I isola es sensi i e da a
113
in o a sepa a e, us ed memo y componen using In el MPK and ensu es ha
only he us ed componen has access o sensi i e da a. To p e en malicious
a acks om escala ing p i ileges using he unp i ileged WRPKRU CPU ins uc ion,
ERIM addi ionally p o ec s he us ed componen ia secu e con ol ans e s and
bina y inspec ion. Secu e con ol ans e s ensu e ha he un us ed componen
canno ele a e access pe missions wi hou he in ol emen o he us ed componen .
Bina y inspec ion gua an ees ha no execu able bina y code sequence ele a es access
pe missions o he us ed componen , while execu ing un us ed code. ERIM’s
isola ion imposes no addi ional o e head on he execu ion and less han 1% un ime
o e head pe 100,000 swi ches/second. Unlike s a e-o - he-a isola ion echniques,
he low swi ch cos and no o e head on execu ion allows ERIM o e icien ly isola e
equen ly-used session keys in web se e s, an in-memo y e e ence moni o ’s p i a e
s a e, and managed un imes om na i e lib a ies as demons a ed in he e alua ion.
5.1 Fu u e Wo k
Gua da and ERIM independen ly p o ec he con iden iali y and in eg i y o sensi i e
pe sis en and in-memo y da a. While Gua da es ic s da a accesses a he s o age
laye , i does no p o ec da a eleased o an applica ion. In con as ERIM es ic s
access o applica ion da a, bu does no p o ec pe sis en da a om malicious
a acks. Al hough beyond he goal o his hesis, in his sec ion we discuss how o
o e come he limi a ions o each echnique and a ain an end- o-end con iden iali y
and in eg i y gua an y o sensi i e pe sis en da a h oughou i s in-memo y use in
an applica ion.
We can o e come he limi a ions o each indi idual sys em by connec ing an
ERIM-isola ed moni o o a Gua da de ice which s o es he sec e s. These pe sis-
en ly s o ed sec e s on a Gua da de ice could be p o ec ed om a bi a y da a
accesses by associa ing a policy which allows da a access only by au ho ized connec-
114
ions. Howe e , wi hou any changes o ERIM, an ERIM-isola ed moni o would
no be able o access hese sec e s. To gain access, he moni o has o connec o
he Gua da de ice and au hen ica e i sel . The cu en design o ERIM does no
p o ide a way o gene a e a unique au ho iza ion sec e ha is eliable and consis en
ac oss en i onmen s and eboo s. In o de o gene a e au ho iza ion sec e s, exis ing
echniques like us ed pla o m modules (TPM) o In el SGX gene a e au ho iza ion
sec e s o code by measu ing he code’s in-memo y oo p in as a secu e hash. This
hash is hen used o de i e a unique au ho iza ion sec e .
Simila o exis ing echniques, we sugges o change ERIM’s ini ializa ion o
gene a e an au ho iza ion sec e by measu ing he us ed moni o ’s oo p in . Once
measu ed, he sec e is placed in he us ed moni o ’s memo y (ou side o he
un us ed applica ion’s each). Using his sec e , he us ed moni o au hen ica es
i sel o he Gua da de ice. This app oach gua an ees ha Gua da only eleases
sec e s o an ERIM-isola ed moni o , while ERIM p o ec s he in-memo y copy o
he sec e om accesses by an un us ed applica ion.
By connec ing an ERIM-isola ed moni o o Gua da we p o ide an end- o-end
con iden iali y and in eg i y gua an ee which would be pa icula ly in e es ing o
se e applica ions ha ely on a secu e connec ion o clien s using asymme ic
c yp og aphy. In oday’s se e and c yp og aphic lib a y implemen a ions (see
Figu e 5.1a) he p i a e and session keys a e no isola ed in memo y o p o ec ed in
pe sis en s o age. Exis ing se e applica ions ead a p i a e key om pe sis en
s o age in o memo y and use he key o es ablish a secu e connec ion by nego ia ing
a session key. The session key is s o ed in memo y and used by bo h pa ies o
enc yp and dec yp messages. Hence, secu i y ulne abili ies and bugs in he
un us ed se e applica ion, he ope a ing sys em, o applica ions wi h access o
he s o age may esul in a con iden iali y o in eg i y iola ion. Fo example,
unau ho ized applica ions may ead he pe sis en p i a e key and elease hem,
iola ing con iden iali y. Simila ly, he key could be modi ied on disk, iola ing
115
(a)
Today’s se e applica-
ions
(b)
Isola ing a c yp o-
g aphic lib a y using ERIM
and connec ing i o a Gua -
da de ice
(c)
Combining ERIMwi h
So wa e Gua d Ex ensions
(SGX) o isola e om he
ope a ing sys em
Figu e 5.1: S eps owa ds an isola ed c yp og aphic lib a y in se e applica ions
in eg i y. Once he keys eside in memo y, malicious a acks like Hea bleed [
90
] can
elease o modi y he keys, iola ing bo h con iden iali y and in eg i y.
To p o ec he keys om hese ypes o h ea s, Gua da in combina ion wi h
ERIM can p o ec he pe sis en keys and he in-memo y keys e icien ly as shown in
Figu e 5.1b. The se e needs o be spli in o a us ed moni o which only holds he
c yp og aphic unc ions and an un us ed se e which handles he communica ion
and p o ides he se ice o he clien . Du ing ini ializa ion ERIM isola es he us ed
moni o ’s memo y om he un us ed se e applica ion. ERIM measu es he us ed
moni o and p o ides he au ho iza ion sec e o he us ed moni o . I hen allows
he us ed moni o o ini ialize, connec o a Gua da de ice, au ho ize using he
p e iously measu ed sec e , and ead he p i a e key. Gua da checks ha he
au hen ica ed clien ac ually is he ERIM-isola ed us ed moni o which is speci ied
in he policy. A e inishing he us ed moni o ’s ini ializa ion, ERIM s a s he
un us ed se e , which begins i s usual ope a ion wai ing o clien s o connec .
Once a clien connec s o es ablish a secu e connec ion, he un us ed se e accep s
he connec ion, s a s he SSL/TLS handshake p o ocol and swi ches o he us ed
moni o whene e enc yp ing o dec yp ing messages using he in-memo y p i a e
key and gene a ing new session keys.
116
In con as o he h ea model o Gua da which assumes in e media e laye s,
such as he OS, o be ulne able o malicious a acks o ci cum en ion, he echnique
p oposed abo e, howe e , assumes he OS o be us ed, since ERIM’s gua an ees
depend on he OS. As a esul , he p oposed solu ion would only p o ec con iden iali y
and in eg i y agains a acks om ou side, e.g., malicious clien s a acking he se e
like Hea bleed [
90
], and any h ea s on he ne wo k be ween he machine unning
he se e and he Gua da de ice. Such a h ea model is common o se e s
unning in he cloud.
Fu he ha dening ERIM agains OS ulne abili ies:
While he cloud h ea
model is commonly assumed, ecen a acks [
24
] show how o he cloud enan s
can access in-memo y sec e s iola ing con iden iali y and in eg i y. In addi ion,
highly sensi i e applica ions may no assume he cloud p o ide o be us ed and,
hence, in e media e laye s like he OS and VMM which p o ide isola ion a e no
longe us ed. In o de o s eng hen he h ea model o he p esen ed echnique,
he memo y isola ion gua an ees o ERIM ha e o be independen o in e media e
so wa e laye s like he OS o VMM.
To de end agains hese h ea s, us ed execu ion en i onmen s (TEE), in
pa icula In el SGX, can be used o shield sensi i e da a om he cloud pla o m
and o he enan s. SGX p o ides in-memo y encla es o s o e sensi i e da a and
execu e code independen o he unning OS o VMM. Se e al esea ch sys ems
[
17
,
13
,
70
] demons a e he use o In el SGX o shield an applica ion agains he
cloud pla o m.
While SGX p o ides s ong memo y isola ion gua an ees, i s high swi ch cos s
compa able o a con ex swi ch hinde s i s adop ion, and p e en s i om being
used o isola e equen ly-used sec e s. Exis ing wo k o e comes hese pe o mance
limi a ions by ei he isola ing in equen ly-used sec e s like p i a e keys o isola ing
an en i e applica ion. Howe e , pushing en i e applica ions (e.g., a se e ) in o
117
an encla e wi hou u he memo y isola ion lea es he applica ion ulne able o
malicious a acks, due o he size and complexi y o hese applica ions. Recen
wo k [
71
] sugges s u he p o ec ing applica ions in SGX encla es by adding memo y
bound checks. Howe e , such checks incu subs an ial un ime o e head, and i is
no su icien o ERIM o simply swap In el MPK o In el SGX. Ins ead, we need
o combine bo h app oaches o allow en i e applica ions o un wi hin SGX encla es,
shielding hem om he emaining cloud so wa e s ack, while isola ing sec e s wi hin
he applica ion using a mechanism simila o In el MPK.
To his end, we sugges amending he SGX speci ica ion, since i has no p o ision
o MPK-like memo y isola ion using pe -page domains and an access pe mission
egis e such as he PKRU egis e . The encla e memo y desc ip o s eside in
p ocesso ese ed memo y which is inaccessible o sys em so wa e (e.g., OS o
VMM). An impo an desc ip o is he encla e page cache map (EPCM), a able-like
s uc u e, which holds in o ma ion abou which memo y pages belongs o an encla e
and holds pe -page access pe mission bi s. Cu en ly he EPCM allows pages o be
accessible wi h ead, w i e, and execu e pe mission and does no allow pages o be
agged wi h a MPK domain.
To allow page-le el memo y isola ion wi hin SGX encla es, we sugges adding
memo y domain iden i ie s o he EPCM and ex ending he CPU’s memo y access
pe mission check o alida e he cu en access pe missions in he PKRU egis e
agains he memo y access’s EPCM domain iden i ie . This app oach ex ends he
each o In el MPK in o SGX encla es. Simila o he use o MPK in ERIM, his
solu ion is ulne able o malicious a acks and hence needs o be combined wi h
ERIM’s secu e con ol ans e s and bina y inspec ion. Fo code in encla es, we can
simpli y ERIM’s bina y inspec ion, since encla e memo y is alloca ed once a he
s a o an encla e and can only be ex ended wi h a special p o ocol including a s ep
in which he encla e app o es he ex ension [
83
]. The bina y inspec ion could scan
once a he s a o unsa e WRPKRUs in all execu able memo y and a un ime i
118
could only app o e new pages which do no con ain unsa e WRPKRUs. The e is no
need o ke nel modi ica ions o signal handle s. By amending he SGX speci ica ion
and combining i wi h ERIM’s secu e con ol ans e s and bina y inspec ion, we
can isola e equen ly-used sec e s wi hin SGX encla e wi hou us ing he sys em
so wa e like he OS (see Figu e 5.1c).
In his sec ion we ha e shown how o ex end he p o ec ion o pe sis en iles om
Gua da o in-memo y da a using ERIM. We discussed he challenge in au hen ica ing
an ERIM-isola ed us ed moni o o a Gua da de ice and desc ibe a echnique
o gene a e an au hen ica ion sec e using code measu emen s. We discuss i s use
in a commonly assumed h ea model o cloud en i onmen s. Fo highly sensi i e
applica ions, we desc ibe an ex ension o In el’s SGX o p o ec agains ogue cloud
p o ide s and o he cloud enan s.
119
Bibliog aphy
[1]
Ma ín Abadi, Mihai Budiu, Úl a E lingsson, and Jay Liga i. Con ol- low
in eg i y. In P oceedings o ACM SIGSAC Con e ence on Compu e and
Communica ions Secu i y (CCS), 2005.
[2]
Ni in Ag awal, William J. Bolosky, John R. Douceu , and Jacob R. Lo ch. A
i e-yea s udy o ile-sys em me ada a. ACM T ansac ions on S o age, 3(3),
2007.
[3]
Ma cos K. Aguile a, Minwen Ji, Ma k Lillib idge, John MacCo mick, E win
Oe li, Da id G. Ande sen, Mike Bu ows, Timo hy Mann, and Chand amohan
Thekka h. Block-Le el Secu i y o Ne wo k-A ached Disks. In P oceedings
o USENIX Con e ence o File and S o age Technologies (FAST), 2003.
[4]
Hussain M. J. Almoh i and Da id E ans. Fidelius Cha m: Isola ing Unsa e
Rus Code. In P oceedings o ACM Con e ence on Da a and Applica ion
Secu i y and P i acy (CODASPY), 2018.
[5]
Amazon. Amazon Simple S o age Se ice (Amazon S3).
h p://aws.amazon.
com/s3/, 2011.
[6] James P. Ande son. Compu e Secu i y Technology Planning S udy (Volume
II), 1972.
[7]
Jason Ansel, Pe Ma chenko, Ul a E lingsson, Elijah Taylo , B ad Chen,
De ek L. Schu , Da id Seh , Cli L. Bi le, and Benne Yee. Language-
independen sandboxing o jus -in- ime compila ion and sel -modi ying code.
In P oceedings o ACM SIGPLAN con e ence on P og amming language design
and implemen a ion (PLDI), 2011.
120
[8]
Apache HTTP Se e P ojec .
h ps://h pd.apache.o g/docs/2.4/
p og ams/ab.h ml.
[9] Apple Inc. Time Machine, 2007.
[10] Apple Inc. Apple Fusion D i e. h ps://www.apple.com/de/imac, 2017.
[11]
ARM. De elope guide: ARM memo y domains.
h p://in ocen e .a m.
com/help/, 2001.
[12]
ARM. ARM Secu i y Technology.
h p://in ocen e .a m.com/
help/ opic/com.a m.doc.p d29-genc-009492c/PRD29-GENC-009492C_
us zone_secu i y_whi epape .pd , 2009.
[13]
Se gei A nau o , Bohdan T ach, F anz G ego , Thomas Knau h, And e Ma in,
Ch is ian P iebe, Joshua Lind, Di ya Mu hukuma an, Ma k L. S illwell, Da id
Gol zsche, Da id Eye s, Pe e Pie zuch, and Ch is o Fe ze . SCONE: Secu e
Linux Con aine s wi h In el SGX. In P oceedings o USENIX Symposium on
Ope a ing Sys ems Design and Implemen a ion (OSDI), 2016.
[14]
Paul Ba ham, Bo is D ago ic, Kei F ase , S e en Hand, Tim Ha is, Alex
Ho, Rol Neugebaue , Ian P a , and And ew Wa ield. Xen and he a o
i ualiza ion. ACM SIGOPS Ope a ing Sys ems Re iew, 37(5), 2003.
[15]
F i zge ald Ba h, Vic o Chin, Moshe Fe be , Sean Hi el, Lau ie Jame-
son, Na heniel Mason, Ha deep Meh o a a, Ashish Meh a, Mihi Mohan y,
K ishna Na ayanswamy, and Michael Roza. Top h ea s o cloud compu ing
plus indus y insigh s.
h ps://www.couldsecu i yaliance.o g/download/
op- h ea s- o-cloud-compu ing-plus-indus y-insigh s/, 2017.
[16]
E ick Bauman, Zhiqiang Lin, and Ke in W. Hamlen. Supe se disassembly:
S a ically ew i ing x86 bina ies wi hou heu is ics. In P oceedings o Ne wo k
and Dis ibu ed Sys ems Secu i y Symposium (NDSS), 2018.
121
[65]
The iSCSI En e p ise Ta ge P ojec .
h p://iscsi a ge .sou ce o ge.
ne /, 2011.
[66]
T. Jim. SD3: A us managemen sys em wi h ce i ied e alua ion. In
P oceedings o IEEE Compu e Socie y Symposium on Resea ch in Secu i y
and P i acy, 2001.
[67]
Douglas Kilpa ick. P i man: A Lib a y o Pa i ioning Applica ions. In
P oceedings o USENIX Anual Technical Con e ence (ATC), 2003.
[68]
Koen Koning, Xi Chen, He be Bos, C is iano Giu ida, and Elias A hana-
sopoulos. No Need o Hide: P o ec ing Sa e Regions on Commodi y Ha dwa e.
In P oceedings o ACM Eu opean Con e ence on Compu e Sys ems (Eu oSys),
2017.
[69]
Ramak ishna Ko la, Tom Rodehe e , Ind aji Roy, Pa ick S uedi, and Ben-
jamin Wes e . Pas u e: Secu e O line Da a Access using Commodi y T us ed
Ha dwa e. In P oceedings o USENIX Symposium on Ope a ing Sys ems Design
and Implemen a ion (OSDI), 2012.
[70]
Robe K ahn, Bohdan T ach, Anjo Vahldiek-Obe wagne , Thomas Knau h,
P amod Bha o ia, and Ch is o Fe ze . Pesos: Policy Enhanced Secu e Objec
s o e. In P oceedings o ACM Eu opean Con e ence on Compu e Sys ems
(Eu oSys), 2018.
[71]
Dmi ii Ku aiskii, Oleksii Oleksenko, Se gei A nau o , Bohdan T ach, P amod
Bha o ia, Pascal Felbe , and Ch is o Fe ze . SGXBOUNDS: Memo y Sa e y o
Shielded Execu ion. In P oceedings o ACM Eu opean Con e ence on Compu e
Sys ems (Eu oSys), 2017.
128
[72]
Volodymy Kuzne so , László Szeke es, and Ma hias Paye . Code-poin e
in eg i y. In P oceedings o USENIX Symposium on Ope a ing Sys ems Design
and Implemen a ion (OSDI), 2014.
[73]
Ninghui Li and John C. Mi chell. Da alog wi h Cons ain s: A Founda ion o
T us Managemen Languages. In P oceedings o ACM Symposium on P ac ical
Aspec s o Decla a i e Languages (PADL), 2003.
[74]
James Li on, Anjo Vahldiek-Obe wagne , Eslam Elnike y, Deepak Ga g,
Bobby Bha acha jee, and Pe e D uschel. Ligh -Weigh Con ex s: An OS
Abs ac ion o Sa e y and Pe o mance. In P oceedings o USENIX Symposium
on Ope a ing Sys ems Design and Implemen a ion (OSDI), 2016.
[75]
Yu ao Liu, Tianyu Zhou, Kexin Chen, Haibo Chen, and Yubin Xia. Thwa ing
Memo y Disclosu e wi h E icien Hype iso -en o ced In a-domain Isola ion.
In P oceedings o ACM SIGSAC Con e ence on Compu e and Communica ions
Secu i y (CCS), 2015.
[76]
Boon Thau Loo. The Design and Implemen a ion o Decla a i e Ne wo ks.
PhD hesis, Uni e si y o Cali o nia, Be keley, 2006.
[77]
Boon Thau Loo, Tyson Condie, Joseph M. Helle s ein, Pe os Mania is, Timo-
hy Roscoe, Ion S oica, Thau Loo, Pe os Mania is, Tyson Condie, Timo hy
Roscoe, and Joseph M. Helle s ein. Implemen ing decla a i e o e lays. In
ACM SIGOPS Ope a ing Sys ems Re iew, olume 39, 2005.
[78]
Kangjie Lu, Chengyu Song, Byoungyoung Lee, Simon P. Chung, Taesoo Kim,
and Wenke Lee. ASLR-Gua d: S opping Add ess Space Leakage o Code
Reuse A acks. In P oceedings o ACM SIGSAC Con e ence on Compu e and
Communica ions Secu i y (CCS), 2015.
129
[79]
Michelle L. Mazu ek, Yuan Liang, William Meliche , Manya Sleepe , Lujo
Baue , G ego y R. Gange , Ni in Gup a, and Michael K. Rei e . Towa d s ong,
usable access con ol o sha ed dis ibu ed da a. In P oceedings o USENIX
Con e ence on File and S o age Technologies (FAST), 2014.
[80]
S ephen Mccaman and G eg Mo ise . E alua ing SFI o a CISC A chi ec u e.
In P oceedings o USENIX Secu i y Symposium, 2006.
[81]
Jona han M. McCune, Yanlin Li, Ning Qu, Zongwei Zhou, Anupam Da a,
Vi gil Gligo , and Ad ian Pe ig. T us iso : E icien cb educ ion and
a es a ion. In P oceedings o IEEE Symposium on Secu i y and P i acy
(Oakland), 2010.
[82]
Jona han M. McCune, B yan J. Pa no, Ad ian Pe ig, Michael K. Rei e , and
Hi oshi Isozaki. Flicke : An Execu ion In as uc u e o TCB Minimiza ion.
In P oceedings o ACM Eu opean Con e ence on Compu e Sys ems (Eu oSys),
2008.
[83]
F ank McKeen, Ilya Alexand o ich, I ai Ana i, D o Caspi, Simon Johnson,
Rebekah Leslie-Hu d, and Ca los Rozas. In el so wa e gua d ex ensions
suppo o dynamic memo y managemen inside an encla e. In P oceedings o
Ha dwa e and A chi ec u al Suppo o Secu i y and P i acy, 2016.
[84]
Aas ha Meh a, Eslam Elnike y, Ka u a Ha ey, Deepak Ga g, and Pe e
D uschel. Qapla: Policy compliance o da abase-backed sys ems. In P oceedings
o USENIX Secu i y Symposium, 2017.
[85]
Michael Mesnie , Feng Chen, Tian Luo, and Jason B. Ake s. Di e en ia ed
s o age se ices. In P oceedings o ACM Symposium on Ope a ing Sys ems
P inciples (SOSP), 2011.
130
[86]
Mike Mesnie , G ego y R. Gange , and E ik Riedel. Objec -based s o age.
IEEE Communica ions Magazine, 41(8), 2003.
[87]
Mic oso Co p. Bi locke .
h ps://docs.mic oso .com/en-us/windows/
secu i y/in o ma ion-p o ec ion/bi locke /bi locke -o e iew.
[88]
Mic oso Co p. Windows Backup and Res o e.
h p://www.mic oso .com/
a home/se up/backupda a.aspx{#} bid=l7X90d97alI.
[89]
Mic oso Co p. Wha Is Volume Shadow Copy Se ice?: Da a Re-
co e y.
h ps:// echne .mic oso .com/en-us/lib a y/cc757854( =ws.
10).aspx, 2003.
[90]
MITRE. CVE-2014-0160.
h ps://n d.nis .go / uln/de ail/
CVE-2014-0160, 2014.
[91] Node.js Founda ion. h ps://nodejs.o g.
[92] Oasis. eX ensible Access Con ol Ma kup Language, 2005.
[93]
OCZ Technology Inc. Dene a 2 Da a Shee .
h ps://d i e.google.com/
ile/d/0B4hWjkpwenosS0VMOWZkZ1B R1E/ iew?usp=sha ing, 2011.
[94]
Angelos Oikonomopoulos, Elias A hanasopoulos, He be Bos, and C is iano
Giu ida. Poking Holes in In o ma ion Hiding. In P oceedings o USENIX
Secu i y Symposium, 2016.
[95]
Oleksii Oleksenko, Dmi ii Ku aiskii, P amod Bha o ia, Pascal Felbe , and
Ch is o Fe ze . In el MPX Explained: A C oss-laye Analysis o he In el
MPX Sys em S ack. In P oceedings o ACM on Measu emen and Analysis o
Compu ing Sys ems, 2018.
[96]
OpenSSL. C yp o (OpenSSL c yp og aphic lib a y).
h p://www.openssl.
o g/, 2012.
131
[97]
B yan Pa no, Jona han M. McCune, and Ad ian Pe ig. Boo s apping T us
in Mode n Compu e s. In P oceedings o IEEE Symposium on Secu i y and
P i acy (Oakland), 2011.
[98]
Da id A. Pa e son, Ga h Gibson, and Randy H. Ka z. A case o edun-
dan a ays o inexpensi e disks (RAID). In P oceedings o ACM SIGMOD
in e na ional con e ence on Managemen o da a (SIGMOD), 1988.
[99]
Adam G. Penning on, John Linwood G i in, John S. Bucy, John D. S unk,
and G ego y R. Gange . S o age-Based In usion De ec ion. ACM T ansac ions
on In o ma ion and Sys em Secu i y, 13(4), 2010.
[100]
And ew Pimlo and Oleg Kiselyo . Sou ei, a Logic-Based T us -Managemen
Sys em. In P oceedings o In e na ional Symposium on Func ional and Logic
P og amming (FLOPS), 2006.
[101]
Sean Quinlan and Sean Do wa d. Ven i: a new app oach o a chi al da a
s o age. In P oceedings o USENIX Con e ence o File and S o age Technologies
(FAST), 2002.
[102]
E ik Riedel, Ch is os Falou os, Ga h a Gibson, and Da id Nagle. Ac i e Disks
o La ge Scale Da a P ocessing. Tc, 34(6), 2001.
[103] Rus language. h ps://www. us -lang.o g/.
[104]
Cai lin Sadowski, Edwa d A andilian, Alex Eagle, Liam Mille -Cushon, and
Cie a Jaspan. Lessons om building s a ic analysis ools a google. Commun.
ACM, 61(4):58–66, Ma ch 2018.
[105]
Je ome H. Sal ze and Michael D. Sch oede . The P o ec ion o In o ma ion
in Compu e Sys ems. In P oceedings o he IEEE, olume 63, 1975.
[106]
Samsung. 830 SSD da a shee .
h p://www.samsung.com/us/sys em/
consume /p oduc /mz/7p/c1/mz7pc128nam/830.pd , 2011.
132
[107]
Nuno San os, Rod igo Rod igues, K ishna P. Gummadi, and S e an Sa oiu.
Policy-sealed da a: A new abs ac ion o building us ed cloud se ices. In
P oceedings o USENIX Secu i y Symposium, 2012.
[108]
F ed B. Schneide , Ke in Walsh, and Emin Gün Si e . Nexus au ho iza ion logic
(NAL): Design a ionale and applica ions. ACM T ansac ions on In o ma ion
and Sys em Secu i y, 14(1), 2011.
[109]
Seaga e Technology LLC. Kine ic Open S o age Pla o m.
h p://www.
seaga e.com/solu ions/cloud/da a-cen e -cloud/pla o ms.
[110]
Seaga e Technology LLC. Sel -Enc yp ing Ha d Disk D i es in he Da a Cen e .
Technical Repo TP583, 2007.
[111]
Seaga e Technology LLC. Ba acuda Da a Shee .
h p://www.seaga e.com/
iles/s a ic iles/docs/pd /da ashee /disc/ba acuda-x -ds1696.
3-1102us.pd , 2012.
[112]
Seaga e Technology LLC. Momen us XT Da a Shee .
h p://www.seaga e.
com/docs/pd /da ashee /disc/ds_momen us_x .pd , 2012.
[113]
Ho a Shacham, Ma hew Page, Ben P a , Eu-Jin Goh, Nagend a Modadugu,
and Dan Boneh. On he e ec i eness o add ess-space andomiza ion. In
P oceedings o ACM SIGSAC Con e ence on Compu e and Communica ions
Secu i y (CCS), 2004.
[114]
Moni ul I. Sha i , Wenke Lee, Weidong Cui, and And ea Lanzi. Secu e in- m
moni o ing using ha dwa e i ualiza ion. In P oceedings o ACM SIGSAC
Con e ence on Compu e and Communica ions Secu i y (CCS), 2009.
[115]
Lei Shi, Yuming Wu, Yubin Xia, Na han Dau enhahn, Haibo Chen, Binyu
Zang, Haibing Guan, and Jiñming Li. Decons uc ing xen. In P oceedings o
Ne wo k and Dis ibu ed Sys em Secu i y Symposium (NDSS), 2017.
133
[116]
Jiwu Shu, Zhi ong Shen, and Wei Xue. Shield: A s ackable secu e s o age
sys em o ile sha ing in public s o age. J. Pa allel Dis ib. Compu ., 74(9),
Sep embe 2014.
[117]
Rui Shu, Peipei Wang, Sigmund A. Go ski III, Benjamin Andow, Adwai
Nadka ni, Luke Desho els, Jason Gion a, William Enck, and Xiaohui Gu. A
S udy o Secu i y Isola ion Techniques. ACM Compu ing Su eys, 49(3), 2016.
[118]
Emin Gün Si e , Willem de B uijn, Pa ick Reynolds, Alan Shieh, Ke in Walsh,
Dan Williams, and F ed B. Schneide . Logical a es a ion: an au ho iza ion
a chi ec u e o us wo hy compu ing. In P oceedings o ACM Symposium
on Ope a ing Sys ems P inciples (SOSP), 2011.
[119]
Gopalan Si a hanu, Swamina han Sunda a aman, and E ez Zadok. Type-sa e
disks. In P oceedings o USENIX Symposium on Ope a ing Sys ems Design
and Implemen a ion (OSDI), 2006.
[120]
Mu hian Si a hanu, Vijayan P abhaka an, Flo en ina I. Popo ici, Timo hy E.
Denehy, And e A paci-Dusseau, and Remzi A paci-Dusseau. Seman ically-
Sma Disk Sys ems. In P oceedings o USENIX Con e ence o File and S o age
Technologies (FAST), 2003.
[121] SQLi e. h ps://www.sqli e.o g.
[122] SQLi e. Speed es 1. h ps://www.sqli e.o g/ es ing.h ml.
[123]
S o age Wo k G oup o he T us ed Compu ing G oup. Sel -Enc yp ing D i es
Take o o S ong Da a P o ec ion.
h ps:// us edcompu ingg oup.o g/
sel -enc yp ing-d i es- ake-o -s ong-da a-p o ec ion/, 2010.
[124]
John D. S unk, Ga h R. Goodson, Michael L. Scheinhol z, C aig A. N.
Soules, and G ego y R. Gange . Sel -Secu ing S o age: P o ec ing Da a in
134
Comp omised Sys ems. In P oceedings o USENIX Symposium on Ope a ing
Sys ems Design and Implemen a ion (OSDI), 2000.
[125] Sun Mic osys ems. Sola is ZFS, 2009.
[126]
Laszlo Szeke es, Ma hias Paye , Tao Wei, and Dawn Song. SoK: E e nal
wa in memo y. In P oceedings o IEEE Symposium on Secu i y and P i acy
(Oakland), 2013.
[127]
TCG. TCG S o age A chi ec u e Co e Spec-
i ica ion.
h ps:// us edcompu ingg oup.o g/
cg-s o age-a chi ec u e-co e-speci ica ion/, 2007.
[128]
Eno The eska, Hi esh Ballani, G eg O’Shea, Thomas Ka agiannis, An Row-
s on, Tom Talepy, Richa d Black, and Timo hy Zhu. Io low: A so wa e-de ined
s o age a chi ec u e. In P oceedings o ACM Symposium on Ope a ing Sys ems
P inciples (SOSP), 2013.
[129]
Robe Wahbe, S e en Lucco, Thomas E. Ande son, and Susan L. G aham.
E icien so wa e-based aul isola ion. In P oceedings o ACM Symposium on
Ope a ing Sys ems P inciples (SOSP), 1993.
[130]
Ke in Walsh and F ed B. Schneide . Cos s o Secu i y in he PFS File Sys em.
Technical epo , Compu ing and In o ma ion Science, Co nell Uni e si y, 2012.
[131]
Da id H.D. Wa en. an Abs ac P olog Ins uc ion Se . Technical Repo
Technical No e 309, SRI In e na ional, 1983.
[132]
Robe N. M. Wa son, Jona han Ande son, Ben Lau ie, and K is Kennaway.
A as e o Capsicum. Communica ions o he ACM, 55(3), 2012.
[133]
Ca s en Weinhold and He mann Hä ig. VPFS: Building a i ual p i a e
ile sys em wi h a small us ed compu ing base. In ACM SIGOPS Ope a ing
Sys ems Re iew, 2008.
135
[134]
Ca s en Weinhold and He mann Hä ig. jVPFS: Adding Robus ness o a
Secu e S acked File Sys em wi h Un us ed Local S o age Componen s. In
P oceedings o USENIX Anual Technical Con e ence (ATC), 2011.
[135]
Jan We ne , Geo ge Bal as, Rob Dalla a, Na han O e ness, Ke in Z. Snow,
Fabian Mon ose, and Michalis Polych onakis. No-Execu e-A e -Read: P e-
en ing Code Disclosu e in Commodi y So wa e. In P oceedings o ACM
SIGSAC Asia Con e ence on Compu e and Communica ions Secu i y (Asia
CCS), 2016.
[136]
Wikimedia Founda ion. Image Dump.
h p://a chi e.o g/de ails/
wikimedia-image-dump-2005-11, 2005.
[137]
Wikimedia Founda ion. S a ic HTML dump.
h p://dumps.wikimedia.o g/
,
2008.
[138]
Wikimedia Founda ion. Page iew s a is ics Ap il 2012.
h p://dumps.
wikimedia.o g/o he /pagecoun s- aw/2012/2012-04/, 2012.
[139]
Edwa d Wobbe , Ma ín Abadi, Michael Bu ows, and Bu le Lampson. Au-
hen ica ion in he Taos ope a ing sys em. ACM T ansac ions on Compu e
Sys ems, 12(1), 1994.
[140]
Ted Wobbe , Aydan Yume e endi, Ma ín Abadi, And ew Bi ell, and Daniel R.
Simon. Au ho izing applica ions in singula i y. In ACM SIGOPS Ope a ing
Sys ems Re iew, olume 41, 2007.
[141]
Rubin Xu, Hassen Saïdi, and Ross Ande son. Au asium: P ac ical Policy
En o cemen o And oid Applica ions. In P oceedings o USENIX Secu i y
Symposium, 2012.
136
[142]
Yuanzhong Xu, Alan M. Dunn, Owen S. Ho mann, Michael Z. Lee, Syed Akba
Mehdi, and Emme Wi chel. Applica ion-de ined decen alized access con ol.
In P oceedings o USENIX Anual Technical Con e ence (ATC), 2014.
[143]
Benne Yee, Da id Seh , G ego y Da dyk, J. B adley Chen, Robe Mu h,
Ta is O mandy, Shiki Okasaka, Neha Na ula, and Nicholas Fullaga . Na i e
clien : A sandbox o po able, un us ed x86 na i e code. In P oceedings o
IEEE Symposium on Secu i y and P i acy (Oakland), 2009.
[144]
Yajin Zhou, Xiaoguang Wang, Yue Chen, and Zhi Wang. ARMlock: Ha dwa e-
based Faul Isola ion o ARM Yajin. In P oceedings o ACM SIGSAC Con e -
ence on Compu e and Communica ions Secu i y (CCS), 2014.
137