TORMENT OpenACC2016: A benchma king ool o OpenACC compile s
Daniel Ba ba, A u o Gonzalez-Esc ibano, Diego R. Llanos
Dp o. de In o m´
a ica
Uni e sidad de Valladolid
Valladolid, Spain
daniel@in o .u a.es, a u o@in o .u a.es, diego@in o .u a.es
Abs ac —OpenACC is a pa allel p og amming model o
ha dwa e accele a o s, such as GPUs o Xeon Phi, which
has been in de elopmen o se e al yea s by now. Du ing
his ime, di e en compile s ha e appea ed, bo h comme -
cial and open sou ce, which a e s ill on de elopmen s age.
Due o he ac ha bo h he OpenACC s anda d and i s
implemen a ions a e ela i ely ecen , we p opose a bench-
ma k sui e speci ically designed o check he pe o mance o
he OpenACC ea u es in he code gene a ed by di e en
compile s on di e en a chi ec u es. Ou benchma k sui e is
named TORMENT OpenACC2016. Along wi h his ool we
ha e de eloped an adequa e me ic o he compa ison o
pe o mance among di e en machine-compile pai s which
we ha e named TORMENT ACC2016 Sco e. The e sion 1 o
TORMENT OpenACC2016 p esen ed in his pape , con ains six
benchma ks, and is a ailable online.
Keywo ds-OpenACC, compile s, benchma king.
I. INTRODUCTION
OpenACC is an open s anda d which de ines a numbe
o compila ion di ec i es o p agmas o pa allel code ag-
men s execu ion using ha dwa e accele a o s such as GPUs
and Xeon Phi. I s objec i e is o ease pa alleliza ion o
sequen ial code using his kind o accele a o s and educing
he equi ed ime bo h o lea ning and coding [1]. The
OpenACC speci ica ion is cu en ly on i s 2.5 e sion [2].
A he ime o w i ing his pape , he e a e se e al bench-
ma ks o OpenACC. Fo example, he EPCC OpenACC
Benchma k Sui e [3] has been de eloped by he Edinbu gh
Pa allel Compu ing Cen e, and consis s on bo h a numbe
o mic obenchma ks in ended o check he compliance o
he s anda d and measu e he o e head o di e en p agmas
implemen a ion. I also measu es he gene al pe o mance o
OpenACC gene a ed code o each benchma k, bu se e al
compile s lack ull suppo o all he ea u es o his
ool ye . This implies ha i is no s aigh o wa d o do
compa a i e pe o mance analysis o di e en compile s. On
he o he hand, he e a e o he benchma k sui es no o igi-
nally designed o OpenACC. Fo example, Rodinia [4] o
OpenACC is a po o he o iginal benchma ks om he sui e
o he same name [5], de eloped by Pa hscale. As i is no
speci ically designed o OpenACC, mos o he benchma ks
included in his sui e p esen many compila ion p oblems
wi h he s a e-o - he-a OpenACC compile s, and hey do
no co e all o he OpenACC ea u es in a sys ema ic way.
I would be in e es ing o he communi y o ha e a bench-
ma k sui e ha co e s he main ea u es o he OpenACC
s anda d and easily allows o compa e he imp o emen s
in code gene a ion o di e en a chi ec u es in e ms o
pe o mance. The con ibu ion o his wo k includes:
•A benchma k sui e designed o OpenACC, ha co e s
he main ea u es o he s anda d based on p og am-
ming pa e ns o di e en kinds o eal applica ions.
•A p oposal o a me ic o p o ide a ela i e sco e
which allows o he compa ison o he pe o mance
ob ained by he implemen a ion o di e en compile s
on di e en pla o ms. The design o his me ic allows
o classi y he esul s in a global anking. We named i
TORMENT ACC2016 Sco e.
•A compila ion-execu ion w appe , along wi h a sys em-
a ic way o include in he sou ce code condi ional com-
pila ion s a emen s o adap he OpenACC syn ax and
de ails o he speci ic pa icula i ies o each compile .
This w appe allows o easily adap he benchma ks
o di e en OpenACC compile s and pla o ms. I also
au oma izes he gene a ion o a inal epo including
he me ic measu e and o he ele an da a o he
compa isons.
We p esen TORMENT OpenACC2016. I is a ool ha
includes he benchma ks and he w appe ha au oma izes
he compila ion and execu ion. The gene a ed epo s a e
p esen ed in a s anda d and easy eadable way, including
in o ma ion abou he execu ion pla o m, he compile s,
s a is ics o execu ion imes, and he inal alues o he
me ic o each case. The epo also o e s s a is ics o
execu ion imes o sequen ial and na i e CUDA e sions o
he benchma ks o u he e e ence. The ool, in i s cu en
s a e, has been alida ed o h ee di e en compile s in wo
di e en GPU pla o ms. The suppo o he used compile s
o he gene a ion o Xeon Phi code is no ye ma u e enough
o ex end ou alida ion o hese kind o de ice.
The es o he pape is s uc u ed as ollows: Sec ion
II desc ibes wi h mo e de ail p e ious benchma king ools.
Sec ion III discusses he ma u i y o OpenACC compile s.
Sec ion IV desc ibes TORMENT OpenACC2016, including
i s goals, s uc u e o he ool, implemen ed benchma ks,
he p oposed me ic, and commen s he epo ob ained in
he expe imen al alida ion. Finally, sec ion V concludes he
pape .
II. EXISTING TOOLS
As i was s a ed in he in oduc ion, a he ime o w i ing
his pape he e a e p e ious pe o mance e alua ion ools
o OpenACC. In his sec ion we will desc ibe mo e ho -
oughly examples o hese ools and he p oblems de ec ed
o use hem o he goals o his wo k.
A. EPCC OpenACC Benchma k Sui e
This benchma k sui e de eloped by he EPCC has been
designed speci ically o OpenACC. I is composed o h ee
di e en pa s, named Le el 0,Le el 1 yApplica ion Le el.
Le el 0 con ains se e al mic obenchma ks whose goal is
measu ing he o e head caused by he implemen a ion o
he di e en p agmas. These mic obenchma ks o e a esul
calcula ed as he di e ence be ween he execu ion imes o
wo di e en e sions o he use o he OpenACC p agmas.
Fo #p agma acc da a di ec i e i simply measu es he
equi ed ime o da a ans e s.
Le el 1 con ains a se o BLAS ype benchma ks based
on Polybench and Polybench/GPU [6]. The esul s o e ed
by hese benchma ks a e he execu ion ime o he di e en
code agmen s. This means hey a e a good pe o mance
indica o o he OpenACC gene a ed code. Howe e , in he
cu en s a e o de elopmen o he di e en compile s some
o he benchma ks use unsuppo ed p agmas. Due o his,
hese benchma ks p esen compila ion o un ime p oblems
wi h a numbe o compile s.
Applica ion Le el con ains h ee benchma ks o a bigge
complexi y han hose om he p e ious sec ion. These
benchma ks also o e as hei esul s he execu ion ime
o he gene a ed code.
The EPCC OpenACC Benchma k Sui e is well planned
and designed. Howe e , he esul s ob ained om he mi-
c obenchma ks a e no adequa e enough o a ela i e pe -
o mance analysis. The o he benchma ks om Le el 1
and Applica ion Le el could be adequa e. Howe e , since
hey a e no speci ically designed o es OpenACC ea u es
suppo ed by all o he a ailable compile s, hey p esen
p oblems. In some cases hey couldn’ be compiled wi h
some compile s, and some imes hey p oduced un ime
e o s.
B. Rodinia o OpenACC
Pa hscale Inc. has de eloped an OpenACC e sion [4] o
he Rodinia benchma k sui e [5], [7]. The cu en e sion a
he ime o w i ing his pape has been published on Gi Hub
(Ap il, 24 h 2014). The sui e is composed o benchma ks
ha e u n as hei esul he execu ion ime. This means ha
hey a e a good s a ing poin o a pe o mance analysis.
The main issues a e he low ma u i y le el o he OpenACC
use in he code, and he lack o compa ibili y wi h he
a ailable compile s. These issues make i impossible o
es ablish a ai pe o mance compa ison.
III. OPENACC COMPILERS
The e is a numbe o choices among compile s wi h
suppo o OpenACC, bo h comme cial and ee. In his
wo k we ocus on hose compile s which a e a ailable
o ee o wi h academic o ial licenses. In gene al, he
OpenACC compile s ha e cu en ly e y limi ed suppo o
any pla o m o he han GPUs. The compile s conside ed
a e:
A. PGI Compile
The PGI compile [8], de eloped by The Po land G oup
and N idia, is, a he ime o w i ing, he compile wi h he
highes ma u i y le el. In ou s udies, he PGI compile has
shown o be he mos s able and he highes ma u i y le el
among he used compile s. I has a p ope implemen a ion
o mos o he OpenACC speci ica ion.
B. OpenUH
OpenUH compile , de eloped by he Uni e si y o Hous-
on, is an open sou ce p ojec . Acco ding o ou es s and
compa ed o he PGI compile , i s ma u i y and obus ness
a e lowe and i lacks se e al unc ionali ies.
C. accULL
The accULL compile [9] has been de eloped by Uni-
e sidad de La Laguna, Spain, and i is an open sou ce
compile like OpenUH. Ou es s show ha compa ed o he
p e ious compile s, his is he one wi h a lowe obus ness,
leading o se e al issues du ing sou ce o sou ce ansla ion,
no suppo ing se e al C cha ac e is ics, like unc ion poin -
e s. Se e al OpenACC unc ionali ies a e also lacking, like
se e al ypes o educ ion.
IV. TORMENT OPENACC2016
Ou p oposal is implemen ed as a ool called TORMENT
OpenACC2016, an ac onym o T asgo pe ORMance and
E alua ioN Tool o OpenACC. The p ojec was bo n o
gi e esponse o he need o pe o mance analysis and
compa ison o OpenACC code gene a ed by he a ious
compile s implemen ing he s anda d.
A. Goals
The main goal o TORMENT OpenACC2016 is o allow
a pe o mance analysis o OpenACC code gene a ed by
di e en compile s in an easy way, gene a ing a esul
summa y ha can be easily analyzed and which o e s he
TORMENT ACC2016 Sco e me ics. This me ic is use ul o
compa e machine-compile pai s in e ms o pe o mance.
Ou p oposal in ends o o e a ool speci ically p epa ed
o OpenACC, de eloped aking in o accoun he ea ly
de elopmen s age o he a ailable compile s. Thus, we y o
ensu e ha compila ion o un ime p oblems a e a oided o
he compile s conside ed, unlike in he p e ious benchma k
sui es a ailable. TORMENT OpenACC2016 is p epa ed o
be compiled and execu ed wi h minimum in e en ion om
he use . The sc ip s accompanying he sui e ge in o ma ion
om he benchma king p ocess and inally o e a HTML
epo wi h he mos ele an da a.
Mo eo e , TORMENT OpenACC2016 makes use o he
GCC and NVCC compile s in o de o ob ain da a om
sequen ial and CUDA-based code execu ions. Wi h his da a,
ou p oposal can o e o he use in o ma ion abou he
speedup compa ed wi h sequen ial and CUDA code execu ed
in he same machine.
B. S uc u e o ou ool
TORMENT OpenACC2016 is composed o a numbe o
shell sc ip s which a e in cha ge o he compila ion, exe-
cu ion and esul s ga he ing p ocesses, emo ing he bu den
om he use . Each benchma k has been chosen o ep esen
a ele an class o p oblems, and has been implemen ed o be
as simple as possible, in o de o a oid compila ion issues.
The benchma ks a e launched en imes, wi h an ex a
in oca ion a he beginning whose esul s a e disca ded. The
alues ha co espond o he bes o he execu ions and
he a i hme ic mean o all o hem a e eco ded o la e
calcula e he p oposed me ic. These alues a e named peak
and a e age espec i ely.
C. Fea u es Co e age
In o de o make a selec ion o benchma ks o ou sui e
we ha e analyzed he de elopmen s a us o he di e en
compile s. We ha e s udied hei capabili ies and hei
comple eness o he OpenACC s anda d. Ou conclussion
is ha he exis ing benchma ks, based on Polybench and
Rodinia, o en equi e p agmas ha a e no implemen ed ye
on one o he compile s a leas . To achie e ull compa ibili y
among he di e en compile s we need o use only he se o
di ec i es a ailable o all o hem a he ime o de eloping
his e sion o TORMENT OpenACC.
We lis below a numbe o GPU cha ac e is ics ha we
a gue a e impo an o be es ed o he cu en s a us o
compile s’ ma u i y:
•Use o sha ed memo y
•In ensi e and non-balanced compu a ion
•Compu a ion wi hou memo y access
•Da a ans e s
•Non-pe ec ly coalescen memo y access
These a e impo an cha ac e is ics ha play a pi o al ole
in he esul ing code o manually w i en CUDA ke nels.
The code gene a ed by he OpenACC compile s will also
be a ec ed by hese cha ac e is ics, hus he pe o mance o
hese gene a ed codes could be measu ed and compa ed.
As mos o he well-known benchma ks equi e he use o
non-suppo ed di ec i es o one compile a leas , we ha e
decided o ollow a bo om-up app oach o de eloping a
wo king sui e o benchma ks, ying o co e he cha ac e -
is ics lis ed abo e. This decision lea es us wi h a se o e y
simple benchma ks, use ul o a compa a i e analysis o he
pe o mance o OpenACC gene a ed code using di e en
compile s, bu no o a ho ough analysis o comple eness,
obus ness and pe o mance o a single compile . Ou wo k
aims o ob ain a ool capable o gene a ing a compa a i e
analysis epo o a wide numbe o compile s.
As he suppo o OpenACC di ec i es imp o es, his
design decision should also be e iewed and adap ed in o de
o use benchma ks o a highe complexi y as he ones p o-
ided in sui es like Polybench (used in he EPCC OpenACC
benchma k sui e), Rodinia, and he CUDA Toolki .
D. Implemen ed Benchma ks
Ve sion 1 o TORMENT OpenACC2016, con ains a se o
six benchma ks. These benchma ks y o co e he di e en
cha ac e is ics enume a ed in he p e ious sec ion, as i is
shown in Table I.
1) T Mon eCa loPi: This is an emba assingly pa allel,
wi hou dependencies, pe ec ly egula , well balanced wi h
a s a ic pa i ion, and compu a ional bound example. This
benchma k consis s on an app oxima ion o Pi using he
Mon e Ca lo me hod. This me hod is based on he andom
gene a ion o coo dina es in a uni squa e. Fo each coo di-
na e, i is checked whe he o no ha coo dina e is loca ed
inside a qua e ci cle and, i ha condi ion is sa is ied, hen
he coo dina e is accumula ed. Finally, he ollowing o mula
is applied:
π≈4∗P
T
Whe e Pis he numbe o coo dina es inside he qua e
ci cle and Tis he o al o gene a ed coo dina es.
T Mon eCa loPi is a benchma k ha con ains no memo y
ans e s and a e y simple compu a ion. Howe e , i can be
op imized i each h ead compu es se e al coo dina es and
i he sha ed memo y in he h ead blocks is used in o de
o a oid global memo y access.
The unc ion used o andom numbe gene a ion can be
p oblema ic. OpenACC code should use unc ions ha can
be o loaded o he GPU and a he same ime can be used
on he hos machine. Thus, we canno use na i e CUDA
unc ions like he ones included in he cu and lib a y. Ou
solu ion has been o eplica e he implemen a ion o he
s and unc ion in he s anda d C lib a y.
2) T S ingMa ch: This example is coalesced wi h low
memo y bandwid h and i egula da a-dependan loads pe
h ead. The T S ingMa ch benchma k is a s ing ma ching
p og am. This algo i hm is in e es ing because i combines
da a ans e wi h he need o e icien use o he memo y,
specially he sha ed memo y. I consis s on he sea ch o
he i s occu ence o a small s ing in a la ge one, using
anai e algo i hm. In his benchma k, he la ge s ing has a
Sha ed
Memo y
In ensi e
Compu a ion
Compu a ion wi hou
memo y access
Da a
T ans e
Non-pe ec ly coalescen
Memo y Access
T Mon eCa loPi X X
T S ingMa ch X X
T 3DS encil X
T Mandelb o X
T Ma ixMul X X
T Re e seA ay X X X
Table I
GPU CHARACTERISTICS TESTED BY THE BENCHMARKS
size o 10 million cha ac e s, ha is, app oxima ely 10MB.
We sea ch ou small s ings o 1000 cha ac e s.
3) T 3DS encil: This is a egula , well balanced example
wi h dependencies ha lead o i e a i e neighbou synch o-
niza ion, and low load pe h ead on each i e a ion. The
T 3DS encil benchma k consis s on a 6 poin 3D s encil.
Each poin o a 3D ma ix is upda ed wi h he a i hme ic
mean o i s neighbou s. This benchma k is in e es ing be-
cause, adi ionally, s encil p og ams can be e y e icien
when execu ed on a GPU. I includes memo y ans e s only
a he beginning and end o he compu a ion.
4) T Mandelb o : This example is emba assingly pa -
allel wi hou dependencies wi h non-balanced load. The
T Mandelb o implemen s he mos amous o he ac al
se s. The implemen ed algo i hm is he escape- ime algo-
i hm. This is an emba assingly pa allel p oblem and he e
a e no dependencies among he compu a ions. The e o e,
he e is no need o use he sha ed memo y. Global memo y
access is also negligible. On he o he hand, he compu a ion
is e y in ensi e, depending on he maximum numbe o
i e a ions. This benchma k should scale e y well on GPU.
5) T Ma ixMul : This example has pe ec ly egula
loads wi h coalesced memo y access i sha ed memo y is
p ope ly used. Ma ix mul iplica ion is widely used in many
domains. I has a la ge ma gin o op imiza ions. When
using a GPU, he mos impo an op imiza ion is he co ec
use o he sha ed memo y, which equi es a co ec h ead
block geome y. As his benchma k has h ee nes ed loops,
i p esen s ce ain complexi y o he OpenACC compile s.
They need o make choices a he di e en le els o pa al-
lelism, including also he geome y o he h ead blocks.
6) T Re e seA ay: This las example has a egula load,
bu a di ec implemen a ion o non-aligned a ay sizes de-
i es in coalesced eads bu non-pe ec ly coalesced w i es.
The las o he benchma ks implemen ed in TORMENT
OpenACC2016 consis s on a e y simple ope a ion; e e s-
ing an in ege a ay. Al hough his seems o be a i ial
ope a ion, unning i on a GPU p esen s a ele an p oblem.
Re e sing an a ay on a GPU can be e y ine icien because
o non-coalescen global memo y access. The co ec solu-
ion in ol es using he sha ed memo y as an in e media e
bu e whe e a pa ial e e se is done. This allows o
coalescen global memo y access.
E. P oposed Me ic
Fo he selec ion o he me ic named
TORMENT ACC2016 Sco e, we use he same me hodology
as SPEC [10] (S anda d Pe o mance E alua ion
Co po a ion). I is a widely known e e ence in e ms
o benchma king and pe o mance analysis. One o i s
s eng hs is o acknowledge ha benchma ks ge olde as
ime passes and, in consequence, hey mus be upda ed.
SPEC uses he ollowing me hodology. Fi s , he p og am
e u ns i s execu ion ime. Then he SPEC a io is calcula ed,
which consis s on he a io ob ained om di iding he e e -
ence execu ion ime (supplied by SPEC) and he execu ion
ime ob ained by he benchma k. Finally, he geome ic mean
o all he SPEC a ios is calcula ed [11].
Ou p oposal ollows a simila idea bu wi h se e al
changes. Fi s , he execu ion o he benchma ks con ained
in TORMENT OpenACC2016 e u n h ee alues. Peak Time
is he bes execu ion ime ob ained, measu ed in seconds.
A e age Time is he a i hme ic mean o all he execu ions o
he same benchma k, also in seconds. Finally, he S anda d
De ia ion o he se o imes ob ained gi es an idea o he
a iabili y among he se e al execu ions o he benchma ks.
Wi h he peak and a e age imes, a a io is calcula ed di id-
ing he e e ence imes p o ided by he ool, which a e he
execu ion imes o he sequen ial e sion o he benchma ks
in a e e ence machine. The cha ac e is ics o his e e ence
machine a e shown in h ps:// o men .in o .u a.es. Finally, i
compu es he ha monic mean o all he a ios, bo h o peak
ime as a e age ime. These alues a e he TORMENT ACC
Peak Sco e and TORMENT ACC A e age Sco e.
The main di e ence be ween TORMENT OpenACC2016
and SPEC me ics, apa om he ime measu emen
me hodology, consis s on he use o he ha monic mean
ins ead o he geome ic mean. Fo he goals o TORMENT
OpenACC2016, he ha monic mean is mo e adequa e han
he geome ic mean. Fi s , e en i he geome ic mean
always p oduces a consis en o de ing, i ’s no necessa ily
he co ec o de ing [11]. This is because his mean is no
in e sely p opo ional o he execu ion ime. In con as , he
ha monic mean is in e sely p opo ional o he execu ion
ime, which makes i an adequa e mean o a ios (see e.g.
[12]). TORMENT ACC2016 Sco e is a HB (Highe is Be e )
me ic.
An example o usage can be ound on
h ps:// o men .in o .u a.es. The TORMENT Repo
Example link shows a epo gene a ed o one o ou
machines. The i s pa o he epo consis s on sys em
in o ma ion. A e ha sec ion, he esul ables o
sequen ial and CUDA code a e ound. These esul s
a e me ely in o ma i e, and hey a e no used in he
TORMENT ACC2016 Sco e. Howe e , hese esul s allow
he use o compa e he pe o mance o he OpenACC
code wi h sequen ial and CUDA e sions execu ed in he
same machine. A e he sequen ial and CUDA esul s,
he OpenACC esul s a e shown o each compile used.
These esul s consis on he ables, wi h he execu ion
ime s a is ics, he TORMENT ACC2016 Sco e, and he
speedup alues using as e e ence he sequen ial and CUDA
execu ions.
In he Repo Example a ailable online, he PGI compile
ob ains he bes esul s, ob aining a 64% o he pe o mance
achie ed wi h CUDA implemen a ions. OpenUH ge s a
second posi ion, wi h a 41% o he pe o mance achie ed
wi h he op imized CUDA implemen a ions. Finally, he
accULL compile ge he hi d posi ion, wi h 16% o he
pe o mance achie ed wi h CUDA.
The TORMENT ACC2016 Sco e also allows o a com-
pa ison among compile -machine pai s. Fo example, he
Hyd a Sys em in he Repo Example a ailable online
has Ti an Black GPUs, and we also an he benchma k
sui in a lap op which has a N idia GTX 850M. The
TORMENT ACC2016 Peak Sco e in Hyd a is 28.69, while
in he lap op is 8.45. On he o he hand he accULL compile
ob ains 7.26 in Hyd a and 4.98 in he lap op. This indica es
he impo an di e ences o pe o mance in oduced by he
code gene a ion and op imiza ion echniques used by bo h
compile s.
V. CONCLUSIONS AND FUTURE WORK
TORMENT OpenACC2016 is a pe o mance analysis and
compa ison ool o OpenACC gene a ed code, aking in o
conside a ion he ma u i y le el o he suppo o OpenACC
in di e en compile s. TORMENT OpenACC2016 con ains
a sui e o benchma ks speci ically designed o OpenACC
and main aining he maximum po abili y.
The esul s o e ed by TORMENT OpenACC2016 include
he so called TORMENT ACC2016 Sco e, designed o he
compa ison o machine-compile pai s. Mo eo e , i o e s a
esul epo o he benchma ks, including a able wi h exe-
cu ion imes s a is ics (bo h o peak and a e age alues) and
he s anda d de ia ion, and also including execu ion esul s
om sequen ial and CUDA code gene a ed by GCC and
NVCC compile s. This o e s he use a compa a i e analysis
o he OpenACC gene a ed code e sus he sequen ial and
CUDA e sions o he benchma ks.
Fu u e wo k consis s on he inclusion o new benchma ks,
specially when he suppo ed compile s become mo e ma-
u e, including a iche s uc u e wi h di e en le els (such
as BLAS and eal wo ld applica ions) in addi ion o he
exis ing syn he ic applica ions. Mo eo e , an impo an pa
o ou u u e wo k consis s on main aining compa ibili y
among he suppo ed compile s and new addi ions. We a e
in he p ocess o adding suppo o GCC, Omni (Uni e si y
o Tsukuba), OpenARC (Oak Ridge Na ional Labo a o y),
and RoseACC (Uni e si y o Delawa e).
ACKNOWLEDGMENTS
This esea ch has been pa ially suppo ed by MICINN
(Spain) and ERDF p og am o he Eu opean Union:
HomP og-He Sys p ojec (TIN2014-58876-P), and COST
P og am Ac ion IC1305: Ne wo k o Sus ainable Ul ascale
Compu ing (NESUS).
REFERENCES
[1] OpenACC-s anda d.o g. Abou OpenACC. [Online]. A ail-
able: h p://www.openacc.o g/abou -openacc
[2] ——. (2015, oc ) OpenACC 2.5 d a o public commen .
[Online]. A ailable: h p://www.openacc.o g/con en /openacc-
25-d a -public-commen
[3] EPCC. (2013, sep) Epcc OpenACC benchma k sui e.
h ps://gi hub.com/EPCCed/epcc-openacc-benchma ks.
[4] Pa hscale. (2014, ap ) Rodinia benchma k sui e 2.1 wi h
OpenACC po . h ps://gi hub.com/pa hscale/ odinia.
[5] S. Che, M. Boye , J. Meng, D. Ta jan, J. W. Shea e , S.-H.
Lee, and K. Skad on, “Rodinia: A benchma k sui e o he -
e ogeneous compu ing,” in Wo kload Cha ac e iza ion, 2009.
(IISWC), 2009 IEEE In e na ional Symposium on. IEEE,
2009, pp. 44–54.
[6] L.-N. Pouche , “Polybench: The polyhed al
benchma k sui e,” URL: h p://www. cs. ucla. edu/˜
pouche /so wa e/polybench/[ci ed July,], 2012.
[7] S. Che, J. W. Shea e , M. Boye , L. G. Sza a yn, L. Wang,
and K. Skad on, “A cha ac e iza ion o he Rodinia bench-
ma k sui e wi h compa ison o con empo a y CMP wo k-
loads,” in Wo kload Cha ac e iza ion (IISWC), 2010 IEEE
In e na ional Symposium on. IEEE, 2010, pp. 1–11.
[8] PGI. (2015, no ) Pgi accele a o compile s wi h OpenACC
di ec i es. h ps://www.pg oup.com/ esou ces/accel.h m.
[9] R. Reyes, I. L´
opez-Rod ´
ıguez, J. J. Fume o, and F. de Sande,
“accULL: an OpenACC implemen a ion wi h CUDA and
OpenCL suppo ,” in Eu o-Pa 2012 Pa allel P ocessing.
Sp inge , 2012, pp. 871–882.
[10] K. M. Dixi , “The spec benchma ks,” Pa allel compu ing,
ol. 17, no. 10, pp. 1195–1209, 1991.
[11] D. J. Lilja, Measu ing compu e pe o mance: a p ac i ione ’s
guide. Camb idge Uni e si y P ess, 2005.
[12] J. R. Mashey, “Wa o he benchma k means: ime o a uce,”
ACM SIGARCH Compu e A chi ec u e News, ol. 32, no. 4,
pp. 1–14, 2004.