Multi-Device Controllers: A Library To Simplify The Parallel Heterogeneous Programming

... manusc ip No. (will be inse ed by he edi o ) Mul i-De ice Con olle s: A Lib a y To Simpli y The Pa allel He e ogeneous P og amming Ana Mo e on-Fe nandez ·A u o Gonzalez-Esc ibano ·Diego R. Llanos Recei ed: da e / Accep ed: da e Abs ac Cu en HPC clus e s a e composed by se e al machines wi h di - e en compu a ion capabili ies and di e en kinds and amilies o accele a o s. P og amming e icien ly o hese he e ogeneous sys ems has become an im- po an challenge. The e a e many p oposals o simpli y he p og amming and managemen o accele a o de ices, and he hyb id p og amming, mixing ac- cele a o s and CPU co es. Howe e , he po abili y comp omises in many cases he e iciency on di e en de ices, and he e a e de ails abou he coo dina ion o di e en ypes o de ices ha should be s ill ackled by he p og amme . In his wo k we in oduce he Mul i-Con ole (MC l), an abs ac en i y implemen ed in a lib a y, ha coo dina es he managemen o he e ogeneous de ices, including accele a o s wi h di e en capabili ies and se s o CPU- co es. Ou p oposal imp o es s a e-o - he-a solu ions, simpli ying he da a pa i ion, mapping, and anspa en deploymen o bo h, simple gene ic ke - nels po able ac oss di e en de ice ypes, and specialized implemen a ions de ined and op imized using speci ic na i e o endo p og amming models (such as CUDA o NVIDIA’s GPUs, o OpenMP o CPU-co es). The un- ime sys em au oma ically selec s and deploys he mos app op ia e imple- men a ion o each ke nel o each de ice, managing he da a mo emen s, and hiding he launching de ails. Resul s o an expe imen al s udy wi h i e s udy cases indica es ha ou abs ac ion allows he de elopmen o lexible and high e icien p og ams, ha adap o he he e ogeneous en i onmen . Keywo ds Accele a o s, Hyb id compu a ion, GPUs, Ke nel cha ac e iza- ion, Memo y ans e s A. Mo e on-Fe nandez ·A. Gonzalez-Esc ibano, ·D.R. Llanos Depa amen o de In o m´a ica, Uni e sidad de Valladolid, Valladolid, Spain A. Mo e on-Fe nandez E-mail: [email p o ec ed]a.es A. Gonzalez-Esc ibano E-mail: [email p o ec ed]a.es D. R. Llanos E-mail: [email p o ec ed]a.es 2 Ana Mo e on-Fe nandez e al. 1 In oduc ion Cu en HPC clus e s a e composed by se e al machines wi h di e en compu- a ion capabili ies and accele a o s, such as G aphics P ocessing Uni s (GPUs) o XeonPhi cop ocesso s [20]. I has been p o ed ha he use o hese he e oge- neous sys ems imp o es he pe o mance in many applica ions [18]. Howe e , p og amming e icien ly o hese he e ogeneous sys ems has become an im- po an challenge. The di e en compu a ional uni s (CPUs, GPUs, XeonPhi, ..) ha can o m a clus e , ypically ha e di e en p og amming equi emen s and cons ain s o achie e he bes pe o mance. Thus, di e en p og amming models a e used o each kind o de ice. Fo example, CUDA p og amming model achie es he bes pe o mance in NVIDIA GPUs [8], and OpenMP has been shown o be an e icien p og amming model o mul i-co es in sha ed- memo y sys ems o XeonPhi cop ocesso s. The e a e many p oposals o simpli y he p og amming and managemen o accele a o de ices, and he hyb id p og amming mixing accele a o s and CPU co es. Howe e , he po abili y comp omises in many cases he e iciency on di e en de ices. Depending on he p oposal, some de ails abou he co- o dina ion o di e en ypes o de ices a e s ill ackled by he p og amme , such as compu a ion pa i ion and balance, da a mapping and locali y, o da a mo emen coo dina ion ac oss di e en memo y hie a chies. In his wo k we in oduce he Mul i-Con ole (MC l), an abs ac en i y implemen ed in a lib a y, ha coo dina es he managemen o he e ogeneous de ices, including accele a o s wi h di e en capabili ies and se s o CPU- co es. I helps he p og amme o handle he compu a ion pa i ion, mapping, and anspa en execu ion o complex asks in such hyb id and he e ogeneous en i onmen , independen ly o he a ge de ices selec ed a un- ime. Ou p oposal allows he exploi a ion o simple gene ic ke nels ha can i in any de ice, e y specialized ke nels de ined and op imized by he p og amme o each a chi ec u e, and e en w appe s o call hi d-pa y p ede ined lib a ies (such as e.g. cuBLAS [14]). This allows he exploi a ion o na i e o endo speci ic p og amming models, in a highly e icien way. The mos app op ia e ke nels o each a ge de ice a e au oma ically selec ed by he en i y du ing he p og am execu ion. Ou wo k is de eloped on he concep o Con olle p esen ed in [1,12]. While he Con olle anspa en ly manages he da a mo emen s and he launching o se ies o ke nels on a gi en a ge de ice, he Mul i-Con olle coo dina es se e al Con olle s associa ed o di e en de ices o g oups o CPU-co es. I is implemen ed as an ex ensible lib a y, and i can use he bes p og amming models, ools, and compile s o each po en ial de ice. We p esen an expe imen al s udy wi h i e case s udies. We show ha ou app oach is highly lexible, wi h minimum p og amming e o o chang- ing he a ge de ices. The esul s o a pe o mance s udy compa ing ou app oach wi h op imized e e ence codes show ha ou implemen a ion does no in oduce signi ican pe o mance penal ies. Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 3 The es o he pape is o ganized as ollows: Sec ion 2 p esen s he ela ed wo k. Sec ion 3 in oduces he lib a ies used o build ou p oposal. Sec ion 4 explains he p oposed model. Sec ion 5 shows he expe imen al s udy, and inally Sec . 6 exposes he conclusions and u u e wo k. 2 Rela ed wo k In his sec ion we analyse di e en wo ks p oposed in he li e a u e ha a - ge he simpli ica ion o pa allel p og amming on he e ogeneous sys ems, com- posed by di e en compu a ional de ices, including accele a o s. These p o- posals a oid he need o using a manual combina ion o he speci ic p o- g amming models o each compu a ion de ice. They in oduce uni ied p o- g amming models o ool abs ac ions o manage he a chi ec u al di e ences be ween compu a ional uni s o di e en na u e o capabili ies. A widesp ead p og amming amewo k o deal wi h he e ogeneous de ices is OpenCL [19]. The OpenCL con ex abs ac ion allows he managemen o mul iple de ices o he same na u e (using he same pla o m in OpenCL no a ion). Howe e , he coo dina ion o de ices o di e en na u es, and he managemen o da a sha ing o pa i ioning, compu a ion mapping, load bal- ancing, and communica ion ac oss hem a e icky, and should be manually sol ed and coded by he p og amme . Mo eo e , he abs ac ions in oduced by OpenCL de i e in many cases in no eaching he bes possible pe o - mance (see e.g. [8]). Many lib a ies o highe le el o abs ac ion, ha ely on OpenCL as execu ion laye , ypically inhe i some o hese p oblems (see e.g. [19]). An in e es ing example o abs ac ion buil on op o OpenCL is he Maa lib a y [16]. I p o ides a uni ied con ex wi h an abs ac iew e- ga dless o he numbe and na u e o de ices, o GPU and CPU pla o ms. The main di e ences wi h ou p oposal a e ela ed o ou choice o in e nally using na i e o endo low-le el p og amming models. We p opose he possi- bili y o decla ing bo h uni ied and specialized/op imized ke nels o di e en a chi ec u es, which a e selec ed a un- ime o each pa icula de ice. Ou p oposal allows he exploi a ion o ea u es o he na i e p og amming models, o specialized hi d-pa y lib a ies op imized o he de ice. We in oduce mo e lexibili y o selec he desi ed de ices associa ed o a mul icon olle , applying echniques such as g ouping CPU-co es as a single de ice, o exploi na i e mul i- h eaded pa allelism inside a ke nel. The pe o mance esul s p esen ed o he Maa lib a y a e only compa ed wi h o he OpenCL implemen a ions. I has no been s a ed ye he pe o mance e ec s compa ing wi h using mo e speci ic p og amming models such as combina ions o CUDA o GPUs, and OpenMP o mul ico e CPUs. Mo e gene al app oaches p opose comple e in eg a ed amewo ks ha ex- ploi lowe -le el speci ic p og amming models. Some examples include OM- PICUDA [9], Casheme e [6], S a PU [7] o he skele on p og amming ame- wo k based on i , SkePU [2]. PACXX [4] is a ans o ma ion sys em in eg a ed in o he LLVM compile amewo k. I gene a es code o di e en kinds o 4 Ana Mo e on-Fe nandez e al. de ices, and i ans o ms explici pa allel cons uc ions ha use he concep o ke nel and launching in an abs ac an elegan o m. Scogland e al [17] also p oposes a mixed app oach by adap ing OpenMP p agmas o he e oge- neous sys ems. In gene al all hese app oaches hide he coo dina ion de ails o he p og amme , o he poin o cons aining he po en ial op imiza ions ha could be achie ed manually. Techniques o selec launching pa ame e s like he h eadblock size a e o example ackled in SkePU using ial-and-e o , wi h no possibili y o ex apola e he esul s o o he ke nel codes o a chi ec u es. Unlike p e ious app oaches, ou lib a y p oposes a lexible ool, ha allows he p og amming wi h gene ic and po able ke nels, and a he same ime he in eg a ion o highe le els o op imiza ion using he na i e o endo p o ided p og amming models, ools, o lib a ies, o highe e iciency and pe o mance. The lexibili y in e ms o p og amme con ol o he de ice selec ion and coo dina ion a un- ime also imp o es p e ious echniques. 3 Backg ound The Mul i-Con olle lib a y we a e p esen ing in his pape is buil on op o wo p e ious ools. The i s one, named Hi map [3], is a lib a y o manage he pa i ion and mapping o da a s uc u es. I is used in ou model o man- age he da a dis ibu ion ac oss de ices, and o p o ide a common in e ace o implemen da a managemen inside gene ic po able ke nels. The second one, named Con olle [1,12], is a lib a y ha de ines an abs ac en i y o anspa en ly manage he da a mo emen s and ke nel launching o a single de ice. Ou p oposal combines hem wi h a new laye o abs ac ion o co- o dina e he use o se e al de ices o di e en a chi ec u es o na u es. This sec ion in oduces he eade o bo h p e ious ools, be o e desc ibing he new abs ac ion p oposed. 3.1 Hi map lib a y Hi map is a lib a y designed o hie a chical iling and mapping o dense and spa se a ays, o g aphs. Hi map is based on a dis ibu ed SPMD p og amming model, using abs ac ions o decla e da a s uc u es wi h a global iew. I au oma izes he pa i ion, mapping, and communica ion o hie a chies o iles, while s ill deli e ing good pe o mance [3]. An objec Hi Shape ep esen s a subspace o domain indexes. Fo dense a ays, i is de ined as an n-dimensional ec angula pa allelo ope. The limi s on each dimension a e ep esen ed wi h a Hi Sig objec , con aining he ange limi s. Each Hi Sig objec is a uple o h ee in ege numbe s S= (b, e, s) (begin, end, and s ide), ep esen ing he indexes in one o he axis o he domain. Begin and s ide membe s o a Signa u e ep esen he coe icien s o a linea unc ion S(x) = sx +b. Hi map de ines he Hi Tile s uc u e, an abs ac en i y o n-dimensional a ays and iles. A Hi Tile s uc u e is a handle o s o e a ay me a-da a, Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 5 1/* Ke nel cha ac e iza ions */ 2KERNEL_CHAR(Ma Add,1, ull,low,low) 3 4/* Gene ic ke nel codes o any de ice */ 5KERNEL(Ma Add, 3, OUT, Hi Tile_ loa , C, IN, Hi Tile_ loa , A, 6IN, Hi Tile_ loa , B ) { 7in x = h ead.x; 8in y = h ead.y; 9hi _ ileElem( C, 2, x, y ) = 10 hi _ ileElem( A, 2, x, y ) + hi _ ileElem( B, 2, x, y ); 11 } Fig. 1 Ke nel de ini ion and con igu a ion o a ma ix addi ion using he Con olle lib a y. along wi h he poin e o he ac ual memo y space o he da a. A Hi Tile maps ac ual da a elemen s o he index subspace de ined by a shape. The e a e only ou unc ions o Hi map needed o wo k wi h ou Mul i-Con olle lib a y. Fi s , hi ileDomain and hi ileDomainAlloc a e used o decla e he index domains o a ile a ay. The second one also alloca es he memo y o he da a. The unc ion hi ileF ee is used o ee he da a memo y and clean he handle . The unc ion hi ileElem is used in hos o ke nels code o access he elemen s o a ile. I ecei es a ile name, a numbe o dimensions, and he indexes alues o he desi ed elemen . The da a a e accessed in ow majo o de in all cases, independen ly o he implemen a ion. Hi map lib a y includes many o he unc ionali ies. Fo example, he man- agemen o hie a chical subselec ions o pa s o he iles, and he anspa en managemen o dis ibu ed-a ays, wi h abs ac pa i ion and communica- ion unc ionali ies ha in e nally use a message-passing pa adigm (exploi - ing MPI). Hi map has been ex ended wi h a much smalle handle (Hi KTile), wi h he minimum in o ma ion needed o da a accesses o mul idimensional a ays h ough a da a poin e , ha is use ul o anspa en ly po he da a s uc u es o accele a o s in a mo e e icien o m. 3.2 Con olle s lib a y The second abs ac ion upon we ha e buil ou p oposal is implemen ed in he Con olle s lib a y. I in oduces an abs ac en i y ha allows he anspa en launching o se ies o asks on a single accele a o de ice, also conside ing a g oup o CPU co es as a many-co e de ice. The Con olle model p esen s se - e al impo an ea u es: (1) A mechanism o de ine common ke nels eusable ac oss di e en ypes o de ices, o specialized ke nels o speci ic de ice kinds; (2) A anspa en mechanism o memo y managemen , including op imized communica ions o he da a s uc u es be ween he hos and he co espond- ing images in he accele a o s; (3) An op imiza ion sys em o selec p ope alues o ke nel-launching con igu a ion pa ame e s (such as he h eadblock 6 Ana Mo e on-Fe nandez e al. geome y), guided by simple quali a i e code cha ac e iza ion p o ided by he p og amme . In ou p oposal we use he Con olle en i y o manage each de ice inside he new Mul i-con olle ha coo dina es hem. In he Con olle lib a y, a ke nel is decla ed by using he p imi i e KERNEL < ype>. Whe e ype may be emp y o indica e a ke nel usable on any kind o de ice, o a speci ic alue o a specialized code o a gi en ype o de ice. This is use ul when di e en op i- miza ions on he ke nel code a e equi ed o di e en de ices. Cu en ly, he lib a y suppo s he speci ic decla a ions KERNEL GPU o CUDA code a ge - ing NVIDIA’s GPUs, KERNEL CPU o hos machine code a ge ing se s o CPU co es, and KERNEL GPU WRAPPER,KERNEL CPU WRAPPER, o hos machine code which includes calls o specialized GPU o CPU lib a ies, such as cuBLAS o MKL ou ines. Mo eo e , he wo k p esen ed in [11] ex ended he Con olle lib a y o suppo he Xeon Phi co-p ocesso . The ke nel-de ini ion p imi i es decla e in b acke s he numbe o pa am- e e s o he ke nel, wi h a uple o in o ma ion o each pa ame e . The pa- ame e in o ma ion includes i s ype, name, and inpu /ou pu ole: –IN: o inpu Hi Tile pa ame e s, whose elemen s a e only ead. –OUT: o ou pu Hi Tile pa ame e s, whose elemen s a e only w i en. –IO: o inpu and ou pu Hi Tile pa ame e s, wi h elemen s can be bo h ead and w i en. –INVAL: o inpu pa ame e s o any ype passed by alue. The p og amme can also p o ide a ke nel cha ac e iza ion, in e ms o code ea u es, ha helps o au oma ically de e mine p ope ke nel launching pa ame e s. In he cu en p o o ype, he CPU h eads g anula i y is de e - mined by a simple egula blocking policy, ha does no equi e a speci ic ke nel cha ac e iza ion. Fo GPU ke nels, he lib a y in eg a es he model p esen ed in [15,21]. This model allows he de e mina ion o con igu a ion pa- ame e s (g id, h eadblock, and L1 cache memo y sizes), o NVIDIA’s GPUs. The p imi i e KERNEL CHAR ecei es he ke nel name, he numbe o dimensions o he h ead space (1, 2, o 3), and desc ip i e alues o he cha ac e iza ion model. These alues a e a quali a i e desc ip ion o cha ac e is ics o he ke - nel code p o ided by he p og amme . They a e ela ed o: (a) The coalescing p ope y o he global memo y access pa e ns ( ull, medium, sca e ); (b) The a io o a i hme ic/logic ope a ions pe global memo y access (high, medium, low); and (c) The a io o da a sha ing accesses in a block pe global memo y access (high, medium, low). Figu e 1 shows an example o he ke nel o a ma ix addi ion compu a ion. We see a ke nel cha ac e iza ion in line 2 and a ke nel de ini ion in lines 5 and 6. 4 Mul iple-De ice Con olle (MC l) lib a y The Mul iple-De ice Con olle (MC l) lib a y p o ides a simpli ied way o p og am applica ions a ge ing he e ogeneous sys ems wi h di e en kinds o Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 7 Execu ion policyQueue C l-0 Execu ion policyQueue C l-1 Execu ion policyQueue C l-2 Many-co e Memo y De -0 De -1 De -2 A[0] Mul i-co e Memo y A[1] Many-co e Memo y A[2] Ke nel di ision Da a pa i ion MC l Hos MA ach MLaunch MDe ach Hi Tile A, B A Code Memo y A ached Mul iCon olle De ices CPU CPU CPU CPU A BB B[0] B[1] B[2] In e nal X Fig. 2 Diag am o he Mul iple-De ice Con olle lib a y (MC l). compu a ional uni s. In his pape , we de ine compu a ional uni / a ge de ice as an accele a o s (GPU, Xeon Phi, e c.) o a g oup o CPU-co es conside ed as a single independen de ice. The goal o his lib a y is: (1) o au oma ize he da a pa i ion and da a ans e be ween he hos and mul iple a ge de ices, as well as (2) o anspa en ly coo dina e he di ision and execu ion o he compu a ion among di e en compu a ional uni s, independen ly o he kind o a ge de ice exploi ed (GPUs, g oup o CPU-co es, e c). The lib a y has an objec -o ien ed design, despi e he ac ha i is mainly de eloped in C language. The classes a e implemen ed as C s uc u es wi h associa ed unc ions. The Mul i-Con olle model a chi ec u e is p esen ed in he Fig. 2. The Mul i-Con olle objec p o ides unc ions o manage: – Mul i-de ice coo dina ion: The Mul i-Con olle is associa ed wi h a se o di e en de ices a cons uc ion. I in e nally c ea es Con olle ob- jec s o in e ac wi h each de ice. The Mul i-Con olle p o ides an ab- s ac in e ace ha enables o manage i as a single compu a ional de ice, independen ly o he in e nally associa ed de ices. – Da a s uc u es: The Mul i-Con olle abs ac ion c ea es an uni ied memo y con ex o all o he associa ed de ices, whe e in e nal da a s uc- u es can be c ea ed, o da a s uc u es om he main hos h ead can be a ached. Da a s uc u es can be eplica ed o pa i ioned and dis ibu ed ac oss he de ices as he p og amme equi es. In he cu en p o o ype, a simple s a ic pa i ioning me hod has been included. I pa s he s uc- u es in as many i egula pa s as numbe o de ices we e selec ed in he Mul i-Con olle cons uc ion. The size o each pa is calcula ed p o- po ionally o a lis o weigh s. Da a mo emen s ac oss di e en de ice memo y hie a chies a e anspa en ly managed by he in e nal Con olle objec s associa ed o each de ice. – Ke nel de ini ion and launching: The Mul i-Con olle model in e- g a es he Con olle s idea o mul i- e sion ke nel de ini ion. Thus, ke nel launching in a Mul i-Con olle simply uses a ke nel name. The in e nal 8 Ana Mo e on-Fe nandez e al. Con olle s selec s, a un- ime, he mos app op ia e ke nel e sion o implemen a ion o he associa ed de ice, among hose p o ided by he p og amme . The Mul i-Con olle in e nally di ides he compu a ion as- socia ed o a ke nel launching among he di e en de ices. The ke nel exe- cu ion on a de ice is pe o med asynch onously wi h espec o he ke nels execu ion in he es o de ices. Synch oniza ions a e equi ed only by da a eques s on he main hos h ead. 4.1 Mul i-Con olle cons uc ion A Mul i-Con olle objec is cons uc ed o manage a speci ic collec ion o de ices. I s cons uc ion unc ionali y ecei es an o de ed lis o de ices spec- i ica ions. Each de ice speci ica ion is used o in e nally c ea e a Con olle objec associa ed o he compu a ional esou ce. In he cu en p o o ype we suppo de ice speci ica ions ha include: (1) NVIDIA’s GPUs, speci ying hei CUDA de ice numbe , and (2) G oups o main hos CPU-co es, speci- ying a ange o co e iden i ie s, acco ding o hei in e nal numbe ing in he CPU in o ma ion p o ided by he ope a ing sys em. The Mul i-Con olle in e nally c ea es a queue o empo a ily s o e he eques s o ke nel launchings, be o e di iding he compu a ion and mapping i o he queues o he in e nal de ice Con olle s. The synch oniza ion and coo dina ion ope a ions o each Con olle a e execu ed on i s own ask, which makes asynch onous he use o a hos h ead only when ac i i y is needed, o minimal in e e ence wi h o he hos h eads. In he cu en p o o ype, he in e nal de ice Con olle s a e implemen ed using OpenMP asks. 4.2 Da a s uc u es and domains One o he objec i es o he Mul i-con olle lib a y is o p o ide an homo- geneous in e ace o wo k wi h da a s uc u es in di e en de ice ypes, p e- se ing he coalescing o ec o iza ion p ope ies o he code due o he da a accesses o de . The p e ious Con olle s lib a y uses Hi map o p o ide such in e ace. Fo Mul i-Con olle we p opose o exploi and ex en Hi map unc ional- i ies o p o ide a anspa en abs ac ion o he pa i ion, subselec ion, and mapping o pa s o he da a s uc u es o he di e en de ices associa ed o a Mul i-Con olle . The Mul i-Con olle model p oposes a single memo y con ex o he whole se o associa ed he e ogeneous de ices. Da a s uc u es om he main hos h ead can be a ached o he Mul i-Con olle con ex , and hey should no be manipula ed on he main hos h ead un il hey a e de ached om he Mul i-Con olle . The Mul i-Con olle can decide when he eal da a mo emen s should be done, synch onously o asynch onously, o he ac ual de ices, depending on he ke nels enqueued o execu ion, and hei da a dependencies. Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 9 To a oid edundan da a mo emen s ac oss memo y hie a chies, he model p o ides he p og amme wi h a lexible a achmen unc ionali y. The cu en p oposal is ocused on applica ions whe e: (1) No da a ans e s be ween de- ices a e needed ac oss se e al ke nel execu ions; and (2) any pa o he com- pu a ion needs ei he , a whole da a s uc u e, o a subse o he da a s uc u e ha does no o e lap wi h o he subpa s. Thus, da a s uc u es should be assigned as a whole o all he de ices, o pa i ioned in non-o e lapping pa s, assigning one o each de ice. To suppo his model, we ha e ex ended he Hi Tile objec s in he Hi map lib a y wi h he capabili y o pa i sel in se e al sub-selec ions, and s o e he in o ma ion abou he pa i ion inside he objec . In e nally, when a Hi Tile is a ached o a Mul i-Con olle , i pe o ms he ollowing s eps: 1. Fi s , i checks ha he Hi Tile is no al eady a ached o any Mul i- Con olle . I ha pa icula Hi Tile objec is al eady a ached, he p o- g am aises an e o , as a second a achmen could lead o ace condi ions due o he concu en execu ion o ke nels in di e en Mul i-Con olle s. 2. I i is an a achmen wi hou he pa i ion op ion, he whole space o indexes o he da a s uc u e a e mapped o each de ice. I i is an a ach- men wi h he pa i ion op ion ac i a ed, he Mul i-Con olle pa s he index space o he Hi Tile da a s uc u e in a numbe o pa s equal o he numbe o de ices de ined in he Mul i-Con olle . The pa i ion policies in oduced in Hi map a e esponsible o di iding he da a s uc u e wi h no-o e lapped domains. The pa i ion size co esponding o each de ice is p opo ional o he weigh s p o ided in an a ay o loa ing poin numbe s, one o each de ice. Mo e sophis ica ed pa i ion policies can be easily added in he u u e hanks o he modula plug-ins sys em in he Hi map lib a y. The in o ma ion abou he mapping is s o ed in he Hi Tile objec o u he e e ence. 3. Finally, he Mul i-Con olle c ea es a Hi Tile s uc u e o he space o sub-space o indexes mapped o each de ice, and p oceeds o do he da a ans e s o he assigned de ice when needed. T ans e s a e no needed o g oups o CPU-co es o he hos , o accele a o s ha can sha e he hos memo y space. The ans e policies inside he Mul i-Con olle can ake decisions abou when and how make he ans e s. Fo example, he cu en Mul i-Con olle p o o ype implemen bo h, immedia e and lazy ans e s. The implemen a ion o asynch onous ans e s is cu en ly an on-going wo k. When he da a s uc u e is de ached, he Mul i-Con olle objec ensu es he consis ency o he whole da a s uc u e in he main hos h ead. This may imply da a ans e s om some o all o he associa ed de ices. The in o ma ion s o ed in he objec s abou he index space mapped o each de ice is used o he ans e s, and elimina ed a he end o he de achmen p ocedu e. The seman ic o his ope a ion makes i synch onous. The main hos h ead should block un il i s s a e is consolida ed. 16 Ana Mo e on-Fe nandez e al. Table 1 De elopmen e o measu es o he ou benchma ks when hey a e p og ammed using Cuda, OpenMP, and he p oposed Mul i-Con olle lib a y. Benchma k Code Lines Cycloma ic Hals ead o Code Complexi y Measu e Ma ix Cuda 72 7 202361 Addi ion OpenMP 49 11 99783 MC l 61 5 103528 Ma ix Cuda 142 5 409862 Mul . OpenMP 45 9 81136 MC l 97 6 201242 Black- Cuda 211 7 742735 Scholes OpenMP 134 8 389556 MC l 163 6 486956 Mandel- Cuda 49 3 159481 b o OpenMP 36 5 66610 MC l 55 3 124212 5.1.4 Mandelb o algo i hm The escape ime algo i hm is he simples algo i hm o gene a ing a ep esen- a ion o ac al geome ic images o he Mandelb o se . This example p esen s an i egula wo kload pe h ead. Ne e heless, he compu a ion space can be di ided be ween wo GPUs gi ing he same amoun o wo k o each GPU, and hus p oducing good esul s. This example will clea ly show he balancing e ec o pa i ioning he wo kload among se e al GPUs. The applica ion does no need any da a ans e o s a he compu a ion, only o e u n he esul s. The pa ame e s chosen o he expe imen s a e: An image size o 2048 ×2048, and a limi o 60,000 i e a ions pe pixel. A isual ep esen a ion o he chosen a ea is shown in Fig. 6. The GPU h eadblock size used is 32 ×32 o all he expe imen s wi h his benchma k. In ou implemen a ion, he same gene ic ke nel de ini ion is used o bo h GPUs and CPUs. 5.2 De elopmen e o In his sec ion we compa e, in e ms o de elopmen e o , he use o ou lib a y wi h he mos common na i e p og amming models o NVIDIAS’s GPUs and mul i-co e CPUs, which a e CUDA and OpenMP espec i ely. Fo his compa ison, we use h ee classical de elopmen e o me ics: COCOMO lines o code, McCabe’s cycloma ic complexi y [10], and Hals ead de elopmen e o [5]. McCabe’s cycloma ic complexi y measu e is a quan i a i e measu e o he numbe o linea ly independen pa hs h ough a p og am’s sou ce code, and he Hals ead de elopmen e o me ic is also a quan i a i e numbe based on he numbe on ope a o s and ope ands in he sou ce code. Low cycloma ic complexi y and Hals ead de elopmen e o esul s imply codes simple o de- elop and debug. These me ics a e ypically used in he assessmen o so wa e design complexi y. The me ics a e applied o he pa s o code ha include: ke nel de ini ions, ke nel cha ac e iza ions, he coo dina ion code in he main hos h ead wi h he Mul i-con olle managemen , and da a s uc u es man- Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 17 agemen . We igno e code de o ed o e o o esul s checking, pe o mance ins umen a ion, and w i ing messages o he s anda d ou pu . Table 1 shows he di e en measu es o he di e en codes e alua ed. The esul s show ha ou lib a y implies less de elopmen e o o he p og am- me han using CUDA o all he s udy cases. On he o he hand, al hough he OpenMP p og amming model needs a less olume o lines o code, he cycloma ic complexi y o ou p oposed is less because ou abs ac ion hides some un- ime decisions and checkings. T ans o ming a CUDA p og am in o an OpenMP e sion, o in he opposi e way, is no a i ial ask. Remind he Fig. 5, whe e we show wo codes, using he Mul i-Con olle abs ac ion, ha pe o m a ma ix addi ion using di e en a ge de ice combina ions. When we compa e bo h codes, we obse e ha he e o equi ed by he p og amme o change he p og am in o de o exploi 1 GPU + 10 CPU-co es, o wo GPU de ices, only in ol es 4 lines o code. We can see hese ou lines highligh ed on bo h codes in he code. This simpli ica ion makes anspa en o he p og amme all he di e ences on da a ans e s, ke nel launchings, and da a managemen s o di e en kind o compu a ional de ices such as accele a o s o g oups o CPU-co es. 5.3 Pe o mance esul s In his sec ion we p esen pe o mance esul s: (1) compa ing ou p oposal wi h pu e CUDA e e ence p og ams, and (2) e alua ing he impac o using in ou model di e en compu a ional uni s o he i e case s udies selec ed. The goal o his s udy is o de e mine he po en ial pe o mance penal y in- oduced by using ou app oach, as well as he pe o mance gain ob ained when exploi ing a combina ion o he e ogeneous de ices wi h di e en compu- a ional capabili ies. The expe imen s ha e been execu ed on a hos machine named Hyd a, wi h wo CPUs In el Xeon E5-2609 3 @1.90GHz, 64Gb DDR3 main memo y, and wo GPUs: an NVIDIA’s GeFo ce Ti an Z (named GPU-0) and a Ti an Black X (named GPU-1). We exploi he wo GPU de ices, and mul iple CPU-co es o ganized in a single i ual de ice in ou model. Fo his es , we ha e decided o a oid pe o mance e ec s de i ed om o e susc ip ion o hype h eading. As ou Mul i-Con olle lib a y uses one hos h ead o each de ice o be con olled, he numbe o CPU-co es we use o compu e and execu e ke nels is 10. The p og ams ha e been compiled using he CUDA Toolki 8.0, and GCC 4.8.3. We ha e used he lags, -O3, and - openmp o exploi pa allelism when using a g oup o CPU-co es as a compu a ional uni . We ha e execu ed all he expe imen s en imes, egis e ing he lowes o al execu ion imes. We ha e also measu ed sepa a ely he imes spen in copying da a o h o and back om he a ge de ices, and he compu a ion ime o he ke nels. We include he ime spen by ou queue sys em inside he compu a ion imes. We ha e es ed h ee kinds o codes: 18 Ana Mo e on-Fe nandez e al. 0 0.2 0.4 0.6 0.8 1 1.2 80 85 90 92.5 95 97.5 100 Cuda Execu ion ime (sec.) CPU % da a Ma ix Addi ion CPU+GPU, SIZE=20000x20000 Compu a ion Da a ans e s 0 0.1 0.2 0.3 0.4 0.5 0.6 80 85 90 92.5 95 97.5 100 Cuda Execu ion ime (sec.) CPU % da a BlackScholes CPU+GPU, SIZE= 3*10^8 Compu a ion Da a ans e s 0 200 400 600 800 1000 1200 1400 80 85 90 92.5 95 97.5 100 Cuda Execu ion ime (sec.) CPU % da a Ma ix Mul iplica ion CPU+GPU, SIZE=12800x12800 Compu a ion Da a ans e s 0 1 2 3 4 5 6 7 8 9 80 85 90 92.5 95 97.5 100 Cuda Execu ion ime (sec.) CPU % da a BlackScholes_2048 CPU+GPU, SIZE= 1*10^6 Compu a ion Da a ans e s 0 0.5 1 1.5 2 2.5 80 85 90 92.5 95 97.5 100 Cuda Execu ion ime (sec.) CPU % da a Mandelb o CPU+GPU, SIZE=2048 x 2048 ITERATIONS=60000 Compu a ion Da a ans e s Fig. 7 Pe o mance esul s (in seconds) o expe imen s on Hyd a using a g oup o 10 CPU- co es and a GPU. The igh -mos columns show he esul o he e e ence CUDA p og ams un on he same GPU. –A na i e CUDA implemen a ion o he di e en benchma ks es ed. We p esen measu es ob ained in one o bo h GPUs in ou a ge sys em de- pending on he s udy. See he igh -mos and/o he le -mos columns in he cha s discussed bellow. Cuda Re : Measu es in NVIDIA’s GeFo ce Ti an Black Z. Cuda Re -2: Measu es in NVIDIA’s GeFo ce Ti an Black X. – CPU+GPU: This code, p og ammed using he MC l lib a y, execu es he p og ammed applica ion on wo de ices, a g oup o 10 CPU-co es, and an NVIDIA’s GeFo ce Ti an Black Z. Di e en mappings ha e been es ed, de e mined by he pe cen age o da a and compu a ion assigned o each de ice. – GPU+GPU: This code, p og ammed using he MC l lib a y, execu es he applica ion on he wo GPUs a ailable in he a ge sys em. Again, Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 19 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Cuda_Re 100 70 65 60 55 50 0 Cuda_Re -2 Execu ion ime (sec.) GPU-0 % da a Ma ix Addi ion GPU+GPU, SIZE=20000x20000 Compu a ion Da a ans e s 0 0.5 1 1.5 2 2.5 3 3.5 Cuda_Re 100 70 65 60 55 50 0 Cuda_Re -2 Execu ion ime (sec.) GPU-0 % da a BlackScholes GPU+GPU, SIZE= 3*10^8 Compu a ion Da a ans e s 0 1 2 3 4 5 6 7 8 9 10 Cuda_Re 100 70 65 60 55 50 0 Cuda_Re -2 Execu ion ime (sec.) GPU-0 % da a Ma ix Mul iplica ion GPU+GPU, SIZE=12800x12800 Compu a ion Da a ans e s 0 0.05 0.1 0.15 0.2 0.25 Cuda_Re 100 70 65 60 55 50 0 Cuda_Re -2 Execu ion ime (sec.) GPU-0 % da a BlackScholes_2048 GPU+GPU, SIZE= 1*10^6 Compu a ion Da a ans e s 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Cuda_Re 100 80 75 70 65 60 0 Cuda_Re -1 Execu ion ime (sec.) GPU-0 % da a Mandelb o GPU+GPU, SIZE=2048 x 2048 ITERATIONS=60000 Compu a ion Da a ans e s Fig. 8 Pe o mance esul s (in seconds) o expe imen s on Hyd a using wo di e en GPUs. The le - and igh -mos columns show he esul s o he e e ence CUDA p og ams un on each o he wo GPUs conside ed in he s udy. di e en mappings ha e been es ed, de e mined by he pe cen age o da a and compu a ion assigned o each de ice. Figu e 7 shows he pe o mance esul s ob ained by ou p oposal when he da a, and hus he compu a ion, is di ided among he g oup o CPU- co es and a GPU, he NVIDIA’s GeFo ce Ti an Black Z. In he applica ions whe e da a ans e s domina e he o al ime (Ma ix Addi ion and Black- Scholes benchma ks) we can achie e a be e pe o mance by gi ing pa o he compu a ion o he g oup o CPU-co es. Despi e he compu a ional powe o he GPU accele a o , he compu a ion di ision imp o es pe o mance by educing he ime spen in da a ans e s o/ om he GPU. When he compu a ional load is high, such as in he ma ix mul iplica ion, and Black-Scholes 2048, he bes pe o mance is ob ained doing he whole compu a ion in he GPU. This is ypical o his kind o p og ams ha e- ally sui he GPU compu a ional model. Fo he Mandelb o benchma k, we 20 Ana Mo e on-Fe nandez e al. obse e ha he pa i ion o he compu a ion ob ains ma ginally be e pe - o mance esul s han he CUDA e e ence codes. This pa icula beha iou is because o he i egula wo kload o his benchma k. Figu e 8 shows he pe o mance esul s ob ained when we di ide he com- pu a ion be ween he wo GPUs. When he compu a ion load is eally low and he e a e mul iple ke nel launchings, he ime spen in he queue managemen s ( emind ha hese imes a e aken in o accoun in he compu a ion ime) can be no iceable, such in he BlackScholes 2048 case (i akes app oxima ely 0.04 seconds). In ou i s p o o ype o he lib a y, da a ans e s o se e al GPUs a e s ill sequen ialized. Thus, in hose applica ions whe e da a ans e s lead he execu ion ime, he bes pe o mance is ob ained when he applica ion is exe- cu ed only in he mos powe ul GPU. Howe e , when he compu a ion ime is much highe han he communica ion imes, a di ision o he compu a ion among he GPUs, p opo ional o hei ela i e compu a ion powe o his p oblem, imp o es he pe o mance. Fo example in ou s udy, wi hou aking in o accoun he da a ans e s, he ke nel execu ion imes a e educed he 34%, 25%, and 41% compa ed o he bes CUDA e e ence codes which use a single de ice, o he ma ix mul iplica ion, Black-Scholes 2048 and Mandel- b o benchma ks espec i ely. This beha iou is shown in he expe imen a ion esul s. 6 Conclusion In his pape we p esen he Mul i-Con olle (MC l), an abs ac en i y im- plemen ed in a lib a y, ha coo dina es he managemen o he e ogeneous de- ices, including accele a o s wi h di e en capabili ies and se s o CPU-co es. This en i y o e s a global iew o he compu a ion, anspa en ly managing he coo dina ion, da a pa i ion, mapping, and execu ion o whole compu a ions on hei associa ed de ices. Ou solu ion allows he use o simple gene ic ke - nels (po able ac oss di e en de ice ypes), o specialized implemen a ions de- ined and op imized using speci ic na i e o endo p og amming models (such as CUDA o NVIDIA’s GPUs, o OpenMP o CPU-co es). The un- ime sys- em au oma ically selec s and deploys he mos app op ia e implemen a ion o each ke nel o each de ice, managing he da a mo emen s, and hiding he launching de ails. Resul s o an expe imen al s udy wi h i e s udy cases indi- ca e ha ou abs ac ion allows he de elopmen o lexible and high e icien p og ams, ha adap o he he e ogeneous en i onmen . On-going and u u e wo k includes he s udy o he e ec o mo e sophis ica ed echniques o da a mo emen which o e lap communica ion and compu a ion, and a new expe - imen al s udy including i egula benchma ks whose da a should be di ided using mo e complex mechanisms. Acknowledgemen s This esea ch has been pa ially suppo ed by MICINN (Spain) and ERDF p og am o he Eu opean Union: HomP og-He Sys p ojec (TIN2014-58876-P), CAPAP- Mul i-De ice Lib a y: Simpli ying Pa allel He e ogeneous P og amming 21 H6 (TIN2016-81840-REDT), and COST P og am Ac ion IC1305: Ne wo k o Sus ainable Ul ascale Compu ing (NESUS). Re e ences 1. Alonso-Mayo, A., O ega-A anz, H., Gonzalez-Esc ibano, A.: Communica o s: An ab- s ac ion o ease he use o accele a o s. In: HLPGPU’2016 (2016) 2. Das gee , U., Enmy en, J., Kessle , C.W.: Au o- uning SkePU: A Mul i-backend Skele- on P og amming F amewo k o mul i-GPU Sys ems. In: P oc. IWMSE’11, pp. 25–32. ACM, New Yo k, NY, USA (2011) 3. Gonzalez-Esc ibano, A., To es, Y., F esno, J., Llanos, D.R.: An ex ensible sys em o mul ile el au oma ic da a pa i ion and mapping. IEEE T ansac ions on Pa allel and Dis ibu ed Sys ems 25(5), 1145–1154 (2014) 4. Haidl, M., Go la ch, S.: PACXX: Towa ds a Uni ied P og amming Model o P og am- ming Accele a o s using C++14. In: P oc. LLVM-HPC’14. IEEE (2014) 5. Hals ead, M.H.: Elemen s o So wa e Science (Ope a ing and p og amming sys ems se ies). Else ie Science Inc. (1977) 6. Hijma, P., Jacobs, C.J., an Nieuwpoo , R.V., Bal, H.E.: Cashme e: He e ogeneous many-co e compu ing. In: Pa allel and Dis ibu ed P ocessing Symposium (IPDPS), 2015 IEEE In e na ional, pp. 135–145. IEEE (2015) 7. Hugo, A.E., Gue mouche, A., Wac enie , P.A., Namys , R.: Composing Mul iple S a PU Applica ions o e He e ogeneous Machines: A Supe ised App oach. In: P oc. IPDPSW’13 PhD Fo um, pp. 1050–1059. IEEE, Washing on, D.C., USA (2013) 8. Ka imi, K., Dickson, N.G., Hamze, F.: A pe o mance compa ison o cuda and opencl. a Xi p ep in a Xi :1005.2581 (2010) 9. Liang, T., Li, H., Chiu, J.: Enabling Mixed OpenMP/MPI P og amming on Hyb id CPU/GPU Compu ing A chi ec u e. In: P oc. IPDPSW’12, PhD Fo um, pp. 2369– 2377. IEEE, Washing on, D.C., USA (2012). DOI 10.1109/IPDPSW.2012.294 10. McCabe, T.J.: A complexi y measu e. So wa e Enginee ing, IEEE T ansac ions on 4, 308–320 (1976) 11. Mo e on-Fe nandez, A., Rod iguez-Gu iez, E., Gonzalez-Esc ibano, A., Llanos, D.R.: Suppo ing he xeon phi cop ocesso in a he e ogeneous p og amming model. In: Eu- opean Con e ence on Pa allel P ocessing, pp. 457–469. Sp inge , Cham (2017) 12. Mo e onFe nandez, A., O egaA anz, H., GonzalezEsc ibano, A.: Con olle s: An ab- s ac ion o ease he use o ha dwa e accele a o s. The In e na ional Jou nal o High Pe o mance Compu ing Applica ions (2017). DOI 10.1177/1094342017702962. URL h p://dx.doi.o g/10.1177/1094342017702962 13. NVIDIA: NVIDIA CUDA C P og amming Guide 7.5 (2015). URL {h p://docs. n idia.com/cuda/pd /CUDA _C _P og amming _Guide.pd }. Las isi : No embe 16 h, 2015 14. N idia, C.: Cublas lib a y. NVIDIA Co po a ion, San a Cla a, Cali o nia 15, 27 (2008) 15. O ega-A anz, H., To es, Y., Gonzalez-Esc ibano, A., Llanos, D.R.: Op imizing an APSP implemen a ion o NVIDIA GPUs using ke nel cha ac e iza ion c i e ia. The Jou nal o Supe compu ing 70(2), 786–798 (2014). DOI 10.1007/s11227-014-1212-z 16. P´e ez, B., Bosque, J.L., Bei ide, R.: Simpli ying p og amming and load balancing o da a pa allel applica ions on he e ogeneous sys ems. In: P oceedings o he 9 h Annual Wo kshop on Gene al Pu pose P ocessing using G aphics P ocessing Uni , pp. 42–51. ACM (2016) 17. Scogland, T.R., Roun ee, B., Feng, W.c., de Supinski, B.R.: He e ogeneous ask scheduling o accele a ed openmp. In: Pa allel & Dis ibu ed P ocessing Symposium (IPDPS), 2012 IEEE 26 h In e na ional, pp. 144–155. IEEE (2012) 18. Shen, J., Va banescu, A.L., Lu, Y., Zou, P., Sips, H.: Wo kload pa i ioning o accel- e a ing applica ions on he e ogeneous pla o ms. IEEE T ansac ions on Pa allel and Dis ibu ed Sys ems 27(9), 2766–2780 (2016) 19. S one, J.E., Goha a, D., Shi, G.: Opencl: A pa allel p og amming s anda d o he - e ogeneous compu ing sys ems. Compu ing in science & enginee ing 12(1-3), 66–73 (2010) 22 Ana Mo e on-Fe nandez e al. 20. TOP500.o g: Top500 supe compu ing si es. WWW (2017). On h p://www. op500. o g/ 21. To es, Y., Gonzalez-Esc ibano, A., Llanos, D.R.: uBench: exposing he impac o CUDA block geome y in e ms o pe o mance. The Jou nal o Supe compu ing 65(3), 1150– 1163 (2013). DOI 10.1007/s11227-013-0921-z