Intel-oneAPI for Heterogeneous Computing

In el-oneAPI pa a Compu ación He e ogénea In el-oneAPI o He e ogeneous Compu ing Ge mán Cas año Roldán T abajo de fin de g ado del G ado en Ingenie ía In o má ica, Facul ad de In o má ica, Uni e sidad Complu ense de Mad id Cu so 2020/2021 Di ec o : Ca los Ga cía Sánchez In el-oneAPI o He e ogeneous Compu ing Índice Acknowledgmen s 3 Resumen 4 Palab as Cla e 4 Abs ac 5 Key Wo ds 5 Chap e 1: In oduc ion 6 Wo k plan 7 Lea ning phase 7 Mig a ion phase 7 Benchma king phase 7 Chap e 2: Mo i a ion and Objec i es 8 Mo i a ion 8 Objec i es 10 Chap e 3: S a e o he A 10 CUDA 11 CUDA Al e na i es 12 High-Le el P og amming 12 OpenMP 12 OpenACC 12 C++ Lib a ies and Ex ensions 13 Py hon and Ja a 13 SYCL and oneAPI 13 FPGAs 15 Fu u e Accele a o s 15 Chap e 4: DPC++ 16 Asynch onous excep ions 16 De ice Selec o 17 Buffe s and Accesso s 18 Unified Sha ed Memo y 19 Queue and pa allel_ o 19 Chap e 5: DPCT & Rodinia 21 DPC++ Compa ibili y Tool 21 Rodinia Benchma ks 22 1 In el-oneAPI o He e ogeneous Compu ing Chap e 6: Me hodology 24 Mig a ion p ocess 24 Benchma king p ocess 29 Ins umen aliza ion 30 CUDA s oneAPI 30 P ofiling 31 O he a chi ec u es 32 Chap e 7: Mig a ion Resul s 33 Wa nings 33 Manual modifica ions 37 P oblems encoun e ed 39 Chap e 8: Pe o mance Resul s 41 Pe o mance o he memo y ope a ions 41 Pe o mance o he ke nel execu ion 42 O e all pe o mance 48 Chap e 9: Conclusions 49 Bibliog aphy 51 Glossa y 54 2 In el-oneAPI o He e ogeneous Compu ing Acknowledgmen s Fo emos , I would like o hank my di ec o in his p ojec , Ca los Ga cía Sánchez, no only o his suppo and guidance bu o a ousing my in e es in he e ogeneous compu ing du ing he subjec "P og amación de GPUs y Acele ado es", wi hou which I wouldn' ha e chosen a p ojec like his o my final deg ee p ojec . I also would like o exp ess my g a i ude o Alina Shad ina om In el o he in e es and he insigh p o ided when p oblems a ose du ing he mig a ion om CUDA o oneAPI. 3 In el-oneAPI o He e ogeneous Compu ing Resumen "oneAPI es un modelo de p og amación unificado, abie o y basado en es ánda es, que o ece una expe iencia de desa ollado común en odas las a qui ec u as de acele ado es, pa a un endimien o de aplicaciones más ápido, más p oduc i idad y una mayo inno ación." -www.oneapi.com La he amien a de compa ibilidad DPC++ de In el es un componen e del oneAPI Base Toolki . es a he amien a ans o ma au omá icamen e código CUDA en Da a Pa allel C++ (DPC++) ayudando en el p oceso de mig ación. Es e p oyec o consis e en un análisis de la he amien a de compa ibilidad DPC++, conside ando la in e ención manual eque ida y los p oblemas encon ados al mig a los benchma ks de Rodinia. Y un es udio compa a i o del endimien o ob enido po el código mig ado. Palab as Cla e In el oneAPI, SYCL, Da a Pa allel C++, He amien a de compa ibilidad Da a Pa allel, CUDA, Compu ación he e ogénea, Rodinia Benchma k Sui e. 4 In el-oneAPI o He e ogeneous Compu ing Abs ac "oneAPI is a c oss-indus y, open, s anda ds-based unified p og amming model ha deli e s a common de elope expe ience ac oss accele a o a chi ec u es— o as e applica ion pe o mance, mo e p oduc i i y, and g ea e inno a ion." -www.oneapi.com The In el DPC++ Compa ibili y Tool is a componen o he In el oneAPI base oolki . This ool au oma ically ans o ms CUDA code in o Da a Pa allel C++ (DPC++) assis ing in he mig a ion p ocess. This p ojec consis s o an analysis o he DPC++ Compa ibili y Tool, conside ing he manual in e en ion equi ed and he p oblems encoun e ed while mig a ing he Rodinia benchma ks. And a compa a i e s udy o he pe o mance ob ained by he mig a ed code. Key Wo ds In el oneAPI, SYCL, Da a Pa allel C++, Da a Pa allel Compa ibili y Tool, CUDA, He e ogeneous compu ing, Rodinia Benchma k Sui e. 5 In el-oneAPI o He e ogeneous Compu ing Chap e 1: In oduc ion Nowadays he e ogenei y is widely p esen in bo h high-pe o mance compu ing and consume elec onics. These sys ems add a mul i ude o co-p ocesso s o accele a o s, such as GPUs, TPUs, and FPGAs, o he adi ional CPU. Howe e , he e isn' a simple, po able and efficien me hod o de elop o hese sys ems. In el oneAPI aims o fill his ole. This p ojec consis ed on an e alua ion o he Da a Pa allel C++ Compa ibili y Tool assessing he manual modifica ions needed by he esul ing code and pe o ming hem in he cases whe e i was assumable, an ins umen aliza ion o he o iginal and he mig a ed code and he benchma king in diffe en he e ogeneous sys ems. All he code gene a ed, along wi h he ou pu s o he mig a ion ool and he necessa y makefiles and da ashee s, is publically a ailable in he gi hub eposi o y: h ps://gi hub.com/CR-G/In el-oneAPI- o -He e ogeneous-Compu ing All he p oblems and conce ns encoun e ed du ing he mig a ion we e discussed by email wi h s aff om In el, con ibu ing o he imp o emen o he Compa ibili y Tool and oneAPI. This documen consis s on he ollowing 9 chap e s: ●Chap e 1: In oduc ion. Includes a b ie in oduc ion o he e ogeneous sys ems, he wo k pe o med, a desc ip ion o he s uc u e o his documen and he wo k plan. ●Chap e 2: Mo i a ion and Objec i es. Discusses he mo i a ion behind his p ojec and i s objec i es. ●Chap e 3: S a e o he A . E olu ion and cu en s a us o he accele a o -based he e ogeneous compu ing. ●Chap e 4: DPC++. Includes an in oduc ion o Da a Pa allel C++ wi h an example p og am. ●Chap e 5: DPCT & Rodinia. Desc ibes he Da a Pa allel C++ Compa ibili y Tool and he Rodinia Benchma k Sui e. ●Chap e 6: Me hodology. Explains he me hodology ollowed by his p ojec , including he mig a ion p ocess, he benchma king and he ins umen aliza ion o he code. ●Chap e 7: Mig a ion Resul s. P esen s he esul s o he mig a ion p ocess, discussing he necessa y manual in e en ion and he p oblems encoun e ed. ●Chap e 8: Pe o mance Resul s. This chap e exposes he esul s o he benchma king. ●Chap e 9: Conclusions. 6 In el-oneAPI o He e ogeneous Compu ing Wo k plan The p ojec was di ided in o h ee phases, a lea ning phase, a mig a ion phase and a benchma king phase. Du ing he leng h o he p ojec egula mee ings we e held wi h he u o o moni o he p og ess and discuss esul s. Due o he COVID-19 pandemic, all he mee ings we e held online ia ideo calls. Lea ning phase Du ing his phase I amilia ized mysel wi h DPC++ language and oneAPI wi h he help o he book Da a Pa allel C++ Mas e ing DPC++ o P og aming o He e ogeneous Sys ems using C++ and SYCL [11], he examples p o ided by In el and he ideo eco dings o he Danyso wo kshops abou oneAPI [17] and wi h he Rodinia Benchma k Sui e, which is widely used and implemen ed in CUDA. This phase ended wi h he manual mig a ion o he nn benchma k o ensu e all he necessa y ap i udes we e acqui ed Mig a ion phase The DPCT was used o mig a e he Rodinia Benchma k sui e du ing his phase. Also emails we e exchanged wi h In el ep esen a i es discussing he a ious p oblems ha appea ed wi h he mig a ion ool. Benchma king phase This phase included he modifica ion o he benchma ks o enable he iming o he execu ion and he benchma king i sel in he Codeplay's docke and he In el De cloud. The analysis wi h he N idia Visual P ofile was also pe o med du ing his phase. 7 In el-oneAPI o He e ogeneous Compu ing Chap e 2: Mo i a ion and Objec i es Mo i a ion In ecen yea s a clea endency owa ds he e ogenei y in compu ing can be obse ed, no only in high-pe o mance compu e s, whe e in he op-500 lis N idia GPUs a e he mos used accele a o s; bu also in desk ops and handheld de ices. Nowadays mos sma phones include a GPU and he Apple M1 [1] is a sys em on chip, used in he la es Apple compu e s and able s, ha includes an ARM CPU, a GPU and o he accele a o s like a 16-co e Neu al Engine o a ificial in elligence applica ions. These he e ogeneous SOCs, mo e p ominen e e y day, a e clea indica o s o his shi owa d he e ogenei y. A shi ha makes i impo an o ha e a simple and unified way o de eloping o diffe en he e ogeneous a chi ec u es wi hou depending on a specific endo . Figu e 1: Accele a o sys em sha e o he No embe 2020 Top-500 lis [2] 8 In el-oneAPI o He e ogeneous Compu ing OneAPI simplifies so wa e de elopmen by p o iding he same languages and p og amming models ac oss accele a o a chi ec u es [8], [5]. FPGAs FPGAs ha en' seen much adop ion un il he las ew yea s. The eal e olu ion o FPGAs, and hei adop ion as a he e ogeneous accele a o , came om he eplacemen o he low-le el Ha dwa e Defini ion Languages used o p og am FPGAs wi h new p og amming app oaches. Se e al high-le el op ions, o High Le el Syn hesis (HLS) ools, ha e been de eloped since he 1990s, howe e , he fi s iable HLS o scien ific pu poses was Al e a's OpenCL SDK eleased in 2013, ollowed by Xilinx's OpenCL SDKs and HLS ools. Due o he widesp ead adop ion o GPUs a his ime, he usage o FPGAs as accele a o s became a mo e p omising idea. In el acqui ed Al e a in 2015 and eb anded i as he SDK as he In el FPGA SDK o OpenCL. This SDK is s ill ac i e nowadays. O he p ojec s aimed o make HLS app oaches mo e accessible, usually by adding so wa e laye s on op o he endo 's HLS backends. Some examples a e he OpenACC ex ension OpenARC (2016), he OmpSs amewo k ex ended in 2017 o suppo FPGAs and he ETH Zu ich DaCe amewo k, ha in oduced a con ol-flow g aph and GUI based in e ace [5]. Fu u e Accele a o s On op o GPUs and FPGAs, mo e ha dwa e is being explo ed o becoming accele a o s o he e ogeneous sys ems; like he TPU, a machine lea ning accele a o fi s eleased by Google in 2018 and quickly adop ed by N idia [5]. F om he same yea , Oak Ridge Na ional Lab has hos ed an in e na ional con e ence on neu omo phic compu ing, which aims o emula e he ope a ion and s uc u e o he human b ain. Two examples o neu omo phic accele a o s a e he IBM T ueNo h and he In el Loihi [5], [19]. Quan um compu ing is also being explo ed o use as he e ogeneous accele a o s by companies such as D-Wa e and IBM wi h he Q sys em [5]. These a e only some o he accele a o s ha will popula e he wo ld o he e ogeneous compu ing and mo e a e explo ed e e y yea , p omising a u u e o ex eme he e ogenei y and ull o challenges o p og amme s [5]. 15 In el-oneAPI o He e ogeneous Compu ing Chap e 4: DPC++ DPC++ is he hea o oneAPI. DPC++ p og ams a e w i en in ISO C++ and use he SYCL pa allel p og amming model o dis ibu e compu a ion ac oss p ocessing elemen s in a de ice. DPC++ ex ends SYCL wi h ea u es o pe o mance and p oduc i i y. DPC++ is single sou ce, de ice and hos code can be included in he same sou ce file. A DPC++ compile gene a es code o bo h he hos and de ice. Any C++ compile can compile p og ams ha only use he hos subse o DPC++ [9]. Figu e 6: DPC++ C oss-A chi ec u e Compiling [24] A ypical Da a Pa allel C++ consis s o he ollowing sec ions: ●Asynch onous excep ions om ke nels ●De ice selec o s o diffe en accele a o s ●Buffe s and accesso s (o unified sha ed memo y) ●Queues ●pa allel_ o ke nel This poin s can be easily explained ollowing he ec o _add [10] example p o ided by in el: Asynch onous excep ions Ke nels can un asynch onously and in diffe en s ack ames so e o s can' be p opaga ed up o he s ack. Fo his and in o de o ca ch asynch onous excep ions, SYCL queues p o ide e o handle unc ions. 16 In el-oneAPI o He e ogeneous Compu ing s a ic au o excep ion_handle = [ ](cl::sycl::excep ion_lis eLis ) { o (s d::excep ion_p cons &e : eLis ) { y { s d:: e h ow_excep ion(e); } ca ch (s d::excep ion cons &e) { s d:: e mina e(); } } }; … … y { sycl::queue q(d_selec o , excep ion_handle ); … … }ca ch (sycl::excep ion cons &e) { … … } De ice Selec o SYCL and oneAPI p o ide selec o s ha can disco e and p o ide access o he ha dwa e ha is a ailable on he en i onmen (see Figu e 7). The de aul _selec o selec s he mos pe o man accele a o a ailable. // The de aul de ice selec o will selec he mos pe o man de ice. sycl::de aul _selec o d_selec o ; Figu e 7: Buil -in de ice selec o s [11] 17 In el-oneAPI o He e ogeneous Compu ing Buffe s and Accesso s When using buffe s, da a decla ed on he hos is w apped in a buffe and is ans e ed o he accele a o s implici ly by he DPC++ un ime. The accele a o s ead o w i e o he buffe h ough an accesso . The un ime also d aws he ke nel dependencies om he accesso s used, hen dispa ches and uns he ke nels in mos efficien o de . sycl::buffe a_bu (a_a ay); sycl::buffe b_bu (b_a ay); sycl::buffe sum_bu (sum_pa allel.da a(), num_i ems); … … q.submi ([&](sycl::handle &h) { // C ea e an accesso o each buffe wi h access pe mission: ead, w i e, o // ead/w i e. The accesso is used o access he memo y in he buffe . sycl::accesso a(a_bu , h, ead_only); sycl::accesso b(b_bu , h, ead_only); // The sum_accesso is used o s o e (wi h w i e pe mission) he sum da a. sycl::accesso sum(sum_bu , h, w i e_only); … … }); The h ee access modes o buffe s a e shown in Figu e 8. Figu e 8: Buffe access modes [11] 18 In el-oneAPI o He e ogeneous Compu ing Unified Sha ed Memo y Unified Sha ed Memo y (USM) is an al e na i e o buffe s o managing and accessing memo y om he hos and de ice. Explici da a mo emen wi h USM is accomplished, like in CUDA, wi h de ice alloca ions and a special memcpy() ound in he queue and handle classes. Da a can be alloca ed o de ice, hos o bo h (sha ed), see Figu e 9, and i is copied be ween hos and de ice memo y be o e and a e he ke nel execu es [11] [12]. in * a_de ice = sycl::malloc_de ice<in > num_i ems); in * b_de ice = sycl::malloc_de ice<in > num_i ems); in * sum_de ice = sycl::malloc_de ice<in > num_i ems); q.memcpy(a_de ice, a_a ay, sizeo (in ) * num_i ems).wai (); q.memcpy(b_de ice, b_a ay, sizeo (in ) * num_i ems).wai (); … … // Ke nel execu ion … … q.memcpy(sum_a ay, sum_de ice, sizeo (in ) * num_i ems).wai (); Figu e 9: USM alloca ion ypes [11] Queue and pa allel_ o All he con ex and s a es needed o ke nel execu ion a e encapsula ed in a DPC++ queue. By de aul , a queue is c ea ed and associa ed wi h an accele a o , as indica ed by Figu e 10, h ough a de aul selec o when no pa ame e is passed. I can also ake a specific de ice selec o and an asynch onous excep ion handle . 19 In el-oneAPI o He e ogeneous Compu ing Figu e 10: A queue is bound o a single de ice [11] Ke nels a e enqueued o he queue and execu ed. The e a e diffe en ypes o ke nels, ec o _add uses he basic da a-pa allel pa allel_ o ke nel. y { sycl::queue q(d_selec o , excep ion_handle ); … … q.submi ([&](sycl::handle &h) { … … h.pa allel_ o (num_i ems, [=](au o i) { sum[i] = a[i] + b[i]; }); }); }ca ch (sycl::excep ion cons &e) { … … } The ke nel body is he addi ion o wo a ays in he Lambda unc ion. num_i ems, he fi s pa ame e o h.pa allel_ o specifies he ange o da a he ke nel can p ocess. When using USM he ke nel call would be: h.pa allel_ o (num_i ems, [=](au o i) { sum_de ice[i] = a_de ice[i] + b_de ice[i]; }); 20 In el-oneAPI o He e ogeneous Compu ing Chap e 5: DPCT & Rodinia DPC++ Compa ibili y Tool The In el DPC++ Compa ibili y Tool (DPCT) is a componen o he In el oneAPI Base Toolki (see Figu e 11) ha assis s he de elope in he mig a ion o a p og am ha is w i en in CUDA o a p og am w i en in DPC++ [13]. Figu e 11: In el oneAPI Base Toolki [25] The ool wo ks by in e cep ing he build p ocess and eplacing CUDA code wi h he oneAPI coun e pa . Al hough DPCT au oma ically mig a es mos o he code, some manual wo k is equi ed o a ull mig a ion. The ool ou pu s wa nings o indica e how and whe e manual in e en ion is needed. These wa nings ha e an assigned ID, o he o m "DPCT10XX", ha can be consul ed in he De elope Guide and Re e ence. This guide con ains a lis o all he wa nings, hei desc ip ion and a sugges ion o fix i . 21 In el-oneAPI o He e ogeneous Compu ing Rodinia Benchma ks Quo ing Rodinia's websi e: "A ision o he e ogeneous compu e sys ems ha inco po a e di e se accele a o s and au oma ically selec he bes compu a ional uni o a pa icula ask is widely sha ed among esea che s and many indus y analys s; howe e , he e a e no ag eed-upon benchma ks o suppo he esea ch needed in he de elopmen o such a pla o m. The e a e many sui es o pa allel compu ing on gene al-pu pose CPU a chi ec u es, bu accele a o s all in o a gap ha is no co e ed by p e ious benchma k de elopmen . Rodinia is eleased o add ess his conce n." [14]. The Rodinia Benchma k Sui e is implemen ed in CUDA, OpenMP and OpenCL and includes he ollowing applica ions: Applica ion Dwa es Domain Implemen a ions Leukocy e S uc u ed G id Medical Imaging CUDA, OMP, OCL Hea Wall S uc u ed G id Medical Imaging CUDA, OMP, OCL MUMme GPU G aph T a e sal Bioin o ma ics CUDA, OMP CFD Sol e Uns uc u ed G id Fluid Dynamics CUDA, OMP, OCL LU Decomposi ion Dense Linea Algeb a Linea Algeb a CUDA, OMP, OCL Ho Spo S uc u ed G id Physics Simula ion CUDA, OMP, OCL Back P opaga ion Uns uc u ed G id Pa e n Recogni ion CUDA, OMP, OCL Needleman-Wunsch Dynamic P og amming Bioin o ma ics CUDA, OMP, OCL Kmeans Dense Linea Algeb a Da a Mining CUDA, OMP, OCL B ead h-Fi s Sea ch G aph T a e sal G aph Algo i hms CUDA, OMP, OCL SRAD S uc u ed G id Image P ocessing CUDA, OMP, OCL S eamclus e Dense Linea Algeb a Da a Mining CUDA, OMP, OCL 22 In el-oneAPI o He e ogeneous Compu ing Pa icle Fil e S uc u ed G id Medical Imaging CUDA, OMP, OCL Pa hFinde Dynamic P og amming G id T a e sal CUDA, OMP, OCL Gaussian Elimina ion Dense Linea Algeb a Linea Algeb a CUDA, OCL k-Nea es Neighbo s Dense Linea Algeb a Da a Mining CUDA, OMP, OCL La aMD2 N-Body Molecula Dynamics CUDA, OMP, OCL Myocy e S uc u ed G id Biological Simula ion CUDA, OMP, OCL B+ T ee G aph T a e sal Sea ch CUDA, OMP, OCL GPUDWT Spec al Me hod Image/Video Comp ession CUDA, OCL Hyb id So So ing So ing Algo i hms CUDA, OCL Ho spo 3D S uc u ed G id Physics Simula ion CUDA, OMP, OCL Huffman Fini e S a e Machine Lossless da a comp ession CUDA, OCL Table 1: Rodinia 3.1 applica ions [14] 23 In el-oneAPI o He e ogeneous Compu ing Chap e 6: Me hodology The wo kflow, isualized in Figu e 12, was di ided in o wo main p ocesses: he mig a ion p ocess and he benchma king p ocess. Figu e 12: Wo kflow Mig a ion p ocess In o de o mig a e he Rodinia benchma ks o CUDA o DPC++ he ollowing s eps we e ollowed: 1. Gene a e a compila ion da abase wi h he ool in e cep -build. This c ea es a json file wi h all he compile in oca ions and s o es he names o he inpu files and he compile op ions. This is done wi h he command in e cep -build make. 2. Use he In el DPC++ Compa ibili y Tool o mig a e he code. The command dpc -p compile_commands.json --in- oo =. --ou - oo =mig a ion mig a es he files in he cu en di ec o y and s o es he esul in he mig a ion olde . Du ing his s ep commen s a e inse ed whe e he ool couldn' mig a e he code o whe e he use should e iew he mig a ion. 3. Ve i y he mig a ion and add ess any DPCT wa nings gene a ed. Consul he Diagnos ics Re e ence [18] o de ailed in o ma ion abou he DPCT wa nings. 24 In el-oneAPI o He e ogeneous Compu ing A e analyzing he esul s o hese measu emen s, he benchma ks ha had he wo s pe o mance compa ed o he o iginal CUDA applica ion ecei ed a mo e in-dep h analysis using he N idia Visual P ofile [21]. Figu e 16: Wo kflow (CUDA s oneAPI) P ofiling On op o he execu ion o he ins umen alized code, he mos ele an applica ions o he Rodinia benchma ks we e analysed using he N idia Visual P ofile , a pe o mance p ofiling ool p o ided by N idia. Figu e 17: Example o he N idia Visual P ofile [21] 31 In el-oneAPI o He e ogeneous Compu ing This ool allows pe o ming a low-le el analysis o he code, showing de ails abou he ke nels and he memo y ans e no isible by o he means. O he a chi ec u es Taking ad an age o he po abili y p o ided by oneAPI, he benchma ks we e also execu ed in he In el De cloud, wi h wo In el XeonGold6128 CPUs and an In el UHDG aphicsP630 GPU as de ices. The In el De cloud allows i s use s o execu e applica ions wi h diffe en ha dwa e en i onmen s ha include a ious In el CPUs, GPUs and FPGAs. Figu e 18: Wo kflow (benchma king) 32 In el-oneAPI o He e ogeneous Compu ing Chap e 7: Mig a ion Resul s Du ing he mig a ion p ocess he DPC++ Compa ibili y Tool gene a ed a se ies o wa nings indica ing possible p oblems and he need o manual in e en ion by he use . These a e discussed in he ollowing sec ions. Wa nings Ac oss all he benchma ks 99 files we e p ocessed by he DPC++ Compa ibili y Tool, wi h a o al o 43485 lines o code. I ga e a o al o 461 wa nings, wi h an a e age o 4.65 wa nings pe file o a wa ning e e y 94.3 lines. Wa ning code Numbe o appea ances DPCT1000 4 DPCT1001 4 DPCT1003 171 DPCT1004 1 DPCT1005 15 DPCT1009 45 DPCT1010 38 DPCT1012 20 DPCT1019 1 DPCT1022 2 DPCT1024 2 DPCT1026 8 DPCT1027 2 DPCT1035 9 DPCT1039 7 DPCT1049 75 33 In el-oneAPI o He e ogeneous Compu ing DPCT1051 2 DPCT1059 14 DPCT1064 2 DPCT1065 29 DPCT1072 1 DPCT1077 9 To al 461 Table 3: Wa nings gi en by DPCT See he In el DPC++ Compa ibili y Tool De elope Guide and Re e ence [18] o mo e in o ma ion abou hese wa nings. These wa nings can be g ouped in he ollowing ca ego ies: ●E o handling ela ed wa nings: DPCT1000, DPCT1001, DPCT1003, DPCT1009, DPCT1010, DPCT1024. To al: 264 - 57.3% ●De ice in o ma ion ela ed wa nings: DPCT1005, DPCT1019, DPCT1022, DPCT1051, DPCT1072. To al: 21 - 4.6% ●Ke nel in oca ion wa nings: DPCT1049. To al: 75 - 16.3% ●Time measu emen wa nings: DPCT1012. To al: 20 - 4.3% ●Wa nings caused by he emo al o unnecessa y unc ion calls: DPCT1026, DPCT1027. To al 10 - 2.1% ●Wa ning gene a ed because SYCL no suppo s some hing: DPCT1059. To al 14 - 3% 34 In el-oneAPI o He e ogeneous Compu ing ●Pe o mance imp o ing sugges ions: DPCT1065. To al 29 - 6.3% ●Mac o ela ed wa nings: DPCT1064, DPCT1077. To al 11 - 2.4% ●O he : DPCT1004, DPCT1035, DPCT1039 To al 17 - 3.7% Figu e 19: Dis ibu ion o he wa nings gene a ed by DPCT Mos o he wa nings gene a ed by he compa ibili y ool (57.3%) a e caused by he ac ha SYCL uses excep ions ins ead o e o codes. Al hough i migh be desi able in o de o handle any e o ha migh occu in un ime, no manual modifica ions we e manda o y o hese wa nings as he ool modifies all e o checks so hey always e u n a success. The second g oup o wa nings appea in e e y ke nel in oca ion and simply eminds he use he ac ha he a ge ed de ice migh ha e a smalle limi o he wo kg oup size. This is usually he case when compa ing an N idia GPU wi h an in eg a ed In el GPU. O he es i should be no ed ha he wa nings ela ed wi h mac os and de ice in o ma ion a e he ones ha equi e he mos manual in e en ion om he use . 35 In el-oneAPI o He e ogeneous Compu ing I is wo h men ioning he ac ha , as he mig a ed code is a benchma k, he amoun o ime measu emen ela ed wa nings is much bigge han i would be in o he codes. In he end wen y ou o he wen y h ee benchma ks we e success ully mig a ed wi hou majo manual in e en ion, esul ing in a success a e o almos 87%. In he cases whe e he mig a ion wasn' success ul i was due o issues known by In el. Wo k is being done o sol e hese issues and a fix is planned in he nex upda e (oneAPI 2021.3) o some o hem. Benchma k Success ul mig a ion b+ ee ✓ backd op ✓ b s ✓ c d ✓ dw 2d ✓ gaussian ✓ hea wall ✓ ho spo ✓ ho spo 3D ✓ huffman ✓ hyb idso X kmeans X la aMD ✓ leukocy e ✓ lud ✓ mumme gpu X myocy e ✓ nn ✓ nw ✓ pa iclefil e ✓ pa hfinde ✓ s ad ✓ s eamclus e ✓ Table 4: Mig a ion successes 36 In el-oneAPI o He e ogeneous Compu ing Manual modifica ions A e he au oma ic mig a ion some manual modifica ions o he code we e necessa y, always add essing he wa ning messages gene a ed by he ool. As a summa y, we can illus a e in he ollowing lines he main modifica ions pe o med: ●The wo kg oup size migh need o be adjus ed depending on he de ice used: DPCT sugges s que ying in o::de ice::max_wo k_g oup_sice o ge he de ice limi and adjus he wo kg oup size acco dingly. ●When he block size is specified wi h a mac o and used o c ea e a sycl:: ange he expanded alue should be changed back o he mac o. When his is he case he ool lea es he o iginal mac o commen ed. Fo example: q_c 1.submi ([&](sycl::handle &cgh) { sycl:: ange<2>weigh _ma ix_ ange_c 1(16 /*HEIGHT*/,16 /*WIDTH*/); ... } should be eplaced wi h q_c 1.submi ([&](sycl::handle &cgh) { sycl:: ange<2>weigh _ma ix_ ange_c 1(HEIGHT ,WIDTH); ... } ●As SYCL uses excep ions ins ead o e o codes e e y check is modified by he ool o always succeed. P ope e o checking migh be added manually wi h y-ca ch cons uc ions as i is de ailed below: A sou ce code line like check_e o (cudaMalloc(( oid **) &a ayX_GPU, sizeo (double)*Npa icles)); will be mig a ed o check_e o ((a ayX_GPU =sycl::malloc_de ice<double>(Npa icles,q_c 1), 0)); In o de o handle any possible un ime e o his should be changed o some hing like y{ a ayX_GPU =sycl::malloc_de ice<double>(Npa icles, q_c 1); ca ch (sycl::excep ion cons &e) { Handle excep ion } 37 In el-oneAPI o He e ogeneous Compu ing ●The de ice selec ion logic mus be manually e iewed as all DPC++ de ices (no only he GPU) can be used o submi asks. I is impo an o ake his in o accoun when he d i e e sion is used o de ec a GPU in he o iginal CUDA sou ce. i (de P op.ge _majo _ e sion() < 1) { … } used in he o iginal CUDA code o de ec he p esence o a GPU can be eplaced by i (!dpc ::ge _cu en _de ice().is_gpu()){ … } In his example he execu ion finished when no GPU was a ailable, bu i has no sense on DPC++ as i allows o un he de ice code in o o he de ices, such as CPU. In his case he check was modified so i con inues he execu ion in ano he de ice i no GPU is a ailable. ●Many CUDA de ice p ope ies don' ha e a SYCL equi alen , a e sligh ly diffe en o a en' cu en ly suppo ed. This will cause in many occasions he e ie al o inco ec alues. Fo his eason he use mus manually e iew and co ec he in o ma ion que ies o he de ice. ●I he ool makes some assump ion o gene a e he DPC++ code, i will place a wa ning ins uc ing he use o e iew he modified code. Fo example DPCT1039: "The In el® DPC++ Compa ibili y Tool deduces whe he he fi s pa ame e o an a omic unc ion poin s o a global memo y add ess space o a local memo y space, using he las assignmen ’s alue o he fi s pa ame e o he a omic unc ion..." ●SYCL only suppo s 4-channel image o ma so he code needs o be manually adjus ed. An example o he needed modifica ions is gi en in he In el DPC++ Compa ibili y Tool De elope Guide and Re e ence [18]. ●When a unc ion call is used in a mac o defini ion i migh need o be mig a ed diffe en ly depending on how he mac o is called. All uses o his mac o mus be e iewed. ●Inside a ke nel, DPCT sugges s eplacing ba ie () wi h ba ie (sycl::access:: ence_space::local_space) o be e pe o mance i he e is no access o global memo y. In his case he use should check he memo y accesses and he modifica ion. ●I a mac o edefines a s anda d SYCL ype i may cause conflic s. The de elope guide and e e ence sugges s he use o ename he mac o. 38 In el-oneAPI o He e ogeneous Compu ing P oblems encoun e ed ●CL_INVALID_IMAGE_SIZE: In e e y benchma k whe e an image da a ype is used he execu ion ends wi h an excep ion (CL_INVALID_IMAGE_SIZE). This is a known issue ha occu s when he in o::de ice::image2d_max_wid h alue o he de ice is less han he wid h o he image passed in o he ke nel. A e discussing his wi h some In el's s aff a wo ka ound was sugges ed o CPU. The e is no solu ion o GPU ye . ●When a unc ion is called inside a complex mac o some imes he ool e oneously eplaces he pa ame e s o he unc ion called wi h names om an in oca ion u he down he code. #define BIND_TEX_ARRAY( ex,a ,desc)do { CUDA_SAFE_CALL(cudaBindTex u eToA ay( ex, a , desc)); ++num_bind_ ex_calls; }while(0) was mig a ed o #define BIND_TEX_ARRAY( ex,a ,desc)do { CUDA_SAFE_CALL((BIND_TEX_ARRAY(node ex, (cudaA ay*) e ->d_node_ ex_a ay, nodeTex u eDesc).a ach(BIND_TEX_ARRAY(node ex, (cudaA ay*) e ->d_node_ ex_a ay,nodeTex u eDesc), BIND_TEX_ARRAY(node ex, (cudaA ay*) e ->d_node_ ex_a ay, nodeTex u eDesc)), 0)); ++num_bind_ ex_calls; }while(0) This is a known bug and a fix is planned o upda e 2021.3 ●The la es CUDA suppo ed e sion is 11.1. This causes small p oblems wi h he in e cep -build ool as i won' find some lib a ies. Re-execu ing he command wi h --append esumes he execu ion om he e o , effec i ely fixing he p oblem. 39 In el-oneAPI o He e ogeneous Compu ing ●Lea ing some posi ions o an a ay unini ialized migh esul in a segmen a ion aul . This occu ed in he pa iclefil e benchma k whe e he unc ion oid s elDisk(in * disk,in adius){ in diame e = adius*2-1; in x,y; o (x=0;x<diame e ;x++){ o (y=0;y<diame e ;y++){ double dis ance = sq (pow((double)(x- adius +1), 2) + pow((double)(y- adius +1), 2)); i (dis ance < adius) disk[x*diame e +y] = 1; } } } le a posi ion unini ialized when he dis ance was g ea e o equal o he adius. The o iginal CUDA e sion an wi hou any p oblem, bu his caused he men ioned segmen a ion aul in DPC++. The solu ion was se ing o 0 he unini ialised memo y posi ions. 40 In el-oneAPI o He e ogeneous Compu ing Figu e 25: dw 2d (oneAPI) la ency analysis 47 In el-oneAPI o He e ogeneous Compu ing O e all pe o mance In el oneAPI has he added ad an age o no being es ic ed o one endo like CUDA. Fo his eason he benchma ks we e also es ed in he In el De cloud using a pai o In el XeonGold6128 CPU and an in eg a ed In el UHDG aphicsP630 GPU as de ices. As seen in Figu e 26, al hough, as expec ed, he dedica ed N idia GPU ou pe o med he CPU and he in eg a ed GPU in almos e e y case, he e a e ins ances whe e he CPU and/o he in eg a ed GPU ma ched o e en ou pe o med he N idia GPU. This was he case when he applica ion didn' ake ad an age o he massi e pa alleliza ion capabili ies o a GPU o he o e head caused by he mo emen o memo y be ween he hos and he de ice wasn' jus ified by he speedup p o ided by he accele a o . Figu e 26: To al execu ion imes ac oss all es ed de ices (loga i hmic scale) Con inuing wi h he example o nn ha calcula es he euclidean dis ance be ween a se o poin s, he execu ion was much as e in he dedica ed GPU due o use o sq () and he high pa alleliza ion o he ke nel. The in eg a ed GPU akes ad an age o his pa alleliza ion, hus ou pe o ming he CPUs, bu lacks he special compu e uni s ha he dedica ed GPU has o calcula ing squa e oo s. By he o he hand, he myocy e es is a clea example o a case whe e he speedup p o ided by he GPU does no jus i y he o e head in oduced by he memo y ans e . In his case he in eg a ed GPU akes ad an age o he educed la ency ha p o ides being in he same chip as he hos and ou pe o ms he dedica ed GPU, bu he as e and mo e complex co es o he CPUs gi es hem he ad an age in his case whe e he massi e pa alleliza ion capabili ies o he GPUs a e no ully u ilized. 48 In el-oneAPI o He e ogeneous Compu ing Chap e 9: Conclusions As s a ed in chap e s 2 and 3, he e is a clea endency owa ds he e ogenei y and, in ecen yea s, many me hods o p og amming hese he e ogeneous sys ems ha e appea ed wi h g ea e o lesse success. OneAPI aims o p o ide a simple and unified p og amming me hod o all hese diffe en he e ogeneous sys ems, independen ly o he accele a o used o i 's endo , allowing high and low le el app oaches acili a ing he p og amme 's wo k in his highly he e ogeneous wo ld we a e mo ing owa ds. The wo k done mig a ing, benchma king and p ofiling he Rodinia Benchma k sui e shows p omising esul s: ●Al hough some wo k emains o be done, he Da a Pa allel C++ Compa ibili y Tool g ea ly s eamlines he mig a ion p ocess om CUDA o oneAPI. Twen y ou o he wen y h ee benchma ks we e success ully mig a ed wi hou majo manual in e en ion, esul ing in a success a e o almos 87% and allowing p e iously N idia specific code o un in o he accele a o s. ●Mos o he p oblems encoun e ed du ing he mig a ion we e caused by known bugs ha will be fixed o e ime. These bugs we e he main eason why he mig a ion ailed in hyb idso , kmeans and mumme gpu. ●The pe o mance o he esul ing mig a ed code is, in many cases, compa able o he o iginal CUDA sou ce e en wi hou applying any op imiza ion. Backp op, gaussian and s eamclus e achie e a pe o mance diffe ence o 1% o less and b s, eule 3D_double, la aMD, lud and pa hfinde keep he o e head unde 17%. ●Rega ding he basic memo y ope a ions, de ice memo y alloca ion, copy memo y hos o de ice, copy memo y de ice o hos and de ice memo y ee, he pe o mance is equi alen and any o e head de ec ed a ound hese ope a ions is caused by o he ins uc ions in oduced by oneAPI. ●In he cases whe e he pe o mance loss is mo e significan i is due o ei he poo memo y and egis e usage o insufficien op imiza ion o he compiled code. Bo h hese p oblems should be sol ed, o a leas mi iga ed, by a mo e ma u e and ad anced compile . 49 In el-oneAPI o He e ogeneous Compu ing E en wi h he pe o mance loss, he capabili y o execu ing he ke nels in a mul i ude o diffe en de ices, no only N idia GPUs, is a g ea ad an age o e CUDA ha jus ifies he effo pu on he mig a ion o , a leas , makes i wo h conside ing i . I is also wo h no icing ha oneAPI wi h he In el XeonGold6128 CPUs achie ed simila pe o mance o he o iginal CUDA e sion o pa iclefil e _nai e and su passed i in eule 3D_double, myocy e and pa iclefil e _floa . Bo h, oneAPI and DPCT, a e in ac i e de elopmen . The esul s o his p ojec we e discussed wi h In el's s aff and mos o he ound p oblems and bugs we e known by In el; some fixes a e e en planned o he nex upda e and u he op imiza ions and compile upg ades migh educe he obse ed pe o mance losses. All o his eaffi ms he belie ha oneAPI and SYCL will become a u u e s anda d o he e ogeneous p og amming. 50 In el-oneAPI o He e ogeneous Compu ing Bibliog aphy [1] "Apple M1 - Wikipedia" [Online]. A ailable: h ps://en.wikipedia.o g/wiki/Apple_M1 [2] "Lis S a is ics | TOP500" [Online]. A ailable: h ps://www. op500.o g/s a is ics/lis [3] "De elopmen o e Time | TOP500" [Online]. A ailable: h ps://www. op500.o g/s a is ics/o e ime [4] "CUDA Zone | NVIDIA De elope " [Online]. A ailable: h ps://de elope .n idia.com/cuda-zone [5] Jacob Lambe , "E olu ion o P og amming App oaches o High-Pe o mance He e ogeneous Sys ems", Compu e Science Uni e si y o O egon, Uni ed S a es, Decembe 14, 2020. A ailable: h ps://www.cs.uo egon.edu/Repo s/AREA-202012-Lambe .pd [6] P adeep Gup a, "CUDA Re eshe : The CUDA P og amming Model | N idia De elope Blog" [Online]. A ailable: h ps://de elope .n idia.com/blog/cuda- e eshe -cuda-p og amming-model [7] "SYCL O e iew - The Kh onos G oup Inc" [Online]. A ailable: h ps://www.kh onos.o g/sycl/ [8] "Welcome o oneAPI - oneAPI Documen a ion" [Online]. A ailable: h ps://docs.oneapi.com/ e sions/la es /index.h ml [9] "DPC++ Re e ence - DPC++ Re e ence Documen a ion" [Online]. A ailable: h ps://docs.oneapi.com/ e sions/la es /dpcpp/index.h ml#da a-pa allel-c-dpc [10] "DPC++ Founda ions Code Sample" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/a icles/dpcpp- ounda ions- code-sample.h ml [11] James Reinde s, e al., "Da a Pa allel C++ Mas e ing DPC++ o P og amming o He e ogeneous Sys ems using C++ and SYCL" A ailable: h ps://link.sp inge .com/book/10.1007%2F978-1-4842-5574-2 51 In el-oneAPI o He e ogeneous Compu ing [12] "Da a Pa allel C++ USM Code Samples" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/a icles/dpcpp-usm-code-sa mple.h ml [13] "Ge S a ed wi h he In el DPC++ Compa ibili y Tool" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/documen a ion/ge -s a ed- wi h-in el-dpcpp-compa ibili y- ool/ op.h ml [14] "s a [Rodinia]" [Online]. A ailable: h p:// odinia.cs. i ginia.edu/doku.php [15] "In el DPC++ Compa ibili y Tool" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/ ools/oneapi/componen s/dp c-compa ibili y- ool.h ml [16] "Ruyk/dpcpp-cuda-examples-docke " [Online]. A ailable: h ps://gi hub.com/Ruyk/dpcpp-cuda-examples-docke [17] "Encuen o Online Desa ollado es In el So wa e" [Online]. A ailable: h ps://www.danyso .com/encuen o-online-desa ollado es-in el-so wa e [18] "Diagnos ics Re e ence" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/documen a ion/in el-dpcpp- compa ibili y- ool-use -guide/ op/diagnos ics- e e ence.h ml [19] "Neu omo phic Compu ing - Nex Gene a ion o AI" [Online]. A ailable: h ps://www.in el.com/con en /www/us/en/ esea ch/neu omo phic-compu ing.h ml [20] "Pandemic dis o s global GPU ma ke esul s | Jon Peddie Resea ch" [Online]. A ailable: h ps://www.jonpeddie.com/p ess- eleases/pandemic-dis o s-global-gpu-ma ke - e sul s [21] "NVIDIA Visual P ofile | NVIDIA De elope " [Online]. A ailable: h ps://de elope .n idia.com/n idia- isual-p ofile [22] "In el oneAPI P og amming Guide - Glossa y" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/documen a ion/oneapi-p og amming-guide/ op/glossa y.h ml [23] "Th ead block (CUDA p og amming) - Wikipedia" [Online]. A ailable: h ps://en.wikipedia.o g/wiki/Th ead_block_%28CUDA_p og amming%29 52 In el-oneAPI o He e ogeneous Compu ing [24] "In el oneAPI DPC++/C++ Compile - Da a Pa allel C++ o C oss-A chi ec u e Applica ions" [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/ ools/oneapi/componen s/dp c-compile .h ml [25] "In el oneAPI Base Toolki o C oss-A chi ec u e De elopmen " [Online]. A ailable: h ps://so wa e.in el.com/con en /www/us/en/de elop/ ools/oneapi/base- oolki .h ml [26] "CUDA Bina y U ili ies :: CUDA Toolki Documen a ion" [Online]. A ailable: h ps://docs.n idia.com/cuda/cuda-bina y-u ili ies/index.h ml#ins uc ion-se - e 53 In el-oneAPI o He e ogeneous Compu ing Glossa y Accele a o :Specialized componen con aining compu e esou ces ha can quickly execu e a subse o ope a ions. Examples include CPU, FPGA, GPU [22]. Accesso : Communica es he desi ed loca ion (hos , de ice) and mode ( ead, w i e) o access [22]. Buffe : Memo y objec ha communica es he ype and numbe o i ems o ha ype o be communica ed o he de ice o compu a ion [22]. Compu e uni :A g ouping o p ocessing elemen s in o a ‘co e’ ha con ains sha ed elemen s o use be ween he p ocessing elemen s and wi h as e access han memo y esiding on o he compu e uni s on he de ice [22]. De ice:An accele a o o specialized componen con aining compu e esou ces ha can quickly execu e a subse o ope a ions. A CPU can be employed as a de ice, bu when i is, i is being employed as an accele a o . Examples include CPU, FPGA, GPU [22]. FPGA: Field-p og ammable ga e a ay. I is an in eg a ed ci cui designed o be configu ed a e manu ac u ing. GPU: G aphics p ocessing uni . Ke nel:Code ha execu es on he de ice [22]. P ocessing elemen :Indi idual engine o compu a ion ha makes up a compu e uni [22]. Sha ed memo y: Special memo y egion ha can be accessed by all h eads in he same g oup. SIMD: Single ins uc ion mul iple da a. SOC: Sys em on Chip. TPU: Tenso P ocessing Uni . I is an AI accele a o o neu al ne wo k machine lea ning. Wa p: A wa p is a se o h eads wi hin a h ead block such ha all he h eads in a wa p execu e he same ins uc ion [23]. Wo kg oup:Collec ion o wo k-i ems ha execu e on a compu e uni [22]. Wo k-i em: Basic uni o compu a ion in he oneAPI p og amming model. I is associa ed wi h a ke nel which execu es on he p ocessing elemen [22]. 54