SynFull-RTL: evaluation methodology for RTL NoC designs

JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 1 SynFull-RTL: E alua ion Me hodology o RTL NoC Designs Neiel Ley a, Ba celona Supe compu ing Cen e , Uni e si a Poli ` ecnica de Ca alunya, Ali eza Monemi, Ba celona Supe compu ing Cen e , and En ique Vallejo, Uni e sidad de Can ab ia, ICS FORTH Abs ac —SynFull is a widely employed ool ha gene a es ealis ic a ic pa e ns o he pe o mance e alua ion o a NoC. In his wo k, we iden i y he main limi a ions o SynFull: high a iabili y and long simula ion ime and also ha hese limi a ions inc ease when SynFull is in eg a ed wi h RTL designs. SynFull-RTL employs a s a is ical app oach, simula ing each applica ion mac o-phase only once and a e aging acco ding o i s p obabili y o occu ence and he measu ed a ic load. SynFull- RTL ob ains highe accu acy han he o iginal e sion and educed a iabili y, wi h obse ed 40× educ ion in simula ion ime and esou ces. A use-case wi h P oSMART alida es he esul s. Index Te ms—SynFull, Ve ila o . I. INTRODUCTION SIMULATION is e y impo an s ep o e alua e he pe o - mance o new designs. The ideli y o he esul s depends mainly on he cha ac e is ics o he ne wo k and he a ic model used. The mo e de ailed a ic in he model is used, he highe he accu acy o he esul s, bu also he highe he load and he simula ion ime. SynFull [1] in oduces a Ma ko -chain-based syn he ic model ha mimics eal applica ion a ic. I is e y a ac i e, because i allows o eed he simula ion wi h a ealis ic a ic pa e n in a simple and as way. SynFull di ides he simula ion in o di e en long pe iods o ime, deno ed mac o-phases, de i ed om aces o a eal execu ion o he applica ion. Howe e , in his pape we iden i y ha SynFull does no execu e all he phases in he p opo ion ound in he eal applica ion. Fo his eason, he a iabili y o he gene a ed a ic may be e y la ge and he esul s o he simula ions become un eliable, since di e en execu ions using he same SynFull model may ecei e a ic ha signi ican ly di e s. This may be emedied by unning e y long simula ions, bu i inc eases compu a ional equi emen s. In his wo k, we modi y SynFull o eed an RTL simula- o . Because o i s inc eased simula ion equi emen s, u he enleng hening simula ions o mi iga e he inna e a iabili y becomes un easible. To a oid hese p oblems, we in oduce a no el me hodology based on SynFull, deno ed SynFull- RTL, which is bo h as e and mo e p ecise han he o igi- nal implemen a ion. Al hough he p oposed me hodology is no exclusi e o RTL simula ion, i is pa icula ly ele an Manusc ip ecei ed XXX, 2022; e ised XXX, 2022; accep ed XXX, 2022. This a icle was p esen ed a he 2022 In e na ional Symposium on Ne wo ks-on-Chip and appea s as pa o he Design & Tes special issue. o hese models because o hei inc eased simula ion cos . SynFull-RTL e alua es each mac o-phase in isola ion, and de- e mines he a e age la ency esul acco ding o he heo e ical p obabili y o occu ence o each mac o-phase in s eady-s a e and i s measu ed a ic le el. The me hodology in oduced in his wo k p o ides measu e- men s as e and wi h high accu acy, le e aging he SynFull models based on eal applica ions. The main con ibu ions a e: •An in eg a ed model o SynFull wi h RTL NoC ou e models, which allows o quickly e alua e he pe o mance o he model using ealis ic a ic. •An analysis o he main limi a ions o SynFull in his en i onmen , mainly he use o a single seed, i s high a iabili y and he long simula ion imes. •SynFull-RTL, an e alua ion me hodology ha samples SynFull mac o-phases and a e ages hem acco ding o hei p obabili y o occu ence and hei a ic le el. •An e alua ion o SynFull-RTL, which shows ha i can p o ide mo e accu a e esul s han he o iginal SynFull app oach while equi ing 8× o 40×less simula ed cycles and educing ime- o-solu ion by up o 40×. II. BACKGROUND A. T a ic Modelling T a ic modelling is e y ele an o accu a e pe o mance e alua ion o NoCs. The e a e mul iple me hodologies o gene a e a ic, including syn he ic models (such as andom uni o m o pe mu a ions), use o la ge aces such as Ne- ace [2] o simula ing a whole applica ion, such as gem5 modelling. T aces o whole applica ion simula ion p o ide he highes accu acy, bu hey equi e huge iles, la ge amoun s o memo y and long simula ion ime. [2] p esen s an in e es ing discussion and o e iew o he limi a ions o di e en models. Fo he designe s, a ele an ea u e is execu ion ime; ha is, he designe s o en need o es a new expe imen al ea u e in he NoC and ha e a hin abou i s es ima ed pe o mance. In hese cases, o wai ing imes is undesi able, because i delays decisions abou new ea u es and possible changes. B. SynFull SynFull is a ool designed o acili a e as and ealis ic NoC e alua ions. SynFull gene a es andom a ic based on eal applica ions, wi hou elying on aces o ull-sys em simula ion. SynFull has been used in many ele an esea ch wo ks, such as [3]–[7]. 0000–0000/00$00.00 © 2021 IEEE © 2022 IEEE. Pe sonal use o his ma e ial is pe mi ed. Pe mission om IEEE mus be ob ained o all o he uses, in any cu en o u u e media, including ep in ing/ epublishing his ma e ial o ad e ising o p omo ional pu poses,c ea ing new collec i e wo ks, o esale o edis ibu ion o se e s o lis s, o euse o any copy igh ed componen o his wo k in o he wo ks. DOI 10.1109/MDAT.2022.3202996 JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 2 Mphase_1 P1=45.78% 100% 96.53% Mphase_2 P2=1.88% 0.86% Mphase_3 P3=52.32% 2.59% 15.78% 78.94% 5.26% 2.46% 0% 97.53% Fig. 1. S a e diag am o he mac o-phases in he Ba nes model. The numbe nex o each a ow indica es he ansi ion p obabili y. SynFull di ides each applica ion in o blocks called mac o- phases, each o hem las ing o an in e al o 500,000 cycles. Each mac o-phase models he beha io o an in e al o he applica ion. Each mac o-phase has di e en cha ac e is ics, such as he packe injec ion a e and he pa e n o des ina ions. SynFull models de ine mac o-phases as nodes o a Ma ko chain. Each model de ines a low numbe (2 o 10 in he a ailable models) o mac o-phase ypes, ob ained om aces o he ac ual applica ion. Inside each mac o-phase, he model ha de ines he a ic is deno ed mic o-phase, and i also ollows an in e nal Ma ko model. The model also de ines he ansi ion p obabili y be ween mac o-phases iand j,pij, and he p obabili y o occu ence o each mac o-phase iin s eady-s a e, Pi. In a Ma ko model, hey a e ela ed by Pj=Pi(Pi×pij). Figu e 1 depic s he mac o-phase s uc u e o he Ba nes model. I comp ises 3 mac o-phase ypes, each one wi h a di e en p obabili y o occu ence (P1,P2, and P3). No e ha hese p obabili ies di e la gely, wi h P2being e y low. T ansi ion p obabili ies (a ows) indica e how likely is o change om one mac o-phase ype o ano he , a e each in e al. The s a ing mac o-phase is always ype 1. A simula ion wi h SynFull injec s a ic acco ding o he cu en mac o-phase ype o he comple e in e al o 500,000 cycles. Then, i andomly changes o ano he mac o-phase, acco ding o he ansi ion p obabili ies in he model. Al hough SynFull implemen s a ha dcoded andom seed, he sequence o mac o-phases simula ed is no always he same o each a ic model. This issue, which is u he analyzed in Sec ion III-B, occu s because he andom sequence a ies wi h packe ecep- ion, which in u n depends on he simula ed NoC model. C. P oSMART P oSMART1[8] is a ully pa ame izable NoC ou e de- sign w i en in Sys emVe ilog, in eg a ed wi h a module ha suppo s mul ihop bypass. The NoC is con igu able wi h many s a e-o - he-a ea u es such as i ual channels, i ual ne - wo ks, ha d-buil -in QoS, mul icas , mul ihop bypass, di e en ou ing algo i hms, and ne wo k ypologies. P oSMART allows injec ed li s o skip mul iple ou e s in a dimension wi hin a single cycle, esul ing in a d as ic 1h ps://gi hub.com/amonemi/P oNoC la ency educ ion. The sys em employs he HPCMax pa- ame e o de ine he maximum mul ihop leng h; in a 4×4 mesh, HPCMax a ies om 1 o 3. This allows o model bo h a adi ional mesh and one wi h mul ihop bypass. The e ec i eness o P oSMART is only e alua ed in [8] using syn he ic a ic pa e ns, and i lacks empi ical e alua ions using ealis ic a ic pa e ns. This wo k employs P oSMART as a use-case o alida e he p oposed e alua ion me hodology. III. SYNFULL-RTL INTEGRATION AND ANALYSIS OF LIMITATIONS This sec ion in oduces he in eg a ion o SynFull wi h an RTL NoC model o suppo ealis ic RTL simula ions. I ollows wi h an analysis o he main limi a ions o SynFull. A. In eg a ion o P oSMART wi h SynFull The P oSMART GUI p o ides a se o au oma ion oolse ha gene a es a simula ion model o any cus om-de ined NoC model and also in eg a es se e al ools o acili a e pe o mance e alua ion. This sec ion discusses wo ools used o in eg a ion wi h SynFull: Ques asim, and Ve ila o . 1) Ques asim: I wo ks di ec ly on RTL code and uses he DPI unc ions o wo k wi h o eign p og amming languages like C++. The C++ module is compiled sepa a ely as a sha ed bina y and he loca ion is passed o Ques asim wi h a lag, so i can ind he bina y a simula ion ime. This ool was used in he i s app oach o SynFull and i was decided o keep he socke communica ion. The DPI unc ion has wo pa s, one is so wa e and ano he one is RTL code. In so wa e, ecei ing packe s om he socke is handled and he packe s a e mapped on a di ec connec ion o he RTL pa o he DPI unc ion, whe e we ha e po s o all he inpu queues whe e each packe is di ided in o li s. 2) Ve ila o : I con e s he NoC RTL code o an equi alen cycle-accu a e C++ model using Ve ila o simula o . Then i alloca es h ee di e en so wa e-based queues o each NoC’s endpoin namely as injec ion, a e sal, and ejec ion queues. These queues keep he desi ed a ic packe s’ in o ma ion such as injec ion imes amp, he des ina ion node add ess, and packe size in li s. The simula ion model moni o s he injec ion queues and eeds a co esponding packe o he NoC acco ding o injec ion imes amp and NoC c edi a ailabili y. Injec ed packe s a e mo ed o a e sal queues and emain he e un il hei co esponding packe s each hei des ina ion. A ejec ion ime, packe s a e ans e ed o ejec ion queues. The simula ion model collec s se e al s a is ical in o ma ion du ing he simula ion and epo s he pe o mance esul s a he end o he simula ion. The P oSMART baseline simula ion model can be e o - lessly adop ed wi h any exis ing a ic gene a o model such as Ne ace o SynFull lib a ies. The in eg a ion is s aigh o wa d, and i only equi es he mapping o injec ion/ejec ion unc ions om he a ic gene a o lib a y o he P oSMART so wa e- based injec ion/ejec ion queues. Fo he ollowing analysis and esul s, he e sion wi h Ve ila o is used, because i allows he execu ion o mul iple simula ions in pa allel wi hou license es ic ions. JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 3 HPC_Max=1 HPC_Max=2 HPC_Max=3 HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 0 2 4 6 8 10 12 14 16 18 20 A g. Fli La ency (Cycles) (a) A e age Fli La ency HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 0 100000 200000 300000 400000 500000 600000 Numbe o Packe s Injec ed (b) Packe Injec ion Fig. 2. Va iabili y obse ed in he simula ion esul s o 10 million cycles (20 mac o-phase i e a ions) using he o iginal SynFull unning he Ba nes model. B. Analysis o SynFull Limi a ions This subsec ion p esen s some limi a ions and challenges ha we e encoun e ed du ing he in eg a ion o SynFull wi h P oSMART. Fi s , i highligh s ha , e en wi h a single ha d- coded seed, mino di e ences in he NoC model cause la ge di e ences in obse ed pe o mance. Nex , i analyzes he a i- abili y, con i ms ha i comes om di e ences in he mac o- phases execu ed, and obse es ha gene a ed a ic may no be comple ely ep esen a i e o he o iginal applica ion. Finally, i discusses simula ion ime. 1) Single seed and la ge a iabili y: SynFull makes use o a pseudo andom numbe gene a o , which elies on a ha d- coded seed alue. Ha ing a ixed seed always gene a es he same sequence o alues o consecu i e andom gene a ions. Howe e , we obse e ha in SynFull his does no gua an ee he same sequence o mac o-phases and, hus, an equal o e y simila a ic pa e n o each model. An analysis o he implemen a ion o SynFull e eals ha andom calls a e employed o mac o- and mic o-phase an- si ions as well as packe gene a ion. Packe gene a ion occu s bo h a he s a o each mac o-phase and in esponse o packe ecep ion. The use o di e en NoC models modi ies he delay o each packe , and in u n he e olu ion o he pseudo andom sequence and he ansi ion be ween mac o-phases. Figu e 2 illus a es he impac o his issue wi h h ee execu ions o Ba nes wi h di e en alues o HP CMax in P oSMART. I shows packe injec ion and a e age li la ency. La ge HP CMax alues allow o highe mul ihop leng h and a e expec ed o educe la ency, which may inc ease a ic. Howe e , esul s in Figu e 2b show ha load is much highe wi h HPCMax = 2 han HPCMax = 1 o HPCMax = 3. Simila ly, Figu e 2a shows ha appa en ly HPCMax = 2 ob ains he bes li la ency. The compa ison is e oneous, because he a ic injec ed in each case di e s signi ican ly. In conclusion, he a ic p esen s a la ge a iabili y in bo h he a ic load and a e age la ency be ween di e en simu- la ions, and he use o a ixed seed p e en s om obse ing di e en ou comes, esul ing in e oneous conclusions. 2) Va iabili y and applica ion ep esen a i eness: We ex- plo e a modi ied e sion o SynFull ha accep s a seed alue as a pa ame e . Figu e 3 shows he amoun o a ic in se e al simula ions o FFT and Ba nes using 10 di e en seeds. Each simula ion uns o 10 million cycles (20 mac o-phases o 43233165 12132064 36361438 91167016 96285072 97865544 13917556 18219912 79952726 49250622 Seeds 0 2000000 4000000 6000000 8000000 Numbe o Packe s Injec ed HPC_Max=1 HPC_Max=2 HPC_Max=3 (a) FFT model 43233165 12132064 36361438 91167016 96285072 97865544 13917556 18219912 79952726 49250622 Seeds 0 400000 800000 1200000 1600000 Numbe o Packe s Injec ed HPC_Max=1 HPC_Max=2 HPC_Max=3 (b) Ba nes model Fig. 3. Numbe o packe s injec ed in 10 million cycles using he o iginal SynFull and wo di e en applica ion models, o di e en seed alues. 500,000 cycles). The la ge a iabili y obse ed wi hin a single seed also occu s o di e en seed alues. SynFull models comp ise di e en mac o-phases, each one wi h di e en mic o-phase pa e ns o a ic injec ion and load. The numbe o mac o-phases depends on he applica ion model; FFT and Ba nes con ain 5 and 3 di e en mac o-phases espec i ely. Figu e 4 dissec s he amoun o a ic injec ed in each mac o-phase in bo h applica ions. Wi hin a mac o-phase he amoun o a ic is almos cons an , and ba ely depends on he mul ihop dis ance used in he ou e model. E o ba s in hese igu es, which ep esen one s anda d de ia ion, a e ba ely isible. Simila ly, he a e age la ency pe mac o-phase p esen ed in Figu e 5 is also qui e cons an . These analyses con i m ha he a iabili y in he applica ion a ic load obse ed in Figu es 2 and 3 does no come om mic o-phase a iabili y, bu i depends on he speci ic mac o- phases simula ed in each case. An analysis o he simula ions con i ms ha he execu ions in Figu e 3 wi h a highe injec ed load also simula e a highe numbe o mac o-phases wi h high load (phase 5 in FFT and phase 2 in Ba nes), and ice e sa. Figu e 6 shows a e age li la ency esul s o h ee appli- ca ions unning o 10 million cycles, a e aging esul s om 10 di e en seeds. The s anda d de ia ion is signi ican , so he esul is no eliable. The high s anda d de ia ion is caused by he high a iabili y o SynFull’s packe injec ion p ocess. To gua an ee ha a simula ion co esponds o he s eady s a e si ua ion, his is, i p ope ly ep esen s he applica ion om which he model was cap u ed, he obse ed pe cen age o occu ence o each mac o-phase should be simila o he s eady-s a e p obabili y o occu ence o such mac o-phase. Howe e , his clea ly does no occu in he di e en execu ions shown in Figu e 3. Indeed, in some cases we obse ed ha a gi en mac o-phase (wi h low p obabili y o occu ence) is JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 4 Mphase_1 Mphase_2 Mphase_3 Mphase_4 Mphase_5 Mac o-phases 0.0 0.2 0.4 0.6 0.8 1.0 1.2 A g. Packe s Injec ed 1e7 HPC_Max=1 HPC_Max=2 HPC_Max=3 (a) FFT Mphase_1 Mphase_2 Mphase_3 Mac o-phases 0 400000 800000 1200000 1600000 2000000 2400000 2800000 3200000 A g. Packe s Injec ed HPC_Max=1 HPC_Max=2 HPC_Max=3 (b) Ba nes Fig. 4. Pe mac o-phase numbe o injec ed packe s, o 10 million cycles (20 mac o-phase i e a ions), a e aging esul s om 10 di e en seeds. Mphase_1 Mphase_2 Mphase_3 Mphase_4 Mphase_5 Mac o-phases 8 10 12 14 16 18 20 22 A g. Fli La ency (Cycles) HPC_Max=1 HPC_Max=2 HPC_Max=3 (a) FFT Mphase_1 Mphase_2 Mphase_3 Mac o-phases 10 12 14 16 18 20 A g. Fli La ency (Cycles) HPC_Max=1 HPC_Max=2 HPC_Max=3 (b) Ba nes Fig. 5. Pe mac o-phase a e age li la ency, unning o 10 million cycles (20 mac o-phase i e a ions), a e aging esul s om 10 di e en seeds. ne e execu ed in ce ain simula ions. Longe simula ions wi h a highe numbe o a e aged simula ions may be used o inc ease he a ic ideli y. Figu e 7 shows he a e age numbe o injec ed packe s o FFT and Ba nes as he numbe o cycles inc eases, a e aging he esul o 10 di e en seeds. As expec ed, he a e age numbe o packe s g ows p opo ionally o he simula ion leng h. How- e e , he obse ed s anda d de ia ion emains e y la ge, e en a e aging simula ions o 400 million cycles. Se e al conclusions can be ob ained om he p e ious anal- ba nes body ack Applica ion Models 11 13 15 17 19 21 A g. Fli La ency (Cycles) HPC_Max=1 HPC_Max=2 HPC_Max=3 Fig. 6. La ency Resul s o 10 million o cycles in P oSMART+SynFull, a e aging 10 di e en seeds. HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 0 1 2 3 4 5 6 7 8 A g. Packe s Injec ed 1e7 5M 10M 20M 50M 100M 200M 400M (a) FFT HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 0.0 0.2 0.4 0.6 0.8 1.0 1.2 A g. Packe s Injec ed 1e7 5M 10M 20M 50M 100M 200M 400M (b) Ba nes Fig. 7. Numbe o injec ed packe s o di e en leng hs o a SynFull simula ion, om 5 o 400 million cycles, and esul ing a iabili y. ysis. Fi s , a single simula ion may no ep esen accu a ely he beha iou o he applica ion om which he SynFull model was ob ained, because he mac o-phases a e no execu ed p o- po ionally o hei expec ed p obabili y o occu ence. Indeed, some mac o-phases may no be execu ed a all. Second, in o de o accu a ely model he applica ion, SynFull equi es a e aging mul iple simula ions wi h long simula ion imes. 3) Simula ion ime: RTL designs a e mo e complex and de ailed han he unc ional models in so wa e simula o s such as BookSim. Fo his eason, RTL simula ions consume mo e esou ces and a e slowe han high-le el so wa e simula ions. The p e ious analysis sugges s ha a la ge numbe o simula ed cycles a e equi ed o con e gence, such as 400 million. Running in a ypical HPC clus e , we measu ed an a e age 5.4 hou s o simula ion using SynFull wi h Booksim. Howe e , using SynFull in eg a ed wi h an RTL design like P oSMART, we measu ed a e age simula ion imes o 39 JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 5 hou s, 7.2×slowe . Besides he inc eased amoun o com- pu a ion esou ces, due o hei slow execu ion speed, RTL simula ions may exceed he maximum allowed job un ime in some o all he clus e queues, o cing he use o longe queues wi h lowe p io i y and ypically mo e conges ion. All hese e ec s educe he bene i o SynFull as a mechanism o as e alua ion o RTL NoC designs wi h ealis ic a ic. IV. SYNFULL-RTL METHODOLOGY This sec ion in oduces SynFull-RTL, which sol es he limi a ions o SynFull in RTL models iden i ied in Sec ion III. SynFull-RTL employs he o iginal SynFull models, bu exe- cu es each mac o-phase in isola ion and a e ages he esul s acco ding o he s eady-s a e p obabili ies in he model. Ou p oposal simula es each mac o-phase indi idually, a oiding a la ge a ia ion in packe injec ion in each simu- la ion. Figu es 4 and 5 show ha he s anda d de ia ion o a e age packe injec ion and a e age li la ency pe mac o- phase can be educed o negligible alues. SynFull-RTL employs wo pa ame e s: Fi s , N ep esen s he numbe o di e en simula ions (di e en seeds) ha a e a e aged pe mac o-phase. N= 10 is used in Figu es 4 o 7. Second, L ep esen s he leng h o each simula ed mac o-phase in each simula ion, in i e a ions. Since each i e a ion co esponds o a ixed alue o 500,000 cycles, Lcan be also indica ed in cycles. Finally, he numbe o dis inc mac o-phases M depends on he applica ion model. The M×Nsimula ions can un in pa allel, educing he ime- o-solu ion. Fo each simula ion i(1 ≤i≤N)co esponding o mac o-phase m(1 ≤m≤M)we ob ain he a e age li la ency Fli La im, he a e age packe la ency Pk La im, and he numbe o sen packe s NumPk im and sen li s NumFli sim. The applica ion model p o ides he p obabili y o each mac o-phase, Pm. F om hese alues, we de i e he a e age and he s anda d de ia ion o he packe and li la ency in s eady s a e. The ollowing discussion explains how o calcula e packe la ency, and he calcula ion o li la ency is analogous. Fo each mac o-phase m, he a e age and he s anda d de ia ion o he packe la ency (A gPk La m and Sde Pk La m) and a e age numbe o packe s (A gNumPk m) a e calcula ed om he esul s o he N simula ions. To ob ain he o e all esul s, we canno simply a e age he esul s om all he Mmac o-phases, because hey di e in hei p obabili y Pmand hei load in ensi y. Ins ead, he alue mus be calcula ed using a weigh ed a e age, in which each weigh wm ep esen s he o e all pe cen age o packe s ha a e injec ed in mac o-phases o ype m. A no malized o e all numbe o packe s is de ined by PM m=1(A gNumPk m×P(m)), which conside s he con- ibu ion o all he mac o-phase ypes. The e o e, he weigh o be used o each mac o-phase is de ined in Equa ion 1: wm=A gNumPk m×P(m) PM m=1(A gNumPk m×P(m)) (1) Wi h hese weigh s ha ep esen he con ibu ion o each mac o-phase ype, he a e age packe la ency can be simply calcula ed as: A gPk La = M X m=1 (wm×A gPk La m)(2) To es ima e he accu acy o he mechanism, we also cal- cula e he s anda d de ia ion o he la ency es ima ion. The s anda d de ia ion is he oo o he a iance. The a iance o a linea combina ion o andom a iables is gi en by: V a ( M X i=1 ai·Xi) = N X i=1 a2 i·V a (Xi) + 2 X 1≤i<j≤N ai·aj·Co (Xi, Xj) (3) Fo independen execu ions he co a iance Co (Xi, Xj)is null, so we can es ima e: V a Pk La = M X m=1 w2 m×Sde Pk La 2 m(4) And inally we ob ain he s anda d de ia ion as Sde Pk La =√V a Pk La . V. EVALUATION This sec ion p esen s an e alua ion o he accu acy and alidi y o he SynFull-RTL model, ollowed by a use-case e alua ing he la ency imp o emen s o he P oSMART ou e . Me hodology: The modelled ne wo k is a 4×4mesh using DOR and 1 i ual channel, wi h he P oSMART ou e [8] wi h HPCMax = 1,2,3in all cases. Unless o he wise no ed, he e alua ions in his sec ion employ SynFull-RTL wi h L= 20 i e a ions pe mac o-phase and N= 5 seeds. In o de o quan i y he inaccu acy o ou s a is ical me hodology2, we also conside a SynFull-RTL-Ideal model, which conside s L= 400 i e a ions and N= 20 seeds. Whi hese la ge pa ame e s, he cycle coun is so la ge ha a a ia ion in he numbe o seeds does no modi y he esul s in a no iceable way. E alua ions using he o iginal SynFull a e age N= 10 di e en seeds ( o ob ain simila con idence in e al), unning o he speci ied ime. Plo s include he con idence in e al o a 95% con idence le el. This in e al is p opo ional o he s anda d de ia ion depic ed in p e ious sec ions. In pa icula , i is calcula ed as CI =±Z·s √N, wi h Z= 1.960 o a 95% con idence le el, s he s anda d de ia ion and N he numbe o samples. SynFull-RTL pa ame e s: Figu e 8 analyzes he impac o he numbe o a e aged seeds Nand he numbe o i e a ions Lo each mac o-phase in SynFull-RTL. The applica ion em- ployed is Ba nes, al hough o he applica ions beha e simila ly. The a e age la ency in Figu e 8a ha dly depends on he numbe o a e aged simula ions, N. Using N≥5p o ides an e o in e al wi hin a 1%. In Figu e 8b, N= 5 simula ions wi h di e en seeds a e a e aged, bu he numbe o in e als simula ed pe mac o-phase ange om L= 1 o L= 20. The ini ializa ion impac , his is, he e o obse ed when he numbe o i e a ions is low because he ne wo k is ini ially 2The inaccu acy in oduced by he model gene a ion is impossible o a oid. JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 6 HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 13 14 15 16 17 A g. Fli La ency (Cycles) N=3 N=5 N=7 N=10 SynFull-RTL-IDEAL (a) numbe o seeds, N(L= 20 in all cases) HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 13 14 15 16 17 18 A g. Fli La ency (Cycles) L=1 L=2 L=5 L=10 L=20 SynFull-RTL-IDEAL (b) mac o-phase i e a ions, L(N= 5 in all cases) Fig. 8. Pa ame e analysis o SynFull-RTL using he Ba nes applica ion. emp y, can be app ecia ed clea ly o small alues; longe simula ions p og essi ely con e ge. In his case, a high alue o Lis mo e ele an o accu acy. Using N= 5 seeds and L= 20 p o ides a e age esul s which di e less han 0.67% o SynFull-RTL-Ideal. O iginal SynFull s SynFull-RTL: Figu e 9 compa es he es ima ion o SynFull-RTL (using N= 5 and L= 20) wi h he o iginal SynFull, wi h di e en numbe o simula ed cycles and N= 10 seeds. SynFull needs a highe numbe o seeds o ob ain a compa able con idence in e al. Fou applica ions a e selec ed: FFT, Ba nes, Blackscholes and Body ack. In he ou cases we obse e ha sho simula ions p o- ide inco ec esul s in SynFull, equi ing many hund ed o i e a ions o ge close o he esul p o ided by SynFull-RTL. The esul s wi h sho simula ions di e by up o 15.8% om he esul p o ided by SynFull-RTL-Ideal. The bias o he e o depends on he la ency o he ini ial mac o-phase 1. In FFT, Ba nes and ( o a lesse ex en ) Body ack he la ency o mac o-phase 1 is high, and sho simula ions o e es ima e he esul because simula ions always s a he e; in Blackscholes he bias is he opposi e, o an analogous eason. E en wi h 400M simula ed cycles, he con idence ma gin ob ained wi h SynFull is clea ly la ge han wi h SynFull-RTL in all cases. SynFull-RTL a e age la ency is wi hin a 0.58% o he ideal alue, whe eas SynFull alues wi h 400M cycles di e by up o 3.2%. Longe SynFull simula ions could no be un, because hey exceeded he 2-day ime limi in ou sys ems. By con as , he di e en simula ions o L= 20 i e a ions (10M cycles) in SynFull-RTL can be un in pa allel, educing ime- o-solu ion by up o 400M/10M= 40×. Wi h hese pa ame e s, SynFull- RTL simula es N·L= 5 ·20 = 100 i e a ions pe mac o- phase ( he numbe o mac o-phases anges om 2 o 10 in he a ailable models). Compa ed o he N·800 = 8000 i e a ions (in he 400M case) in SynFull, compu ing esou ces a e educed by 8× o 40×, while p o iding mo e accu a e esul s. Two aspec s o Ba nes a e ele an . Fi s , sho simula ions wi h HPCMax = 2 and HPCMax = 3 p esen excessi e la ency bu educed con idence in e al. This occu s because he i s mac o-phase has bo h high p obabili y and la ency. In he “s eady-s a e” execu ion mode (which ends simula ion p ema u ely when esul s con e ge) his can p oduce inco ec esul s, because only one mac o-phase is un. Second, he la ency esul wi h 400M cycles is o e es ima ed, compa ed o he esul om SynFull-RTL. We ound ha mac o-phase 2 has a e y low p obabili y (P2= 1.88%) bu a signi ican weigh (w2= 22.6%) because i injec s abou 9 imes mo e a ic han he wo o he s. Addi ionally, i has he lowes la ency, as seen in Figu e 5b. An analysis o ou SynFull simula ions shows ha i is unde ep esen ed, wi h less han 1% o he o e all i e a ions. We a ibu e his p oblem o i s low ansi ion p obabili y and he bias owa ds he esul om mac o-phase 1 because SynFull always s a s om i . P oSMART e alua ion using SynFull-RTL: Finally, Fig- u e 10 p esen s SynFull-RTL esul s o all he a ailable mod- els, cha ac e izing P oSMART. As expec ed, he con idence in e al is e y na ow. All he applica ions con i m ha inc easing he mul ihop leng h is p o i able. Addi ionally, he use o HPCMax = 3 does no yield as much bene i as inc easing om HPCMax = 1 o 2: In a 4×4mesh, he occu ence o mul ihops o leng h 3 (i.e. om side o side o he mesh) is less equen . O e all, a e age li la ency is educed by 11.3% and 14.8% by inc easing HPCMax o 2 o 3 espec i ely. VI. RELATED WORK In his wo k, we use SynFull as a ool o eed an RTL design. In he p ocess, we de ec ed se e al limi a ions ela ed o simula ion ime and applica ion ep esen a i eness. Simila p oblems we e de ec ed in [9], whe e SynFull is used o eed he e ogeneous sys ems wi h CPU+GPU. In hei case, he simula ion ime does no allow o explo e all he GPU phases. The e o e, esul s do no e lec i s highe a ic load. They modi y he mac o-phases o balance he ela ion be ween he wo kloads and he ansi ions be ween hem, and hey also modi y he size o he mac o-phases o include mo e mic o- phases du ing he execu ion. To gua an ee ha all mac o- phases occu , hey eco d and eplay he mac o-phase o iginal o de . In con as , ou SynFull-RTL app oach uses a sampling solu ion based on a weigh ed a e age ha depends on bo h s eady-s a e p obabili y and a ic load o each mac o-phase, wi h mino changes o he SynFull design using. This educes he need o a la ge numbe o cycles o ob ain eliable esul s. The o iginal SynFull mechanism [1] al eady conside s s eady-s a e con e gence. Thei p oposal dynamically analyzes he con e gence o he pe o mance me ics and e mina es he simula ion p ema u ely. Howe e , we ha e obse ed pa icula cases in which his al e na i ely ins ead inc eases he e o o he esul . Ou app oach elies on he s eady-s a e p obabili ies o each mac o-phase, de i ed om he ansi ion p obabili ies, and sampled simula ion. JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 7 5M 10M 20M 50M 100M 200M 400M SynFull-RTL SynFull-RTL-IDEAL HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 11 12 13 14 15 16 17 A g. Fli La ency (Cycles) (a) FFT HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 13 14 15 16 17 18 19 20 A g. Fli La ency (Cycles) (b) Ba nes HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 16 17 18 19 20 21 22 23 A g. Fli La ency (Cycles) (c) Blackscholes HPC_Max=1 HPC_Max=2 HPC_Max=3 Maximum Mul ihop Leng h 15 16 17 18 19 20 21 A g. Fli La ency (Cycles) (d) Body ack Fig. 9. SynFull (blue) and SynFull-RTL (pu ple) esul s o a e age la ency wi h a ying numbe o simula ed cycles (in millions). SynFull-RTL uns each mac o-phase o 10M cycles using N=5 seeds. Each SynFull ba a e ages esul s om N=10 seeds. swap ions cholesky wa e _spa ial wa e _nsqua ed acesim lu_cb lu_ncb adiosi y blackscholes ba nes body ack luidanima e ol end ay ace adix a e age Applica ion Models 12 16 20 24 28 A g. Fli La ency (Cycles) HPC_Max=1 HPC_Max=2 HPC_Max=3 Fig. 10. P oSMART a e age li la ency unning he comple e se o a ailable models using SynFull-RTL. O he wo ks ocus on he SynFull me hodology o cap u e he a ic bu s in eal applica ions. This is he case o Mock- ails [10], which employs a simila app oach o ec ea e he space- ime memo y access beha io in he e ogeneous sys ems based on memo y IP blocks. This is a e y di e en app oach o he SynFull-RTL me hodology. The me hodology in FNoC [11] employs FPGAs o e alua e ou e models. While he simula ion speed may be highe han ou app oach, i equi es he de elopmen o complex a ic gene a ion uni s and is limi ed by he FPGA esou ces. A s ochas ic me hodology o NoC a ic gene a ion is in oduced in [12], including a eedback loop ha p opaga es delays h ough he memo y hie a chy, absen in SynFull. The applica ion o ou sampling me hodology o his app oach could be conside ed in he u u e. VII. CONCLUSIONS The o iginal me hodology in SynFull p esen s signi ican limi a ions ha a e analyzed h oughou his pape . The gene - a ed a ic has a la ge a iabili y and i can be no comple ely ep esen a i e o he o iginal applica ion being modelled, because some mac o-phases may be unde ep esen ed o no execu ed a all in a gi en simula ion, despi e eaching he s a us o s eady s a e. To mi iga e a iabili y, long execu ion imes JOURNAL OF L A TEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 8 a e equi ed, and his is exace ba ed when in e acing wi h complex RTL models. The me hodology in oduced wi h SynFull-RTL simula es each mac o-phase in isola ion and a e ages he esul s ac- co ding o he s eady-s a e p obabili y o occu ence and he measu ed a ic. While he me hodology can be applied o adi ional unc ional-le el so wa e simula ions, i has shown o be pa icula ly ele an o ou RTL modelling app oach, in which a s aigh o wa d connec ion could exceed he ypical simula ion imes o many compu e clus e s and delay he esul s o many days. Indeed, ou me hodology no only educes he equi ed compu ing esou ces by up o 40×, bu also educes ime- o-solu ion by up o 40×. Fu he mo e, his speedup is ob ained wi h inc eased accu acy: we main ain he e o in e al wi hin a 1%, lowe han he igu e measu ed wi h he o iginal mechanism. ACKNOWLEDGMENTS We would like o hank he anonymous e iewe s o hei help ul insigh s. We would also hank Miquel Mo e ´ o o help- ul sugges ions on his wo k. This wo k has been suppo ed by he Spanish Science and Technology Commission unde con ac PID2019-105660RB-C22 and he Eu opean HiPEAC Ne wo k o Excellence. En ique Vallejo has been pa ially suppo ed by he Minis y o Uni e si ies, Subp og ama Es- a al de Mo ilidad, g an numbe PRX21/00757. This wo k also ecei ed unding om he Eu opean Union Ho izon 2020 esea ch and inno a ion p og amme unde g an ag eemen s numbe 826647 (EPI) and 946002 (MEEP). REFERENCES [1] M. Bad and N. E. Je ge , “SynFull: Syn he ic a ic models cap u - ing cache cohe en beha iou ,” in 2014 ACM/IEEE 41s In e na ional Symposium on Compu e A chi ec u e (ISCA), 2014, pp. 109–120. [2] J. Hes ness, B. G o , and S. Keckle , “Ne ace: dependency-d i en ace- based ne wo k-on-chip simula ion,” 01 2010. [3] F. Alazemi, A. AziziMaz eah, B. Bose, and L. Chen, “Rou e less ne wo k-on-chip,” in 2018 IEEE In e na ional Symposium on High Pe o mance Compu e A chi ec u e (HPCA), 2018, pp. 492–503. [4] Z. Li, J. S. Miguel, and N. E. Je ge , “The unahead ne wo k-on-chip,” in 2016 IEEE In e na ional Symposium on High Pe o mance Compu e A chi ec u e (HPCA), 2016, pp. 333–344. [5] S. Saxena, D. S. Manu , M. S. Shamim, and A. Ganguly, “A olded wi eless ne wo k-on-chip using g aphene based THz-band an ennas,” in P oceedings o he 4 h ACM In e na ional Con e ence on Nanoscale Compu ing and Communica ion, se . NanoCom ’17. New Yo k, NY, USA: Associa ion o Compu ing Machine y, 2017. [Online]. A ailable: h ps://doi.o g/10.1145/3109453.3109455 [6] N. Jindal, S. Gup a, D. P. Ra ipa i, P. R. Panda, and S. R. Sa angi, “Enhancing ne wo k-on-chip pe o mance by eusing ace bu e s,” IEEE T ansac ions on Compu e -Aided Design o In eg a ed Ci cui s and Sys ems, ol. 39, no. 4, pp. 922–935, 2020. [7] A. Ejaz, V. Papae s a hiou, and I. Sou dis, “DDRNoC: Dual da a- a e ne wo k-on-chip,” ACM T ans. A chi . Code Op im., ol. 15, no. 2, jun 2018. [Online]. A ailable: h ps://doi.o g/10.1145/3200201 [8] A. Monemi, I. P´ e ez, N. Ley a, E. Vallejo, R. Bei ide, and M. Mo e ´ o, “PlugSMART: a pluggable open-sou ce module o implemen mul ihop bypass in Ne wo ks-on-Chip,” in 2021 15 h IEEE/ACM In e na ional Symposium on Ne wo ks-on-Chip (NOCS), 2021, pp. 41–48. [9] J. Yin, O. Kayi an, M. Po emba, N. E. Je ge , and G. H. Loh, “E i- cien syn he ic a ic models o la ge, complex SoCs,” in 2016 IEEE In e na ional Symposium on High Pe o mance Compu e A chi ec u e (HPCA), 2016, pp. 297–308. [10] M. Bad , C. Delcon e, I. Edo, R. Jag ap, M. And eozzi, and N. E. Je ge , “Mock ails: Cap u ing he memo y beha iou o p op ie a y mobile a chi ec u es,” in 2020 ACM/IEEE 47 h Annual In e na ional Symposium on Compu e A chi ec u e (ISCA), 2020, pp. 460–472. [11] T. V. Chu, S. Sa o, and K. Kise, “Fas and cycle-accu a e emula ion o la ge-scale ne wo ks-on-chip using a single FPGA,” ACM T ans. Recon igu able Technol. Sys ., ol. 10, no. 4, dec 2017. [Online]. A ailable: h ps://doi.o g/10.1145/3151758 [12] Y. Wang, G. Balak ishnan, and Y. Solihin, “MeToo: S ochas ic modeling o memo y a ic iming beha io ,” in 2015 In e na ional Con e ence on Pa allel A chi ec u e and Compila ion (PACT), 2015, pp. 457–467. Neiel Ley a is a PhD. S uden a Uni e si a Poli ` ecnica de Ca alunya and Resea ch enginee a Ba celona Supe compu ing Cen e (BSC). His cu en esea ch in e es a e Ne wo ks-on-Chip (NoC) design and memo y hie a chies o many-co e sys ems. He ecei ed he M.S deg ee in Compu e Enginee ing om Compu ing Resea ch Cen e o he Na ional Poly echnic Ins i u e o Mexico. neiel.ley [email p o ec ed]. Ali eza Monemi ecei ed he M.S. and Ph.D. deg ees in elec ical enginee ing om Uni e si i Teknologi Malaysia, Malaysia, in 2011 and 2017, espec i ely. He is cu en ly a Pos doc o al esea ch associa e a Ba celona Supe compu ing Cen e (BSC). His cu en esea ch in e es is cache-cohe en Ne wo ks-on- Chip (NoC) design. [email p o ec ed]. En ique Vallejo is an associa e p o esso in he Uni e si y o Can ab ia and a isi ing esea che a FORTH. His esea ch in e es s a e in com- pu e a chi ec u e, mainly in e connec ion ne wo ks and pa allel a chi ec- u es. He holds a PhD in compu e a chi ec u e om he U. Can ab ia. en ique. [email p o ec ed].