Developing telemetry tools for Alveo Accelerator cards

id192251


DEVELOPING TELEMETRY TOOLS FOR ALVEO
ACCELERATOR CARDS
JOAN TERUEL JURADO
Thesis supe iso
XAVIERMARTORELLBOFILL(Depa men o Compu e A chi ec u e)
Thesis co-supe iso
BEHZADSALAMI(Ba celonaSupe compu ingCen e )
Deg ee
Bachelo 'sDeg eeinIn o ma icsEnginee ing(Compu e Enginee ing)
Bachelo 's hesis
Facul a d'In o mà ica de Ba celona (FIB)
Uni e si a Poli ècnica de Ca alunya (UPC) - Ba celonaTech
21/01/2025

Con en s
1. Con ex ...........................................................................................................................................10
1.1. In oduc ion.............................................................................................................................10
1.2. P oblem o sol e.....................................................................................................................11
1.3. S akeholde s............................................................................................................................11
1.4. P e ious concep s...................................................................................................................12
2. Jus i ica ion.....................................................................................................................................13
3. Scopes.............................................................................................................................................14
3.1. Objec i es...............................................................................................................................14
3.2. Requi emen s..........................................................................................................................14
3.2.1. Func ional:......................................................................................................................14
3.2.2. Non-Func ional:..............................................................................................................15
3.3. Obs acles and isks..................................................................................................................15
3.3.1. No enough in o ma ion..................................................................................................15
3.3.2. No enough ime..............................................................................................................15
3.3.3. Un easibili y....................................................................................................................15
3.3.4. Ha dwa e/so wa e no a ailable....................................................................................16
4. Me hodology and igou .................................................................................................................16
4.1. Me hodology...........................................................................................................................16
4.2. Tools........................................................................................................................................16
5. P ojec planning..............................................................................................................................18
5.1. Tasks.......................................................................................................................................18
5.1.1. P ojec managemen asks...............................................................................................19
5.1.2. De elopmen Tasks.........................................................................................................20
5.1.3. W i ing he memo y........................................................................................................22
5.2. Resou ces................................................................................................................................22
5.2.1. Human esou ces.............................................................................................................22
5.2.2. So wa e esou ces..........................................................................................................23
5.3. Risk managemen and impac .................................................................................................23
5.3.1. Al eady encoun e ed p oblems........................................................................................23
5.3.2. Po en ial isks..................................................................................................................24
5.4. Table o asks..........................................................................................................................25
6. Gan diag am.................................................................................................................................26
7. Budge .............................................................................................................................................27
7.1. Pe sonnel cos s........................................................................................................................27
7.2. Gene ic cos s...........................................................................................................................29
7.2.1. So wa e..........................................................................................................................29
7.2.2. Ha dwa e.........................................................................................................................29
7.2.3. Gene ic cos s...................................................................................................................30
7.3. Es ima ed budge .....................................................................................................................31
7.3.1. Po en ial isks cos ..........................................................................................................31
7.3.2. Final budge wi h con ingencies.....................................................................................32
7.4. Budge Con ol........................................................................................................................32
8. Miles one acking..........................................................................................................................33
8.1. Modi ica ions done.................................................................................................................33
8.2. Al e na i es explo ed..............................................................................................................35
9. Sus ainabili y epo ........................................................................................................................36
9.1. En i onmen al:........................................................................................................................36
9.2. Economic:...............................................................................................................................36
9.3. Social:.....................................................................................................................................37
10. De elopmen o he p ojec ..........................................................................................................38
De eloping eleme y ools o Al eo accele a o ca ds
11. Ou -o -Band..................................................................................................................................40
11.1. In oduc ion...........................................................................................................................40
11.2. Implemen a ion.....................................................................................................................43
11.2.1. Baseboa d managemen con olle wo k low...............................................................43
11.2.2. Baseboa d managemen con olle -less wo k low........................................................44
11.3. Issues and conclusions om Ou -o -band.............................................................................48
12. In-Band moni o ing......................................................................................................................51
12.1. In oduc ion...........................................................................................................................51
12.2. Implemen a ion.....................................................................................................................52
12.2.1. Xilinx Run-Time lib a ies.............................................................................................52
12.2.2. Base Add ess Regis e s.................................................................................................54
13. Cha ac e izing and p o iling.........................................................................................................63
13.1. Cha ac e izing.......................................................................................................................63
13.2. P o iling................................................................................................................................73
14. Conclusions..................................................................................................................................76
14.1. Sa is ied objec i es...............................................................................................................76
14.2. Technical skills......................................................................................................................77
14.3. In eg a ion o knowledge......................................................................................................78
15. Laws and egula ions....................................................................................................................79
16. Glossa y........................................................................................................................................80
APPENDIX A)...................................................................................................................................82
Repo on eleme y capabili ies on Ve sal V80............................................................................82
APPENDIX B)...................................................................................................................................83
Table: Suppo ed senso s pe Al eo ca d.......................................................................................83
APPENDIX C)...................................................................................................................................85
Table: Resou ce u iliza ion o CMS.............................................................................................85
APPENDIX D)...................................................................................................................................86
Table: Compila ion o known senso s, o se s, p esence mask, and di ision ac o ......................86
17. Re e ences....................................................................................................................................89
Figu e Index
Figu e 1: T ello webpage, example s a us o cu en p ojec ; Own compila ion..............................15
Figu e 2: T ello ca d example o ask called: 'Deli e able1: Con ex and scope o he p ojec '; Own
compila ion.........................................................................................................................................15
Figu e 3: Gan ini ial empo al diag am; Own compila ion (using Google D i e Shee s)...............24
Figu e 4: Gan e ised empo al diag am; Own compila ion (using Google D i e Shee s).............31
Figu e 5: Al eoTM U2XX, U50X, U55X, UL3524 block diag am; Modi ied Xilinx image...........38
Figu e 6: Al eo sa elli e con olle in e ed in e nal s uc u e; Own compila ion.............................39
Figu e 7: Requi emen s om hos sys em; own compila ion.............................................................40
Figu e 8: Mock e sion o OOB p og am; Own compila ion............................................................45
Figu e 9: Al eoTM U2XX, U50X, U55X, UL3524 block diag am; Modi ied Xilinx image...........49
Figu e 10: Ca d Managemen Solu ion IP in e ace image o U280 and U55C; Sou ce Xilinx.......52
Figu e 11: Final implemen a ion o he FPGA Shell wi h CMS, block design iew; own compila ion
............................................................................................................................................................54
Figu e 12: CMS ou pu exce p on h ee di e en nodes o he Meep Wo ks a ions; own compila ion
............................................................................................................................................................56
3/91
De eloping eleme y ools o Al eo accele a o ca ds
Figu e 13: Exce p o he p og am wi h 8 FPGAs on one node simul aneously on he Makino e node
wel e; own compila ion....................................................................................................................57
Figu e 14: Figu e showing a success ul connec ion wi h a XDMA FPGA; own compila ion...........60
Figu e 15: A ea and esou ce u ilisa ion o W appe shell, Laga o Hun wi h 4 co es and 2 VPUs;
Own compila ion................................................................................................................................61
Figu e 16: Fan speed o e ime in e olu ions pe minu e; Own compila ion...................................62
Figu e 17: Tempe a u e o e ime (U280); Own compila ion............................................................62
Figu e 18: To al powe o e ime (U280); Own compila ion.............................................................62
Figu e 19: Tempe a u e o e ime (U55C); Own compila ion...........................................................64
Figu e 20: To al powe o e ime (U55C); Own compila ion............................................................64
Figu e 21: To al powe o e ime using eigh U55C FPGAs; Own compila ion...............................66
Figu e 22: Tempe a u e o e ime using eigh U55C FPGAs; Own Compila ion.............................66
Figu e 23: Powe o e ime o s essed U55Cs; Own compila ion...................................................69
Figu e 24: Tempe a u e o e ime o non-s essed U55Cs; Own compila ion..................................69
Figu e 25: S anda d cell schema ic; sou ce........................................................................................71
Figu e 26: FPGA cell schema ic; sou ce............................................................................................71
Figu e 27: Image showcasing he V80 SMBus implemen a ion; sou ce Xilinx................................80
Table Index
Table 1: Risk able wi h p obabili ies and hei impac ; own compila ion.........................................22
Table 2: Desc ibing asks, hou s in es ed, dependencies, and esou ces needed; own compila ion..23
Table 3: hou ly cos pe pe sonnel ole; own compila ion.................................................................25
Table 4: Table de ining asks and pe sonnel oles; own compila ion.................................................26
Tabla 5: So wa e cos able; own compila ion..................................................................................27
Table 6: Table wi h Ha dwa e de ices ela ed expenses; own compila ion.......................................28
Table 7: Table exposing gene ic expenses; own compila ion.............................................................29
Table 8: Table de ining po en ial isks and hei cos s; own compila ion..........................................29
Table 9: Final able wi h he p ojec es ima ed cos ; own compila ion..............................................30
Table 10: Table de ining de ia ion o mulas; own compila ion.........................................................30
Table 11: Re ised able de ining asks and pe sonnel oles (incomple e); own compila ion.............32
Table 12: Re ised inal able wi h he p ojec es ima ed cos ; own compila ion...............................32
Table 13: Bene i s pe eleme y me hod; own compila ion...............................................................36
Table 14: Ou pu om command 'sudo i2cde ec -l'; Own compila ion.............................................43
Table 15: Ou pu om command 'sudo i2cde ec -y 0'; Own compila ion.........................................43
Table 16: Suppo ed I2C/SMBus commands; sou ce Xilinx..............................................................44
Table 17: Code exce p o he pla o m iden i ica ion pa ame e s; Own compila ion.......................58
Table 18: Table wi h he basic da a ou pu ; own compila ion............................................................59
Table 19: Exce p o ex ended da a using an Al eo U55C; own compila ion....................................60
Table 20: Analysis o powe and empe a u e inc emen ; Own compila ion.....................................67
Table 21: A e age absolu e alues o p e ious able; Own compila ion..........................................67
Table 22: Compila ion o s anda d de ia ion (SD) pe senso ; Own compila ion.............................68
Table 23: Powe inc ease due o s essing emula ed p ocesso s on FPGAs; own compila ion..........70
Table 24: U iliza ion o esou ces; Own compila ion.........................................................................73
Table 25: Suppo ed senso s pe Al eo ca d; Sou ce Xilinx..............................................................83
Table 26: Resou ce u iliza ion o ca d managemen solu ion; Sou ce Xilinx...................................83
Table 27: Compila ion o known egis e s, hei names, o se s, p esence mask, di ision ac o s and
loca ion; Own compila ion, exce p om cus om C lib a y, in e ed om Xilinx sou ces...............86
4/91

De eloping eleme y ools o Al eo accele a o ca ds
ABSTRACT
The p ojec would p opose solu ions and de elop ools o ge ing eleme y da a, such as
empe a u e, powe , cu en , o ol age alues, om he embedded senso s inside he
AMD/Xilinx Al eo FPGAs accele a o ca ds. We will use his da a o ack he
powe /ene gy consump ion and empe a u e o di e en FPGA componen s a un- ime,
e.g., memo y, IOs, and logics. This analysis will help o be e unde s and he pe o mance
o he applica ion unning on FPGA and hus, p opose solu ions o imp o ing he
powe /ene gy e iciency wi h lowe empe a u e impac s. We plan o de elop hese ools
on op o Makino e (i.e., BSC’s FPGA clus e ) o collec esul s on a massi e numbe o
de ices. The a ia ion ac oss di e en FPGA ca ds would also p o ide use ul in o ma ion
o be e unde s and he design placemen and op imise i . The eleme y oolse will be
in eg a ed in FPGA Shell, which is a ool o gene a e FPGA designs au oma ically by using
he Xilinx IPs o connec ing he ha dwa e design o pe iphe als.
5/91
De eloping eleme y ools o Al eo accele a o ca ds
RESUMEN
Es e p oyec o iene la in ención de p opone soluciones y desa olla he amien as pa a
consegui in o mación de eleme ía cómo la empe a u a, po encia, co ien e o ol ajes,
es os alo es se ob ienen de los senso es embebidos den o de las Al eo de acele ación
FPGA de AMD/Xilinx. Usa emos es os da os pa a el seguimien o de la ene gía/po encia
consumida y los alo es de empe a u a en iempo eal de módulos como la memo ia,
pue os de en ada/salida o secciones lógicas. Es e análisis ayuda á a comp ende el
endimien o de una aplicación den o de la FPGA y po an o, p opone mejo es
soluciones pa a inc emen a la e iciencia en é minos de consumo y empe a u a.
Planeamos desa olla es a he amien a den o de Makino e (es deci el clús e de FPGA
del BSC) pa a ecolec a los esul ados en can idad. La a iación en e di e en es a je as
FPGA ambién nos puede se ú il pa a el en endimien o del diseño in e no y op imiza lo. El
conjun o de he amien as de eleme ía se á in eg ado den o del FPGA Shell, que es una
he amien a que gene a diseños pa a FPGA au omá icamen e usando las IP de Xilinx
pa a conec a nues os diseños con sus espec i os pe i é icos.
6/91
De eloping eleme y ools o Al eo accele a o ca ds
RESUM
Aques p ojec e é la in enció de p oposa solucions y desen olupa eines pe aconsegui
in o mació elemé ica com la empe a u a, po encia, co en o ol a ges, aques s alo s
s’ob enen dels senso s encas a s dins de les FPGAs d’accele ació Al eo de AMD/Xilinx.
U ili za em aques es dades pel seguimen de la ene gia/po encia consumides y els alo s
de empe a u a en emps eal dels mòduls com les memò ies, po s d’en ada i so ida o
seccions lògiques. Aques anàlisi ens ajuda à a comp end e el endimen d’una aplicació
dins de les FPGAs i pe an , p oposa millo s solucions pe inc emen a l’e iciència en
e mes de consum i empe a u a. Planajem desen olupa aques a eina dins de Makino e
(un clús e de FPGAs del BSC) pe a ecolec a els esul a s en quan i a . La a iació en e
di e en s a ge es FPGA ambé ens po se ú il pe al en enimen del disseny in e n i
op imi za -lo. El conjun d’eines de eleme ía se à in eg a dins del FPGA Shell, que és
una eina que gene a dissenys pe a FPGAs au omà icamen u ili zan IPs de Xilinx pe
connec a -los amb els seus espec ius pe i è ics.
7/91
De eloping eleme y ools o Al eo accele a o ca ds
1. Con ex
This is a bachelo hesis o he Compu e Science deg ee a he Facul a d’In o mà ica de
Ba celona, Uni e si a Poli ècnica de Ca alunya, on he specialisa ion o Compu e
Enginee ing, di ec ed by Xa ie Ma o ell Bo ill wi h co-supe iso Behzad Salami, head o
he FPGA Technologies eam a he Ba celona Supe compu e Cen e (BSC).
The p ojec will wo k wi hin he bounda ies o he compu e science depa men , wo king
on he new compu e a chi ec u e p ojec s, such as he Ze ascale, which is one o he
mos impo an sou ces o inancing. In his p ojec , BSC wo ks o de elop a RISC-V
p ocesso en i ely designed locally; addi ionally, he e a e also di e en p ojec s o
in e na ional coope a ion wi h he Eu opean Union and p i a e en e p ises.
1.1. In oduc ion
A Field P og ammable Ga e A ay (FPGA) is a ype o de ice ha , using an a ay made o
basic elemen s o logic, is capable o emula ing he beha iou o de ined ha dwa e [1].
The FPGA ma ke was alued in 2020 a 6 billion US dolla s, and some o ecas s udies
expec i o g ow o 13.6 billion dolla s by he end o 2031 [2]. Inc easingly FPGAs a e
being used in da a cen e s and cloud compu ing, as hey can be dynamically modi ied o
compu e-in ensi e asks such as analysis o da a, machine lea ning, ne wo k p ocessing
and o he specialised compu ing-in ensi e asks.
FPGAs a e gaining space in he global ma ke , no as emula ion pla o ms o ha dwa e
design, bu as inal compu ing pla o ms. In ac , BSC is using he FPGA no o he
in ended ole o he Al eos which in hei name speci ies (accele a o asks), bu o
emula ion. Con a y o his, big companies like Amazon o Mic oso a e buying an
inc easing amoun o FPGA o hei da a cen e s o p o ide new compu a ion pla o ms o
hei cus ome s and hemsel es [3].
FPGAs a e c ucial whe e a ha dwa e p oduc is being designed. These de ices a e used
o emula ion pu poses, which help p o ide debugging da a o he design eam. Wi hou
FPGAs, he ime ha would ake o he de elopmen o such p oduc s would inc ease
signi ican ly, due o he ac ha simula ions ha a e un on classic compu e s can ake a
lo o hou s, i no days, in he case o big designs [4].
As he impo ance o he p oduc i i y o he Regis e T ans e Le el (RTL) eam is o he
u mos impo ance, de eloping ools ha ocus on he s a us o hose machines is a key
se ice ha helps o imp o e he eliabili y o hose expensi e de ices, which ansla es o
be e e iciency and managemen o his ha dwa e esou ces.
8/91
De eloping eleme y ools o Al eo accele a o ca ds
•Done: Tasks ha a e 100% inished.
•Re iew/Imp o e: Tasks ha a e almos inished bu equi e es ing, e iew om
o he ac o s o some le el o polishing.
•Disca ded: Tasks ha we e s a ed bu no longe a e ac i ely ollow h ough due o
p oblems, obs acles o mis akes.
Figu e 1: T ello webpage, example s a us o cu en p ojec ; Own compila ion
Each e ical column is e e ed o as a ‘lis ’ which con ains a se o ‘ca ds’ and each ca d
can ha e an amoun o di e en asks. In he Figu e 2 a he immedia e igh we can see
exac ly how we ha e de ined a due da e o he ask and a checklis , in his case, o w i e
he sel - epo and w i e he documen o deli e . See ha I ha e also added a ag wi h a
colou , his I use i a li le di e en om he sugges ed me hod explained a [9], a he I use
he colou s o exp ess he se e i y o he ask, ed wi h high deg ee o impo ance, g een
o a low p io i y.
15/91
Figu e 2: T ello ca d example o ask called:
'Deli e able1: Con ex and scope o he p ojec '; Own
compila ion

De eloping eleme y ools o Al eo accele a o ca ds
- Gi lab: One o he mos impo an ool in he ac i e de elopmen will be Gi lab, which is a
powe ul ool ha enables cons an de elopmen and cons an in eg a ion [10]. Apa om
he eposi o ies which sa e he p og ess, Gi lab enables new me hods o ac i ely epo
issues and ecommenda ions om he di e en ac o s.
- Zoom: To communica e be ween he di ec o and co-di ec o o his p ojec , apa om
he physical eunions we ha e and he s anda d communica ion channels such as Google
Mail o D i e, Zoom akes a big ole, he e iciency in in e changing mails canno be
compa ed o he one o people ac i ely in e ac ing wi h each o he , when he di e en
ac o s a e no able o euni e oge he i is decided ha emo e mee ings will be he second
op ion.
5. P ojec planning
This p ojec is made wi h a BSC con ac , al hough he modali y o i is ype AI. Ini ially, he
numbe o hou s dedica ed o he p ojec we e i e a day, i e days a week; his a leas
was he ini ial p oposal; he daily hou s dedica ed o he p ojec ha e been modi ied as he
p ojec p og essed. The con ac was signed on he 1 s o Ma ch 2024, and he in en ion is
o ge i done be o e he 13 h o Janua y. This p ojec o icially s a ed as a TFG when i
was insc ibed and ma icula ed in he i s week o July. Tha means some o he asks
desc ibed ha a e planned a e asks al eady comple ed, and he de ini ion o isks and
p oblems encoun e ed a e obs acles ha al eady happened o can possibly happen, so
his sec ion will look in e ospec i e a he wo k ha has al eady been done, he wo k ha
is being done ac i ely, and planned wo k o do in he u u e.
5.1. Tasks
In his chap e , he asks o he p ojec will be de ined be ween hose conside ed p ojec
managemen , de elopmen , edac ing he memo y, and inally he p ojec moni o ing. Each
ask will de ine he ime o comple e, wha he ask consis s o , and wha will be done. A
he end o he desc ip ion in Chap e 5.4. Table o asks, whe e addi ionally he
dependencies be ween asks a e de ined and he esou ces needed.
I Modali y A indica es ha he p ojec is de eloped as an uni e si y p ojec .
16/91
De eloping eleme y ools o Al eo accele a o ca ds
5.1.1. P ojec managemen asks
He e, all asks a e de ined ha a e ela ed o p ojec managemen (PM).
•PM.1: Con ex and scope o he p ojec (25h)
He e he p ojec scope, con ex , and jus i ica ion o he p ojec a e cla i ied,
iden i ying he p oblems and solu ions he p ojec has o sol e, he equi emen s o
he p ojec , he pa ies in ol ed, and he me hodology. A documen is w i en and
deli e ed.
•PM.2: P ojec planning (20h)
The planning o he p ojec is de ined by he ask decomposi ion, he schedule o
each ask wi h he Gan diag am, he esou ces needed, and p obable isks he
p ojec migh encoun e . A documen is w i en and deli e ed.
•PM.3: Economic and sus ainabili y managemen (20h)
An economic and sus ainabili y epo is made, desc ibing he o al budge o he
p ojec and i s en i onmen al consequences. A documen is w i en and deli e ed.
•PM.4: Final in eg a ion (15h)
All he p e ious documen s a e in eg a ed in o one, which will be he basis o he
memo y o he p ojec . A documen is w i en, e iewed, and deli e ed.
•MT: Mee ings o moni o ing (45h)
An amalgama ion o all he mee ings and eunions made is de ined as a ask o
accoun o all he hou s in es ed. Since he 1s o Ma ch, we de ined one eunion
o one hou a week wi h he hesis co-supe iso , and since he i s o Sep embe ,
an addi ional mee ing wi h bo h he supe iso and co-supe iso o also one hou .
The eam mee ings ake app oxima ely one hou each; hese mee ings a e aken
e e y Tuesday om ele en o’clock o wel e o’clock.
17/91
De eloping eleme y ools o Al eo accele a o ca ds
5.1.2. De elopmen Tasks
A lis o he asks ha a e ela ed o he de elopmen o he p ojec is de ined. The p ojec
de elopmen asks will be subdi ided in o hose ela ed o Ou -o -band, In-Band and
w i ing asks. The o de o he lis does no speci y he o de in which he asks a e
comple ed.
•RT.1: Gene al esea ch on eleme y (25h)
A gene al esea ch on Xilinx’s documen s is pe o med on how o ge eleme y da a
ou o he Al eo ca ds. This in oduc o y ask will lead o he subsequen decisions
on he de elopmen o he p ojec .
•RT.2: In es iga ing XRT me hodology (25h)
The XRT lib a ies and APIs a e in es iga ed o disco e whe he hose ools a e
use ul o no . Also, a small p og am in bash sc ip s is de eloped.
- Ou o Band de elopmen g oup o asks:
•OOB.1: Resea ch on OOB me hodology (25h)
A speci ic esea ch on OOB me hodology is done, in es iga ing exac ly he in e nal
s uc u e o he Al eo Accele a o ca ds and how o communica e wi h ha channel.
•OOB.2: Ini ial de elopmen o OOB ool (25h)
He e a p og am ha uses he OOB in e ace is de eloped, his ini ial ask consis s
o ins alling necessa y ools and lib a ies, con igu ing he FPGAs, and an ini ial
Py hon p og am.
•OOB.3: Addi ional esea ch o sol e p oblems (25h)
In e ospec i e, we encoun e ed di e en p oblems ha made us s op de eloping
he OOB ool and o ced us o sea ch o possible solu ions.
•OOB.4: Baseboa d Manage Con olle De elopmen (25h)
Includes he esea ch o wha is a BMC, he es ing o BMC inside ou ha dwa e,
and inding solu ions o aul y BMC beha iou .
18/91
De eloping eleme y ools o Al eo accele a o ca ds
•OOB.5: Ques ion o Xilinx and conclusions (15h)
Ha ing obse ed a mul i ude o p oblems almos e e ywhe e when de eloping OOB
ela ed eleme y, we es ablished a channel wi h Xilinx enginee s h ough emails
and Xilinx’s o um whe e we asked o solu ions o he p oblems encoun e ed, upon
Xilinx answe ing, we came o some conclusions and made a change in he cou se
o he de elopmen o OOB.
- In-Band de elopmen :
•IB.1: Resea ch on IB me hodology (25h)
Resea ching IB me hodology wi h Xilinx documen a ion.
•IB.2: De elopmen o a simple Vi ado p ojec wi h CMS (25h)
Wi h Vi ado IDE, de elop a simple design using a ious ypes o FPGAs, which
enable CMS.
•IB.3: In eg a ion o CMS in o W appe -Shell (25h)
Include and in eg a e he CMS in o W appe -Shell, addi ionally, check ha his new
ea u e does no b eak he p e ious in as uc u e and CMS can s ill be add essed.
•IB.4: De olpmen o CMS Py hon P og am (25h)
De elopmen o a CMS p og am ha uses he p e ious designs loaded in o he
FPGAs and eads he da a and w i es i in o CSV o ma o la e analysis. The
p og am is made in Py hon bu also calls a C execu able.
•IB.5: Ea ly ecopila ion o eleme y da a (10h)
F om Al eo U280 and U55c models, ex ac da a o la e analysis.
•IB.6: In eg a ion o CMS in o Embedded-Shell (25h)
Expand he CMS o be included inside Embedded-Shell, making su e he addi ion
o CMS does no b ake he s anda d pipeline, and wo ks o U200/U250, U280, and
U55c FPGAs.
•IB.7: Imp o emen o eleme y p og am (15h)
Imp o e he eleme y in-band p og am so ha i includes suppo o Makino e
deploymen and mul iple FPGAs can be connec ed o one ins ance o he p og am
a once.
19/91
De eloping eleme y ools o Al eo accele a o ca ds
•IB.8: De elopmen o D x and p o iling o Cinco anch (50h)
De elop Dynamic Func ion eXchange o embedded-shell so we can collec da a
indi idually om each Cinco anch co e.
•IB.9: Ex ended ecopila ion o eleme y da a (20h)
F om p e ious addi ions, ex end he amoun o da a aken wi h he addi ion o
Cinco anch suppo and DFx o be e p o iling.
5.1.3. W i ing he memo y
These de ini ions o asks co espond wi h he w i ing o a documen called memo y and
he de ence p epa a ion be o e he academic ibunal.
•WM.1: Concu en W i ing (50h)
This ask de ines he ime used when w i ing he memo y while de eloping he
p ojec .
•WM.2: Final W i ing o he Memo y (40h)
The asks co espond o he inal syn hesis o he memo y o deli e once all asks
and ecompila ion o da a a e done.
•WM.3: P epa ing he de ence o he hesis (20h)
C ea ing a Powe Poin p esen a ion, possible ques ions and hose answe s, and a
sc ip ha makes up o he de ence o he hesis.
5.2. Resou ces
5.2.1. Human esou ces
- [AP] Au ho o he p ojec : s uden who de elops he p ojec .
- [TS] Thesis supe iso : UPC u o , who moni o s he p ojec .
- [TC] Thesis co-supe iso : BSC u o , who moni o s he p ojec .
Addi ional human esou ces ha a e mo e gene alis ic, like:
- [SA] BSC Se e s adminis a o s: adminis a o s who help us ins all equi ed ools.
- [XP] Xilinx pe sonnel: Xilinx enginee s, which answe ou pe i ions.
20/91

De eloping eleme y ools o Al eo accele a o ca ds
- [CW] BSC cowo ke s: When we equi e addi ional help om BSC companions.
5.2.2. So wa e esou ces
[GD] Google D i e: de ined a conglome a e o di e en ools like Powe Poin , D aw,
Shee s, and mo e.
[LO] Lib eo ice: Lib eO ice IDE is used o w i ing he o icial documen s o be deli e ed.
[TR] T ello: T ello is he ool ha is used wi h Kanban me hodology and moni o ing.
[VD] Vi ado: Vi ado is an IDE ool de eloped by Xilinx in which we c ea e designs and use
o loading hem in o he FPGAs.
[VS] Visual S udio Code: Will be he main IDE ool o use when de eloping Py hon and C
p og ams o be used on emo e clus e s.
[ZM] Zoom: This ool is used o emo e mee ings.
5.3. Risk managemen and impac
Du ing he cou se o he p ojec he e will be obs acles and p oblems ha would impac he
de elopmen ime, o his eason he planning is done wi h a ma gin o ime so ha in he
wo s -case scena io he schedule would s ill be me .
In ac , some o he isks below we e al eady ound when de eloping he p ojec , so I will
di ide hem be ween he al eady-exis ing and po en ial new ones:
5.3.1. Al eady encoun e ed p oblems
1. No enough in o ma ion: When de eloping a eleme y ool, we encoun e ed a se o
documen s ha explained di e en ways o accessing moni o ing da a, bu he p oblem is
ha hey a e no p ecise enough o he e a e missing cla i ica ions. Fo example, he
Vi ado IDE, when showing all a ailable FPGAs o load he bi s eam, le s you also iew a
cha wi h he empe a u e o he ca d, bu he o icial documen a ion does no cla i y how
o access h ough ha same me hod he da a. Addi ionally, some in o ma ion misses
s eps, o e en wo se, he documen a ion is w ong. Expe imen ally, we ha e p o en ha
some o he s a emen s o he OOB me hodology a e no ue. The impac on hou s can be
s ablished as 25, as we had o con ac mul iple imes wi h Xilinx o ge answe s and also o
s a looking o in o ma ion om uno icial si es.
21/91
De eloping eleme y ools o Al eo accele a o ca ds
2. Un easibili y: Ini ially in his p ojec we we en’ sea ching o an in-band solu ion,
because om bo h (OOB and IB) he bes is wi hou a doub OOB, bu o he OOB ask,
we ha e seen ha he documen a ion is w ong in some key speci ica ions, wi hou en e ing
he speci ic de ails, he w ong s a emen s make OOB o ally un easible, hus OOB was
abandoned and he in eg a ion o IB began. The consequences o un easibili y we e a
eaching, OOB me hod was abandoned and he esea ch and de elopmen o a ool had
o be es a ed, we ha e s ablished ha om his p oblem, he impac on addi ional hou s
is he o al amoun o ime dedica ed o OOB minus he esea ch ime, which is 90 hou s.
5.3.2. Po en ial isks
1. No enough ime: As he p ojec s a ed a long ime ago, he schedule planned o he
missing asks and ea u es al eady has a ma gin o possible addi ional p oblems, which
we belie e is a low isk. Bu , we conside an addi ional 10 hou s o possible de ia ions.
2. Ha dwa e/so wa e no a ailable: The p ojec is de eloped on BSC in as uc u e, and
some imes i is needed some main enance and mee ings wi h he ope a ions eam, he
eam ha o e sees he Makino e clus e , and pe i ions ha e o be made h ough hem,
which akes ime like ins alling new p og ams, downloading lib a ies, o accessing he
ha dwa e physically. The isk is conside ed 'medium’ and he impac on hou s in a wo s -
case scena io is 25 (because mee ings wi h ope a ions a e made each F iday).
Risk P obabili y/Se e i y Impac
No enough in o ma ion Medium 25 hou s
Un easibili y High 90 hou s
No enough ime Low 10 hou s
Ha dwa e/so wa e no a ailable Medium 25 hou s
Table 1: Risk able wi h p obabili ies and hei impac ; own compila ion
22/91
De eloping eleme y ools o Al eo accele a o ca ds
5.4. Table o asks
Table 2: Desc ibing asks, hou s in es ed, dependencies, and esou ces needed; own compila ion
Code Task Du a
ion
Depend
encies
Resou ces (human
and echnological)
P ojec Managemen (80 hou s)
PM.1 Con ex and scope o he p ojec 25 h --- LO, TR, AP
PM.2 P ojec Planning 20 h PM.1 LO, GD, TR, AP
PM.3 Economic and sus ainabili y managemen 20 h PM.2 LO, GD, AP
PM.4 Final In eg a ion 15 h PM.3 LO, AP
MT Mee ings o moni o ing 45 h --- ZM, TR, AP, TS, TC
P ojec De elopmen (375 hou s)
RT.1 Gene al esea ch on eleme y 25 h --- AP
RT.2 In es iga ing XRT me hodology 25 h RT.1 AP
OOB.1 Resea ch on OOB me hodology 25 h RT.1 AP
OOB.2 Ini ial de elopmen o OOB ool 25 h OOB.1 VS, VD, AP
OOB.3 Addi ional esea ch o sol e p oblems 25 h OOB.2 AP, CW
OOB.4 Baseboa d Mange Con olle de elopmen 25 h OOB.3 AP, SA
OOB.5 Ques ion o Xilinx and conclusions 15 h OOB.4 AP, XP, TC
IB.1 Resea ch on IB me hodology 25 h RT.1 AP
IB.2 De elopmen o simple Vi ado p ojec wi h
CMS
25 h IB.1 VS, AP, VD
IB.3 In eg a ion o CMS in o W appe -Shell 25 h IB.2 VS, VD, AP, CW
IB.4 De olpmen o CMS py hon p og am 25 h IB.3 VS, VD, AP
IB.5 Ea ly ecopila ion o eleme y da a 10 h IB.4 VS, AP
IB.6 In eg a ion o CMS in o Embedded-Shell 25 h IB.5 VS, VD, AP
IB.7 Imp o emen o eleme y p og am 15 h IB.5 VS, VD, AP
IB.8 De elopmen o D x and p o iling o
Cinco anch
50 h IB.3 VS, VD, AP, CW
IB.9 Ex ended ecopila ion o eleme y da a 20 h IB.8 VS, AP
W i ing he memo y (110 hou s)
WM.1 Concu en W i ing 40 h OOB.4
o IB.2
LO, GD, AP
WM.2 Final W i ing o he memo y 50 h WM.1 LO, GD, AP
23/91
De eloping eleme y ools o Al eo accele a o ca ds
WM.3 P epa ing he de ense o he hesis 20 h WM.1 GD, AP
The o al amoun o ime dedica ed o he p ojec is a ound 550 hou s (wi hou isks),
ha ing in mind ha o icially he TFG accoun s o 18 c edi s and each akes 30 hou s, o
a o al o 540 hou s, we su pass he expec ed amoun o ime in es ed in he p ojec .
6. Gan diag am
In he ollowing Figu e 3, we can see he Gan empo al diag am. Each week is de ined as
i e wo k days, o example, ‘W01’ only is con abilized as he i s o Ma ch, and ‘W10’ has
wo Ap il days and h ee May days, so is accoun ed as a May week; as ime wen by, he
hou s dedica ed o he p ojec we e modi ied, and so he numbe o weeks needed o
comple e a ask inc eased bu no he o al hou s o such a ask. The igu e below was
c ea ed on he 29 h o Sep embe , which co esponds wi h W31.
The me hodology used is Agile wi h a cus om Kanban; some modi ica ions ha e been
made, so, o example, e en hough we de ine each sp in as a wo-week ask, each week
we do he agile e iew, and as men ioned be o e, on la e s ages o he p ojec , he
quan i y o sp in s needed o a ask inc eased as he daily hou s dedica ed dec eased.
Indi idually o each sp in , i is no speci ied ha he planning is an indi idual ask; a he ,
i is implici on [MT] mee ings o moni o ing; when e iewing he p og ess made, a e-
planning is done i needed.
This p ojec elapsed almos 10 mon hs, he en i e mon h o Augus he e was no p og ess
due o summe aca ions. Addi ionally, we can see ha he moni o ing o he p ojec
s opped om W10 o W15 because we encoun e ed se e al p oblems o de eloping
OOB, and i was decided ha we would s op un il ge ing answe s om Xilinx; in he
mean ime, he au ho would wo k on asks ha a e no ela ed o his p ojec .
24/91
Figu e 3: Gan ini ial empo al diag am; Own compila ion (using Google D i e Shee s)
De eloping eleme y ools o Al eo accele a o ca ds
8. Miles one acking
8.1. Modi ica ions done
The miles one acking is done some weeks be o e he p esen a ion o he echnical
memo y, which is gi en be o e he p esen a ion o he bachelo ’s hesis. In his case, his
e iew has been done on 3 d o Decembe 2024. In his sec ion, only he changes made
a e indica ed, like a changelog.
Con ex ualiza ion: A new legal no e has been added in chap e ‘1.1. In oduc ion’ whe e
i is speci ied he almos nonexis en legal cons ain s o he p ojec . Unde ‘3.3. Obs acles
and isks’ a new isk has been added, ‘3.3.4. Ha dwa e/so wa e no a ailable’ because
he e we e only h ee isks desc ibed, bu in chap e ‘5.3. Risk managemen and impac ’,
ou appea ed.
Tempo al planning: A es uc u e has been made o he p e ious ‘6. Gan diag am’,
whe e he plani ica ion has been modi ied o i eali y; asks IB.8 and IB.9 ha e been
e ised o he le o he diag am. The ime o comple e Task IB.8 has been doubled, o i y
hou s. Tempo ally, he a io o ime dedica ed be ween Oc obe and No embe o asks
ela ed o BSC has inc eased, which means less ime is in es ed pe week in o his
p ojec .
Budge : The budge has been modi ied due o he ask IB.8 aking longe han usual; his
e ision cos s an addi ional 665.25€ o he p ojec . Following he p o ocol men ioned in
‘7.4. Budge Con ol’ we ha e o calcula e he de ia ion; in his case, he de ia ion is he
inc ease o cos s. The pe cen age o he con ingency money used o pay o hese cos s is
(-665.25) ÷ 2947.54 = -22.53%; as i is unde he -50% limi , i is accep able. The ollowing
able 11 depic s only he changes made on he espec i e ows. I has been added a no e
on ‘Budge Con ol’ ha o calcula e he de ia ion, i is needed o di ide by he
con ingency.
31/91
Figu e 4: Gan e ised empo al diag am; Own compila ion (using Google D i e Shee s)

De eloping eleme y ools o Al eo accele a o ca ds
Code Task Du a
ion
Hou s pe Role Cos (€)
SM SD JR DA
P ojec Managemen (125 hou s)
...
P ojec De elopmen (385 hou s)
...
IB.8 De elopmen o D x and p o iling o
Cinco anch
50 h 50 1330,50€
...
W i ing he memo y (110 hou s)
...
To al: 18537,49€
Table 11: Re ised able de ining asks and pe sonnel oles (incomple e); own compila ion
The inal budge is upda ed as ollows :
Sou ce Cos (€)
Cos s ela ed o pe sonnel 18537,49€
So wa e 73,70€
Ha dwa e 112,51€
Gene ic cos s 1591,83€
To al be o e con ingency: 20315,53€
Con ingency (15%) 3047,33€
Po en ial isks 359,24€
To al: 23722,10€
Table 12: Re ised inal able wi h he p ojec es ima ed cos ; own compila ion
Conclusions: A new chap e called ‘Conclusions’ has been w i en wi h subsec ions. Bea
in mind he ollowing sec ion canno be o ally comple ed un il he inal deli e y.
14.1. Sa is ied objec i es: This chap e indica es upon he ini ial scopes sugges ed, i
hose ha e been success ully comple ed.
14.2. Technical skills : The echnical skills associa ed wi h he p ojec and i s
achie emen s.
14.3. In eg a ion o knowledge: Which UPC subjec s ha e been bene icial o his p ojec
and he eason why.
32/91
De eloping eleme y ools o Al eo accele a o ca ds
O he mino changes such as g amma ical e o s, ex ending some pa ag aphs o be e
explain he si ua ion a e no indica ed he e.
8.2. Al e na i es explo ed
The explo a ion o al e na i es has been concu en wi h he de elopmen o he p ojec .
The al e na i es we ha e explo ed a e ocused on he educ ion o powe consump ion.
Which ini ially was no inside he scopes o he p ojec bu was added by my own ini ia i e,
no as a p oduc , a he as a heo ised exploi a ion o he p og am we ha e been
de eloping. These explo a ions ha e no been accoun ed o in he Gan diag am, no in
he ask decomposi ion o he p ojec , because i is cen ed a ound he expe imen al usage
o he FPGA and some manipula ions o cu en designs.
The main al e na i e was o de elop ye ano he shell, called ‘li e shell,’ which could be
able o gi e eleme y da a o he hos while using he leas amoun o powe consump ion,
because, unlike o he shells, eleme y does no need memo y con olle s, he ull AXI
in e ace, o addi ional pe iphe als like UART, JTAG, o o he s. The main p oblem wi h his
idea is ha i will likely ne e ge implemen ed (a leas no in BSC’s clus e s), because i
elies on he de elopmen o a ‘daemon,’ a backg ound ask wi h p i ileges, which equi es
ex ensi e and ca e ul p og amming; o he wise i could become a cybe -a acke ec o .
The main idea is ha , upon a use lea ing he se e , his ‘li e shell’ would ge loaded in o
he FPGA, p o iding eleme y. Ano he issue is ha a use ha wan s a p og am o keep
unning a e hei depa u e om he se e would ge hei FPGA disabled.
Al e na i es o p o ide eleme y da a a e bad; ou -o -band would ne e be able o wo k on
ou cu en se e s. Sysmon da a h ough Xilinx’s Vi ado GUI is no a solu ion ei he ,
p ima ily because he e is no enough da a o know i s inne wo kings. XRT d i e s would
no be able o bind o ou designs, and e en i hey did, o he unc ionali ies like p o iling
wouldn’ wo k. Since ely, I hink we chose he bes decision among he bad ones, e en
hough i has seemed o be somewha imp ac ical.
33/91
De eloping eleme y ools o Al eo accele a o ca ds
9. Sus ainabili y epo
Inc easingly, companies all o e he wo ld include in hei p ojec s a sus ainabili y epo ,
which measu es he impac o hei p oduc s and se ices no only in ecological bu also
economic and social ma e s oo.
9.1. En i onmen al:
In chap e 7.2.3. Gene ic cos s I es ima ed he consump ion o elec ici y as I do no ha e
any ca , he CO2 emissions ela ed o his p ojec a e app oxima ely o 273 gCO2 / kWh[17]
pe 1,1665kW imes 550 hou s; esul ing in 175,17 Kg o CO2.
P ojec pu in o p oduc ion:
I is di icul o accu a ely es ima e he eal impac he de elopmen o he p ojec will ha e.
Compa ed o he mac o-economic scope, a lone s uden wo king wi h a pai o sc eens
and a lap op is negligible. Bu he comple ion o he p ojec could lead o bene i s, such as
he con ol o cooling sys ems o imp o e hei e iciency and so hei elec ical
consump ion.
Use ul li e:
Cu en ly, he e is no solu ion ha sui s he BSC as well as mine. Yes, Vi ado o e s a he
XRT ools bu only wo ks on hei in as uc u e, which we cu en ly do no suppo , so he
answe is ha is he e is no solu ion igh now ha sui us. This ool gi es he designe he
abili y o use wo new me ics, consump ion and empe a u e, wi h which o choose he
designs ha ha e he bes alues.
Risks:
We don’ o esee any ype o isks ela ed wi h he en i onmen due o his p ojec , in ac i
he e a e any i would p o ide ools o dec emen hem such as FPGA op imiza ion.
9.2. Economic:
P ojec pu in o p oduc ion:
I ha e es ima ed he cos s ela ed o human esou ces, ha dwa e and so wa e usage,
gene ic cos s such as in e ne , elec ical consump ion, o wa e , and also aking in o
accoun possible isks and con ingencies o he p ojec . The con ingency is a ound 15%
o he ini ial budge , and al hough he e ha e been ce ain isks, hese ha e only
consumed a qua e o he con ingency, his e u ns a posi i e di e ence o €2282.29.
34/91
De eloping eleme y ools o Al eo accele a o ca ds
Use ul li e:
This p ojec esol es one o he p oblems o he ac ual s a e o he a p og ams we used,
ha a en’ sui ed o he cus om BSC low. We do no expec addi ional expenses
ega ding his p ojec , he e is almos no main enance o upda es ela ed cos s.
Risks:
The main isk o his p ojec is ha is only compa ible o Ul ascale FPGAs and hus only
use ul as long as he BSC has hose FPGAs in hei in en o y. Risk which is e y low as
he FPGAs ha e s ill a good use ul li e ahead.
9.3. Social:
P ojec pu in o p oduc ion:
The de elopmen o his p ojec will gi e me a huge amoun o knowledge, i s ly because
his is my i s job, so knowing how is he en i onmen inside a company, how o be e
communica e wi h ou pa ne s and supe iso s, and ob iously he knowledge o
de eloping a p ojec , wha is a FPGA, how does i wo k, wha is hei in e nal s uc u e,
de elop p ojec s o FPGAs, di e en p o ocols and ools...
Use ul li e:
The cu en p og ams a e only used o know he empe a u e o he machines wi hou
doing any hing wi h hem, so hey a e a kind o en e ainmen ool because he p ac icali y
is limi ed. Addi ionally, he use s ha e o o e see he p og ams (wi h a GUI) and a e no
made o un independen ly in he backg ound.
This ool aims o be as easy o use as possible wi h he minimum use o o e sigh , only
equi ing an ini ial ins allmen and con igu a ion.
This p ojec has d awn he in e es o my supe iso since a long ime ago; he al eady
w o e an academic pape abou a simple eleme y ool [6], and we ha e been alking
abou w i ing a pape abou his p ojec when i ends. The e is also in e es om he
main enance eam because, as we do no know he empe a u e o he FPGAs, we ha e o
pu hei cooling sys ems a 100%, which is no op imal. Addi ionally, we wan o see o
which le el his could be used o p o iling he co es we de elop a BSC.
Risks:
We do no expec any scena io in which his could be ha m ul o anyone o any collec i e.
And he p ojec is de eloped o be as sel -con ained as possible wi h he leas ex e nal
dependencies as possible.
35/91
De eloping eleme y ools o Al eo accele a o ca ds
10. De elopmen o he p ojec
As i is es ablished on 1.4. P e ious concep s, he e a e wo main me hods o acqui e
eleme y da a om an FPGA: one is he ou -o -band me hod and he o he is he in-band
me hod, which di e in how he da a ansac ions a e done, ei he independen ly om he
da a ans e s o e PCI o wi h some side channels.
OOB does no consume bandwid h o e he PCIe e en hough he pins hemsel es a e
ound in he PCI in e ace because hey a e ese ed o ha special pu pose only, and so
a hos sys em would ne e use hem o memo y accesses, o example.
As In-Band is achie ed hanks o addi ional logic in he p og ammable logic on he FPGA,
i occupies esou ces. Those esou ces consume powe , and hus, using he in-band
me hod is less e icien a p o iling o cha ac e ising he FPGA. On he o he hand, OOB
uses he embedded p ocesso , which has i s own powe sou ce, and i is always
ope a ional.
Wi h IB we need a bi s eam wi h ha specialised IP, al hough he golden/ ac o y images,
which a e he s anda d bi s eams ha ge loaded when he e is a cold eboo , ca y ha
IP.
Luckily, all he Ul ascale de ices (Al eo ca ds) ca y he same embedded p ocesso ,
meaning ha OOB has wide suppo . IB is also suppo ed, bu he use has o con igu e
he IP o each a ge de ice.
Table 13 sums up all he bene i s om each me hod:
Ou -o -Band In-Band
Consump ion o
bandwid h:
No o PCIe ansac ions, only
o hos ead/w i e ope a ions.
Yes, I uses PCIe memo y
accesses
Dis u bance in
eadings: No Yes, I is pa o he logic
design
A ailabili y when
plugged: Always Only when a sui able
bi s eam is loaded
Mul ipla o m
(among Al eo): Yes Yes, bu econ igu a ion is
needed o each Al eo
Table 13: Bene i s pe eleme y me hod; own compila ion.
36/91

De eloping eleme y ools o Al eo accele a o ca ds
The embedded p ocesso , also known as sa elli e con olle is suppo ed on hese Al eo
Ca ds: U200, U250, U280, U50, U50LV, U55C, U55N, UL3xxx, U30, V70 and MA35D.
Mainly we a e going o wo k using he U55C, U280 and U250. The Ve sal V80s ha e a
new in e ace and do no house a sa elli e con olle .
FPGAs o he BSC:
•96 Al eos U55C on Makino e (Clus e ) di ided in o 12 nodes
•2 Al eos V80 on Jonu (Meep wo ks a ions)
•4 Al eos U280 on Nanu, Picu and wo on Raju (Meep wo ks a ions)
•2 Al eos U55C on Te u and Cucu (Meep wo ks a ions)
•1 Al eos U250 on Sa u (Meep wo ks a ion)
•2 Al eos U250 on Qua
•2 Al eos U50LV unplugged
In he ollowing chap e s he eade will ind an in dep h explana ion o all he me hods o
de elop eleme y o he Ul ascale FPGAs and wha can be done wi h ha in o ma ion.
37/91
De eloping eleme y ools o Al eo accele a o ca ds
11. Ou -o -Band
11.1. In oduc ion
The mos sensible solu ion o p o ide eleme y is Ou -o -band and in his chap e all he
implemen a ion de ails will be explained.
The sa elli e con olle is an embedded p ocesso made by Texas Ins umen s, ype MSP-
432. The SC is connec ed o he ol age egula o s h ough a powe managemen bus, o
an EEPROM, he mal and elec ical senso s h ough I2C p o ocol.
To access he SC da a an I2C bus is ou ed o he PCIe edge connec o ; ou dedica ed
pins a e used o ansmi ing da a: SMDa , SMClk, g ound, and Vcc. This bus does no
use a pu e I2C p o ocol di ec ly; a he , i uses a modi ied e sion wi h i s own s anda d,
SMBus, which s ands o Sys em Managemen Bus. SMBus uses addi ional wo king
condi ions such as a minimum clock equency o a imeou .
The p o ocol uses mas e and sla e de ices ha a e connec ed o he bus; each sla e has
a dedica ed add ess and should no be used by ano he de ice in he same bus. These
de ices sni he bus o know i anyone has a pe i ion wi h hei own add ess. I wo o
mo e de ices use he same add ess, hen he bus would be ende ed useless whene e
he e is a pe i ion o any o hem.
F om igu e 5 i can be obse ed ha he main way o access he SC is h ough he called
Baseboa d Managemen Con olle (BMC). The eason his is ecommended is because
he BMC which is an independen chip ha has al eady all he i mwa e needed o send all
eques s o he FPGA. This could become a p oblem (like in ou case) i he e a e no any
BMCs housed inside he mo he boa ds.
38/91
Figu e 5: Al eoTM U2XX, U50X, U55X, UL3524 block diag am; Modi ied Xilinx image
De eloping eleme y ools o Al eo accele a o ca ds
Figu e 6 explains he OOB pa h, i can be obse ed how he sa elli e con olle has ou
di e en sla es, his image makes i clea e how o access he SC, we can see ha he
sla es a e connec ed o he same bus, each ha ing hei own I2C add ess. Tha bus
begins a he Al eo ca d and i connec s o he PCIe slo , om hen i has a bi u ca ion
going o he hos sys em (i.e. he CPU) con olle o he Baseboa d Managemen
Con olle (BMC). The con olle is pa o he pla o m con olle hub (PCH), which is a
dedica ed chip ha manages all pe iphe al connec ions, p o ocols and communica ion in
gene al.
O e he PCI he pins which con ol he SMBus a e Smda pin 6, SMClk pin 5, addi ionally,
pin 7 and 8 p o ide g ound and 3.3V espec i ely all om ace B [18].
These ou sla es which can be accessed by he use , a e dedica ed each o a di e en
p o ocol, he sla es a e IPMI FRU a add ess 0x50, Al eo I2C a 0x65, SMBus (ARP) a
0x61 and inally MCTP/PLDM a 0x18 [19]. Also we can access he da a au oma ically i
we ha e a baseboa d managemen con olle a ailable on he mo he boa d which he
Al eo ca ds a e connec ed o, wi h he BMC he e is no need o c ea e a dedica ed
p og am as i uses a s anda d IPMI p o ocol.
Inside he Sa elli e Con olle , he e is a memo y module. Connec ed o he same bus as
he EEPROM, is he senso s' module, which is a simpli ica ion o all he senso s ha can
be connec ed and house he Al eo ca d. On he o he side, we can ind he Vol age
39/91
Figu e 6: Al eo sa elli e con olle in e ed in e nal s uc u e; Own compila ion
De eloping eleme y ools o Al eo accele a o ca ds
Regula o Module, which is con olled by he SC and makes su e all powe equi emen s
a e me .
The de ice has ou a ailable sla es, each answe ing o a di e en add ess as pe he
s anda d o each p o ocol. Each sla e beha es di e en ly, as he e a e di e en
cons ain s, such as he size o he packe s o da a o he a ailable space o eading. The
basic cha ac e is ics o hese p o ocols a e he ollowing:
- IPMI FRU: Usually used o eading and w i ing da a o EEPROMs. Suppo s mul i-by e
ead/w i e ansac ions.
- SMBus: A subse o I2C designed o sys em managemen communica ion. Suppo s
enables mul i-by e ead/w i e ansac ions.
- I2C: Gene al-pu pose communica ion p o ocol o connec ing mul iple de ices on a wo-
wi e bus. The packe size is lexible.
- PLDM: In ended o managing pla o m-le el da a. Has hea he s and payloads wi h
di e en ields o con igu e he ansac ion.
F om hese p o ocols, we unde s and ha he minimum equisi es o o ally exploi he
sa elli e con olle senso a chi ec u e a e a con olle six een bi s wide wi h he capabili ies
o sla e and mas e . The wid h will cons ain he amoun o logical add esses a de ice
o e SMBus can pe o m. As he ollowing igu e explains, his con olle is embedded in o
he hos sys em inside he pe iphe al con olle hub (PCH). The PCH is whe e all he
con olle s o he di e en in e aces om and o he p ocesso a e posi ioned.
40/91
Figu e 7: Requi emen s om hos sys em; own compila ion
De eloping eleme y ools o Al eo accele a o ca ds
We ied on he ollowing mo he boa ds:
-MSI MAG Z490 TOMAHAWK [MSI TOWER]
-Supe mic o X9DR3-F [Abuelo]
-Dell Inc. Op iPlex 9020 (00V62H) [Sandy]
-Dell Inc. Powe Edge T640 (07978V) [Qua ]
-Leno o ThinkSys em SR670 V2 Se e [MAKINOTE]
-Gigaby e X570 AORUS PRO [MEEP wo ks a ions]
A e a manual upda e o he i mwa e om he d i e ’s eposi o y a ailable on Linux and
an upda e o he BIOS in which no op ion o con igu a ion had o do wi h SMBus enabling
o disabling, he same aul y beha iou was obse ed.
O he p oblems obse ed we e ha eigh o he se e s om he MEEP wo ks a ions,
which had one FPGA each, we e ha he SMBus con olle did no bind i sel p ope ly wi h
he d i e s, and i seems he e a e no any a ailable d i e s o ou con olle , “FCH
SMBus Con olle '1022:790e' ”. E en a e opening icke s wi h Gigaby e, no solu ion
could be ound.
Finally, he e is ano he p oblem: in he o icial documen a ion om Xilinx, i is s a ed ha
he SMBus has ARP (can be seen a igu e 5). ARP s ands o add ess esolu ion p o ocol,
a ea u e ha di e en ia es i om I2C. The main p oblem wi h I2C is he collision o
add esses, and so ARP was designed o de ec and a oid collisions, changing he
add esses o some o he sla es connec ed o he bus.
Expe imen ally, i was es ed on Uni e si a Au onoma de Ba celona’s se e s, which we e
he only ones capable o i2c/SMBus in e changes, ha when wo o mo e Al eos we e
connec ed in pa allel o he PCIe connec o s he p og ams w i en in py hon ailed and
e en he i2c- ools, as when unning he ‘i2cde ec ’ command hose add esses
disappea ed, one o hose FPGA was one o BSC’s U50LV he o he an U55C.
Addi ionally, in hose se e s, we es ed some o he unc ionali ies like he cold and wa m
FPGA ese command (see able 16). I did no wo k.
Theo e ically, i was p o ed wi h he documen a ion o he ac ual manu ac u e o he SC
chip, Texas Ins umen s, model MSP-432. In hei o um appea ed a ques ion asking o
he implemen a ion o he ac ual ARP p o ocol inside hei lib a ies; hei answe was
nega i e. Addi ionally, Xilinx s a es ha he SC i mwa e is p op ie a y and would no sha e
i . The conclusion is hen ha ei he Xilinx's s a emen abou he exis ence o ARP is
47/91

De eloping eleme y ools o Al eo accele a o ca ds
inco ec , some p io managemen has o be done ha is no explained/documen ed, o i
does no wo k p ope ly.
The las hypo hesis is ha being he SMBus a e y in aused bus, he di e en
manu ac u e s do no include he suppo necessa y o he ansmission o he da a
be ween he chipse and he PCIe edge connec o ; his would be done o educe he cos s
o he manu ac u ing p ocess by educing he amoun o cables ou ed in he p in ed ci cui
boa d. This was es ed o he ‘MSI MAG Z490 TOMAHAWK’ wi h an oscilloscope, and
while he Al eo was connec ed, no signal om he SMBus clock o da a pin was obse ed.
This eason migh explain why, e en hough success ully enabling he BMC o he
‘Supe mic o X9DR3-F’ mo he boa d and connec ing o he webpage, no da a om he
Al eo accele a o ca ds we e de ec ed inside he lis o senso s a ailable o ead
in o ma ion excep o he na i e CPU and GPU. Su ely he mo he boa d does no ha e
ha SMBus ou ing.
This belie was ein o ced when we ead he s anda d o SMBus implemen a ion on PCI
[22]. The PCI speci ica ion e ision 3.0. s a es (page 271) “The SMBus in e ace
connec ions on he PCI connec o a e op ional” and addi ionally he ARP is conside ed an
op ional ea u e, which is also said a he speci ica ion o he SMBus [23].
No mally, he I2C p o ocol has ei he some jumpe s o a logical add ess ha can be se .
This is ue o he MSP430, a page 8 [24], Texas Ins umen s indica es ha he e is a
unc ion o he i mwa e ha se s he sla e’s add ess. This does no mean i has he ARP
ea u e, which i does no . Also looking a page 5 we can see in he able ha he e is no
ARP o MSP430, his in o ma ion is no a ailable o he MSP432 bu in he o ums o
Texas Ins umen s i is said: “some ea u es o SMBUS (like ARP, which equi es sla e-
side a bi a ion no suppo ed di ec ly in ou I2C pe iphe al) a en' a ailable in ha dwa e
and hus mus be bi -banged” [25].
In conclusion, al hough he o icial documen a ion om Xilinx s a es ha he SC has he
abili y o pe o m ARP, i is e oneous. This limi s he numbe o FPGAs capable o being
connec ed o a sui able mo he boa d o pe o m OOB o one, imp ac ical in ou case.
Supposing he hos is capable o a leas accessing 16-bi add esses and beha es like a
mas e and/o sla e, and ha i has suppo o SMBus.
In ou case, as no SMBus suppo was obse ed and we could no unplug 84 FPGAs om
he se e s (lea ing one pe mo he boa d). Fo ha eason, he de elopmen o he OOB
solu ion was s opped, and we began he implemen a ion o he in-band solu ion.
48/91
De eloping eleme y ools o Al eo accele a o ca ds
12. In-Band moni o ing
12.1. In oduc ion
Da a is ead h ough a Xilinx IP co e called Ca d Managemen Solu ion (CMS), which in
u n accesses he SC wi h some dedica ed pins designed o in e change communica ion
wi h ha module. Wi h he CMS we can ead a lo o egis e s ha a e housed inside he
SC and a e upda ed con inuously by he di e en senso s dispe sed in he ca d and
communica e in e nally wi h he I2C p o ocol. The ollowing igu e 9 shows he In-Band
me hod.
Whene e a de ice is connec ed wi h PCI, i can ask he hos o some memo y space o
alloca e da a. The CMS is accessed by he dedica ed ese ed memo y space inside he
PCI sessions. Those sessions a e subdi ided in o unc ions, and each unc ion has
di e en Base Add ess Regis e s (BAR). Also, hey can be ead h ough wha a e called
Di ec Memo y Access (DMA) commands o e he in e nal AXI p o ocol; al hough i may
wo k, we ha e ound some issues, and we do no ecommend i .
The physical unc ions (PF) a e used o di e en ia e unc ionali ies and no mally a e
ese ed o di e en d i e s. XRT lib a ies can use PF 0 o PF 1. Apa om he PF we
also ha e an amoun o BARs inside a PF. As he access o he CMS is done h ough a
special Axi Li e bus, i is compulso y ha i esides a BAR 2.
The CMS is con igu ed h ough he Vi ado wo k low, a p og am designed by Xilinx o
c ea e all he necessa y in as uc u e o de elop he bi s eams, which would be la e
inse ed inside he FPGA egion. Once a bi s eam has been c ea ed and loaded in o a
sui able FPGA, we would hen ha e only o de elop a p og am ha accesses ha memo y
space, ans o ms i , and analyses he da a.
49/91
Figu e 9: Al eoTM U2XX, U50X, U55X, UL3524 block
diag am; Modi ied Xilinx image
De eloping eleme y ools o Al eo accele a o ca ds
Wi h a C o Py hon p og am, hose BAR egions can be ead di ec ly. Knowing om he
Vi ado add ess edi o he di ec ion in which he CMS is housed, we can hen pe o m
eadings inside he BAR2 egion gi en all p io con igu a ion has co ec ly been sa is ied.
Excep o he Xilinx lib a ies, he e is a small p o ocol by which o ead he da a
success ully, which is common o all he di e en me hods explained in he subsequen
chap e s. Fi s , we pe o m a w i e ope a ion wi h a alue o jus one in o he egis e called
“MB_RESETN_REG”, as his gi en egis e is ac i e low (ac i a es ese on ze o), we
mus make su e i s con en is di e en om ze o. Once he w i e ope a ion has ended, we
can pe o m all he ead ope a ions om he lis o a ailable senso s speci ied in he
documen a ion [26]. We can oggle addi ional con igu a ion o ead he HBM empe a u e
senso s o he QSFP low-speed managemen ea u e, al hough s ange beha iou has
been ound on some senso s.
12.2. Implemen a ion
To implemen he In-Band eleme y, he e a e wo dis inc ways. Fi s , using he Xilinx
Run-Time lib a ies, which a e al eady a ailable, eliable, and easy o use i used on Xilinx
pla o ms. And secondly, when ac ually in eg a ing he IP co e used o he In-Band
module, we can do i h ough he BAR egion o he PCIe.
12.2.1. Xilinx Run-Time lib a ies
Xilinx Run-Time lib a ies ha e al eady made all necessa y so wa e in as uc u e a ailable
o ob aining all he desi ed in o ma ion equi ed o he analysis o da a; he issue esol es
in he ac ual wo k low a he Ba celona Supe compu ing Cen e .
This cen e wo ks wi h an al eady es ablished in as uc u e, such as he OpenPi on
p ojec . OpenPi on is he wo ld's i s open-sou ce, gene al-pu pose, mul i h eaded
manyco e p ocesso . I is a iled manyco e amewo k scalable om one o 1/2 billion
co es. I is a 64-bi a chi ec u e using SPARC 9 ISA wi h a dis ibu ed di ec o y-based
cache cohe ence p o ocol ac oss on-chip ne wo ks, and i is highly con igu able.
When designing a p ocesso , i is i al o ac ually es i . Compu e simula ions could ake
up o h ee days o ge comple ed, so he main way o debug he p ocesso s is o emula e
hem on FPGAs. The p oblem comes when uploading a design in o he FPGA; i done
di ec ly, a lo o issues may a ise abou he designing o a socke o he p ocesso , and o
e e y FPGA, a di e en socke would be needed o design.
So, o abs ac he egis e ans e le el (RTL) eam, all he necessi ies o each FPGA,
such as all he di e en pins and in e aces con ained, he OpenPi on p ojec was bo n. A
subp ojec cu en ly used by BSC is he w appe shell, OpenPi on, plus ACME, and la e
he embedded shell p ojec , known also as he " pga shell, was bo n.
50/91
De eloping eleme y ools o Al eo accele a o ca ds
The main issue esides in he ac ha all hese shells e ol e a ound he use o he QDMA
engine, which s ands o ‘Queue Di ec Memo y Access.’ This engine p o ides a di e en
in e ace om i s compe i o , XDMA, made by Xilinx. These wo engines a e no di ec ly
compa ible because hey use di e en in e aces, and so o his p ojec , al hough i has
been esea ched, he po en ial o XRT lib a ies has no been used in he inal p og am, as
XRT only wo ks wi h XDMA, which he BSC does no use.
These lib a ies should ge ins alled alongside Vi ado o Vi is. On Ubun u-based
en i onmen s, i can be ound in he ‘/op /xilinx/x /bin’ olde . Be o e p oceeding in o he
commands, he e a e some conside a ions.
The ollowing use ul commands a e om he XRT lib a ies:
$ sudo /op /xilinx/x /bin/xbmgm examine
Ou pu s he in o ma ion abou he amoun and which Xilinx FPGAs de ices a e connec ed,
gi ing he s a es o he ca ds and i s BDFs.
$ sudo /op /xilinx/x /bin/xbmgm examine - all
Ou pu s he in o ma ion abou he FPGA con igu a ion such as empe a u e h esholds and
mo e.
$ /op /xilinx/x /bin/xbu il examine -d <BDF> - elec ical
he mal
Ou pu s he in o ma ion ega ding elec ical and he mal, such as ol ages, cu en s, powe
consump ion and empe a u e o some senso s.
$ sudo /op /xilinx/x /bin/xbu il alida e -d <BDF> - all
Pe o ms all a ailable es s o he gi en de ice.
Some commands equi e he use o p i ileges. Be o e execu ing hese commands, make
su e ha Xbmgm and Xocl a e he same e sion, i no incompa ibili ies may a ise and
also he la es e sion is ins alledV.
The e should no be any p oblem in de eloping a ool ha om he p e ious commands,
ex ac s and pa ses he in o ma ion in o a g aphical use in e ace, bu as men ioned
be o e, he inal implemen a ion o he eleme y ool does no use he XRT.
The eason XRT wo ks is hanks o he so-called ‘pla o ms’ which is a comp ehensi e
ecosys em o ools, so wa e, ha dwa e and me hodologies p o ided o suppo all he
wo k low, such as designing, de eloping, p og amming and deploying un o he FPGAs.
The pla o ms when loaded wi h he golden bi s eam come al eady wi h he CMS
in eg a ed, ha 's why o e w i ing hese pla o ms disables he XRT en i onmen and hus
he moni o ing ools.
V Addi ional obuleshoo ing on:
h ps://xilinx.gi hub.io/Al eo-Ca ds/mas e /debugging/build/h ml/docs/x - oubleshoo ing.h ml
51/91
De eloping eleme y ools o Al eo accele a o ca ds
12.2.2. Base Add ess Regis e s
The pa h we ha e decided is bes is h ough he BAR egions, as hey can be accessed in
a mo e s aigh o wa d manne .
The in e aces decla ed in he CMS a e:
•s_axi_c l: Axi-Li e con ol sla e in e ace.
•sa elli e_gpio: GPIO signals om SC.
•aclk_c : Sou ce clock, make su e i is a 50MHz Clock.
•a ese n_c l: Rese pin ac i e low ese . (We ecommend o connec i o PCI ese )
•in e up _hbm_ca ip: O ed HBM ca iped signals.
•Hbm_ emp_1 and 2: HBM empe a u e senso bus.
•sa elli e_ua : UART signals o SC.
•in e up _hos : In e up gene a ed by CMS.
In he ollowing explana ion i will only be desc ibed he connec ion wi h he CMS om he
U55C and U280 which a e equi alen .
52/91
Figu e 10: Ca d Managemen Solu ion IP in e ace image o U280 and U55C;
Sou ce Xilinx

De eloping eleme y ools o Al eo accele a o ca ds
Exhaus i e s eps o build success ully a CMS p ojec :
C ea e o load a p ojec o Vi ado, p e e ably wi h an al eady exis ing DMA engine such
as he QDMA/XDMA IP, an HBM con olle , and also a PCI co e. Easily you can add a new
IP wi h a igh click in o he block design; sea ch o he ‘Ca d Managemen Solu ion
Subsys em.’ This exac module does no need any addi ional con igu a ion, only he
placemen and ou ing o i s in e aces. When he CMS is ins an ia ed in o he block
design, a boa d p ese is applied; his means he boa d-speci ic CMS will ge selec ed as
di e ences exis among hese modules.
Addi ionally, add a ‘P ocesso Sys em Rese ,’ he same way as he CMS. Make su e
double-clicking he QDMA IP, on he op bu on 'PCIe: BARs,' he size o he 'AXI Li e
Mas e ' is a leas 256KB; o he wise, p oblems should be expec ed.
Connec bo h HBM’s DRAM_STAT_TEMP o he CMS’s ones. O all he signals om he
HBM’s ca ip and place i o in e up _hbm_ca ip. The clock sou ce o he CMS mus be
a 50 MHz clock and also he slowes _sync_clk om he ese con olle , and inally, we
ecommend placing he PCIe ese o he ex _ ese _in.
I he e exis s an AXI in e connec li e connec ed o he QDMA M_AXI_LITE po , c ea e an
addi ional mas e ou pu , wi h he clock connec ed o he same sou ce as he aclk_c l and
he ese o he mas e in e ace o he p ocesso sys em ese in e connec _a ese n. The
new mas e in e ace ou es i o s_axi_c l.
F om he P ocesso Sys em Rese , connec he ‘ex _ ese _in’ wi h he ‘axi_a ese n’ om
he QDMA IP. Basically, his module will synch onise all ese s wi h he clock sou ce om
he CMS (QDMA and CMS ha e di e en clock sou ces).
Once all in a-signals ha e been connec ed, we mus make wo ex e nal in e aces o he
sa elli e_gpio and sa elli e_ua , le -click hose in e aces and click ‘make ex e nal.’ To
connec he in e aces, i is needed o modi y he ‘sys em_ op.s ’ ile, w i en in Sys em
Ve ilog, he e is needed o add he CMS new signals o he in e aces sec ion as:
inpu [0:0] sa elli e_ua _ xd,
ou pu [0:0] sa elli e_ua _ xd,
inpu [3:0] sa elli e_gpio,
And inally, o end he implemen a ion, add he speci ic cons ain s o he FPGA signals o
he cons ain s olde , hose can be ex ac ed om he Xilinx websi e o each speci ic
Al eoVI. FPGA pin cons ain s de ine he ules and equi emen s o how FPGA pins a e
connec ed, con igu ed, and u ilised in ha dwa e designs.
VI Example o U280: h ps://www.xilinx.com/p oduc s/boa ds-and-ki s/al eo/u280.h ml# i ado
53/91
De eloping eleme y ools o Al eo accele a o ca ds
Figu e 11 shows he inal block design o he socke .
54/91
Figu e 11: Final implemen a ion o he FPGA Shell wi h CMS, block design iew; own compila ion
APB_ s _and
U ili y Vec o Logic
Op1[0:0]
Op2[0:0] Res[0:0]
E h100GbSys _w_hbm
m_axi_ x
m_axi_sg
m_axi_ x
qs p_4x
qs p_4x_g x_n[3:0]
qs p_4x_g x_p[3:0]
qs p_4x_g x_n[3:0]
qs p_4x_g x_p[3:0]
s_axi
qs p_ e ck
qs p_ e ck_clk_n
qs p_ e ck_clk_p
in c[1:0]
x_clk
x_ s n[0:0]
s_axi_clk
s_axi_ ese n
x_clk
x_ s n[0:0]
MEEP_ua _0
S_AXI
in e ace_ua
s_axi_aclk
s_axi_a ese n
ip2in c_i p
in
sin
sou
SHELL_ROM
S_AXI
s_axi_aclk
s_axi_a ese n
axi_gpio_0
AXI GPIO
S_AXI GPIO
gpio_io_o[4:0]
s_axi_aclk
s_axi_a ese n
axi_xba _pcie
AXI In e connec
S00_AXI
M00_AXI
M01_AXI
ACLK
ARESETN
S00_ACLK
S00_ARESETN
M00_ACLK
M00_ARESETN
M01_ACLK
M01_ARESETN
axi_xba _pcie_li e
AXI In e connec
S00_AXI
M00_AXI
M01_AXI
ACLK
ARESETN
S00_ACLK
S00_ARESETN
M00_ACLK
M00_ARESETN
M01_ACLK
M01_ARESETN
bscan_p im
Debug B idge
m0_bscan
m1_bscan
bscan2j ag
BSCAN o JTAG Con e e
S_BSCAN M_JTAG
clk_wiz_1
Clocking Wiza d
CLK_IN1_D
clk_ou 1
clk_ou 2
locked
cms_ ese
P ocesso Sys em Rese
slowes _sync_clk
ex _ ese _in
aux_ ese _in
mb_debug_sys_ s
dcm_locked
mb_ ese
bus_s uc _ ese [0:0]
pe iphe al_ ese [0:0]
in e connec _a ese n[0:0]
pe iphe al_a ese n[0:0]
cms_subsys em
Ca d Managemen Solu ion Subsys em
s_axi_c l
sa elli e_ua
sa elli e_gpio[3:0]
aclk_c l
a ese n_c l in e up _hos
in e up _hbm_ca ip[0:0]
hbm_ emp_1[6:0]
hbm_ emp_2[6:0]
dbg_j ag
debug_hub
Debug B idge
S_BSCAN
clk
e h_axi
e h_axi_a s n[0:0]
e h_i q[1:0]
hbm_ca ip[0:0]
hbm_ca ip_o
U ili y Vec o Logic
Op1[0:0]
Op2[0:0] Res[0:0]
hbm_0
HBM IP
SAXI_00
SAXI_01
SAXI_02
SAXI_29
SAXI_30
SAXI_31
HBM_REF_CLK_0
HBM_REF_CLK_1
AXI_00_ACLK
AXI_00_ARESET_N
AXI_00_WDATA_PARITY[31:0]
AXI_01_ACLK
AXI_01_ARESET_N
AXI_01_WDATA_PARITY[31:0]
AXI_02_ACLK
AXI_02_ARESET_N
AXI_02_WDATA_PARITY[31:0]
AXI_29_ACLK
AXI_29_ARESET_N
AXI_29_WDATA_PARITY[31:0]
AXI_30_ACLK
AXI_30_ARESET_N
AXI_30_WDATA_PARITY[31:0]
AXI_31_ACLK
AXI_31_ARESET_N
AXI_31_WDATA_PARITY[31:0]
APB_0_PCLK
APB_0_PRESET_N
APB_1_PCLK
APB_1_PRESET_N
AXI_00_RDATA_PARITY[31:0]
AXI_01_RDATA_PARITY[31:0]
AXI_02_RDATA_PARITY[31:0]
AXI_29_RDATA_PARITY[31:0]
AXI_30_RDATA_PARITY[31:0]
AXI_31_RDATA_PARITY[31:0]
apb_comple e_0
apb_comple e_1
DRAM_0_STAT_CATTRIP
DRAM_0_STAT_TEMP[6:0]
DRAM_1_STAT_CATTRIP
DRAM_1_STAT_TEMP[6:0]
mc_ s n[0:0]
mem_axi
mem_calib_comple e[0:0]
ncmem_axi
pci_exp ess_x16
pcie_gpio[4:0]
pcie_ e clk
Ex A s n
chipse _clk
e h_axi_aclk
mc_clk
pcie_pe s n
qs p_ e _clk_n
qs p_ e _clk_p
ese n
s232_ xd
s232_ xd
ua _i q
pu_clk
p oc_sys_ s _pcie
P ocesso Sys em Rese
slowes _sync_clk
ex _ ese _in
aux_ ese _in
mb_debug_sys_ s
dcm_locked
mb_ ese
bus_s uc _ ese [0:0]
pe iphe al_ ese [0:0]
in e connec _a ese n[0:0]
pe iphe al_a ese n[0:0]
qdma_0
Queue DMA Subsys em o PCI Exp ess
M_AXI
M_AXI_LITE
pcie_mg us _i q
dsc_c d _in m_dsc_s s
m_dsc_s s_ dyus _ l
qs s_ou
qs s_ou _ dy
sys_clk
sys_clk_g
sys_ s _n use _lnk_up
axi_aclk
axi_a ese n
so _ ese _n
phy_ eady
qs p_4x_g x_n[3:0]
qs p_4x_g x_p[3:0]
qs p_4x_g x_n[3:0]
qs p_4x_g x_p[3:0]
s _ea_CLK0
P ocesso Sys em Rese
slowes _sync_clk
ex _ ese _in
aux_ ese _in
mb_debug_sys_ s
dcm_locked
mb_ ese
bus_s uc _ ese [0:0]
pe iphe al_ ese [0:0]
in e connec _a ese n[0:0]
pe iphe al_a ese n[0:0]
s _ea_CLK1
P ocesso Sys em Rese
slowes _sync_clk
ex _ ese _in
aux_ ese _in
mb_debug_sys_ s
dcm_locked
mb_ ese
bus_s uc _ ese [0:0]
pe iphe al_ ese [0:0]
in e connec _a ese n[0:0]
pe iphe al_a ese n[0:0]
sa elli e_gpio[3:0]
sa elli e_ua
sysclk0
sysclk1
ua _axi
u il_ds_bu
U ili y Bu e
CLK_IN_D IBUF_OUT[0:0]
IBUF_DS_ODIV2[0:0]
u il_ds_bu _hbm_clk
U ili y Bu e
CLK_IN_D IBUF_OUT[0:0]
dd_0
Cons an
dou [0:0]
De eloping eleme y ools o Al eo accele a o ca ds
The CMS p ojec has been inco po a ed in o he FPGA-SHELL; his enables all p ocesso s
made a BSC ha use his w appe o p o ide eleme y on FPGAs wi hou any change in
he cu en wo k low, as he CMS has i s own embedded p ocesso called Mic oBlaze.
P io o any eading, we mus disable he ese egis e ; as i is an ac i e low, we mus
w i e a '1' o ha egis e , posi ioned on o se 0x20000 om he s a ing posi ion o he
CMS module. Once w i en, we can ead he egis e s, aking in o accoun he s a ing
add ess and he o se o he gi en egis e s. We can gua an ee he CMS wo ks by
pe o ming a ead in o he 0x28000 which is he Regis e Map ID, wi h a ixed alue o
0x74736574 o o he 0x28014 which s o es he ype o Al eo ca d we a e using (e.g.
0x0x5535354E o U55C).
The in o ma ion is ead hanks o he di e en lib a ies in C and Py hon. The Py hon lib a y
in ques ion is ‘pypcie’. The inpu s equi ed o make i wo k a e he BDF and he ini ial
add ess whe e he CMS is placed.
The BDF can be known using he ‘lspci’ command and sea ching o ‘P ocessing
accele a o s: Xilinx Co po a ion De ice’; ha iden i ie is loca ed on he igh side. I he ‘-
’ op ion is added, we can obse e addi ional impo an da a, such as he egions, also
known as BAR. I mus be seen in he ollowing “Region 2: Memo y a … [size=...]” he size
a leas being 256Kb i done co ec ly (in FPGA-SHELL he minimum is 512Kb). And as
al eady men ioned, he ini ial add ess can be e ched om he Vi ado add ess edi o .
The p og am is sel -con ained; ha means i will au oma ically de ec which ype o Al eo
ca d is connec ed o he hos sys em. A e he ini ialisa ion o he da a s uc u es, i will
ou pu whe he i has been connec ed success ully. I no e o s a e shown, he p og am
will c ea e a lis o he a ailable senso s each ca d has, and e e y hal a second i will
que y all o hose, no malise hem (i.e., change he uni s o s anda d), and ou pu he
imes amp once all alues a e shown on he sc een.
In he igu e 12 is an example o a un made in h ee di e en nodes. Two o hose house a
U280 and he igh mos an U55C. The p og am on he heade shows he Al eo ype, i s
BDF and he endo and he de ice numbe .
Xilinx speci ies ha he o al powe consump ion is he sum o he powe s consumed in he
PEX and AUX channels and o each ol age:
To al powe = Powe (12V_PEX + 12V_AUX + 3V3_PEX + 3V3_AUX)
Ob iously, each powe alue is he mul iplica ion o x_<PEX/AUX> * x_<PEX/AUX>_I
whe e ‘x’ is ei he 3V3 o 12V and i comes om PEX o AUX senso s (‘I’ meaning
In ensi y/Cu en ).
55/91
De eloping eleme y ools o Al eo accele a o ca ds
Fo example, on he U55C he powe consump ion can only be calcula ed om he PEX
channels, as he e a en’ auxilia y ones, and o he U280 he e a e only senso s o he
12V channels (12V_AUX+12V_PEX).
S ange beha iou s ha e been obse ed, such as missing o disabled senso s, as can be
seen in igu e 12. Fo example, when he p og am was un on di e en nodes wi h he
same ype o Al eo ca ds connec ed, he e we e senso s whose eading was ze o on one
and da a on he o he ; maybe some senso s ha e been damaged o ende ed inope able.
In his case, he PICU node ‘CAGE_TEMP0’ is ze o, and in NANU wi h he same ca d, i s
alue is 32ºC.
Addi ionally, e en a e doing he manual enabling o he HBM senso s, he p og am has
been able o ead alues om hose senso s only when he p ocesso has been boo ed.
The enabling o hese is also included in he p og am. As he o icial documen a ion says,
he 27 h bi mus be w i en wi h one; a e w i ing i and wai ing some ime, he e a e also
di e en ypes o egis e , and he ype o egis e ha holds he a e age alue can be
ese o ze o i necessa y.
56/91
Figu e 12: CMS ou pu exce p on h ee di e en nodes o he Meep Wo ks a ions; own compila ion
De eloping eleme y ools o Al eo accele a o ca ds
We can obse e some beha iou s ha we did no know. The U280 an beha es wi h a
ixed cons an eloci y, as we can see in igu e 16, he e is no inc ease in e olu ions
despi e he inc ease in empe a u e o he FPGA ( igu e 17). The ed e ical line a igu e
17 and 18 is due o he ‘Boo acme’ e en , ha is, he loading o he ope a ing sys em on
he FPGA. When loading he ope a ing sys em, he es a signal ceases o be ope a ional,
and he p ocesso begins o consume powe almos immedia ely; he main consume is
he HBM memo y.
In igu e 17 and 18 he e is a end lineVIII[33] gene a ed au oma ically by Google D i e
Shee s; his way we can pe cei e be e he inc ease o he alues. The main di e ence
be ween hose wo igu es is ha consump ion ises ab up ly due o he enabling o some
o he modules and s abilizes. Tempe a u e inc ease is much slowe .
The main p oblem wi h all he igu es is he amoun o noise and poo p ecision.
Tempe a u e is unca ed o he in ege alue, hus we can see some small ises in igu e
17. Noise is also p esen a he an speed and powe senso ; in he case o he la e , he
p ecision ge s wo se because he powe da a is no a esul o a di ec eading o a senso
bu a he he mul iplica ion and addi ion o he da a om he cu en and ol ages, which
inc eases noise and dec emen s p ecision. Vol ages and cu en s a e in e p e ed as
milli ol s and milliampe es.
VIIIR² indica es he de e mina ion coe icien . Which indica es how a model (in his case he black line) is as close as
he eal da a. The close he R², he close o he eal da a.
63/91

De eloping eleme y ools o Al eo accele a o ca ds
13.1.2. Second es ; One U55C FPGA
The second es was launched wi h he W appe -Shell using HBM memo y alongside ou
co es om he Laga o Hun plus wo ec o ial p ocessing uni s on a U55C om he Meep
wo ks a ions (CUCU). The U55C is a passi ely cooled FPGA. The es las ed
app oxima ely wo hou s.
64/91
Figu e 19: Tempe a u e o e ime (U55C); Own compila ion
Figu e 20: To al powe o e ime (U55C); Own compila ion
De eloping eleme y ools o Al eo accele a o ca ds
Wi h he U55C he beha iou obse ed in empe a u e and powe di e s g ea ly om
hose in he p e ious igu es. Fi s we obse e in igu e 19 ha he ini ial empe a u e o
he U55C is much highe han ha o he U280, ha i is because hey ha e a an and he
U55C is passi ely cooled, also pe haps because in he da a cen e, hey a e close o he
en ila ion.
The me hodology we’ e ollowed is o i s load he bi s eam o he design and le he
empe a u e s abilise, hen boo he FPGA and wai some ime, and inally, a e i has
s abilised as well, eload he same bi s eam again. Following his p ocedu e we can make
appa en he ela ion be ween he empe a u e and powe .
Tempe a u e inc eases mo e sha ply, om he ini ial 64ºC o abou 85ºC. Wha we can
obse e is ha as he empe a u e inc eases, so oo does he powe consumed. This
canno be obse ed in he U280 ( igu e 18), a leas as clea ly as in igu e 20. “The
ela ionships be ween powe and empe a u e a e complex: an inc ease in dynamic powe
inc eases he o al powe , which inc eases he empe a u e, which inc eases he leakage
powe , which inc eases he o al powe , which inc eases he empe a u e (and so on).
The e is hus a cyclic dependence be ween powe and empe a u e, which means ha a
small inc ease in powe leads o a small inc ease in empe a u e, which u he leads o a
small inc ease in powe , and so on. In mos cases, his p ocess con e ges; howe e , in
some cases i does no and his leads o a e y dange ous condi ion called he mal
unaway.” [34].
A di e ence be ween he ini ial consump ion o he U55C ( igu e 20) and U280 ( igu e 18)
can be explained p ima ily by he addi ional consump ion o he an, he di e ence is
app oxima ely o ou wa s, which makes sense; he U280 speci ica ion doesn’ s a e he
an consump ion [35]. We mus emembe ha wo implemen a ions uns o he same
design on he same ool can esul in di e en bi s eams. As some o he s ages in he
FPGA design low a e non-de e minis ic when unning he low on mo e han one h ead
[36].
In he eloading e en he FPGA is i ually elimina ed om he PCI de ices un il he
bi s eam is comple ely loaded, ha is why he e is an app oxima e wo minu e gap in he
da a ( igu e 20, second e ical ed line).
13.1.3. Thi d es ; Eigh simul aneous U55C FPGAs
The hi d es was made on he Makino e node wel e wi h eigh FPGAs. Loaded wi h he
W appe -Shell alongside ou co es om he Laga o Hun plus wo ec o ial p ocessing
uni s in o eigh U55C FPGAs. The es las ed o y- i e minu es.
65/91
De eloping eleme y ools o Al eo accele a o ca ds
In he ollowing igu es 21 and 22 he e a e di e en FPGAs unning di e en es s based
on he same bi s eam. In he legend on he igh o he igu es, you can see he BDF
iden i ie , along wi h he ype o es ha has been pe o med. 'Boo ed' implies ha he
FPGA has only loaded he ope a ing sys em. 'S essed' indica es ha es s ha e been
launched wi hin he ope a ing sys em session ha s ess he co es. Finally, 'idle' indica es
ha only he bi s eam has been loaded in o he FPGA and no hing else.
The mos cu ious ea u e is ha he ini ial consump ion among di e en FPGAs di e s wi h
a high deg ee o a iance. Be ween he leas (yellow line) and he highes (pale blue) ini ial
consump ions, he e is app oxima ely a whole wa .
66/91
Figu e 21: To al powe o e ime using eigh U55C FPGAs; Own compila ion
Figu e 22: Tempe a u e o e ime using eigh U55C FPGAs; Own Compila ion
De eloping eleme y ools o Al eo accele a o ca ds
When launching he p ocedu e o boo he di e en FPGAs, i has o be done one by one,
he e o e he e is a delay be ween hem. To ensu e ha he e is a co ela ion be ween he
empe a u e inc ease and he powe , we ha e calcula ed he alue. As i is e y close o '1'
we unde s and ha i is p ac ically pe ec .
Powe
12V PEX
Powe
12V AUX
Powe
3V3 PEX
To al
powe Tempe a u e Co ela ion
be ween powe
and empe a u e
inc ease
Inc ease Inc ease Inc ease Inc ease Inc ease
BOOTED 19:00.0 41.23% 42.01% 6.71% 34.67% 24.67% 99.24%
1a:00.0 37.46% 48.64% 6.84% 33.94% 26.32% 99.48%
BOOTED +
STRESSED
33:00.0 39.41% 52.26% 8.00% 36.08% 26.83% 99.78%
34:00.0 40.31% 46.69% 7.33% 35.30% 22.67% 98.62%
Table 20: Analysis o powe and empe a u e inc emen ; Own compila ion
Fo able 21, ‘p e’ means be o e and ‘pos ’ a e he boo ing e en .
Powe 12V PEX Powe 12V AUX Powe 3V3 PEX To al powe
A e age
p e
A e age
pos
A e age
p e
A e age
pos
A e age
p e
A e age
pos
A e age
p e
A e age
pos
BOOTED 19:00.0 9.54 13.48 4.19 5.96 3.34 3.56 17.07 22.99
1a:00.0 10.53 14.48 3.61 5.37 3.33 3.56 17.48 23.41
BOOTED
STRESSED
33:00.0 10.05 14.00 3.52 5.36 3.22 3.48 16.79 22.85
34:00.0 10.12 14.21 3.61 5.30 3.28 3.52 17.02 23.03
IDLE
b3:00.0 10.44 3.75 3.28 17.47
b4:00.0 9.84 4.59 3.26 17.68
cc:00.0 10.18 4.58 3.25 18.01
cd:00.0 10.31 4.08 3.30 17.69
Table 21: A e age absolu e alues o p e ious able; Own compila ion
67/91
De eloping eleme y ools o Al eo accele a o ca ds
A inal obse a ion ha we made is ha we saw, as seen in able 22, ha he s anda d
de ia ion o some eco ds ends o be ze o, which is e y su p ising o us since i is e y
a e o he alues no o a y o e he whole ime he es was unning. These alues a e
indica ed wi h a ed cell.
Powe 12V
PEX
Powe 12V
AUX
Powe 3V3
PEX To al powe Tempe a u e
SD
p e
SD
pos
SD
p e
SD
pos
SD
p e
SD
pos
SD
p e
SD
pos SD p e SD
pos
BOOTED 19:00.0 0.031 0.145 0.001 0.062 0 0.008 0.031 0.213 0 0.608
1a:00.0 0.047 0.143 0.023 0.064 0.004 0.009 0.050 0.216 0.456 0.412
BOOTED
STRESSED
33:00.0 0.001 0.137 0.004 0.080 0 0.009 0.004 0.218 0 0.442
34:00.0 0.021 0.141 0.028 0.087 0.013 0.009 0.031 0.215 0.488 0.649
IDLE b3:00.0 0.003 0.044 0 0.044 0.146
b4:00.0 0 0.001 0 0.001 0.443
cc:00.0 0.048 0.004 0 0.049 0
cd:00.0 0.051 0.037 0 0.055 0.482
Table 22: Compila ion o s anda d de ia ion (SD) pe senso ; Own compila ion
68/91

De eloping eleme y ools o Al eo accele a o ca ds
13.1.4. Fou h es ; Eigh simul aneous U55C FPGAs
The las es was made on he Makino e node wel e wi h eigh FPGAs. All he FPGAs
ha e a bi s eam ha uses a di e en shell called Embedded-shell wi h only one co e o
Sa gan anaIX.
F om hese wo g oups, he i s was boo ed and s essed, he second only boo ed. To
s ess he co es, a se ies o concu en p og ams we e execu ed con inuously ha do no
access memo y. O he wise, as he HBM memo y consumes a lo o powe , i would add
oo much noise, up o eigh een wa s, acco ding o a powe epo om Vi ado. The
impo an da a om his es is o see he inc ease o powe consump ion when unning
so wa e on he emula ed p ocesso s.
The pe iod be ween he e ical black lines o bo h igu e 23 and 24 indica es he p ocess
o loading he s ess es on each FPGA; he da a be ween he wo black lines has no
been used o he calcula ions on able 23.
IX Sa gan ana is an in o de p ocesso wi h a RISC-V a chi ec u e made a he BSC.
69/91
Figu e 23: Powe o e ime o s essed U55Cs; Own compila ion
Figu e 24: Tempe a u e o e ime o non-s essed U55Cs; Own compila ion
De eloping eleme y ools o Al eo accele a o ca ds
Addi ionally, o each igu e, he e is a pale line colo ed which indica es he endency o he
alues. In igu e 24 i is almos impossible o see due o alues no changing.
Tempe a u e in his es was moni o ed oo, bu in his case he Makino e FPGAs a e
su icien ly egula ed, main aining a s eady a e age among all FPGAs, which can be seen
in able 23.
When unning his es , we ound ha , while i is common sense o obse e an inc ease in
powe consump ion when unning so wa e on an emula ed pla o m, as he e is an
inc ease in he ac i i y ac o o he ansis o s, in p e ious es s, i was no shown as
clea ly as in his es ( igu e 23). Al hough he e is no a s ong inc ease in empe a u e as
a esul ( igu e 24).
AVG Tempe a u e (T) AVG To al Powe Cons. (W)
P e s ess Pos s ess P e s ess Pos s ess Tempe a u e
inc.
To al Powe
inc.
19 38.00 38.00 18.80 18.99 0.00% 1.01%
33 38.13 38.22 18.57 18.56 0.23% -0.04%
B3 38.03 38.07 19.24 19.43 0.12% 0.99%
CC 39.01 39.00 19.82 19.82 -0.02% 0.01%
1A 37.00 37.00 19.29 19.29 -0.01% 0.02%
34 38.00 38.00 18.76 18.92 0.00% 0.85%
b4 38.00 38.00 19.44 19.44 0.00% -0.04%
CD 38.70 38.79 19.50 19.69 0.21% 0.98%
Table 23: Powe inc ease due o s essing emula ed p ocesso s on FPGAs; own compila ion
Vi ado powe epo es ima es he Sa gan ana co e o consume 0.405. F om able 23 he
a e age inc ease in empe a u e o said HDL design is es ima ed o be 0.95%, 180mW.
70/91
De eloping eleme y ools o Al eo accele a o ca ds
13.2. P o iling
Teleme y would no p o ide insigh s o emula ed p ocesso s, since he e a e no di ec
co ela ions be ween he consumed powe om an FPGA and he inal p ocesso s on a
p in ed ci cui boa d. B ie ly below I will explain he eason.
P in ed ci cui boa ds use s anda d cells, which a e a g ouping o ansis o s and an
in e connec ing s uc u e ha p o ides Boolean logic unc ions o s o age. The boolean
logic can hen c ea e complex beha iou s, which a e wha almos all modules a e made o
[37].
FPGA, in con as , wo ks wi h lookup ables, which a e memo ies illed wi h all he di e en
inpu s and, o each inpu , i s ou pu . The p inciple is ha he beha iou o an emula ed
ha dwa e is calcula ed and s o ed in he memo ies. F om igu es 25 and 26, he
di e ences can be seen be ween a s anda d cell and an FPGA cell. No e ha no all FPGA
cells a e equal; some ha e di e en ypes o esou ces like BRAM iles, a i hme ic uni s,
LUTRAM, among o he s [38].
Al hough we can p o i om he ac
ha any FPGA would be less e icien
han any ASIC one. Ou p ocess o
de elop any ha dwa e design is o
c ea e he mos e icien unc ion ha
p o ides he equi ed unc ionali ies.
Tha is done h ough Ka naugh[39],
which simpli ies he algeb aic
exp essions in o Boolean and hen
ou pu s he mos e icien solu ion.
FPGAs do he opposi e; hey p o ide
all he solu ions, e en he ones we do
no ca e o . No only ha , he s uc u e
o he FPGA signi ican ly inc eases he
leakage powe due o he ex a ou ing
and s uc u es needed.
71/91
Figu e 25: S anda d cell schema ic; sou ce
Figu e 26: FPGA cell schema ic; sou ce
De eloping eleme y ools o Al eo accele a o ca ds
The ques ion is hen, i an FPGA is less e icien , hen calcula ing he consump ion o he
FPGA p o ides a highe eading and hus a op ma gin o powe consump ion once i is
p in ed. The s anda d low o calcula e he consump ion o any chip is h ough simula o s,
which calcula e he o al consump ion based on he simula ion uns and a chi ec u e.
The o mula o he powe consump ion is he ollowing:
Ps a ic = VDD * IDD
Pdynamic =
1
2
C * VDD² * * α
Pleakage = VDD * Ileak
Psho ci cui = * VDD * Isho *
The key p oblem comes om he unce ain y o he alues om he p e ious o mulas. To
ex apola e he FPGA design o he ASIC design is o make a lo o assump ions, and he
accumula ion o e o will ine i ably induce a lo o di e ence be ween he calcula ed
alues and he eal ones: capaci ance, ac i i y ac o , echnological di e ences om one
scale o ano he . E en wi h he da a om he syn hesising simula ions o ASICs, he
di e ence can be o se e al ac o s.
Now, doing p o iling on an accele a o design in ended o a speci ic FPGA can gi e be e
esul s. The use can use his ool o calcula e he di e ences be ween e sions o he
HDL code and he di e en Vi ado s a egiesX.
To pe o m be e p o iling o he accele a o s on he FPGA, we explo ed he called DFx,
which s ands o (Dynamical Func ion eXchange). Using his unc ionali y an use can load
on he ly di e en segmen s o he FPGA wi hou he need o ew i e all he FPGA
bi s eam [40]. This can dec emen he ime o syn hesis and implemen a ion because he
use can speci y a s a ic pa i ion and a econ igu able one. The s a ic pa i ion can be
sa ed once implemen ed wi h a checkpoin ile, which enables euse o he ne lis s [41].
When p o iling designs o he FPGA, one o he p oblems is ha all he in e aces om he
Shell (including he CMS) consume hei own powe . To di e en ia e om he consump ion
om he accele a o , shell DFx is used. Independen ly implemen ing he Shell and
measu ing i s consump ion as a s andalone bi s eam wouldn’ be p ecise enough because
he ne lis s o he Shell wi h he accele a o (only aking he sec ion ega ding he shell)
and he s andalone shell would be di e en .
X Vi ado s a egies a e a se o con igu a ion ha cons ain he syn hesis and implemen a ion algo i hms o sui a
speci ic need, like powe consump ion, u ilisa ion, un- ime, iming enclosu e op imisa ions.
72/91
De eloping eleme y ools o Al eo accele a o ca ds
- ARP: Add ess Resolu ion p o ocol, a ea u e in SMBus designed o sol e I2C add ess
collisions.
- XRT: Xilinx Run-Time, hese a e a se o lib a ies implemen ed by Xilinx o abs ac he
ha dwa e om he use and ob ain some use ul se ices.
- XDMA: Xilinx Di ec Memo y Access engine, which p o ides di ec ans e s be ween
FPGA de ices and hos memo y wi hou CPU in ol emen [47].
- QDMA: I is an engine which s ands o ‘Queue Di ec Memo y Access’, he main
di e ences is ha i has a se o queues [48].
- IP: In ellec ual p ope y co e; a eusable logic uni which is used in a gi en design o add
a unc ionali y which is licensed om a endo .
- VPU: Vec o ial P ocessing Uni .
- ASIC: Applica ion-Speci ic In eg a ed Ci cui .
79/91

De eloping eleme y ools o Al eo accele a o ca ds
APPENDIX A)
Repo on eleme y capabili ies on Ve sal V80
Recen ly he BSC acqui ed wo V80 FPGAs om Xilinx. These a e he mos powe ul ones
we ha e, and he ques ion o whe he we could also add eleme y capabili y a ose. Tha is
why his appendix only wan s o epo he cha ac e is ics and how a u u e p og am on he
V80 would ha e o be implemen ed. I will also alk abou some di e ences and
ad an ages.
The V80 doubles he logic densi y, and he PCIe bandwid h compa ed o he U55C, almos
doubles he memo y bandwid h and quad uples he ne wo k bandwi h [49]. Addi ionally he
V80 in e nal s uc u e is di ided in o wo pa s, PL ‘p og ammable logic’ and PS
‘p ocessing subsys em’.
Compa ed o he Ul ascale de ices he g ea es ad an age is ha hey ha e managed o
sol e he I2C collision o add esses [50], bu i has some downsides, he i s being ha
e e y hing ela ed o he SMBus has o be implemen ed as an IP, ha cu en ly is a ailable
ajd s ill being de eloped by Xilinx, igu e 27 shows he schema ic o he sys em. This IP
can be ins an ia ed in o he use design. Xilinx al eady w o e a lib a y wi h he APIs o he
eleme y ea u e[51].
The e o e, one o he g ea ad an ages ha he e sion wi h an independen con olle had
is los , since i he use wan s eleme y, he will always ha e o make su e ha he has his
module included in a bi s eam and loaded. In my opinion i is no e y ideal.
80/91
Figu e 27: Image showcasing he V80 SMBus implemen a ion; sou ce Xilinx
De eloping eleme y ools o Al eo accele a o ca ds
APPENDIX B)
Table: Suppo ed senso s pe Al eo ca d.
Senso Name U200/U250 U280 U50 U55 U45N X3 UL3422 UL3524
1V2_VCCIO
2V5_VPP23
3V3_AUX ✓ ✓ ✓ ✓
3V3_PEX ✓ ✓ ✓ ✓ ✓ ✓ ✓
3V3PEX_I_IN ✓ ✓ ✓ ✓ ✓
12V_AUX ✓ ✓ ✓ ✓ ✓ ✓
12V_AUX1
12V_AUX_I_IN ✓ ✓ ✓ ✓ ✓ ✓
12V_PEX ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
12V_SW ✓ ✓
12VPEX_I_IN ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
AUX_3V3_I ✓ ✓
CAGE_TEMP0 ✓ ✓ ✓ ✓ ✓ ✓
CAGE_TEMP1 ✓ ✓ ✓ ✓ ✓
CAGE_TEMP2
CAGE_TEMP3
DDR4_VPP_BTM ✓ ✓
DDR4_VPP_TOP ✓ ✓ ✓ ✓
DDR4_VTT ✓
DIMM_TEMP0 ✓ ✓
DIMM_TEMP1 ✓ ✓
DIMM_TEMP2 ✓
DIMM_TEMP3 ✓
FAN_SPEED ✓ ✓
FAN_TEMP ✓ ✓
FPGA_TEMP ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
GTAVCC
GTVCC_AUX
HBM_1V2 ✓ ✓
HBM_1V2_I ✓
81/91
De eloping eleme y ools o Al eo accele a o ca ds
Senso Name U200/U250 U280 U50 U55 U45N X3 UL3422 UL3524
HBM_TEMP1 ✓ ✓ ✓
HBM_TEMP2 ✓ ✓ ✓
MGT0V9AVCC ✓ ✓ ✓ ✓ ✓ ✓
MGTAVCC ✓ ✓
MGTAVCC_I ✓ ✓
MGTAVTT ✓ ✓ ✓ ✓ ✓ ✓
MGTAVTT_I ✓
PEX_3V3_POWER ✓
PEX_12V_POWER ✓
POWER_GOOD ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
SE98_TEMP0 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
SE98_TEMP1 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
SE98_TEMP2 ✓
SYS_5V5 ✓ ✓ ✓ ✓
V12_IN_AUX0_I
V12_IN_AUX1_I
V12_IN_I
VCC0V85 ✓ ✓
VCC1V2_BTM ✓ ✓
VCC1V2_I
VCC1V2_TOP ✓ ✓ ✓ ✓
VCC1V5 ✓
VCC1V8 ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
VCC3V3 ✓ ✓ ✓ ✓ ✓ ✓
VCC_5V0
VCCAUX ✓ ✓
VCCAUX_PMC
VCCINT ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
VCCINT_I ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
VCCINT_IO ✓ ✓
VCCINT_IO_I ✓ ✓
VCCINT_POWER
VCCINT_TEMP ✓ ✓ ✓ ✓ ✓ ✓ ✓
VCCINT_VCU_0V9
82/91
De eloping eleme y ools o Al eo accele a o ca ds
Senso Name U200/U250 U280 U50 U55 U45N X3 UL3422 UL3524
VCCRAM
VCCSOC
VPP2V5 ✓ ✓
Table 25: Suppo ed senso s pe Al eo ca d; Sou ce Xilinx.
Indica es he senso is suppo ed. Emp y cells indica e ha he senso is no suppo ed.✓
APPENDIX C)
Table: Resou ce u iliza ion o CMS.
Pa In o ma ion Resou ce U iliza ion
De ice Package Speed-G ade LUTs FFs DSPs 36k BRAMs 18k BRAMs
xcu280 s h2892 2L 3702 3700 4 34 0
xcu50 s h2104 2L 3702 3700 4 34 0
xcu50 s h2104 2LV 3701 3700 4 34 0
xcu55c s h2892 2L 3697 3700 4 34 0
xcu200 sgd2104 2 3820 3883 4 34 0
xcu250 igd2104 2L 2818 3883 4 34 0
Table 26: Resou ce u iliza ion o ca d managemen solu ion; Sou ce Xilinx
83/91
De eloping eleme y ools o Al eo accele a o ca ds
APPENDIX D)
Table: Compila ion o known senso s, o se s, p esence mask,
and di ision ac o .
//RESET_REG (CMC/XMC/CMS) HAS A DIFFERENT OFFSET
#de ine RESET_REG 0x0
#de ine DISABLE_RESET 0x0
#de ine ENABLE_RESET 0x1
//THESE REGISTERS HAVE A DIFFERENT OFFSET
#de ine FIRMWARE_SIZE 0x20000
#de ine REG_MAP_1_SIZE 0x2000
#de ine REG_MAP_ID_REG_VALUE 0x74736574 // Magic alue
#de ine REG_MAP_ID_REG 0x000 // Magic egis e
#de ine FW_VERSION_REG 0x004
#de ine STATUS_REG 0x008
#de ine ERROR_REG 0x00C
#de ine PROFILE_NAME_REG 0x014
#de ine CONTROL_REG 0x018
//FPGA BOARD TYPE ( ead om PROFILE_NAME_REG)
#de ine U200_U250 0x55325858
#de ine U280 0x55323830
#de ine U50 0x55353041
#de ine U55 0x5535354E
#de ine U45N 0x55323641
#de ine X3 0x58334100
#de ine UL3422 0x55333432
#de ine UL3524 0x55333234
#de ine MASK_U200_U250 0b10000000
#de ine MASK_U280 0b01000000
#de ine MASK_U50 0b00100000
#de ine MASK_U55 0b00010000
#de ine MASK_U45N 0b00001000
#de ine MASK_X3 0b00000100
#de ine MASK_UL3422 0b00000010
#de ine MASK_UL3524 0b00000001
//O se s o MAX, AVG and INS egs
#de ine MAX 0x0 //Maximum alue egis e
#de ine AVG 0x4 //A e aged alue egis e
#de ine INS 0x8 //Ins an aneous alue egis e
//SENSORS NAME, Senso s ing o se , Mask, DIV_Fac o , Known Loca ion
#de ine REG_12V_PEX "12V_PEX (V)", {0x020, 0b11111111, 1000}
#de ine REG_3V3_PEX "3V3_PEX (V)", {0x02C, 0b11111110, 1000}
#de ine REG_3V3_AUX "3V3_AUX (V)", {0x038, 0b11000101, 1000}
#de ine REG_12V_AUX "12V_AUX (V)", {0x044, 0b11011011, 1000}
#de ine REG_DDR4_VPP_BTM "DDR4_VPP_BTM (V)", {0x050, 0b11000000, 1000} //DDR ol age ¿BANK 1?
#de ine REG_SYS_5V5 "SYS_5V5 (V)", {0x05C, 0b11110000, 1000}
#de ine REG_VCC1V2_TOP "VCC1V2_TOP (V)", {0x068, 0b11000011, 1000}
84/91

De eloping eleme y ools o Al eo accele a o ca ds
#de ine REG_VCC1V8 "VCC1V8 (V)", {0x074, 0b11111111, 1000}
#de ine REG_VCC0V85 "VCC0V85 (V)", {0x080, 0b11000000, 1000}
#de ine REG_DDR4_VPP_TOP "DDR4_VPP_TOP (V)", {0x08C, 0b11000011, 1000} //DDR ol age ¿BANK 2?
#de ine REG_MGT0V9AVCC "MGT0V9AVCC (V)", {0x098, 0b11110011, 1000}
#de ine REG_12V_SW "12V_SW (V)", {0x0A4, 0b11000000, 1000}
#de ine REG_MGTAVTT "MGTAVTT (V)", {0x0B0, 0b11110011, 1000}
#de ine REG_VCC1V2_BTM "VCC1V2_BTM (V)", {0x0BC, 0b11000000, 1000}
#de ine REG_12V_PEX_I_IN "12V_PEX_I_IN (A)", {0x0C8, 0b11111111, 1000}
#de ine REG_12V_AUX_I_IN "12V_AUX_I_IN (A)", {0x0D4, 0b11011011, 1000}
#de ine REG_VCCINT_V "VCCINT_V (V)", {0x0E0, 0b11111111, 1000} // LUTs + DSP ...
#de ine REG_VCCINT_I "VCCINT_I (A)", {0x0EC, 0b11111111, 1000}
#de ine REG_FPGA_TEMP "FPGA_TEMP (C)", {0x0F8, 0b11111111, 0x01}
#de ine REG_FAN_TEMP "FAN_TEMP (C)", {0x104, 0b11000000, 0x01} //FAN
#de ine REG_DIMM_TEMP0 "DIMM_TEMP0 (C)", {0x110, 0b11000000, 0x01}
#de ine REG_DIMM_TEMP1 "DIMM_TEMP1 (C)", {0x11C, 0b11000000, 0x01}
#de ine REG_DIMM_TEMP2 "DIMM_TEMP2 (C)", {0x128, 0b10000000, 0x01}
#de ine REG_DIMM_TEMP3 "DIMM_TEMP3 (C)", {0x134, 0b10000000, 0x01}
#de ine REG_FAN_SPEED "FAN_SPEED (RPM)", {0x164, 0b11000000, 0x01} //FAN
#de ine REG_SE98_TEMP0 "SE98_TEMP0 (C)", {0x140, 0b11111111, 0x01} //PCB TOP FRONT
#de ine REG_SE98_TEMP1 "SE98_TEMP1 (C)", {0x14C, 0b11111111, 0x01} //PCB TOP REAR
#de ine REG_SE98_TEMP2 "SE98_TEMP2 (C)", {0x158, 0b10000000, 0x01} //PCB BTM FRONT
#de ine REG_CAGE_TEMP0 "CAGE_TEMP0 (C)", {0x170, 0b11111100, 0x01} //QSFP 0
#de ine REG_CAGE_TEMP1 "CAGE_TEMP1 (C)", {0x17C, 0b11011100, 0x01} //QSFP 1
#de ine REG_CAGE_TEMP2 "CAGE_TEMP2 (C)", {0x188, 0b00000000, 0x01} //QSFP 2
#de ine REG_CAGE_TEMP3 "CAGE_TEMP3 (C)", {0x194, 0b00000000, 0x01} //QSFP 3
#de ine REG_HBM_TEMP "HBM_TEMP (C)", {0x260, 0b01110000, 0x01}
#de ine REG_VCC3V3 "VCC3V3 (V)", {0x26C, 0b00111111, 1000}
#de ine REG_3V3_PEX_I "3V3_PEX_I (A)", {0x278, 0b00111110, 1000}
#de ine REG_VCC0V85_I "VCC0V85_I (A)", {0x284, 0b00000000, 1000}
#de ine REG_HBM_1V2 "HBM_1V2 (V)", {0x290, 0b00110000, 1000} //HBM ol age
#de ine REG_VPP2V5 "VPP2V5 (V)", {0x29C, 0b00110000, 1000}
#de ine REG_VCCINT_BRAM "VCCINT_BRAM (V)", {0x2A8, 0b11111111, 1000}
#de ine REG_HBM_TEMP2 "HBM_TEMP2 (C)", {0x2B4, 0b01110000, 0x01} //HBM
#de ine REG_12V_AUX1 "12V_AUX1 (V)", {0x2C0, 0b00000000, 1000}
#de ine REG_VCCINT_TEMP "VCCINT_TEMP (C)", {0x2CC, 0b10111111, 0x01}
#de ine REG_3V3_AUX_I "3V3_AUX_I (A)", {0x2F0, 0b00000101, 1000}
#de ine HEARTBEAT 0x2FC // Sys em egis e s, p esen on all FPGAs
#de ine HOST_MSG_OFFSET 0x300 // Sys em egis e s, p esen on all FPGAs
#de ine HOST_MSG_ERROR 0x304 // Sys em egis e s, p esen on all FPGAs
#de ine HOST_MSG_HEADER 0x308 // Sys em egis e s, p esen on all FPGAs
#de ine STATUS2 0x30C // Sys em egis e s, p esen on all FPGAs
#de ine HEARTBEAT_ERR_CODE 0x310 // Sys em egis e s, p esen on all FPGAs
#de ine REG_VCC1V2_I "VCC1V2_I (A)", {0x314, 0b00000000, 1000} // HBM cu en
#de ine REG_V12_IN_I "V12_IN_I (A)", {0x320, 0b00000000, 1000}
#de ine REG_V12_IN_AUX0_I "V12_IN_AUX0_I (A)", {0x32C, 0b00000000, 1000}
#de ine REG_V12_IN_AUX1_I "V12_IN_AUX1_I (A)", {0x338, 0b00000000, 1000}
#de ine REG_VCCAUX "VCCAUX (V)", {0x344, 0b00000011, 1000}
#de ine REG_VCCAUX_PMC "VCCAUX_PMC (V)", {0x350, 0b00000000, 1000}
#de ine REG_VCCRAM "VCCRAM (V)", {0x35C, 0b00000000, 1000}
#de ine REG_VCCINT_VCU_0V9 "VCCINT_VCU_0V9(V)", {0x380, 0b00000000, 1000}
#de ine REG_1V2_VCCIO_MAX "1V2_VCCIO (V)", {0x38C, 0b00000000, 1000}
#de ine REG_GTAVCC_MAX "GTAVCC (V)", {0x398, 0b00000000, 1000}
#de ine REG_VCCSOC_MAX "VCCSOC (V)", {0x3B0, 0b00000000, 1000}
#de ine REG_VCC_5V0_MAX "VCC_5V0 (V)", {0x3BC, 0b00000000, 1000}
#de ine REG_2V5_VPP23_MAX "2V5_VPP23 (V)", {0x3C8, 0b00000000, 1000}
#de ine REG_GTVCC_AUX_MAX "GTVCC_AUX (V)", {0x3D4, 0b00000000, 1000}
85/91
De eloping eleme y ools o Al eo accele a o ca ds
#de ine REG_HBM_1V2_I_MAX "HBM_1V2_I (A)", {0x410, 0b00010000, 1000} //HBM cu en
#de ine REG_VCC1V5_MAX "VCC1V5 (V)", {0x41C, 0b00000001, 1000}
#de ine REG_MGTAVCC_MAX "MGTAVCC (V)", {0x428, 0b00000011, 1000}
#de ine REG_MGTAVTT_I_MAX "MGTAVTT_I (A)", {0x434, 0b00000010, 1000}
#de ine REG_MGTAVCC_I_MAX "MGTAVCC_I (A)", {0x440, 0b00000011, 1000}
#de ine CMC_HOST_MSG_REG 0x1000 // Sys em egis e s, p esen on all FPGAs
#de ine CMC_SOC_OPCODE_REG 0x1004 // Sys em egis e s, p esen on all FPGAs
#de ine CMC_SOC_LENGTH_REG 0x1008 // Sys em egis e s, p esen on all FPGAs
#de ine CMC_SOC_PAYLOAD_REG 0x100C // Sys em egis e s, p esen on all FPGAs
Table 27: Compila ion o known egis e s, hei names, o se s, p esence mask, di ision ac o s and
loca ion; Own compila ion, exce p om cus om C lib a y, in e ed om Xilinx sou ces.
86/91
17. Re e ences
[1]. GEEKSFORGEEKS. Xilinx FPGA A chi ec u e - Geeks o Geeks. Geeks o Geeks
[online]. 7 h Ma ch 2022 [ iewed 7 h Oc obe 2025]. A ailable om:
h ps://www.geeks o geeks.o g/xilinx- pga-a chi ec u e/
[2]. Field-P og ammable Ga e A ay (FPGA) Ma ke Analysis ill 2031. Ma ke Resea ch
Repo s, Business Consul ing | In-dep h Insigh s | TMR [online]. [Consul ed 22nd
Sep embe 2024]. A ailable a : h ps://www. anspa encyma ke esea ch.com/ ield-
p og ammable-ga e-a ay.h ml
[3]. MOOR INSIGHTS AND STRATEGY. Amazon's Xilinx FPGA Cloud: Why This May Be
A Signi ican Miles one. Fo bes [online]. 13 h Decembe 2016 [Consul ed 22nd
Sep embe 2024]. A ailable a :
h ps://www. o bes.com/si es/moo insigh s/2016/12/13/amazons-xilinx- pga-cloud-why-
his-may-be-a-signi ican -miles one/
[4]. Helena K upno a, Gab iele Saucie , FPGA-Based Emula ion: Indus ial and Cus om
P o o yping Solu ions [online]. Ins i u Na ional Poly echnique de G enoble/CSI, 1s o
Janua y 2002 [Consul ed 23 d Sep embe 2024]. ISBN : 978-3-540-67899-1. Doi:
10.1007/3-540-44614-1_8
[5]. BALKIND, Jona han e al. OpenPi on. En: ASPLOS '16: A chi ec u al Suppo o
P og amming Languages and Ope a ing Sys ems [online]. New Yo k, NY, USA: ACM,
2016 [Consul ed 23 d Sep embe 2024]. ISBN 9781450340915. Doi:
10.1145/2872362.2872414
[6]. SALAMI, Behzad, Osman S. UNSAL y Ad ian CRISTAL KESTELMAN. Comp ehensi e
E alua ion o Supply Vol age Unde scaling in FPGA on-Chip Memo ies. En: 2018 51s
Annual IEEE/ACM In e na ional Symposium on Mic oa chi ec u e (MICRO) [online].
IEEE, 2018 [Consul ed 23 d Sep embe 2024]. ISBN 9781538662403. Doi:
10.1109/mic o.2018.00064
[7]. In oduc ion — Sa elli e Con olle documen a ion. Gi Hub Pages [online]. [Consul ed
23 d Sep embe 2024]. A ailable a :
h ps://xilinx.gi hub.io/Al eo-Ca ds/mas e /managemen -speci ica ion/oob-in o.h ml
[8]. T ello - Wikipedia. [online]. 16 h Sep embe 2011 [Consul ed 23 d Sep embe 2024].
A ailable a : h ps://en.wikipedia.o g/wiki/T ello
[9]. Fe an Saba é. 2.1.2. Ges ión ágil de p oyec os. Campus Vi ual UPC [online].
[Consul ed 23 d Sep embe 2024]. No a ailable o e e yone, a :
h ps://a enea.upc.edu/plugin ile.php/6119723/mod_ olde /con en /0/Mòdul%202.1.b
%20-%20Ges ion%20agil%20de%20p oyec os.pd ? o cedownload=1
[10]. Gi Lab - Wikipedia. [online]. 25 h Oc obe 2014 [Consul ed 23 d Sep embe 2024].
A ailable a : h ps://en.wikipedia.o g/wiki/Gi Lab
[11]. Jooble [online]. [Consul ed 8 h Oc obe 2024]. A ailable a : h ps://es.jooble.o g
[12]. Amo ización de inmo ilizado. Sis emas y p og amas in o má icos.
Supe Con able.com - Con abilidad, Plan con able, aseso ía, aseso con able y
P og amas. [online]. [ iewed 9 h Oc obe 2024]. A ailable om:
h ps://www.supe con able.com/in o macion/impues o_sociedades/Amo izacion_de_in
mo ilizado.Sis emas_y_p og amas_.h ml
17. Re e ences
[13]. Cómo calcula la amo ización de equipos in o má icos. Holded [online]. [ iewed 9 h
Oc obe 2024]. A ailable om: h ps://www.holded.com/es/blog/amo izacion-de-
equipos-in o ma icos
[14]. ¿Cuán o cues a el kilo a io ho a de luz (kWh) en España? a i aluzho a.es [online].
[ iewed 9 h Oc obe 2024]. A ailable om: h ps:// a i aluzho a.es/in o/p ecio-kwh
[15]. O icinas en Ped albes. Idealis a [online]. [ iewed 9 h Oc obe 2024]. A ailable om:
h ps://www.idealis a.com/inmueble/102386101/
[16]. ¿Cuán o se paga al mes po los suminis os básicos? | Ras ea o . Ras ea o
[online]. 19 h Janua y 2022 [ iewed 9 h Oc obe 2024]. A ailable om:
h ps://www. as ea o .com/ a i as-ene gia/guias/ ac u a-mes-suminis os-basicos
[17]. Fac o de emisión de la ene gía eléc ica: el mix eléc ico. Cambio climá ico [online].
[ iewed 8 h Janua y 2025]. A ailable om:
h ps://can iclima ic.genca .ca /es/ac ua/ ac o s_demissio_associa s_a_lene gia/
[18]. CONTRIBUTORS TO WIKIMEDIA PROJECTS. PCI Exp ess - Wikipedia. Wikipedia,
he ee encyclopedia [online]. 2nd No embe 2002 [ iewed 4 h Janua y 2025]. A ailable
om: h ps://en.wikipedia.o g/wiki/PCI_Exp ess
[19]. In oduc ion — Sa elli e Con olle documen a ion. Si e no ound · Gi Hub Pages
[online]. [no da e] [ iewed 4nJanua y 2025]. A ailable om: h ps://xilinx.gi hub.io/Al eo-
Ca ds/mas e /managemen -speci ica ion/oob-in o.h ml
[20]. AWATI, Rahul. Wha is a baseboa d managemen con olle (BMC)? | De ini ion om
TechTa ge . Sea ch Ne wo king [online]. 11 h Janua y 2024 [ iewed 20 Decembe
2024]. A ailable om:
h ps://www. ech a ge .com/sea chne wo king/de ini ion/baseboa d-managemen -con olle
[21]. Al eo I2C Teleme y — Sa elli e Con olle documen a ion. Gi Hub Pages [online].
[ iewed 4 h Janua y 2025]. A ailable om:
h ps://xilinx.gi hub.io/Al eo-Ca ds/mas e /managemen -speci ica ion/al eo-I2C-
eleme y.h ml
[22]. PCI-SIG. PCI Local Bus Speci ica ion Re ision 3.0. [online]. 3 d Feb ua y 2004
[ iewed 21 No embe 2024]. A ailable om:
h ps://lekens eyn.nl/ iles/docs/PCI_SPEV_V3_0.pd
[23]. Sys em Managemen Bus (SMBus) Speci ica ion. [online]. 12 h May 2024 [ iewed 21
No embe 2024]. A ailable om: h ps://smbus.o g/specs/
[24]. SMBus Lib a y o MSP430G2xx3 De ices; Use 's Guide. SMBus Lib a y o
MSP430G2xx3 De ices [online]. 13 Feb ua y 2015 [ iewed 3 Janua y 2025]. A ailable
om: h ps://so wa e-dl. i.com/msp430/msp430_public_sw/mcu/msp430/SMBusLib/
la es /expo s/msp430_smbuslib_1_10_00_00/docs/MSP430G2xx3/
MSP430_SMBus_Use s_Guide-1_10_00_00.pd
[25]. MSP432P401R: SMBUS lib a y o MSP432 - MSP low-powe mic ocon olle o um -
MSP low-powe mic ocon olle s - TI E2E suppo o ums. TI E2E suppo o ums
[online]. 6 h Sep embe 2018 [ iewed 4 h Janua y 2025]. A ailable om: