RETHINK big: Eu opean Roadmap o Ha dwa e and
Ne wo king Op imiza ions o Big Da a
Gina Alio o, Paul Ca pen e ,
Ad ián C is al, Osman Unsal
Ba celona Supe compu ing Cen e
Ba celona, Spain
Ma cus Leich
Technische Uni e si ä Be lin
Be lin, Ge many
Ch is ophe A a e
Thales Resea ch & Technology
Palaiseau cedex F ance
Abs ac —This pape discusses he esul s o he RETHINK big
P ojec , a 2-yea Collabo a i e Suppo Ac ion unded by he
Eu opean Commission in o de o w i e he Eu opean Roadmap o
Ha dwa e and Ne wo king op imiza ions o Big Da a. This
indus y-d i en p ojec was led by he Ba celona Supe compu ing
Cen e (BSC), and i included la ge indus y pa ne s, SMEs and
academia. The oadmap iden i ies business oppo uni ies om 89
in-dep h in e iews wi h 70 Eu opean indus y s akeholde s in he
a ea o Big Da a and p edic s he u u e echnologies ha will
dis up he s a e o he a in Big Da a p ocessing in e ms o
ha dwa e and ne wo king op imiza ions. Mo eo e , i p esen s
coo dina ed echnology de elopmen ecommenda ions ( ocused on
op imiza ions in ne wo king and ha dwa e) ha would be in he
bes in e es o Eu opean Big Da a companies o unde ake in
conce as a ma e o compe i i e ad an age.
Keywo ds— oadmap, Big Da a, ne wo k, ha dwa e
I.
I
NTRODUCTION
Big Da a is a la ge and di e se ield [1] [2] [3] which is
cha ac e ized by la ge da a olumes, eloci ies and a ie ies,
wi h s anda d and adi ional me hods and a chi ec u es unable
o p ocess and s o e he da a wi hin a easonable ime ame.
P ocessing and s o age bo lenecks a e leading o he adop ion
o specialized Big Da a-op imized ha dwa e and ne wo king
echnologies, especially by he majo Big Da a playe s.
Fo ins ance, Mic oso has employed FPGA accele a ion o
“mee he needs o Da acen e wo kloads [ ha ] demand high
compu a ional capabili ies, lexibili y, powe e iciency, and
low cos ” [4], esul ing in a 29% educ ion in ail la ency o
i s p op ie a y Bing sea ch engine. In 2015, Mic oso ook
Big Da a ha dwa e op imiza ion a s ep u he by employing
FPGAs o “ a ious kinds o deep lea ning app oaches” [5]
and i has ins alled FPGAs in e e y Azu e cloud se e
wo ldwide, in o de o c ea e a cloud o a i icial in elligence
[6]. In June 2016, N idia announced plans o “ ap in o he big
da a business [which] will be a illion-dolla business o e he
nex ew yea s” [7]. The deep lea ning ha dwa e pla o m
ea u ing GPU-accele a ed aining combined wi h ASIC-
accele a ed gameplay was c i ical o Google DeepMind’s
AlphaGo p og am, which bea he Eu opean Go champion in
ou namen condi ions [8]. N idia is also pushing GPUs in he
DRIVE PX on-boa d compu e pla o m o compu e ision
and deep lea ning o Ad anced D i e Assis ance Sys ems
(ADAS) [9]. Facebook is ocused on he OpenCompu e open
ha dwa e ini ia i e [10] [11], and i is decoupling he so wa e
om he ha dwa e in he ne wo k swi ch business, while a he
same ime mo ing owa d disagg ega ed ne wo k a chi ec u e.
O e all, he e is a end owa ds no el ha dwa e o Big Da a-
ela ed op imiza ion a he la ge hype scale s, as hey a e o en
he i s o see he p oblem as well as he i s o sol e i .
This pape discusses he esul s o he RETHINK big P ojec ,
a 2-yea Collabo a i e Suppo Ac ion unded by he Eu opean
Commission in o de o w i e he Eu opean Roadmap o
Ha dwa e and Ne wo king op imiza ions o Big Da a.
Sec ion II desc ibes he RETHINK big P ojec and
me hodology and Sec ion III desc ibes how he RETHINK big
oadmap i s in o o he ela ed ini ia i es in he Eu opean
con ex . Sec ion IV discusses he oadmap con ibu ions
ela ing o ne wo k a chi ec u e, node a chi ec u e and low-
le el so wa e suppo , and Sec ion V summa izes he
oadmap’s key indings and ecommenda ions. Mo e de ail is
ound in he comple e RETHINK big oadmap [12]. Finally,
Sec ion VI concludes he pape .
II. RETHINK
BIG
P
ROJECT
The Eu opean Union, ecognizing he ends o in e na ional
Big Da a playe s and in an e o o ge he bes e u n on hei
in es men in companies in he a ea o Big Da a analy ics,
called o a Roadmap o p o ide a coo dina ed se o
echnology de elopmen ecommenda ions ( ocused on
op imiza ions in ne wo king and ha dwa e) ha would be in
he bes in e es o Eu opean Big Da a companies o unde ake
in conce as a ma e o compe i i e ad an age.
The p oduc ion o his oadmap has been unded as a p ojec ,
RETHINK big EC-GA No. 619788, which was led by he
Ba celona Supe compu ing Cen e and included pa ne s om
la ge indus y o SME o academia, as shown in Table 1. In his
oadmap, we iden i ied business oppo uni ies om Eu opean
indus y s akeholde s in he a ea o Big Da a and p edic ed
u u e echnologies ha will dis up he s a e o he a in Big
Da a p ocessing in e ms o ha dwa e and ne wo king
op imiza ions. We hen iden i ied a c i ical mass o hese
s akeholde s ha see a clea compe i i e ad an age enabled by
emb acing speci ic u u e echnologies. Finally, we de eloped
ecommenda ions o he Eu opean Commission ha will
ul ima ely acili a e imely Eu opean indus y access o hese
u u e echnologies.
© 20xx IEEE. Pe sonal use o his ma e ial is pe mi ed. Pe mission om IEEE mus be
ob ained o all o he uses, in any cu en o u u e media, including ep in ing/
epublishing his ma e ial o ad e ising o p omo ional pu poses, c ea ing new collec i e
wo ks, o esale o edis ibu ion o se e s o lis s, o euse o any copy igh ed
componen o his wo k in o he wo ks.
III. A
E
UROPEAN
R
OADMAP
The RETHINK big oadmap is one piece o he amewo k o
oadmaps (Figu e 1) being pu oge he o he Eu opean
Commission. While compiling he oadmap, i was de e mined
ha many Big Da a compu e p oblems we e ins ances o
compu e p oblems linked o he ending o Moo e’s law and
Denna d scaling, and beyond. As such, he oadmap scope is
limi ed o p oblems wi h a di ec impac on Big Da a Eu opean
indus y, and gene al compu e p oblems a e handled by he
Eu opean Technology Pla o m (ETP) Roadmaps (NEM,
NESSI, EPoSS and Pho onics21). The oadmap was de eloped
wi h conside a ion o High Pe o mance Compu ing (HPC),
since HPC also ou inely uses ex emely la ge da ase s [13].
Ou oadmap, howe e , is limi ed o ac i i ies wi h a clea
bene i o EU Big Da a Indus y, as HPC- ela ed aspec s a e
co e ed by ETP4HPC. The same is ue o so-called In e ne
o Things on i s way o becoming he In e ne o E e y hing.
The key o his nascen compu e a ea seems o be he da a
i sel , and we belie e ha he oppo uni ies p o ided by IoT
will be “enabled by and dependen on he emendous da a
collec ions and compu e capaci ies in he back-end machines
and da acen e s ha use such da a” [14]. As such, we main ain
ou ocus on hese back-end machines and da a cen e s,
lea ing all o he aspec s o be co e ed unde he Alliance o
In e ne o Things Ini ia i e and he egula ion and s anda ds
o communica ion a he ne wo k le el unde he wo k o he
5G-PPP (Public P i a e Pa ne ship). Finally, while no
discussion ega ding Big Da a is comple e wi hou discussion
o so wa e, he ea men o so wa e in he oadmap was
limi ed o suppo o ha dwa e and ne wo king op imiza ions
o Big Da a. We lea e he mo e de ailed discussion o he
Big Da a analy ics applica ions and he da a i sel o he
oadmap o he Big Da a Value Associa ion (BDVA).
IV. T
ECHNICAL
D
ISCUSSION
A. Ne wo k
The ne wo k is he mos pe asi e elemen o any mode n
echnology-based business. As such, he oadmap explains he
po en ial op imiza ions o Big Da a wi h inno a i e
echnologies applied o hese appliances - speci ically ou e s
and swi ches - as ela ed o i ualiza ion.
The analysis conside s ne wo k equi emen s o Big Da a
wo kloads, whe he inside a public cloud, a p i a e da a cen e
o e en in a u u e High Pe o mance / Big Da a embedded
sys em. We examine hese equi emen s om he pe spec i e
o he “da a ecei ing end”, meaning he ne wo k
communica ion inside o he Da a Cen e . As a esul , we
conside he nascen IoT senso s ma ke , he In e ne o mobile
in as uc u e challenges aced by he global elecom
ne wo ks, and he ac ual access o he da a by businesses
(including egula o y and p i acy conce ns) om his
pe spec i e. The “ne wo k” consis s o mul iple unc ions
embedded a di e en laye s ac oss many physical de ices
anging om he se e mo he boa d and in e aces o he op
o ack swi ches, ou e s and ope a o in as uc u e. Un il
now, he ne wo king ha dwa e li ecycle has been d i en by he
ques o inc easing bandwid h. Bu oday’s ma ke landscape
is apidly changing unde he p essu e o demand coming om
Big Da a, mobile phones and IoT combined equi emen s.
1) Ne wo k appliance ha dwa e: specialized o ba e me al
In eac ion o a compe i i e new landscape, hype scale s like
Google and Facebook a e acing o be he i s o achie e
s a e-o - he-a bandwid h (100GE). They a e also conside ing
mo ing o a new a chi ec u e based on ei he ba e me al
swi ches o specialized “pu pose-buil ” swi ches ha a e able
o be e cope wi h hei speci ic Big Da a wo kloads. Ba e
me al [16] e e s o commodi y (low-cos ) swi ches o which
cus ome s mus p ocu e, sepa a ely, a hi d-pa y ne wo k
ope a ing sys em (NOS) — like Big Swi ch Ligh OS,
Cumulus Linux OS, Pica8 PicOS — o build hei own like
Facebook did. Ne wo k ope a ing sys em suppo and se ices
mus be ob ained om he hi d-pa y NOS. Addi ionally,
he e a e Whi e Box swi ches ha a e commodi y-based ba e-
me al swi ches wi h a p eloaded ne wo k ope a ing sys em
om a hi d-pa y o adi ional ne wo king endo .
2) Ha dwa e o “so wa iza ion” o i ualiza ion This
end in ne wo k a chi ec u e, howe e , goes well beyond his
ba e me al ha dwa e. The p e iously men ioned
“so wa iza ion” begins wi h So wa e De ined
Ne wo king (SDN) which allows o he sepa a ion o con ol
and da a planes, espec i ely, ia so wa e ha can un on ba e
me al swi ches and o se e s wi h he addi ion o ne wo k
Figu e 1: ETP/PPP Collabo a ion ( om [15])
Pa ne Name Expe ise
Ba celona Supe compu ing Cen e (BSC) Compu e a chi ec u e and
sys em a chi ec u e
Technische Uni e si a Be lin (TUB) Da abase sys ems and
in o ma ion managemen
École Poly echnique Fédé ale de
Lausanne (EPFL)
Da abase sys ems and
applica ions
Cen um Voo Wiskunde en In o ma ica
(CWI)
Ha dwa e-conscious da abase
echnologies
Uni e si y o Manches e (UoM) Compu e a chi ec u e
Uni e sidad Poli écnica de Mad id (UPM) Da a mining and wa ehousing
ARM L d. (ARM) Silicon IP p o ide
In e ne Memo y Resea ch (IMR) Web-scale sou cing pla o m o
business in elligence
Thales SA (THALES) Si ua ion and decision analysis,
planning and op imiza ion
Table 1: RETHINK big P ojec Conso ium
ca ds [16]. This has he po en ial o b ing down he cos
signi ican ly and can g ea ly inc ease lexibili y. As explained
by Google [17], SDN is abou “a so wa e con ol plane ha
abs ac s and manages complexi y…and can make 10,000
swi ches look like one.” This a chi ec u e con inues wi h
Ne wo k Func ion Vi ualiza ion (NFV), which allows o he
implemen a ion o secu i y, i ewalls, ou ing schemes and
o he unc ions sepa a ely, again ia so wa e allowing o
inc eased con ol, lexibili y and scalabili y.
3) Decons uc ing he da a cen e (beyond 400 GbE)
High-end (beyond 400 Gigabi E he ne o GbE) ne wo k
appliances should be a ailable a e 2020 [18], bu by hen, he
en i e o ganiza ion inside he da a cen e may ha e changed.
The con inuous demand o lexibili y and lowe ope a ing
cos s may equi e adical ans o ma ions [19], wi h high
bandwid h a ailable a all key in e connec nodes leading o
composable ha dwa e – CPU, memo y, I/O and s o age ha is
pu chased à la ca e and suppo ed by so wa e ha can
econ igu e he ne wo k o speci ic wo kloads.
The bene i s a e clea ; by disagg ega ing he da a cen e , we
acili a e egula upg ades and po en ially elimina e he need
and cos o eplacing en i e se e s, cabling and econ igu ing
e e y hing. This ision will no be ealis ic wi hou new
so wa e capable o e icien ly managing he complexi y o
such a he e ogeneous pool o esou ces – each esou ce
po en ially loca ed anywhe e in a da a cen e . This could lead
o in e es ing oppo uni ies o SMEs, due o he end owa d
open ha dwa e and ne wo king and po en ially mo e he
ecosys em ou he hands o he big e ical chip make s.
B. A chi ec u e
Rega ding Big Da a compu e node ha dwa e, he e a e ou
impo an ends o conside , which a e b ie ly discussed
below and ou lined in de ail in he ull oadmap [12].
1) He e ogeneous compu ing The e is a no iceable end
away om gene al-pu pose a chi ec u es owa ds
he e ogeneous sys ems and accele a o s. This change is mainly
d i en by a slowdown in Moo e’s Law [20][21], which leads
o combina ions o mul iple kinds o p ocesso s and
accele a o s, GPUs, many-co es, FPGAs, and applica ion-
speci ic accele a o s in o he same de ice.
Despi e he po en ial bene i s o mo ing owa d he e ogeneous
sys ems, he ba ie s o en y a e subs an ial, in pa icula he
cos o pu chasing accele a o ha dwa e and so wa e
complexi y. The e o o un a Big Da a applica ion on
he e ogeneous sys ems equi es specialized skills and
knowledge o ha dwa e due o he complex ools and
p og amming models. E en a e in es ing in he app op ia e
human capi al, a sui able Re u n on In es men (ROI) is no
gua an eed, since such sys ems o en equi e hand
op imiza ion. On op o his, so wa e o he e ogeneous
sys ems is no po able and subjec o endo lock-in. In
addi ion, many open-sou ce communi ies a e philosophically
opposed o accep ing ha dwa e-speci ic so wa e pa ches [22],
so only open languages and APIs a e likely o be suppo ed
beyond speci ic d i e modules connec ed o using gene al-
pu pose and o en es ic i e in e aces. Finally, many new
echnologies ha e ye o be p o en in e ms o pe o mance
due o he lack o s anda d eal-wo ld benchma ks.
As a esul , o Eu opean so wa e endo s o adop
he e ogeneous sys ems, hey mus keep pace wi h successi e
candida e echnologies, which is no economically iable. This
is e iden in ou p ojec su eys, in which he majo i y o
Eu opean so wa e endo s epo ed ha hey had no ha dwa e
oadmap and p e e ed o wai un il new echnologies became
widely adop ed inexpensi e commodi ies.
2) Specializa ion and endo lock-in Gene al-pu pose
GPU (GPGPU) is a ma u ing echnology wi h a g owing a e
o adop ion, especially in he a ea o High Pe o mance
Compu ing (HPC). The GPGPU ma ke is cu en ly domina ed
by N idia (>95% o GPU-accele a ed sys ems in he TOP500
use N idia). GPGPUs ha e no ye achie ed wide-scale
pene a ion in o da a cen e s due he unce ain ROI. Small o
medium-sized da a cen e ope a o s a e unwilling o deploy
GPGPUs a la ge scale, as he powe consump ion is oo high
and u iliza ion oo low o jus i y he in es men . As is he case
o mo ing om a GPGPU-based he e ogeneous a chi ec u e
o an FPGA-based one, he e is conside able Non- ecu ing
Enginee ing (NRE) cos equi ed o a change in GPU endo .
3) In eg a ion wi hin he compu e node The e is a end
owa ds g ea e in eg a ion, o imp o e pe o mance and
educe ene gy consump ion. In es ing in a ma ke -speci ic
se e SoC is likely o be cos -p ohibi i e, howe e , unless he
design can be suppo ed by a e ical business o i can add ess
a la ge- olume ma ke (such as mobile). SoCs p o ide no
lexibili y: adding a new in e ace (e.g. 40 GbE) equi es a
cos ly edesign. In addi ion, an SoC mus be implemen ed
using a single silicon p ocess. Since he SoC includes he
pe o mance- and ene gy-c i ical p ocesso co es, he die mus
be ab ica ed using an expensi e leading edge silicon
echnology.
An al e na i e is Sys em-in-Package (SiP), as pionee ed by he
EC EUROSERVER p ojec [23]. Ha ing mul iple dies in he
same package p o ides lexibili y in ha as e e ol ing
echnologies may be sepa a ed om mo e slowly e ol ing
ones and hus eplaced wi hou a ec ing he es o he design.
In addi ion, ma ke -speci ic p oduc s can be buil om
commodi y compu e chiple (s) wi h specialized chiple (s) o
accele a o s and I/O in e aces wi hou designing an en i e
SoC. This lexibili y may gi e smalle companies a be e
oppo uni y o compe e due o igh e sys em in eg a ion.
4) Ve icaliza ion and hype scale s The inal majo end
is he inc easing dominance o a small numbe o e ically-
in eg a ed companies ha co-design all o pa s o he se e
s ack, o a ying deg ees, anging om he use - isible
so wa e, h ough (Big Da a) amewo ks, down o sys em
in eg a ion and po en ially e en chip design. These companies
ha e eno mous ma ke sha e and economies o scale, made
mo e so h ough e iciencies om hei e ically-in eg a ed
pe spec i e.
In Eu ope, howe e , he indus y is agmen ed, wi h a la ge
disconnec be ween echnology p o ide s and analy ics
companies. Almos all analy ics companies exp essed ha hey
ha e no ha dwa e oadmap, ake li le no ice o new ha dwa e
ends and a e only looking a exis ing commodi y ha dwa e.
Since Eu ope cu en ly has no ma ke sha e in se e compu e
CPUs, he e is limi ed oppo uni y o hese companies o
engage wi h he incumben supplie (s). This la ge disconnec
be ween echnology p o ide s and analy ics companies ca ies
a signi ican isk o being le behind by he la ge U.S.
companies.
C. So wa e Suppo
1) Big Da a p ocessing: que y languages o amewo ks In
he ea ly yea s o da a p ocessing, da a analys s knew which
answe s hey we e looking o and hei da ase s we e clean, so
que y languages we e he ool-o -choice. Que y languages
we e bound o a speci ic pu pose, o ins ance SQL o
ela ional que ying, SPARQL o que ying Resou ce
Desc ip ion F amewo k da a, and XPa h/XQue y o XML.
Mo eo e , he highe -le el so wa e could be easily w i en in
a way ha was independen o he ha dwa e.
O e he yea s, wi h he ad en o Big Da a, se e al changes in
he da a p ocessing landscape ha e ende ed que y languages
di icul o use. Fi s ly, he e has been a nea ly exponen ial
inc ease in he olume and a ie y o da a – da a ha is
inc easingly he e ogeneous, uns uc u ed, “di y” and
unp ocessed, and he e o e unsui able o he SQL abs ac ion.
In addi ion, he e has been a b oad shi om local o
dis ibu ed compu ing, which has equi ed he use o
dis ibu ed amewo ks, such as MapReduce, Spa k and Flink,
ha hide he complexi y o dis ibu ed ha dwa e. These lowe -
le el amewo ks ha e no been di ec ly compa ible wi h
exis ing que y languages. The consequence has been a shi
away om que y languages owa ds da a analysis lib a ies and
APIs a ge ing Machine Lea ning (ML) and Na u al Language
P ocessing (NLP).
2) Towa ds ha dwa e dependence The push om he
business in elligence communi y owa d Big Da a analy ics
has esul ed in widesp ead adop ion o dis ibu ed amewo ks.
A he same ime, open sou ce communi ies a e ying o
p o ide sui able ML code highe -le el lib a ies (MLlib) o
hese amewo ks. Addi ionally, and simila ly o he
widesp ead adop ion o SQL, la ge companies and
hype scale s ha e al eady s a ed o de elop hei own
solu ions o NLP and ML, as specialized highe -le el lib a ies
such as IBM’s Sys emT and Sys emML and Google’s Tenso -
Flow. The ad an age o hese companies is ha hese highe -
le el lib a ies can be un on nea ly any dis ibu ed amewo k.
They allow use s o speci y compu a ions in a way ha
ensu es ha he p og am can be execu ed in pa allel and he
amewo ks can hen un his code on a suppo ed se o
ha dwa e. E e y ime no el ha dwa e becomes a ailable, hese
amewo ks need o be adap ed o suppo he ha dwa e.
3) Too many abs ac ions A he lowe le els o he
so wa e s ack, he e a e a la ge numbe o p og amming
abs ac ions. He e ogeneous a chi ec u e app oaches come
wi h di e se p og amming in e aces ha usually need o be
add essed explici ly by Big Da a applica ion de elope s and
canno be easily in eg a ed in o exis ing wo k lows
au oma ically.
E en hough e e y p og amming concep ele an o Big
Da a equi es pa allelizing wo k ac oss a ailable ha dwa e,
and e en hough all ele an ha dwa e s uc u es suppo
pa allel p ocessing in some way, he speci ics a e
incompa ible, in ha he e a e no common abs ac ions ha
wo k o e e y hing. Abs ac ion o pa allelism a he da a
cen e le el means using MapReduce o ano he dis ibu ed
amewo k while abs ac ion o pa allelism (mul ico e) a
node le el equi es ye ano he laye such as OpenMP (mul i-
co e), he e ogeneous a chi ec u es (CPU, GPU, FPGA) equi e
an abs ac ion such as OpenCL, and so on.
On a la ge scale, MapReduce and i s successo s o ba ch and
s eam p ocessing implemen ed by he Apache Spa k and
Apache Flink p ojec s allow he pa allel execu ion o code on
sha ed-no hing clus e s. All o hese amewo ks speci y in a
decla a i e way he da a placemen and uni o pa alleliza ion,
while lea ing he ac ual p ocessing in each pa allel ins ance o
con en ional unc ional o p ocedu al code. The uni o
pa alleliza ion suppo ed he e is an ope a ing sys em h ead.
Any ha dwa e ha can execu e such a h ead, such as a CPU
co e, is po en ially a ailable o MapReduce, while e e y hing
else needs o be add essed explici ly by he p og amme .
A he ha dwa e le el on a single node, he e a e simila
misma ches: GPUs and CPUs a e ailo ed o p ocessing
mul iple da a i ems a once. Fo each piece o ha dwa e,
howe e di e en p og amming app oaches a e necessa y.
GPUs ely on SIMT, and CPUs p o ide SIMD unc ionali y o
achie e a simila goal a leas on a single CPU co e. Mul i-
co e ope a ions need o be implemen ed explici ly. Mode n
compile s and amewo ks like OpenMP a e capable o
abs ac ing away some o hese di e ences. The ke nel-based
p og amming abs ac ions o OpenCL ansla e e y well o all
o he p e iously men ioned pa alleliza ion concep s, howe e
e en OpenCL only ensu e co ec ness o he compu a ion on
each pla o m. I does no ensu e ha he compu a ion has
been op imized o execu ion on hese same pla o ms. The
si ua ion wi h mo e ad anced accele a o ha dwa e in
he e ogeneous a chi ec u es is simila . While FPGAs usually
suppo s anda d Ha dwa e Desc ip ion Languages (HDLs)
such as VHDL and Ve ilog, hese languages desc ibe he
unc ionali y equi ed a a e y low le el, in a way ha is
di icul o so wa e enginee s o unde s and. Finally,
specialized solu ions such as ASICs, DSPs and neu omo phic
ha dwa e equi e p og amming in e aces ha a e likewise
specialized and o he ime being canno be accessed
au oma ically om con en ional Big Da a amewo k code.
V. K
EY
F
INDINGS AND
R
ECOMMENDATIONS
A. Indus y Key Findings
Ou indings a e he esul o 89 in-dep h in e iews wi h key
s akeholde s om mo e han 70 dis inc Eu opean companies,
in addi ion o se e al mee ings wi h a b oad spec um o
indus y s akeholde s. These companies included majo and
up-and-coming playe s om elecommunica ions, ha dwa e
design and manu ac u e s as well as s ong ep esen a ion
om heal h, au omo i e, inancial and analy ics sec o s.
(1) Indus y is s ill ocused on how o ex ac alue om
hei da a, and hey a e s ill looking o he business model o
u n his alue in o p o i . Consequen ly, hey a e no
ocused on p ocessing (and s o age) bo lenecks, le alone on
he unde lying ha dwa e. The o e whelming esponse is ha
indus y does no see Big Da a ha dwa e p ocessing p oblems,
only Big Da a alue oppo uni ies. We belie e ha his is
la gely because he indus y is no ye ma u e enough o mos
companies o ully unde s and he kind o analy ics and Big
Da a p ocessing ha leads o undesi able bo lenecks.
(2) Eu opean companies a e no con inced o he Re u n
on In es men o using no el ha dwa e. In gene al, Eu opean
companies a e con en o use cu en ly a ailable ha dwa e as
long as hey con inue o ecei e he mos compe i i e p icing.
In some cases his was due o ex eme p ice-sensi i i y, bu o
many i was simply due o isk, exace ba ed by he lack o a
clean me ic o benchma k o side-by-side compa isons o
no el ha dwa e. O e all, he majo i y o he companies we e
no con inced ha he in es men in expensi e ha dwa e
coupled wi h he pe son mon hs equi ed o make hei
p oduc s wo k wi h new ha dwa e would be wo hwhile.
(3) Eu ope has limi ed oppo uni ies o ha dwa e /
so wa e a chi ec s o wo k oge he . The Eu opean
ecosys em is highly agmen ed while media and in e ne
gian s such as Google, Amazon, Facebook, Twi e and Apple
and o he s (also known as hype scale s) a e pu suing
e icaliza ion and designing hei own in as uc u es om
he g ound up. Eu opean companies ha a e no closely
conside ing ha dwa e and ne wo king echnologies as a means
o cu ing cos and o e ing be e u u e se ices un he isk
o alling u he behind. Hype scale s will con inue o ake
isks and ans o m hemsel es because hey a e he
“ecosys em”, mo ing e e ybody else in hei ail.
(4) Dominance o non-Eu opean companies in he se e
ma ke complica es he possibili y o new Eu opean en an s
in he a ea o specialized a chi ec u es. The as majo i y o
se e ha dwa e is based on In el p ocesso s. As a esul , In el
has a huge in luence o e he di ec ion o he indus y, and
hey a e wo king o u he expand his in luence, including
ia hei ecen acquisi ion o Al e a, in hopes o de eloping
powe ul new in eg a ed ha dwa e echnologies.
B. High-le el Ac ions Summa y
The RETHINK big oadmap [12] makes he ollowing wel e
conc e e ecommenda ions. Mo e de ail is a ailable in he
comple e oadmap documen .
1) P omo e adop ion o cu en and upcoming ne wo king
s anda ds Eu ope should accele a e he adop ion o he cu en
and upcoming s anda ds (10 and 40Gb E he ne ) based on
low-powe consump ion componen s p oposed by Eu opean
Companies and connec hese companies o end use s and
da a-cen e ope a o s so ha hey can demons a e hei alue
compa ed o he bigge playe s.
2) P epa e o he nex gene a ion o ha dwa e and ake
ad an age o he con e gence o High Pe o mance
Compu ing (HPC) and Big Da a in e es s In pa icula , Eu ope
mus ake ad an age o i s s eng hs in HPC and embedded
sys ems by encou aging dual-pu pose p oduc s ha b ing hese
di e en communi ies oge he (e.g. HPC/Big Da a ha dwa e
ha can be di e en ia ed in so wa e). This would allow new
companies o sell o a bigge ma ke and dec ease he isk
associa ed wi h de elopmen o new p oduc . The e is al eady
a clea con e gence be ween High Pe o mance Compu ing
and Big Da a. Mo eo e , la ge scien i ic expe imen s,
including he La ge Had on Collide and Squa e Kilome e
A ay in ol e p ocessing huge s eams o da a and a e
inc easingly adop ing Big Da a echnologies.
3) An icipa e he changes in Da a Cen e design o
400Gb E he ne ne wo ks (and beyond) This includes paying
special a en ion o ha dwa e de elopmen s such as pho onics-
on-silicon in eg a ion and no el Da a Cen e in e connec
designs equi ed a 400Gb ope a ion.
4) Reduce isk and cos o using accele a o s No el
specialized ha dwa e has he po en ial o inc ease compu ing
pe o mance and ene gy e iciency o app op ia e
applica ions by a ac o o en o mo e. In o de o achie e
hese gains, i is necessa y o e-enginee he so wa e, which
is expensi e and ime consuming, and i is di icul o p edic
he le el o gains ahead o ime. The use o FPGAs o
compu ing is mos p ominen in inancial and oil indus ies,
wi h only a small numbe o companies add essing hese
ma ke s. Eu ope mus lowe he ba ie o en y o
he e ogeneous sys ems and accele a o s; collabo a i e p ojec s
should b ing oge he end use s, applica ion p o ide s and
echnology p o ide s o demons a e signi ican (10x) inc ease
in h oughpu pe node on eal analy ics applica ions.
5) Encou age sys em co-design o new echnologies
Eu ope mus b ing oge he end use s, applica ion p o ide s,
sys em in eg a o s and echnology p o ide s o build he mos
e icien in eg a ed comple e ha dwa e—so wa e solu ions.
This equi es ha dwa e o mee he e ol ing needs o Big
Da a, in eg a ing mo e subsys ems in o he p ocesso de ice as
well as new non- ola ile memo ies and I/O in e aces.
6) Imp o e p og ammabili y o FPGAs Eu ope should also
und esea ch p ojec s in ol ing p o ide s o ools,
abs ac ions and high-le el p og amming languages o
FPGAs o o he accele a o s wi h he aim o demons a ing he
e ec i eness o his app oach using eal applica ions. Eu ope
should also encou age a new en an in o he FPGA indus y.
7) Pionee ma ke s o neu omo phic compu ing and
inc ease collabo a ion Fo neu omo phic compu ing and o he
dis up i e echnologies, he p incipal issue is he lack o a
ma ke ecosys em, wi h insu icien appe i e o isk and ew
Eu opean companies wi h he size and clou o in es in such a
isky di ec ion. Eu ope should encou age collabo a i e
esea ch p ojec s ha b ing oge he ac o s ac oss he whole
chain: end use s, applica ion p o ide s and echnology
p o ide s o demons a e eal alue om neu omo phic
compu ing in eal applica ions.

8) C ea e a sus ainable business en i onmen including
access o aining da a Eu ope should add ess access o
aining da a by encou aging he collec ion o open
anonymized aining da a and encou aging he sha ing o
anonymized aining da a inside EC- unded p ojec s. To
add ess he lack o in o ma ion sha ing, Eu ope should
encou age in e ac ion be ween ha dwa e p o ide s and Big
Da a companies using ne wo k-o -excellence o simila .
9) Es ablish s anda d benchma ks I is di icul o
Indus y o assess he bene i s o using no el ha dwa e. We
p opose es ablishing benchma ks o compa e cu en and no el
a chi ec u es using Big Da a applica ions.
10) Iden i y and build accele a ed building blocks We
p opose o iden i y o en- equi ed unc ional building blocks
in exis ing p ocessing amewo ks and o eplace hese blocks
wi h (pa ially) ha dwa e-accele a ed implemen a ions.
11) In es iga e use o he e ogeneous esou ces Wi h edge
compu ing and cloud compu ing en i onmen s calling o
he e ogeneous ha dwa e pla o ms, we p opose c ea ion o
dynamic scheduling and esou ce alloca ion s a egies.
12) Con inue o ask he ques ion – Do companies hink ha
ha dwa e and ne wo king op imiza ions o Big Da a can
sol e he majo i y o hei p oblems? As mo e companies lea n
how o ex ac alue om Big Da a and de e mine which
business models lead o p o i s, he numbe o se ice
o e ings and p oduc s based on Big Da a analy ics will g ow
sha ply. This g ow h will likely lead o an inc ease in
consume expec a ions wi h espec o hese Big Da a-d i en
p oduc s and se ices, and we expec companies o un in o
mo e and mo e undesi able pe o mance bo lenecks ha will
equi e op imized ha dwa e.
VI. CONCLUSIONS
This pape desc ibes he RETHINK big P ojec and i s
me hodology, and i gi es a b ie o e iew o he key indings
and ecommenda ions in he RETHINK big oadmap [12]. In
summa y, he RETHINK big P ojec has c ea ed an indus y-
led s a egic oadmap ha will maximize Eu opean indus y
compe i i eness o Big Da a ha dwa e o e he nex 10 yea s.
A
CKNOWLEDGMENT
This p ojec has ecei ed unding om he Eu opean Union’s
Se en h F amewo k P og amme o esea ch, echnological
de elopmen and demons a ion unde g an ag eemen n°
619788. I has also been suppo ed by he Spanish
Go e nmen (g an SEV2015-0493 o he Se e o Ochoa
P og am), by he Spanish Minis y o Science and Inno a ion
(con ac TIN2015-65316) and by Gene ali a de Ca alunya
(con ac s 2014-SGR-1051 and 2014-SGR-1272).
R
EFERENCES
[1] G. P ess, “12 Big Da a De ini ions: Wha 's You s?” 2014 [Online],
A ailable: h p://www. o bes.com/si es/gilp ess/2014/09/03/12-big-da a-
de ini ions-wha s-you s/#32c2a68821a9.
[2] G. Nunbe g, “Fo ge YOLO: Why 'Big Da a' Should Be The Wo d O
The Yea ,” 2012 [Online], A ailable:
h p://www.np .o g/2012/12/20/167702665/geo ey-nunbe gs-wo d-o -
he-yea -big-da a?li e=1.
[3] Helpdesk esponse, “Wha is Big Da a and wha classi ies as Big Da a?”
2016 [Online], A ailable: h p://s acko e low.com/ques ions/35560823/
wha -is-big-da a-wha -classi ies-as-big-da a.
[4] A. Pu nam, e al. “A Recon igu able Fab ic o Accele a ing La ge-Scale
Da acen e Se ices,” P oceedings o he 41s Annual In e na ional
Symposium on Compu e A chi ecu u e (ISCA 2014), 2014 [Online],
A ailable: h ps://www.mic oso .com/en-us/ esea ch/publica ion/a-
econ igu able- ab ic- o -accele a ing-la ge-scale-da acen e -se ices/.
[5] T. P icke Mo gan, “Mic oso Ex ends FPGA Reach om Bing o
Deep Lea ning,” 2015 [Online], A ailable:
h p://www.nex pla o m.com/2015/08/27/mic oso -ex ends- pga- each-
om-bing- o-deep-lea ning/.
[6] M. Feldman, “Mic oso Goes All in o FPGAs o Build Ou AI Cloud,”
2016 [Online], A ailable: h ps://www. op500.o g/news/mic oso -goes-
all-in- o - pgas- o-build-ou -cloud-based-ai/.
[7] A. Chan hada ong, “N idia Tou s GPU P ocessing as he Fu u e o Big
Da a, ” 2016 [Online], A ailable: h p://www.zdne .com/a icle/n idia-
ou s-gpu-p ocessing-as- he- u u e-o -big-da a/.
[8] A. Woodie, “GPU-Powe ed Deep Lea ning Eme ges o Ca y Big Da a
To ch Fo wa d,” 2016 [Online] , A ailable:
h p://www.da anami.com/2016/04/13/gpu-powe ed-deep-lea ning-
eme ges-ca y-big-da a- o ch- o wa d/.
[9] DRIVE PX, N idia websi e, h p://www.n idia.com/objec /d i e-
px.h ml (accessed 25 No 2015).
[10] N. Ahmad in in e iew wi h T. P icke Mo gan, “Facebook Ops
Di ec o On B eaking Open The Swi ch” 2015 [Online], A ailable:
h p://www.nex pla o m.com/2015/03/16/ acebook-ops-di ec o -on-
b eaking-open- he-swi ch/.
[11] The Open Compu e Ini ia i e, h p://www.opencompu e.o g/.
[12] RETHINK big D5.3 Roadmap [Online], A ailable:
h p://www. e hinkbig-p ojec .eu/ oadmap.
[13] Ea l Joseph, IDC, “HPC T ends, Big Da a and he Eme ging Ma ke o
High Pe o mance Da a Analysis (HPDA)” 2013 [Online], A ailable:
h p://www.lanl.go /con e ences/salishan/salishan2013/IDC%20Salisha
n%20P esen a ion%204.18.2013.pd ( o BDEC).
[14] H. Alkha ib, P. Fa aboschi, E. F ach enbe g, e . al. IEEE CS 2022
Repo , Feb 2014 [Online], A ailable:
h ps://www.compu e .o g/cms/Compu ingNow/2022Repo .pd .
[15] BDVA SRIA 2.0, Jan 2016 [Online], A ailable: h p://www.bd a.eu/
si es/de aul / iles/Eu opeanBigDa aValuePa ne ship_SRIA__ 2.pd .
[16] A. Kindness – Fo es e blog, “Wha is he di e ence be ween whi e box
ba e me al b anded ba e me al and oem ne wo k swi ches?” 2015
[Online], A ailable: h p://blogs. o es e .com/and e_kindness/15-02-23-
wha _is_ he_di e ence_be ween_whi e_box_ba e_me al_b anded_ba e
_me al_and_oem_ne wo k_swi ches.
[17] T. Ho - High Scalabili y blog, “How Google In en ed An Amazing
Da acen e Ne wo k Only They Could C ea e” 2015 [Online], A ailable:
h p://highscalabili y.com/blog/2015/8/10/how-google-in en ed-an-
amazing-da acen e -ne wo k-only- hey.h ml
[18] D. Coleman, “Op ical T ends in he Da a Cen e ” 2012 [Online],
A ailable: h ps://www.bicsi.o g/uploadedFiles/BICSI_Websi e/Global
_Communi y/P esen a ions/Ca ibbean/1.07%20Co ning.pd .
[19] T. P icke Mo gan, “Smashing he Se e o Pu i Back Toge he
Again” 2015 [Online], A ailable: h p://www.nex pla o m.com/2015/
11/04/smashing- he-se e - o-pu -i -back- oge he -be e /.
[20] A. Huang, “The Dea h o Moo e’s Law Will Spu Inno a ion,” IEEE
Spec um Ma ch 2015 [Online], A ailable: h p://spec um.ieee.o g/
semiconduc o s/design/ he-dea h-o -moo es-law-will-spu -inno a ion.
[21] M. Mi chell Wald op, “The chips a e down o Moo e’s law”, Na u e,
Vol. 520, Issue 7589. [Online], A ailable:
h p://www.na u e.com/news/ he-chips-a e-down- o -moo e-s-law-
1.19338.
[22] LWN.ne , “Linux and TCP o load engines,” 2005 [Online] A ailable:
h ps://lwn.ne /A icles/148697.
[23] M. Ma azakis e al. EUROSERVER: Sha e-Any hing Scale-Ou Mic o-
Se e Design. Design, Au oma ion and Tes in Eu ope (DATE), 2016