OpenNAS: Open Sou ce Neu omo phic Audi o y Senso HDL code
gene a o o FPGA implemen a ions
D. Gu ie ez-Galan
a,
⇑
, J.P. Dominguez-Mo ales
a
, A. Jimenez-Fe nandez
a,b
, A. Lina es-Ba anco
a,b
,
G. Jimenez-Mo eno
a,b
a
Robo ics and Technology o Compu e s Lab, Uni e sidad de Se illa, Se ille, Spain
b
SCORE Lab - I3US, Uni e sidad de Se illa, Se ille, Spain
Keywo ds:
Neu omo phic Audi o y Senso
AER
VHDL
Open ha dwa e
abs ac
OpenNAS is an open-sou ce ool o au oma ically gene a ing he sou ce iles o c ea e a Neu omo phic Audi o y
Senso (NAS) VHDL p ojec o FPGA. OpenNAS guides he use wi h a iendly in e ace ha allows con igu ing he
NAS’ pa ame e s using a i e-s ep wiza d o code gene a ion. OpenNAS p o ides suppo o se e al audio inpu
in e aces (AC’97 audio codec, I2S-ADC and PDM mic ophones), di e en p ocessing a chi ec u es (cascade and
pa allel), and neu omo phic ou pu in e aces (pa allel AER, SpiNNake ). A e NAS gene a ion, use s ha e e e y hing
eady o building, simula ing, and syn hesizing he VHDL p ojec o a a ge FPGA. OpenNAS is ully modula , which
allows p o iding suppo o new ea u es in an easy way.
1. In oduc ion
A i icial cochleae a e senso s inspi ed by he way ha he bio-
logical inne ea wo ks. Se e al models (ha dwa e and so wa e)
can be ound in he li e a u e, wi h a simila a chi ec u e. They
all ha e an inpu s age, whe e he inpu sound s imuli a e col-
lec ed; a p ocessing s age consis ing o a se o band-pass il e s,
commonly wi h a cascaded opology, and inally he ou pu s age,
whe e he il e s’ ou pu s a e ob ained and p e-p ocessed o sub-
sequen s eps. Ha dwa e implemen a ions can be di ided in o wo
g oups: econ igu able and non- econ igu able a chi ec u es. Fo
he la e , he implemen a ion is ixed (analog [1] o digi al silicon
[2]) and only ew pa ame e s can be uned.
The main con ibu ion o his pape is he in oduc ion o he
i s so wa e ool o au oma ically gene a ing a ull-cus om Neu-
omo phic Audi o y Senso (NAS) o FPGA, which was p esen ed
in [3]. One o he main ad an ages o de eloping a NAS in an FPGA
is i s lexibili y and e sa ili y. Howe e , hese e en ually become
a se ious disad an age, as he complexi y o design building and
pa ame e uning inc eases he di icul y o designing NAS. Wi h
he aim o dis ibu ing NAS along he neu omo phic esea ch com-
muni y, we p esen his ool, known as OpenNAS. OpenNAS builds
a design by ins an ia ing i s di e en blocks and au oma ically
compu ing all he pa ame e s (including di e en il e gains,
cu -o equencies, FIFO memo ies and in e aces), guiding use s
s ep by s ep along he p ocess.
NAS is cu en ly used o se e al neu omo phic applica ions
de eloped by di e en in e na ional esea ch g oups, demons a -
ing he u ili y o OpenNAS o se ing up and in eg a ion in cus om
p ojec s. This includes pa e n ecogni ion in audio samples [4] and
sound sou ce localiza ion [5].
2. NAS a chi ec u e and design low
The NAS a chi ec u e is mainly composed o h ee blocks, p e-
sen ed in Fig. 1. Fi s ly, he inpu audio signal is acqui ed by an
audio on -end ha con e s audio in o ma ion o pulse-
equency modula ion (PFM) spike-coded signals. Nex , he spikes
gene a ed by he i s block exci e a spike-based il e bank (SFB),
which decomposes he in o ma ion in di e en equency bands
using a cascade- ashion o a pa allel- ashion p ocessing a chi ec-
u e (use selec able). Finally, he spikes ob ained om he ou pu
o he SFB a e collec ed by a neu omo phic ou pu in e ace o
p opaga e he NAS in o ma ion o any ollowing p ocessing laye .
The cu en OpenNAS e sion suppo s a pa allel AER moni o as
ou pu , as well as he SpiNNake in e ace, which is used o con-
nec he NAS o a SpiNNake boa d [6].
Use s should ollow he design low p esen ed in Fig. 2,
adjus ing he se ings o he h ee blocks o con igu e a new
⇑
Co esponding au ho .
E-mail add ess: [email p o ec ed] (D. Gu ie ez-Galan).
NAS. A e his s ep, NAS’ pa ame e s will be compu ed and he
sou ce code will be gene a ed. Finally, using a de elopmen
sui e o FPGAs, such as Xilinx’s Vi ado o Al e a’s Qua us,
he NAS can be syn hesized and deployed in o an FPGA. A
de ailed explana ion o he concep s p esen ed is a ailable in
he OpenNAS wiki
1
.
3. So wa e a chi ec u e
To ep esen NAS’ componen s, we used a se o classes, whe e
each class con ains all he pa ame e s and HDL in o ma ion o a
speci ic componen . The class hie a chy is p esen ed in Fig. 3.
The main NAS class is ‘‘OpenNasA chi ec u e”, which con ains an
ins ance o common pa ame e s (OpenNASComponen s) and one
a ibu e o each o he h ee NAS componen s: AudioInpu ,
AudioP ocesingA chi ec u e and SpikesOu pu In e ace (Fig. 3
mid). These a e abs ac classes ha inhe i om he ‘‘HDLGene a-
ble” abs ac class (Fig. 3 op), which con ains he me hods o gen-
e a ing he HDL code. Finally, speci ic componen classes inhe i
om AudioInpu , AudioP ocesingA chi ec u e and SpikesOu -
pu In e ace, implemen ing each o he componen ea u es. Using
his inhe i ance ee, a NAS is ully modeled and s uc u ed, eady
o u u e expansions wi h new NAS ea u es.
To guide he use , OpenNAS implemen s a wiza d-based g aph-
ical use in e ace (GUI) w i en in Windows P esen a ion Founda-
ion (WPF). The las s ep is HDL gene a ion, which pe o ms he
ollowing s eps: (1) Each componen w i es i s HDL dependency
iles and op en i y o an ou pu des ina ion olde . (2)
OpenNasA chi ec u e c ea es he op NAS HDL ile. (3) Sequen-
ially, each componen w i es I/O signals in he NAS op ile. (4)
In e ace signals be ween componen s a e added o he NAS op
ile. (5) Sequen ially, each componen w i es i s op componen
a chi ec u e. (6) To he NAS op ile, each componen adds an in o-
ca ion o i s ins ance, and hese a e connec ed o each o he using
in e ace signals. (7) A empla e o cons ain iles is gene a ed
Fig. 1. Block diag am o he comple e a chi ec u e o a binau al NAS.
Fig. 2. Design low diag am o ull NAS syn hesis.
Fig. 3. OpenNAS class diag am.
Table 1
OpenNAS pe o mance compa ison be ween wo di e en p ocesso s.
NAS SFB
E o
AMD Ryzen 3900X
(3.80 GHz)
In el Co e i7 6700HQ
(2.60 GHz)
32ch.
S e eo
0.48% 63.48 ms 139.14 ms
64ch.
Mono
0.51% 64.36 ms 156.81 ms
128ch.
Mono
0.53% 84.62 ms 247.4 ms
256ch. 0.55% 127.28 ms 313.38 ms
1
h ps://gi hub.com/RTC- esea ch-g oup/OpenNAS/wiki
4. OpenNAS execu ion esul s
To measu e he uning e o and execu ion ime, di e en NAS
we e gene a ed wi h di e en CPUs. The esul s a e p esen ed in
Table 1. The gene a ed NAS ha e an a e age e o o a ound 0.5%,
which is lowe han he e o epo ed in [7] (a ound 1.573% o
a 64-channel NAS). The ime ha he so wa e akes o gene a e
a NAS was measu ed, including in e nal pa ame e uning, wi h
wo di e en p ocesso s: AMD Ryzen 3900X and In el Co e i7
6700HQ. Fo all he di e en cases, he gene a ion ime is below
a ew hund ed milliseconds, inc easing wi h he numbe o NAS’
channels.
5. Conclusions
The main con ibu ion o his wo k is a no el IP co e gene a o
ool ha allows esea che s o easily design hei own NAS o
speci ic applica ions. Thanks o i s iendly in e ace and he au o-
ma ic compu a ion o i s pa ame e s by only ollowing 5 s eps in a
GUI, he NAS a chi ec u e can be eely dis ibu ed o he neu o-
mo phic communi y, eady o low-cos FPGAs
2
. OpenNAS
3
was
designed ollowing a hie a chical class s uc u e o ep esen NAS’
componen s, wi h i s HDL desc ip ion and pa ame e s, allowing
de elope s o inc ease OpenNAS componen s and unc ionali ies
easily.
Decla a ion o Compe ing In e es
The au ho s decla e ha hey ha e no known compe ing inan-
cial in e es s o pe sonal ela ionships ha could ha e appea ed
o in luence he wo k epo ed in his pape .
Acknowledgemen s
This wo k was suppo ed by he Spanish g an (wi h suppo
om he Eu opean Regional De elopmen Fund) COFNET
(TEC2016-77785-P). Daniel Gu ie ez-Galan was suppo ed by a
Fo mación de Pe sonal In es igado Schola ship om he Spanish
Minis y o Educa ion, Cul u e and Spo .
Re e ences
[1] M. Yang e al., A 0.5 V 55
l
W64 2 channel binau al silicon cochlea o e en -
d i en s e eo-audio sensing, IEEE J. Solid-S a e Ci cui s 51 (11) (2016) 2554–
2569.
[2] Y. Xu e al., A FPGA implemen a ion o he CAR-FAC cochlea model, F on .
Neu osci. 12 (2018) 198.
[3] A. Jimenez-Fe nandez e al., A binau al neu omo phic audi o y senso o pga:
a spike signal p ocessing app oach, IEEE T ans. Neu al Ne w. Lea ning Sys . 28
(4) (2017) 804–818.
[4] J.P. Dominguez-Mo ales e al., Deep neu al ne wo ks o he ecogni ion and
classi ica ion o hea mu mu s using neu omo phic audi o y senso s, IEEE
T ans. Biomed. Ci cui s Sys . 12 (1) (2017) 24–34.
[5] T. Schoepe e al., Neu omo phic senso y in eg a ion o combining sound sou ce
localiza ion and collision a oidance, in: 2019 IEEE Biomedical Ci cui s and
Sys ems Con e ence (BioCAS), IEEE, 2019, pp. 1–4.
[6] E. Paink as e al., SpiNNake : a 1-W 18-co e sys em-on-chip o massi ely-
pa allel neu al ne wo k simula ion, IEEE J. Solid-S a e Ci cui s 48 (8) (2013)
1943–1953.
[7] A. Jimenez-Fe nandez e al., Building blocks o spikes signals p ocessing, in: In .
Join Con . on Neu al Ne wo ks, IEEE, 2010, pp. 1–8.
Daniel Gu ie ez-Galan ecei ed he B.S. deg ee in
compu e enginee ing in 2014 and he M.S. deg ee in
compu e enginee ing and ne wo ks in 2016, bo h om
he Uni e si y o Se ille, Se illa, Spain. Since Sep embe
2017, he has been a Ph.D. s uden in he Depa men o
Compu e A chi ec u e and Technology, a Uni e si y o
Se ille. His esea ch in e es s include embedded sys-
ems p og amming, digi al design, FPGA, spiking neu al
ne wo ks in embedded sys ems o audio p ocessing,
neu omo phic audi o y senso s and neu omo phic
obo s.
Juan P. Dominguez-Mo ales was bo n in Se illa
(Se illa, Spain) in 1992. He ecei ed he B.S. deg ee in
compu e enginee ing, he M.S. deg ee in compu e
enginee ing and ne wo ks, and he Ph.D. deg ee in
compu e enginee ing (specializing in neu omo phic
audio p ocessing and spiking neu al ne wo ks) om he
Uni e si y o Se ille, in 2014, 2015 and 2018, espec-
i ely. F om Oc obe 2015 o Decembe 2018, he was a
PhD s uden in he A chi ec u e and Technology o
Compu e s Depa men o he Uni e si y o Se ille wi h
a esea ch g an om he Spanish Minis y o Educa ion
and Science. Since Janua y 2019, he has been wo king as
Assis an P o esso in he same depa men . His esea ch in e es s include medical
image analysis, con olu ional neu al ne wo ks, compu e -aided diagnosis sys ems,
neu omo phic enginee ing, spiking neu al ne wo ks, neu omo phic senso s and
audio p ocessing. In 2016 he became a membe o he Eu opean Neu al Ne wo k
Socie y, he has been a membe o IEEE o ou yea s.
Angel F. Jimenez-Fe nandez ecei ed he B.S. Deg ee in
Compu e Enginee ing in 2005, he M.S. Deg ee in
Indus ial Compu e Science in 2007 and he Ph.D. in
Neu omo phic Enginee ing in 2010 om he Uni e si y
o Se ille, Se illa, Spain. Since Oc obe 2007, he has been
an Assis an P o esso o Compu e A chi ec u e and
Technology a he Uni e si y o Se ille. In Ap il 2011, he
was p omo ed o Associa e P o esso . His esea ch
in e es s include neu omo phic enginee ing applied o
obo ics, eal- ime spikes signal p ocessing, neu omo -
phic senso s, ield p og ammable ga e a ay (FPGA)
digi al design, embedded sys ems de elopmen , high-
speed se ial communica ions, and sma senso s ne wo king.
Alejand o Lina es-Ba anco (M’06) ecei ed he B.S.
deg ee in compu e enginee ing, he M.S. deg ee in
indus ial compu e science, and he Ph.D. deg ee in
compu e science (specializing in compu e in e aces
o neu omo phic sys ems) om he Uni e si y o
Se illa, Se illa, Spain, in 1998, 2002, and 2003, espec-
i ely. F om Janua y 1998 o June 1998, he was Second
Lieu enan in he Spanish Ai Fo ce wo king as Sys em
Adminis a o and So wa e De elope . F om 1998 o
2000, he was a Membe o he Technical S a a he
Se illa Mic oelec onics Ins i u e (IMSE-CNM-CSIC).
F om 2000 o 2001, he was a De elopmen Enginee
wi h he Resea ch and De elopmen Depa men , a ABENGOA Company, Se illa,
wo king on VHDL-based ield p og ammable ga e a ay (FPGA) sys ems o he
INSONET Eu opean p ojec on powe line communica ions. F om 2001 o 2006, he
was an Assis an P o esso a he A chi ec u e and Technology o Compu e s
Depa men o he Uni e si y o Se illa. In 2009, he was p omo ed o Associa e
P o esso (ci il-se an ). Since 2013 he is he sec e a y o he depa men . In 2014
he has been isi ing p o esso wi h he UZH. His Robo ics and Compu e Technology
Labo a o y has de eloped a se o AER- ools o debugging and connec ing AER
sys ems s a ing unde he EU p ojec CAVIAR. His esea ch in e es s include VLSI
o FPGA digi al design, neu o-inspi ed chip- o-chip and chip- o-compu e in e -
aces, spike based p ocessing, e en -based senso in o ma ion il e ing and ea u e
ex ac ion, mo o con ol, ision and audio o FPGAs, wi eless senso ne wo ks and
embedded applica ions based on mic ocon olle s, bus emula ion, and compu e
a chi ec u es. P o . Lina es-Ba anco has been a Re iew Commi ee Membe o
ISCAS since 2009. In 2010, he became a membe and o ice o he Technical
Commi ee on Neu al Sys ems and Applica ions (NSATC) o he IEEE Ci cui s and
Sys ems Socie y. In 2012 he became membe o he Senso y Sys ems Technical
Commi ee (SSTC).
2
h ps://gi hub.com/RTC- esea ch-g oup/OpenNAS#suppo ed-ides-simula o s-an
d-de ices
3
h ps://gi hub.com/RTC- esea ch-g oup/OpenNAS#license
wi h all NAS I/O signals. (8) Finally, a NAS summa y is w i en as a
XML ile.
mo phic sys ems. His esea ch in e es s include neu al ne wo ks, ision p ocessing
sys ems, embedded sys ems, compu e in e aces, and compu e a chi ec u es.
Gab iel Jimenez-Mo eno ecei ed he M.S. Deg ee in
Physics (elec onics) and he Ph.D. Deg ee om he
Uni e si y o Se ille (Se ille, Spain), in 1987 and 1992,
espec i ely. A e wo king wi h Alca el, he was g an ed
a Fellowship om he Spanish Science and Technology
Commission (CICYT). Cu en ly, he is an Associa e P o-
esso o compu e a chi ec u e a he Uni e si y o
Se ille. F om 1996 un il 1998, he was Vice-Dean o he
E.T.S. Ingenie ia In o ma ica, Uni e si y o Se ille. He
pa icipa ed in he c ea ion o he Depa men o Com-
pu e A chi ec u e (also a he Uni e si y o Se ille) and
since 2013 has been i s Di ec o . He is he au ho o
a ious pape s and esea ch epo s on
obo ics, ehabili a ion echnology, and
compu e a chi ec u e. He has di ec ed h ee na ional esea ch p ojec s on neu o-