Mul i- ask Implemen a ion o Image Recons uc ion o
an AER Communica ion
C. Luján-Ma inez, A. Lina es-Ba anco, A. Jiménez-Fe nandez,
G. Jiménez-Mo eno, and A. Ci i -Balcells1
Depa amen o de A qui ec u a y Tecnología de Compu ado es.
Uni e sidad de Se illa.
A . Reina Me cedes s/n, 41012-Se illa, Spain
[email p o ec ed]
h p://www.a c.us.es
Abs ac . Add ess-E en -Rep esen a ion (AER) is a communica ion p o ocol
o ans e ing spikes be ween bio-inspi ed chips. Such sys ems may consis o
a hie a chical s uc u e wi h se e al chips ha ansmi spikes among hem in
eal ime, while pe o ming some p ocessing. The e exis se e al AER ools o
help in de eloping and es ing AER based sys ems. These ools equi e he use
o a compu e o allow he p ocessing o he e en in o ma ion, eaching e y
high bandwid h a he AER communica ion le el. We p opose o use an
embedded pla o m based on mul i- ask ope a ing sys em o allow bo h, he
AER communica ion and he AER p ocessing wi hou a lap op o a compu e .
We ha e connec ed and p og ammed a Gums ix compu e o p ocess Add ess-
E en in o ma ion and measu ed he pe o mance e e ed o he p e ious AER
ools solu ions. In his pape , we p esen and s udy he pe o mance o a new
philosophy o a ame-g abbe AER ool based on a mul i- ask en i onmen ,
composed by he In el XScale p ocesso go e ned by an embedded GNU/Linux
sys em.
1 In oduc ion
The Add ess-E en Rep esen a ion (AER) was p oposed by he Mead lab in 1991 [1]
o communica ing be ween neu omo phic chips wi h spikes (Fig. 1). Each ime a cell
on a sende de ice gene a es a spike, i communica es wi h he a ay pe iphe y and a
digi al wo d ep esen ing a code o add ess o ha pixel is placed on he ex e nal
in e -chip digi al bus ( he AER bus). Addi ional handshaking lines (Acknowledge and
Reques ) a e used o comple ing he asynch onous communica ion. In he ecei e
chip he spikes a e di ec ed o he pixels whose code o add ess was on he bus. In his
way, cells wi h he same add ess in he emi e and ecei e chips a e i ually
connec ed by s eams o spikes. These spikes can be used o communica e analog
1 This wo k was suppo ed by Spanish g an TEC2006-11730-C03-02 (SAMANTA 2). We would
also like o hank he NSF sponso ed Tellu ide Neu omo phic Enginee ing Wo kshop, whe e
his idea was bo n in a discussion g oup pa icipa ed by Daniel Fasnach , Giacomo Indi e i,
Alejand o Lina es-Ba anco and F ancisco Gomez-Rod íguez.
in o ma ion using a a e code, bu his is no a equi emen . Mo e ac i e cells access
he bus mo e equen ly han hose less ac i e. A bi a ion ci cui s usually ensu e ha
cells do no access he bus simul aneously. Usually, hese AER ci cui s a e buil using
sel - imed asynch onous logic by e.g. Boahen [2].
T ansmi ing he cell add esses allows pe o ming ex a ope a ions on he e en s
while hey a el om one chip o ano he . Fo example he ou pu o a silicon e ina
can be easily ansla ed, scaled, o o a ed by simple mapping ope a ions on he
emi ed add esses. These mapping can ei he be lookup-based (using, e.g. an
EEPROM) o algo i hmic. Fu he mo e, he e en s ansmi ed by one chip can be
ecei ed by many ecei e chips in pa allel, by p ope ly handling he asynch onous
communica ion p o ocol. The e is a g owing communi y o AER p o ocol use s o
bio-inspi ed applica ions in ision, audi ion sys ems and obo con ol, as
demons a ed by he success in he las yea s o he AER g oup a he Neu omo phic
Enginee ing Wo kshop se ies [3]. The goal o his communi y is o build la ge mul i-
chip and mul i-laye hie a chically s uc u ed sys ems capable o pe o ming
massi ely-pa allel da a-d i en p ocessing in eal ime [4].
Fig. 1. Ra e-coded AER in e -chip communica ion scheme
I is essen ial o ha e a se o ins umen s which make possible he igh
communica ion o hese AER chips and can be used o debugging pu poses. So his
se o ins umen s has o allow he ollowing asks: (a) o sequence: o p oduce
syn he ic AER e en s eams ha can be used as con olled inpu s while es ing and
adjus ing a chip o a se o hem, (b) o moni o : o obse e he ou pu o any elemen
in he sys em, and (c) o map: o al e he s eam p oduced by an emi e and send i o
a ecei e .
The e is a se o AER ools based on econ igu able ha dwa e (FPGA) ha can be
connec ed o a compu e . They achie e hese pu poses wi h a e y high AER
bandwid h, bu wi h he necessi y o a PC o E en p ocessing pu poses. A new
philosophy was bo n in he las Wo kshop on Neu omo phic Enginee ing (Tellu ide,
2006) o imp o e his, which is based in he use o an embedded GNU/Linux sys em
o e a ela i ely powe ul mic op ocesso .
In his pape , we s udy a o ally mic op ocesso based solu ion. The e o e, he e a e
ei he no econ igu able and speci ic ha dwa e o manage he AER a ic o o
p ocess he e en in o ma ion. We ha e de eloped so wa e solu ions o cap u e and
manage e en sequences and we ha e compa ed hem. They a e based on he
ope a ing sys em’s policies o manage p ocesses. We p esen h ee di e en solu ions:
one in e up based solu ion and wo polling ones (a p ocesses implemen a ion and a
h eads one).
2 Real-Time Model o Add ess-E en -Rep esen a ion Da a
Recep ion
AER was de eloped o mul iplexing in ime he spike esponse o a se o neu o-
inspi ed VLSI cells. These cells a e implemen ed oge he in o he same chip o
FPGA. Se e al housands o VLSI cells can be implemen ed in o he same chip using
a high speed digi al bus o implemen he AER communica ion, as he equency o
spikes o a neu on is in he o de o milliseconds. A se e al Me en s pe second a e
has o be suppo ed by he digi al bus as he only es ic ion. In he o he hand, he
AER scheme is asynch onous because he VLSI neu o-inspi ed cells a e no
synch onized. They send a spike o e en when hey need o send i . Then, he AER
pe iphe y is esponsible o send i in o AER o ma wi h he minimum possible delay.
This equi emen explains he use o an asynch onous p o ocol. This p o ocol is able
o send he e en as soon as possible. Fu he mo e, he communica ion is s onge
enough because an acknowledgmen is sen om he ecei e .
The e o e, he AER ecep o and emi e can wo k a di e en speeds, because o
he AER p o ocol speed adjus men o he slowe de ice. So he e en ecep ion is an
asynch onous ask and i is no possible o p esume he e en la ency o he emi e .
I is necessa y o sol e he ollowing ques ions o se he pa ame e s o es ic ions
o a Real-Time ask in he AER p o ocol:
-Wha is he ypical In e -Spike-In e al ime o he same add ess, bo h in he
emi e and in he ecei e ? This pa ame e will be he neu on ypical ac i i y.
-How many cells a e he e in he emi e and in he ecei e ? The AER channel
connec s hese wo chips. The ypical AER h oughpu in his channel can be de ined
by joining hese pa ame e s wi h he p e ious one.
-How many e en s can be los in he communica ion o a speci ic applica ion
wi hou de e io a ion in i s objec i e? Because many imes he sys em is no able o
p ocess a numbe o e en s (due o speed limi a ions), bu his doesn’ imply
necessa ily a di e en esul espec o he case whe e no e en is los .
- How much can be educed he h oughpu in he AER channel wi hou
de e io a ion in he AER p ocessing? The speed o he communica ion is de ined by
he slowe de ice. This should no necessa ily be ansla ed in o a wo se p ocessing.
The e o e, o de ine he eal- ime in o an AER sys em is necessa y o de ine he
limi s o he p e ious ques ions. The e is no limi a ion on how slow he ecep o
de ice can be. The e en ecep ion is gua an eed by he handshake p o ocol. Bu an
app oxima ion o eal- ime is desi able when de eloping some AER de ice.
3 The Pla o m
The pla o m is composed by an embedded p ocesso and a mul i- ask gene al
pu pose ope a ing sys em. The i s one is he In el XScale PXA255 400MHz. This 32
bi p ocesso o e s 32KB o cache memo y o da a and he same amoun o
ins uc ions, an MMU, 84 GPIO po s ha can be p og ammed o wo k as unc ion
uni s o manage se ial po s, I2C, PWM, LCD, USB clien 1.1, ... This p ocesso is
connec ed o 64MB o RAM and 16MB o Flash Memo y as he s o age medium o
he OS oo ile sys em. Ano he boa d is a ached o he p ocesso ’s one, p o iding
wi eless connec i i y o he pla o m (IEEE 802.11b). This ha dwa e is go e ned by a
mul i- ask gene al pu pose ope a ing sys em. I is based on a Linux ke nel 2.6, wi h
only a chi ec u e dependen pa ches applied o i s sou ces. The whole sys em, and
ob iously he c oss-compile ool chain, is compiled using he uClibc [7], a C lib a y
o de eloping embedded Linux sys ems, which suppo s sha ed lib a ies and
h eading. This le s he applica ion’s bina ies o be ligh e . No o he change has been
done o he sys em e e ed o a common GNU/Linux one. The use console and he
debug one a e se o a se ial po . Two se ices a e he o he p o ided use in e aces,
a emo e secu e shell se e and a HTTP one.
Fig. 2. How he sys em is connec ed. The USB-AER boa d will ansla e a bina y image in o an
e en s eam and send i o he mic op ocesso GPIO po s. The e en s ean will be used o
egene a e he bina y image. I could be iewed by ans o ming i in o a BMP ile and
connec ing o he HTTP se e . The RQ signal will be used o measu e he EER using an
oscilloscope.
As shown in Fig. 2, an USB-AER boa d will play he ole o he AER emi e . I
will be esponsible o ans o m a bina y ep esen a ion o a ame in o he
co esponding e en s and o send hem. These will be sen o he pla o m ia he
AER bus, whose pins will be di ec ly connec ed o he p ocesso ’s GPIO po s.
Fig. 3. The USB-AER boa d [5] (on he le image). The Gums ix-connex 400MHz (le ),
Wi is ix ( igh ) and Waysmall (up) boa ds om www.gums ix.com (on he igh image).
An oscilloscope p obe will be clipped o he Reques signal pin and i will be used
o measu e he e en ecep ion a e, E RR. The usual mechanisms o compu e he
execu ion ime o a ask and i s du a ion, ei he p o ided by he ha dwa e o he
ope a ing sys em, would in e e e on he ob ained alue by inc emen ing i .
So he need o including his kind o ins uc ions is a oided by using he
oscilloscope. The E RR will be he equency o he Reques signal, which will be
USB-AER
(emi e )
Mul i- ask embedded
pla o m
PC
(de elopmen hos ) OSCILLOSCOPE
AER
HTTPD
R
Q
USB
cable
calcula ed by i . The ime ha he p ocess is eady o un and wai ing o ake he
p ocesso o i s execu ion is also conside ed in his alue, which makes i a eal
measu e o he E RR
4 AER P ocessing in a Mul i- ask En i onmen Example:
ae 2image
The e is a p e ious ha dwa e implemen a ion o a ame-g abbe used as a moni o
[5]. The idea is o con inuously collec e en s om he AER bus o a de ined pe iod
o ime, called in eg a ion ime. The ame will be cons uc ed by compu ing each
e en when i is ecei ed and i will be sen o a compu e ia USB p o ocol when he
in eg a ion ime has expi ed. The e en s ha will be ecei ed du ing he ame
ans e o he compu e a e los . This p ocess will be es a ed again when he ans e
will ha e inished.
We p esen a new way o cons uc ing a ame in his sec ion using he
mic op ocesso GPIO po s. We ha e de eloped se e al implemen a ions and ha e
compa ed hem o he as es case: oggling he Acknowledge signal when he
Reques one is se o low, which will le o e alua e he pe o mance o his mul i- ask
en i onmen when p ocessing Add ess-E en -Rep esen a ion da a.
The job o cons uc ing a ame om AER e en s can be di ided in o wo asks.
One is o acqui e he e en s om he AER bus and he o he is o cons uc he ame
om hose acqui ed e en s. This concep ual scheme can be quickly ansla e o a
“double-bu e ing” implemen a ion, hinking in he as es way o do i in a mul i- ask
en i onmen . The e a e wo app oaches o his p og amming echnique imple-
men a ion in his scena io. One is he use o one p ocess o each ask and he o he is
he use o one p ocess o bo h, implemen ing hese asks as h eads. We will call
hem “p ocesses implemen a ion” and “ h eads implemen a ion”, espec i ely.
Bo h implemen a ions ha e he same philosophy. E en s will be con inuously
collec ed and pu in o a bu e . When his bu e is ull, a signal will be sen o he
o he p ocess o h ead and he new ecei ed e en s will be pu in o he o he bu e .
So, his is a wo s -case linea ime algo i hm ha will le o con inuously ecei e
e en s. The o he p ocess o h ead will be gene a ing he ame in o memo y om a
bu e o wai ing o ecei e a signal. The e o e, i is also a wo s -case linea ime
algo i hm which le o con inuously gene a e he ame o wai un il a bu e is eady
o i s ea men . When a signal is ecei ed, he e e ence o he app op ia e bu e
will be changed depending on he ecei ed signal. This is a wo s -case cons an ime
algo i hm o he signal handle ha le he double bu e ing bu e -change o be
implemen ed.
We will use IPC Sha ed Memo y me hod in he i s implemen a ion and global
a iables in he second one, which makes bo h implemen a ions equi alen om he
access o memo y poin o iew. “Polling” will be used o implemen he e en
acquisi ion o bo h. So, hey a e also equi alen in his o he sense. The e o e, he
di e ence be ween he implemen a ions akes place in how hey a e a ec ed by he
ope a ing sys em schedule , which will be discussed la e .

Finally, ano he p ocess will be used o debugging pu poses, independen ly o he
double bu e ing implemen a ion. This p ocess will be wai ing o ecei e a signal ha
will be pe iodically sen by he ope a ing sys em. Then, i will wake up and pu he
ame in memo y in o a BMP ile. This las could be iewed by connec ing o he
HTTP se e on he pla o m. Also, hey will be used o es he implemen a ions
unde si ua ions wi h o he p ocesses unning.
4.1 The Schedule In luence
The e a e wo main pa ame e s which de ine he schedule in luence, he scheduling
policy and he equency o he ime in e up s.
The scheduling policy de e mines how he p ocesses will be execu ed in a mul i-
ask ope a ing sys em. The Linux ke nel 2.6 e sion p esen s se e al ones. These can
be chosen wi hou ecompiling he sou ces. The ke nel o e s sys em calls o le he
p ocesses o choose he scheduling policy ha will ule hei execu ion. A dynamic
p io i y based on execu ion ime scheduling policy, a eal- ime ixed p io i y FIFO
one and a eal- ime ixed p io i y ound obin one a e o e ed by he ke nel. The i s
one is he common policy on UNIX sys ems. Basically, a base p io i y is ini ially
assigned o he p ocess. I s new p io i y is calcula ed by he schedule when his las is
execu ed using he execu ion ime associa ed o he p ocess. This p io i y will
de e mine when he p ocess will be execu ed again. The o he wo scheduling policies
di e om each o he in how p ocesses wi h he same p io i y a e eo ganized o ake
he mic op ocesso again, using a FIFO c i e ion o a ound obin one, espec i ely. A
p ocess whose execu ion is managed by one o hese wo policies is, ob iously, no
in luenced by he i s o all. E en mo e, p e e ence will be gi en, o cou se, o a
p ocess in hese scheduling si ua ions han he managed by he i s policy ones.
The eal- ime scheduling policies y o ensu e a sho esponse ime o a uled by
hem unning p ocess. Also, no lowe -p io i y p ocesses should block i s execu ion
bu his si ua ion ac ually happens. The ke nel code is no always assumed o be p e-
emp i e2. So a sys em call om a lowe -p io i y p ocess may block he execu ion o
highe -p io i y one un il i has inished. The e o e, he suppo o eal- ime
applica ions is weak al hough he p ocesses esponse ime is imp o ed e e ed o he
common scheduling policy. E e y p ocess in a Linux sys em is no mally uled by he
i s one. The e o e, a p ocess unning con inuously canno be se o be uled by one
o he o e ed eal- ime policies wi hou making he whole es o he sys em
un esponsi e. We will use he common scheduling policy o ou implemen a ions in
his s udy in o de o e alua e he pe o mance o hem in a gene al pu pose mul i-
ask en i onmen .
The equency alue o he ime in e up s is he o he pa ame e ha mainly
in luences on a mul i- ask ope a ing sys em pe o mance. The pe iod o ime assigned
o a p ocess o i s execu ion in he mic op ocesso is gene ally called quan um,
whose alue is de ined by he equency o he ime in e up s one and is
dec emen ed each ime in e up . The e o e, a mo e ine-g ained esolu ion sys em
can be achie ed by aising i . On he o he hand, an ex a ins uc ion o e head has o
be paid due o a highe numbe o ime in e up s. This implies con ex swi ches om
p ocess o in e up handle and om his las o he i s , he handle execu ion, and
2 I has o be compiled wi h his op ion and i is only suppo ed in 2.6 e sions.
possible cache and TLB3 pollu ion, which may esul in an impo e ishmen o he
sys em pe o mance. This alue is se be o e he Linux ke nel compila ion p ocess.
The de aul one is 100Hz o he ARM a chi ec u e. We ha e s udy he pe o mance
o bo h implemen a ions unde his de aul alue and a 1000Hz one. The esul s will
be p esen ed in he nex sec ion.
4.2 Resul s
We p esen in his sec ion he di e en alues o he e en ecep ion a e, E RR,
ob ained wi h he wo implemen a ions e e ed be o e (using p ocesses o h eads).
Fig. 4 shows he E RR o e he ime o each case. I is mainly s able a i s highes
alue, which is b ie ly dec eased due o he p ocesso assignmen o o he p ocesses.
This educ ion e ol es some imes o a ha sh alue when he equency o he ime
in e up s is se o he de aul , 100Hz. This undesi able alue is 200keps o he
p ocesses implemen a ion and 259keps o he h eads one. The p ocesses based
implemen a ion p esen s an E RR oscilla ing om 530keps o 450keps o a
equency o ime in e up s o 100Hz and om 500keps o 430keps o 1000Hz,
being he i s alues he s able ones. The in e als o he h eads implemen a ion a e
770keps o 620keps and 770keps o 660keps, espec i ely, being again he s able
ones. These s able alues a e he unique ones when no o he use p ocess is unning,
and so we called hem he s able alues.
Fig. 4. E en Recep ion Ra e (E RR) in Kilo-e en s pe second (keps) o e he ime o all he
cases o s udy
We ha e also implemen ed an applica ion which only pe o ms he esponse o he
handshake p o ocol. No e en di ec ion is calcula ed om he mic op ocesso GPIO
po s o e en s o age is done. The E RR is 1’3Meps in his case, which implies a
ime be ween e en s o 760ns. We ha e also measu ed he ime be ween e en s when
3 T ansla e Lookaside Bu e , a cache used o imp o e he speed o i ual add ess ansla ion
con aining pa s o he ope a ing sys em’s page able.
he e is ei he he e en di ec ion calcula ion and i s s o age in o a bu e du ing he
handshake p o ocol. The esul is 1’16µs, which should be he ideal case. We ha e
also se he implemen a ions o be uled by he ound obin p io i y ixed eal- ime
scheduling policy, achie ing an E RR o 840keps. The e o e, he ime be ween
e en s is 1’19µs. This alue is nea he ideal one bu as we explained be o e, and so
expec ed, he sys em was un esponsi e o o he asks. The h eads implemen a ion
p esen s 770keps, which implies ha i pe o ms he e en acquisi ion and he e en
ea men wi h a ime be ween e en s o 1’3µs, app oxima ely. The e o e, i o e s a
mul i- ask en i onmen use ul o o he simul aneous asks wi h an 11% de ia ion
om he ideal.
Al hough he p ocesso o e s a mechanism o de ec any le el change a any o i s
GPIO po s, gene a ing ha dwa e in e up when i occu s, he minimum pulse wid h
du a ion o gua an ee his de ec ion is 1µs [8]. The e o e, he ime be ween e en s is, a
leas , 2’4µs, because he ime due o in e up s handle s o e head, con ex changes …
a e no conside ed and so his op ion was uled ou .
5 Conclusions
We ha e p esen ed a new philosophy o cons uc ing a ame-g abbe using a mul i-
ask en i onmen di ec ly connec ed o he AER bus, achie ing an E RR o 770Keps.
This alue is sus ained wi h o he p ocesses unning in he mic op ocesso , le ing he
execu ion o o he in e es ing and help ul ones like ne wo k connec i i y, a mo e
complex ea men , e c, o debugging pu poses al hough his a e is no as as as
hose go en by he ha dwa e implemen a ions. A u u e s udy o scheduling policies
combina ion a un ime based on he applica ion s a e ( ecei ing e en s o wai ing o
hem), could inc ease he pe o mance o he sys em wi h no deg ada ion on he
mul i- ask en i onmen esponse.
Re e ences
1. Si ilo i, M.: Wi ing Conside a ions in analog VLSI Sys ems wi h Applica ion o Field-
P og ammable Ne wo ks, Ph.D. Thesis, Cali o nia Ins i u e o Technology, Pasadena CA
(1991)
2. Boahen, K.A.: Communica ing Neu onal Ensembles be ween Neu omo phic Chips. In:
Neu omo phic Sys ems, Kluwe Academic Publishe s, Bos on (1998)
3. Cohen, A., e al.: Repo o he Na ional Science Founda ion: Wo kshop on Neu omo phic
Enginee ing, Tellu ide, Colo ado, USA (June-July 2004) www.ini.unizh.ch/ ellu ide
4. Mahowald, M.: VLSI Analogs o Neu onal Visual P ocessing: A Syn hesis o Fo m and
Func ion. PhD. Thesis, Cali o nia Ins i u e o Technology Pasadena, Cali o nia (1992)
5. Paz, R., Gomez-Rod iguez, F., Rod iguez, M.A., Lina es-Ba anco, A., Jimenez, G., Ci i ,
A.: Tes In as uc u e o Add ess-E en -Rep esen a ion Communica ions. In: IWANN
2005. LNCS, ol. 3512, pp. 518–526. Sp inge , Heidelbe g (2005)
6. Lina es-Ba anco, A.: Es udio y e aluación de in e aces pa a la conexión de sis emas
neu omó icos median e Add ess-E en -Rep esen a ion. Ph.D. Thesis, Uni e si y o Se ille,
Spain (2003)
7. h p://uclibc.o g
8. In el PXA255 P ocesso De elope ’s Manual, In el P ess (2004)