An App oach o Dis ance Es ima ion wi h S e eo Vision
Using Add ess-E en -Rep esen a ion
M. Domínguez-Mo ales1, A. Jimenez-Fe nandez1, R. Paz1,
M.R. López-To es1, E. Ce ezuela-Escude o1, A. Lina es-Ba anco1,
G. Jimenez-Mo eno1, and A. Mo gado2
1 Robo ic and Technology o Compu e s Lab, Uni e si y o Se ille, Spain
2 Elec onic Technology Depa men , Uni e si y o Cadiz, Spain
[email p o ec ed]
Abs ac . Image p ocessing in digi al compu e sys ems usually conside s he
isual in o ma ion as a sequence o ames. These ames a e om came as ha
cap u e eali y o a sho pe iod o ime. They a e enewed and ansmi ed a a
a e o 25-30 ps ( ypical eal- ime scena io). Digi al ideo p ocessing has o
p ocess each ame in o de o ob ain a esul o de ec a ea u e. In s e eo
ision, exis ing algo i hms used o dis ance es ima ion use ames om wo
digi al came as and p ocess hem pixel by pixel o ob ain simila i ies and
di e ences om bo h ames; a e ha , depending on he scene and he
ea u es ex ac ed, an es ima e o he dis ance o he di e en objec s o he
scene is calcula ed. Spike-based p ocessing is a ela i ely new app oach ha
implemen s he p ocessing by manipula ing spikes one by one a he ime hey
a e ansmi ed, like a human b ain. The mammal ne ous sys em is able o
sol e much mo e complex p oblems, such as isual ecogni ion by
manipula ing neu on spikes. The spike-based philosophy o isual in o ma ion
p ocessing based on he neu o-inspi ed Add ess-E en -Rep esen a ion (AER) is
achie ing nowadays e y high pe o mances. In his wo k we p opose a wo-
DVS- e ina sys em, composed o o he elemen s in a chain, which allow us o
ob ain a dis ance es ima ion o he mo ing objec s in a close en i onmen . We
will analyze each elemen o his chain and p opose a Mul i Hold&Fi e
algo i hm ha ob ains he di e ences be ween bo h e inas.
Keywo ds: S e eo ision, dis ance calcula ion, add ess-e en - ep esen a ion,
spike, e ina, neu omo phic enginee ing, co-design, Hold&Fi e, FPGA, VHDL.
1 In oduc ion
In ecen yea s he e ha e been nume ous ad ances in he ield o ision and image
p ocessing, because hey can be applied o scien i ic and comme cial pu poses o
nume ous ields such as medicine, indus y o en e ainmen .
As we all know, he images a e wo dimensional while he daily scene is h ee
dimensional. The e o e, in he ansi ion om he scene ( eali y) o he image, wha
we call he hi d dimension is los . Nowadays, socie y has expe ienced a g ea
ad ance in hese aspec s: 2D ision has gi en way o 3D iewing. Indus y and
esea ch g oups ha e s a ed hei u he esea ch in his ield, ob aining some
mechanisms o 3D ep esen a ion using mo e han one came a [14]. T ying o
simula e he ision o human beings, esea che s ha e expe imen ed wi h wo-
came a-based sys ems inspi ed in human ision ([12][13]). Following his, a new
esea ch line has been de eloped, ocused on s e eoscopic ision [1]. In his b anch,
esea che s y o ob ain h ee-dimensional scenes using wo digi al came as. Thus,
we y o ge some in o ma ion ha could no be ob ained wi h a single came a, i.e.
he dis ance a which objec s a e.
By using digi al came as, esea che s ha e made a b eak h ough in his ield, going
up o c ea e sys ems able o achie e he abo e. Howe e , digi al sys ems ha e some
p oblems ha , e en oday, ha e no been sol ed. A logical and impo an esul in
s e eoscopic ision is he calcula ion o dis ances be ween he poin o iew and he
objec ha we a e ocused on. This p oblem is s ill comple ely open o esea ch and
he e a e lo s o esea ch g oups ocusing on i . The p oblems ela ed o his a e he
compu a ional cos needed o ob ain app op ia e esul s and he e o s ob ained a e
dis ance calcula ion. The e a e lo s o high-le el algo i hms used in digi al s e eo
ision ha sol e he dis ance calcula ion p oblem, bu his implies a compu e
in e en ion in o he p ocess and i is compu a ionally expensi e.
The equi ed compu a ional powe and speed make i di icul o de elop a eal-
ime au onomous sys em. Howe e , b ains pe o m powe ul and as ision
p ocessing using millions o small and slow cells wo king in pa allel in a o ally
di e en way. P ima e b ains a e s uc u ed in laye s o neu ons, whe e he neu ons o
a laye connec o a e y la ge numbe (~104) o neu ons in he ollowing one [2].
Mos imes he connec i i y includes pa hs be ween non-consecu i e laye s, and e en
eedback connec ions a e p esen .
Vision sensing and objec ecogni ion in b ains a e no p ocessed ame by ame;
hey a e p ocessed in a con inuous way, spike by spike, in he b ain-co ex. The isual
co ex is composed o a se o laye s [2], s a ing om he e ina. The p ocessing
s a s when he e ina cap u es he in o ma ion. In ecen yea s signi ican p og ess
has been made in he s udy o he p ocessing by he isual co ex. Many a i icial
sys ems ha implemen bio-inspi ed so wa e models use biological-like p ocessing
ha ou pe o m mo e con en ionally enginee ed machines ([3][4][5]). Howe e , hese
sys ems gene ally un a ex emely low speeds because he models a e implemen ed
as so wa e p og ams. Di ec ha dwa e implemen a ions o hese models a e equi ed
o ob ain eal- ime solu ions. A g owing numbe o esea ch g oups a ound he wo ld
a e implemen ing hese compu a ional p inciples on o eal- ime spiking ha dwa e
h ough he de elopmen and exploi a ion o he so-called AER (Add ess E en
Rep esen a ion) echnology.
AER was p oposed by he Mead lab in 1991 [8] o communica ing be ween
neu omo phic chips wi h spikes. E e y ime a cell on a sende de ice gene a es a
spike, i ansmi s a digi al wo d ep esen ing a code o add ess o ha pixel, using an
ex e nal in e -chip digi al bus ( he AER bus, as shown in igu e 1). In he ecei e he
spikes a e di ec ed o he pixels whose code o add ess was on he bus. Thus, cells
wi h he same add ess in he emi e and ecei e chips a e i ually connec ed by
s eams o spikes. A bi a ion ci cui s ensu e ha cells do no access he bus
simul aneously. Usually, AER ci cui s a e buil wi h sel - imed asynch onous logic.
Fig. 1. Ra e-coded AER in e -chip communica ion scheme
Se e al wo ks a e al eady p esen in he li e a u e ega ding spike-based isual
p ocessing il e s. Se ano e al. p esen ed a chip-p ocesso able o implemen image
con olu ion il e s based on spikes ha wo k a e y high pe o mance pa ame e s
(~3GOPS o 32x32 ke nel size) compa ed o adi ional digi al ame-based
con olu ion p ocesso s ( e e ences [6],[7],[5]).
The e is a communi y o AER p o ocol use s o bio-inspi ed applica ions in ision
and audi ion sys ems, as e idenced by he success in he las yea s o he AER g oup
a he Neu omo phic Enginee ing Wo kshop se ies. One o he goals o his
communi y is o build la ge mul i-chip and mul i-laye hie a chically s uc u ed
sys ems capable o pe o ming complica ed a ay da a p ocessing in eal ime. The
powe o hese sys ems can be used in compu e based sys ems unde co-p ocessing.
Fi s , we desc ibe elemen by elemen o ou p ocessing chain un il ob ain he
comple e sys em. Then we p opose an AER algo i hm, which can be de eloped in
AER sys ems using a FPGA o p ocess he in o ma ion; and ha is able o ob ain
di e ences om bo h e inas and calcula e a dis ance es ima ion o he objec in
mo emen . Finally, we p esen dis ance es ima ion esul s o he whole sys em, and
compa e i wi h he eal dis ance.
2 Sys em Desc ip ion
In his sec ion we will desc ibe in de ail he sys em used and each one o he
componen s ha o m ou sys em. We can see om igu e 2 a block diag am o he
whole sys em.
Vi ex‐5 FPGA
Boa d
USBAERmini2
Fig. 2. Comple e sys em wi h all he elemen s used
All he elemen s ha compose ou sys em a e hese ( om le o igh ): wo
DVS128 e inas [10], wo USB-AER, a Vi ex-5 FPGA boa d, an USBAERmini2 [9]
and a compu e o wa ch he esul s wi h jAER so wa e [11]. Nex , we will alk abou
he USB-AER and he Vi ex-5 FPGA boa d (Fig. 3).
USB-AER boa d was de eloped in ou lab du ing he CAVIAR p ojec , and i is
based on a Spa an II FPGA wi h wo megaby es o ex e nal RAM and a cygnal 8051
mic ocon olle .
To communica e wi h he ex e nal wo ld, i has wo pa allel AER po s (IDE
connec o ). One o hem is used as inpu , and he o he is he ou pu . In ou sys em we
ha e used wo USB-AER boa ds, one o each e ina. In hese boa ds we ha e
syn he ized in VHDL a il e called Backg ound-Ac i i y-Fil e , which allows us o
elimina e noise om he s eam o spikes p oduced by each e ina. This noise (o
spu ious) is due o he na u e o analog chips and since we canno do any hing o
a oid i in he e ina, we a e il e ing i . So, a he ou pu o he USB-AER we ha e
he in o ma ion il e ed and eady o be p ocessed.
Fig. 3. Le , USB-AER boa d; igh , Vi ex-5 FPGA boa d
The o he boa d used is a Xilinx Vi ex-5 boa d, de eloped by AVNET [17]. This
boa d is based on a Vi ex-5 FPGA and mainly has a big po composed o mo e han
eigh y GPIOs (Gene al Pu pose Inpu s/Ou pu s po s). Using his po , we ha e
connec ed an expansion/ es ing boa d, which has s anda d pins, and we ha e used
hem o connec wo AER inpu s and one ou pu .
The Vi ex-5 implemen s he whole p ocessing p og am, which wo ks wi h he
spikes coming om each e ina, p ocesses hem and ob ains he di e ences be ween
bo h e inas and he spikes a e o hese di e ences. The whole p og am block
diag am is shown in igu e 4. The sys em beha io and i s unc ionali y a e shown in
he ollowing sec ions.
3 Mul i Hold and Fi e Algo i hm
Once we ha e all he elemen s o ou chain, we can s a hinking abou he
algo i hms used. In his wo k we ecei e he a ic om bo h e inas and calcula e he
di e ences be ween hem. To do ha we ha e used he idea o he Hold&Fi e
building block [16] o ob ain he di e ence o wo signals. Wi h his block wo king
co ec ly, we ha e ex apola ed his o a 128x128 signals sys em (one o each pixel
o he e inas) and ob ained a Mul i Hold&Fi e sys em ha allows us o calcula e he
di e ences be ween bo h e inas’ spikes s eams.
The Hold&Fi e sub ac s wo pulses, ecei ed om wo di e en po s. When i
ecei es an e en , i wai s a sho ixed ime o ano he e en wi h he same add ess.
I i does no ecei e a second e en and he ixed ime is o e , i i es he pulse. I ,
o he wise, i ecei es ano he e en wi h he same add ess, hen, i he new e en
comes om he o he e ina, he e en is cancelled and no e en is ansmi ed, bu i
his second e en comes om he same e ina, he i s e en is dispa ched and his
second e en akes he ole o he i s e en and he sys em wai s again he sho
ixed ime. This Hold&Fi e ope a ion o sub ac ing o cancelling wo s eams o
spikes is desc ibed in dep h in he pape indica ed be o e.
To summa ize, i bo h e en s wi h he same add ess ha e he same pola iza ion and
come om di e en po s hey a e e ased and he Hold&Fi e block does no i e
any hing. In his case i i es only when bo h e en s ha e di e en pola iza ions. On
he o he hand, when bo h e en s come om he same po , hen he opposi e
happens: i only i es when bo h e en s ha e he same pola iza ion. The e a e o he
cases o comple e he u h able.
Ou algo i hm is based on he Hold&Fi e block, as i is said be o e, bu i has one
Hold&Fi e block o each pixel o he e ina. I ea s each pixel sepa a ely and
ob ains he di e ence be ween his pixel in he le e ina and he same pixel in he
igh e ina. A he end, we ha e he di e ence o bo h e inas in ou sys em ou pu .
The comple e VHDL sys em, which is comple ely implemen ed on he Vi ex-5
FPGA boa d, consis s o :
-One handshake block o each e ina: hese blocks es ablish he
communica ion p o ocol wi h he e ina. They a e based on a s a e machine
ha wai s o he e ina eques signal, ecei es he AER spike and e u ns he
acknowledge signal.
-Two FIFOs: o s o age a g ea amoun o spikes and lose none.
-One A bi a o : selec spikes om bo h FIFOs depending on he occupa ion o
hem.
-Mul i Hold&Fi e module: applies he algo i hm explained be o e o he
s eam o spikes ecei ed. To s o age he in o ma ion o he i s pulse o each
sequence, his block uses a dual-po RAM block.
-Dis ance es ima ion block: his block will be explained in he nex sec ion.
-Handshake ou : his block es ablishes he communica ion be ween ou
p ocessed in o ma ion and he ou wo ld.
Righ FIFOLe FIFO
Fig. 4. VHDL block diag am inside Vi ex-5 FPGA

4 Calcula ing Dis ances
In his sec ion we will alk abou he schema ic block ha was only named in he
p e iously sec ion: dis ance es ima ion module.
The algo i hm explained is a i s app oxima ion o he dis ance calcula ion using
spikes. Nowadays, he e is no algo i hm ha can sol e his p oblem using only spikes
in a eal- ime en i onmen . Tha is why we a e ying o ocus on his ield.
Exis ing algo i hms in digi al sys ems ex ac ea u es om bo h came as, p ocess
hem, and y o ma ch objec s om bo h came as ame by ame [15]. This p ocess
in ol es high compu a ional cos s and does no wo k in eal ime.
We wan o do his same hing in eal ime using AER. As a i s s ep o achie e
his goal we p opose an algo i hm based on he spikes a e o he Mul i Hold&Fi e
ou pu .
Theo e ically we ha e o explain he meaning o wha we a e going o ob ain and,
a e ha , we will show he p ac ical esul s and we will es i bo h ma ch.
In ou sys em, bo h e inas a e calib a ed wi h a ce ain angle o ob ain a ocus
dis ance o 1 m. To do ha , we ha e pu ou e inas in a base, sepa a ed 13’5 cm. We
ha e ob ained he sys em shown below (Fig. 5).
Fig. 5. Re inas si ua ions and igonome y
Applying Py hago as and igonome ic ules, we can ob ain:
h2 = 6’752 + 1002  h = 100’22755 cm
sin α = 100 / 100’22755 = 0’99773
a csin 0’99773 = 86’1387º
So, ou e inas a e calib a ed wi h an angle o 86’1387º o ob ain a ocal dis ance o
one me e . A e ha , we ha e measu ed he spikes a es a he ou pu o ou Mul i
Hold&Fi e algo i hm using a eco ded ideo abou one objec in mo emen . This
ideo has been played many imes a di e en dis ances om he e inas and we ha e
anno a ed he numbe o spikes i ed du ing his ideo. The esul ing spike s eam was
eco ded using jAER so wa e [11]. A e measu emen s, we ha e eco ded all he
esul s and ha e made a g aph wi h all o hem. This g aph indica es he numbe o
spikes e sus dis ance.
I is logical o hink ha , a he cen al ma ch poin o he ocal leng h o each e ina,
he Mul i Hold&Fi e ac s like a pe ec sub ac o and do no i e any spike a all (excep
o he e inas’ spu ious), so he spike a e a his poin is nea ze o. I we app oach he
e inas, he spike a e will be inc eased because he objec becomes bigge and each e ina
sees i om a di e en poin o iew, so he sub ac o will no ac so pe ec ly.
O he wise, i we pu he ideo u he om he e inas, spike a e will be sligh ly
inc eased due o he sub ac ion esul (di e en poin s o iew o he e inas), bu he
objec becomes smalle , so i compensa es his ailu e: as he u he he objec is, he
smalle i is and he e o e, he lowe he spike a e is; because less spikes a e i ed, bu
he sub ac ion ac s wo se and i es mo e spikes. Tha is why he second aspec is
balanced wi h he i s one.
So, in heo y, we will see a g aph whe e he numbe o spikes is g ea ly inc eased
nea he e inas and is sligh ly inc eased as we mo e away he objec om he ocal
collision poin . We can see he expe imen al esul s in he nex igu e. We ha e
s imula ed ou sys em using wo ypes o s imuli: an oscilla ing pendulum and a
ib a ing ule . Measu emen s we e aken om he s a ing poin o 10 cen ime e s o
150 cen ime e s. They we e aken e e y 10 cen ime e s.
Fig. 6. Spike a e e sus Dis ance. Up: Pendulum. Down: Rule .
In igu e 6 we can see he expe imen al esul s ob ained. I is in e es ing o ema k
ha , app oxima ely, a a dis ance o 100 cen ime e s ( ocal collision o bo h e inas)
we ob ained he lowes spike a e. I we see measu emen s aken close , i can be seen
ha spike a e inc eases, and a away om he ocal collision poin , he spike a e is
inc eased a li le. Finally, we ob ained he esul s we expec ed. In he i s case, wi h
he pendulum, i can be seen be e han he o he one; bu , wha is ue is ha , in a
sho dis ance, we can es ima e he dis ance o he objec in mo emen wi h a
quad a ic eg ession (line in blue shown in igu e 6).
I is e y in e es ing o see ha ou sys em beha es simila ly o human pe cep ion:
a p io i, wi hou knowing he size o he objec , we canno gi e exac dis ances, only
an app oxima ion. This app oxima ion depends on ou expe ience, bu he sys em
p oposed canno lea n. Howe e , we can measu e he dis ance quali a i ely, and
in e ac ing wi h an objec in a nea by en i onmen .
Fig. 7. Ha dwa e used o es ou sys em ( igh ) and s imulus used (le )
5 Conclusions
The exis ing di icul ies o calcula e dis ances in digi al sys ems ha e been shown.
Tha is why a biological app oach (Add ess-E en -Rep esen a ion) o wo k wi h has
been p esen ed. We ha e in oduced he Add ess-E en -Rep esen a ion no a ion o
communica e neu o-inspi ed chips as a new pa adigm in Neu omo phic Enginee ing.
We ha e e alua ed he ad an ages o his me hod and explained why we wo k wi h i .
In his wo k we p opose a i s app oxima ion o dis ance es ima ion using spikes
in a close en i onmen . To do ha , a s e eoscopic ision sys em wi h wo DVS e inas
has been used, wo king wi h VHDL o e a Vi ex-5 FPGA.
We ha e desc ibed and shown he whole sys em used and each one o i s elemen s
equi ed o ob ain he dis ance es ima ion. Wi h he ha dwa e sys em desc ibed, we
ha e explained he algo i hms used. The i s algo i hm uses a me hod o ob ain
di e ences be ween bo h e inas in eal ime and wi hou sampling. Wi h hese
di e ences he second algo i hm was explained, which wo ks wi h he spike a e
ob ained in ou sys em a e he di e ences calcula ion.
Wi h he esul s o hese wo algo i hms, we ha e been able o model he spike a e
e sus he dis ance o he objec . The simula ion esul s a e e y encou aging, because
we can see in he g aphs shown ha he e is a ela ionship be ween dis ance and he
spike a e a e ou p ocessing and ha his sys em wo ks qui e simila o human
pe cep ion.
Acknowledgemen s. Fi s , we wan o hank he con ibu ion o Tobias Delb uck,
whose e inas a e used in his wo k; as well as Raphael Be ne , whose USBAERmini2
was used oo. Wi hou hei con ibu ions his wo k could no ha e been done. A las ,
bu no leas , we wan o hank he Spanish go e nmen , which has suppo ed he
p ojec VULCANO (TEC2009-10639-C04-02).
Re e ences
1. Ba na d, S.T., Fischle , M.A.: Compu a ional S e eo. Jou nal ACM CSUR 14(4) (1982)
2. Shephe d, G.M.: The Synap ic O ganiza ion o he B ain, 3 d edn. Ox o d Uni e si y P ess
(1990)
3. Lee, J.: A Simple Speckle Smoo hing Algo i hm o Syn he ic Ape u e Rada Images.
Man and Cybe ne ics SMC-13 (1981)
4. C immins, T.: Geome ic Fil e o Speckle Reduc ion. Applied Op ics 24, 1438–1443
(1985)
5. Lina es-Ba anco, A., e al.: AER Con olu ion P ocesso s o FPGA. In: ISCASS (2010)
6. Cope, B., e al.: Implemen a ion o 2D Con olu ion on FPGA, GPU and CPU. Impe ial
College Repo (2006)
7. Cope, B., e al.: Ha e GPUs made FPGAs edundan in he ield o ideo p ocessing? In:
FPT (2005)
8. Si ilo i, M.: Wi ing Conside a ions in analog VLSI Sys ems wi h Applica ion o Field-
P og ammable Ne wo ks. Ph.D. Thesis, Cal ech (1991)
9. Be ne , R., Delb uck, T., Ci i -Balcells, A., Lina es-Ba anco, A.: A 5 Meps $100 USB2.0
Add ess-E en Moni o -Sequence In e ace. In: ISCAS, New O leans, pp. 2451–2454
(2007)
10. Lich s eine , P., Posh, C., Delb uck, T.: A 128×128 120dB 15 us Asynch onous Tempo al
Con as Vision Senso . IEEE Jou nal on Solid-S a e Ci cui s 43(2), 566–576 (2008)
11. jAER so wa e: h p://sou ce o ge.ne /apps/ ac/jae /wiki
12. Benosman, R., De a s, J.: Pano amic s e eo ision senso . In: In e na ional Con e ence on
Pa e n Recogni ion, ICPR (1998)
13. Benosman, R., e al.: Real ime omni-di ec ional s e eo ision and planes de ec ion. In:
Medi e anean Elec o echnical Con e ence, MELECON (1996)
14. Dou e , J., Benosman, R.: A mul i-came as 3D olume ic me hod o ou doo scenes: a
oad a ic moni o ing applica ion. In: In e na ional Con e ence on Pa e n Recogni ion,
ICPR (2004)
15. Dominguez-Mo ales, M., e al.: Image Ma ching Algo i hms using Add ess-E en -
Rep esen a ion. In: SIGMAP (2011)
16. Jimenez-Fe nandez, A., e al.: Building Blocks o Spike-based Signal P ocessing. In:
IEEE In e na ional Join Con e ence on Neu al Ne wo ks, IJCNN (2010)
17. AVNET Vi ex-5 FPGA boa d: h p://www.em.a ne .com/d c