A programmable VLSI filter architecture for application in real-time vision processing systems

A P og ammable VLSI Fil e A chi ec u e o Applica ion in Real-Time Vision P ocessing Sys ems Te esa Se ano-Go a edona1, And eas G. And eou2, and Be nabé Lina es-Ba anco1 1Ins i u o de Mic oelec ónica de Se illa (IMSE), Cen o Nacional de Mic oelec ónica (CNM), Ed. CICA, A . Reina Me cedes s/n 41012 Se illa, SPAIN. Phone: 34-5-4239923, Fax: 34-5-4231832, E-mail: [email p o ec ed] 2Dep . o Elec ical and Compu e Enginee ing, The Johns Hopkins Uni e si y, Bal imo e, MD 21218, USA Abs ac An a chi ec u e is p oposed o he ealiza ion o eal- ime edge-ex ac ion il e ing ope a ion in an Add ess-E en -Rep esen a ion (AER) ision sys em. Fu he mo e, he app oach is alid o any 2D il e ing ope a ion as long as he con olu ional ke nel F(p,q) is decomposable in o an x-axis and a y-axis componen , i.e. F(p,q)=H(p)V(q), o some o a ed coo dina e sys em {p,q}. I i is possible o ind a coo dina e sys em {p,q}, o a ed wi h espec o he absolu e coo dina e sys em a ce ain angle, o which he abo e decomposi ion is possible, hen he p oposed a chi ec u e is able o pe o m he il e ing ope a ion o any angle we would like he ke nel o be o a ed. This is achie ed by aking ad an age o he AER and manipula ing he add esses in eal ime. The p oposed a chi ec u e, howe e , equi es one app oxima ion: he p oduc ope a ion be ween he ho izon al componen H(p) and e ical componen V(q) should be able o be app oxima ed by a signed minimum ope a ion wi hou signi ican pe o mance deg ada ion. I is shown ha o edge-ex ac ion applica ions his il e does no p oduce pe o mance deg ada ion. The p oposed a chi ec u e is in ended o be used in a comple e ision sys em known as he Bounda y-Con ou -Sys em and Fea u e-Con ou - Sys em Vision Model, p oposed by G ossbe g and collabo a o s. The p esen pape p oposes he a chi ec u e, p o ides a ci cui implemen a ion using MOS ansis o s ope a ed in weak in e sion, and shows beha io al simula ion esul s a he sys em le el ope a ion and elec ical simula ion and expe imen al esul s a he ci cui le el ope a ion o some c i ical subci cui s. I. In oduc ion Human beings ha e he capabili y o ecognizing objec s, igu es,andshapese eni heyappea embedded wi hin noise, a e pa ially occluded o look dis o ed. To achie e his, he human ision p ocessing sys em is s uc u ed in o a numbe o massi ely in e connec ed neu allaye s wi h eed o wa dand eedbackconnec ions among hem. Neu ons communica e by means o elec ical s eams o pulses. Each neu on b oadcas s i s ou pu o a la ge numbe o o he neu ons, which can be inside he same o a di e en laye s, and he way his is doneis h oughphysicalconnec ionscalledsynapses. One big p oblem encoun e ed by enginee s when i comes o implemen bio-inspi ed ( ision) p ocessing sys ems is o o e come he massi e in e connec ions. An in e es ing way o ying o sol e his is by Add ess E en Rep esen a ion(AER)[1]-[3].InAEReachneu oncodes i s ac i i y as a pulse s eam signal wi h e y low du y cycle, i.e. pulse wid h mus be minimum bu sepa a ion be ween pulses should be ai ly la ge. Each neu on has a code o add ess, and e e y ime i p oduces a pulse i will y o w i e i s code on a common digi al bus. A ecei ing sys em will con inuously be eading his bus and send he pulse o hose neu ons who ough o be connec ed o he sending neu on. In his manne he ac i i y o a la ge numbe o neu onscan be imemul iplexedon acommon bus. This p inciple allows o s uc u e hie a chically a e y complex neu al sys em. Fo example, a e ina chip wi h AER ou pu is con inuously pu ing add esses on a bus ep esen ing he sensed images. Se e al chips, each wi h an AER ecei e sys em, can be eading he same bus, doing some specialized p ocessing and b oadcas ing he ou pu s o all hei neu ons using again AER on ano he ex e nal bus, and so on. Fu he mo e, ex a p ocessing can be added easily while he “add esses” go om one chip o he nex . Fo ins ance, image o a ion o ansla ion can be pe o med in a s aigh o wa d manne by inse ing an EEPROM o which he ans o ma ion ope a ionhasbeenp og ammedpixelbypixel(o add ess by add ess). In he a chi ec u e p oposed in his pape we akead an ageo his ac osimpli y hep ocessingchip. As neu oscien is s manage o un old he in e nal s uc u e and unc ions o he ision sys em, i becomes mo e easible o ma hema iciansandcompu e scien is s op oposeandunde s andbio-inspi ed isionmodelsand algo i hms, and o enginee s o build bio-inspi ed a i icial ision sys ems. One powe ul ision model p oposed ecen lybyG ossbe ge al.[5]is heBounda y- Con ou -Sys em (BCS) and Fea u e-Con ou -Sys em (FCS) ision model. I consis s o nine laye s which a e local illumina ion no maliza ion and con as enhancemen o an inpu image, pe o ms local edge ex ac ion o di e en spa ialo ien a ionsandscales,and hen is able o iden i y consis en long ange con ou s o he shapes in he inpu image h ough p ocessing laye s wi h eed o wa d and eedback connec ions. In his ision model one o he s ages pe o ms a 2D il e ing ope a ion o edge ex ac ion, and o he s ages pe o m o he 2D il e ing ope a ions.Thep ocessinga chi ec u e p oposedin hispape isin ended obeused in his ision model o pe o m a simpli ied e sion il e doing he edge-ex ac ion ope a ion. The same p ocessing a chi ec u ecanbe ep og ammed ope o msomeo he o he 2D il e sneededin heBCS-FCS isionmodel. Thep esen pape iss uc u edas ollows.In henex Sec ion we will b ie ly desc ibe he s uc u e, unc ionali y, and ope a ions pe o med by he BCS-FCS ision model. In Sec ion III we in oduce modi ica ion o he edge-ex ac ion ke nel which subs i u es a p oduc ope a ion by a minimum ope a ion in he o iginal ke nel. Sec ion IV desc ibes b ie ly he essence o AER, and in Sec ion V we in oduce a VLSI a chi ec u e capable o implemen ing a 2D p og ammable il e . Sec ion VI p o idessys emle elbeha io alsimula ion esul so his a chi ec u ep og ammedwi hake nel odoanex ac ion o e ically o ien ed edges, and inally Sec ion VII indica es heconclusionsand u u ewo k. II. The Bounda y-Con ou -Sys em and Fea u e-Con ou -Sys em Vision Model Fig. 1 shows a schema ic ep esen a ion o he s uc u eo heBCS-FCSmodel[5].TheBCSconsis so se e al iden ical subsys ems ( h ee in he case o Fig. 1) each o which is uned o a di e en spa ial scale. Each BCS spa ial subsys em consis s o 8 laye s. Consecu i e laye s ha e been d awn in Fig. 1 as connec ed by hick shaded a ows. We may hink o hese a ows as he ep esen a ion o a con olu ion (o il e ) ope a ion applied o he s a e o he p e ious laye and esul ing in he s a e o he nex laye . Fo ins ance, he 2D inpu image su e s h ee di e en il e ing o con olu ional ope a ions, each o which is he s a ing poin o a BCS subsys em. F om he e on, each BCS subsys em ope a es au onomously. F om Laye 1 oLaye 3 he e a e only eed o wa d il e ing ope a ions, while Laye s 4 o 8 a e connec edina eedback loopcon igu a ion, whichmeans hesys em will eacha s eadys a e a e a ce ainnumbe o i e a ions(i hesys emisimplemen edsequen iallyon a compu e ) o a e a ce ain ime cons an (i he sys em ope a es asynch onously and ully pa allel, like in biological b ains). The ou pu s o Laye s 1 and 5 o he h ee BCS subsys ems a e ed o he FCS. Nex we will b ie lydesc ibe hep ocessing pe o medon hedi e en laye s. A.S age 1: Cen e -ON OFF-Su ound Le usassume (1) is an N×Minpu image p o ided by a ision sensing on end. This inpu image is applied o a 2D il e whose impulsi e esponse o ke nel o adial symme y is shown in Fig. 2(a). We can see ha pixels close o he cen e egion o he ke nel a e going o con ibu e wi h posi i e weigh s o he con olu ion, while pixels u he away will con ibu e nega i ely. The esul o such a con olu ion is local illumina ion no maliza ion and con as enhancemen . The ma hema ical exp ession o his ke nel is (2) whe e , a e posi i e pa ame e s, , and con ols he spa ial scale o he il e ing. In he case o Fig. 1 he e a e h ee BCS subsys ems, which means h ee Cen e -ON OFF-Su ound il e s a e applied in pa allel o he same inpu image, each wi h a speci ic ( ). F om now on he p ocessing in each BCS subsys em is independen . B.S age 2: Simple Cells The second s age o he BCS sys em applies an o ien a ion speci ic con olu ion o de ec ing edges o ien ed wi hin a na ow angle ange. This is pe o med bycon ol ing heou pu o Laye 1wi h heke nelshown in Fig. 2(b) o di e en o ien a ions. This is why he ou pu o Laye 1 in Fig. 1 su e s se e al con olu ions in pa allel, one o each o ien a ion, esul ing in as many “Laye 2” as o ien a ions ha e been conside ed. The ke nel o Fig. 2(b) is ma hema ically desc ibed by he di e encebe ween wodisplacedgaussians Laye 6 Laye 7 Laye 2 Laye 3 Laye 4 Laye 1 Laye 5 Laye 8 Inpu Image Laye 9 Ou pu Image Fig. 1: Schema ic Rep esen a ion o G ossbe g’s e al. Bounda y Con ou Sys em (BCS) and Fea u e Con ou Sys ems (FCS) Vision Model BCS FCS Ipq,() p1…N,= q1…M,=    S1pq,()A1e p2q2 + σg ----------------- A2e p2q2 + ασg ----------------- –= A1A2 >α1>σ g σg g123,,= (3) whe e he coo dina e sys em is o a ed a ce ain angle wi h espec o he coo dina e sys em o he inpu image , (4) wi h being he o al numbe o o ien a ions o be conside ed. C.S age 3: Complex Cells A e applying he il e ing o S age 2, a pixel in Laye 2 o o ien a ion kwilldisplayahighposi i e alue i he inpu image p esen s a posi i e change in con as wi h espec o he k- h o ien a ion axis. I he change in con as is nega i e, he ou pu o his pixel would be a highnega i e alue.Ino de ode ec whe he o no he e is an edge a ha o ien a ion a ound he gi en pixel he e is no need o dis inguish be ween posi i e and nega i e alues. The e o e, he pu pose o his p ocessing s age is simply o ec i y heou pu o hep e iousone. D.S age 4: Hype complex Cells, Compe i ion ac oss Space A Laye 3 o o ien a ion k, pixels will p esen a posi i e alue i a ound ha pixel he e is an edge a ha o ien a ion. The highe he pixel alue, he clea e he edge was. A his s age, and independen ly o each o ien a ion, a 2D Cen e -ON OFF-Su ound il e is applied o con as enhance he p e ious image. This is equi alen o pe o ming a spa ial compe i ion among pixels, a o ing hosewi hhighe alues. E.S age 5: Hype complex Cells, Compe i ion ac oss O ien a ions A his s age, all Laye 4 pixels o he same spa ial posi ion bu o all possible o ien a ions k, a e going o compe e among hem o con as enhance hose o ien a ions wi h highe pixel alues. This is done by applying a 1D Cen e -ON OFF-Su ound il e o pixels o Laye 4 o he same spa ial posi ion bu o all k o ien a ion alues. F.S age 6: Bipole Cells, Long-Range Coope a ion Theope a iono hiss ageis hemos complica ed.I ies o iden i y long e m “Con ou s”, which can be de ined as edges ha emain consis en o e la ge space anges. This is achie ed by pe o ming o each o ien a ionk he ollowingsumo con olu ions, (5) whe e is he esul ing s a e o pixel o Laye 6 o spa ial scale gand o ien a ion k, subsc ip deno es o ien a ion, and each con olu ion is gi en by (6) whe e deno es he s a e o pixel o Laye 5 o o ien a ion , deno es o ien a ion pe pendicula o , and he ke nel is de ined by (7) wi h , , and being posi i e pa ame e s. Fig. 2(c) depic s his ke nel o he case . G.S age 7: Hype complex Cells, Compe i ion ac oss Space Thiss agepe o ms hesameope a ion hanS age4. (a) Fig. 2: Con olu ional Ke nels used in he BCS sys em o Fig. 1: (a) Cen e -ON OFF Su ound Ke nel used by S ages 1, 4, and 7. (b)Edge-Ex ac ion Ke nel used by S age 2. (c) Bipole Ke nel used by S age 6. (b) (c) 010 20 30 40 50 60 70 0 20 40 60 80 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 010 20 30 40 50 60 70 0 20 40 60 80 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 010 20 30 40 50 60 70 0 20 40 60 80 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Fgpkqk ,() 1 2πσghσg ------------------------ e 1 2 ---pk σgh --------   2 – e 1 2 ---qk σg --------1 2 ---+   2 – e 1 2 ---qk σg --------1 2 ---–   2 – –= pkqk ,() pq,() pkpπk nR ------cos qπk nR ------sin–= qkpπk nR ------sin qπk nR ------cos+= nR Apqk gCpqk ,,,() 1= nR ∑ = Apqk gpq,() Cpqk ,,,()y pq,()y ˆpq,()–[]Z pkqk ,()⊗= y pq,() pq,() ˆ Z pkqk ,() pk ()sgn eβpk 2qk 2 +()–eµqk/pk 2 () 2 –×= k –()π nR ------------------- pk ()sgn 2 2qk pk -------- pk ,   a an–    cos× βµ γ k 0== H.S age 8: Hype complex Cells, Compe i ion ac oss O ien a ions Thiss agepe o ms hesame ope a ion han S age 5. The ou pu o S age 8, Laye 8,is combined wi h he ou pu o S age 3, Laye 3, o o m he inpu image o S age 4. This way a eedback loop is o med, which once se led will yield he p ope ou pu o each BCS subsys em. I.S age 9: Fea u e Con ou Sys em (FCS) The in o ma ion abou consis en long ange con ou s can be aken om Laye 5 (once he eedback loop has se led), o all compu ed o ien a ions. The FCS akes he o iginal (local illumina ion no malized and con as enhanced)imagep esen inLaye 1andpe o ms a selec i e di usion ope a ion be ween pixels, using he con ou in o ma ion p esen a Laye 5: he con ou s a Laye 5 ac as ba ie s o he di usion ope a ion. The esul o all his p ocessing is a clean noise- ee image wi hclea andconsis en long angecon ou s. III. An Edge-Ex ac ion Fil e In he es o his pape wewill concen a eonS ages 2 and 3, he il e o edge-ex ac ion and subsequen ec i ica ion.Wewill i s in oduceasimpli ica iono he ke nel o eq. (3) which will allow us o p opose a e y compac and e icien ha dwa e ha akes ad an age o heAERaswell. The ke nel o eq. (3) is decomposable in o wo ac o s, each o which depends only on ei he he x- coo dina e o hey-coo dina e , , (8) wi h (9) The simpli ica ion p oposed he e consis s in subs i u ing he p oduc ope a ion be ween and by he signed minimum, (10) Fig. 3(b) shows he esul o applying he il e ing o eq. (8) o he inpu image o Fig. 3(a), while Fig. 3(c) esul s when using he ke nel o eq. (10). As can be seen he e is no app eciable di e ence in he esul ing images. To e alua e quan i a i ely he e ec o he p oposed app oxima ion we can use he No malized Squa e E o de inedas, . (11) This quan i y helps us o e alua e he di e ence be ween he o iginal ke nel and he modi ied ke nel ob ained when he p oduc ope a ion is subs i u ed by he signed minimum. Table 1 gi es he compu ed NSE o se e al ke nels. All ke nels in Table 1 a e decomposable in he p oduc o wo unc ions ha depend sepa a ely on he and componen s. IV. Using Add ess E en Rep esen a ion (AER) Fig.4 shows aschema ic igu e ou lining heessence behind he AER. Suppose we ha e an “emi e ” chip con ainingala genumbe o neu onso cellsD1,D2,D3, ... whose ac i i y changes in ime wi h a “ ela i ely slow” ime cons an . Fo example, i Chip 1 is a e ina chip and each neu on’s ac i i y ep esen s he illumina ion sensed by a pixel, he ime cons an wi h which his ac i i y pkqk Fgpkqk ,() 1 2π ------ Hgpk ()Vgqk ()= Hgpk () 1 σgh -------- e 1 2 ---pk σgk --------   2 – = Vgqk () 1 σg -------- e 1 2 ---– qk σg --------1 2 ---+   2 e 1 2 ---– qk σg --------1 2 ---–   2 –= Hg.() Vg.() Fg'pkqk ,() 1 2π ------ Hgpk ()[]sgn Vgqk ()[]×sgn= min Hgpk ()Vgqk (),{}× NSE Fxy,()Fmxy,()–2xdyd ∫∫ Fxy,() 2xdyd ∫∫ -----------------------------------------------------------------------= 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 (a) (b) Fig. 3: Beha io al Simula ion Resul s when pe o ming 2D Fil e ing o e ical edge-ex ac ion. (a) Inpu Image, (b) using he edge-ex ac ion il e ke nel o eq. (8), (c) using he modi ied il e ke nel o eq. (10) (c) Fxy,() Fmxy,() xy changesis,a hemos ,equi alen oF ame-Ra e(i.e.,25- 30 changes pe second o a ime cons an o abou 30- 40ms). The pu pose o an AER based communica ion scheme is o be able o ep oduce he ime e olu ion o each neu on’s ac i i y inside a second o “ ecei e ” chip, usinga as digi al buswi ha smallnumbe o pins. In he “emi e ” chip he ac i i y o each pixel has o be ans o med in o a pulse s eam signal such ha pulse wid h is minimum and he spacing be ween pulses is easonably high o ime mul iplex he ac i i y o a ela i ely la ge numbe o neu ons. E e y ime a neu on p oduces a pulse i s add ess o code should be w i en on he bus. Fo he case mo e han one pulses a e p oduced simul aneously by se e al neu ons, a classical a bi a ion ee can be in oduced [1]-[3], o one based in Winne - Take-All(WTA) ow-wise compe i ions[6], o simplyby making no neu on accessing he bus in case o a “collision”[7].Wha e e me hodisused he esul willbe he p esence o a con inuous sequence o add esses o codes on he digi al bus ha one o mo e ecei e chips can ead. Each ecei e chip mus con ain a decoding ci cui y so ha a pulse eaches he neu on (o neu ons) speci ied by he add ess ead on he bus. I each neu on in eg a es he sequence o pulses p ope ly, he o iginal ac i i y o he neu ons in he emi e chip will be ep oduced.No e ha inAER hoseneu ons ha a emo e ac i e access he bus mo e equen ly. This p ope y allows o op imize he use o he bus, since neu ons wi h low ac i i y will no consume much communica ion bandwid h. This is he simples AER based communica ion schemeamong chips.Howe e , AERallowseasily oadd mo ecomplica edp ocessing. Fo example, inpu images can be ansla ed o o a ed by emapping he add esses while hey a el om one chip o he nex . By p ope ly p og amminganEEPROMasalook-up ableanyadd ess emapping can be implemen ed, by simply inse ing he EEPROM be ween he wo chips. Fu he mo e, many EEPROMs can be connec ed in pa allel each pe o ming, o example, a o a ion a a speci ic angle, and each deli e ing he emapped add esses o a se o specialized p ocessing chips. I is also possible o include synap ic weigh ingbyha ing heEEPROMs o e heweigh alue, dumping i on a da a bus, ha e he “ ecei e ” chip ead bo h headd essand heda abus,andpe o maweigh ed in eg a ion in he des ina ion(s) neu on(s). I is also possible o implemen “p ojec i e ields”, i.e. o e e y add ess ha appea s on he bus a small digi al sys em couldgene a easequenceo add essesa oundi andsend i o he“ ecei e ”chip.Thiswouldbea ime-mul iplexed p ojec ion ield gene a ion. In he a chi ec u e p oposed in his pape , we implemen a synap ically weigh ed p ojec ion ield o each add ess ead on he bus, and no in a ime-mul iplexed manne bu in pa allel. As we will see, he ecei e chip will pe o m he ollowing ope a ions: o e e y add ess ead on he bus i will send pulses o a bubble o neu ons a ound ha add ess. The wid h o hose pulses is modula ed acco ding o some weigh s s o ed on chip. Time in eg a ion o hose pulses o he comple e a ay o neu ons in he ecei e chip implemen s a con olu ion ope a ion. In he es o he pape we will concen a e on desc ibing he ci cui s able oimplemen suchacon olu ionalo il e ingchip. V. Sys em Design Fig. 5 shows he basic ope a ing p inciple o he p oposed a chi ec u e. The add ess bus p o ides he coo dina es o heneu on(o pixel)a oundwhich he ke nel o eq. (10) should be applied. Pulses will be applied o all ows wi h y-coo dina e in he in e al , and all columns wi h x-coo dina e in he in e al , whe e is he wid h conside ed o heke nel. Table 1 ke nel pa ame e s NSE (dB) Gaussian -24.92 E en Gabo -19.04 Odd Gabo -19.03 Displaced Gaussians -22.73 Fxy,()Hx()Vy()= e 1 2 ---– x σx -----   2 e 1 2 ---– y σy -----   2σx10= σy15= e 1 2 ---– x σ ---   2 e 1 2 ---– y σ ---   2 2πy ys ----   sin σ15= ys20= e 1 2 ---– x σ ---   2 e 1 2 ---– y σ ---   2 2πy ys ----   cos σ15= ys20= e 1 2 ---– x σx -----   2 e 1 2 ---– yy s – σy -------------   2 e 1 2 ---– yy s + σy -------------   2 –     σx15= σy5= ys5= D1 D2 D3 CHIP1 ARBITER+ENCODER DIGITAL BUS DECODER D1 D2 D3 CHIP2 Fig. 4: Add ess E en Rep esen a ion In e chip Communica ion Scheme x0y0 ,() y0L–y0L+,[] x0L–x0L+,[]2L1+ Pulses will be modula ed in wid h acco ding o unc ion (see eq. (8)) o he ows, and unc ion o he columns. A each pixel he e is an AND ga e which p o ides a pulse o wid h equal o he minimumo and .Thispulsewillgene a ea ixed magni ude cu en pulse o he same wid h which will be in eg a ed on a capaci o . Each pixel con ains wo in eg a o s. One o hem, called he “posi i e in eg a o ”, in eg a es he pulse o leng h when ; while he o he , called he “nega i e in eg a o ”, in eg a es he pulse when . The alues o and ( ) a e s o ed digi ally on chip on a smallRAM. Fig. 6 shows he loo plan diag am o he sys em. I consis s o wo inpu decode s ha decode he add ess o he a i ing pulses, a C-elemen equi ed o he AER communica ion p o ocol [1]-[3], an a ay o in eg a o cells , wo se s o p og ammable monos ables and whose pulse wid hs a e con olled by he bi s s o ed in wo RAMs, and (which s o e he digi al wo ds and , espec i ely), wo a ays o and selec ingcells and , espec i ely, wo ou pu decode s o selec he min H x() Vy(),() Vy() Hx() Hx() y0 x0 x0 BUS X BUS Y y0 Fig. 5: Schema ic Rep esen a ion o he Basic Ope a ion P inciple behind he p oposed A chi ec u e Vy() Vy() Hx() Vy() Hx() min H x() Vy(),() Hx()()sgn Vy()()sgn 0> Hx()()sgn Vy()()sgn 0<Vx() Hy() xy,L…0…L,,–= c1M c11 cNM cN1 cij x1xN xi Px+1Px-1Px+iPx-iPx+NPx-N Mx-L MxL Tx-L Sx-L TxL SxL My-L MyL Ty-L Sy-L TyL SyL yM yj y1 Py+M Py-M Py+j Py-j Py+1 Py-1 Ack Rqs Vmonob C-elemen La ch X La ch Y Inpu Decode X Inpu Decode Y IM+ + IM- Ij+ Ij- I1+ I1- ScanM Scanj Scan1 Io Ou pu Decode X Ou pu Decode Y Scx1ScxN Scxi ScyM Scyj Scy1 Cx1,-L Cx1L CxN,-L CxNL La ch Rqs m Rqs Ack La ch Rqs Ack Mys Mxl CyM,-L CyM,L Cy1,-L Cyj0 Cxi0 Rqs m RyL Ry-L Rys Rxl Rx-L RxL Random Scan Bus 8 4 4 Add ess Bus 4 4 8 4 4 Cxi-l,l Cyj-s,s Cy1,-L RAM Y RAM X Fig. 6: Floo plan o Comple e 2D Fil e ing Sys em Tys Sys Txl Sxl NM× cijMx L–... Mx0... MxL ,, ,, My L–... My0... MyL ,, ,, RAM X RAM Y Rx L–... Rx0... RxL ,, ,, Ry L–... Ry0... RyL ,, ,, 2L1+()N×2L1+()M×Cxil–l, Cyjs–s, cells o be scanned, and a column o scanning ci cui s o eadou hein eg a o sanalogou pu cu en . No e ha in he p esen p o o ype o Fig. 6 he sys em does no gene a e an AER ou pu . This can be sol ed by ei he adding he necessa y ci cui y o each pixel [1]-[3] which will dec ease he cell densi y, o by adding a pos - p ocessing chip ha scans sequen ially all cells in he a ayo Fig.6andgene a esanAERou pu . The ope a ion o he sys em in Fig. 6 is as ollows. In and digi alwo dso bi sa es o ed ( and ). The i s bi (o )indica es hesigno he unc ion (o ). The ollowing bi s indica e he absolu e alue (o ).These bi slinea lycon ol heleng h o he pulse igge ed by monos ables (o ). The pulses gene a ed by he monos ables a e sen h ough lines (o )anda e igge ed whene e an ex e nal pulsea i es o he sys em(whene e signalRqs pulses). When an ex e nal pulse a i es, he inpu decode s ac i a e lines and co esponding o he add ess o he a i ing pulse. The selec ion cells con olled by (cells in Fig. 6, ) connec he pulse in line o line i he sign bi is ‘1’. I hesignbi is‘0’line isconnec ed o henega i e line .Thisway,pulses (o )a esen h ough lines o ( o )dependingon he signo heweigh s o edin (o ). Each neu on has wo in eg a o s. The posi i e in eg a o accumula es cha ge when pulses a e simul aneously a i ing h ough ho izon al and e ical lines o he same sign. Tha is, i in eg a es a pulse when lines and (o lines and ) a e simul aneously high, o equi alen ly i pe o ms he ope a ion . Hence, he posi i e in eg a o in cell compu es along ime he ollowingsum (12) whe e , , is he (lossy) in eg al o e ime o he numbe o pulses pixel is ecei ing, and is he ixed magni ude o he cu en pulses being in eg a ed. Simila ly, henega i ein eg a o accumula escha ge whenpulsesa i ing h oughho izon aland e icallines o opposi e sign and (o and ) a e simul aneously high, ha is, i pe o ms he ope a ion . Hence, along ime i compu es he ollowingsum (13) Consequen ly, he di e ence be ween he ou pu s o he posi i e and nega i e in eg a o s is gi en by, , (14) which is he il e ope a ion we wan o implemen . In wha ollows we will desc ibe he ci cui componen s and ope a ions o each block in Fig. 6. A.Communica ion P o ocol: The C-elemen To pe o m a p ope communica ion be ween wo chips a communica ion p o ocol mus be implemen ed [1]-[3]. In he AER scheme, he sende chip indica es when he add ess o a pulsing neu on is eady on he bus and he ecei e chipmus acknowledge ha hepulsehas been ecei edand ha i is eady o ecei eanewpulse. Fig. 7 shows he iming diag am o a alid communica ion p o ocol o he wo chips. The sende chip gene a es a eques signal and he ecei e gene a es an acknowledge signal . When he sende has pu he add ess on he bus i pulls he eques signal o a high alue. Once he ecei e de ec s a high signal i la ches he ecei ed add ess and pulls he acknowledge signal high. The sende can pu now low and begin o p ocess he ollowing pulse. The ecei e mus wai un il all he monos ables ha e sen hei pulses o he co esponding neu ons and he has gone low o pu he signal low. Once he signal is low he sende can ac i a e he signal o a high alue again. Fig. 8 shows he schema ic o he cell used in he ecei e chip o gene a e he signal. This cell is known as “C-elemen ”. This elemen ecei es wo inpu signals: a eques signal gene a ed by he sende sys em, and signal which is he wi ed- NOR o all he monos able ou pu pulses . The C-elemen gene a es an ou pu acknowledge signal which is sen back o he sende sys em. When no pulses a e being ecei ed, is low. Signal is high as no pulses a e being ScanjIo RAM X RAM Y n 1+ Rx L–... Rxl... RxL ,, ,, Ry L–... Rys... RyL ,, ,, SxlSysHx() Vy() n Hx() Vy() nMxlMys TxlTys xiyjxi Cxil–l,lL…0…L,–= TxlPxil– +Sxl SxlTxl Pxil– -TxlTys Pxil– +Pxil– -Pyjs– +Pyjs– - RxlRys cij Pxi +Pyj +Pxi -Pyj - Pxi +Pyj + ∩()Pxi -Pyj - ∩()∪cij Iij +Iwmin H(xpi)V(yqj),()npq pq, Hx pi ()()sgn Vy qj ()()sgn 0> ∑ = xpi xpxi –= yqj yqyj –= npq xpyq ,() Iw Pxi +Pyj -Pxi -Pyj + Pxi +Pyj - ∩()Pxi -Pyj + ∩()∪ Iij -Iwmin H(xpi)V(yqj),()npq pq, Hx pi ()()sgn Vy qj ()()sgn 0< ∑ = IwHx pi ()()sgn Vy qj ()()sgn min H(xpi)V(yqj),()npq pq, ∑ CHIP1 CHIP2 Add ess Rqs Ack Rqs Add ess Bus Pulse Ack Sende Recei e Fig. 7: Timing diag am o he add ess-e en communica ion p o ocol Bus Rqs Ack Rqs Rqs Ack Rqs Rqs Ack Ack Rqs Ack Rqs Vmonob Tx L–…TxLTy L–…TyL ,, , ,, Ack Rqs Vmonob gene a ed by he monos ables. Consequen ly, he signal is low. When a alid add ess pulse a i es he sende pu s he signal high. The ising edge o his signal is used o igge he monos ables so ha signal becomes low. Once is high and has been se o low he C-elemen se s o a high alue, meaning ha he pulse has been ecei ed. The high alue o is used o la ch he p esen bus add ess and signal ha igge s he monos ables. La ching he add ess assu es ha he co esponding neighbo hood is kep selec ed un il all monos ables inish hei pulses. By la ching we assu e ha he monos able pulses do no end i signal goes low be o e he monos able pulses ha e inished.TheC-elemen wai sun il goeslowandall he monos able pulses inish ( ) o pu he signal low again. Once he acknowledge is low he sende is allowed o pull up again and a new communica ioncyclecanbegin. B.The Monos ables The schema ic o a monos able wi h con olling bi s is shown in Fig. 9. T ansis o s and a e equally sized, as well as ansis o s and . Swi ches a e con olled by a digi al n-bi wo d ha se he capaci ance connec ed o node . When no add ess is being ecei ed, and a e bo h low and hence is also low. T ansis o is cu o so ha node is low. Node is also se low h ough hose ansis o s wi h a high alue. I all bi s a e low will always be high (by ) and no pulse will be gene a ed. When an inpu pulse a i es signal and hence become high. As soon as goeshighnode goeshigh.Cu en begins o low h ough he swi ch o med by ansis o s and cha ging node a he a e se by bi s . When node eaches ol age alue , he cu en h ough ansis o becomes highe han he cu en supplied by so ha he ou pu node lips om high o low. The leng h o he pulse a is he ime aken by cu en o cha ge node up o a ol age o .This imeisgi enby (15) whe e is he o al capaci ance p esen a node and is se by he bi s s o ed in he co esponding RAM wo d o . Wi h his scheme, he leng h o he monos able pulses is linea ly con olled be ween 0 and , wi h being he numbe o bi s con olling each monos able pulse leng h, and he uni capaci ance in Fig. 9. Fig. 10 depic s he pulse wid hs ob ained wi h Hspice e sus he alue o he digi al con ol wo d, o a monos able con olled by bi s. Fo his simula ion, alues o , and we e used. C.The Selec ion Cell Fig. 11 depic s he schema ic o he selec ion cell (o ) used o selec he neighbo hood o cells whe e he monos able pulses ha e o be sen . Each selec ioncell consis so woNANDga escon olling he ga es o wo PMOS ansis o s ( and ) ha beha e like swi ches, and wo NMOS pull down ansis o s ( and wi h a cons an ga e ol age ).Eachselec ioncell( o example, inFig.6) has wocon olsignals( hedecode ou pu and hesign bi om ), one inpu signal ( he monos able ou pu ) and wo ou pu s ( and ). When a pulse a i es wi h add ess , i ac i a es he decode s ou pu and , espec i ely. The decode ou pu con ols all he selec ion cells wi h . When is high, i he sign bi is ‘1’, heselec ion cell connec s he monos ableou pu line o he posi i e line . I he sign bi is ‘0’, line is connec ed o he nega i e line . The sameis alid o he coo dina eselec ioncells. D.The Co e Cell Theschema ico cell o Fig.6isshowninFig.12. I consis s o wo diode-capaci o in eg a o s [3]. The Rqs Vmonob Ack Ack Fig. 8: Schema ic o he C-elemen used o he a bi a ion o he add ess-e en inpu s Ack Rqs Rqs Vmonob Rqs Vmonob Ack Rqs Ack Rqs m Rqs m Rqs Rqs Vmonob 1= Ack Rqs n Rqs m Rqs m Rqs m Rqs m Rqs m V hm b1bn Vm IT M1 M2 M5 M3 M4 Vou Fig. 9: Schema ic o a Monos able Cell S1Sn 20Cu2n-1Cu Mb1Mbn M6 M7 M8 OR{bi}VT M1M2 M3M4 S1…Sn ,, b1…bn ,, Vm Ack Rqs Rqs m M5 Vou Vm Mbibi biVmM8 Rqs Rqs m Rqs m Vou ITM6 M7Vmbi VmV hm M2 M1Vou Vou ITVm V hm TCmono IT --------------- V hm = Cmono Vm b1…bn ,,{} RxlRys 2n1–()CuV hm/ITn Cu 0 5 10 15 20 25 30 35 0 50 100 150 200 250 Fig. 10: Monos ables pulse leng h exp essed in nanoseconds e sus alue o con olling digi al wo d ob ained h ough Hspice simula ion Pulse Wid h (ns) n5= Cu0.2pF=IT75µA=V hm 2.5V= Cxil–l,Cyis–s, MP+MP- MN+MN- VPD Cxil–l, xi SxlRAM X TxlPxil– +Pxil– - xiyj ,() xiyj xiCxil–l, lL–…L,,[]∈xiSxl Cxil–l, TxlPxil– +Sxl TxlPxil– - Y cij posi i e in eg a o in eg a es he ANDED pulses ha a i e in ow and column lines wi h he same sign, ha is, and (o and ). The nega i e in eg a o in eg a es he ANDED pulses ha a i e in ow and column lines o opposi e sign, ha is, and (o and ). Each diode-capaci o in eg a o consis s o wo ansis o s and ( and )ope a ingin he sub h eshold egion, a capaci o and a ansis o (o ) ac ing as a cu en sou ce o alue (con olled by bias ol age ) wi h i s sou ce pulsed by heou pu o heNORga e.The inpu andou pu cu en s and o he posi i e in eg a o a e ela ed o he ol age a node h ough he ollowing di e en ial equa ions( he ea men o henega i ein eg a o would be hesame o cu en s , and ol age )[3], (16) (17) whe e is he he mal ol age and , a e model pa ame e s o he MOS ansis o ope a ing in he sub h eshold egion. F om eqs. (16) and (17) we can ge an exp ession ha ela es he ou pu and inpu cu en s o he diode , (18) whe e (19) No e ha cu en mi o gain Ais con olled by ol age . Du ing he ime in which and (o and ) a e simul aneously high, he sou ce o ansis o is low, and his ansis o is ac ing as a cu en sou ce sinking a cons an cu en om he in eg a ion node . In his case in eq. (18). Suppose ha a ain o pulses o cons an equency , pulse wid h and in e spike in e al (as depic ed in Fig. 13) is applied simul aneously o lines and (o and ). In eg a ing equa ion (18) om o wi h , esul sin , (20) whe e he in eg a o ime cons an is gi en by . When he ANDED pulses a e ze o, he sou ce o ansis o becomes high and . I he pulses go low a ime and s ay low o an in e spike ime (see Fig. 13), he ou pu cu en a ime jus be o e a new pulse is applied, is gi en by . (21) When pulses o wid h a e applied a a cons an equency as shown in Fig. 13, a s eady s a e is eached in which he cha ge injec ed by he diode du ing he inac i e pe iods equals he cha ge sank by he xi SxlTxl Pxi-l+ Pxi-l- VPD VPD Fig. 11: Schema ic o a neighbo hood selec ion cell MP+ MP- MN+ MN- Pxi+ Pxi- Pyj+ Pyj- Vw VA Scxi Pxi+ Pxi- Pyj- Pyj+ Vw VA ScxiIj- VREF C C Iij- Fig. 12: Schema ic o he Co e Cell wi h he wo diode-capaci o in eg a o s g- g+ Mw+ Mw- M1+M2+ M1-M2- Iin+ Iin- VREF Iij+ Ij+ Pxi +Pyj +Pxi -Pyj _ Pxi +Pyj - Pxi -Pyj + M1 +M2 +M1 -M2 - C Mw +Mw -Iw Vw Iin +Iij + g + Iin -Iij - g - Iin +C g + d d -------- –Iop VAκ g + – ----------------------    exp+= Iij +Iop Vdd κ g + – -------------------------    exp= Iop κ QT Iij + d d -------- Iij +Iin +1 A --- Iij + –   = AVdd VA – -----------------------   exp= QT C κ --------= VAPx+iPy+jPx-i Py-jMw+ Iw g +Iin +Iw = 1T⁄ThTl Px+iPy+jPx-iPy-j Pxi+ 1 2 2+Tl ThTl T nT (n+1)T Fig. 13: Timing diag am o he pulses applied o lines , , o Pxi +Pxi -Pyj +Pyj - 1 2Iin +Iw = 1 Iij + 1Th +() ----------------------------1 AIw ----------1 Iij + 1 () ---------------1 AIw ----------–   Th τ ------–   exp+= τC /κIw = Mw+Iin +0= 2 Tl 2Tl + 1 Iij + 2Tl +() ---------------------------1 Iij + 2 () ---------------Tl AQT -----------+= Th 1T⁄