Applications of event‐based image sensors—Review and analysis

Recei ed 26 Ap il 2018; Re ised 6 June 2018; Accep ed 6 July 2018 DOI: xxx/xxxx RESEARCH ARTICLE Applica ions o E en -based Image Senso s —Re iew and Analysis† Juan A. Leñe o-Ba dallo* | Rica do Ca mona-Galán | Ángel Rod íguez-Vázquez 1Ins i u e o Mic oelec onics o Se ille (IMSE-CNM), Consejo Supe io de In es igaciones Cien í icas y Uni e sidad de Se illa, C/ Amé ico Vespucio s/n, 41092, Se ille, Spain Co espondence *Juan A. Leñe o-Ba dallo. Email: [email p o ec ed] Summa y The sp ead o e en -d i en asynch onous ision senso s du ing he las yea s has inc eased signi ican ly he indus ial in e es and he applica ion scena ios o hem. This a icle e iews he main ields o applica ion ha e en -based image senso s ha e ound du ing he las wen y yea s. We ocus in he desc ip ion o applica ions whe e such de ices can ou pe o m con en ional ame-based senso s. The p ac ical unc ions o he h ee main amilies o asynch onous e en -based senso s a e ana- lyzed. The a icle also s udies wha a e he ac o s ha inc ease nowadays he demand o senso s ha minimize he powe and bandwid h consump ion. Mo eo e , he ech- nological ac o s ha ha e acili a ed he de elopmen o asynch onous senso s a e discussed. KEYWORDS: Image senso s, e en , spike, Add ess E en Rep esen a ion, AER, asynch onous 1 INTRODUCTION Since Mahowald and he colleagues published in he ea ly nine ies he i s in eg a ed asynch onous bio-inspi ed ision sen- so 1, he sp ead o e en -d i en de ices has g own signi ican ly2. Depending on he in o ma ion om he isual scene ha is p ocessed, he e a e h ee main ca ego ies o e en -based (also known as e en -d i en o jus spiking) ision senso s. The i s , and he oldes g oup, co esponds o senso s ha de ec spa io- empo al con as wi hin he isual scene. In hei o igins, hey we e bio-inspi ed senso s ha y o emula e he ea ly p ocessing o he isual scene ha is pe o med in he human e ina. The second and he mos success ul ca ego y so a co esponds o Dynamic Vision Senso s (DVS). These a e de ices whose pix- els spike whe he hey de ec ansien illumina ion changes. They can ack e y as mo ing objec s wi h e y low powe and bandwid h da a consump ion. Finally, he hi d ca ego y co esponds o spiking luminance senso s, also known in he li e a u e as oc opus e inas. Al hough hey jus pe o m a ligh - o- equency con e sion, hey p ese e he asynch onous communica ion †This wo k was suppo ed in pa by he Spanish Minis y o Economy and Compe i i eness unde G an TEC2015-66878-C3-1-R, Co-Funded by ERDF-FEDER; in pa by Jun a de Andalucía CEICE unde G an TIC 2012-2338 (SMARTCIS-3D); and in pa by ONR unde G an N000141410355 (HCELLVIS). 2JUAN A. LEÑERO-BARDALLO ET AL based on spikes o encode he illumina ion le els. Only illumina ed pixels can send in o ma ion o -chip. Thei dynamic ange and hei la ency a e qui e compe i i e. Thus, hey a e a ac i e o applica ions whe e is necessa y o de ec illumina ion le els o ce ain egions wi h low esou ces consump ion. Al hough he comme cial success o asynch onous ision senso s is s ill a away om ame-based senso s, he e a e se e al companies like P ophesee o Inilabs dedica ed o he ab ica ion o e en -based came as. The ise o he demand o ision senso s wi h low powe and bandwid h consump ion, and he capabili y o p ocessing he isual scene has pu spiking senso s in o he scene. D one ision and au onomous ehicles p e e de ices wi h such ea u es ins ead ones ha p o ide high image quali y. Fo hese easons, asynch onous image senso s ind accommoda ion and a e highly compe i i e in si ua ions whe e image quali y is no a mus . The e a e se e al e iews and compa isons be ween he di e en e en -based senso s published so a 2,3. Howe e , o he bes o ou knowledge, s ill he e is no s ill published a e iew o he speci ic ields o applica ion whe e hey ha e been employed. In his a icle, we will ocus on a desc ip ion o a he ields o applica ion whe e e en -based ision senso s ha e been success ully accommoda ed. We will pay special a en ion o applica ions o he amily o spiking luminance senso s in which he au ho s ha e made ele an con ibu ions du ing he las yea s. 2 SENSORS BASED ON SPATIAL CONTRAST DETECTION Spa ial con as de ec ion is he e y i s -p ocessing o he isual scene pe o med by he human isual sys em4. I is imple- men ed by he cells ha compound he e ina. Thei ou pu s a e ansmi ed as ain o spikes h ough he op ic ne e. The de ec ion o spa io- empo al con as allows o de ec edges, and shapes. Tha is c ucial o an ea ly in e p e a ion and classi ica- ion o he elemen s wi hin he scene. A i s ins ance, spiking senso s capable o compu ing spa ial con as on he ocal plane pu sue o mimic a biological modelling o he human e ina4. The i s silicon e ina chip was p oposed by Mahowald,1. The o iginal pixel opology is shown in Fig. 1 . Pixels de ec ligh wi h a pho o ecep o ci cui which ou pu is a ol age ha is a e - aged wi hin a neighbo hood wi h a di usi e esis i e ne wo k ha in e connec s all he pixels. I he a e age illumina ion alue in he neighbo hood, 𝑉𝑝𝑛𝑒𝑖𝑔ℎ𝑏. , di e s om he pixel’s local illumina ion alue, 𝑉𝑝𝑖,𝑗 , he pixel will spike, indica ing he de ec ion o spa ial con as . These ci cui s, in he li e a u e, s ill hey a e known as silicon e inas due o hei biological inspi a ion. The e a e imp o ed e sions o his p elimina y app oach5,6,7. This amily o senso s opened a ield o esea ch ha lead o ul e io de elopmen o e en -based ision senso s. They also pu emphases in he de elopmen o asynch onous ci cui y o implemen he well-known AER (Add ess E en Rep esen a ion) communica ion p o ocol8,1,9. Mode n a bi a ion schemes10,11 a e based on he ea ly e sions implemen ed on he i s asynch onous bio-inspi ed ision senso s. JUAN A. LEÑERO-BARDALLO ET AL 3 + _ + _ Pho o ecep o Righ Top Bo om Le C Pixel’s Ou pu R R R R Vpi,j Vpneighb. ___ Vpneighb. ___ FIGURE 1 Re ina con as de ec ion pixel acco ding o1. A e age illumina ion alues wi hin a pixel neighbo hood (𝑉𝑝𝑛𝑒𝑖𝑔ℎ𝑏. ) a e compu ed wi h a di usi e esis i e ne wo k. The a e age illumina ion alue in he neighbo hood (𝑉𝑝𝑛𝑒𝑖𝑔ℎ𝑏. ) is compa ed o he local pixel illumina ion (𝑉𝑝𝑖,𝑗 ). I bo h alues di e , he pixel spikes, wi h a equency p opo ional o he spa ial con as magni ude. Once he possibili y o implemen ing hese senso s was demons a ed, he communi y d ew i s a en ion abou how o imp o e he quali y o hei images. Pa icula ly, misma ch was an impo an challenge associa ed o he i s gene a ion o bio-inspi ed senso s, ope a ing wi h ansis o s in sub h eshold egion. Se e al au ho s poin ed ou he need o educing pixel Fixed-Pa e n- Noise (FPN) associa ed o ansis o misma ch. In ha sense, calib a ion p ocedu es and ci cui y obus o misma ch we e de eloped12,7. Nowadays, he e en -d i en spa ial con as compu a ion is no highly compe i i e in e ms o spa ial noise agains he adi ional me hod: ead-ou an en i e pixel ma ix illumina ion alues and p ocess hem on he digi al domain o de ec edges and shapes. The main eason is ha asynch onous compu a ion o spa ial con as on he ocal plane equi es non-linea ope a ions ha a e di icul o implemen p ecisely in he analog domain. Howe e , hey dese e o be men ioned in his su ey because hey led o he de elopmen o o he amilies o asynch onous senso s whose applica ions will be desc ibed in he nex sec ions. 4JUAN A. LEÑERO-BARDALLO ET AL + _ Cph V ese + _ + _ -A C 1 C 2 Rese V bo V op OFF EVENT ON EVENT α log(I ) iph = −C /C·(d /d ) di 1 2 i FIGURE 2 A chi ec u e o a DVS senso . Ligh is de ec ed by a pho ologa i hmic ecep o wi h nega i e eedback and gain −𝐴. I s ol age ou pu s eed a di e encing ci cui s ha ampli ies ansien ol age a ia ions a i s inpu , 𝑣𝑖. I he di e encing ci cui ou pu , 𝑣𝑑𝑖𝑓𝑓 , exceeds a nega i e o a posi i e ol age h eshold, e en s a e ansmi ed indica ing empo al con as . The sign o he empo al con as (posi i e o nega i e) is de ec ed wi h wo compa a o s and ansmi ed h ough wo independen pa hways. 3 DVS SENSORS The pixel’s a chi ec u e o a Dynamic Vision Senso (DVS) senso s is depic ed in Fig. 2 . Ligh is p ocessed by a loga i hmic pho o ecep o . I s ou pu ol age, 𝑣𝑖, eds a di e encing ci cui ha compu es and ampli ies i s empo al de i a i e, 𝑣𝑑𝑖𝑓𝑓 . I he ansien a ia ion o di e encing ol age exceeds a p og ammable h eshold (ei he posi i e o nega i e), and e en is sen o - chip, indica ing a pixel’s illumina ion change, i s coo dina es, and he sign o he empo al con as . DVS senso s can de ec as changes o illumina ion, wi h low powe and da a bandwid h consump ion. Mo emen de ec ion is ou pe o med o e ansi ional p ocedu es, based on ame p ocessing. P oo o ha is he ac ha se e al companies like Inilabs o P ophesee ha e pu on hei scope he de elopmen o hese amily o senso s. The i s unc ional DVS senso was p esen ed by Lich s eine e al. in 200813. In his i s publica ion, he au ho s al eady de ised possible applica ions o he senso , i.e. a ic moni o ing, gaming, sleep moni o ing, e c. In mo e ecen publica ions, se e al au ho s p esen ed imp o ed e sions o he o iginal DVS senso 14,15,16,17. Rega ding he applica ion ields o hese senso s, we can highligh he same examples: 1. Da ase s o implemen a ion and aining o spiking neu al ne wo ks o objec acking, ac ion ecogni ion and/o objec ecogni ion. Spiking neu al ne wo ks needs o be ed wi h spiking inpu s. Images gene a ed wi h asynch onous image senso s a e ideal o his pu pose. The sp ead and popula iza ion o DVS senso s among he communi y has d i en he c ea ion o e en -based image da ase s eco ded wi h DVS senso s18,19. JUAN A. LEÑERO-BARDALLO ET AL 5 2. T a ic moni o ing. The con ol o mo ing objec s is an ideal ask o DVS senso s. Only pixels ha de ec illumina ion changes send da a o -chip, minimizing he bandwid h and he powe consump ion. The e a e speci ic pape s add essing his opic, o ins ance,20. Also DVS senso s ha e been used o implemen p oximi y de ec o s,21. 3. Fall de ec o s. P ocessing he ou pu da a low o a DVS senso , he au ho s p opose a all de ec o 22. I was in ended o home- ca e o elde people. In can be in eg a ed among he senso s o an in elligen house. Ges u e ecogni ion is also possible,23. 4. Robo ics in eg a ion, con ol and na iga ion24. The idea is o p o ide images o isual scenes wi h came as ha a e moun ed on sys ems ha a e con inuously mo ing o obse ing a non-s a ic isual scene. 5. T acking and pa icles de ec ion on luids. The excellen empo al esolu ion o DVS senso s has made possible o ack as mo emen s o pa icles in luids educing he da a h oughou 25. 6. Colo change de ec ion26. Combining he pixel a chi ec u e o a DVS senso wi h s acked pho odiodes a di e en dep hs in s anda d silicon, i was possible o c ea e an image senso whose pixels only eac s o colo changes ( ansien shi s wi hin he spec um). 7. S e eo ision27,28. The combina ion o wo o mo e DVS senso s eco ding images om di e en localiza ions allows de e - mine dis ances and dep h, a e p ocessing hei ou pu s. The DVS senso educed ou pu da a low allows o alle ia e he bandwid h and powe p ocessing consump ion wi h ega ds o classic ame-based p ocessing algo i hms. 4 SPIKING LUMINANCE SENSORS In his pape , we will ocus on summa izing p ac ical applica ions whe e asynch onous spiking senso s ha e been employed. These de ices, also known as oc opus e inas, pe o m a ligh - o- equency con e sion o encode luminance le els wi hin he isualscene29.Thei ope a ion p inciple is depic ed in Fig. 3 .Pixelope a ionis ini ia edby ese ing he ol age a hein eg a ion capaci ance, 𝐶. Then, i s ol age dec eases linea ly wi h a slope ha is p opo ional o he pixel’s pho ocu en un il i eaches a p og ammable analog ol age, 𝑉𝑟𝑒𝑓 . The ea e , he pixel sel - ese s and s a s in eg a ing cha ge again. The pixel’s oscilla ion pe iod is app oxima ely gi en by his exp ession: 𝑓𝑜𝑠𝑐 ≈𝐼𝑝ℎ 𝐶⋅(𝑉𝐷𝐷 −𝑉𝑟𝑒𝑓 )=𝐼𝑝ℎ 𝐶⋅Δ𝑉(1) Whe e 𝐼𝑝ℎ is he pixel pho ocu en and 𝐶is he in eg a ion capaci ance. 𝑉𝑟𝑒𝑓 is a ol age h eshold o con ol he pixel sensi i i y o ligh . Al hough hese senso s do no implemen any kind o p ocessing, jus measu ing luminance le els, hey ha e inhe en ad an ages o e con en ional ame-based image senso s. The i s one is speed; pixel spiking equency can 6JUAN A. LEÑERO-BARDALLO ET AL c V ( ) spike 1 spike 2 spike 3 M1 D1 Iph V c V c V e V e V DD -I /C ph + _ C VDD spike VDD V e _ eq_x ese _x _ eq_y _ack_y ese _y AER Communica ion Ci cui y ∆V FIGURE 3 In eg a e-and- i e (I&F) pixel block diag am. Pixels spike wi h a equency p opo ional o ligh in ensi y. E e y ime ha his occu s, pixels’ add esses a e ansmi ed asynch onously o -chip. Asynch onous digi al signals in ol ed in he da a ansmission a e displayed on he igh . easily each KHz a es in indoo en i onmen s30,29. This implies an equi alen empo al esolu ion in he o de o Kilo- ames pe second. The second ad an age is hei au onomous ope a ion. Pixels ha e no o be scanned pe iodically o gauge hei illumina ion alues. They pulse con inuously whene e hey a e illumina ed. I hey a e da k, hey do no send any da a. Tha is also an ad an age o applica ion scena ios whe e da k pixels a e meaningless. Ano he s eng h o hem is hei High Dynamic Range (HDR) ope a ion. Pixel spiking equencies can spam mo e han i e decades31,29. This implies he capaci y o de ec ing in a-scene illumina ion le els wi h a dynamic ange highe han 100dB. Also, hei powe consump ion is educed wi h espec ame-based senso s. Thei main penal y is image quali y. FPN noise is ypically highe wi h hese senso s30,31. Howe e , he e many applica ions ha can sac i ice image quali y in bene i o speed, HDR ope a ion, and low powe consump ion. We summa ize and explain some o hem in he nex subsec ions. 4.1 Sun Senso s Sun senso s de ec he sun posi ion (azimu h and la i ude) e e ed o hei cen oid. They a e demanded o g een ene gy p o- duc ion in sola plan s and o de elop na iga ion sys ems o spacec a s and sounding ocke s, ha ake he sun as a e e ence o o ien a ion. The e a e wo main classic amilies o sun senso s: analog33 and digi al34,35. Analog ones a e mo e simple and JUAN A. LEÑERO-BARDALLO ET AL 7 θ x y FD D T Pixel Ma ix Lid & Pinhole Rmax φ (a) (b) 10010110210 310 4 Deg ees (º) 72.5 72.6 72.7 72.8 72.9 La i ude (θ) E o (º) 0 0.1 0.2 0.3 0.4 θ (º) E o (º) Numbe o Spikes (N ) 10010110210 310 4 Deg ees (º) 34 34.5 35 35.5 36 36.5 Azimu h (φ ) E o (º) 0 0.5 1 1.5 2 2.5 φ (º) E o (º) Ν =1 spikes Ν =1 spikes spikes (c) FIGURE 4 (a) Ope a ion o a sun senso 32 buil wi h a pinhole lens. (b) Snapsho o an asynch onous sun senso implemen ed wi h a pinhole lens. (c) Senso ’s ou pu da a. The sun posi ion is calcula ed compu ing he cen oid o a ce ain numbe o incoming e en s. Wi h only one e en , i is possible o gauge he sun posi ion wi h an e o lowe han 2.2%. easy o design, bu hey a e mo e p one o noise and sensi i e o scene dis ac o s. Digi al ones a e he mos ex ended ones nowa- days. They a e eliable and easy o build. Deploying a dedica ed op ics on op o a ame-based senso , i is possible o con e i in o a sun senso . Depending on he sun posi ion, a g oup o pixels will be illumina ed. I s cen oid will depend on he sun localiza ion. By p ocessing he ou pu da a, he cen oid is calcula ed. Then, es ablishing basic igonome ic ela ions, he sun posi ion can be de e mined. The main limi a ions o his app oach a e he da a edundancy and he limi ed ope a ion speed. Da a edundancy is an impo - an d awback because da k pixels needs o be eadou and a e useless o compu e he sun posi ion. This also d i es o high bandwid h and powe consump ion de i ed o he da a eadou and he da a p ocessing. The second limi a ion is he ope a ion speed. Since he en i e pixel ma ix has o be eadou , e e y ime ha he sun posi ion has o be upda ed, he ope a ion speed is condi ioned by he ame a e. Se e al au ho s ha e p oposed s a egies o a oid eading da k pixels34. Howe e , s ill some o hem ha e o be eadou pe iodically. A ecen al e na i e o build digi al sun senso s is he usage o oc opus e inas36,32. By placing a dedica ed op ic o e hem, only pixels exposed o sunligh will send da a. The e o e, da a edundancy is a oided, simpli ying he da a p ocessing, he senso ope a ion and educing powe consump ion. Mo eo e , he excellen empo al esolu ion o oc opus e inas make hese de ices sui able o space na iga ion. In Fig. 4 .a, we display he a chi ec u e o a spiking sun senso wi h a pinhole lens on op o i . In Fig. 4 .b, he ab ica ed spiking sun senso is displayed. Da a eco ded wi h he senso is p o ided in Fig. 4 .c. The cen oid compu a ion o he illumina ed egion is as and simple. Jus compu ing he cen oid o he pixels ha i es, i is possible o do ha . The ope a o has eedom o decide how many spikes a e employed o compu e i . Usually he cen oid is close o he highes 8JUAN A. LEÑERO-BARDALLO ET AL 20 40 60 80 20 40 60 80 100 120 0 500 1000 1500 2000 2500 3000 3500 4000 4500 Seconda y Combus ion zone In e zonal Region P ima y Combus ion Zone Rela i e NIR le els 4 5 6 7 8 9 10 Time (s) 100 150 200 250 300 Rela i e Ene gy in he NIR band T ansien Va ia ions o NIR ene gy in he Scene -100 -80 -60 -40 -20 0 20 40 60 80 100 F equency (Hz) 40 50 60 70 80 90 |X(jw)| (dB) (a) (b) FIGURE 5 Flames moni o ing wi h an asynch onous NIR spiking senso 38,39. (a) Snapsho o a lame aken wi h a luminance asynch onous senso wi h an op ical NIR il e . (b) Top: T ansien NIR le els a ia ions de ec ed wi h he senso in a isual scene wi h a lame. Bo om: FFT o he p e ious da a o s udy he lame equency componen s. illumina ed pixel ha spikes i s han he o he s. Thus, jus aking in o accoun i s coo dina es, i is possible o gauge he sun posi ion wi h accep able p ecision. In e ms o dynamic ange, spiking senso s allow o encode he illumina ion le els wi hou sa u a ing. Usually, ame-based senso s dynamic ange is limi ed o 70dB37, ha is no enough o de ec illumina ion le els inside e y b igh ligh sou ces like he sun ha equi e senso s wi h mo e han 100dB dynamic ange o be cha ac e ized37. 4.2 Flame moni o ing Asynch onous spiking senso s can be used in he de ec ion and he analysis o lames ac i i y,38,39. Flame licke ing and lame emissions ha e o be moni o ed in some indus ial p ocesses40,41. These asks we e implemen ed, a i s ins ance, wi h in a ed (IR) came as o mic obolome e s. Thei main limi a ions a e he high cos , he limi ed ame a e, he need o calib a ion in some cases, and hei agili y42. As al e na i e o hem, CMOS o CCD came as wo king in he Nea In a ed Band (NIR) ha e been employed. By placing a NIR il e o e hem, he become NIR image s. Radia ion inside his band is emi ed by lames. Hence, wi h adequa e image p ocessing, lames ac i i y can be moni o ed43,44,45. Flames licke wi h equency componen s ha ange be ween DC and 100Hz40. Thus, he analysis o hei ansien ac i i y equi es senso s wi h good empo al esolu ion. This equi emen also JUAN A. LEÑERO-BARDALLO ET AL 9 P- N- P- N+ I&F I&F I&F IB IGB IRG RB GRGB (a) (b) FIGURE 6 Asynch onous colo senso implemen a ion,46. (a) S acked pho odiodes o achie e spec al selec i i y wi hin he isible spec um. Each one is connec ed o in eg a e-and- i e (I&F) block as he one depic ed in Fig. 4 . (b) Inpu s imulus and colo images ende ed wi h an asynch onous senso implemen ed wi h he p oposed pixel a chi ec u e. implies ha , ope a ing wi h a ame-based came a, he amoun o da a o be p ocessed is be e y high, limi ing he da a analysis o o -line ope a ion40. Recen ly, he au ho s ad anced how oc opus e inas can used o moni o lame ac i i y,39. The excellen empo al esolu ion o hese senso s can be exploi ed o ack hei ansien ac i i y e icien ly. Fil e ing he adia ion below he NIR band, he ou pu da a is limi ed o he egions exposed o NIR adia ion. Hence, hei ou pu da a low can be p ocessed eal- ime39, ou pe o ming he ope a ion ob ained wi h classic NIR image s40. In Fig. 5 .a, a snapsho o a lame aken wi h an oc opus asynch onous senso ab ica ed in s anda d CMOS echnology is displayed. An op ical il e was placed o e he senso o emo e he adia ion ou side he NIR band. Resul s a e alike he ones ha can be ob ained wi h a comme cial in a ed came a. I is possible o iden i y he main egions inside he lame. On he op o he igh plo in Fig. 5 .b, we ha e plo ed he ansien NIR le els de ec ed by he senso in a isual scene wi h lames. Flames p o oke as a ia ions o he NIR le els. On he bo om o he same plo , he Fas Fou ie T ans o m (FFT) o he p e ious da a is displayed. Main equency componen s ange om DC o 30Hz, o his lame. 4.3 High speed colo de ec ion wi h s acked pho odiodes Cu en ly, s anda d CMOS echnologies o e he possibili y o inco po a ing a deep n-well o a deep p-well o he chip design. These laye s allow o s ack pho odiodes a di e en dep hs, placing a deep n-well benea h an n-well, as i is depic ed in Fig. 6 .a. S acked pho odiodes ha e di e en sensi i i y o ligh wa eleng h, depending on hei dep h47,48. Hence, simila spec al disc imina ion can be achie ed han employing adi ional me hods wi h disc e e op ical il e s. In his p e ious wo k46, he e diodes we e s acked o encode colo a high speed. In Fig. 6 .a, he e is a ske ch o he colo pixels. Each one is connec ed o an in eg a e-and- i e (I&F) module wi h he same a chi ec u e han an oc opus senso as he one shown in Fig. 3 . P ocessing he 16 JUAN A. LEÑERO-BARDALLO ET AL 24. Delb uck T., Villanue a V., Longino i L.. In eg a ion o dynamic ision senso wi h ine ial measu emen uni o elec on- ically s abilized e en -based ision. In: 2014 IEEE In e na ional Symposium on Ci cui s and Sys ems (ISCAS):2636-2639; 2014. 25. D azen Da id, Lich s eine Pa ick, Hä lige Philipp, Delb ück Tobi, Jensen A le. Towa d eal- ime pa icle acking using an e en -based dynamic ision senso . Expe imen s in Fluids. 2011;51(5):1465. 26. Fa ian L., Leñe o-Ba dallo J.A., Hä lige P.. A Bio-Inspi ed AER Tempo al T i-Colo Di e en ia o Pixel A ay. Biomedical Ci cui s and Sys ems, IEEE T ansac ions on. 2015;9(5):686-698. 27. Domínguez-Mo ales Manuel Jesus, Ce ezuela-Escude o Elena, Pe ez-Peña Fe nando, Jimenez-Fe nandez Angel, Lina es- Ba anco Alejand o, Jimenez-Mo eno Gab iel. On he AER S e eo-Vision P ocessing: A Spike App oach o Epipola Ma ching. In: Neu al In o ma ion P ocessing:267–275Sp inge Be lin Heidelbe g; 2013; Be lin, Heidelbe g. 28. Domínguez-Mo ales M., Jimenez-Fe nandez A., Paz R., e al. An App oach o Dis ance Es ima ion wi h S e eo Vision Using Add ess-E en -Rep esen a ion. In: Neu al In o ma ion P ocessing:190–198Sp inge Be lin Heidelbe g; 2011; Be lin, Heidelbe g. 29. Culu ciello E., E ienne-Cummings R., Boahen K.A.. A biomo phic digi al image senso . Solid-S a e Ci cui s, IEEE Jou nal o . 2003;38(2):281-294. 30. Leñe o-Ba dallo Juan A., Ca mona-Galán R., Rod íguez-Vázquez A.. A wide linea dynamic ange image senso based on asynch onous sel - ese and agging o sa u a ion e en s. IEEE Jou nal o Solid-S a e Ci cui s. 2017;52(6):1605-1617. 31. Leñe o-Ba dallo Juan A., Ca mona-Galán R., Rod íguez-Vázquez. A.. A bio-inspi ed ision senso wi h dual ope a ion and eadou modes. Senso s Jou nal, IEEE. 2016;16(2):1-14. 32. Leñe o-Ba dallo J. A., Fa ian L., Gue e o-Rod íguez J. M., Ca mona-Galán R., Rod íguez-Vázquez Á.. Sun Senso Based on a Luminance Spiking Pixel A ay. IEEE Senso s Jou nal. 2017;17(20):6578-6588. 33. O ega P., Lopez-Rod iguez G., Rica J., e al. A Minia u ized Two Axis Sun Senso o A i ude Con ol o Nano-Sa elli es. IEEE Senso s Jou nal. 2010;10(10):1623-1632. 34. Xie N., Theuwissen A. J. P.. A Minia u ized Mic o-Digi al Sun Senso by Means o Low-Powe Low-Noise CMOS Image . IEEE Senso s Jou nal. 2014;14(1):96-103. 35. Bold ini F., Monnini E., P ocopio D., e al. A i ude senso s on a chip: Feasibili y s udy and b eadboa ding ac i i ies. In: P oceedings 32nd Annual AAS Guided Con ol Con e ence:1197âĂŞ1216; 2009. JUAN A. LEÑERO-BARDALLO ET AL 17 36. Fa ian L., Hä lige P., Leñe o-Ba dallo J. A.. A Minia u ized Two-Axis Ul a Low La ency and Low-Powe Sun Sen- so o A i ude De e mina ion o Mic o Space P obes. IEEE T ansac ions on Ci cui s and Sys ems I: Regula Pape s. 2018;65(5):1543-1554. 37. Da mon A naud. Me hods o ex end he dynamic ange o snapsho ac i e pixel senso s. In: P oceedings o SPIE Vol. 6816, 681603 (2008); 2008. 38. Leñe o-Ba dallo J. A., Gue e o-Rod íguez J. M., Ca mona-Galán R., Rod íguez-Vázquez Á.. On he analysis and de ec ion o lames wi h an asynch onous spiking image senso . IEEE Senso s Jou nal. 2018;:1-9. 39. Leñe o-Ba dallo J.A., B yn D.H., Hä lige P.. Flame moni o ing wi h an AER colo ision senso . In: Ci cui s and Sys ems (ISCAS), 2013 IEEE In e na ional Symposium on:2404-2407; 2013. 40. Yan Yong, M. Colechin, R. Hill. Moni o ing o oscilla o y cha ac e is ics o pul e ized coal lames h ough image p ocessing and spec al analysis. IEEE T ansac ions on Ins umen a ion and Measu emen . 2006;55:226-231. 41. Jones A R. Flame ailu e de ec ion and mode n boile s. Jou nal o Physics E: Scien i ic Ins umen s. 1988;21(10):921. 42. B iz S., Cas o A.J., A anda J.M., Melendez J., Lopez F.. Reduc ion o alse ala m a e in au oma ic o es i e in a ed su eillance sys ems. Remo e Sensing o En i onmen . 2003;:19-29. 43. Cheon Jimin, Lee Jeonghwan, Lee Inhee, Chae Youngcheol, Yoo Youngsin, Han Gunhee. A Single-Chip CMOS Smoke and Tempe a u e Senso o an In elligen Fi e De ec o . IEEE Senso s Jou nal. 2009;9:625-628. 44. Maoul Y. Le, Sen enac T., O eu J.J., A cens J. P.. A New App oach Based on a Low Cos CCD Came a in he Nea In a ed. Fi e De ec ion. 2007;:193-206. 45. Bendiscio P., J F. F ancescon, Malco a P., Malobe F., Pole M., Valacca R.. A CMOS in eg a ed in a ed adia ion de ec o o lame moni o ing. ISCAS. 1998;6:625-628. 46. Leñe o-Ba dallo Juan A., B yn D.H., Hä lige P.. Bio-Inspi ed Asynch onous Pixel E en T icolo Vision Senso . Biomed- ical Ci cui s and Sys ems, IEEE T ansac ions on. 2014;8(3):345-357. 47. Lange R.. 3D Time-o -Fligh dis ance measu emen wi h cus om solid-s a e image senso s in CMOS/CCD- echnology. PhD hesisDepa men o Elec ical Enginee ing and Compu e Science a Uni e si y o Siegen2000. 48. Sze S.M.. Semiconduc o De ices Physics and Technology. AT&T Bell Labo a o ies: Wiley; 1985. 49. Leñe o-Ba dallo Juan A., Ca mona-Galán R., Rod íguez-Vázquez Á.. Li e Demons a ion HDRLVS. In: h ps://www.you ube.com/wa ch? =K dpUpBRD60. 18 JUAN A. LEÑERO-BARDALLO ET AL 50. Leñe o-Ba dallo Juan A., Hä lige P.. A Dual-Ope a ion-Mode Bio-Inspi ed Pixel. Ci cui s and Sys ems II: Exp ess B ie s, IEEE T ansac ions on. 2014;61(11):855-859. 51. Ruedi P.-F., Heim P., Kaess F., e al. A 128 ×128 pixel 120dB dynamic- ange ision-senso chip o image con as and o ien a ion ex ac ion. Solid-S a e Ci cui s, IEEE Jou nal o . 2003;38(12):2325-2333. 52. Ba ba o M., Bu gi P. Y., Mo a a A., Nussbaum P., Hei ge F.. A 100×100 pixel silicon e ina o g adien ex ac ion wi h s ee ing il e capabili ies and empo al ou pu coding. IEEE Jou nal o Solid-S a e Ci cui s. 2002;37(2):160-172. 53. Shoushun Chen, Be mak A.. A bi a ed Time- o-Fi s Spike CMOS Image Senso Wi h On-Chip His og am Equaliza ion. Ve y La ge Scale In eg a ion (VLSI) Sys ems, IEEE T ansac ions on. 2007;15(3):346-357. 54. Son B., Suh Y., Kim S., e al. 4.1 A 640×480 dynamic ision senso wi h a 9µm pixel and 300Meps add ess-e en ep esen a ion. In: 2017 IEEE In e na ional Solid-S a e Ci cui s Con e ence (ISSCC):66-67; 2017. 55. Leñe o-Ba dallo J. A., Rod íguez-Vázquez Á.. Analog Elec onics o Radia ion De ec ion. In: Analog Elec onics o Radia ion De ec ion. 2016 (pp. 47-67). JUAN A. LEÑERO-BARDALLO ET AL 19 TABLE 1 Compa a i e be ween di e en spiking senso amilies Type o senso / Func ionali y Au ho / Yea / Key i s implemen a ion Key pa ame e s Key applica ions Spa ial con as de ec ion Mahowald 1991,1Spa ial con as sensi i - i y Bio-inspi ed modelling, applica ions wi h educed ou pu da a low, su eillance, ea ly scene in e p e a ion DVS senso s Lich s eine 2008,13 Tempo al con as sen- si i i y Da ase s o spiking neu al ne wo ks, a - ic moni o ing, obo ic in eg a ion, domo ic sys ems, s e eo ision Oc opus senso s Culu ciello 2003,29 Dynamic ange and la ency Ligh - o- equency con e sion, sun senso s, lame moni o ing, as colo encoding Hyb id senso s Se e al au ho s 2010,16,14,30 Dynamic ange, la ency, powe consump ion, spa io- empo al con as de ec ion Combina ions o ea u es om di e en senso s amilies, HDR image s, mo ion de ec ion