A programmable VLSI filter architecture for application in real-time vision processing systems

A Programmable VLSI Filter Architecture for Application in Real-Time Vision Processing Systems Teresa Serrano-Gotarredona1, Andreas G. Andreou2, and Bernabé Linares-Barranco1 1Instituto de Microelectrónica de Sevilla (IMSE), Centro Nacional de Microelectrónica (CNM), Ed. CICA, Av. Reina Mercedes s/n 41012 Sevilla, SPAIN. Phone: 34-5-4239923, Fax: 34-5-4231832, E-mail: [email protected] 2Dept. of Electrical and Computer Engineering, The Johns Hopkins University, Baltimore, MD 21218, USA Abstract An architecture is proposed for the realization of real-time edge-extraction filtering operation in an Address-Event-Representation (AER) vision system. Furthermore, the approach is valid for any 2D filtering operation as long as the convolutional kernel F(p,q) is decomposable into an x-axis and a y-axis component, i.e. F(p,q)=H(p)V(q), for some rotated coordinate system {p,q}. If it is possible to find a coordinate system {p,q}, rotated with respect to the absolute coordinate system a certain angle, for which the above decomposition is possible, then the proposed architecture is able to perform the filtering operation for any angle we would like the kernel to be rotated. This is achieved by taking advantage of the AER and manipulating the addresses in real time. The proposed architecture, however, requires one approximation: the product operation between the horizontal component H(p) and vertical component V(q) should be able to be approximated by a signed minimum operation without significant performance degradation. It is shown that for edge-extraction applications this filter does not produce performance degradation. The proposed architecture is intended to be used in a complete vision system known as the Boundary-Contour-System and Feature-Contour- System Vision Model, proposed by Grossberg and collaborators. The present paper proposes the architecture, provides a circuit implementation using MOS transistors operated in weak inversion, and shows behavioral simulation results at the system level operation and electrical simulation and experimental results at the circuit level operation of some critical subcircuits. I. Introduction Human beings have the capability of recognizing objects,figures,andshapeseveniftheyappearembedded within noise, are partially occluded or look distorted. To achieve this, the human vision processing system is structured into a number of massively interconnected neurallayers withfeedforwardandfeedbackconnections among them. Neurons communicate by means of electrical streams of pulses. Each neuron broadcasts its output to a large number of other neurons, which can be inside the same or at different layers, and the way this is doneisthroughphysicalconnectionscalledsynapses. One big problem encountered by engineers when it comes to implement bio-inspired (vision) processing systems is to overcome the massive interconnections. An interesting way of trying to solve this is by Address Even Representation(AER)[1]-[3].InAEReachneuroncodes its activity as a pulse stream signal with very low duty cycle, i.e. pulse width must be minimum but separation between pulses should be fairly large. Each neuron has a code or address, and every time it produces a pulse it will try to write its code on a common digital bus. A receiving system will continuously be reading this bus and send the pulse to those neurons who ought to be connected to the sending neuron. In this manner the activity of a large numberofneuronscan betimemultiplexedon acommon bus. This principle allows to structure hierarchically a very complex neural system. For example, a retina chip with AER output is continuously putting addresses on a bus representing the sensed images. Several chips, each with an AER receiver system, can be reading the same bus, doing some specialized processing and broadcasting the outputs of all their neurons using again AER on another external bus, and so on. Furthermore, extra processing can be added easily while the “addresses” go from one chip to the next. For instance, image rotation or translation can be performed in a straightforward manner by inserting an EEPROM for which the transformation operationhasbeenprogrammedpixelbypixel(oraddress by address). In the architecture proposed in this paper we takeadvantageofthisfacttosimplifytheprocessingchip. As neuroscientists manage to unfold the internal structure and functions of the vision system, it becomes morefeasibleformathematiciansandcomputerscientists toproposeandunderstandbio-inspiredvisionmodelsand algorithms, and for engineers to build bio-inspired artificial vision systems. One powerful vision model proposedrecentlybyGrossbergetal.[5]istheBoundary- Contour-System (BCS) and Feature-Contour-System (FCS) vision model. It consists of nine layers which after local illumination normalization and contrast enhancement of an input image, performs local edge extractionfordifferentspatialorientationsandscales,and then is able to identify consistent long range contours of the shapes in the input image through processing layers with feedforward and feedback connections. In this vision model one of the stages performs a 2D filtering operation for edge extraction, and other stages perform other2Dfiltering operations.Theprocessingarchitecture proposedin thispaper isintended tobeused inthis vision model to perform a simplified version filter doing the edge-extraction operation. The same processing architecturecanbereprogrammedtoperformsomeofthe other2DfiltersneededintheBCS-FCSvisionmodel. Thepresentpaperisstructuredasfollows.Inthenext Section we will briefly describe the structure, functionality, and operations performed by the BCS-FCS vision model. In Section III we introduce modification of the edge-extraction kernel which substitutes a product operation by a minimum operation in the original kernel. Section IV describes briefly the essence of AER, and in Section V we introduce a VLSI architecture capable of implementing a 2D programmable filter. Section VI providessystemlevelbehavioralsimulationresultsofthis architectureprogrammedwithakerneltodoanextraction of vertically oriented edges, and finally Section VII indicatestheconclusionsandfuturework. II. The Boundary-Contour-System and Feature-Contour-System Vision Model Fig. 1 shows a schematic representation of the structureoftheBCS-FCSmodel[5].TheBCSconsistsof several identical subsystems (three in the case of Fig. 1) each of which is tuned for a different spatial scale. Each BCS spatial subsystem consists of 8 layers. Consecutive layers have been drawn in Fig. 1 as connected by thick shaded arrows. We may think of these arrows as the representation of a convolution (or filter) operation applied to the state of the previous layer and resulting in the state of the next layer. For instance, the 2D input image suffers three different filtering or convolutional operations, each of which is the starting point of a BCS subsystem. From here on, each BCS subsystem operates autonomously. From Layer 1toLayer 3 there are only feedforward filtering operations, while Layers 4 to 8 are connectedinafeedback loopconfiguration, whichmeans thesystem willreacha steadystate aftera certainnumber ofiterations(ifthesystemisimplementedsequentiallyon a computer) or after a certain time constant (if the system operates asynchronously and fully parallel, like in biological brains). The outputs of Layers 1 and 5 of the three BCS subsystems are fed to the FCS. Next we will brieflydescribetheprocessing performedonthedifferent layers. A.Stage 1: Center-ON OFF-Surround Letusassume (1) is an N×Minput image provided by a vision sensing front end. This input image is applied to a 2D filter whose impulsive response or kernel of radial symmetry is shown in Fig. 2(a). We can see that pixels close to the center region of the kernel are going to contribute with positive weights to the convolution, while pixels further away will contribute negatively. The result of such a convolution is local illumination normalization and contrast enhancement. The mathematical expression for this kernel is (2) where , are positive parameters, , and controls the spatial scale of the filtering. In the case of Fig. 1 there are three BCS subsystems, which means three Center-ON OFF-Surround filters are applied in parallel to the same input image, each with a specific ( ). From now on the processing in each BCS subsystem is independent. B.Stage 2: Simple Cells The second stage of the BCS system applies an orientation specific convolution for detecting edges oriented within a narrow angle range. This is performed byconvolvingtheoutputofLayer1withthekernelshown in Fig. 2(b) for different orientations. This is why the output of Layer 1 in Fig. 1 suffers several convolutions in parallel, one for each orientation, resulting in as many “Layer 2” as orientations have been considered. The kernel of Fig. 2(b) is mathematically described by the differencebetweentwodisplacedgaussians Layer 6 Layer 7 Layer 2 Layer 3 Layer 4 Layer 1 Layer 5 Layer 8 Input Image Layer 9 Output Image Fig. 1: Schematic Representation of Grossberg’s et al. Boundary Contour System (BCS) and Feature Contour Systems (FCS) Vision Model BCS FCS Ipq,() p1…N,= q1…M,=    S1pq,()A1e p2q2 + σg ----------------- A2e p2q2 + ασg ----------------- –= A1A2 >α1>σ g σg g123,,= (3) where the coordinate system is rotated a certain angle with respect to the coordinate system of the input image , (4) with being the total number of orientations to be considered. C.Stage 3: Complex Cells After applying the filtering of Stage 2, a pixel in Layer2fororientation kwilldisplayahighpositivevalue if the input image presents a positive change in contrast with respect to the k-th orientation axis. If the change in contrast is negative, the output of this pixel would be a highnegativevalue.Inordertodetectwhetherornotthere is an edge at that orientation around the given pixel there is no need to distinguish between positive and negative values. Therefore, the purpose of this processing stage is simplytorectifytheoutputofthepreviousone. D.Stage 4: Hypercomplex Cells, Competition across Space At Layer 3 for orientation k, pixels will present a positive value if around that pixel there is an edge at that orientation. The higher the pixel value, the clearer the edge was. At this stage, and independently for each orientation, a 2D Center-ON OFF-Surround filter is applied to contrast enhance the previous image. This is equivalent to performing a spatial competition among pixels,favoringthosewithhighervalues. E.Stage 5: Hypercomplex Cells, Competition across Orientations At this stage, all Layer 4 pixels of the same spatial position but for all possible orientations k, are going to compete among them to contrast enhance those orientations with higher pixel values. This is done by applying a 1D Center-ON OFF-Surround filter to pixels of Layer 4 of the same spatial position but for all k orientationvalues. F.Stage 6: Bipole Cells, Long-Range Cooperation Theoperationofthisstageisthemostcomplicated.It tries to identify long term “Contours”, which can be defined as edges that remain consistent over larger space ranges. This is achieved by performing for each orientationkthefollowingsumofconvolutions, (5) where is the resulting state of pixel of Layer 6 for spatial scale gand orientation k, subscript r denotes orientation, and each convolution is given by (6) where denotes the state of pixel of Layer 5 for orientation r, denotes orientation perpendicular to r, and the kernel is defined by (7) with , , and being positive parameters. Fig. 2(c) depicts this kernel for the case . G.Stage 7: Hypercomplex Cells, Competition across Space ThisstageperformsthesameoperationthanStage4. (a) Fig. 2: Convolutional Kernels used in the BCS system of Fig. 1: (a) Center-ON OFF Surround Kernel used by Stages 1, 4, and 7. (b)Edge-Extraction Kernel used by Stage 2. (c) Bipole Kernel used by Stage 6. (b) (c) 010 20 30 40 50 60 70 0 20 40 60 80 −0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 010 20 30 40 50 60 70 0 20 40 60 80 −0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 010 20 30 40 50 60 70 0 20 40 60 80 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Fgpkqk ,() 1 2πσghσgv ------------------------ e 1 2 ---pk σgh --------   2 – e 1 2 ---qk σgv --------1 2 ---+   2 – e 1 2 ---qk σgv --------1 2 ---–   2 – –= pkqk ,() pq,() pkpπk nR ------cos qπk nR ------sin–= qkpπk nR ------sin qπk nR ------cos+= nR Apqk gCpqkr,,,() r1= nR ∑ = Apqk gpq,() Cpqkr,,,()yrpq,()yr ˆpq,()–[]Zrpkqk ,()⊗= yrpq,() pq,() r ˆ Zrpkqk ,() pk ()sgn eβpk 2qk 2 +()–eµqk/pk 2 () 2 –×= kr–()π nR ------------------- pk ()sgn 2 2qk pk -------- pk ,   atan–    cos× βµ γ kr0== H.Stage 8: Hypercomplex Cells, Competition across Orientations Thisstageperforms thesame operationthan Stage 5. The output of Stage 8, Layer 8,is combined with the output of Stage 3, Layer 3,to form the input image for Stage 4. This way a feedback loop is formed, which once settled will yield the proper output of each BCS subsystem. I.Stage 9: Feature Contour System (FCS) The information about consistent long range contours can be taken from Layer 5 (once the feedback loop has settled), for all computed orientations. The FCS takes the original (local illumination normalized and contrastenhanced)imagepresentinLayer1andperforms a selective diffusion operation between pixels, using the contour information present at Layer 5: the contours at Layer 5 act as barriers to the diffusion operation. The result of all this processing is a clean noise-free image withclearandconsistentlongrangecontours. III. An Edge-Extraction Filter Intherestofthis paperwewill concentrateonStages 2 and 3, the filter for edge-extraction and subsequent rectification.Wewillfirstintroduceasimplificationofthe kernel of eq. (3) which will allow us to propose a very compact and efficient hardware that takes advantage of theAERaswell. The kernel of eq. (3) is decomposable into two factors, each of which depends only on either the xcoordinate orthey-coordinate , , (8) with (9) The simplification proposed here consists in substituting the product operation between and by the signed minimum, (10) Fig. 3(b) shows the result of applying the filtering of eq. (8) to the input image of Fig. 3(a), while Fig. 3(c) results when using the kernel of eq. (10). As can be seen there is no appreciable difference in the resulting images. To evaluate quantitatively the effect of the proposed approximation we can use the Normalized Square Error definedas, . (11) This quantity helps us to evaluate the difference between the original kernel and the modified kernel obtained when the product operation is substituted by the signed minimum. Table 1 gives the computed NSE for several kernels. All kernels in Table 1 are decomposable in the product of two functions that depend separately on the and components. IV. Using Address Event Representation (AER) Fig.4 shows aschematicfigure outliningtheessence behind the AER. Suppose we have an “emitter” chip containingalargenumberofneuronsorcellsD1,D2,D3, ... whose activity changes in time with a “relatively slow” time constant. For example, if Chip 1 is a retina chip and each neuron’s activity represents the illumination sensed by a pixel, the time constant with which this activity pkqk Fgpkqk ,() 1 2π ------ Hgpk ()Vgqk ()= Hgpk () 1 σgh -------- e 1 2 ---pk σgk --------   2 – = Vgqk () 1 σgv -------- e 1 2 ---– qk σgv --------1 2 ---+   2 e 1 2 ---– qk σgv --------1 2 ---–   2 –= Hg.() Vg.() Fg'pkqk ,() 1 2π ------ Hgpk ()[]sgn Vgqk ()[]×sgn= min Hgpk ()Vgqk (),{}× NSE Fxy,()Fmxy,()–2xdyd ∫∫ Fxy,() 2xdyd ∫∫ -----------------------------------------------------------------------= 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 100 200 300 400 500 600 50 100 150 200 250 300 350 400 450 (a) (b) Fig. 3: Behavioral Simulation Results when performing 2D Filtering for vertical edge-extraction. (a) Input Image, (b) using the edge-extraction filter kernel of eq. (8), (c) using the modified filter kernel of eq. (10) (c) Fxy,() Fmxy,() xy changesis,atthemost,equivalenttoFrame-Rate(i.e.,25- 30 changes per second or a time constant of about 30- 40ms). The purpose of an AER based communication scheme is to be able to reproduce the time evolution of each neuron’s activity inside a second or “receiver” chip, usinga fastdigital buswitha smallnumber ofpins. Inthe “emitter” chip the activity of each pixel has to be transformed into a pulse stream signal such that pulse width is minimum and the spacing between pulses is reasonably high to time multiplex the activity of a relatively large number of neurons. Every time a neuron produces a pulse its address or code should be written on the bus. For the case more than one pulses are produced simultaneously by several neurons, a classical arbitration tree can be introduced [1]-[3], or one based in Winner- Take-All(WTA) row-wise competitions[6], orsimplyby making no neuron accessing the bus in case of a “collision”[7].Whatevermethodisusedtheresultwillbe the presence of a continuous sequence of addresses or codes on the digital bus that one or more receiver chips can read. Each receiver chip must contain a decoding circuitry so that a pulse reaches the neuron (or neurons) specified by the address read on the bus. If each neuron integrates the sequence of pulses properly, the original activity of the neurons in the emitter chip will be reproduced.NotethatinAERthoseneuronsthataremore active access the bus more frequently. This property allows to optimize the use of the bus, since neurons with low activity will not consume much communication bandwidth. This is the simplest AER based communication schemeamong chips.However, AERallowseasily toadd morecomplicatedprocessing. Forexample, input images can be translated or rotated by remapping the addresses while they travel from one chip to the next. By properly programminganEEPROMasalook-uptableanyaddress remapping can be implemented, by simply inserting the EEPROM between the two chips. Furthermore, many EEPROMs can be connected in parallel each performing, for example, a rotation at a specific angle, and each delivering the remapped addresses to a set of specialized processing chips. It is also possible to include synaptic weightingbyhavingtheEEPROMstoretheweightvalue, dumping it on a data bus, have the “receiver” chip read boththeaddressandthedatabus,andperformaweighted integration in the destination(s) neuron(s). It is also possible to implement “projective fields”, i.e. for every address that appears on the bus a small digital system couldgenerateasequenceofaddressesarounditandsend ittothe“receiver”chip.Thiswouldbeatime-multiplexed projection field generation. In the architecture proposed in this paper, we implement a synaptically weighted projection field for each address read on the bus, and not in a time-multiplexed manner but in parallel. As we will see, the receiver chip will perform the following operations: for every address read on the bus it will send pulses to a bubble of neurons around that address. The width of those pulses is modulated according to some weights stored on chip. Time integration of those pulses for the complete array of neurons in the receiver chip implements a convolution operation. In the rest of the paper we will concentrate on describing the circuits able toimplementsuchaconvolutionalorfilteringchip. V. System Design Fig. 5 shows the basic operating principle of the proposed architecture. The address bus provides the coordinates oftheneuron(orpixel)aroundwhich the kernel of eq. (10) should be applied. Pulses will be applied to all rows with y-coordinate in the interval , and all columns with x-coordinate in the interval , where is the width consideredforthekernel. Table 1 kernel parameters NSE (dB) Gaussian -24.92 Even Gabor -19.04 Odd Gabor -19.03 Displaced Gaussians -22.73 Fxy,()Hx()Vy()= e 1 2 ---– x σx -----   2 e 1 2 ---– y σy -----   2σx10= σy15= e 1 2 ---– x σ ---   2 e 1 2 ---– y σ ---   2 2πy ys ----   sin σ15= ys20= e 1 2 ---– x σ ---   2 e 1 2 ---– y σ ---   2 2πy ys ----   cos σ15= ys20= e 1 2 ---– x σx -----   2 e 1 2 ---– yy s – σy -------------   2 e 1 2 ---– yy s + σy -------------   2 –     σx15= σy5= ys5= D1 D2 D3 CHIP1 ARBITER+ENCODER DIGITAL BUS DECODER D1 D2 D3 CHIP2 Fig. 4: Address Even Representation Interchip Communication Scheme x0y0 ,() y0L–y0L+,[] x0L–x0L+,[]2L1+ Pulses will be modulated in width according to function (see eq. (8)) for the rows, and function for the columns. At each pixel there is an AND gate which provides a pulse of width equal to the minimumof and .Thispulsewillgeneratea fixed magnitude current pulse of the same width which will be integrated on a capacitor. Each pixel contains two integrators. One of them, called the “positive integrator”, integrates the pulse of length when ; while the other, called the “negative integrator”, integrates the pulse when . The values of and ( ) are stored digitally on chip on a smallRAM. Fig. 6 shows the floorplan diagram of the system. It consists of two input decoders that decode the address of the arriving pulses, a C-element required for the AER communication protocol [1]-[3], an array of integrator cells , two sets of programmable monostables and whose pulse widths are controlled by the bits stored in two RAMs, and (which store the digital words and , respectively), two arrays of and selectingcells and , respectively, two output decoders to select the min H x() Vy(),() Vy() Hx() Hx() y0 x0 x0 BUS X BUS Y y0 Fig. 5: Schematic Representation of the Basic Operation Principle behind the proposed Architecture Vy() Vy() Hx() Vy() Hx() min H x() Vy(),() Hx()()sgn Vy()()sgn 0> Hx()()sgn Vy()()sgn 0<Vx() Hy() xy,L…0…L,,–= c1M c11 cNM cN1 cij x1xN xi Px+1Px-1Px+iPx-iPx+NPx-N Mx-L MxL Tx-L Sx-L TxL SxL My-L MyL Ty-L Sy-L TyL SyL yM yj y1 Py+M Py-M Py+j Py-j Py+1 Py-1 Ack Rqst Vmonob C-element Latch X Latch Y Input Decoder X Input Decoder Y IM+ + IM- Ij+ Ij- I1+ I1- ScanM Scanj Scan1 Io Output Decoder X Output Decoder Y Scx1ScxN Scxi ScyM Scyj Scy1 Cx1,-L Cx1L CxN,-L CxNL Latch Rqstm Rqst Ack Latch Rqst Ack Mys Mxl CyM,-L CyM,L Cy1,-L Cyj0 Cxi0 Rqstm RyL Ry-L Rys Rxl Rx-L RxL Random Scan Bus 8 4 4 Address Bus 4 4 8 4 4 Cxi-l,l Cyj-s,s Cy1,-L RAM Y RAM X Fig. 6: Floorplan of Complete 2D Filtering System Tys Sys Txl Sxl NM× cijMx L–... Mx0... MxL ,, ,, My L–... My0... MyL ,, ,, RAM X RAM Y Rx L–... Rx0... RxL ,, ,, Ry L–... Ry0... RyL ,, ,, 2L1+()N×2L1+()M×Cxil–l, Cyjs–s, cells to be scanned, and a column of scanning circuits toreadouttheintegratorsanalogoutputcurrent . Note that in the present prototype of Fig. 6 the system does not generate an AER output. This can be solved by either adding the necessary circuitry to each pixel [1]-[3] which will decrease the cell density, or by adding a postprocessing chip that scans sequentially all cells in the arrayofFig.6andgeneratesanAERoutput. The operation of the system in Fig. 6 is as follows. In and digitalwordsof bitsarestored ( and ). The first bit (or )indicatesthesignofthefunction (or ). The following bits indicate the absolute value (or ).These bitslinearlycontrolthelength of the pulse triggered by monostables (or ). The pulses generated by the monostables are sent through lines (or )andaretriggered wheneveran external pulsearrives tothe system(whenever signalRqst pulses). When an external pulse arrives, the input decoders activate lines and corresponding to the address of the arriving pulse. The selection cells controlled by (cells in Fig. 6, ) connect the pulse in line to line if the sign bit is ‘1’. If thesignbit is‘0’line isconnectedto thenegative line .Thisway,pulses (or )aresentthrough lines or ( or )dependingonthe signoftheweightstoredin (or ). Each neuron has two integrators. The positive integrator accumulates charge when pulses are simultaneously arriving through horizontal and vertical lines of the same sign. That is, it integrates a pulse when lines and (or lines and ) are simultaneously high, or equivalently it performs the operation . Hence, the positive integrator in cell computes along time the followingsum (12) where , , is the (lossy) integral over time of the number of pulses pixel is receiving, and is the fixed magnitude of the current pulses being integrated. Similarly,thenegativeintegratoraccumulatescharge whenpulsesarrivingthroughhorizontalandverticallines of opposite sign and (or and ) are simultaneously high, that is, it performs the operation . Hence, along time it computesthefollowingsum (13) Consequently, the difference between the outputs of the positive and negative integrators is given by, , (14) which is the filter operation we want to implement. In what follows we will describe the circuit components and operations of each block in Fig. 6. A.Communication Protocol: The C-element To perform a proper communication between two chips a communication protocol must be implemented [1]-[3]. In the AER scheme, the sender chip indicates when the address of a pulsing neuron is ready on the bus andthereceiverchipmustacknowledgethatthepulsehas beenreceivedandthatitisreadytoreceiveanewpulse. Fig. 7 shows the timing diagram of a valid communication protocol for the two chips. The sender chip generates a request signal and the receiver generates an acknowledge signal . When the sender has put the address on the bus it pulls the request signal to a high value. Once the receiver detects a high signal it latches the received address and pulls the acknowledge signal high. The sender can put now low and begin to process the following pulse. The receiver must wait until all the monostables have sent their pulses to the corresponding neurons and the has gone low to put the signal low. Once the signal is low the sender can activate the signal to a high value again. Fig. 8 shows the schematic of the cell used in the receiver chip to generate the signal. This cell is known as “C-element”. This element receives two input signals: a request signal generated by the sender system, and signal which is the wired- NOR of all the monostable output pulses . The C-element generates an output acknowledge signal which is sent back to the sender system. When no pulses are being received, is low. Signal is high as no pulses are being ScanjIo RAM X RAM Y n 1+ Rx L–... Rxl... RxL ,, ,, Ry L–... Rys... RyL ,, ,, SxlSysHx() Vy() n Hx() Vy() nMxlMys TxlTys xiyjxi Cxil–l,lL…0…L,–= TxlPxil– +Sxl SxlTxl Pxil– -TxlTys Pxil– +Pxil– -Pyjs– +Pyjs– - RxlRys cij Pxi +Pyj +Pxi -Pyj - Pxi +Pyj + ∩()Pxi -Pyj - ∩()∪cij Iij +Iwmin H(xpi)V(yqj),()npq pq, Hx pi ()()sgn Vy qj ()()sgn 0> ∑ = xpi xpxi –= yqj yqyj –= npq xpyq ,() Iw Pxi +Pyj -Pxi -Pyj + Pxi +Pyj - ∩()Pxi -Pyj + ∩()∪ Iij -Iwmin H(xpi)V(yqj),()npq pq, Hx pi ()()sgn Vy qj ()()sgn 0< ∑ = IwHx pi ()()sgn Vy qj ()()sgn min H(xpi)V(yqj),()npq pq, ∑ CHIP1 CHIP2 Address Rqst Ack t Rqst t t Address Bus Pulse Ack Sender Receiver Fig. 7: Timing diagram of the address-event communication protocol Bus Rqst Ack Rqst Rqst Ack Rqst Rqst Ack Ack Rqst Ack Rqst Vmonob Tx L–…TxLTy L–…TyL ,, , ,, Ack Rqst Vmonob generated by the monostables. Consequently, the signal is low. When a valid address pulse arrives the sender puts the signal high. The rising edge of this signal is used to trigger the monostables so that signal becomes low. Once is high and has been set to low the C-element sets to a high value, meaning that the pulse has been received. The high value of is used to latch the present bus address and signal that triggers the monostables. Latching the address assures that the corresponding neighborhood is kept selected until all monostables finish their pulses. By latching we assure that the monostable pulses do not end if signal goes low before the monostable pulses have finished.TheC-elementwaitsuntil goeslowandall the monostable pulses finish ( ) to put the signal low again. Once the acknowledge is low the sender is allowed to pull up again and a new communicationcyclecanbegin. B.The Monostables The schematic of a monostable with controlling bits is shown in Fig. 9. Transistors and are equally sized, as well as transistors and . Switches are controlled by a digital n-bit word that set the capacitance connected to node . When no address is being received, and are both low and hence is also low. Transistor is cut off so that node is low. Node is also set low through those transistors with a high value. If all bits are low will always be high (by ) and no pulse will be generated. When an input pulse arrives signal and hence become high. As soon as goeshighnode goeshigh.Current begins to flow through the switch formed by transistors and charging node at the rate set by bits . When node reaches voltage value , the current through transistor becomes higher than the current supplied by so that the output node flips from high to low. The length of the pulse at is the time taken by current to charge node up to a voltage of .Thistimeisgivenby (15) where is the total capacitance present at node and is set by the bits stored in the corresponding RAM word or . With this scheme, the length of the monostable pulses is linearly controlled between 0 and , with being the number of bits controlling each monostable pulse length, and the unit capacitance in Fig. 9. Fig. 10 depicts the pulse widths obtained with Hspice versus the value of the digital control word, for a monostable controlled by bits. For this simulation, values of , and were used. C.The Selection Cell Fig. 11 depicts the schematic of the selection cell (or ) used to select the neighborhood of cells where the monostable pulses have to be sent. Each selectioncell consistsof twoNANDgatescontrollingthe gates of two PMOS transistors ( and ) that behave like switches, and two NMOS pull down transistors ( and with a constant gate voltage ).Eachselectioncell(forexample, inFig.6) hastwocontrolsignals(thedecoderoutput andthesign bit from ), one input signal (the monostable output ) and two outputs ( and ). When a pulse arrives with address , it activates the decoders output and , respectively. The decoder output controls all the selection cells with . When is high, if the sign bit is ‘1’, theselection cell connectsthe monostableoutput line to the positive line . If the sign bit is ‘0’, line is connected to the negative line . The sameisvalidforthe coordinateselectioncells. D.The Core Cell Theschematicofcell ofFig.6isshowninFig.12. It consists of two diode-capacitor integrators [3]. The Rqst Vmonob Ack Ack Fig. 8: Schematic of the C-element used for the arbitration of the address-event inputs Ack Rqst Rqst Vmonob Rqst Vmonob Ack Rqst Ack Rqstm Rqstm Rqst Rqst Vmonob 1= Ack Rqst n Rqstm Rqstm Rqstm Rqstm Rqstm Vthm b1bn Vm IT M1 M2 M5 M3 M4 Vout Fig. 9: Schematic of a Monostable Cell S1Sn 20Cu2n-1Cu Mb1Mbn M6 M7 M8 OR{bi}VT M1M2 M3M4 S1…Sn ,, b1…bn ,, Vm Ack Rqst Rqstm M5 Vout Vm Mbibi biVmM8 Rqst Rqstm Rqstm Vout ITM6 M7Vmbi VmVthm M2 M1Vout Vout ITVm Vthm TCmono IT --------------- Vthm = Cmono Vm b1…bn ,,{} RxlRys 2n1–()CuVthm/ITn Cu 0 5 10 15 20 25 30 35 0 50 100 150 200 250 Fig. 10: Monostables pulse length expressed in nanoseconds versus value of controlling digital word obtained through Hspice simulation Pulse Width (ns) n5= Cu0.2pF=IT75µA=Vthm 2.5V= Cxil–l,Cyis–s, MP+MPMN+MNVPD Cxil–l, xi SxlRAM X TxlPxil– +Pxil– - xiyj ,() xiyj xiCxil–l, lL–…L,,[]∈xiSxl Cxil–l, TxlPxil– +Sxl TxlPxil– - Y cij positive integrator integrates the ANDED pulses that arrive in row and column lines with the same sign, that is, and (or and ). The negative integrator integrates the ANDED pulses that arrive in row and column lines of opposite sign, that is, and (or and ). Each diode-capacitor integrator consists oftwotransistors and ( and )operatingin the subthreshold region, a capacitor and a transistor (or ) acting as a current source of value (controlled by bias voltage ) with its source pulsed by theoutputof theNORgate.The inputandoutputcurrents and of the positive integrator are related to the voltage at node through the following differential equations(thetreatmentfor thenegativeintegratorwould bethesameforcurrents , andvoltage )[3], (16) (17) where is the thermal voltage and , are model parameters of the MOS transistor operating in the subthreshold region. From eqs. (16) and (17) we can get an expression that relates the output and input currents of the diode , (18) where (19) Note that current mirror gain Ais controlled by voltage . During the time in which and (or and ) are simultaneously high, the source of transistor is low, and this transistor is acting as a current source sinking a constant current from the integration node . In this case in eq. (18). Suppose that a train of pulses of constant frequency , pulse width and interspike interval (as depicted in Fig. 13) is applied simultaneously to lines and (or and ). Integrating equation (18)from to with ,resultsin , (20) where the integrator time constant is given by . When the ANDED pulses are zero, the source of transistor becomes high and . If the pulses go low at time and stay low for an interspike time (see Fig. 13), the output current at time just before a new pulse is applied, is given by . (21) When pulses of width are applied at a constant frequency as shown in Fig. 13, a steady state is reached in which the charge injected by the diode during the inactive periods equals the charge sank by the xi SxlTxl Pxi-l+ Pxi-l- VPD VPD Fig. 11: Schematic of a neighborhood selection cell MP+ MPMN+ MN- Pxi+ Pxi- Pyj+ Pyj- Vw VA Scxi Pxi+ Pxi- Pyj- Pyj+ Vw VA ScxiIj- VREF C C Iij- Fig. 12: Schematic of the Core Cell with the two diode-capacitor integrators vgvg+ Mw+ Mw- M1+M2+ M1-M2- Iin+ Iin- VREF Iij+ Ij+ Pxi +Pyj +Pxi -Pyj _ Pxi +Pyj - Pxi -Pyj + M1 +M2 +M1 -M2 - C Mw +Mw -Iw Vw Iin +Iij + vg + Iin -Iij -vg - Iin +Cvg + d td -------- –Iop VAκvg + – vt ----------------------    exp+= Iij +Iop Vdd κvg + – vt -------------------------    exp= vtIop κ QT Iij + d td -------- Iij +Iin +1 A --- Iij + –   = AVdd VA – vt -----------------------   exp= QT Cvt κ --------= VAPx+iPy+jPx-i Py-jMw+ Iw vg +Iin +Iw = 1T⁄ThTl Px+iPy+jPx-iPy-j t Pxi+ t1t2t2+Tl ThTl T nT (n+1)T Fig. 13: Timing diagram of the pulses applied to lines , , orPxi +Pxi -Pyj +Pyj - t1t2Iin +Iw = 1 Iij +t1Th +() ----------------------------1 AIw ----------1 Iij +t1 () ---------------1 AIw ----------–   Th τ ------–   exp+= τCvt/κIw = Mw+Iin +0= t2 Tl t2Tl + 1 Iij +t2Tl +() ---------------------------1 Iij +t2 () ---------------Tl AQT -----------+= Th 1T⁄