Asynchronous spiking pixel with programmable sensitivity to illumination

1 Asynchronous spiking pixel with programmable sensitivity to illumination Juan A. Le˜ nero-Bardallo, Manuel Delgado-Restituto, Senior Member, IEEE, Ricardo Carmona-Gal´ an, Senior Member, IEEE, and ´ Angel Rodr´ ıguez-V´ azquez, Fellow, IEEE Abstract— A spiking pixel to be used in image sensor arrays for asynchronous frame-based operation is presented. The pixel features both local and global adaptive sensitivity to the illumination level. Local adaptation is performed by adjusting the voltage stored in an embedded analog memory according to the average illumination within a neighborhood. Global adaptation to the overall illumination of the array is implemented by adjusting a voltage value common to all the pixels. These programming capabilities allow full control on the sensor sensitivity, pixel output data flow, and energy consumption, thus, ovecoming the limitations oberved in current image sensors based on spiking pixels. Experimental results validate the functionality of the proposal. Index terms – Integrate and fire pixels, AER, Light adaptation, Image sensors, Spiking pixels. 1 I. INTRODUCTION Asynchronous luminance spiking sensors [1]–[3] consist of pixels that fire with a frequency proportional to the illumination level. Such systems can outperform classic frame-based image sensors in terms of dynamic range, latency and bandwidth consumption and, accordingly, they target application scenarios where these features are preferred over image quality [4], [5]. Current spiking sensors are not able to locally adapt to light, as occurs in biological vision systems [6], [7]. On the contrary, they can only be globally adjusted by setting a common bias parameter. This, however, poses a trade-off between sensitivity and the amount of data to be handled. In order to detect the spiking activity of pixels exposed to low illumination levels, the overall system sensitivity should be large thus leading to large spiking rates in pixels operating under high illumination conditions. In medium and large pixel arrays, the total event rate which can be processed by the readout circuitry is limited (e.g., modern asynchronous arbitration systems can tolerate spiking rates in the order of 20Meps [8]) and, therefore, the global sensor sensitivity to light cannot be made arbitrarily large to extend the dynamic range and expose darker regions. 1This work was supported in part by the University of C´ adiz under Grant PR2016-072; in part by the Spanish Ministry of Economy and Competitiveness under Grant TEC2015-66878-C3-1-R TEC2012-33634 (CLEPSYDRA); in part by the Spanish Ministry of Economy and Competitiveness under Grant TEC2015-66878-C3-1-R, Co-Funded by ERDF-FEDER; in part by Junta de Andaluc´ ıa CEICE under Grant TIC 2012-2338 (SMARTCIS-3D); and in part by ONR under Grant N000141410355 (HCELLVIS) Juan A. Le˜ nero-Bardallo, Manuel Delgado-Restituto, Ricardo Carmona-Gal´ an, and ´ Angel Rodr´ ıguez-V´ azquez are with the Institute of Microelectronics of Seville (IMSE-CNM), CSIC-Universidad de Sevilla, Spain (E-mails: {juanle,mandel,rcarmona,angel}@imse-cnm.csic.es). To overcome this trade-off, it would be desirable to locally adapt the sensitivity of low illuminated pixels to reduce their latency without increasing the firing activity of pixels under higher illumination values. Further, it would be also beneficial to keep the possibility of global adaptation to control the total data throughout and the bandwidth and power consumption requirements. Emerging 3D fabrication technologies facilitate the design of pixels with processing capabilities on the focal plane and allow higher programmability without degrading the pixel fill factor [9], [10]. In this scenario, analog memories are key components to store local pixel parameters or to incorporate on-chip intelligence [11]. Memories can be placed on tiers below/above the sensing photodiode without degrading the fill factor. The reported pixel, with an in-pixel analog memory, is an example of how these novel technologies could be exploited efficiently to make event-driven luminance sensors more competitive. In this article, we analyze the performance of a spiking pixel that implements global and local adaptation to light intensity. Local adaptation is accomplished by storing a voltage reference on an in-pixel memory implemented with a floating diffusion capacitance [11]. The presented pixel is intended for a new generation of image sensors that combine framebased processing with asynchronous operation to increase dynamic range [3]. In this paper, we focus on the asynchronous operation and the local and global adaptation to light intensity. Experimental results obtained with a prototype pixel implemented in a 180nm HV fabrication process are reported. These results validate the proposed approach. II. PIXEL’SOPERATION Fig. 1 shows the proposed pixel architecture. Main building blocks are highlighted with different colors. One block (surrounded with a red dashed line) performs asynchronous light-to-frequency conversion. It generate spikes, i.e., trains of pulses with a frequency proportional to illumination. Another block (surrounded with a green dashed line) is used to store the integration voltage, Vc(t), on the floating diffusion capacitance FD2, whenever the control signal SH is enabled. Finally, the circuit surrounded with a blue dashed line is an asynchronous logic block to read the pixel outputs. The pixel has two possible operation modes: High Dynamic Range (HDR) and octopus [3]. In both cases, the pixel operation starts after a global reset period during which the voltage in the integration capacitance C1is cleared. 2 _req_y _req_x _ack_y Mn8 Mn10 Mn9 C2Mn7 reset_x reset_y Reset_arb Mn6 Mp1 MEM C V c Global Reset V ref V reset SH FD SEL_Pixel FD 1 Mn1 Mn2 Mn3 Mn4 Mn5 spike V pix Analog frame-based readout Asynchronous readout logic Light-to-frequency conversion stage Vinp Vinn Vinp V inn Vboost 1 2 CS j RS i Vbias Vbias Fig. 1. Pixel’s schematics. Main functional blocks are highlighted. The blocks surrounded by a red and a blue dash line are involved in spike generation and spike communication, respectively. The block surrounded by a green dash line is activated to operate in the HDR mode. It is used to read the voltage at the integration capacitance C1, at the end of the integration period, Tint. Transistor sizes are (W/L, µm/µm): Mn1=Mn2=Mn3=Mn5=Mn6=Mn9=Mn10=0.5/0.7, Mn4=1/0.5, Mp1=0.5/1, Mn8=1/0.7, C1=35fF, FD1= 45fF, FD1=C2=30f. In the HDR mode, operation is defined by an user-defined integration time, Tint, as shown in Fig. 2. During this period, the number of pixel over-exposures (spikes) are transmitted and stored off-chip. To assure good linearity, voltage Vreset should be higher than the transistor threshold voltage, VT, in the selected technology. Once the integration time is over, signal SH is enabled and voltage Vc(Tint)is stored on the floating diffusion capacitance F D2. A source follower, formed by transistor Mn4and a biasing transistor shared by columns in a prospective pixel array (not shown in Fig. 1), buffers the voltage at F D2and obtains the analog output Vpix, whenever the pixel is selected. Finally, voltage Vpix is digitized with a ramp ADC shared by all the pixels per column, as done in a prior implementation using the same technology [3]. Illumination values can be encoded by linearly combining the result of the ADC conversion (less significant bits) and the number of generated spikes (more significant bits). In the octopus mode, there is no defined integration time and the control signal SH is always disabled. Hence, circuit operation is continuous an the pixel spikes asynchronously with a frequency proportional to light intensity. Analyzing the light- to-frequency conversion block, it can be easily demonstrated that the spiking frequency is given by: fosc ≈Iph C1·(Vreset −Vref )=Iph C1·∆V(1) where C1is the integration capacitance and Iph is the photodiode current. In this mode, images are encoded taking into account the average spiking frequency within an observation interval. To transmit the spikes, the asynchronous communication Address Event Representation (AER) protocol is used [12], [13]. The in-pixel asynchronous circuitry needed to transmit spikes is shown in Fig. 1 (block surrounded by a blue dashed line). This circuit was already implemented in V(t) t V (Local Adaption) ref V (Global Adaption) reset V pix Tint t spike I /C ph t SH V c Global Reset 1 Fig. 2. Timing diagram of the pixel operating in the HDR mode. Initially, signal Global Reset is enabled to reset the voltages at C1and FD2. Then, the voltage Vc(t)rises with a slope proportional to illumination. Events are transmitted asynchronously every time that Vc(t)reaches the value Vreset. When signal SH is on, voltage Vc(t)is stored on FD2(analog readout). prior designs [3], [14], [15] in conjunction with an external arbitration circuitry to handle the entire pixel array [16]. In this study, the focus is on the control of the spiking frequency of the light-to-frequency conversion block (module surrounded by a red dashed line in Fig. 1) assuming that pixel operation is asynchronous and continuous. Hence, unless otherwise stated, it is assumed the octopus operation mode. 3 The block has been devised to allow both local and global adaption to light intensity. After monitoring the overall system event rate, global adaptation is implemented by tuning the analog voltage Vreset, shared by all the pixels of a prospective pixel array. By increasing its value, the spiking frequency decreases, thus avoiding the risk of event overrun, at the expense of increasing the sensor latency globally. As shown in Fig. 2, the maximum value of signal Vc(t)is set by Vreset. In some situations, local adaptation can be found useful to reduce the pixel latency of low illuminated regions. This can be done by controlling the analog voltage Vref . This voltage is stored in an in-pixel analog memory which is implemented with a floating diffusion capacitance (FD1). The higher the voltage Vref , the lower the pixel latency. With the dedicated peripheral circuitry (shown in Section III), different values of Vref can be set on each pixel. The target is to equalize the pixel array latency dynamically. Pixels with lower spike rates can be tuned to provide faster response to illumination. As will be shown, the tuning procedure is implemented with an adhoc synchronous logic, without interrupting the asynchronous in-pixel light-to-frequency conversion. Fig. 3 shows the schematic of the OTA used in Fig. 1. It is a two-stage Miller compensated OTA. Transistor Mn4boots the bias current of the OTA if the pixel is about to reset the integrator (spike signal becomes active). The comparator in Fig. 1 is implemented in a similar manner, but the Miller capacitance and the boost transistor are eliminated to save pixel area. The boosting mechanism operates as follows: if the voltage at the integration capacitance C1is below the voltage threshold Vreset, the boost transistor is disabled and the OTA is biased with a small current of 90nA to minimize the static power consumption. If the voltage at the integration capacitance reaches the voltage threshold Vreset, the boost transistor is temporarily on until the asynchronous readout logic enables transistor Mn2. The arbitration hand-shaking cycle lasts typically below 100ns [8] since the signal spike becomes active. Hence, the boost transistor is active during a short amount of time, just to speed up the reset of the integration capacitance. III. PIXEL’SCONTROL CIRCUITRY Fig. 4 shows the circuit used to locally program the reference voltage, Vref , per pixel. The pixel matrix readout circuitry is not displayed for simplicity. Circuit operation is controlled by an external FPGA or microntroller attached to the sensor. On the bottom, a column selection block sequentially addreses the columns CSj,j= 0, . . . , N −1, where Nis the number of columns of the sensor array. Only one column can be enabled at a time. The row selection block on the left addresses the rows RSi,i= 0, . . . , M −1, where Mis the number of rows of the sensor array. In this case, different rows can be activated simultaneously. The Vref voltages are generated by a K-bit DAC controlled by the external FPGA or microcontroller. This latter is also responsible to define the Vref value per pixel, depending on the desired sensitivity to light and data throughput. For each V+ Mp1 Mp4 Mp2 Mp3 Mn1 Mn2 Mn3 Vbias V-V out Cm Vboost Mn4 Fig. 3. Schematic of the in-pixel OTA. Transistor Mn4increases the bias current when the pixel is being self-reset. Cmand Mn4are not present in the comparator’s design. Transistor sizes are (W/L, µm/µm): Mp1=Mp4= 1/1,Mp2=Mp3=Mn1=Mn2=Mn4= 1/0.7.Mn3= 3/0.7. The Miller capacitance amounts Cm= 40fF. DAC V ref V ref V ref V ref Column Sel CS0 CS1 CSN-2 CSN-1 Row Sel RS0 RS1 RSM-2 RSM-1 CLK Reset CLK Reset SDRAM CLK Reset store FPGA/ Microcontroller DAC Control Data Display CLK Reset store Off-chip On-chip Fig. 4. Pixel matrix and digital circuitry devised to program the pixel values of the Vref voltage. The column selector block selects sequentially the pixel columns. The row selection block selects simultaneously all the pixels that operate with the same Vref value (red lines in the figure). The Vref values that correspond to each pixel are stored on-chip on a SDRAM memory. All the possible values of Vref are sequentially generated with a DAC. selected column, all the possible values of Vref are swept sequentially. The row selection block activates those rows for which pixels operate with the same Vref reference. A SDRAM memory stores the DAC programming words associated to the different Vref values. This information is transferred to the external FPGA/microcontroller and it is updated every time that a new column is selected. The programming procedure is illustrated in the timing diagram of Fig. 5. First, the pixels of the first column, j= 0, are selected by activating the signal CS0. While CS0is on, the different values of Vref are swept and set by the DAC. Signals RSiare active whenever the Vref value that corresponds to the pixel (i, 0) is set. The same operation is repeated column 4 Reset CLK store CS CS CS DAC control Vref 0 1 Nval 0 1 Nval 01Nval 0 T =400µs col T =N ·T refresh col col 0 1 N -1 col Fig. 5. Time sequence with the digital signals involved in the programming of the different Vref voltages. The amount of time to refresh or program all the Vref values of a pixel column is Tcol. by column until all the in-pixel FD1memories are updated. A resistor-string architecture is proposed for the DAC. It is shown in Fig. 6. Every column uses an individual buffer to drive the internal memories of the selected pixels. The resistor string is biased between voltages Vtop and Vbot. On the one hand, Vtop should be lower than Vreset, otherwise, if they were similar, the spiking frequency would be quite large. On the other, Vbot should be higher than the threshold voltage of transistor Mn4in Fig. 1 to assure the linearity of the pixel response. Fig. 7 illustrates how the pixel spiking frequency can be adjusted by varying the Vref value. In the figure, Vreset is set to 4V and voltage Vref is varied from 0.5 to 4V. The rest of parameters in (1) are kept constant. Two different DAC resolutions, K= 4 and K= 8, have been considered. In both cases, voltages Vbot and Vtop were set to 0.5V and 4V, respectively. The relative spiking frequencies span almost three decades with the 8-bit DAC, and more than one decade with the 4-bit DAC. In practice, a 4-bit DAC can give enough programming flexibility with the added benefits of reduced complexity and fast operation speed. It is worth observing that while the peripheral readout circuit per pixel operates in an asynchronus manner, the control circuit for selecting, programming, and storing in local memories the Vref settings is essentially synchronous. The strategy for making both operations compatible has been described in detail in a previous work [3]. IV. PIXELS’PROGRAMMABILITY PROCEDURE The FPGA or microcontroller of Fig. 4 implements a control algorithm for selecting and adjusting the local in-pixel Vref voltages of the pixel array. The goal is to equalize the pixel output frequencies. Let us assume that the average target pixel spiking frequency for the whole array is favg. The procedure for updating the Vref values per pixel is as follows: 1) Calculate the average spiking rate fi,j,i= 0, . . . , M −1 and j= 0, . . . , N −1, for every pixel in the array during an observation interval. This is done by keeping track of the number of spikes which are transmitted through the AER protocol from every coordinates (i, j)of the array. V bot V top V standby Reset_DAC CLK_DAC DAC SELECTION LOGIC V DAC SEL<16> SEL<15> SEL<14> SEL<1> SEL<0> V DAC V R R R R0 13 14 15 ref Vref Vref Fig. 6. Proposed resistive DAC implementation to generate the Vref values. 0 0.5 1 1.5 2 2.5 3 3.5 4 Vref (V) 10-1 100 101 102 Relative spiking frequency (Hz) 8-bit DAC 4-bit DAC Fig. 7. Pixel spiking frequency versus Vref voltage. The Vreset voltage is set to 4V and Vref is swept using an 8-bit and a 4-bit DAC. As mentioned, The fi,j value depends on the locally stored voltage Vrefi,j and the global voltage Vreset. 2) If the average spiking frequency of the pixel with coordinates (i, j)exceeds a fraction αof the target frequency, i.e., if fi,j > α ·favg, the DAC is adjusted so that the updated reference voltage V0 refi,j amounts: V0 refi,j ←$Vrefi,j ·favg fi,j %(2) where b·c denotes the next available value lower than the argument. 3) If the average spiking frequency of the pixel with coordinates (i, j)is lower than a fraction βof the target frequency, i.e., if fi,j < β ·favg, the reference voltage Vrefi,j is updated as 5 TABLE I MAIN FEATURES OF THE PIXEL Technology AMS 0.18µm HV Power Supply 5V Pixel Size 25µm×22µm Pixel Complexity 45 Transistors + 5 Capacitors Fill Factor 6.25% Dynamic Range >120dB Analog power consumption 450nA Pixel sensitivity to light 1.76 pulses / ∆V·lux Temporal noise 1.1% Chip to chip spiking fdeviation 1.3% Sense Node Capacitance 35fF 22µm1mm (a) (b) 25µm Fig. 8. (a) Pixel layout. Vertical and horizontal metal lines are arranged to assemble with the neighbouring pixels. Its dimensions are 22µm×25 µm. (b) Microphotograph of the fabricated chip containing one prototype pixel, its biasing circuitry, and two scan buffers. V0 refi,j ←&Vrefi,j ·favg fi,j '(3) where d·e denotes the next available value higher than the argument. 4) If the average spiking frequency of the pixel with coordinates (i, j)is comprised between the above limits, i.e., if β·favg < fi,j < α ·favg, the reference voltage keeps the previous value, V0 refi,j ←Vrefi,j (4) Of course, although the procedure has been described for individual pixels, the same approach can be followed if groups of pixels are collectivelly updated. Indeed, this helps to alleviate the computational burden and speed up the process of in-pixel sensitivity programming. V. PIXEL’SEXPERIMENTAL CHARACTERIZATION A dedicated chip in a standard 180nm HV technology has been fabricated. In Fig. 8(a) the pixel layout is depicted. Its dimensions are 25 ×22µm2. A chip photograph is displayed in Fig. 8(b). It contains one pixel prototype, the external analog biasing circuitry for its operation, two analog buffers [17] to scan the analog voltages Vc(t)and Vpix (t), and logic to handle the pixel asynchronous readout. The main features of the proposed pixel are summarized in Table I. 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0 1 2 3 4 5 Amplitude (V) Global Adaptation Vc (t) V reset (t) 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Time (s) 0 2 4 Amplitude (V) SPIKE Fig. 9. Top: Effect of the global adaptation parameter Vreset on the voltage Vcat the integration capacitance C1. Bottom: pulses fired by the pixel. 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Amplitude (V) 0 1 2 3 4Local Adaptation V c (t) V ref (t) Time (s) 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Amplitude (V) 0 1 2 3 4 5 SPIKE Fig. 10. Top: Effect of the local adaptation parameter Vref on the voltage Vcat the integration capacitance C1. Bottom: pulses fired by the pixel. A. Local and global adaptation to light In order to test the global adaption to light, the voltage Vreset has been linearly swept and voltages Vcand spike have been recorded under constant illumination conditions. Plots are shown in Fig. 9. Note that the pixel firing frequency decreases for large Vreset values. Similarly, the local adaptation to light has been tested by sweeping the value of Vref . Results are displayed in Fig. 10. In this case, the recorded spiking frequency increases with Vref . B. Memory leakage impact Fig. 11 illustrates how the leakage from the diffusion capacitance FD1affects the voltage Vcat the integration capacitance of the pixel and, hence, the performance of the 6 0.8 1 1.2 1.4 1.6 1.8 2 Amplitude (V) 0 1 2 3 4 5Analog Memory Test V c (t) Time (s) 0.8 1 1.2 1.4 1.6 1.8 2 Amplitude (V) 0 1 2 3 4 5 MEM Fig. 11. Test of the impact of the leakage from the analog memory FD1. After storing a Vref value, the pixel is operated continuously without refreshing the memory to gauge the transient voltage variation of Vc. local pixel adaptation to light. This effect is of major relevance as it implies a trade-off between the maximum time slot during which voltage Vref can be stored without refreshing and the size of the diffusion capacitor. In the presented design, the measured analog memory leakage is L=3.34 mV/ms. Assuming a 4-bit DAC and a clock operation speed of fclk = 100MHz, the time interval allocated to sweep the Vref values in one column is Tcol = 2K/fclk = 1.6µs. In a pixel array with N= 256 columns, the Vref values can be thus refreshed or reprogrammed every Trefresh =N×Tcol = 0.4ms. Considering that the minimum voltage excursion is ∆V=Vresetmin −Vrefmax = 0.5V, the worst-case relative error due to memory leakage is given by (%) = Trefresh ·L ∆Vmax (5) which amounts 2.7% with the proposed pixel circuit. C. Pixel sensitivity to light and dynamic range Fig. 12 shows the pixel sensitivity to light. The plot is obtained by calculating the spiking rate for different illumination values. Together with the measured results, the plot also includes a linear data fitting (in red). The pixel exhibits a linear response to illumination over five decades, in accordance to (1). In the experiment, ∆ = Vreset −Vref =3.7V and the obtained pixel sensitivity is 1.76 pulses/∆V·lux. The pixel dynamic range can be computed as the ratio between the maximum and the minimum photocurrent that can be measured during a time interval: DRdB = 20 log10 Iphmax Iphmin (6) Examining the experimental data of Fig. 12 and considering that the pixel photocurrent is proportional to illumination, it Chip Illuminance (lux) 100101102103104 Spiking Frequency (Hz) 100 101 102 103 104 105 106 Measured data Linear Fitting Fig. 12. Pixel sensitivity to light measured interms of the spiking frequency for different chip illuminance values. V reset Time Voltage V ref ∆V σsens σarb σ reset σV ref σVreset Fig. 13. Illustration of the impact that the different noise sources have on the voltage at the integration capacitance, Vc. The error induced by each noise source in the spiking performance is indicated. can be deduced that the sensor is able to linearly encode illumination levels spanning for more than four decades. In HDR mode, the dynamic range can be increased beyond 120dB, as it is discussed in a prior work [3]. In this mode, the voltage variation at the integration capacitance can be digitized with an external ADC and, hence, the effictive ∆Vnecessary to encode one illumination value is lower. D. Noise sources Fig. 13 illustrates the impact of the main error sources on the light-to-frequency conversion performance. There are two types of noise sources: static and dynamic. Static noise sources are due to inter-pixel design parameter deviations. Dynamic noise sources are due to the AER arbitration logic, as well as, to thermal and shot noise contributions. As shown in Fig. 13, both types of noise sources cause inter-pixel frequency oscillation variations. The dominant static noise sources are: 7 a) Variations of the comparator’s threshold (σVreset ) cause that integration capacitances in the array are not reset at the same voltage value. b) Inter-pixel variations of the well capacitance and the integration capacitance affects the pixel sensitivity [see (1)]. This leads to slope variations (σsens) in the transient voltage at the integration capacitance. c) Variations of the analog memories capacitances F D1 (σVref ). This creates mismatch between the effective Vref voltages stored on each pixel. d) Variations on the time required to discharge the integration capacitance (σreset). Transistor mismatch in the differential pair of the OTA in Fig. 3 and inter-pixel variations of the integration capacitance are the main responsible for this behaviour. The dominant dynamic noise sources are: a) The AER arbitration logic [8] introduces transient delays (σarb) in the time needed to re-start the pixel operation when the voltage at the integration capacitance reaches the voltage threshold Vreset. Such delays mainly depend on the global pixel activity. If the AER communication logic is not congested, typical arbitration delays are below 100ns [8], [12]. According to the measurement results of Fig. 12, pixel spiking frequencies with high illumination are normally below 100Khz. Thus, the error introduced by the arbitration logic is not usually significant. b) Thermal and shot noise cause random variations in the Vcvoltage that affect the pixel oscillation frequency. The overall pixel temporal noise (TN), due to the dynamic noise sources, has been measured by exposing the pixel to a constant illumination of 400lux and gauging the time intervals between consecutive spikes during 10s. TN is computed as TN = Q P k=1 qfk−¯ f2 Q·¯ f=σf ¯ f(7) where fkis the measured spiking frequency between two consecutive spikes at the k-th time interval, Qis the number of spikes generated by the pixel during the 10s time window, and ¯ fis the average spiking frequency during the entire observation window. The measured TN value is 1.1%. The inter-pixel frequency deviation introduced by static noise sources has been also tested. For a given illumination value, the average spiking frequencies have been measured on ten different chips. The obtained chip to chip spiking frequency deviation is 1.3%. E. HDR mode readout test Fig. 14 illustrates the pixel performance in HDR mode. In this mode, after an integration time, Tint,Vc(Tint)is transferred to a floating diffusion capacitance FD2, when the control signal SH is active. Fig. 14 shows the buffered version, Vpix, of the voltage stored on FD2for different values of Vc(t). As shown in Fig. 1, the buffer consists in a source follower with the biasing transistor placed off-pixel as it is intended to be shared by all the pixels in a column. 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Amplitude (V) 0 1 2 3 4 5Voltage Readout on HDR Mode Vc(t) V pix (t) Time (s) 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 Amplitude (V) 0 1 2 3 4 5 SH(t) Tint Fig. 14. Test of the pixel analog readout in the HDR mode. Pixel analog voltage Vpix is stored on the diffusion capacitance FD2at the end of an integration time, Tint, by enabling the signal SH. In the experiment, Vref =1.5V and Vreset =4.3V. F. System scalability considerations In terms of scalability, the main limitation of event-based luminance sensors is the arbitration speed of the AER logic. As it is shown in Fig. 12, the pixel spiking frequency increases linearly with illumination. If the global event rate in a pixel array exceeds the maximum event rate that can be handled by the arbitration logic, random delays are introduced in the pixel spiking period [8]. This limits the dynamic range and, hence, the maximum illumination value that can be measured. Furthermore, spiking frequency variations increase noise (σarb contribution in Fig. 13). Thus, the proposed technique to adapt locally the sensitivity to light benefits scalability. Further, by reducing the sensitivity of highly illuminated regions, the data load to be processed by the AER logic is minimized. VI. FAST PIXEL LUMINANCE ESTIMATION One relevant question is how to conveniently set the local Vref voltages in the pixel array, the first time the sensor is initiated. On the one hand, if a low illuminated pixel stores initially a very low value of Vref , its response latency will be very slow and transient illumination changes might be undetected because of the low pixel sensitivity. On the other, if the Vref values are set initially close to Vreset, the pixel sensitivity will be high and the pixel response will be fast, however, the global event rate may saturate the arbitration logic [8] and the power consumption of the sensor will be very large until the algorithm steps down the Vref voltages for the higher illuminated pixels. This suggests the convenience to count on a fast procedure for estimating the pixel illumination and, eventually, reprogram the Vref voltages. In particular, it could be useful in environments where the illumination levels within the visual scene change at high speed. If the illumination levels are not known, it is difficult to achieve a 8 V (t) reset V (0) reset T0T1T2T4 T scan Tosc t V (t) c I /C ph 1 -a V ref V resetmin Fig. 15. Proposed method to gauge the illumination levels. The analog voltage Vreset (red trace), shared by all the pixels, is swept between two values during a time interval of duration Tscan. The transient Vcvoltage at the integration capacitance is shown in blue. Pixel spiking frequency increases when the Vreset voltage is close to Vref . By determining the ratio between two time intervals associated to the reception of two consecutive spikes, Tn/Tn−s, the photocurrent value can be determined. good balance between sensitivity and data throughput if the Vref values are arbitrarily defined. To program the pixel sensitivity to light in such situations, the authors advanced in a prior publication [18] a procedure to obtain a fast estimation of the pixel luminance without increasing the global event rate significantly. The idea is to force a response (at least two spikes) for all the pixels within a user-defined time interval, Tscan. Initially, all the pixels store the same value of Vref and are reset. Then, the Vreset voltage, that is shared by all the pixels of the array, is swept as it is depicted in Fig. 15. It starts from the value Vreset0. Then, it decreases with a slope, −a, until it reaches the final value Vresetmin . Whenever the Vreset value is close to Vref , pixels exposed to low illumination will be forced to fire during Tscan. By processing the events received during this interval, it is possible to gauge pixels illumination levels. Before receiving the first spike, t<T0, we can express the transient voltages Vreset (t)and Vc(t)as follows: Vc(t) = Vref +Iph ·t C1 (8) Vreset (t) = Vreset0−a·t(9) Using (8) and (9), the first spike time stamp, T0, and the voltage value Vc(T0) = Vreset (T0)can be determined as T0=Vreset0−Vref a+Iph C1 (10) Vreset (T0) = Vreset0−a·T0(11) Similarly, T1and Vref (T1)are given by T1=Vreset (T0)−Vref a+Iph C1 (12) Vreset (T1) = Vreset0−a·(T0+T1)(13) These equations can be generalized to obtain Vreset (Tn) and Tn, for n > 1, as Tn=Vreset (Tn−1)−Vref a+Iph C1 = =T0−a(T0+T1+... +Tn−1) a+Iph C1 = =T0−A·(T0+T1+... +Tn−1) (14) Vreset (Tn) = Vreset0−a·(T0+T1+... +Tn)(15) where A=a a+Iph/C1<1. From (14), a general expression for Tn, as a function of T0can be derived: T0=T0 T1=T0−AT0= (1 −A)T0 T2=A(T0+T1) = (1 −A)2T0 . . .=. . . Tn= (1 −A)n·T0 (16) where 0<1−A < 1and lim n→∞ Tn=T0lim n→∞ (1 −A)n= 0 (17) Hence, from (16), we can state that the ratio between two consecutive time intervals is constant and depends on Iph: Tn Tn−1 = (1 −A)(18) The proposed method is mainly targeted for a fast measurement of low illuminated regions. For these pixels, the slope of Vreset (t)is much higher than the slope of Vc(t), i.e. a >> Iph/C1, with Iph/C1<1. Thus, the ratio between two consecutive time intervals is approximately constant and depends on the photocurrent value, Iph: A=a a+Iph/C1 ≈1−Iph a·C1 (19) Tn Tn−1 = (1 −A)≈Iph a·C1 (20) Therefore, only two spikes fired by a pixel are required to estimate its illumination. The proposed method achieves a good balance between the amount of time required for the measurement and bandwidth consumption. Pixels sensitivity is increased progressively to avoid saturating the arbitration logic. The duration of the time interval Tscan can be adjusted varying the value of the slope, −a. In Fig. 16, measurements taken with one pixel after sweeping the Vreset voltage are shown. The pixel has been exposed to uniform and constant illumination. On top, transient voltages Vreset (t)and VC(t)are displayed. In the middle plot, the time stamps that correspond to each spike are plotted. At time instant Tn,Vreset (Tn) = VC(Tn)and the pixel fires. As expected, the ratio between consecutive time intervals Tn/Tn−1is approximately constant and lower than one. Variations are mainly due to the arbitrary delays introduced 9 100 105 110 115 120 125 0 1 2 3 4 Voltage (V) Vc (t) Vreset (t) 0 0.5 1 Spike Time Stamps 100 105 110 115 120 125 100 105 110 115 120 125 Time (milliseconds) 0.5 0.6 0.7 0.8 Tn/T n-1 I /(a·C ) ph 1 Fig. 16. Estimation of the pixel photocurrent sweeping the Vref voltage. Top: Transient voltages Vreset (t)and Vc(t)are plotted. Middle plot: Time stamps of the different spikes fired by the pixel. Every time that Vreset (t) voltage reaches the voltage value at the pixel integration capacitance Vc(t), the pixel spikes and resets the Vc(t)voltage. Bottom plot: Ratio between consecutive spikes time stamps. The ratio should be constant and proportional to the photocurrent value. by the asynchronous communication logic. The reception of two spikes is required to determine the photocurrent value. If a pixel fires more than twice during Tscan, results can be improved by averaging the ratios, Tn/Tn−1. In the example, Tscan is set to 25ms. This value can be programmed depending on the latency requirements. VII. CONCLUSIONS The capabilities of a new spiking pixel prototype have been presented and analyzed. It implements both global and local control of the pixel sensitivity to light. Thus, the proposal overcomes a classic limitation of asynchronous sensors based on a light-to-frequency conversion: local adaptation to light is not available for increasing the sensitivity to light in poorly illuminated pixels or regions. Local adaptation is implemented by storing a voltage threshold parameter on an in-pixel diffusion capacitance. The technique increases the dynamic range when a time limit to render images is established. The proposed pixel is compatible with the mixed frame-based and event-based pixel HDR operation previously proposed by the authors. As future work, a whole pixel array, together with the required external control circuitry, will be fabricated and tested. REFERENCES [1] E. Culurciello, R. Etienne-Cummings, and K. Boahen, “A biomorphic digital image sensor,” Solid-State Circuits, IEEE Journal of, vol. 38, no. 2, pp. 281–294, Feb 2003. [2] D. Chen, D. Matolin, A. Bermak, and C. Posch, “Pulse-modulation imaging –Review and performance analysis,” Biomedical Circuits and Systems, IEEE Transactions on, vol. 5, no. 1, pp. 64–82, Feb 2011. [3] J. A. Le˜ nero-Bardallo, R. Carmona-Gal´ an, and A. Rodr´ ıguez-V´ azquez, “A wide linear dynamic range image sensor based on asynchronous self-reset and tagging of saturation events,” IEEE Journal of Solid-State Circuits, vol. 52, no. 6, pp. 1605–1617, April 2017. [4] J. A. Le˜ nero-Bardallo, L. Farian, J. M. Guerrero-Rodr´ ıguez, R. Carmona-Gal´ an, and . Rodr´ ıguez-V´ azquez, “Sun sensor based on a luminance spiking pixel array,” IEEE Sensors Journal, vol. 17, no. 20, pp. 6578–6588, Oct 2017. [5] J. Le˜ nero-Bardallo, D. Bryn, and P. H¨ afliger, “Flame monitoring with an AER color vision sensor,” in Circuits and Systems (ISCAS), 2013 IEEE International Symposium on, May 2013, pp. 2404–2407. [6] J. E. Dowling, The Retina: An Approachable Part of the Brain, 2nd ed. The Belknap Press of Harvard University Press, 2012. [7] K. A. Boahen, “Retinomorphic vision systems: Reverse engineering the vertebrate retina,” Ph.D. dissertation, Cal. Inst. of Tech., Pasadena, California, 1996. [8] K. Boahen, “A throughput-on-demand address-event transmitter for neuromorphic chips,” in Advanced Research in VLSI, 1999. Proceedings. 20th Anniversary Conference on, Mar 1999, pp. 72–86. [9] P. Garrou, C. Bower, and P. Ramm, Handbook of 3D Integration: Technology and Applications of 3D Integrated Circuits. Wiley-VCH, 2008. [10] M. Suarez, V. M. Brea, J. Fernandez-Berni, R. Carmona-Galan, G. Linan, D. Cabello, and . Rodriguez-Vazquez, “CMOS-3D smart imager architectures for feature detection,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 2, no. 4, pp. 723–736, Dec 2012. [11] J. Ohta, Smart CMOS Image Sensors And Applications. CRC Press, 2008. [12] M. Silvilotti, “Wiring considerations in analog VLSI systems with application to field-programmable networks,” Ph.D. dissertation, Cal. Inst. of Tech., Pasadena, California, 1991. [13] M. Mahowald, An Analog VLSI System for Stereoscopic Vision. Kluwer, 1994. [14] J. A. Le˜ nero-Bardallo, R. Carmona-Gal´ an, and A. Rodr´ ıguez-V´ azquez., “A bio-inspired vision sensor with dual operation and readout modes,” Sensors Journal, IEEE, vol. 16, no. 2, pp. 1–14, January 2016. [15] J. A. Le˜ nero-Bardallo, D. Bryn, and P. H¨ afliger, “Bio-inspired asynchronous pixel event tricolor vision sensor,” Biomedical Circuits and Systems, IEEE Transactions on, vol. 8, no. 3, pp. 345–357, June 2014. [16] P. H¨ afliger, “A spike based learning rule and its implementation in analog hardware,” Ph.D. dissertation, ETH Z¨ urich, Switzerland, 2000, http://www.ifi.uio.no/ hafliger. [17] C.-W. Lu, “High-speed driving scheme and compact high-speed lowpower rail-to-rail class-b buffer amplifier for LCD applications,” Solid- State Circuits, IEEE Journal of, vol. 39, no. 11, pp. 1938–1947, Nov 2004. [18] J. Le˜ nero-Bardallo and F. Garc´ ıa-Pacheco, “Fast luminance measurement method for asynchronous spiking pixels,” Electronics Letters, pp. 1–2, March 2018.