Live demonstration: Neuro-inspired system for realtime vision tilt correction

Live Demonstration: Neuro-inspired system for realtime vision tilt correction A. Jimenez-Fernandez, J.L. Fuentes-del-Bosh, R. Paz-Vicente, A. Linares-Barranco, G. Jiménez Dpto. Arquitectura y Tecnología de Computadores. Universidad de Sevilla. Av. Reina Mercedes s/n, 41012-Sevilla, SPAIN [email protected] Abstract — Correcting digital images tilt needs huge quantities of memory, high computational resources, and use to take a considerable amount of time. This demonstration shows how a spikes-based silicon retina dynamic vision sensor (DVS) tilt can corrected in real time using a commercial accelerometer. DVS output is a stream of spikes codified using the address-event representation (AER). Event-based processing is focused on change in real time DVS output addresses. Taking into account this DVS feature, we present an AER based layer able to correct in real time the DVS tilt, using a high speed algorithmic mapping layer and introducing a minimum latency in the system. A co-design platform (the AER-Robot platform), based into a Xilinx Spartan 3 FPGA and an 8051 USB microcontroller, has been used to implement the system. The demonstration is almost self-explanatory and captivating. It shows the power of combining highperformance spike-based sensors with commercial cheap sensors. It has been running in our lab for the last months and we show it to students to understand the power of the spikebased representation of information. Presented system is an AER based layer that corrects in real time the tilt of a DVS vision sensor, called AER Tilt Correction Layer, ATCL. This layer tries to solve a common problem in vision systems over mobile robots, vision tilt and vibrations. DVS output is composed by stream of AER events, where every event address is the pixel coordinate that fires a spike. Changing event addresses when they are flowing from one layer to other allows performing several kind of image processing in real time. This kind of processing can be implemented using mapping tables, like look-up-tables, or an algorithmic mapping. ATCL implements an algorithmic mapping, because use mapping tables will demand huge quantities of memory and can introduce some latency time. ATCL has been implemented in the AER-Robot platform. AER-Robot platform have designed using a Spartan 3 FPGA (XC3S400) and 8051 microcontroller (C8051F320). These elements work together to correct rotate AER images using an accelerometer. The idea is use the 8051 analog-to-digitalconverts (ADC) to measure accelerometer tilt. One time this information is obtained, the microcontroller sends the value of tilt sine and cosine values to the FPGA. Inside the FPGA there the elements necessary to calculate the rotation of every incoming AER event according to accelerometer tilt. More detailed information about how ATCL has been implemented can be found in a paper submitted to Eventbased Neuromorphic Systems track in ISCAS 2010. For this demonstration we will bring to ISCAS an AERRobot board together with a small PCB which contains the accelerometer. We also need an element to monitor AER information, for this purpose we bring an USBAERmini2 board, which represents a bridge between AER buses and a standard USB 2.0 high-speed port. This board is able to monitor AER information, and visualize tilt corrected images in a computer, thanks to the jAER open source software. If a DVS is available, it will be connected to the AER-Robot, correcting in real-time DVS tilt. If is not possible to dispose of one DVS, we will use the USBAERmini2 to sequence some DVS data samples, rotating these samples tilting the accelerometer. In addition we will bring a small power supply, an USB cable, AER cables for connect all the system, and a computer to visualize (and sequence if is needed) the AER information. Visitors will interact with our system tilting the DVS, and viewing in the computer how the image stands horizontally, if it is available. Otherwise, without DVS, visitors will tilt the accelerometer board, watching in the computer how sequencing AER samples are rotated in real time. Understanding the advantages of representing vision systems using the AER, and how can be images processed thanks to communicate pixels coordinates instead of its values. Photo of ATCL, implemented inside the AER-Robot with the acceloromter attached at bottom, connected to a USBAERmini2, at top.