SIRENA: A simulation environment for CNNs

CNNA-94 Third IEEE International Workshop on Cellular Neural Networks and their Applications Rome. Italy, December 18-21.1994 SIRENA: A Simulahn Environment for CNNs R. Dom'nguez-Castro, S. l%pejo, A. Rodriguez-Vhquez, I. Garcia-Vargas, J.F. Ramos and R. Carmona Centro Nacional de Micm&ctriwica-Universidad de Sevilla Edificio CICA, marfia dn, 41012-Sevilla, SPAIN Phone: 34 - 5 - 423 99 23. Fax: 34 - 5 - 462 45 06. E-mail: [email protected] Abstract - SIRENA is a general simukztion environment for arhficial neural networks, with emphasis towards CNNs. A especial interast has been placed in allowing the simulation and modelling of the non-ideal eflects expectedw VLSI implementations. SIRENA allows the simulation of CNNs in greater detail than conventional CNN simulators, and much more eflciently that SPICE-type electrical simulators. 1. Introduction CNNs are massively-parallel processing systems, specially suited for real-time image processing applications. Since its algorithmic description in 1988 [ 11. [2] many applications have been reported [2], [3], [4], [5], and several analog integrated circuits have been developed for real-time specific applications, [61,[71, [81. Efficient simulators are needed to develop new applications (at an algorithmic level) as well as for functional verification of complex integrated circuits (lower description level). While algorithmic simulators are being widely usad [9], proper CAD tools for the development of CNN VLSI integrated circuits are still lacking. The widely available SPICE-type electrical simulators give a high level of detail in tha system description but arc limited to working with around 10,OOO transistors. For example, it has been estimated that an HSPICE simulator [lo] running on a Sparc-10 workstation (loohiPS), can simulate circuits of at most 20.000 transistors and may take several days of CPU in a simple &nulation. An average size (32x32) programmable CNN designed by the authors bas around 250,000 transistors [ll]. Consequently, conventional design Wls an poorly suited for the verification of largescale analog integrated circuits. On the other hand, actual algorithmic level Simulators are unable to consider the non-ideal effects of mal electronic circuits. SIRENA has been designed to overcome this limitation. This simulation environment allows the description and simulation of large neural networks with different levels of description. In addition to a high simulation efficiency, SIRENA is equipped with a powerful and friendly graphical interface under the X-windows framework. 2. S~kA~~~tf~c"dlMlbtion8ndmodcling Anschemuicblockdi~ofSIR$NAisshowninFig. 1.Shdedanasnpnsentsthe major pans of the system: the nucleus or actual simulator (NSS), the model generator (GMS), and the graphic interface between the user and the simulator nucleus. 0-7803-20704/94/$4.00 Q 1994 IEEE 41 7 V~sualion and Data file - FW I - Conceptml Mock diagram of SIRENA. SIRENA allows simulation of general neural networks, and can be considered as functional simulator in which the description and use of general macromodels is possible and simple. However, its development has been focuscd on the simulation of CNNs. The simulation enviranment hps been developed using object-oriented programing techniques, and highly portable languaepes like C and C++. Cumntly, it runs under the UNIX operating system and the X-Window system. These characteristics make of SIRENA an efficient, actual and friendly simulation environment. The following sections describe in greater detail each of the major components of SIRENA. 3. The Simulator Core (NS) The principal function of the simulator core is to solve systems of nonlinear differential equations, as required to simulate the dynamic evolution of CNNs [I]. The user can select among a set of well known integration algorithms (fourth-order Runge-Kutta with either automatic integration-time-step control or with fixed time-step). Other integration algorithms can be easily incorporated, (for instance Forward-Euler, Backward-Euler, Trapezoidal, Gear Methods, etc.), and will be included in fum releases of SIRENA. The simulator core allows the following types of analysis: this basic capability evaluates the time evolution of the network, providing the output signals oc images specified by the user with time intervals also defined by the user. This is the main type of malysis, upon which the following ones are based. this functionality consists in the automatic realization of multiple transient analysis, sweeping the value of a single parameter or variable with a specified increment and range. The motivation for the inclusion of this capability is to facilitate the evaluation of the sensibility of a given network to variations on the different parameters of the algorithm (template coefficients, offset parameter, time constant, etc.) or on other parametcm describing the behavior of real circuit blocks (for instance the power supplies or some process parameter). It is also very convenient to optimize the robustness of a given application (design-centering). it daws the f&Wation of multiple simulations of a given network, with randam variapions of ollc or more parameters Following some specific probability . .. 41 8 distribution (gaussian or uniform). This type of analysis is useful for the evaluation of the statistical yield of a given network untier random deviations of its parameters, either local variations (different deviation on 4 cell) of global (same deviation 00 all cells). his type of evaluation is a prerequisite for the fabrication of highcomplexity analog VLSI circuit employing small devices. It is tdwever extremely expensive in terms of CPU time, and hence, its realization with traditional SPICE-like electrical simalators is virtually impossible. Noise this feature is inten& for the evaluation of noise influence on the evolution of a given network. Because CNNs am strongly nodi systems. the conventional approach of traditional electrical simulators, in which noise analysis is associated to small signal equivalents, can not be applied. Iu our case, the simulator injects noise on user-specified points (signals or parpeters). The probability distribution and spectral density of the noise can be arbitrarily shaped. A combined noise and sensibility analysis allows the evaluation of the noise *bold that can be tolerated by the network. aver &&& the simulator core of SIRENA allows the simultaneous simulation of several CNN layers, which can be connected among them. These layers can be described by the same cell model (three-dimensional CNNs), or by different models (multilayered CNN). work 4nalrsis; in addition tQ the simulation of three-dimensional and multilayered CNNs, NSS allows the evaluation of complex systems compound of several CNNs, which interchange information at specified time instants or following a described protocol (information interchange after convwgency of the individual network processes). This functionality is controlled by the task-scheduler. 3. I. Task scheduler NSS permits the independent simulation of several CNN layers, controlling fhe sequence of processes and allowing information interchange among the different layers at specific time instants. This permits the evaluatiap of sequential and multipath algorithms [ 121. From a general point of view, NSS allows an arbii flow control with conditional and jump ogerators, as well as a wide catalog of algebraic operations. An example of this type of algorithms is shown in Fig.2, which describes the objectcounting application [ 121. 4. The Model Generator (GMS) A previous step in the simulation of a CM Black and-htc image w Hole filling L I 1 Figure 2 - Flmv diagram ofthe object counting apvlic4th d CNNs I I2 1. -_ -- is the description of the individual cells. The capability of employing userdefined modsls of individual cells is a major feature of SIRENA. Cell model can range from the pun algorfthmic level to highly detailed models with a large number of non ideal characteristics accounting for predictable effects in electronic implemntations (impedance coupling among input pd output nodes of the cells, undesired nonlinearities, parasitic element effects, non-unifomity caused by element mismatch, etc.) GMS allows the user to define, in a simple manner, his own CNN models, which later on can be used for CNN simulations. The definition of a CNN requires the specification of its components and their behavior. 41 Q Components include the set of layer and sublayers of the network, as well as their characteristics (layer siz in term of number of cells, type and parameters of boundary cells, neighborhood s&, dorm or nonuniform type of laJnr, and others). The behavior is described by a set of alpbmc and differential equations relating the different pre-specified components, which detqmine the dynamic evolution of the network. Behavior description is easily made using an specific purpose programming language developed for this purpose: DECEL. After the cell models have been entered, GMS compiles and links them with NSS in order to use them in the simulation. 5. The Graphics Front End The graphic front end of the simulation environment has been developed under the X-Windows framework and performs multige functions like the edition of input signals (usually two-dimensional images), the visualization of the simulation output (like individual element values as a function of time, images, Q sequences of images), and interactive control of the simulation. The following is a brief description of the main tools included in the frontad package. 5.1. Input/Ourput interface (IESS) IESS is in fact a library of dedicated functions for information-file management. It provides a common base for individual tools in SIRENA, allowing the integration of the different packages in a compact environment. This interface include the parsers required to analyze the input and output files, detect their errors, and provide the proper error messages. In addition, it contains a set of utilities dedicated to translate different graphic fonnats *quid for the visualization of signals with external tools (for instance “xgraph”). 5.2. Matrix visualizer (XVMS) mand line. The major features are This tool allows the visualization of an image or a sequence of images specified in a com- - Zooming - Image sequence visualization. The sequence can be compound of a set of images corresponding to different time instants in a particular C” process, or by images obtained from different networks (for instance in sequerltial algorithm). Fig. 3 illustrates an example of this last possibility, borders extraction of the black and white image obtained using the noise-filtering template on a grey-level image. - Visualization of different layers of three-dimensional networks. 5.3. Matrix editor (XEVMS) XEVMS is a graphic editor, which can be considered as enhanced version of the matrix visualizer XYMS. Added capabilities are dedicated to allow the creation and modification of images, which can be easily used as inputs to the simulator. These capabilities are: - File management operations: saving, kading. creating, etc. - Irnport/expon of specified sections of a given image frodto graphic files in conventional fonnats (PBM format). - Image resolution modification, in order to adapt a given image to a particular network Size. - Graphical and numerical edition of individual pixel values. - Image sequence order modification. 420 Figure 3 - Example of image-sequence visualization. a) Original gmy-levels image. 6) Image afer noise filter. ins. c) Borders extracted fmm image b. - Adding and deleting images from an image sequence. - Drawing some common geometric elements (circles, ellipses, rectangles, etc.) Fig. 3a illustrates the graphical appearance of this tool. 5.4. Simulator core interface (XINSS) This tool is a graphical interface with the simulator core (NSS) of SIRENA under the X-windows system. Its main objective is to provide interactive access to the output generated by the simulator. This interactive access permits the visualization of the results of the simulation while it is running. In addition, the user is allowed to modify the simulation parameters without interrupting the simulator. In this manner, the user can observe and modify the simulation without reinitializing the process. The major capabilities of this tool are: - Starting a simulation on NSS. - Simulation supervision by interactive observation of the output being generated. - On-line parameter modification. - Access to output files generated by previous simulations. - Images visualization, individually or in sequences, calling XVMS. I Figure 4 - a) Matrix ediror (XEVMS). b) Sirnularor core interface window (INSS). 421 - Waveforms visualitation using Irsraph. - output storage on information-files. 6. Results The efficiency of SIRENA is illusby the following example: the tiae nquind to simthe borders extraction CWwlr’p the image shown ia Rg. 3b (143x159 pixels) was of 52s, running on a Sparc Statim 10 (1@MIPS) and using the d@thmic kvel &scription. Ref- [l] L.O. Chua and L. Yaag; “Cellul orks: Theory”. IEEE Trans. Circuits and [2] L.O. Chua and L. Yang: “Cellular Neur ’ Networks: Applications”. IEEE Trans. Circuits [3] T. Roska, T. Boros, P. Thiran and L.O. Chua: “D*tacting Simple Motian Using Cellular Neural Networks”. Proc. First IEEE Int. Workshop on Cellular Neuml Networks and their Applications, pp 127-138, Budapest, Detdmber 1990. [4] T. Sziranyi and J. Csicsvari: “High-speed Character Recognition Using a Dual Cellular New Network Architecture”. IEEE Tm. Circuifs and Systems II: Analog and Digital Signal Processing, Vol. 40, pp 223-231, March 1993. [SI H. Suzuki. T. Matsumoto and L.0. Chw: “A CNN Handwritten Character Recognizer”. Int. J. Circuit Theory and Applications, Vol. 20, pp 601-612, John Wiley & Sons, September-October 1992. [6] J. M. Cruz and L.O. Chua: “A CNN Chip for Connected Component Detection”. IEEE Trans. Circuits and Systems. Vol. CAS-38, pp. 812-817, July 1991. [7] H. Harrer, J. A. Nossek and R. Stela: “An Analog Implementation of Discrete-Time Cellular Neural Networks”. IEEE Trans. Neural Networks, Vol. 3, pp. 466476, May 1992. [8] A. Rodriguez-Vhquez, S. Espejo. R. Dominguez-Castro, J.L. Huertas and E. Shchez-Si- nencio: “Current-Mode Techniqws for the Implementation of Continuous-Time and Discrete-Time Cellular Neural Networks”, IEEE Trans. Circuits and Systems II: Analog and Digital SSgnrJ Processing, Vol. 40, pp. 132-146. Mamh 1993. [9] T. Roska, G. BWay, P. Szolpy, T. Szirhyi, A. Radvhyi, T. Xozek and 2s. Ugray: “A Hardwe~e Agelerator BaMz for Cellular Ned Neeworlrs: CNN-HAC‘, IEEE Int. Workshop on CdWr Ir4erVal Ne+ and Their wim*ons, pp. 160-168, Dec. 1990. S-, Vol. 35. p~ 12S7-1272, mrdSystemr, Vol. 35, pp 1273-1290, OCt ber 1988. [ 10l“HSPICE User Manual” Meta sofnkrarc Inc. 1988. [llJR. Domfnguee-Cz”, S. Espj~, A. ROdrigucz-V@~ and R. Carmona: “A CNN Universal Chip in CMOS Tecfmology”. In this proceeding. l12lL.O. chua, T. Roslra, P.L. VeWaner and A. My: “Some Novel Capabilities of CNN: Game of Life and Exampks of Multipath Al&th”’. Proc. Second ZEEE In?. Workshop on Cellular Neural Network ad kir Applidations, pp 276-281, Munich, October 1992. 422