CMOS current-mode chaotic neurons

499 CMOS Current-Mode Chaotic Neurons Manuel Delgado-Restituto and Angel Rodriguez-V&quez Departament of Analog Design Centro Nacional de Microelectronica, cmarlia sn, 41 012-Sevilla, Spain This paper presents two nonlinear CMOS current-mode circuits that implement neuron soma equations for chaotic neural networks, and another circuit to realize programmable current-mode synapse using CMOS-compatible BJT's. They have been fabricated in a double-metal, single-poly 1.6pm CMOS technology and their measured performance reached the expected function and specifications. The neuron soma circuits use a novel, highly accurate CMOS circuit strategy to realize piecewise-linear characteristics in current-mode domain. Their prototypes obtain reduced area and low voltage power supply (down to 3v) with clock frequency of 5OOkHz. As regard to the synapse circuit, it obtains large linearity and continuous, linear, weight adjustment by explotation of the exponential-law operation of CMOS-BJT's. The full accordance observed between theory and measurements supports the development of future analog VLSI chaotic neural networks to emulate biological systems and advanced computation. INTRODUCTION Recent studies on real nerve membranes in neurophysiological experiments have shown that the dynamical behavior of biological neurons are much more complex (including chaotic response) than that exhibited by conventional models used in artificial networks, typically represented by simple threshold or sigmoid elements [l], [2]. Consequently, in the last years, new schemes of artificial neural networks have emerged whose purpose is to more realistically emulate the chaotic responses experimentally observed from biological systems. One of the simplest chaotic neural networks has been reported in [3], and constitutes a modification of the Nagumo-Sat0 model [4], where instead of following an all-or-none law for the action potential, modelled by an unit step output function U(*), the neuron shows a continuously graded stimulus-response curve, represented by a non-linear function Am). The model in [3], illustrated in Fig. 1, is defined by the following finite-difference equations: ni(n + 1) = kxi(n) - a&(n)) +Ai@) , n = 0, 1, ... (1) yi(n + 1) =&(n + I)) wherex,(n+l) and y,(n+l) are the internal state and the output of the ith chaotic neuron at the discrete time n+l, respectively; a and k are the scaling and damping factors of refractoriness (residual effect of a neuron once fired), respectively; and A,(n) is the input excitation at the instant n, given by Figure 1 Analog computer concept for the chaotic neuron circuit. The nonlinear block is given by g(x) = kr - ax), according to (1). M N Ai(n) = Wjjyj(n) + Vj,Ij(n) - Bi (2) j= 1 j= 1 where the first term computes the influence of the A4 neurons driving the ith neuron; the second, the excitation from the N external inputs, 4; and 0, is the threshold of the ith neuron. Lastly,A-) constitutes the neuron output function represented by the piecewise-linear model where E is a positive number defining the steepness of the function. Many studies on chaotic neural networks in general, and using the previous model in particular, reveal that such networks not only serve as an experimental vehicle in the study of sensory nerve systems, but also provide the means for important engineering applications. In this sense, chaotic neural networks have been proposed to solve difficult optimization problems [5], [6] ; for dynamical associative pattem classification [7]; and for signal detection and classgcation in noisy environments [8], and it is foreseeable that new applications will arise in the near future. In spite of the strong economical interest involving these applications, few up-to-date physical implementations of chaotic neurons have been proposed. Thus, it is advisable to give circuit realizations of these models. Furthermore, due to the technological trend towards system integration, these circuits must be well-suited for VLSI, and, if possible, compatible with standard low cost CMOS technologies. The purpose of this communication is to provide monolithic implementations of the discrete-time chaotic 500 neuron described in (l), as well as for the Nagumo-Sato model that, previously stated, substitutes the nonlinear function in (3) by a unit step U(*). Also, a synapse circuit with continuous, linear programmability which employs CMOS-compatible BJT's is reported. CURRENT MODE IMPLEMENTATIONS Current-mode techniques have been employed to implement both neurons, following the conceptual diagram shown in Fig.1. Summation is easily realized exploiting KCL. The delay operation can be realized as a cascade of two track-and-hold switched-current stages, as proposed by Hughes et al. [9]. Fig. 2 shows the schematics for this block. Nonlinearities have been achieved using a novel, highly accurate CMOS circuit strategy to realize PL characteristics in current-mode domain. It is based on the rectifying characteristics of the current switch [lo], which provides very high resolution and virtually zero current offset, not 9 I : 1+1 : B Figure 2 Current-mode track and hold circuit. + -a I (a) (b) Nagumo-Sato Model (4 Aihara Mode Figure 3 Piecewise linear mapping :I circuits for the neurons. influenced by transistor mismatches (indeed, minimum size transistors were used in both prototypes). Fig. 3 shows the corresponding schematics for both PL functions. Current amplifiers can be implemented by properly ratioed bilateral current mirrors. Firstly, let us consider implementation of the nonlinear block, g(-), for the Nagumo-Sat0 model, according to the concept of Fig.1. Fig.3(a), which consists of two current sources (realized in practice by current mirror output branches), four transistors and two digital inverters, shows a conceptual schematic for the realization of the PL characteristics of Fig.3(b). Transistors MCSn and M,, in Fig.3(a) operate as a current-controlled-current-switch, wkle transistors MVSn and MVSp operate as voltage-controlled current switches. Any positive input current increases the input voltage, turning Qs device ON, and since both devices in the current switci have the same gate voltage, MCSn OFF. Simultaneously, the voltage at the second inverter output evolves to the high logic state, turning Mvs,, ON and Mvs OFF. Thus, a current h(n) -a (obtained by KCL at noBe NI) is directed to the output node through the transistor MvSn -- the right-hand piece of Fig.3(b) is implemented in this manner. Similarly, negative input currents turn MCSn and MVsp ON, so that a current kx(n) + a circulates through Mvsp to the output node. The output response of the neuron can be taken from the voltage at the output of the inverter IN2 which swings from rail to rail depending upon the sign of the input current, thus resembling the required unit step function in voltage mode. Obviously this binary signal must be combined with analog switches and current sources for properly driving other neurons in the network. Since current discrimination in the proposed circuit relies on integration function performed at the input node, resolution is very high, not influenced by transistor mismatches (measurements from the CMOS prototype display resolution of 12pA's). Operation speed is also very high, limited mainly by nonlinear transients in the transistors that implement the current sources used to drive nodes NI and N2. Also, the feedback created by inverter IN1 yields significant reduction of the dead-zone exhibited by the driving point characteristics measured at the input node, that is proportional to (VTn + IVTp() / Ai",, where VTn and VTp are the threshold voltages for the transistors and A,, is the inverter DC gain. This is an appealing feature that enables reduction of interstage loading errors caused by finite equivalent MOS transistors Early voltages. Concerning the implementation of the PL characteristic shown in Fig. 3(d) for the Aihara model, Fig. 3(c) shows the corresponding schematic. The current switches, together with the current sources, are used to discriminate the input current into three paths according if x(n) is lower than -E, greater than E, or is comprised between both values. In the two first cases, the paths are routed to the same node and amplified by k, while if x(n)E [-E, E] the current is amplified by -y, where y = WE - k, according to (1) and (3). The output of the neuron can be easily taken in this case by replication and scaling the output current of the amplifier with gain -y. 50 1 PROGRAMMABILITY ISSUES FOR CURRENT MODE BLOCKS In current-mode domain two different approaches to programmability can be considered depending on the nature of the associated controlling signals: discrete programmability, where controlling signals are digital, and continuous programmability, where controlling signals are analog. Discrete programmability can be incorporated in a very simple way, by analog multiplexing of current contributions from different mirrors. These mirrors can either implement fixed templates (with application, for instance, in cases where well-defined tasks must be sequentially performed), or be binary-weighted (for more general application). Discrete programmability provides ease of controllability and accurate results, at the cost of a strong area penalty. For reduced area and continuous weight adjustment, analog programmability should be considered. A simple way to achieve analog programmability is using tunable transconductors, as the one shown in Fig. 4(a). Fig. 4(b) shows a programmable current mirror using this transconductor. Two different situations arise depending on whether transistors operate in weak or in strong inversion. Analysis for both operating conditions shows the following, - iq = 1s 21 = 2 (4) i. In strong PllBl 'In weak 'El As can be seen, the dependence is linear for weak inversion; hence, this latter case provides larger weight adjustment ranges. It is illustrated in Fig. 5, showing the current weight as a function of IB2/IB for different values of IB,/I,, where IB is a normalization factor of value lOnA for weak inversion (Fig. 5(a)) and 50pA for strong inversion (Fig. 5(b)). Also, nonlinearity cancellation is exact in weak inversion due to the exponential nature of current to voltage characteristics, while it is only approximate for strong inversion: nonlinearity in the weak inversion case is less than I% up to io=IB2, while the corresponding value for strong inversion is i,=O.13lB2. Drawbacks of weak inversion are low accuracy, due to mismatch, and reduced speed. These can be overcome by using CMOS compatible lateral BJTs [ 1 I], which exhibit exponential feature for larger current ranges, and with excellent matching properties [ 121. EXPERIMENTAL RESULTS Both neurons have been fabricated in a double-metal, single-poly 1.6pm CMOS technology. Fig. 6 shows the corresponding microphotographs. Some extra miscellaneous circuitry has been added to both circuits to enable testing the output current and the possibility to either open or close the feedback loop. Dummy switches were also added to reduce the influence of clock feedthrough. All current amplifiers were binary-weighted for the issue of programmability. Bias current I, for the delay stages was set (4 (b) Figure 4 (a)Tunable transconductor. (b) Programmable current mirror. i - P Figure 5 Weight (iJih) variation with IBZIIB for different values of IB~IlB in Fig. 4(b). (a) Transconductors operating in weak inversion I - 10nA. (b) 'kansconductors operating in strong inversio;, 50pA. to 50pA. Total area occupation is 0.096mmL for the Nagumo-Sat0 neuron, and 0.225mm2 for the Aihara neuron. Fig. 7 shows the characteristics measured in open loop for both circuits, using the HP41 45 semiconductor analyzer, with a rail-to rail power supply of only 3v. For the Nagumo-Sat0 neuron (Fig.7(a)), a = 20pA, k = 1 and the input ranges from -20pA to 20pA. For the Aihara neuron (Fig.7(b)), E = 2pA, y= 10 , k = 1 and the input sweeps from -1OpA to IOW. In both prototypes deviation from linearity is less than 0.2%, and the measured current offset amounts to few PA'S. (a) Nagumo-Sato Model (b) Aihara Model Figure 6 Microphotographs of the prototypes. ah) 6w 20.0 4.0 Idiv 0.0 -20.0 -20.0 0.0 20.0 I 2.Wdlv (U) (a) Nagumo-Sato Model -20.0 0.0 20.0 I 2.Wdiv (pA) (b) Aihara Model Figure 7 Measured open-loop characteristics. Fig. 8 shows the experimental bifurcation trees for both neurons, when the damping factor k = 0.5 and the neuron excitation A, taken as the bifurcation parameter, vanes from -20pA to 20~A. All other parameters were fixed to the values previously cited. Clock frequency was set to 500kHz. All these measurements are in full accordance with the theoretical results. Also, given the reduced area, as well as operating with only 3v power supply, we believe both neuron implementations are very appropriate for high density chaotic neural networks on a single chip. RERERENCES G. Matsumoto, K. Aihara, Y. Hanyu, N. Takahashi, S. Yoshizawa and J. Nagumo: “Chaos and Phase Locking in Normal Squid Axons”. Physics Letters A, Vol. 123 pp. 162-166, Aug. 1987. C. A. Skarda and W. J. Freeman: “How Brains make chaos in order to make sense of the world’. Brain and Behavioral Science, Vol. 10, pp. 161-195, 1987. K. Aihara, T. Takabe and M. Toyoda: “Chaotic Neural Networks”. Phys. Lett. A, Vol. 144. pp. 333-340, Mar. 1990. 502 (a) Nagumo-Sato Model (b) Aihara Model Figure 8 Bifurcation.diagrams. J. Nagumo and S. Sato: “On a Response Characteristic of a Mathematical Neuron Model”. Kybernetics, Vol. 10. pp. 155-164. 1972. K. Judd and K. Aihara: “Neural Networks and Chaos”. Proc. of the 1992 IEEE Int. Synp. on Circuits and S?stems. pp. 2765-2768, May 1992. M. lnoue and A. Nagayoshi: “A Chaos Neuro-Computer”. M. Adachi et al.: “Nonlinear Associative Dynamics in a Chaotic Neural Network”. Proc. ofthe 2nd International Conference on Fuzzy Logic & Neural Networks, pp. 947-950. Jul. 1992. D. L. Birx and S. J. Pipenberg: “Chaotic Oscillators and Complex Mapping Feed Forward Networks (CMFFNS) for Signal Detection in Noisy Environments”. lnt. Joint Conf on Neirrul Networks, Vol.11, pp. 881-888, Jun. 1992. J. B. Hughes, I. C. Macbeth and D. M. Patullo: “Switched Currents Filters”. IEE Proceedings, Vol. 137. Pt. G, No. 2, pp. 156-162, April 1990. Z. Wang: “Current-Mode Analog Integrated Circuits and Linearization Techniques in CMOS Technology“. Hartung-Gorre Verlag, Konstanz 1990. E.A. Vittoz: “MOS Transistors Operated in the Lateral Bipolar Mode and Their Application in CMOS Technology”. IEEE J. Solid-State Circuits, Vol. SC-18, pp 273-279. 1983. T. Pan and A.A. Abidi: “A 50dB Variable Gain Amplifier Using the Parasitic Bipolar Transistors in CMOS”. IEEE J. Solid-state Circuits, Vol. SC-24. pp 95 1-961, 1989. Phy~. Lett. A, V01.158, pp. 373-376, 1991.