Current-mode building blocks for CMOS-VLSI design of chaotic neural networks

Current-Mode Building Blocks for CMOS-VLSI Design of Chaotic Neural Networks Manuel Delgado-Restituto and Angel Rodriguez-VAquez Departanent of Analog Design Centro Nacional de Microelectronica cnarfia sn, 41012-Sevilla, Spain Abstruct -- This paper presents two nonlinear CMOS current-mode circuits that implement neuron soma equations for chaotic neural networks, and another circuit to realize programmable current-mode synapse using CMOScompatible BJT’s. They have been fabricated in a double-metal, single-poly 1.6pm CMOS technology and their measured performance reached the expected function and specifications. The neuron soma circuits use a novel, highly accurate CMOS circuit strategy to realize piecewiselinear characteristics in current-mode domain. Their prototypes obtain reduced area and low voltage power supply (down to 3v) with clock frequency of SOOkHz. As regard to the synapse circuit, it obtains large linearity and continuous, linear, weight adjustment by explotation of the exponential-law operation of CMOS-BJT’s. The full accordance observed between theory and measurements supports the development of future analog VLSI chaotic neural networks to emulate biological systems and advanced computation. ~.INTRODUCTION Most artificial neural networks use a simple neuron model where the processing redized by the soma involves a static nonlinear transformation. with either sigmoid or threshold characteristics [ 11. However. recent studies on real nerve membranes in neurophysiological experiments have shown that the dynamic behavior of biological neurons is much more complex (including chaotic response) than that exhibited by simple models [23, [3]. Consequently new schemes of artificial neural networks have emerged to more realistically emulate the chaotic responses experitnentally observed in biological systems. In particular, some remarkable chaotic neuron models have been reported by Nagumo and Sato [4]; and Aihara, Takabe and Toyoda 151. Many studies on chaotic neural networks in general, and using the previous models in patticular. reveal that such networks serve not only as an experimental vehicle in the study of sensory nerve systems. but also lead to important engineering applications. In this sense. chaotic neural networks have been proposed to solve difficult optimization problems [63. [7]; for dynamical associative pattern class[ficatiorr [81: and for signal detmion and class$cation in noisy eiwiiuiinzeiits [C,]. and it is predictable that new applications will arise in the near future. 0-7803-1901-X/94 $4.00 01994 IEEE 3973 -~~ In spite of the strong economical interest involving these applications, few up-to-date physical implementations of chaotic neural networks have been proposed. Thus, it is advisable to give circuit realizations of these models. Furthermore, due to the technological trend towards system integration, these circuits must be wellsuited for VLSI. and, if possible, compatible with standard low cost CMOS technologies. The purpose of this communication is to report some circuit building blocks for CMOS-VLSI design of chaotic neural networks using analog current-mode techniques. Neuron soma and synapse circuits are proposed and validated using silicon prototypes in a single-poly double-metal 1 .@in CMOS technology. In particular, Nagumo-Sato’s and Aihara-Takabe-Toyoda’s chaotic neuron models, and a synapse circuit with continuous, linear, programmability which employs CMOScompatible BJT’s. Proposed circuits use innovative currentmode techniques and operate correctly for bias voltage down to 3v. 11. NEURON MODELS Fig. 1 is a block diagram for an artificial neuron, where we can distinguish a dendriticlsynuptic structure and a soma. The dendritic structure collects input signals (represented by vector I) and signals from other neurons (vector y), so that the input strength is given by, M N A;M = C w,~v~(~L) + C Vjj/hn) - e, (1) j= I j= 1 where the first term coinputes the influence of the M neurons driving the ith neuron: the second. the excitation from theN external inputs. f,; and 8, is the threshold of the ith neuron. For convenience, in Fig. 1 and (1) we have assumed that signals are discrete time. with time index represented by ri. In a conventional artificial neuron, processing performed at the soma is a simple nonlinear transformation. xi(n + 1) =,f(.xi(n)) +Aj(n) .n = 0. 1, ... (2) yi(n + 1) = fl.r;(n + 1)) where x,(n+l) and y,(n+l) are the internal state and the output of the ith chaotic neuron at the discrete time n+I, respectively: andfi.) is a threshold function. In Nagumo-Sato's model that processing also involves dynamic operators. ~,(n + 1) = I;s,(N) - au(.~-,(n)) + A,(N) .rt = 0. 1. ..(3) y,(n + 1) = t4(.1,(n + 1)) where a and k are the scaling and damping factors of refractoriness (residual effect of a neuron once fired). respectively: and I{(.) is a unit-step function. Numerical studies realized with this model show that chaotic responses occur only for a set of parameters with zero measure. To extend the range of parameters lor which chaotic behavior cm be observed, the chaotic rrerrral rmvork reported in [4] constitutes ii modification of the NagumoSato's model [5], where instead of following a11 all-ornone law for the action potential. modelled by an unit step output function it(-), the neuron shows a continuously graded stimulus-response curve, represented by n non-linear functionfi.) The model in [4] is defined by the following finite-difference equations: wherefi.) constitutes the neuron output function and can be represented by the piecewise-linear mcxiel. where E is a positive number defining the steepness of the function. 111. CURRENT MODE IMIZEMENTATIONS Cirrrent-mode techniques have been employed to implement (3) and (4). following the conceptud diagram shown in Fig.1. Summation is easily realized exploiting KCL. The delay operation can be realized as a cascade of two track-and-hold switched-current stages, as proposed by Hughes et al. [9]. Fig. 2 shows the schematics for this block. 4 1 : 1+1 : B Fig. 2. Current-mode track and hold circuit. Nonlinearities have been achieved using a novel, highly accurate CMOS circuit strategy to realize piecewise-linear (PL) characteristics in current-mode domain. It is based on the rectifying characteristics of the current switch [ 103, which provide very high resolution and virtually zero current offset. uninfluenced by transistor mismatches (in fact. minimum size transistors were used in both prototyp). Fig. 3 shows the corresponding schematics for both PL functions. Current amplifiers can be implemented by properly ratioed bilateral current mirrors. whose programmability will be adressed in the next sec tion. First. let us consider implementation of the nonlinear block. g(.). for the Nagumo-Sato model according to the concept of Fig. 1. Fig3a). which consists of two current 4 I (a) (b) Nagurno-Sato Model (4 Aihara Model Fig. 3. Piecewise linear mapping circuits for the neurons. 3974 sources (realized in practice by current mirror output branches), four transistors, and two digital inverters, shows a conceptual schematic for the realization of the PL characteristics of Fig3(b). Transistors Wsn and bSp in Fig.3(a) operate as a current-controlled-currentswitch, while transistors MVSn and MVsp operate as voltage-controlled current switches. Any positive input current increases the input voltage, turning the Mcsp device ON, and since both devices in the current switch have the same gate voltage, Qsn OFF. Siinultaneously, the voltage at the second inverter output evolves to the high logic state, turning Mvsl, ON and MVsp OFF. Thus, a current kx(n) - a (obtained by KCL at node NI) is directed to the output node through the transistor Mvsn -- the right-hand piece of Fig.3(b) is implemented in this inanner. Similarly, negative input currents turn MCSn and MVsp ON, so that a current kx(rr) + 0: circulates through MVsp to the output node. The output response of the neuron can be taken from the voltage at the output of the inverter IN2 which swings from rail to rail depending on the sign of the input current. thus resembling the required unit step function in voltage mode. Obviously. this binary signal must be combined with analog switches and current sources to properly drive other neurons in the network. Since current discrimination in the proposed circuit relies on the integration function performed at the input node. resolution is very high, uninfluenced by transistor mismatches (measurements from the CMOS prototype display resolution of 12pAs). Operation speed is also very high, limited mainly by nonlinear transients in the transistors that implement the current sources used to drive nodes NI and N2. Also. the feedback created by inverter INI yields significant reduction of the dead-zone exhibited by the driving point characteristics measured at the input node, that is proportional to (VT,~ + IVT,I)/ A,,,, where VT,~ and Vrp are the threshold voltages for the tram sistors and A,,,,, is the inverter DC gain. This is an appealing feature that enables reducing the interstage loading errors caused by finite equivalent MOS transistors EiUly voltages. Fig. 3(c) shows the corresponding schematic to iinplemeiit the PL characteristic shown in Fig. 3(d) for the Aihara model, . The current switches. together with the current sources, are used to discriminate the input current into three paths depending on whether .u(I~) is lower than -E, greater than E. or comprised between both values. In the two first cases. the paths are routed to the same node and amplified by k, while if s(n)~ [-E. E] the current is amplified by -y. where y = a/& - k. according to ( I) and (3). In this case, the output of the neuron can be easily taken by replication and scaling the output current of the amplifier with gain -y. Iv. CURRENT MODE SYNAPSE AND DENDRITIC Smucr URE The synaptic structure in Fig.1 involves the operation of analog-weighted summation. Summation is easily performed in current-mode domain by KCL. Digital programmability c,m be incorporated by analog multiplexing of current contributions from different mirrors. These mirrors can either implement fixed weights (with application. for instance, in cases where well-defined tasks must be sequentially performed), or be binaryweighted (for more general application). Digital programmability provides ease of controllability and accurate results. at the cost of a strong area penalty. Analog programmability should be considered for reduced area and continuous weight adjustment. A simple way to achieve analog programmability is to use tunable transconductors. like that shown in Fig. 4(a). Fig. 4(b) shows a programmable current mirror using this transconductor. Two different situations arise depending on whether tr'msistors operate in weak or strong inversion. Analysis for both operating conditions shows the following , IB 2 'irr weak IB, ?I = - (6) As can be seen. the dependence is linear for weak inversion: hence. this latter case provides larger weight adjustment ranges. It is illustrated in Fig. 5, showing the current weight as a function of Ig_?/fg for different values of IgIIIg. where 18 is ii lionnalization factor of lOnA for weak inversion (Fig. 5(a)) and 5OpA for strong inversion (Fig. 5(b)). Also. nonlinearity cancellation is exact in weak inversion due to the exponential nature of current - I (4 (b) Fig. 4. (a )Tunable transconductor. (b) Progranunable current mirror. 3975 to voltage characteristics. while it is only appmximate for strong inversion: nonlinearity in the weak inversion case is less than 1% up to io=lBz, while the corresponding value for strong inversion is io=O. 1318,. Drawbacks of weak inversion are low accuracy, due to mismatch. and reduced speed. This paper overcomes this drawback using CMOS compatible lateral BJTs. which exhibit exponential feature for larger current ranges. and have excellent matching properties [ 111. V. EXPERIMENTAL RESULTS Both neurons have been fabricated in a double-metal. single-poly 1.6pm CMOS technology. Fig. 6 shows the corresponding microphotographs. Some extra miscellaneous circuitry has been added to both circuits to enable testing the output current and rhe abilily to either open or close the feedback loop. Dummy swilches were also added to reduce the influence of clock feedthrough. All i' g; 1 1 0 0.00 0.20 040 om OM loo 4IJb 1.0 - -- 2 no 1 a+- / 160 I A0.8 14+--- 1 Fig. 5. Weight (i i ) vanaiion with /,?/I8 for diffelrni values of I ,//s in Fig. J&). (a{%i~rccli~ductors uprating in wd inversion. 4 = 1OnA.e) Transconductors operaiing in sirong inversion. IS = 5OpA. current amplifiers were binary-weighted for the issue of programmability. Bias current I, for the delay stages was set to 5OpA. Total am occupation is 0.096mm2 for the Nagumo-Sat0 neuron, and 0.225"' for the Aihara neuron. Fig. 7 shows the characteristics measured in open loop for both circuits. using the HP4145 semiconductor analyzer. with a rail-to rail power supply of only 3v. For the Nagumo-Sat0 neuron (Fig. 7(a)), a = 2OpA. A: = 1, and the input ranges from -2w to 2OpA. For the Aihm neuron (Fig. 7(b)), E = 2+lA, y= 10 , A: = 1, and the input sweeps from -lo@ to lw. In both prototypes deviation from linearity is less than 0.2%. and the measured current offset amounts to few pA's. Fig. 8 shows the experimental bifurcation trees for both neurons. where the damping factor k = 0.5 and the neuron excitation A, taken as the bifurcation parameter, varies from -2OpA to 2OpA. All other parameters were fixed lo the values previously cited. Clock frequency was set lo 5OOkHz. Fig. 9 shows the schematics of a tunable transconductor using CMOS-BJTs for a n-well technology. We have fabricated a programmable current mirror using this transconductor in a 1.6pm technology and have obtained continuous weight variation in four decades range with very low current off set (nA's level). All these measurements are in full accordance with the theoretical results. Also, given the reduced area. as well as operating with only 3v power supply, we believe (a) Naguinu-Saicr-Model (b) Aihara Model Fig. 6. Micropholographs of the prolotypes. 3976 (a) Nagumo-Sato Model -3 I I I I IWI I (b) Aihara Model Fig. 7. Measured open-loop characteristics. both neuron implementations are very appropriate for high density chaotic neural networks on a single chip. RERERENCES G. Matsumoto et al.: ''Chaos and Phase Locking in Normal Squid Axons". Physics Lerrers A. Vol. 123 pp. 162166. Aug. 1987. C. A. Skarda and W. J. Frexm": "How Brains make chaos in order tci make sense of the world". Brain ond Brhavioral Science. Vol. 10. pp. 161-195. 1987. K. Aihara et al.: "Chaotic Neural Networks". Ptiys. Lerr. J. Nagumo and S. Sato: "On a Response Characteristic of a Mathematical Neuron Model". Kybrrnerics. Vol. 10, pp. K. Judd and K. Aihara: "Neural Networks and Chaos". Proc. of rile 1992 IEEE Inr. Synip. on Circriits und Sysrenis, pp. 2765-2768. May 1992. M. Inoue and A. Nagayoshi: "A Chaos Neuro-Computer". Phys. Lerr. A. Vo1.158. pp. 373-376. 1991. M. Adachi et al.: "Nonlinear Associative Dynamics in ;1 Chaotic Neural Network". Prm. qfrhe 2nd lnr. Cot$ or1 Fuzzy Lr?gic & Ncrtrcrl Networks. pp. 947-950, J ul . 1912. D. L. Birx and S. J. Pipenberg: "Chaotic Oscillators and Complex Mapping Feed Forward Networks (CMFFNS) for Signal Detection in Noisy Environments". Inr. .loin1 Conf. on NeurcrlNerworks. Vol.11. pp. 881 -888. Jun. 1992. A. Vol. 144. pp. 333-340, MU. I!2"0. 155-164, 1972. 40.0.. ... SO 30.0 , , . , . . , 1 -40.0 ao -15.0 -1ao -50 0.0 5.0 10.0 15.0 20.0 BibMiMPI.ar.A(lrA) Nagumo-Sat0 Model 40.0 I . *. BibMiMPlrm.AW) Aihara Model Fig. 8. Bifurcation diagrams. J. B. Hughes et al.: "Switched Currents Filters". IEE Proceedings. Vol. 137. Pt. G, No. 2, pp. 156-162, April 1990. 2. Wang: "Current-Mode Analog Inregraied Circuits and Linearizarion Techniques in CMOS Technology". Hartung-Gorre Verlap. Konstanz 1990. E.A. Vittoz: "MOS Transistors Operated in the Lateral Bipolar Mode and Their Application in CMOS Technology". IEEE J. Solid-Srare Circuirs. Vol. SC-18, pp 273279. 1983.. T Fig. 9. TunaMe transconductor using CMOS-BIT'S 3977