A Programmable Neural Oscillator Cell

756 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 A Programmable Neural Oscillator Cell BERNABE LINARES-BARRANCO, EDGAR SANCHEZ-SINENCIO, SENIOR MEMBER, IEEE, ANGEL RODR~GUEZ-VAZQUEZ, MEMBER, IEEE, AND JOSE L. HUERTAS, MEMBER, IEEE Abstruct -A programmable analog neural oscillator cell architecture is presented. The proposed neuron circuit is of hysteretic neural nature with its implementation based on operational transconductance amplifiers (OTA’s). The hysteresis loop as well as the frequency of oscillation are voltage (or current) dependent. The architecture, which involves two OTA’s, a current +rror, a capacitor, a diode, and a resistor is very suitable for monolithic integrated circuits. Experimental results confirm the expected flexibility of the synthetic neuron. I. INTRODUCTION HE MOST efficient computational system is the hu- T man brain. Biological systems ordinarily perform reasoning and logcal manipulations beyond the capabilities of our most modern sophisticated computer systems. Many researchers motivated by that efficiency try to mimic the human nervous system to solve a variety of complex engineering problems, among them: control systems, knowledge processing and signal/image (sensor) processing. A neural system consists of many highly interconnected neurons. Since in general neural networks involve a very large number of parallel arrays of basic cells (neurons), they are very appropriate for VLSI technology. Besides, there are some interesting applications where only a limited number of neurons are used, e.g., low-order optimization problems [l], [2], crude respirator control systems simulation [l], [3]. A biological neuron has both excitatory and inhibitory connections. Each neuron has inputs which are the outputs of other neurons as well as external (stimuli) signals, and if the sum of the inputs is greater than a certain threshold level, a firing sequence of pulses is generated, otherwise below ths level the neuron will not fire. The number of pulses fired out depends on the intensity and duration of the input pulse, as well as on the inherent characteristics of the neuron. Manuscript received June 24, 1988; revised December 20, 1988. This paper was recommended by Guest Editors R. W. Newcomb and N. El-Leithy. B. Linares-Barranco was with the University of Seville, 41012 Seville, Spain. He is now with the Department of Electrical Engineering, Texas A&M University, College Station, TX 77843. E. Sanchez-Sinencio is with the Department of Electrical Engineering, Texas A & M University, College Station, TX 77843-3128. A. B. Rodriguez-Vasqua and J. L. Huertas are with the Department of Electronics and Electromagnetics, Faculty of Physics, University of Seville, 41012 Seville, Spain. IEEE Log Number 8826710. Synthetic neurons behave in a crude similar way to the actual neurons. In ths paper, a programmable neuron circuit architecture is presented. The implementation is based on the operational transconductance amplifier (OTA), this voltage (or current) programmable device is very suitable for synthetic neurons. The proposed neuron circuit is of the hysteretic neural-type pulse oscillator [4] with the inhibitory and excitatory properties as function of the input sum exceeding (or not) a firing threshold level. 11. MATHEMATICAL DESCRIPTION OF A HYSTERETIC NEURON A basic first-order state-variable type equation [ 11, [4] simulating (in a primitive way) a neuron is given by dx dt C- = - G,x - G,H(x) - GJJ y=x (W where U = input, y = output, x = internal state variable; C, G,, G,, and G, are non-negative constants, and H(x) describes the hysteresis nature of the neuron model if x>x, - H-, H, I, ifx=-x- orx=x+ (2) Observe that there are other neuron models with similar mathematical characterization. For instance, the term - G,H(x) in other characterizations [2], [5] is related to -C,”=,DJ,h(x) where DJ, is a weighting function, and f,( .) are nonlinear functions imposing constraints and can be considered as threshold-activation functions. In what follows, we rewrite (1) in a form most useful for our circuit implementation, i.e., dx C-=-GG,(~)~+Z,(x) dt y=x (3b) where I,(x) is - G,H(x), and the input variable U con- 0098-4094/89/0500-0756$01 .OO 01989 IEEE Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply. LINARES-BARRANCO er al. : PROGRAMMABLE OSCILLATOR CELL Fig. 1. Equilibrium point characteristics trols the value of GJu), and the - G,u term has been deleted. The equilibrium points are obtained when dx/dt = 0, this yields Z,(X) =G,(u)x. (4) The intersections of the hysteresis function Zo(x) with resistive (load-line) G,( u)x provide the equilibrium points, as shown in Fig. 1. It can be observed from Fig. 1 that point A is a stable point, and point B is an unstable point. The asymmetry of the hysteresis loop is needed to guarantee only one stable point. For our circuit implementation we need a mechanism to change the slope G,(u) such that the straight line moves from point A to point A' and becomes unstable when U becomes excitatory. Hence, the circuit oscillates and fires negative pulses when the input signal U exceeds a certain threshold level; ths package of pulses is terminated when U falls below threshold. Thus, there is a critical value G, which separates points like A and A', e.g., G,( U) > G, stable point A (5) Gx( U) < G, unstable point A'. Fig. 2 illustrates the time response of this neuron for a current pulse u(t) that makes G,(t) to jump between G, and G, so that G, < G, < G,. This input pulse makes the stable equilibrium point A to change to point A' which is unstable. At the initial time of change of U the output of the hysteretic element starts at I+, and, since x cannot change instantaneously, the differential equation characterizing the neuron yields dx I+ C- = - G,x + If with x(0) = -. dt Gl (6a) Hence, the time solution initially is given by This equation holds until x(t) becomes x, = Z+/G,, at which time the output of the hysteretic element changes to - I- (we assume the ideal case where no time is taken in going from I, to - I- on the hysterisis curve) and let to be this time, with x(t,) = I+/G,. Therefore solving for to II I to' tl' t2 Fig. 2. Characteristic time response of neuron cell we obtain to=-ln( C 1-G2/Gl 1 G2 1 - G2/Gc Now the characterizing differential equation becomes (7) which solution is given by This situation is maintained until x(t) becomes - x-, and Io(x) then changes back to + I+ to return to point A and, hence to complete the cycle that will generate a neural-type pulse output. Thus x(t,) = - x-. (10) Solving for t, yields C t, = - In G, I'G, 1+- I- G, G2 . 1--x- I- Now the differential equation becomes (6a) again dx dt C- = - G,x + I+ but with x(0) = - x- (12a) and the solution is given by The time at which Io(x) changes from I+ to - I- occurs Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply. 758 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 - Fig. 3. Block diagram of neural-pulse simulator. when in (12b) I+ x(t,) = - GC thus t, results If T is the duration of the input pulse U and np is the number of pulses generated by the neuron, the next inequalities hold: to + (np -l)(t,+ t,) < T< to+ n,(t, + t,). (15) Hence, if one were to hold G, constant at G,<Gc then the number of pulses that would occur while G, remains constant is n,, = Int (:;;:)+1 - where Int(.) produces the integer part of the function ((T- to)/(t,+ t2)). 111. A PROPOSED NEURAL PULSE SIMULATOR A block diagram of the proposed neural-type circuit which implements (3a) and (3b) is shown in Fig. 3. In our circuit, the outputs of the hysteretic element Io(x) and of the externally controllable linear amplifier G,(u) are currents, while their inputs are voltages. Accordingly, the integrator is built with a simple capacitor and the current summer element simply implements KCL and, hence needs no additional circuitry. In order to implement the transconductance hysteretic element to yield Zo(x), we used a differential input transconductance comparator with multiple outputs all taking values as shown in Fig. 4. An antimetric transconductance-hyster- etic element can be implemented as shown in Fig. 5 where R =l/Gc, and G, is as defined before. Furthermore, i I- Fig. 4. A transconductance comparator with multiple outputs. (b) Fig. 5. Antimetric transconductance-hysteretic element. (a) Circuit implementation. (b) Characteristics of load. f'h -E -*x 2 -I - Fig. 6. Hysteresis characteristics of circuit of Fig. 5. observe that since ui = V, - x 1) if x < V, then ui > 0 and I,, = I+ which back biases 2) if x > V, then ui < 0 and I,, = - I- and if further we the diode, giving V, = I'R and x < Z'R; choose E, < I-R then V, = - E, and x > - E,; where I+ = I- = Ibis [8]. Ibis is the controlling current of the OTA. These observations are summarized in Fig. 6. The other element needed to build up the block diagram of Fig. 3 is an externally current controlled linear transconductance amplifier. A conventional OTA is employed [SI, [9]. The transconductance gain G, is controlled by the bias current U. This is shown in Fig. 7. Now we can put together the basic elements to form the programmable neural oscillator cell as shown in Fig. 8. The equation describing the behavior of the circuit of Fig. 8 is given by (3) for which x is chosen as the capacitor voltage, ZJx) has Z+=Z-= Ibis, and G,=hu with U the bias current in the bottom OTA in Fig. 8, and h [9] is a proportional constant which depends upon temperature, device geometry, and the process. Neurons are interconnected through synapses. A synapse can be considered as a weighted relation between the output of a neuron and the input received by another one. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply. LINARES-BARRANCO et al. : PROGRAMMABLE OSCILLATOR CELL Fig. 9. Hysteresis loop measurement. / I Fig. 7. A linear transconductance amplifier. &A -- Fig. 8. A hysteretic neural cell implementation. This relation can be inhibitory or excitatory. In the proposed neuron circuit, the negative current pulses can be arbitrarily considered to be excitatory signals, while other positive current pulses could be considered the inhibitory signals. This excitatory and inhibitory property of the neuron circuit can be implemented as an integrated circuit version by means of complementary current mirrors for taking output of one neuron to feed other neurons. IV. EXPERIMENTAL RESULTS The circuit of Fig. 8 was built as a test protoboard. The commercial transconductance devices used were the VA703 OTA's. For the linearized OTA implementing G,( U), the linearizing diodes [8] provided within the chip were used as well as an additional resistive attenuator to increase the input voltage swing. The input linear range obtained was about +3 V for a k5-V power supply. 759 For the transconductance comparator with multiple outputs, three OTA's (VA703) were used. These three OTA's had their inputs and bias connected together so that they would work like an OTA with three equal outputs. The input impedances of both transconductance devices (the comparator and the linearized one, G,) are not very high, so a voltage buffer had to be added. The values (arbitrarily chosen) of the passive elements were R =1.8 k!d and C = 33 nF. The measured hysteresis loop of Io,' versus x is depicted in Fig. 9. The bias current of the comparator was adjusted to Ibis = 0.8 mA, and the voltage E, = 0.5 V. The upper trace of Fig. 10(a) shows the output current of the neuron (Io2 = I,,, = Ih) of Fig. 8 when the input U is a square wave (shown in the lower trace) of 4.4 mA. This square wave makes the transconductance of G,( U) to jump from G, = 7.7 X mho, (unstable point) being the width of the pulse (excitation time) T = 900 ps. The critical point of G,( U) is given by G, = 1/R = 5.6 x mho. If T, C, G,, G,, G,, I+, and E, are known, we can calculate the values of to, t,, t,, (via (7), (11) and (14)) as well as the number of pulses np fired during the excitation time T (eq. (16)). mho (stable point) to G, = 1.6 X The calculated values are to = 22 ps t, = 75 ps t2 = 89 ps np = 6. It can be seen from the upper trace of Fig. 10(b), where the voltage across capacitor C is shown for the same input signal U as in Fig. lO(a) that the experimental values of the previous calculated times are approximately: to = 24 ps t, = 80 p~ t, = 94 ps np = 6. 'Iol was measured across a resistor of 1.0 kQ. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply. 760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 (b) current U. Time response Vc for a square wave input U. Fig. 10. (a) Current time response V,/R for a square wave input Fig. 11. Same as Fig. 10(a) with smaller amplitude and duration of U. The difference between measured and calculated values is less than 10 percent, which can be due to component tolerances and parasitic elements. Fig. 11 shows the output response V, of circuit of Fig. 8 when the input signal becomes a pulse of smaller magnitude and duration, (T-300 ps and np =1 since (T- 4J/(4 + 12) <I. v. CONCLUSIONS AND FUTURE WORK An analog synthetic neuron architecture with voltage (or current) programmable properties has been presented. In fact, referring to Fig. 8, the proposed architecture has two degrees of freedom (Ibia and U) to modify the neuron output; they are functions of the two OTA’s which can be voltage (or current) dependent. This current dependence modifies H(I+) and G, in (1). If needed, the weight of the interconnections between neurons (synapses) could be determined by means of current mirrors. The addition of current signals from other output neurons or external stimulus at each neuron circuit is accomplished through the controlling terminal of the OTA’s (i.e., see signal U). Contrary to some other analog neural circuits where it is difficult to change their parameters [6], this neuron architecture has additional flexibility to modify its parameters according to a certain predetermined algorithm. We are currently working on the MOS integrated version of the proposed neuron architecture taking into account area [7], flexibility, and power consumption. ACKNOWLEDGMENT The authors wish to thank Professor R. W. Newcomb for his valuable comments, suggestions, and encouragement in the preparation of this paper. REFERENCES [31 [41 [51 F. C. Hoppensteadt, An Introduction to the Mathematics of Neurons. New York: Cambridge Univ. Press, 1286. D. W. Tank and J. J. Hopfield, Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533-541, May 1986. C. Von Euler, “Central pattern generation during breathing,” Trenh in Neurosci., pp. 275-277, Nov. 1980. G. Kiruthi, R. C. Ajmera, R. Newcomb, T. Yami, and H. Yazdani, “A hysteretic neural type oscillator,” in Proc. IEEE/ISCAS, vol. 3, pp. 1173-1175, May 1983. A. F. Murray and A. V. W. Smith, “Asynchronous VLSI neural networks using pulse-stream arithmetic,” IEEE J. Solid-State Circuits, vol. 23, pp. 688-697, June 1988. C. Koch, J. Marroquin, and A. Yuille, “Analog ‘neural‘ networks in early vision,” in Proc. Natl. Acad. Sci. (Biophysics), vol. 83, pp. 4263-4267, June 1986. N. El-Leithy and R. W. Newcomb, “Hysteresis in neural-type circuits,” in Proc. IEEE/ISCAS, vol. 2, pp. 993-996, June 1988. VTC Incorporated, “High Performance Integrated Circuits,” Data Book (supplement) 1987. R. L. Geiger and E. Sinchez-Sinencio, “Active filter design using operational transconductance amplifiers: A tutorial,” IEEE Circuits Deoices Mag., vol. 1, pp. 20-32, Mar. 1985. B. Linares-Barranco et al., “A novel CMOS analog neural oscillator cell,” to be presented at the ISCAS, May 1989. BernaM Linares-Barranco received the B.Sc. degree in electronic physics in 1986, and the M.S. degree in microelectronics in 1987, from the University of Seville, Spain. Currently, he is with the Department of Electrical Engineering, Texas A&M University, Texas, where he is working towards the Ph.D. degree. His research interest is in the area of nonlinear analog microelectronic design and neural networks. Edgar Shchez-Sinencio (S’72-M74-SM83) received the M.S.E.E. degree from Stanford University, CA, in 1970, and the Ph.D. degree from the University of Illinois, Champaign-Urbana, in 1973. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply. LINARES-BARRANCO et al. : PROGRAMMABLE OSCILLATOR CELL 761 In 1974, he held an industrial Post-Doctoral Since October 1978, he has been with the position with the Central Research Laboratories, Departamento de Electricidad y Electr6nica at Nippon Electric Company Ltd., Kawasaki, the University of Seville, where he is currently Japan. From 1976 to 1983 he was the Head of employed as an Associate Professor. His research Instituto Nacional de Astrofisica, Optica y Elecinterest lies in the fields of analog/digital intetrbnica, Puebla, Mexico. He was a visiting prograted circuit design, neural networks and nonfessor in the Department of Electrical Engineerlinear networks analysis and synthesis. ing at Texas A&M University during the academic years of 1979-1980 and 1983-1984, where he is currrently a Professor. He was the General Chairman of the 1983 16th Midwest S$nposium on Circuits and Systems. He was an Associate Editor of ZEEE Circuits and Systems Magazine (1983-1985), IEEE Circuifs and Devices Magazine (1985-1988) and of the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS (1985-1987). He is the co-author of the book, Switched-Capacitor Circuits (Van Nostrand- Reinhold, 1984). His current research interest include operational transconductance amplifiers and applications, computer-aided design of solid-state circuits and neural network circuit implementations. rfc J& L. Hwrtas (M’74) received the Licenciado en Fisica and the Doctor en Ciencias Fisicas degrees from the University of Seville, Spain, in 1969 and 1973, respectively. From October 1970 to September 1971, he was with the Philips International Institute, Eindhoven, The Netherlands, as a Post-Graduate Student. Since October 1971, he has been with the Departamento de Electricidad y Electr6nica at the University of Seville, where he became Assistant Professor in 1973 and a Professor in Electronics in 1981. His research interest lies in the field of multivalued logic, sequential machines, analog circuit design, and nonlinear network analysis and synthesis. Angel Rodriguez-Vbquez (M80) received the Licenciado en Fisica and the Doctor en Ciencias Fisicas degrees from the University of Seville, Spain, in 1977 and 1983, respectively. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore. Restrictions apply.