A Programmable Neural Oscillator Cell

756 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 A P og ammable Neu al Oscilla o Cell BERNABE LINARES-BARRANCO, EDGAR SANCHEZ-SINENCIO, SENIOR MEMBER, IEEE, ANGEL RODR~GUEZ-VAZQUEZ, MEMBER, IEEE, AND JOSE L. HUERTAS, MEMBER, IEEE Abs uc -A p og ammable analog neu al oscilla o cell a chi ec u e is p esen ed. The p oposed neu on ci cui is o hys e e ic neu al na u e wi h i s implemen a ion based on ope a ional ansconduc ance ampli ie s (OTA’s). The hys e esis loop as well as he equency o oscilla ion a e ol age (o cu en ) dependen . The a chi ec u e, which in ol es wo OTA’s, a cu en + o , a capaci o , a diode, and a esis o is e y sui able o monoli hic in eg a ed ci cui s. Expe imen al esul s con i m he expec ed lexibili y o he syn he ic neu on. I. INTRODUCTION HE MOST e icien compu a ional sys em is he hu- T man b ain. Biological sys ems o dina ily pe o m ea- soning and logcal manipula ions beyond he capabili ies o ou mos mode n sophis ica ed compu e sys ems. Many esea che s mo i a ed by ha e iciency y o mimic he human ne ous sys em o sol e a a ie y o complex engi- nee ing p oblems, among hem: con ol sys ems, knowl- edge p ocessing and signal/image (senso ) p ocessing. A neu al sys em consis s o many highly in e connec ed neu- ons. Since in gene al neu al ne wo ks in ol e a e y la ge numbe o pa allel a ays o basic cells (neu ons), hey a e e y app op ia e o VLSI echnology. Besides, he e a e some in e es ing applica ions whe e only a limi ed numbe o neu ons a e used, e.g., low-o de op imiza ion p oblems [l], [2], c ude espi a o con ol sys ems simula ion [l], [3]. A biological neu on has bo h exci a o y and inhibi o y connec ions. Each neu on has inpu s which a e he ou pu s o o he neu ons as well as ex e nal (s imuli) signals, and i he sum o he inpu s is g ea e han a ce ain h eshold le el, a i ing sequence o pulses is gene a ed, o he wise below hs le el he neu on will no i e. The numbe o pulses i ed ou depends on he in ensi y and du a ion o he inpu pulse, as well as on he inhe en cha ac e is ics o he neu on. Manusc ip ecei ed June 24, 1988; e ised Decembe 20, 1988. This pape was ecommended by Gues Edi o s R. W. Newcomb and N. El-Lei hy. B. Lina es-Ba anco was wi h he Uni e si y o Se ille, 41012 Se ille, Spain. He is now wi h he Depa men o Elec ical Enginee ing, Texas A&M Uni e si y, College S a ion, TX 77843. E. Sanchez-Sinencio is wi h he Depa men o Elec ical Enginee ing, Texas A & M Uni e si y, College S a ion, TX 77843-3128. A. B. Rod iguez-Vasqua and J. L. Hue as a e wi h he Depa men o Elec onics and Elec omagne ics, Facul y o Physics, Uni e si y o Se ille, 41012 Se ille, Spain. IEEE Log Numbe 8826710. Syn he ic neu ons beha e in a c ude simila way o he ac ual neu ons. In hs pape , a p og ammable neu on ci cui a chi ec u e is p esen ed. The implemen a ion is based on he ope a ional ansconduc ance ampli ie (OTA), his ol age (o cu en ) p og ammable de ice is e y sui able o syn he ic neu ons. The p oposed neu on ci cui is o he hys e e ic neu al- ype pulse oscilla o [4] wi h he inhibi o y and exci a o y p ope ies as unc ion o he inpu sum exceeding (o no ) a i ing h eshold le el. 11. MATHEMATICAL DESCRIPTION OF A HYSTERETIC NEURON A basic i s -o de s a e- a iable ype equa ion [ 11, [4] simula ing (in a p imi i e way) a neu on is gi en by dx d C- = - G,x - G,H(x) - GJJ y=x (W whe e U = inpu , y = ou pu , x = in e nal s a e a iable; C, G,, G,, and G, a e non-nega i e cons an s, and H(x) desc ibes he hys e esis na u e o he neu on model i x>x, - H-, H, I, i x=-x- o x=x+ (2) Obse e ha he e a e o he neu on models wi h simila ma hema ical cha ac e iza ion. Fo ins ance, he e m - G,H(x) in o he cha ac e iza ions [2], [5] is ela ed o -C,”=,DJ,h(x) whe e DJ, is a weigh ing unc ion, and ,( .) a e nonlinea unc ions imposing cons ain s and can be conside ed as h eshold-ac i a ion unc ions. In wha ollows, we ew i e (1) in a o m mos use ul o ou ci cui implemen a ion, i.e., dx C-=-GG,(~)~+Z,(x) d y=x (3b) whe e I,(x) is - G,H(x), and he inpu a iable U con- 0098-4094/89/0500-0756$01 .OO 01989 IEEE Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply. LINARES-BARRANCO e al. : PROGRAMMABLE OSCILLATOR CELL Fig. 1. Equilib ium poin cha ac e is ics ols he alue o GJu), and he - G,u e m has been dele ed. The equilib ium poin s a e ob ained when dx/d = 0, his yields Z,(X) =G,(u)x. (4) The in e sec ions o he hys e esis unc ion Zo(x) wi h esis i e (load-line) G,( u)x p o ide he equilib ium poin s, as shown in Fig. 1. I can be obse ed om Fig. 1 ha poin A is a s able poin , and poin B is an uns able poin . The asymme y o he hys e esis loop is needed o gua an- ee only one s able poin . Fo ou ci cui implemen a ion we need a mechanism o change he slope G,(u) such ha he s aigh line mo es om poin A o poin A' and becomes uns able when U becomes exci a o y. Hence, he ci cui oscilla es and i es nega i e pulses when he inpu signal U exceeds a ce ain h eshold le el; hs package o pulses is e mina ed when U alls below h eshold. Thus, he e is a c i ical alue G, which sepa a es poin s like A and A', e.g., G,( U) > G, s able poin A (5) Gx( U) < G, uns able poin A'. Fig. 2 illus a es he ime esponse o his neu on o a cu en pulse u( ) ha makes G,( ) o jump be ween G, and G, so ha G, < G, < G,. This inpu pulse makes he s able equilib ium poin A o change o poin A' which is uns able. A he ini ial ime o change o U he ou pu o he hys e e ic elemen s a s a I+, and, since x canno change ins an aneously, he di e en ial equa ion cha ac- e izing he neu on yields dx I+ C- = - G,x + I wi h x(0) = -. d Gl (6a) Hence, he ime solu ion ini ially is gi en by This equa ion holds un il x( ) becomes x, = Z+/G,, a which ime he ou pu o he hys e e ic elemen changes o - I- (we assume he ideal case whe e no ime is aken in going om I, o - I- on he hys e isis cu e) and le o be his ime, wi h x( ,) = I+/G,. The e o e sol ing o o II I o' l' 2 Fig. 2. Cha ac e is ic ime esponse o neu on cell we ob ain o=-ln( C 1-G2/Gl 1 G2 1 - G2/Gc Now he cha ac e izing di e en ial equa ion becomes (7) which solu ion is gi en by This si ua ion is main ained un il x( ) becomes - x-, and Io(x) hen changes back o + I+ o e u n o poin A and, hence o comple e he cycle ha will gene a e a neu al- ype pulse ou pu . Thus x( ,) = - x-. (10) Sol ing o , yields C , = - In G, I'G, 1+- I- G, G2 . 1--x- I- Now he di e en ial equa ion becomes (6a) again dx d C- = - G,x + I+ bu wi h x(0) = - x- (12a) and he solu ion is gi en by The ime a which Io(x) changes om I+ o - I- occu s Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply. 758 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 - Fig. 3. Block diag am o neu al-pulse simula o . when in (12b) I+ x( ,) = - GC hus , esul s I T is he du a ion o he inpu pulse U and np is he numbe o pulses gene a ed by he neu on, he nex in- equali ies hold: o + (np -l)( ,+ ,) < T< o+ n,( , + ,). (15) Hence, i one we e o hold G, cons an a G,<Gc hen he numbe o pulses ha would occu while G, emains cons an is n,, = In (:;;:)+1 - whe e In (.) p oduces he in ege pa o he unc ion ((T- o)/( ,+ 2)). 111. A PROPOSED NEURAL PULSE SIMULATOR A block diag am o he p oposed neu al- ype ci cui which implemen s (3a) and (3b) is shown in Fig. 3. In ou ci cui , he ou pu s o he hys e e ic elemen Io(x) and o he ex e nally con ollable linea ampli ie G,(u) a e cu - en s, while hei inpu s a e ol ages. Acco dingly, he in eg a o is buil wi h a simple capaci o and he cu en summe elemen simply implemen s KCL and, hence needs no addi ional ci cui y. In o de o implemen he ansconduc ance hys e e ic elemen o yield Zo(x), we used a di e en ial inpu ansconduc ance compa a o wi h mul iple ou pu s all aking alues as shown in Fig. 4. An an ime ic ansconduc ance-hys e - e ic elemen can be implemen ed as shown in Fig. 5 whe e R =l/Gc, and G, is as de ined be o e. Fu he mo e, i I- Fig. 4. A ansconduc ance compa a o wi h mul iple ou pu s. (b) Fig. 5. An ime ic ansconduc ance-hys e e ic elemen . (a) Ci cui im- plemen a ion. (b) Cha ac e is ics o load. 'h -E -*x 2 -I - Fig. 6. Hys e esis cha ac e is ics o ci cui o Fig. 5. obse e ha since ui = V, - x 1) i x < V, hen ui > 0 and I,, = I+ which back biases 2) i x > V, hen ui < 0 and I,, = - I- and i u he we he diode, gi ing V, = I'R and x < Z'R; choose E, < I-R hen V, = - E, and x > - E,; whe e I+ = I- = Ibis [8]. Ibis is he con olling cu en o he OTA. These obse a ions a e summa ized in Fig. 6. The o he elemen needed o build up he block diag am o Fig. 3 is an ex e nally cu en con olled linea anscon- duc ance ampli ie . A con en ional OTA is employed [SI, [9]. The ansconduc ance gain G, is con olled by he bias cu en U. This is shown in Fig. 7. Now we can pu oge he he basic elemen s o o m he p og ammable neu al oscilla o cell as shown in Fig. 8. The equa ion desc ibing he beha io o he ci cui o Fig. 8 is gi en by (3) o which x is chosen as he capaci o ol age, ZJx) has Z+=Z-= Ibis, and G,=hu wi h U he bias cu en in he bo om OTA in Fig. 8, and h [9] is a p opo ional cons an which depends upon empe a u e, de ice geome y, and he p ocess. Neu ons a e in e connec ed h ough synapses. A synapse can be conside ed as a weigh ed ela ion be ween he ou pu o a neu on and he inpu ecei ed by ano he one. Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply. LINARES-BARRANCO e al. : PROGRAMMABLE OSCILLATOR CELL Fig. 9. Hys e esis loop measu emen . / I Fig. 7. A linea ansconduc ance ampli ie . &A -- Fig. 8. A hys e e ic neu al cell implemen a ion. This ela ion can be inhibi o y o exci a o y. In he p o- posed neu on ci cui , he nega i e cu en pulses can be a bi a ily conside ed o be exci a o y signals, while o he posi i e cu en pulses could be conside ed he inhibi o y signals. This exci a o y and inhibi o y p ope y o he neu on ci cui can be implemen ed as an in eg a ed ci cui e sion by means o complemen a y cu en mi o s o aking ou pu o one neu on o eed o he neu ons. IV. EXPERIMENTAL RESULTS The ci cui o Fig. 8 was buil as a es p o oboa d. The comme cial ansconduc ance de ices used we e he VA703 OTA's. Fo he linea ized OTA implemen ing G,( U), he linea izing diodes [8] p o ided wi hin he chip we e used as well as an addi ional esis i e a enua o o inc ease he inpu ol age swing. The inpu linea ange ob ained was abou +3 V o a k5-V powe supply. 759 Fo he ansconduc ance compa a o wi h mul iple ou - pu s, h ee OTA's (VA703) we e used. These h ee OTA's had hei inpu s and bias connec ed oge he so ha hey would wo k like an OTA wi h h ee equal ou pu s. The inpu impedances o bo h ansconduc ance de ices ( he compa a o and he linea ized one, G,) a e no e y high, so a ol age bu e had o be added. The alues (a bi a ily chosen) o he passi e elemen s we e R =1.8 k!d and C = 33 nF. The measu ed hys e esis loop o Io,' e sus x is de- pic ed in Fig. 9. The bias cu en o he compa a o was adjus ed o Ibis = 0.8 mA, and he ol age E, = 0.5 V. The uppe ace o Fig. 10(a) shows he ou pu cu en o he neu on (Io2 = I,,, = Ih) o Fig. 8 when he inpu U is a squa e wa e (shown in he lowe ace) o 4.4 mA. This squa e wa e makes he ansconduc ance o G,( U) o jump om G, = 7.7 X mho, (uns able poin ) being he wid h o he pulse (exci a- ion ime) T = 900 ps. The c i ical poin o G,( U) is gi en by G, = 1/R = 5.6 x mho. I T, C, G,, G,, G,, I+, and E, a e known, we can calcula e he alues o o, ,, ,, ( ia (7), (11) and (14)) as well as he numbe o pulses np i ed du ing he exci a ion ime T (eq. (16)). mho (s able poin ) o G, = 1.6 X The calcula ed alues a e o = 22 ps , = 75 ps 2 = 89 ps np = 6. I can be seen om he uppe ace o Fig. 10(b), whe e he ol age ac oss capaci o C is shown o he same inpu signal U as in Fig. lO(a) ha he expe imen al alues o he p e ious calcula ed imes a e app oxima ely: o = 24 ps , = 80 p~ , = 94 ps np = 6. 'Iol was measu ed ac oss a esis o o 1.0 kQ. Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply. 760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 (b) cu en U. Time esponse Vc o a squa e wa e inpu U. Fig. 10. (a) Cu en ime esponse V,/R o a squa e wa e inpu Fig. 11. Same as Fig. 10(a) wi h smalle ampli ude and du a ion o U. The di e ence be ween measu ed and calcula ed alues is less han 10 pe cen , which can be due o componen ole ances and pa asi ic elemen s. Fig. 11 shows he ou pu esponse V, o ci cui o Fig. 8 when he inpu signal becomes a pulse o smalle mag- ni ude and du a ion, (T-300 ps and np =1 since (T- 4J/(4 + 12) <I. . CONCLUSIONS AND FUTURE WORK An analog syn he ic neu on a chi ec u e wi h ol age (o cu en ) p og ammable p ope ies has been p esen ed. In ac , e e ing o Fig. 8, he p oposed a chi ec u e has wo deg ees o eedom (Ibia and U) o modi y he neu on ou pu ; hey a e unc ions o he wo OTA’s which can be ol age (o cu en ) dependen . This cu en dependence modi ies H(I+) and G, in (1). I needed, he weigh o he in e connec ions be ween neu ons (synapses) could be de- e mined by means o cu en mi o s. The addi ion o cu en signals om o he ou pu neu ons o ex e nal s imulus a each neu on ci cui is accomplished h ough he con olling e minal o he OTA’s (i.e., see signal U). Con a y o some o he analog neu al ci cui s whe e i is di icul o change hei pa ame e s [6], his neu on a chi- ec u e has addi ional lexibili y o modi y i s pa ame e s acco ding o a ce ain p ede e mined algo i hm. We a e cu en ly wo king on he MOS in eg a ed e sion o he p oposed neu on a chi ec u e aking in o accoun a ea [7], lexibili y, and powe consump ion. ACKNOWLEDGMENT The au ho s wish o hank P o esso R. W. Newcomb o his aluable commen s, sugges ions, and encou age- men in he p epa a ion o his pape . REFERENCES [31 [41 [51 F. C. Hoppens ead , An In oduc ion o he Ma hema ics o Neu ons. New Yo k: Camb idge Uni . P ess, 1286. D. W. Tank and J. J. Hop ield, Simple neu al op imiza ion ne wo ks: An A/D con e e , signal decision ci cui , and a linea p og amming ci cui ,” IEEE T ans. Ci cui s Sys ., ol. CAS-33, pp. 533-541, May 1986. C. Von Eule , “Cen al pa e n gene a ion du ing b ea hing,” T enh in Neu osci., pp. 275-277, No . 1980. G. Ki u hi, R. C. Ajme a, R. Newcomb, T. Yami, and H. Yazdani, “A hys e e ic neu al ype oscilla o ,” in P oc. IEEE/ISCAS, ol. 3, pp. 1173-1175, May 1983. A. F. Mu ay and A. V. W. Smi h, “Asynch onous VLSI neu al ne wo ks using pulse-s eam a i hme ic,” IEEE J. Solid-S a e Ci - cui s, ol. 23, pp. 688-697, June 1988. C. Koch, J. Ma oquin, and A. Yuille, “Analog ‘neu al‘ ne wo ks in ea ly ision,” in P oc. Na l. Acad. Sci. (Biophysics), ol. 83, pp. 4263-4267, June 1986. N. El-Lei hy and R. W. Newcomb, “Hys e esis in neu al- ype ci cui s,” in P oc. IEEE/ISCAS, ol. 2, pp. 993-996, June 1988. VTC Inco po a ed, “High Pe o mance In eg a ed Ci cui s,” Da a Book (supplemen ) 1987. R. L. Geige and E. Sinchez-Sinencio, “Ac i e il e design using ope a ional ansconduc ance ampli ie s: A u o ial,” IEEE Ci - cui s Deoices Mag., ol. 1, pp. 20-32, Ma . 1985. B. Lina es-Ba anco e al., “A no el CMOS analog neu al oscilla o cell,” o be p esen ed a he ISCAS, May 1989. Be naM Lina es-Ba anco ecei ed he B.Sc. de- g ee in elec onic physics in 1986, and he M.S. deg ee in mic oelec onics in 1987, om he Uni- e si y o Se ille, Spain. Cu en ly, he is wi h he Depa men o Elec ical Enginee ing, Texas A&M Uni e si y, Texas, whe e he is wo king owa ds he Ph.D. deg ee. His esea ch in e es is in he a ea o nonlinea analog mic oelec onic design and neu al ne - wo ks. Edga Shchez-Sinencio (S’72-M74-SM83) ecei ed he M.S.E.E. de- g ee om S an o d Uni e si y, CA, in 1970, and he Ph.D. deg ee om he Uni e si y o Illinois, Champaign-U bana, in 1973. Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply. LINARES-BARRANCO e al. : PROGRAMMABLE OSCILLATOR CELL 761 In 1974, he held an indus ial Pos -Doc o al Since Oc obe 1978, he has been wi h he posi ion wi h he Cen al Resea ch Labo a o ies, Depa amen o de Elec icidad y Elec 6nica a Nippon Elec ic Company L d., Kawasaki, he Uni e si y o Se ille, whe e he is cu en ly Japan. F om 1976 o 1983 he was he Head o employed as an Associa e P o esso . His esea ch Ins i u o Nacional de As o isica, Op ica y Elec- in e es lies in he ields o analog/digi al in e- bnica, Puebla, Mexico. He was a isi ing p o- g a ed ci cui design, neu al ne wo ks and non- esso in he Depa men o Elec ical Enginee - linea ne wo ks analysis and syn hesis. ing a Texas A&M Uni e si y du ing he aca- demic yea s o 1979-1980 and 1983-1984, whe e he is cu en ly a P o esso . He was he Gene al Chai man o he 1983 16 h Midwes S$nposium on Ci cui s and Sys ems. He was an Associa e Edi o o ZEEE Ci cui s and Sys ems Magazine (1983-1985), IEEE Ci cui s and De ices Magazine (1985-1988) and o he IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS (1985-1987). He is he co-au ho o he book, Swi ched-Capaci o Ci cui s (Van Nos and- Reinhold, 1984). His cu en esea ch in e es include ope a ional ansconduc ance ampli ie s and applica ions, compu e -aided design o solid-s a e ci cui s and neu al ne wo k ci cui implemen a ions. c J& L. Hw as (M’74) ecei ed he Licenciado en Fisica and he Doc o en Ciencias Fisicas deg ees om he Uni e si y o Se ille, Spain, in 1969 and 1973, espec i ely. F om Oc obe 1970 o Sep embe 1971, he was wi h he Philips In e na ional Ins i u e, Eind- ho en, The Ne he lands, as a Pos -G adua e S u- den . Since Oc obe 1971, he has been wi h he Depa amen o de Elec icidad y Elec 6nica a he Uni e si y o Se ille, whe e he became Assis- an P o esso in 1973 and a P o esso in Elec- onics in 1981. His esea ch in e es lies in he ield o mul i alued logic, sequen ial machines, analog ci cui design, and nonlinea ne wo k analy- sis and syn hesis. Angel Rod iguez-Vbquez (M80) ecei ed he Licenciado en Fisica and he Doc o en Ciencias Fisicas deg ees om he Uni e si y o Se ille, Spain, in 1977 and 1983, espec i ely. Au ho ized licensed use limi ed o: Uni e sidad de Se illa. Downloaded on Ap il 01,2020 a 16:04:47 UTC om IEEE Xplo e. Res ic ions apply.