STDP and STDP Variations with Memristors for Spiking Neuromorphic Learning Systems

FOCUSED REVIEW published: 18 Feb ua y 2013 doi: 10.3389/ nins.2013.00002 STDP and STDP a ia ions wi h mem is o s o spiking neu omo phic lea ning sys ems T. Se ano-Go a edona1,T. Masquelie 2,3,T. P od omakis4,G. Indi e i5 and B. Lina es-Ba anco 1* 1Depa men o Analog and Mixed-Signal Design, Ins i u o de Mic oelec ónica de Se illa, IMSE-CNM-CSIC, Se illa, Spain 2Uni o B ain and Cogni ion, Depa men o In o ma ion and Communica ion Technologies, Uni e si a Pompeu Fab a, Ba celona, Spain 3Labo a o y o Neu obiology o Adap i e P ocesses, UMR 7102, CNRS - Uni e si y Pie e and Ma ie Cu ie, Pa is, F ance 4Cen e o Bio-inspi ed Technology, Ins i u e o Biomedical Enginee ing, Impe ial College London 5Ins i u e o Neu oin o ma ics, Uni e si y o Zu ich and ETH Zu ich, Zu ich, Swi ze land In his pape we e iew se e al ways o ealizing asynch onous Spike-Timing-Dependen - Plas ici y (STDP) using mem is o s as synapses. Ou ocus is on how o use indi idual mem is o s o implemen synap ic weigh mul iplica ions, in a way such ha i is no necessa y o (a) in oduce global synch oniza ion and (b) o sepa a e mem is o lea ning phases om mem is o pe o ming phases. In he app oaches desc ibed, neu ons i e spikes asynch onously when hey wish and mem is i e synapses pe o m compu a ion and lea n a hei own pace, as i happens in biological neu al sys ems. We dis inguish be ween wo di e en mem is o physics, depending on whe he hey espond o he o iginal “mo ing wall” o o he “ ilamen c ea ion and annihila ion” models. Independen o he mem is o physics, we discuss wo di e en ypes o STDP ules ha can be implemen ed wi h mem is o s: ei he he pu e iming-based ule ha akes in o accoun he a i al ime o he spikes om he p e- and he pos -synap ic neu ons, o a hyb id ule ha akes in o accoun only he iming o p e-synap ic spikes and he memb ane po en ial and o he s a e a iables o he pos -synap ic neu on. We show how o implemen hese ules in c oss-ba a chi ec u es ha comp ise massi e a ays o mem is o s, and we discuss applica ions o a i icial ision. Keywo ds: mem is o /cmos, a i icial-lea ning-synapses, spike- iming-dependen -plas ici y, spiking-neu al- ne wo ks Edi ed by: Ge Cauwenbe ghs, Uni e si y o Cali o nia, San Diego, USA Re iewed by: Em e O. Ne ci, Ins i u e o Neu oin o ma ics, Swi ze land Siddha h Joshi, Uni e si y o Cali o nia, San Diego, USA *Co espondence: B. Lina es-Ba anco, BSc Physics 1986, MSc 1987, PhD 1990 om Uni e si y o Se ille, Spain, and 2nd PhD 1991 om Texas A&M Uni e si y, USA. He is Full p o esso o Resea ch a he Ins i u o de Mic oelec ónica de Se illa (IMSE-CNM-CSIC). He has been in ol ed wi h ci cui design o elecommunica ion ci cui s, VLSI emula o s o biological neu ons, VLSI neu al based pa e n ecogni ion sys ems, hea ing aids, p ecision ci cui design o ins umen a ion equipmen , bio-inspi ed VLSI ision p ocessing sys ems, AER, mem is o s and eme ging nanode ices, and VLSI ansis o misma ch pa ame e s cha ac e iza ion. He is IEEE Fellow. be [email protected] 1. INTRODUCTION Fo many yea s, he ield o neu omo phic enginee ing has s uggled o de elop p ac i- cal neu o-compu ing de ices ha mimicked he p inciples and ope a ions o biological b ains, by di ec ly exploi ing he physics o elec onic de ices in mixed analog/digi al VLSI (Indi e i and Ho iuchi, 2011). Howe e , he e always was aclamo o acompac anddis ibu ednon- ola ile memo y, possibly igh ly coupled o he signal p ocessing componen s (neu ons), so ha he biological synapses coun e pa s could be p ope ly emula ed. The ecen ad en o nanoscale mem is i e-like de ices (Wu ig and Yamada, 2007; S uko e al., 2008; Yang e al., 2008; Jo e al., 2010; Go o eanu e al., 2011; Lee e al., 2011; Chan hbouala e al., 2012; Kuzum e al., 2012; P od omakis e al., 2012a)opens he possibili y o la ge-scale bio-inspi ed neu al ne wo k implemen a ions wi h minimal size- equi emen s o hose elemen s in he ci cui ha a e mos nume ous and he e o e mos F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |1 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s space-in ense: plas ic synap ic connec ions. The s eng h o a synap ic link be ween wo neigh- bo ing neu ons depends on i s his o y and mo e explici ly by he o e all amoun o neu o ans- mi e s ha has been p opaga ed h ough i a e a ele an neu al spike. In simila ashion, he s eng h o a mem is o , i.e., i s mem is ance Mem is o Two e minal elec onic de ice which ope a es simila o a esis o , bu whose esis ance changes dynamically as he de ice is being used. Spike-Timing-Dependen -Plas ici y (STDP) One ype o lea ning ule o a i icial synapses in spiking neu al ne wo ks, whe e he synap ic upda e depends on he iming cha ac e is ics o indi idual spikes a he synapse e minals. (o ins an aneous esis ance) is dic a ed by he amoun o cha ge q ha has lown h ough i o he accumula ed ol age lux φ. Addi ionally, he in insic non-linea na u e o p ac ical solid- s a e mem is o s esembles he beha io o neu- al synapses. On he o he hand, and independen ly o he new nanoscale de ices a ailabili y, he neu- omo phic enginee ing ield e ol ed na u ally owa d ci cui s and sys ems exploi ing spik- ing signal encoding, as in biology. Fo exam- ple, a la ge collec ion o spike-d i en ision senso s ha e been epo ed, such as senso s o luminance (Culu ciello e al., 2003; Chen e al., 2011), empo al con as (Ba ba o e al., 2002; Mallik e al., 2005; Chan e al., 2007a; Lich s eine e al., 2008; Leñe o-Ba dallo e al., 2011; Posch e al., 2011; Se ano-Go a edona e al., 2013), mo ion (K ame , 1996; Sa peshka e al., 1996; Ozale li and Higgins, 2005), and spa ial con as (Ruedi e al., 2003; Zaghloul and Boahen, 2004; Cos as-San os e al., 2007; Massa i e al., 2008; Leñe o-Ba dallo e al., 2010). Spike-d i en p inciples ha e also been used o audi o y sys ems (Sa peshka e al., 2005; Wen and Boahen, 2006, 2009; Chan e al., 2007b), compe i ion and Winne -Take- All ne wo ks (Indi e i, 2000; Chicca e al., 2007; Os e e al., 2008), lea ning (Mill e al., 2011), classi ica ion (Mi a e al., 2009), all de ec- ion (Fu e al., 2008), and sys ems dis ibu ed o e wi eless senso ne wo ks (Teixei a e al., 2005; Massa i e al., 2008). Apa om eal- ime sensing, spike-d i en p ocessing sys ems can p oduce ex emely as esponses. Examples o spike-d i en p ocessing modules (chips) a e hose ha , emula ing biological neoco i- cal s uc u es, pe o m spa io- empo al ea u e ex ac ion such as ixed-ke nel (Venie e al., 1997; Choi e al., 2005) o p og ammable ke nel (Se ano-Go a edona e al., 2006; Camuñas- Mesa e al., 2011, 2012) 2D con olu ions, and gene ic massi e neu al p ocessing (Vogels ein e al., 2007; Fie es e al., 2008; Khan e al., 2008; Se ano-Go a edona e al., 2009; Zama eno- Ramos, 2012). Una oidably, he lea ning capabili y is one key cha ac e is ic ha is equi ed o building cogni i e a i icial neu al sys ems. Recen ly p o- posed a i icial neu al p ocessing sys ems spend g ea esou ces o his ask: he mul i-million Eu opean ini ia i e FACETS/B ainScales (Fie es e al., 2008) is de eloping a 200.000 neu on wa e 1whe e mos o he silicon a ea is used o implemen ing Spike-Timing-Dependen - Plas ici y (STDP) lea ning mechanisms in he synapses. The UK ini ia i e SpiNNake (Khan e al., 2008) based on mul i-p ocesso s ARM echnology has o use hyb id packaging echnology in o de o encapsula e wo sepa a e Silicon chips in o each chip package: one chip is being he genuine SpiNNake chip wi h 18 ARM 2CPUs, and he second chip being a comme cial 128MB DRAM chip o local synap ic s o age. Bo h he lea ning mechanisms and he s o age o lea ned pa ame e s equi e subs an ial silicon eal-es a e in adi ional silicon-based chip echnology. Howe e , he ad en o new nanoscale ech- nologies has shed new expec a ions, gi ing hopes o he de elopmen o ul a-compac , as and e icien lea ning and s o age mecha- nisms ha may esul in a o dable, low powe , compac , la ge scale, a i icial neu al sys ems (Wu ig and Yamada, 2007; S uko e al., 2008; Yang e al., 2008; Jo e al., 2010; Go o eanu e al., 2011; Lee e al., 2011; Chan hbouala e al., 2012; Kuzum e al., 2012; P od omakis e al., 2012a). A e y p omising new class o nanoscale de ices is he one ha comp ises he so called mem is- o s (Chua, 1971; Chua and Kang, 1976; S uko e al., 2008; Bo ghe i e al., 2009; Jo e al., 2009, 2010), whose dis inc cha ac e is ic is ha hey ha e memo y while hey ope a e like a iable wo- e minal esis o s.I was ecen lypos u- la ed ha such iny nanoscale de ices, when d i en by app op ia ely shaped ol age pulses, could be embedded wi hin adi ional CMOS3 mic ochips, esul ing in uly asynch onous4 a i icial lea ning neu al “ issue” equipped 1Wa e : mic ochips a e ab ica ed on silicon wa e s (wi h diame e anging om 1 o abou 18), which can hold se e al hund eds o housands o indi idual chips ha a e la e on cu and encapsula ed in o chip packages. The FACETS/B ainScaleS p ojec is a wa e -scale design, meaning ha he wa e is no cu in o indi idual chips, bu he whole wa e is used as a uni ci cui . 2ARM s ands o “Aco n Risk Mic op ocesso ” and is he name o a company p o iding embedded mic op oces- so s o a a ie y o mo e complex chips, like cellula phones, usb-s icks, e c. 3CMOS s ands o “Complemen a y Me al Oxide Semiconduc o ” and e e s o he mos s anda d echnol- ogy used o mic ochip ab ica ion. 4In asynch onous sys ems no clock is equi ed, as opposed o con en ional digi al compu ing sys ems. Consequen ly, he e is no cen alized ime keepe ha en o ces ac ions o happen in lock-s ep wi h each o he . F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |2 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s wi h STDP (Lina es-Ba anco and Se ano- Go a edona, 2009b,a; Zama eño-Ramos e al., 2011). Al hough his s ill needs o be p o en expe - imen ally and all p ac ical limi a ions a e ye o be iden i ied, while mem is o s a e con inu- ously being imp o ed and op imized o e many labs wo ldwide, he po en ial o building e y dense hyb id mem is i e-CMOS lea ning sys- ems is he e. The esul ing implemen a ions can be ex emely compac STDP-equipped sys- ems, which con as wi h pu e CMOS-based a emp s ha ei he ha e esul ed in physical STDP synapses consuming signi ican chip eal- es a e (Fie es e al., 2008)o complexcom- pu a ional wo k a ounds in mo e algo i hmic solu ions (Ras e al., 2010; Da ies e al., 2012). In his pape we quickly e iew he basic p in- ciples behind exploi ing mem is ance o asyn- ch onous STDP and ex end he o iginal indings o o he ypes o STDP. In he nex sec ion we quickly e iew he mem is o concep as well as some o he pos ula ed physical mech- anisms esponsible o i s ope a ion. A e his, sec ion 3summa izes STDP and some a ia- ion o i , as well as addi i e, mul iplica i e and quad a ic STDP. Sec ions 4and 5 e iew how o combine mem is o s wi h speci ic CMOS neu ons o esul in di e en ypes o STDP. Sec ion 6men ions an applica ion in he con- ex o a i icial ision, and sec ion 7p o ides conclusions. 2. MEMRISTORS Mem is ance was pos ula ed in 1971 by Chua (1971) as he ou h missing canonical ci cui elemen h ough his amous symme y a gu- men , illus a ed in Figu e 1. Acco ding o ci - cui heo e ical undamen als, he e a e ou basic elec ical quan i ies (Chua e al., 1987): (1) ol age di e ence be ween wo e minals “ ,” (2) cu en lowing h ough in o a de ice e minal “i,” (3) cha ge lowing h ough a de ice e mi- nal o in eg al o cu en q=i(τ)dτ,and(4) lux o in eg al o ol age φ= (τ)dτ.A wo- e minal de ice is said o be canonical (Chua e al., 1987) i ei he wo o he ou basic elec i- cal quan i ies a e ela ed by a s a ic5 ela ionship, as shown in Figu e 1. A esis o has a s a ic ela- ionship be ween e minal ol age and de ice cu en i,asshowninFigu e 1B. A capaci o shows a s a ic ela ionship be ween cha ge qand ol age ,asshowninFigu e 1C.Aninduc o has a s a ic ela ionship be ween i s cu en i and lux φ,asshowninFigu e 1D.These h ee de ices ha e been e y well known since he o igins o Elec onics and Elec ici y. Howe e , he e a e o he possibili ies o combining he 5By “s a ic” we mean i is no al e ed by changes o he abo e elec ical quan i ies, o by hei his o y, in eg als, de i a i es, e c. These “s a ic” cu es can, howe e , be ime- a ying i he change is caused by an ex e nal agen . Fo example, a mo o d i en po en iome e would ha e a “s a ic” i/ cu e ha is ime a ying. FIGURE 1 | Fou a iables o ci cui heo y linked by six ma hema ical ela ions consis ing o he unc ional ela ionships o he ou passi e ci cui elemen s, Fa adays law o induc ion and he de ini ion o elec ic cu en . (A) Chua’s symme y a gumen and (B–E) desc ip ions o he ou canonical wo- e minal de ices. (B) A esis o is de ined by a s a ic ela ionship be ween a de ice’s ol age and cu en . (C) A capaci o is de ined by a s a ic ela ionship be ween a de ice’s cha ge and ol age. (D) An induc o is de ined by a s a ic ela ionship be ween a de ice’s cu en and lux. (E) And a mem is o is de ined by a s a ic ela ionship be ween a de ice’s cha ge and lux. F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |3 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s ou basic elec ical quan i ies: (q,i),( ,φ),and (q,φ). Igno ing he combina ions o a quan- i y wi h i s own ime de i a i e lea es us wi h one single addi ional possibili y: (q,φ).This easoning led Chua o pos ula e he exis ence o a ou h basic wo- e minal elemen , which he called he Mem is o .Mem is o s beha e as esis ances in which he esis ance changes h ough some o he basic elec ical quan i- ies, and is somehow memo ized. The mem is- o would show a s a ic ela ionship be ween cha ge qand lux φ,asshowninFigu e 1E.I he q s. φ ela ionship is linea , he mem is- o degene a es in o a linea esis o . Al hough none o he so- a epo ed mem is o s can be desc ibed by a s a ic cons i u i e ela ionship in he (q,φ)plane (and hus, s ic ly speaking, he 1971 ou h canonical elemen is s ill missing), hey all all wi hin Chua’s 1976 gene aliza ion o Mem is i e Sys ems (Chua and Kang, 1976). F om he e on we will use he e m mem is o o Chua’s 1976 de ini ion o mem is i e sys em. Consequen ly, he simple concep o mem is- ance as de ined in Figu e 1D can be ex ended o e e o any de ice exhibi ing esis i e beha io (i s i/ cu es c oss he o igin) whose esis ance can change h ough some o he ou basic elec- ical quan i ies (o a combina ion o hem, o hei ime de i a i es o in eg als, e c.), while a he same ime exhibi ing memo y o ha esis ance. In ha case, mo e elabo a e ma h- ema ical desc ip ions a e equi ed (Chua and Kang, 1976). Mem is ance has ecen ly been demon- s a ed (wi h ex ao dina y impac among he esea ch communi y) in nanoscale wo- e minal de ices, such as ce ain i anium- dioxide (S uko e al., 2008; Bo ghe i e al., 2009; P od omakis e al., 2011, 2012a)and amo phous Silicon (Jo e al., 2009) c oss- poin swi ches. Howe e , mem is i e de ices we e epo ed ea lie by o he g oups (A gall, 1968; P od omakis e al., 2012b). Mem is ance a ises na u ally in nanoscale de ices because small ol ages can yield eno mous elec ic ields ha p oduce he mo ion o cha ged a omic o molecula species, changing s uc u al p ope - ies o a de ice (such as i s doping p o ile) while i ope a es. I s unc ional cha ac e is ic has been a pinched hys e esis loop in he i– domain (Figu es 2C,D); a signa u e ha has been obse ed in a ious dissipa i e de ices (P od omakis e al., 2012b). Pa icula ly nowa- days a ious eme ging esis i e andom-access memo y (ReRAM) nano-de ices (Chua, 2011), wi h one scaling ex eme being he a omic swi ch (Te abe e al., 2005), a e classi ied as being mem is o s, and show a ibu es ha esemble biological synapses (Ohno, 2011)p o- iding exci ing p ospec s o demons a ing neu omo phic applica ions (A izienis e al., 2012). Hys e esis is ypically no iced in sys- ems/de ices ha possess ce ain ine ia, causing he alue o a physical p ope y o lag behind changes in he mechanism causing i ; mani- es ing memo y (Pe shin and Di Ven a, 2011). Pa icula ly in he case o nanoscale mem is o s, his ine ia has been asc ibed o Joule hea - ing (Fu sina e al., 2009), he elec ochemical mig a ion o oxygen ions (Nian e al., 2007) and acancies (Yang e al., 2008), he low- e ing o Scho ky ba ie heigh s by apped cha ge ca ie s a in e acial s a es (Hu e al., 2010), he phase-change (Wu ig and Yamada, 2007), he o ma ion/ up u e o conduc i e il- amen s (Kwon e al., 2010), Yang e al. (2012) in a de ice’s co e, o e en o some ex en a combina ion o he a o emen ioned swi ching mechanisms. Clea ly, he impac o mem is o s is o e- seen o be ealized h ough hei nanome ic dimensions (see Figu e 2B which is a c oss sec- ion o one o he s uc u es in Figu e 2A), hei capaci y o s o e mul iple bi s o a con- inuum o in o ma ion pe elemen (Figu e 2E) and he minuscule ene gy equi ed o w i e dis inc s a es, esul ing in high spa ial- and high s o age-densi y well beyond he cu - en s a e-o - he-a (Go o eanu e al., 2011). None heless, he ac ha he unc ional p op- e ies o such elemen s a e associa ed wi h a e-limi ing ( equency-dependen ) elec o- o he mo-dynamic changes ha a e con ingen on bo h he p esen as well as he pas en i onmen , p esen s us wi h oppo uni ies in exploi ing hem as no el compu a ion elemen s. By de ini ion, mem is o s can be ei he ol age/ lux d i en o cu en /cha ge d i en. Depending on he pola i y o he se and ese po en ials equi ed o change esis i e s a es (RS), he de ices can be classi ied as unipola (URS) o bipola (BRS) (Schindle e al., 2007) and consequen ly, hei ci cui symbol mus indica e somehow hei pola i y, as depic ed in Figu e 3A. Vol age/ lux d i en mem is o s can be desc ibed by (Chua and Kang, 1976) iMR =G(w, MR, ) MR (1) ˙w= (w, MR, )(2) while cu en /cha ge d i en mem is o s would be desc ibed as (Chua and Kang, 1976) MR =R(w,iMR, )iMR (3) ˙w= (w,iMR, )(4) F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |4 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s FIGURE 2 | Solid-s a e TiO2-based mem is o s ab ica ed a Impe ial College London. (A) Mic opho og aph o a mem is o c oss-ba a ay, wi h a close-up SEM illus a ion o a single cell appea ing in he inse o (A).(B) CHEMI-STEM map o a lamella c oss-sec ion o one o he de ices shown in (A): blue deno es P ( op and bo om elec odes) while g een and ed co espond o Ti and O2species (P od omakis e al., 2012a). (C) Simula ed and measu ed pinched hys e esis I-V cha ac e is ics (absolu e mem is o cu en |I| s. signed mem is o ol age V) (P od omakis e al., 2011) in log scale o (D) linea scale, and (E) mul i-s a e p og amming o a TiO2-based mem is o : ead pulses a e posi i e and small ampli ude (1V) ha do no al e he esis ance (mem is ance) o he mem is o , while successi e se pulses ha e nega i e high ampli ude (3V) and do p og essi ely al e he esis ance (mem is ance) o he mem is o . He e w ep esen s some s uc u al p ope y pa ame e o he mem is o . Fo example, in he 2008 HP pape (S uko e al., 2008) he ope a ion o he epo ed mem is o was pos- ula ed as desc ibed by he mo ing wall model depic ed in Figu e 3B. In his simpli ied model a mem is o o heigh L, sandwiched be ween wo elec odes, has a low esis ance egion o heigh wand a high esis ance egion o heigh L−w. The mem is o is conside ed o be di ided in o wo egions. Bo h egions a e sepa a ed by a bounda y wall a posi ion w,whichmo esup F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |5 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s FIGURE 3 | (A) Mem is o asymme ic symbols. (B) Illus a ion o mo ing wall model desc ibing mem is o ope a ion as wo a iable esis o s in se ies. (C) Illus a ion o ilamen o ma ion/annihila ion model desc ibing mem is o ope a ion as wo a iable esis ances in pa allel. (D) Expe imen ally measu ed STDP unc ion ξ(T)on biological synapses (da a om Bi and Poo, 1998, 2001). (E) Ideal STDP upda e unc ion used in compu a ional models o STDP synap ic lea ning. (F) An i-STDP lea ning unc ion o inhibi o y STDP synapses. (G) Shape o mem is o weigh upda e unc ion ( MR),(H) spike-shape wa e o m. and down wi h he amoun o cha ge ha has lown h ough he mem is o (in he case o being cu en /cha ge d i en) o he accumu- la ed lux (in case o being ol age/ lux d i en). The mem is o would beha e as wo a iable esis o s in se ies. The o al e ec i e esis ance o he mem is o would be desc ibed by R=RON w L+ROFF 1−w L(5) This mo ing wall model can app oxima e phe- nomena like mig a ion o oxygen ions (Nian e al., 2007) and acancies (Yang e al., 2008), he lowe ing o Scho ky ba ie heigh s by apped cha ge ca ie s a in e acial s a es (Hu e al., 2010), and he phase-change in some PCM (phase change ma e ials) de ices (Wu ig and Yamada, 2007). Howe e , esis i e swi ching e ec s in dielec ic-based de ices ha e no mally been assumed o be caused by conduc ing ilamen o ma ion ac oss he elec odes, al hough he unde s anding and modeling o hese phenomena emains con o e sial. As a ma - e o ac , some esea che s a e obse ing he o ma ion and annihila ion o nanoscale wid h conduc ing ilamen s in mem is o s (Kwon e al., 2010; Yang e al., 2012). P ecise modeling o his phenomenon is s ill unde F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |6 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s esea ch (Shihong e al., 2012). Howe e , le us he e p opose he ollowing e y simpli ied iew o app oxima e his physical mechanism. Figu e 3C illus a es schema ically a mem is o wi h se e al conduc ing ilamen s be ween he wo elec odes. The numbe o ilamen s o hei c oss-sec ional a ea would inc ease o dec ease wi h mem is o ope a ion. Le us call now w he o al c oss sec ional a ea o he e ec i e conduc ing ilamen s a a gi en ins an in ime, and S he o al c oss sec ion a ea o he mem is o . The ilamen s p esen high conduc i i y (low esis i i y), while he bulk p esen s much lowe conduc i i y (high esis i i y). All o med pa allel ilamen s beha e as one e ec i e esis ance o low esis ance, while he es o hebulkbeha esasano he highe esis i i y esis o . The e o e, now he mem is o beha es as wo a iable esis o s in pa allel. Consequen ly, i s o al conduc ance (in e se o esis ance) could be desc ibed as G=GON w S+GOFF 1−w S(6) whe e GON is he conduc ance pe e ec i e c oss sec ion a ea o he ilamen s, and GOFF is he conduc ance pe e ec i e c oss sec ion a ea o he ilamen -less bulk ma e ial. Pa ame e w would change om 0 o wmax, he maximum possible e ec i e c oss sec ion a ea o o al con- duc ing ilamen s(wmax ≤S). This changing c oss sec ion desc ip ion no only app oxima es ilamen o ma- ion/annihila ion phenomena, bu also some o he g adual c oss sec ion a ea a ia ions obse ed in some phase-change o e oelec ic- domains-based ma e ials (Chan hbouala e al., 2012). As we will highligh la e in sec ions 4and A i icial lea ning synapses A i icially manu ac u ed de ice ha beha es simila o a biological synapse, e.g., i ’s communica ion s eng h (o synap ic weigh ) changes as he de ice is used acco ding o some lea ning ule. Nanoscale a i icial synapse This is an a i icial synapse made using some de ice whose dimensions a e below he mic on (10−6m). Tunable STDP STDP lea ning ule whose ma hema ical desc ip ion can be made o change in ime. 5, whe he a mem is o is be e desc ibed by he mo ing wall model o he ilamen o - ma ion/annihila ion model, impac s se e ely on he esul ing ype o STDP lea ning mecha- nism. The la e yields an addi i e ype o STDP, while he o me esul s in a quad a ic ype STDP. No e ha a mem is o can be ei he ol age/ lux o cu en /cha ge d i en, indepen- den ly o whe he i is a “wall” o a “ ilamen ” mem is o . 3. SPIKE-TIMING-DEPENDENT-PLASTICITY STDP is he abili y o na u al o a i icial synapses o change hei s eng h acco ding o he p ecise iming o indi idual p e- and/o pos -synap ic spikes (Ge s ne e al., 1993, 1996; Ma k am e al., 1997; Bi and Poo, 1998, 2001; Zhang e al., 1998; Feldman, 2000; Mu and Poo, 2006; Cassenae and Lau en , 2007; Jacob e al., 2007; Young, 2007; Finelli e al., 2008; Masquelie e al., 2008, 2009). A nice o e iew o STDP and i s his o y can be ound else- whe e (Sjös öm and Ge s ne , 2010). STDP lea ning in biology is inhe en ly asynch onous and on-line, meaning ha synap ic inc emen- al upda e occu s while neu ons and synapses ansmi spikes and pe o m compu a ions. This con as s o mo e adi ional lea ning ules, like backp opaga ion (Rojas, 1996), whe e i s neu ons and synapses pe o m signal agg ega- ion and neu al s a e upda e (we call his he e “pe o ming phase”) and hen synap ic upda es a e compu ed and applied (we call his he e “weigh upda e phase”) al e na ing hese wo phases du ing aining. E en ea ly p oposals o mem is o -based STDP lea ning implemen a- ions used a i icial ime-mul iplexing o al e - na e con inuously and synch onously be ween “pe o ming” and “weigh upda e” phases (Snide , 2008), hus equi ing global sys em- wide synch oniza ion. This can become a se e e handicap when scaling up sys ems o a bi a y size. He e we show a ully asynch onous imple- men a ion o mem is o -based STDP whe e “pe o ming” and “weigh upda e” phases hap- pen simul aneously in a na u al manne , as in biology (Lina es-Ba anco and Se ano- Go a edona, 2009b,a; Zama eño-Ramos e al., 2011), whe e he e is no need o any global syn- ch oniza ion. O he esea che s ha e p oposed a ia ions a ound hese ideas (Bichle e al., 2012a; Kuzum e al., 2012). Figu e 3D shows he change o synap ic s eng h (in pe cen ) measu ed expe imen ally om biological synapses as unc ion o ela- i e iming T= pos − p e be ween he a i al ime p e o a p e-synap ic spike and he ime pos o gene a ion o a pos -synap ic spike. Al hough he da a shows s ochas ici y, we can in e an unde lying in e pola ed unc ion ξ(T)as shown in Figu e 3E. ξ(T)=a+e−T/τ+i T>0 −a−eT/τ−i T<0(7) Fo a causal p e o pos spike iming ela- ion (T>0) he s eng h o he synapse is inc eased, while o an an i-causal ela ion (T<0) i is dec eased. In he case o synapses wi h nega i e synap ic s eng h (as in some a i- icial ealiza ions), he e e sed e sion shown in Figu e 3F can be used. Mic ochip CMOS ci cui implemen a ions o STDP ules ha ollow he desc ip ion o Equa ion (7) ha e been epo ed (Indi e i e al., 2006), which esul in abou 30 F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |7 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s ansis o s pe plas ic synapse, hus demons a - ing he e y high cos o hei ha dwa e ealiza- ion. Le us call his double-spike STDP,since he weigh will be upda ed a e he a i al o he second spike (ei he p e- o pos -synap ic). Al e na i e a ia ions o STDP ha e been p oposed ha do no equi e he in e en ion o bo h p e- and pos -synap ic spikes (B ade e al., 2007), esul ing in sligh ly less complex ci cui implemen a ions (Mi a e al., 2009). Le us call his single-spike STDP,since heweigh willbe upda ed a e he a i al o p e-synap ic spikes only.Thissingle-spikeSTDP uleupda es he synap ic weigh depending on he alue o wo local neu al soma s a e a iables. The i s one is he memb ane ol age V( )and he second one is an auxilia y s a e a iable C( )p opo ional o he neu on’s i ing a e and equi alen o he biological neu on’s Calcium concen a ion, which has he ollowing dynamics ˙ C=− C( ) τC +JC i δ( − i)(8) whe e JC ep esen s he con ibu ion o one single pos -synap ic spike and he ime con- s an τCis compa able o he STDP lea ning window T. The synap ic weigh a iable ξis upda ed only when a p e-synap ic spike occu s a ime p e. The synap ic s eng h is inc eased o dec eased by ixed size s eps |a±|depending on he ins an aneous alues o V( p e)and C( p e) wi h espec o a gi en se o global h esholds {θ ,θl up,θh up,θl down,θh down},as: ξ( p e)=⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ a+i V( p e)>θ and θl up <C( p e)<θh up −a−i V( p e)<θ and θl down <C( p e)<θh down (9) Addi ionally, in his model he synap ic s eng h d i s slowly owa d i s uppe o lowe bound depending on whe he i is abo e o below an in e media e h eshold. Bo h ypes o STDP ules, double-spike and single-spike, a e e y expensi e o implemen in con en ional CMOS mic ochips (Indi e i e al., 2006; Fie es e al., 2008; Khan e al., 2008; Mi a e al., 2009). Howe e , as we will see in he nex sec ion, bo h can be implemen ed wi h jus one mem is o pe synapse i app op ia e pe iphe al signal condi ioning neu ons a e used in hyb id CMOS/mem is o ealiza ions. Hyb id nano/CMOS neu al sys em A i icial neu al ne wo k sys em buil using con en ional mic ochip echnology (CMOS) combined wi h p esen ly eme ging nanoscale de ices. Independen ly on whe he STDP is ei he double-spike o single-spike, i is said o be ei he addi i e, mul iplica i e o quad a ic i i addi ionally depends o no on he ac ual synap- ic s eng h. I he STDP upda e is independen o he ac ual synap ic s eng h, i is said o be addi i e. Addi i e STDP equi es he weigh al- ues o be bounded o an in e al because weigh s will s abilize a one o hei bounda y alues ( an Rossum e al., 2000; Rubin e al., 2001). I he synap ic upda e is p opo ional o ac ual synap ic s eng h, i is called mul iplica i e STDP and weigh s may s abilize o alues in e me- dia e o hei bounda ies ( an Rossum e al., 2000; Rubin e al., 2001; Gü ig e al., 2003). I he synap ic weigh upda e is p opo ional o he squa e o ac ual synap ic s eng h, we call i quad a ic STDP (Zama eño-Ramos e al., 2011). 4. MEMRISTORS AND CMOS NEURONS FOR DOUBLE-SPIKE STDP The mo e adi ional double-spike STDP lea n- ing ule [as modeled by Equa ion (7)] can, in heo y, be implemen ed by (Zama eño-Ramos e al., 2011) (a) using a pa icula ype o ol age/ lux d i en mem is o (Jo e al., 2010) whose ope a ion migh be app oxima ed by Equa ion (2)wi h(seeFigu e 3G) ( MR)=⎧ ⎨ ⎩ Iosign( MR)e| MR|/ o−e h/ o i | MR|> h 0o he wise (10) and bounded synap ic s eng h w∈ [wmin,wmax], while (b) p o iding app o- p ia ely shaped p e- and pos -synap ic spikes a ailable a bo h synapse (mem is o ) elec- odes (Zama eño-Ramos e al., 2011). Fo example, conside a pai o iden ical p e- and pos -synap ic spikes wi h a shape esembling ha o biological spikes, wi h an on-se du a ion | + ail|and a ail o du a ion | − ail|,asshownin Figu e 3H, spk( )=⎧ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎩ A+ mp e /τ+−e− + ail/τ+ 1−e− + ail/τ+i − + ail < <0 −A− mp e− /τ−−e− − ail/τ− 1−e− − ail/τ−i 0 < < − ail 0o he wise (11) Unde hese ci cums ances, mem is o ol age is MR( , )=αposspk( )−αp espk( + )and om Equa ions (2,10) synap ic s eng h upda e can be compu ed as w(T)= ( MR( ,T))d =ξ(T) (12) F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |8 Se ano-Go a edona e al. STDP and STDP a ia ions wi h mem is o s which has been shown o esul in he same shape illus a ed in Figu e 3E (Zama eño- Ramos e al., 2011). Fu he mo e, by eshaping hespikewa e o monecan ine uneo com- ple ely al e he STDP lea ning unc ion ξ( ), as illus a ed in Figu e 4 (Lina es-Ba anco and Se ano-Go a edona, 2009a). This way, by building neu ons wi h a gi en deg ee o shape p og ammabili y, i is possible o change he STDP lea ning unc ion a will, depending on he applica ion, o make i e ol e in ime as lea ning p og esses. Figu e 5A shows a way o in e connec ing mem is o s and CMOS neu ons o STDP lea ning. T iangles ep esen he neu on soma, being he la side i s inpu (dend i es) and he sha p side he ou pu (axon). Da k ec angles a e mem is o s, ep esen ing each one synap ic junc ion. Each neu on con ols he ol age a i s inpu (Vpos in Figu e 5B)andou pu (Vp e in Figu e 5B) nodes. When he neu on is no spiking i o ces a cons an ol age a bo h nodes, while collec ing h ough i s inpu node he sum o inpu synap ic spike cu en s coming om he mem is o s, which con ibu e o changing he neu on in e nal s a e. When he neu on spikes, i se s a one-spike wa e o m a bo h inpu and ou pu nodes. This way, hey send hei ou pu spikes o wa d as p e-synap ic spikes o he des ina ion synap ic mem is- o s, bu also backwa d o p eceding synap ic mem is o s as pos -synap ic spikes. Zama eño e al. showed ex ensi e simula ions on hese concep s, and how one can change om STDP o an i-STDP by swi ching pola i ies o spikes o mem is o s (Zama eño-Ramos e al., 2011). FIGURE 4 | Illus a ion o in luence o ac ion po en ial shapes on he esul ing STDP mem is o weigh upda e unc ion ξ(T). Mem is o uppe and lowe h esholds a e no malized o ampli udes ±1.0. F om (A1,A2) o (E1,E2) he same spike wa e o m a els o wa d and backwa d. In (F1,F2) he o wa d and backwa d wa e o ms a e he same bu ha e opposi e pola i y. In (G1,G2) o (H1,H2) he o wa d and backwa d wa e o ms a e di e en . In (G1,G2), he posi i e pulse o he backwa d wa e o m exceeds ampli ude +1.0, hus p oducing nega i e STDP upda e whene e he e is a pos -synap ic spike alone (G2); o he wise i p e- and pos -synap ic spikes happen wi hin a gi en ime window, he e will be posi i e STDP upda e. F on ie s in Neu oscience www. on ie sin.o g Feb ua y 2013 | Volume 7 | A icle 2 |9