Rockpool Documentation

Rockpool Release 3.0.3 Dylan Muir , Felix Bauer , Philipp W eidel Dec 17, 2025 CONTENTS 1 A bout R oc kpool 3 2 Installing Roc kpool and contributing 7 3 A simple intr oduction to spiking neurons 11 4 Introduction to R oc kpool 17 5 Hello MNIST with R oc kpool 29 6 W orking with time series data 43 7 Standard modules 57 8 The sharp points of R oc kpool 59 9 Lo w -le v el Module API 67 10 High-le v el TimedModule API 75 11 [] Lo w -le v el functional API 89 12 T raining a Roc kpool netw ork with Jax 93 13 A dvanced Jax training topics 103 14 Building R oc kpool modules with T orc h 107 15 T raining a Roc kpool netw ork with T orc h 117 16 Ho w T o: Configure and perf orm constrained optimization in R oc kpool 123 17 Gradient descent training of a rate-based recurr ent netw or k 133 18 T raining a spiking netw ork with Jax 143 19 T raining a spiking netw ork with T orc h 159 20 Easter with R oc kpool 165 21 A dv ersarial training 179 22 T raining an audio classification t ask using T orch 191 i 23 W a v eSense: T raining a Spiking N eural Ne tw ork with T emporal Con v olutions 201 24 The SynNe t ar chitectur e 215 25 Ov ervie w of the X ylo ™ f amil y 223 26 Introduction to X y lo ™ A udio 229 27 Quic k -start with X ylo ™ SNN cor e 247 28 T raining a spiking netw ork to deplo y to the X ylo ™ digital SNN 259 29 Using the analog frontend model 277 30 Quic k start with X ylo ™ A udio 3 285 31 Operation types of X yloA udio 3 287 32 Using AFESim as an audio transf orm 289 33 Using X yloSamna and X yloMonitor to deplo y a model on X yloA udio 3 HDK 293 34 Bandpass Filtering In A udio Front End (AFE) Of X ylo ™ A udio 3 301 35 Introduction to X y lo ™ IMU 307 36 The X ylo ™ IMU prepr ocessing inter face 329 37 Specifying IMU prepr ocessing parameters 335 38 Ov ervie w of Dynap-SE2 347 39 Quic k Start with Dynap-SE2 353 40 DynapSim Neur on Model 371 41 T raining a spiking netw ork to deplo y to Dynap-SE2 389 42 Computational graphs in R ockpool 403 43 Graph mapping 411 44 Import / e xport betw een toolchains with NIR 417 45 R oc kpool P arameter handling 421 46 T ypehints in R oc kpool 423 47 P er f ormance benchmarks f or LIF la y ers in R oc kpool 425 48 Full API summary f or Roc kpool 431 49 Chang e log 1647 50 UML diagrams f or Roc kpool 1663 51 Using the bac k end management sy stem 1671 52 Dynap-SE2 de v eloper notes 1675 ii 53 No tes f or de v elopers 1691 Python Module Inde x 1695 iii iv Rockpool, Release 3.0.3 R ockpool is a Python pac kage f or w orking with dynamical neural netw ork architectures, par ticular l y f or designing e v ent-dr iv en netw orks f or Neuromorphic computing hardware. R ockpool pro vides a con v enient interface f or designing, training and e v aluating recur rent netw orks, which can operate both with continuous-time dynamics and e v ent-dr iv en dynamics. R ockpool is an open-source project manag ed b y SynSense. CONTENTS 1 Rockpool, Release 3.0.3 2 CONTENTS CHAPTER ONE ABOUT R OCKPOOL R ockpool is an open source project released b y SynSense. 1.1 Citing Rockpool If y ou use R ockpool in y our academic w ork, w e w ould appreciate a citation! Y ou can cite us as DR Muir , F Bauer & P W eidel. (2019). R oc kpool Documentaton. Zenodo. https:// doi.org/ 10.5281/ zenodo.3773845 Or using the bibte x block: @misc {muir_dylan_2019_4639684, author = {Muir, Dylan R . and Bauer, Felix and Weidel, Philipp}, title = {Rockpool Documentaton}, month = sep, year = 2019 , publisher = {Zenodo}, doi = { 10.5281 / zenodo .3773845 }, url = {https: // doi . org / 10.5281 / zenodo .3773845 } } 3 Rockpool, Release 3.0.3 10 Chapter 2. Installing Rockpool and contributing CHAPTER THREE A SIMPLE INTRODUCTION T O SPIKIN G NEURONS Spiking neural netw orks are considered to be the thir d g eneration of neural netw orks, preceeded b y McCulloch-Pitts threshold neurons (“first g eneration”) which produced digital outputs and Artificial Neural N etw orks with continuous activ ations, lik e sigmoids and h yperbolic tang ets, (“second g eneration”) that are commonl y used these day s. 3.1 Ar tificial N euron (AN) Model The standard artificial neuron model used most commonl y in ANNs and DNNs is a simple equation  𝑦 = Θ( 𝑊 . 𝑥 + 𝑏 ) where Θ is typicall y a non linear activ ation function such as a sigmoid or hyperbolic tang ent function. [1]: from IPython.display import Image Image(filename = "AN-neuron.png" , width = 300 ) [1]: No te that the output depends onl y on the instant aneous in puts. The neuron does no t ha v e an internal st ate that w ould affect its output. This is where spiking neurons and spiking neural netw orks (SNNs) come into pla y . The y add an additional dependence on the cur rent state of the neuron, and include e xplicit temporal dynamics. The y also communicate with each other using pulsed e v ents called “spik es”. 3.2 Leaky Integrate and Fire (LIF) model [2]: Image(filename = "LIF-neuron-full.png" , width = 500 ) 11 Rockpool, Release 3.0.3 [2]: Diagram of a simple spiking neuron. In puts and outputs are ser ies of e v ent pulses called “spik es”. The neuron imple- ments temporal dynamics when integrating the input spik es, and when updating its internal state. 3.2.1 Internal state — “membrane potential” One of the simplest models of spiking neurons is the Leaky Integrate and Fire (LIF) neuron model, where the state of the neuron, kno wn as the membr ane pot ential 𝑉 𝑚 ( 𝑡 ) , depends on its previous s tate in addition to its in puts. The dynamics of this state can be described as f ollo ws: 𝜏 𝑚 · 𝑑𝑉 𝑚 /𝑑𝑡 = − 𝑉 𝑚 ( 𝑡 ) + 𝐼 𝑠 ( 𝑡 ) + 𝐼 𝑏 where 𝜏 𝑚 is the membrane time cons tant, which determines ho w muc h the neuron depends on its pre vious s tates and inputs and theref ore defines the neuron ’ s memory; and 𝐼 𝑏 is a cons tant bias. The linear equation abo v e descr ibes the dynamics of a sys tem with “leaky” dynamics ie o v er time, the membrane potential 𝑉 slo w ly “leaks” to 0 . This is wh y the neuron model is ref er red to as a Leaky Integrate and Fire N euron model . N ote that there is an e xplicit notion of time, which does not occur in the s tandard ar tificial neuron model abo v e. 3.2.2 Synaptic inputs Spiking input is integrated onto the membrane of the neuron via a synaptic cur rent 𝐼 𝑠 ( 𝑡 ) , which has its o wn dynamics: 𝜏 𝑠 · 𝑑𝐼 𝑠 /𝑑𝑡 + 𝐼 𝑠 ( 𝑡 ) = ∑︀ 𝑖 𝑤 𝑖 · ∑︀ 𝑗 𝛿 ( 𝑡 − 𝑡 𝑖 𝑗 ) where ∑︀ 𝑗 𝛿 ( 𝑡 − 𝑡 𝑖 𝑗 ) is a stream of spik es from presynaptic neuron 𝑖 occur r ing at times 𝑡 𝑖 𝑗 . The v alues 𝑤 𝑖 represent the cor responding w eights. The synaptic cur rents deca y to w ards zero under the synaptic time constant 𝜏 𝑠 . Let ’ s simulate the synapse and membrane dynamics with a random spike train in put, and see ho w the y e v ol v e. F irs t w e will g enerate a random (“P oisson”) spike train, b y thresholding random noise. [3]: # - Import some useful packages import numpy as np import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt (continues on ne xt page) 12 Chapter 3. A simple introduction to spiking neurons Rockpool, Release 3.0.3 (continued from pre vious pag e) % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 [4]: # - Generate a random spike train T = 100 spike_prob = 0.1 spikes = ( np . random . rand( T, ) < spike_prob ) # - Plot the spike train plt . figure() plt . stem(spikes, use_line_collection = True ) plt . xlabel( "Time (ms)" ) plt . title( "Random spikes" ); Belo w w e implement the synaptic and membrane dynamics b y iterating o v er time, and computing the synapse and membrane v alues in the simples t possible w a y . [5]: # - A simple Euler solver for the synapse and membrane. One time step is 1 ms tau_m = 10 # ms tau_s = 10 # ms V_m = 0 # Initial membrane potential I_s = 0 # Initial synaptic current I_bias = 0 # Bias current V_m_t = [] I_s_t = [] # - Loop over time and solve the synapse and membrane dynamics for t in range (T): (continues on ne xt page) 3.2. Leaky Integrate and Fire (LIF) model 13 Rockpool, Release 3.0.3 (continued from pre vious pag e) # Take account of input spikes I_s = I_s + spikes[t] # Integrate synapse dynamics dI_s = - I_s / tau_s I_s = I_s + dI_s # Integrate membrane dynamics dV_m = ( - V_m + I_s + I_bias) / tau_m V_m = V_m + dV_m # Save data V_m_t . append(V_m) I_s_t . append(I_s) [6]: plt . figure() spike_times = np . argwhere(spikes) plt . plot(spike_times, np . zeros( len (spike_times)), "." ) plt . plot(I_s_t) plt . xlabel( "Time (ms)" ) plt . ylabel( "$I_s$" ) plt . title( "Synaptic current $I_s$" ) plt . legend([ "Input spikes" , "Synaptic current" ]) plt . figure() plt . plot(spike_times, np . zeros( len (spike_times)), "." ) plt . plot(V_m_t, c = "purple" ) plt . xlabel( "Time (ms)" ) plt . ylabel( "$v_m$" ) plt . legend([ "Input spikes" , "Membrane potential" ]) plt . title( "Membrane potential $V_m$" ); 14 Chapter 3. A simple introduction to spiking neurons Rockpool, Release 3.0.3 3.2.3 Spiking neuron output The output of the LIF neuron is binar y and instantaneous. It is 1 only when 𝑉 reac hes a threshold value 𝑉 𝑡ℎ = 1 , upon which the membrane potential is immediatel y reset b y subtracting one. 𝑉 𝑚 ( 𝑡 + 𝛿 ) = 𝑉 𝑚 ( 𝑡 ) − 𝑉 𝑡ℎ , if 𝑉 𝑚 ( 𝑡 ) ≥ 𝑉 𝑡ℎ 𝑉 𝑚 ( 𝑡 + 𝛿 ) = 𝑉 𝑚 ( 𝑡 ) , other wise This instantanious output of 1 is typicall y ref er red to as a “spike” at time 𝑡 . A series of such spik es will hence f or th be ref er red to as 𝑠 ( 𝑡 ) . 𝑠 ( 𝑡 )=1 , if 𝑉 𝑚 ( 𝑡 ) ≥ 𝑉 𝑡ℎ 𝑠 ( 𝑡 )=0 , other wise W e can include spik e g eneration and reset into the LIF simulation: [7]: # - A simple Euler solver for the synapse and membrane, including spike generation. One ␣ ˓ → time step is 1 ms tau_m = 10 # ms tau_s = 10 # ms V_m = 0 # Initial membrane potential V_th = 1 # Spiking threshold potential I_s = 0 # Initial synaptic current I_bias = 0 # Bias current V_m_t = [] I_s_t = [] spikes_t = [] # - Loop over time and solve the synapse and membrane dynamics for t in range (T): # Take account of input spikes I_s = I_s + spikes[t] # Integrate synapse dynamics dI_s = - I_s / tau_s (continues on ne xt page) 3.2. Leaky Integrate and Fire (LIF) model 15 Rockpool, Release 3.0.3 (continued from pre vious pag e) I_s = I_s + dI_s # Integrate membrane dynamics dV_m = ( 1 / tau_m) * ( - V_m + I_s + I_bias) V_m = V_m + dV_m # - Spike detection if V_m > V_th: # - Reset V_m = V_m - 1 # - Save the spike time spikes_t . append(t) # Save data V_m_t . append(V_m) I_s_t . append(I_s) [8]: plt . figure() plt . plot(spikes_t, np . zeros( len (spikes_t)), "." ) plt . plot(V_m_t, c = "purple" ) plt . plot([ 0 , T], [V_th, V_th], "k:" ) plt . xlabel( "Time (ms)" ) plt . ylabel( "$v_m$" ) plt . legend([ "Output spikes" , "Membrane potential" , "Spiking threshold" ]) plt . title( "Membrane potential $V_m(t)$" ); The code abo v e solv es the dynamic eq uations of the neuron and synaps using a F orw ar d Euler sol v er . This is a v er y simple fix ed time-step sol v er , which uses the s tate at 𝑡 to to produce the state at 𝑡 + 𝑑𝑡 . The dynamic equations are con v er ted into temporal difference equations to sol v e f or ˙ 𝐼 𝑠 ( 𝑡 ) and ˙ 𝑉 𝑚 ( 𝑡 ) , then uses these to produce 𝐼 𝑠 ( 𝑡 + 𝑑𝑡 ) and 𝑉 𝑚 ( 𝑡 + 𝑑𝑡 ) . F or w ard Euler sol v ers are conv enient f or their simplicity , but the y ha v e dra w-bac ks in ter ms of numer ical accuracy . In par ticular , the time constants 𝜏 𝑠 and 𝜏 𝑚 mus t be at leas t 10 time long er than 𝑑𝑡 to produce an accurate simulation of the dynamics. 16 Chapter 3. A simple introduction to spiking neurons CHAPTER F OUR INTRODUCTION T O ROCKPOOL R ockpool is designed to let y ou design, simulate, train and test dynamical neural netw orks – in contrast to s tandard ANNs, these netw orks include e xplicit temporal dynamics and simulation of time. R oc kpool contains se v eral types of neuron simulations, including continuous-time (“rate”) models as w ell as e v ent-dr iv en spiking models. R ockpool suppor ts se v eral simulation back -ends, and la y ers with v ar ying dynamics can be combined in the same netw ork. 4.1 No use cr ying o ver spilt time R ockpool isn ’ t y our Grandma ’ s deep lear ning librar y . R ockpool is, at its hear t, a simulator of dynamical sy s tems. This is a pr imal difference to most ANN / deep learning librar ies, which treat neurons as little functional units of s tateless linear alg ebra that y ou push data through on an implicit global cloc k. Each neuron in R oc kpool is a dynamical object that e xists and e v ol v es o v er time. Y ou can influence its state b y pro viding input, but its s tate can e v ol v e e v en in the absence of input. This introduces some subtle differences in ho w in puts, states and outputs are treated in R ockpool. The aim of R ockpool is to mak e this as transparent as possible, and assist y ou in building signal processing and com- puting ar tef acts in an intuitiv e w a y . The approach is to engineer sy stems to perf or m desired tasks, with the resulting sy s tems e xisting as dynamical units that interact with the w or ld in real-time. This approach lets y ou design solutions f or real-time computing hardw are such as spiking neural netw ork inf erence engines. 4.2 Impor ting Rockpool modules [1]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Import classes to represent time series data from rockpool import TimeSeries, TSContinuous, TSEvent # - Import ` Module ` classes to use from rockpool.nn.modules import Rate, Linear [2]: # - Import numpy import numpy as np # - Import the plotting library import sys !{ sys.executable } -m pip install --quiet matplotlib rich (continues on ne xt page) 17 Rockpool, Release 3.0.3 (continued from pre vious pag e) import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 # - Rich printing from rich import print 4.3 My First Module Neurons and their s tate in R oc kpool are encapsulated in Module s, which are small Python objects that deriv e from the Module base class. These are similar in design to Modules in other mac hine lear ning librar ies: the y manag e the parameters, state and computations associated with a group of neurons. W e will first define a small population of non-spiking rate neurons using the Rate class, b y specifing the shape of the module. These will be rate-based neurons with a linear -threshold (R eLU) transf er function. R ockpool modules almos t alwa y s e xpect shape as their first parameter . This is either an integ er (f or modules that ha v e the same number of input and output c hannels), or a tuple (N_in, N_out) , f or modules that can ha v e different numbers of input and ouput c hannels. In this case, w e pro vide a single number to specify the number of neurons in the module. [3]: # - Define a feed-forward module with ` N ` neurons N = 4 mod = Rate(N) print (mod) Rate with shape ( 4 , ) Let ’ s take a look inside and see what w e ha v e. All Modules support inspection methods state() , parameters() and simulation_parameters() , which return nested dictionaries of parameters f or the module and an y sub-modules. In this case w e only ha v e a single top-lev el module, so there will be no nes ted parameters. “P arameters” are g enerall y the set of model configuration that y ou w ould modify to train a model, and that y ou w ould need to communicate to someone to giv e them y our trained model. e.g. w eights, neuron time constants, biases. “S tate” are inter nal v ariables that are maintained dur ing and between e v olutions, that influence the dynamics of a model. If y ou thro w them a w a y , the trained model still e xists. e.g. ins tantaneous synaptic cur rents and membrane potentials. “Simulation parameters” are configuration elements that the model needs to kno w about, but which aren ’ t trained to configure y our model. e.g. time-step duration, noise s td. dev . These are fix ed dur ing training. [4]: print (mod . state()) print (mod . parameters()) print (mod . simulation_parameters()) { ' x ' : array ([ 0 ., 0 ., 0 ., 0 . ])} { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : ␣ ˓ → array ([ 0 ., 0 ., 0 ., 0 . ])} 18 Chapter 4. Introduction to Rockpool Rockpool, Release 3.0.3 { ' dt ' : 0.001 , ' noise_std ' : 0.0 , ' act_fn ' : < function H_ReLU at 0x172ca81f0 >} W e can ev ol v e the state of these neurons o v er time, b y pro viding some input to the module b y calling it. Lo w -le v el modules accept cloc k ed input data as ra w numpy ar ra y s with shape (T, N) , or with shape (batches, T, N) . If an ar ra y (T, N) is pro vided, it will be con v er ted to a single batch. Let ’ s create some simple input. [5]: T = 100 input_clocked = np . random . randn(T, N) * 3 # - Evolve the module with the input output_clocked, new_state, recorded_state = mod(input_clocked) # - Update the state of the module mod = mod . set_attributes(new_state) output_clocked represents the output of the neurons, whic h is a n numpy ar ra y with shape (1, T, N) . W e can visualise this using matplotlib . [6]: plt . plot(output_clocked[ 0 ]); 4.4 Getting things done in time R ockpool also offers a higher -lev el TimedModule interface, which accepts in put and retur ns output as TimeSeries objects. TimeSeries objects pro vide a con v enience encapsulation f or handling time-ser ies data, including both con- tinuous signals and streams of e v ents. [7]: # - Use the ` .timed() ` method to obtain a TimeSeries interface tmod = mod . timed() print (tmod) TimedModuleWrapper with shape ( 4 , 4 ) { Rate ' _module ' with shape ( 4 , ) } with Rate ' _module ' as module timed() con v er ts a lo w-le v el module into one which nativ el y understands time series data. T o test this out, w e need to g enerate a time series! 4.4. Getting things done in time 19 Rockpool, Release 3.0.3 (continued from pre vious pag e) ) self . w_output = Parameter( shape = self . shape[ 1 :], family = "weights" , init_func = lambda size: np . random . normal(size = size), ) # - Build submodules self . mod_recurrent = Rate((shape[ 1 ], shape[ 1 ]), has_rec = True ) # - Every module needs an ` evolve() ` method def evolve ( self , input , record: bool = False ) -> (np . array, np . array, dict ): # - Initialise output arguments new_state = {} record_dict = {} # - Pass input through the input weights and submodule x, mod_state, mod_record_dict = self . mod_recurrent( np . dot( input , self . w_input), record ) # - Maintain the submodule state and recorded state dictionaries new_state . update({ "mod_recurrent" : mod_state}) record_dict . update({ "mod_recurrent" : mod_record_dict}) # - Pass data through the output weights output = np . dot(x, self . w_output) # - Return outputs return output, new_state, record_dict N o w w e can ins tantiate a module based on our net class, and test it b y passing data through. [17]: # - Define the network size input_size = 4 rec_size = 3 output_size = 2 # - Instantiate a module net_mod = net(shape = (input_size, rec_size, output_size)) print (net_mod) net with shape ( 4 , 3 , 2 ) { Rate ' mod_recurrent ' with shape ( 3 , 3 ) } [18]: # - Define an input T = 100 input = np . random . randn(T, input_size) # - Evolve the module (continues on ne xt page) 26 Chapter 4. Introduction to Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) output, new_state, recorded_state = net_mod( input ) net_mod = net_mod . set_attributes(new_state) plt . plot(output[ 0 ]); If w e inspect our netw ork parameters, w e can see the y f ollo w the structure of the module heirarch y , and include param- eters defined b y the sub-modules. [19]: print (net_mod . parameters()) { ' w_input ' : array ([[ 1.41978838 , -1.13184915 , -0.74752138 ] , [ -0.95091877 , 0.30172439 , 0.35265 ] , [ -0.09663117 , 0.10782144 , -1.36449476 ] , [ -0.72311345 , 0.10093025 , -0.2414413 ]]) , ' w_output ' : array ([[ -0.20141784 , 0.62143563 ] , [ 1.22627445 , -0.14979523 ] , [ -1.53157303 , 0.67460225 ]]) , ' mod_recurrent ' : { ' w_rec ' : array ([[ 0.15648791 , 0.401766 , -1.14040351 ] , [ 0.42543312 , 0.5842613 , -0.30617398 ] , [ -0.81748013 , -1.23444556 , -0.26348427 ]]) , ' tau ' : array ([ 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 . ]) } } Y ou can set the parameters of the netw ork directl y b y setting attr ibutes: [20]: # - Define weights weights_in = np . random . rand(input_size, rec_size) - 0.5 weights_rec = np . random . randn(rec_size, rec_size) / rec_size weights_out = np . random . rand(rec_size, output_size) - 0.5 # - Set the weights net_mod . w_input = weights_in net_mod . mod_recurrent . w_rec = weights_rec net_mod . w_output = weights_out (continues on ne xt page) 4.6. Building a custom nested modules 27 Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Display the network print (net_mod . parameters()) { ' w_input ' : array ([[ 0.28506196 , 0.02794232 , 0.05052549 ] , [ -0.39742499 , 0.13996292 , -0.191019 ] , [ -0.0180503 , -0.03285261 , 0.01811005 ] , [ 0.05156022 , -0.23567097 , -0.29003133 ]]) , ' w_output ' : array ([[ -0.28148003 , -0.02212604 ] , [ -0.23931323 , 0.24881792 ] , [ -0.41053964 , -0.33932346 ]]) , ' mod_recurrent ' : { ' w_rec ' : array ([[ -0.91369668 , 0.15297224 , 0.4534814 ] , [ 0.05805358 , 0.35401688 , -0.29566064 ] , [ 0.2393718 , -0.02951023 , -0.00684574 ]]) , ' tau ' : array ([ 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 . ]) } } 4.7 N e xt steps R ockpool pro vides se v eral additional niceties to mak e de v eloping y our o wn Module s easier . See the documentation f or the Module class f or more inf or mation. Y ou can also lear n about R ockpool Parameter s, and about the facilities pro vided b y the rockpool.utilities pac kag e. See a simple e xample f or training SNNs: Hello MNIST with R oc kpool Lear n about the lo w -le v el API in detail: Low-lev el Module API Lear n about the high-le v el API in detail: High-lev el TimedModule API Lear n about the functional API in detail: [] Low-lev el functional API Lear n about training netw orks with Jax: T raining a R oc kpool netw ork wit h Jax Lean about the T orch API: Building Roc kpool modules with T or c h Lear n about training netw orks with T orch: T r aining a Roc kpool netw ork wit h T orc h Lear n about training netw orks f or deplo yment to hardware: Ov er view of the X ylo ™ f amily 28 Chapter 4. Introduction to Rockpool CHAPTER FIVE HELL O MNIST WITH ROCKPOOL R ockpool is a neural netw ork training library , with a f ocus on spiking neural netw orks (SNNs) and other neurons that ha v e inter nal s tate and temporal dynamics. The goal is to mak e training and deplo ying SNNs as simple as training a standard DNN . Here w e sho w the steps to train the MNIS T digit classification task — a common dataset f or g etting s tar ted with neural netw ork librar ies. [9]: # - Install required packages import sys !{ sys.executable } -m pip install --quiet rockpool tonic tqdm torch torchvision matplotlib Note: you may need to restart the kernel to use updated packages. [10]: # - Basic imports import torch import torchvision import numpy as np import matplotlib.pyplot as plt plt . rcParams[ ' figure.dpi ' ] = 300 plt . rcParams[ ' figure.figsize ' ] = [ 9.6 , 3.6 ] plt . rcParams[ ' font.size ' ] = 12 from tqdm .autonotebook import tqdm, trange from IPython.display import Image 5.1 Spiking neurons The main difference betw een a spiking neuron (SN) and a standard ar tificial neuron (AN) is in their understanding of time. S tandard ANs operate instantaneousl y , b y simply summing their w eighted inputs and appl ying a transf er function such as a R eL U . Spiking neurons, on the other hand, ha v e an inter nal state that e v olv es o v er time in response to their input. As a result, SNNs are great f or processing temporal signals — and as a consequence, w e need to consider ho w to pro vide input data to an SNN o v er time, and consider e.g. the time constants 𝜏 as additional netw ork parameters. In R ockpool, w e ha v e a simple Leaky Integ rate-and-Fire (LIF) spiking neuron, defined by the module LIF . The cell belo w sho ws y ou ho w to build a single spiking neuron, pro vide in put e v ents, simulate the neuron and e xamine its inter nal state. Y ou can read more in depth about spiking neurons in A simple intr oduction to spiking neur ons . 29 Rockpool, Release 3.0.3 [11]: # - Import the SNN modules from rockpool from rockpool.nn.modules import LIF # - Generate a spiking neuron neuron = LIF( 1 ) # - Simulate this neuron for 1 sec with poisson spiking input z(t) num_timesteps = int ( 1 / neuron . dt) input_z = 0.06 * (np . random . rand(num_timesteps) < 0.0125 ) output, _, rec_dict = neuron(input_z, record = True ) # - Display the input, internal state and output events plt . figure() plt . plot(rec_dict[ ' isyn ' ] . squeeze(), label = ' $I_s(t)$ ' ) plt . plot(rec_dict[ ' vmem ' ] . squeeze(), label = ' $V_m(t)$ ' ) b ,t ,n = np . nonzero(output) plt . scatter(t, n, marker = ' | ' ,c = ' k ' , label = ' $o(t)$ ' ) plt . plot([ 0 , num_timesteps], [neuron . threshold] * 2 , ' k: ' , label = ' $\Theta$ ' ) plt . xlabel( ' Time (dt) ' ) plt . ylabel( ' $V_m$, $I_s$ (a.u.) ' ) plt . legend(); 5.2 Data and encoding N o w w e can load the MNIS T dataset, and decide ho w to encode the data f or processing b y an SNN. W e make use of the torchvision pac kag e to obtain the MNIS T dataset and manag e the dataset and data loader classes. The data will be accessed as torch.Tensor objects. [46]: # - Number of samples per batch batch_size = 256 # - Download and access the MNIST training dataset train_data = torchvision . datasets . MNIST( (continues on ne xt page) 30 Chapter 5. Hello MNIST with Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) root = "." , train = True , download = True , transform = torchvision . transforms . ToTensor(), ) # - Create a data loader for the training dataset train_loader = torch . utils . data . DataLoader( train_data, batch_size = batch_size, shuffle = True ) # - Create a test dataset test_loader = torch . utils . data . DataLoader( torchvision . datasets . MNIST( root = "." , train = False , transform = torchvision . transforms . ToTensor(), ), batch_size = batch_size, ) N o w w e need to decide ho w to encode the data samples f or the SNN. The MNIS T samples are 28 × 28 resolution imag es, with no temporal component. W e will encode the images b y ar ranging the 28 × 28 pix els into a 784-element v ector , and creating 784 spiking input c hannels. W e’ll use a P oisson process to g enerate an a v erag e input rate f or each channel, according to the intensity of the corresponding pix el. Pix els with v alue 0 will ha v e g enerate no input e v ents; pix els with value 1 will g enerate the highest rate of e v ents. T o do so we need to define ho w man y time-steps eac h sample will tak e, and what the duration of a single time-step will be. [13]: # - Define the temporal aspects of a data sample num_timesteps = 100 dt = 10e-3 # - Extract the number of classes and input channels num_classes = len (torchvision . datasets . MNIST . classes) input_channels = train_data[ 0 ][ 0 ] . numel() # - Define a function to encode an input into a poisson event series def encode_poisson (data: torch . Tensor, num_timesteps: int , scale: float = 0.1 ) -> torch . ˓ → Tensor: num_batches, frame_x, frame_y = data . shape data = scale * data . view((num_batches, 1 , - 1 )) . repeat(( 1 , num_timesteps, 1 )) return (torch . rand(data . shape) < (data * scale)) . float() W e also need to deter mine what the targ et output of the netw ork will look lik e. The MNIST dataset has 10 classes. W e ’ll build a netw ork with 10 output neurons, one f or each class, and train the netw ork to produce a high ev ent rate f or the targ et class and no e v ents f or the non-targ et classes. Our targ et will also be a time series of ev ents, with the same duration as the input time series, and with 10 channels. [14]: # - Define a function to encode the network target def encode_class (class_idx: torch . Tensor, num_classes: int , num_timesteps: int ) -> torch . ˓ → Tensor: num_batches = class_idx . numel() (continues on ne xt page) 5.2. Data and encoding 31 Rockpool, Release 3.0.3 (continued from pre vious pag e) target = torch . nn . functional . one_hot(class_idx, num_classes = num_classes) return target . view((num_batches, 1 , - 1 )) . repeat(( 1 , num_timesteps, 1 )) . float() [15]: # - Get one sample frame, class_idx = train_data[ 0 ] # - Encode the input and targets data = encode_poisson(frame, num_timesteps) target = encode_class(torch . tensor(class_idx), num_classes, num_timesteps) # - Plot the poisson input for this sample plt . figure() b ,t ,n = torch . nonzero(data, as_tuple = True ) plt . scatter(t * dt, n, marker = ' | ' ) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Channel ' ) plt . title( ' Input ' ) # - Plot the target event series for this sample b ,t ,n = torch . nonzero(target, as_tuple = True ) plt . figure() plt . scatter(t * dt, n, marker = ' | ' ) plt . ylim([ - 1 , num_classes + 1 ]) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Channel ' ) plt . title( ' Target ' ); 32 Chapter 5. Hello MNIST with Rockpool Rockpool, Release 3.0.3 5.3 Building a netw ork R ockpool allo w s y ou to build SNNs with a simple syntax, similar to PyT orch. W e suppor t sev eral computational bac k -ends in R ockpool, one of whic h is PyT orch, whic h w e will use here. LIFTorch is a R ockpool and torch module which pro vides a trainable simulation of LIF spiking neurons. LinearTorch is a linear w eight matr ix, comparable to the torch nn.Linear module. T o build simple f eed-f or w ard netw orks, w e use the Sequential combinator from R ockpool, whic h functions lik e the torch nn.Sequential combinator . By def ault, R oc kpool allo w s y ou to train the time constants 𝜏 and other parameters of an SNN . For simplicity , here w e ’ll define them as constant (i.e. non-trainable), using the Constant parameter decorator . The netw ork architecture w e will use is sho wn belo w ; a simple tw o-la y er SNN . [16]: Image( ' mnist-architecture.png ' ) [16]: [17]: # - Import network packages from rockpool.nn.modules import LIFTorch, LinearTorch from rockpool.nn.combinators import Sequential from rockpool.parameters import Constant # - Define a simple network num_hidden = 64 (continues on ne xt page) 5.3. Building a network 33 Rockpool, Release 3.0.3 (continued from pre vious pag e) tau_mem = Constant( 100e-3 ) tau_syn = Constant( 50e-3 ) threshold = Constant( 1. ) bias = Constant( 0. ) # - Define a two-layer feed-forward SNN snn = Sequential( LinearTorch((input_channels, num_hidden)), LIFTorch(num_hidden, tau_syn = tau_syn, tau_mem = tau_mem, threshold = threshold, ␣ ˓ → bias = bias, dt = dt), LinearTorch((num_hidden, num_classes)), LIFTorch(num_classes, tau_syn = tau_syn, tau_mem = tau_mem, threshold = threshold, ␣ ˓ → bias = bias, dt = dt) ) print (snn) TorchSequential with shape (784, 10) { LinearTorch ' 0_LinearTorch ' with shape (784, 64) LIFTorch ' 1_LIFTorch ' with shape (64, 64) LinearTorch ' 2_LinearTorch ' with shape (64, 10) LIFTorch ' 3_LIFTorch ' with shape (10, 10) } Let ’ s e xamine the un-trained output of this netw ork. W e simulate the netw ork identicall y as with the single LIF neuron abo v e, just b y passing input data to the module. W e’ll use the single data sample w e encoded abo v e. The record = True argument tells R oc kpool to record all the inter nal state of the SNN dur ing e v olution. [18]: # - Simulate the untrained network, record internal state output, _, rec_dict = snn(data, record = True ) # - Display the internal state and output plt . plot(rec_dict[ ' 1_LIFTorch ' ][ ' vmem ' ][ 0 ] . detach()); plt . plot([ 0 , num_timesteps], [threshold] * 2 , ' k: ' ) plt . ylim([ - 2.1 , 1.1 ]) plt . xlabel( ' Time (dt) ' ) plt . ylabel( ' $V_m$ (a.u.) ' ) plt . title( ' Hidden $V_m$ ' ) plt . figure() plt . plot(rec_dict[ ' 3_LIFTorch ' ][ ' vmem ' ][ 0 ] . detach()); plt . plot([ 0 , num_timesteps], [threshold] * 2 , ' k: ' ) plt . ylim([ - 2.1 , 1.1 ]) plt . xlabel( ' Time (dt) ' ) plt . ylabel( ' $V_m$ (a.u.) ' ) plt . title( ' Output $V_m$ ' ) plt . figure() t ,n = torch . nonzero(rec_dict[ ' 3_LIFTorch_output ' ][ 0 ] . detach(), as_tuple = True ) plt . scatter(t, n, marker = ' | ' , label = ' Output events ' ) plt . plot( 0 , class_idx, ' g> ' , markersize = 20 , label = ' Target class ' ) plt . ylim([ - 1 , num_classes + 1 ]) plt . xlim([ - 1 , num_timesteps + 1 ]) (continues on ne xt page) 34 Chapter 5. Hello MNIST with Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) plt . xlabel( ' Time (dt) ' ) plt . ylabel( ' Channel ' ) plt . title( ' Output (events) ' ) plt . legend(); 5.3. Building a network 35 Rockpool, Release 3.0.3 42 Chapter 5. Hello MNIST with Rockpool CHAPTER SIX W ORKIN G WITH TIME SERIES D A T A 6.1 Concepts In R ockpool, temporal data (“time series” data) is encapsulated in a set of classes that der iv e from TimeSeries . T ime ser ies come in tw o basic fla v ours: “continuous” time ser ies, which ha v e been sampled at some set of time points but which represent v alues that can e xist at an y point in time; and “e v ent” time ser ies, which consist of discrete e v ent times. The TimeSeries subclasses pro vide methods f or e xtracting, resampling, shifting, tr imming and manipulating time ser ies data in a con v enient f ashion. Since R ockpool naturall y deals with temporal dynamics and temporal data, Time- Ser ies objects are used to pass around time ser ies data both as input and as output. TimeSeries objects ha v e an implicit shared time-base at 𝑡 0 = 0 sec. Ho w e v er , the y can easily be offset in time, concatenated, etc. Housek eeping and import statements [1]: # - Import required modules and configure # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Required imports import numpy as np from rockpool.timeseries import ( TimeSeries, TSContinuous, TSEvent, set_global_ts_plotting_backend, ) from IPython.display import Image # - Use HoloViews for plotting import sys !{ sys.executable } -m pip install --quiet colorcet holoviews import colorcet as cc import holoviews as hv hv . extension( "bokeh" ) (continues on ne xt page) 43 Rockpool, Release 3.0.3 (continued from pre vious pag e) % opts Curve [width=600] % opts Scatter [width=600] Data type cannot be displa y ed: application/ja vascript, application/vnd.holo view s_load.v0+json Data type cannot be displa y ed: application/ja vascript, application/vnd.holo view s_load.v0+json 6.2 Continuous time series represented b y TSContinuous Continuous time ser ies are represented b y tuples [ 𝑡 𝑘 , 𝑎 ( 𝑡 𝑘 )] , where 𝑎 ( 𝑡 𝑘 ) is the amplitude of a signal, sampled at the time 𝑡 𝑘 . A full time ser ies is theref ore the set of samples [ 𝑡 𝑘 , 𝑎 ( 𝑡 𝑘 )] f or 𝑘 = 1 . . . 𝐾 . Continuous time ser ies in R ockpool are represented b y the TSContinuous class. A time ser ies is constructed b y pro viding the sample times in seconds and the cor responding sample v alues. The full syntax f or constructing a TSContinuous object is giv en by def __init__ ( self , times: Optional[ArrayLike] = None , samples: Optional[ArrayLike] = None , num_channels: Optional[ int ] = None , periodic: bool = False , t_start: Optional[ float ] = None , t_stop: Optional[ float ] = None , name: str = "unnamed" , interp_kind: str = "linear" , ) [2]: # - Build a time trace vector duration = 10.0 dt = 0.01 times = np . arange( 0.0 , duration, 0.1 ) theta = times / duration * 2 * np . pi # - Create a TSContinuous object containing a sin-wave time series ts_sin = TSContinuous( times = times, samples = np . sin(theta), name = "sine wave" , ) ts_sin 44 Chapter 6. W orking with time series dat a Rockpool, Release 3.0.3 [2]: non-periodic TSContinuous object ` sine wave ` from t=0.0 to 9.9. Samples: 100. Channels: 1 TSContinuous pro vides a con v enience method from_clocked() to g enerate a TSContinuous object from a regular l y -sampled v ector of data with a fix ed time-step dt . This is the optimal w a y to w ork with cloc k ed data im- por ted from outside R ockpool, and ensures that the data will be used intuitiv el y within R ockpool. from_clocked() will retur n a TSContinuous time ser ies with e xtent from t=0 to t=N*d t , using sample- and-hold inter polation betw een samples. Each data sample is assumed to occur at the beginning of a time bin of dt duration, and the data is assumed to be v alid f or the entir ity of the dt time bin. Y ou can offset the retur ned time series accurately b y specifying the time of the initial sample with the argument t_start . [3]: # - Generate some data samples N = 100 dt = 0.01 samples = np . random . rand(N, 1 ) ts_reg = TSContinuous . from_clocked(samples, dt = dt, t_start = 5.0 ) ts_reg [3]: non-periodic TSContinuous object ` unnamed ` from t=5.0 to 6.0. Samples: 100. Channels: 1 TSContinuous objects pro vide a con v enience plotting method plot() f or visualisation. This mak es use of holo view s / bokeh or matplotlib plotting libraries, if a v ailable. If both are a vailable y ou can c hoose betw een them using the timeseries.set_global_plotting_backend func- tion. [4]: # - Set backend for Timeseries to holoviews set_global_ts_plotting_backend( "holoviews" ) # # - Alternatively, it can be set for specific Timeseries instances # ts_sin.set_plotting_backend("holoviews") # - Plot the time series ts_sin . plot() Global plotting backend has been set to holoviews. [4]: :Curve [Time] (y) TSContinuous objects can represent multiple ser ies simultaneousl y , as long as the y share a common time base: [5]: # - Create a time series containing a sin and cos trace ts_cos_sin = TSContinuous( times = times, samples = np . stack((np . sin(theta), np . cos(theta))) . T, name = "sine and cosine" , ) # - Print the representation print (ts_cos_sin) # - Plot the time series ` ts_cos_sin . plot() 6.2. Continuous time series repr esented b y TSContinuous 45 Rockpool, Release 3.0.3 non-periodic TSContinuous object ` sine and cosine ` from t=0.0 to 9.9. Samples: 100. ␣ ˓ → Channels: 2 [5]: :Overlay .Curve.I :Curve [Time] (y) .Curve.II :Curve [Time] (y) F or con v enience, TimeSeries objects can be made to be per iodic. This is particularl y useful when simulating netw orks o v er repeated tr ials. T o do so, use the periodic flag when cons tr ucting the TimeSeries object: [6]: # - Create a periodic time series object ts_sin_periodic = TSContinuous( times = times, samples = np . sin(theta), periodic = True , name = "periodic sine wave" , ) # - Print the representation print (ts_sin_periodic) # - Plot the time series plot_trace = np . arange( 0 , 100 , dt) ts_sin_periodic . plot(plot_trace) periodic TSContinuous object ` periodic sine wave ` from t=0.0 to 9.9. Samples: 100. ␣ ˓ → Channels: 1 [6]: :Curve [Time] (y) Continuous time ser ies per mit inter polation betw een sampling points, using scipy.interpolate as a bac k -end. By def ault sample-and-hold inter polation is used, but an y inter polation method suppor ted b y scipy.interpolate can be pro vided as a string when constr ucting the TSContinuous object (f or e xample, linear ). The inter polation inter f ace is simple: TSContinuous objects are callable with a list-lik e set of time points; the inter - polated v alues at those time points are returned as a numpy.ndarray . [7]: # - Interpolate the sine wave print (ts_sin([ 1 , 1.1 , 1.2 ])) [[0.58778525] [0.63742399] [0.63742399]] As a con v enience, TSContinuous objects can also be inde x ed using [] , whic h uses inter polation to build a ne w time ser ies object with the reques ted data. A second inde x can be pro vided to choose specific c hannels. Inde xing will return a ne w TSContinuous object. [8]: # - Slice a time series object ts_cos_sin[: 1 : 0.09 , 0 ] . print() non-periodic TSContinuous object ` sine and cosine ` from t=0.0 to 0.99. Samples: 12. ␣ ˓ → Channels: 1 0.0: [0.] 0.09: [0.] 0.18: [0.06279052] (continues on ne xt page) 46 Chapter 6. W orking with time series dat a Rockpool, Release 3.0.3 (continued from pre vious pag e) 0.27: [0.12533323] ... 0.72: [0.42577929] 0.8099999999999999: [0.48175367] 0.8999999999999999: [0.48175367] 0.99: [0.53582679] F or non-per iodic TSContinuous objects, the time rang e in which data can be sampled or interpolated lies between the ear liest and the lates t sample or , in other w ords, betw een the first and las t point of its times attr ibute. It is possible that the limits of the time ser ies, which are defined b y the t_start and t_stop attr ibutes, are bey ond the sampling times. In this case v alues in the cor responding inter v als are deter mined b y the fill_value attr ibute. The def ault beha vior is to e xtrapolate. [9]: # - Extrapolate between last sample and t_stop ts_cos_sin . t_stop = 12 print ( "Extrapolated fill value at t=11:" , ts_cos_sin( 11 )) # - Use constant fill value instead of extrapolation ts_cos_sin . fill_value = 42 print ( "New fill value at t=11:" , ts_cos_sin( 11 )) # - Back to extrapolation ts_cos_sin . fill_value = "extrapolate" Extrapolated fill value at t=11: [[-0.06279052 0.99802673]] New fill value at t=11: [[42. 42.]] When tr ying to sample outside of this rang e, the def ault beha vior is to raise a ValueError : [10]: # - Sampling at a point beyond the series ' time range: try : ts_cos_sin( 13 ) except ValueError as e: print ( "Caught the following exception:" ) print (e) Caught the following exception: TSContinuous ` sine and cosine ` : Some of the requested time points are beyond the first ␣ ˓ → and last time points of this series and cannot be sampled. If you think that this is due to rounding errors, try setting the ` approx_limit_times ` ␣ ˓ → attribute to ` True ` . If you want to sample at these time points anyway, you can set the ` beyond_range_ ˓ → exception ` attribute of this time series to ` False ` and will receive ` NaN ` as values. If the beyond_range_exception attr ibute is set to False , nan v alues will ins tead be retur ned and a w arning will be issued. (Y ou will not see the w arning here, because warnings are suppressed in this tutor ial) [11]: ts_cos_sin . beyond_range_exception = False print (ts_cos_sin( 13 )) [[nan nan]] Sometimes it can happen due to numer ical er rors that the pro vided sampling time is slightl y be y ond rang e, al- though the intention ma y hav e been to sample right at the beginning or end of a time ser ies. This will result in a ValueError or nan v alues being returned, depending on the value of beyond_range_exception . Ho w e v er , when 6.2. Continuous time series repr esented b y TSContinuous 47 Rockpool, Release 3.0.3 the approx_limit_times attr ibute is set to True (the def ault case), v alues that are only slightl y be y ond the defined rang e will be appro ximated b y the firs t or last time point of the series. For a time series with approx_limit_times set to True , the threshold f or this appro ximation is 1e-6 * ts.duration or 1e-9 , whiche v er is less. In either case a w arning will be raised. [12]: # - ` t ` should be 12, but is slightly larger due to numerical errors. times = np . repeat( 0.001 , 12000 ) t = np . sum(times) print ( f"We want to sample about { t - ts_cos_sin . t_stop : .1e } seconds after the series ends." ) # - Sampling at ` t ` will give same value as if sampled at t=12 print ( "Sampling in spite of rounding errors:" , ts_cos_sin(t)) print ( "Values at end of series:" , ts_cos_sin( 12 )) We want to sample about 5.3e-15 seconds after the series ends. Sampling in spite of rounding errors: [[-0.06279052 0.99802673]] Values at end of series: [[-0.06279052 0.99802673]] [13]: # - Disable time approximation in such cases ts_cos_sin . approx_limit_times = False # - Sampling at ` t ` will result in a warning and nan-values being returned. print ( "Now we get:" , ts_cos_sin(t)) Now we get: [[nan nan]] TSContinuous pro vides a larg e number of methods f or manipulating time ser ies. For e x ample, binar y operations such as addition, multiplication etc. are suppor ted betw een tw o time ser ies as w ell as betw een time ser ies and scalars. Mos t operations retur n a ne w TSContinuous object. See the api ref erence f or TSContinuous f or full detail. A ttributes ( TSContinuous ) Attribut e name Description times V ector 𝑇 of sample times samples Matr ix 𝑇 × 𝑁 of samples, cor responding to sample times in times num_channels Scalar repor ting 𝑁 : number of ser ies in this object num_traces Synon ym to num_channels t_start , t_stop Firs t and last sample times, respectiv el y duration Duration betw een t_start and t_stop plotting_backend Cur rent plotting back end f or this instance. fill_value Data to use to fill samples that f all outside t_start and t_stop 6.2.1 Examples of time series manipulation [14]: # - Perform additions, powers and subtractions of time series ts_sin . beyond_range_exception = False ( (ts_sin + 2 ) . plot() + (ts_cos_sin ** 6 ) . plot() (continues on ne xt page) 48 Chapter 6. W orking with time series dat a Rockpool, Release 3.0.3 (continued from pre vious pag e) + (ts_sin - (ts_sin ** 3 ) . delay( 2 )) . plot() ) . cols( 1 ) [14]: :Layout .Curve.Sine_wave.I :Curve [Time] (y) .Sine_and_cosine.I :Overlay .Curve.I :Curve [Time] (y) .Curve.II :Curve [Time] (y) .Curve.Sine_wave.II :Curve [Time] (y) 6.3 Ev ent-based time series represented b y TSEvent Sequences of e v ents (e.g. spike trains) are represented b y the TSEvent class, which inherits from TimeSeries . Discrete time ser ies are represented b y tuples ( 𝑡 𝑘 , 𝑐 𝑘 ) , where 𝑡 𝑘 are sample times as bef ore and 𝑐 𝑘 is a “channel” associated with each sample (e.g. the source of an e v ent). Multiple samples at identical time points are e xplictl y per mitted such that (f or e xample) multiple neurons could spik e simultaneousl y . U nless the ser ies is empty , the ar gument t_stop mus t be pro vided and has to be strictly lar ger than the time of the las t e v ent. TSEvent objects are initialised with the syntax def __init__ ( self , times: ArrayLike = None , channels: Union[ int , ArrayLike] = None , periodic: bool = False , t_start: Optional[ float ] = None , t_stop: Optional[ float ] = None , name: str = None , num_channels: int = None , ) [15]: # - Build a time trace vector times = np . sort(np . random . rand( 100 )) channels = np . random . randint( 0 , 10 , ( 100 )) ts_spikes = TSEvent( times = times, channels = channels, t_stop = 100 , ) ts_spikes [15]: non-periodic ` TSEvent ` object ` unnamed ` from t=0.001941216549281144 to 100.0. Channels: ␣ ˓ → 10. Events: 100 [16]: # - Plot the events ts_spikes . plot() 6.3. Event-based time series represented b y TSEvent 49 Rockpool, Release 3.0.3 [16]: :Scatter [Time] (Channel) If TSEvent is called, it retur ns ar ra y s of the ev ent times and channels that f all within the defined time points and cor respond to selected channels. [17]: # - Return events between t=0.5 and t=0.6 ts_spikes( 0.5 , 0.6 ) ts_spikes( 0.5 , 0.6 , channels = [ 3 , 7 , 8 ]) [17]: (array([0.51589083, 0.58430924, 0.59280105]), array([3, 8, 7])) TSEvent also suppor ts inde xing, where indices cor respond to the indices of the e v ents in the times attr ibute. A ne w TSEvent will be retur ned. F or e xample, in order to g et a new series with the first 5 e v ents of ts_spikes one can do: [18]: ts_spikes[: 5 ] [18]: non-periodic ` TSEvent ` object ` unnamed ` from t=0.001941216549281144 to 100.0. Channels: ␣ ˓ → 10. Events: 5 TSEvent pro vides se v eral methods f or combining multiple TSEvent objects and f or e xtracting data. See the API ref erence f or TSEvent f or full details. A ttributes ( TSEvent ) Attribut e name Description times V ector 𝑇 of sample times channels V ector of channels corresponding to sample times in times num_channels Scalar 𝐶 : number of channels in this object t_start , t_stop Firs t and last sample times, respectiv el y duration Duration betw een t_start and t_stop plotting_backend Cur rent plotting back end f or this instance. 6.4 Impor ting time series dat a T ime ser ies data impor ted from outside R ockpool often comes in a “cloc ked” f or mat, where the data is presented as a v ector of samples on an implicit time base. Sample times are often on a fix ed cloc k dt or sample frequency fs = 1 / dt . These representations can easil y be impor ted into R ockpool, but some care is required to mak e sure the data is handled cor rectl y . F or continuous-time data, TSContinuous pro vides the method from_clocked() . def TSContinuous . from_clocked( samples: numpy . ndarray, dt: float , t_start: float = 0.0 , periodic: bool = False , name: str = None , ) -> TSContinuous This method will accept regular l y sampled data on a specified sample cloc k dt , from multiple channels, and will retur n a cor rectl y -f or matted TSContinuous object with e xtents set cor rectly . This object will use “sample-and-hold” inter polation, as this seems to be the most common mental model that de v elopers ha v e f or data resampled within a time bin. 50 Chapter 6. W orking with time series dat a Rockpool, Release 3.0.3 [19]: # - Get a data sample T = 100 C = 3 dt = 0.1 data = np . random . rand(T, C) # - Create the time series ts = TSContinuous . from_clocked(data, dt = dt) print (ts) non-periodic TSContinuous object ` unnamed ` from t=0.0 to 10.0. Samples: 100. Channels: 3 Similar l y , impor ted ev ent data often appears in a “ras ter” f or mat, which is regular ly cloc ked. In this f or mat, a train of e v ents on 𝐶 channels o v er 𝑇 time bins is represented as a matr ix ( 𝑇 , 𝐶 ) , where the element ( 𝑡, 𝑐 ) indicates the number of e v ents occur r ing dur ing integ er time bin 𝑡 on channel 𝑐 . In the e xample belo w , se v en channels emit e v ents o v er sev en time bins. W e will use the TSEvent.from_raster() method to con v er t the set of e v ents from a numpy.ndarray into a TSEvent object. This will ensure that t_start , t_stop and num_channels attr ibutes are set appropratel y . def from_raster ( raster: np . ndarray, dt: float = 1.0 , t_start: float = 0.0 , t_stop: Optional[ float ] = None , name: Optional[ str ] = None , periodic: bool = False , num_channels: Optional[ int ] = None , spikes_at_bin_start: bool = False , ) -> TSEvent: [20]: Image( "raster_to_TSEvent.png" ) [20]: N ote that the e v ents are placed in the middle of each time bin. This beha viour can be modif ed with the spikes_at_bin_start argument to TSEvent.from_raster() . [21]: # - Generate a boolean raster T = 10 C = 20 rate = 0.1 raster = np . random . rand(T, C) <= 0.1 # - Convert to a time series using ` .from_raster() ` (continues on ne xt page) 6.4. Impor ting time series dat a 51 Rockpool, Release 3.0.3 58 Chapter 7. Standard modules CHAPTER EIGHT THE SHARP POINTS OF ROCKPOOL R ockpool aims to be an intuitiv e br idg e betw een super vised machine learning and dynamic signal processing, hiding the comple xities of the underl ying dynamical sy stems as muc h as possible. Ho w e v er , there are a f e w places where y ou need to consider that a R ockpool-based solution is actuall y a dynamical sy s tem, and not a simple stac k of stateless linear alg ebra. This notebook illustrates some of the sharp points that can jab y ou when y ou dip y our hands into R ockpool. Our goal is to mak e this list disappear o v er time. [1]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Rockpool imports from rockpool import TSContinuous # - General imports and configuration import numpy as np import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 8.1 How t o use sam pled time series data (in)correctly in Rockpool T ime ser ies data loaded from else where probably comes in a cloc ked ras ter f or mat. Y ou can easil y use this data in R ockpool, but there are a couple of tricky points to w atch out f or . 8.1.1 W rong: how to generate a time base for clock ed dat a [2]: T = 1000 dt = 1e-3 data = np . random . rand(T) t_start = 23.6 time_base = np . arange(t_start, t_start + len (data) * dt, dt) 59 Rockpool, Release 3.0.3 This approach can sometimes lead to rounding errors such that time_base is one sample too shor t, especially f or floating point numbers with < 64 bit precision. The better w a y is to g enerate integ er time s teps, then scale and shift them: [3]: time_base = np . arange(T) * dt + t_start 8.1.2 W rong: how to define a time series from clock ed dat a Sa y w e ha v e a 20-second sample, sampled on a 100ms cloc k. Let ’ s con v er t this into a continuous time ser ies: [4]: dt = 100e-3 T = round ( 20 / dt) data = np . random . rand(T) time_base = np . arange( len (data)) * dt ts = TSContinuous(time_base, data) ts [4]: non-periodic TSContinuous object ` unnamed ` from t=0.0 to 19.900000000000002. Samples: ␣ ˓ → 200. Channels: 1 Huh? The time ser ies is too shor t, it ’ s one dt off! But there are 200 samples. . . ? T ime ser ies objects don ’ t ha v e an intr isic cloc k; y ou can define samples at any point in time. So TSContinuous has no w a y to kno w that y ou e xpected a 20 second duration. By def ault, the time ser ies ends at the point in time where the last sample occurs. But f or clock ed data, y ou probabl y e xpected there to be an e xtra dt . The cor rect lo w-le v el wa y to define the time ser ies is to specify t_stop e xplicitly : [5]: ts = TSContinuous(time_base, data, t_stop = 20.0 ) ts [5]: non-periodic TSContinuous object ` unnamed ` from t=0.0 to 20.0. Samples: 200. Channels: 1 Ho w e v er , w e pro vide a con v enience method TSContinuous.from_clocked() to mak e this easier: [6]: ts = TSContinuous . from_clocked(data, dt = dt) ts [6]: non-periodic TSContinuous object ` unnamed ` from t=0.0 to 20.000000000000004. Samples: ␣ ˓ → 200. Channels: 1 TSContinuous.from_clocked() is the canonical w a y to impor t clock ed time-ser ies data into R oc kpool. If y ou use from_clocked() then e v er ything should beha v e as y ou e xpect it to. 8.2 Defining e xtents for TSEvent time series data Ev ent time ser ies are represented in R ockpool using the TSEvent class, which represents ev ents as occuring at discrete moments in time, on one of a number of channels. This is different from an “ev ent ras ter” representation, where e v ents are placed into discrete time bins, and the temporal resolution is limited to bin durations dt . [7]: from rockpool import TSEvent from matplotlib import pyplot as plt times = [ 0.2 , 0.8 , 1.2 , 1.4 , 1.8 , 2.2 , 3.3 , 3.5 , 3.6 , 4.2 , 4.8 , 5.2 , 5.8 , 6.2 , 6.5 , 6.8 ] (continues on ne xt page) 60 Chapter 8. The sharp points of Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) channels = [ 0 , 3 , 3 , 1 , 6 , 6 , 3 , 3 , 5 , 5 , 4 , 0 , 5 , 1 , 1 , 2 ] ts = TSEvent(times, channels, t_start = 0.0 , t_stop = 7.0 ) ts . plot(); [8]: from IPython.display import Image Image( "TSEvent_to_raster.png" ) [8]: The imag e abo v e sho w s the relationship between an e v ent time ser ies and one possible raster representation of that time ser ies. N ote that a raster is inherentl y lossy — it can onl y represent ev ents do wn to a minimum temporal resolution dt . W e can con v er t a TSEvent object into a raster b y using the appropr iatel y -named raster() method: def raster ( dt: float , t_start: float = None , t_stop: float = None , num_timesteps: int = None , channels: numpy . ndarray = None , add_events: bool = False , include_t_stop: bool = False , ) -> numpy . ndarray: With an e v ent raster , the time base is e xplicitl y defined. Time proceeds from t = t_start at the beginning of the raster , and continues until t = t_start + dt * T at the end of the raster (after T time bins). If y ou con v er t a ras ter directl y to a TSEvent object using the from_raster() method, then t_start and t_stop are inf er red. But a freshl y -created TSEvent object can ’ t inf er t_start and t_stop — w e need to supply these e xplicitl y when 8.2. Defining e xtents for TSEvent time series data 61 Rockpool, Release 3.0.3 creating the TSEvent . If w e don ’ t, things can g et confusing: [9]: tsBad = TSEvent(times, channels, t_stop = 7.0 , name = "No extents " ) print (tsBad) tsGood = TSEvent(times, channels, t_start = 0.0 , t_stop = 7.0 , name = "With extents " ) print (tsGood) non-periodic ` TSEvent ` object ` No extents ` from t=0.2 to 7.0. Channels: 7. Events: 16 non-periodic ` TSEvent ` object ` With extents ` from t=0.0 to 7.0. Channels: 7. Events: 16 Imagine y ou tr ied to evolve() a la y er with the first ts — the evolve() method w ould probabl y g et the ev olution duration wrong. Things w ould also g et messy if y ou tr ied to con v er t the first ts to a raster() . The time bins w ould begin at t = 0.2 rather than t=0 . The moral of the story is, y ou should alway s set the e xtents f or a TSEvent , if y ou are creating it from a list of e v ent times. 8.3 TimeSeries and Module s share an e xplicit global time base All TimeSeries and Module objects in R ockpool are defined with a shared global time base. That means t=0 ref ers to the same point in time f or all objects. This ma y slip y ou up if y ou pla y with a la y er f or while, ev ol ving with some input time series, and e xpect that e v olution will begin from the start of the time ser ies data f or each e v olution. [10]: # - Imports from rockpool.nn.modules import Rate from rockpool import TSContinuous # - Define a Timed module with a single rate neuron tmod = Rate( 1 ) . timed() print ( "tmod:" , tmod) # - Define a time series dt = 1e-3 data = np . sin(np . arange( 0 , 10 , dt) / 4 * ( 2 * np . pi)) ts_input = TSContinuous . from_clocked(data, dt = dt) ts_input . plot(); tmod: TimedModuleWrapper with shape (1, 1) { Rate ' _module ' with shape (1,) } with Rate ' _module ' as module 62 Chapter 8. The sharp points of Rockpool Rockpool, Release 3.0.3 N o w w e e v ol v e tmod using ts_input , and look at the result. [11]: output_ts, _, _ = tmod(ts_input) output_ts . plot() [11]: [<matplotlib.lines.Line2D at 0x7fe2aefbc550>] But if w e tr y to ev ol v e mod ag ain in the same w a y , we recie v e an er ror . [12]: tmod(ts_input); --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /var/folders/fj/ptcl18sd0yd8n9bm9fs_qtrc0000gn/T/ipykernel_10376/2584389186.py in ˓ → <module> ----> 1 tmod ( ts_input ); ~/SynSense Dropbox/Dylan Muir/LiveSync/Development/rockpool_GIT/rockpool/nn/modules/ ˓ → timed_module.py in __call__ (self, *args, **kwargs) 725 record_state ( dict ): If the argument `` record `` is `` True `` , ␣ ˓ → `` record_state `` must contain a dictionary of the recorded states o this and all sub - ˓ → modules during evolution . Otherwise it may be an empty dict . 726 """ (continues on ne xt page) 8.3. TimeSeries and Module s share an explicit global time base 63 Rockpool, Release 3.0.3 (continued from pre vious pag e) --> 727 return self . evolve (* args , ** kwargs ) 728 729 @ property ~/SynSense Dropbox/Dylan Muir/LiveSync/Development/rockpool_GIT/rockpool/nn/modules/ ˓ → timed_module.py in _evolve_wrapper (self, ts_input, duration, num_timesteps, kwargs_ ˓ → timeseries, record, *args, **kwargs) 289 """ 290 # - Determine number of timesteps --> 291 num_timesteps = self . _determine_timesteps ( ts_input , duration , num_ ˓ → timesteps ) 292 293 # - Call wrapped evolve ~/SynSense Dropbox/Dylan Muir/LiveSync/Development/rockpool_GIT/rockpool/nn/modules/ ˓ → timed_module.py in _determine_timesteps (self, ts_input, duration, num_timesteps) 344 duration = ts_input . t_stop - self . t --> 346 raise ValueError( 345 if duration <= 0 : 347 self . full_name 348 + "Cannot determine an appropriate evolution ␣ ˓ → duration." ValueError : TimedModuleWrapper Cannot determine an appropriate evolution duration. ' ts_ ˓ → input ' finishes before the current evolution time. This is because the inter nal time of the la y er is after the defined times of the timeser ies. There are se v eral possibilities to resol v e this issue, one of which is simpl y dela ying ts_input on creation b y using the t_start argument to from_clocked() : [13]: ts_input = TSContinuous . from_clocked(data, dt = dt, t_start = tmod . t) print ( "ts_input:" , ts_input) output, _, _ = tmod(ts_input) ts_input: non-periodic TSContinuous object ` unnamed ` from t=10.0 to 20.000000000000004. ␣ ˓ → Samples: 10000. Channels: 1 or b y using the TSContinuous.start_at() or TSContinuous.delay() methods when calling evolve() : [14]: # - Using ` .start_at() ` tmod(ts_input . start_at(tmod . t)) # - Using ` .delay() ` tmod(ts_input . delay(tmod . t - ts_input . t_start)); The final possibility is to specify that the time ser ies is per iodic, using the periodic = True flag to TSContinuous. from_clocked() . The time ser ies will then be defined o v er all time: [15]: ts_input = TSContinuous . from_clocked(data, dt = dt, periodic = True ) ts_input . plot(np . arange( - 10 , 20 , 0.1 )); 64 Chapter 8. The sharp points of Rockpool Rockpool, Release 3.0.3 and w e can ev ol v e as w e like: [16]: tmod(ts_input) tmod(ts_input); 8.4 Module s in Rockpool are dynamical syst ems, and don’ t get reset implicitly Neurons and modules in R oc kpool behav e quite differentl y from standard ANNs. The y are intr insicall y s tateful, e v en in the absence of e xplicit recur rent connectivity . Single neurons ha v e an internal state, which e v ol v es dynamically o v er time f ollo wing a set of differential equations. S tandard ANNs are stateless, meaning that each frame or sample is processed completel y independently . R ecur rent ANNs or LS TMs are often implicitl y reset at the s tar t of each trial. Neurons in R ockpool are onl y e v er reset e xplicitl y using nn.modules.Module.reset_state() . An e xplicit reset could become impor tant dur ing training, to ensure that a netw ork’ s response to a trial is not contami- nated b y the response to the previous trial. 8.4. Module s in Rockpool are dynamical systems, and don ’t get reset implicitly 65 Rockpool, Release 3.0.3 66 Chapter 8. The sharp points of Rockpool CHAPTER NINE L O W-LEVEL MODULE API The lo w -le v el API in R ockpool is designed f or minimal efficient implementation of s tateful neural netw orks. The Module base class pro vides f acilities f or configur ing, simulating and e xamining netw orks of stateful neurons. 9.1 Constructing a Module All Module subclasses accept minimall y a shape ar gument on construction. This should specify the in put, output and inter nal dimensionality of the Module completel y , so that the code can determine ho w many neurons should be g enerated, and the sizes of the s tate v ariables and parameters. Some Module subclasses allo w y ou to specify the module shape by setting concrete parameter arra y s, e.g. b y setting a v ector of length (N,) as the bias parameters f or a set of neurons. These concrete parameter v alues will be used to initialise the Module , and if the Module is reset, then the parameters will retur n to those concrete v alues. Other wise, all Module subclasses will set reasonable def ault initialisation v alues f or the parameters. [1]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Useful imports try : from rich import print except : pass # - Example of constructing a module from rockpool.nn.modules import Rate import numpy as np # - Construct a Module with 4 neurons mod = Rate( 4 ) print (mod) Rate with shape ( 4 , ) [2]: # - Construct a Module with concrete parameters mod = Rate( 4 , tau = np . ones( 4 )) print (mod) 67 Rockpool, Release 3.0.3 74 Chapter 9. Low-lev el Module API CHAPTER TEN HIGH-LEVEL TIMEDMODULE API The high-le v el API in R ockpool allo w s y ou to build netw orks of stateful neurons that accept and manipulate time-series data nativ el y . T ime-ser ies data in R ockpool is encapsulated b y TimeSeries classes: TSContinuous f or continuous signals, and TSEvent f or discrete e v ents (see W or king with time series data f or an o v er vie w). The TimedModule API allo w s y ou to specify modules that receiv e TimeSeries objects as input; process them; and retur n TimeSeries objects as output. 10.1 Constructing a TimedModule Instantiating a TimedModule w orks in a v er y similar w a y to the lo w-le v el API. Y ou must specify the simulation time- step dt , if a useful default is not pro vided. TimedModule will tr y to set a sensible dt inter nally if it can, but y ou usuall y w ant to ha v e control o v er this. [1]: # - Switch off warnings import warnings warnings . filterwarnings( ' ignore ' ) # - Useful imports import numpy as np try : from rich import print except : pass from rockpool.nn.combinators import Sequential from rockpool.nn.modules import Rate, Linear # - Construct the TimedModule Nin = 1 Nout = 5 dt = 1e-3 tmod = Sequential(Linear((Nin, Nout)), Rate(Nout)) . timed(dt = dt) print (tmod) TimedModuleWrapper with shape ( 1 , 5 ) { ModSequential ' _module ' with shape ( 1 , 5 ) { Linear ' 0_Linear ' with shape ( 1 , 5 ) Rate ' 1_Rate ' with shape ( 5 , ) (continues on ne xt page) 75 Rockpool, Release 3.0.3 (continued from pre vious pag e) } } with ModSequential ' _module ' as module 10.2 Ev olving a TimedModule P assing time ser ies data through a TimedModule is v er y con v enient. A TimedModule kno w s ho w to handle TimeSeries objects nativ el y , and handles an y required sampling, resampling, ras ter isation etc. inter nally . Let ’ s star t b y defining a sinusoid wa v ef or m to use as input. [2]: # - Import time-series handling classes from rockpool import TSContinuous # - Import and configure matplotlib for plotting import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ ' figure.figsize ' ] = [ 12 , 4 ] plt . rcParams[ ' figure.dpi ' ] = 300 # - Create a sinusoidal input signal T = 1000 omega = 10 time_base = np . arange(T) * dt ts_sin = TSContinuous . from_clocked( np . sin(time_base * 2 * np . pi * omega), dt = dt, periodic = True , name = ' Sinusoid ' , ) ts_sin . plot(); N o w w e can e v ol v e the TimedModule b y simply calling it with the in put time series. The evolve() arguments are slightl y different to that f or the lo w -le v el API: 76 Chapter 10. High-lev el TimedModule API Rockpool, Release 3.0.3 output_ts, new_state, recorded_state = evolve(ts_input: TimeSeries, duration: float, num_ ˓ → timesteps: int) Y ou can specify the e v olution duration either e xplicity in ter ms of real time ( duration in seconds), or in units of dt ( num_timesteps ); or implicitl y b y simpl y pro viding the input signal ts_input . If y ou just pro vide ts_input , then R oc kpool will e v ol v e the TimedModule f or the remaining v alid duration of time ser ies. R emember that R oc kpool [3]: # - Evolve the TimedModule output_ts, _, _ = tmod(ts_sin) plt . figure(); print (output_ts); output_ts . plot(); non-periodic TSContinuous object ` Output samples ` from t = 0.0 to 1.0 . Samples: 1000 . ␣ ˓ → Channels: 5 10.3 Inspecting a TimedModule Inspecting a TimedModule uses the common R ockpool API. All attributes can be accessed using standard Python “dot” inde xing notation, or the module can be inspected using the parameters() , state() , simulation_parameters() and modules() methods. [4]: # - Examine the parameters print ( ' Parameters: ' , tmod . parameters()) # - Examine the state print ( ' State: ' , tmod . state()) # - Examine the simulation parameters print ( ' Simulation parameters: ' , tmod . simulation_parameters()) # - ` TimedModules ` also have time print ( ' Time: ' , tmod . t) 10.3. Inspecting a TimedModule 77 Rockpool, Release 3.0.3 Parameters: { ' _module ' : { ' 0_Linear ' : { ' weight ' : array ([[ 1.1175366 , -1.50156676 , -1.88202227 , -0. ˓ → 95408104 , -0.08374108 ]])} , ' 1_Rate ' : { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) } } } State: { ' _module ' : { ' 0_Linear ' : {} , ' 1_Rate ' : { ' x ' : array ([ -0.56143606 , 0.75436789 , 0.94550386 , 0.47931808 , 0. ˓ → 04207044 ])} } } Simulation parameters: { ' dt ' : 0.001 , ' _module ' : { ' dt ' : 0.001 , ' 0_Linear ' : {} , ' 1_Rate ' : { ' dt ' : 0.001 , ' noise_std ' : 0.0 , ' act_fn ' : < function H_ReLU at ␣ ˓ → 0x7f05e46a4ee0 >} } } Time: 1.0 10.4 Building a netw ork using nested TimedModule s Y ou can inher it from the TimedModule base class to build arbitrar y netw orks, combining other TimedModule s as y ou lik e. TimedModule tak es care of man y things f or y ou, including all the handling of TimeSeries data. Y ou can k eep all data in TimeSeries objects and maniupulate the data in that f or mat, using the operator suppor t from TimeSeries subclases. Alternativ el y y ou can use con v enience methods pro vided b y TimedModule to assist with con v er ting signals to and from TimeSeries objects. T o define a netw ork as a TimedModule subclass, y ou need to minimally define an __init__() method and an evolve() method. The __init__() method specifies and initialises parameters f or y our netw ork, and initialises an y required submodules. The submodules tak e care of specifying and initialise their o wn parameters and state. In the e xample belo w , w e build a small f eed-f or ward netw ork as a TimedModule , with one la y er of linear input w eights and tw o additional la y ers of ReL U rate units with w eighting in-betw een. W e sho w ho w to maintain all signals as 78 Chapter 10. High-lev el TimedModule API Rockpool, Release 3.0.3 TimeSeries objects through e v olution. T o do so, all submodules must themsel v es be TimedModule subclasses. [5]: # - Import the base class and a TimedModule to use as a submodule from rockpool.nn.modules import TimedModule, Rate from rockpool.parameters import Parameter from rockpool import TimeSeries # - Define a new TimedModule class ffwd_tmod_net (TimedModule): # - Provide an ` __init__ ` method to specify required parameters and modules # Here you check, define and initialise whatever parameters and # state you need for your module. def __init__ ( self , shape, dt = 1. , * args, ** kwargs, ): # - Call superclass initialisation # This is always required for a ` Module ` class super () . __init__ (shape = shape, dt = dt, * args, ** kwargs) # - Specify weights attributes # We need a weights matrix for our input weights. # We specify the shape explicitly, and provide an initialisation function. # We also specify a family for the parameter, "weights". This is used to # query parameters conveniently, and is a good idea to provide. self . w_0 = Parameter( shape = self . shape[ 0 : 2 ], init_func = lambda s: np . zeros(s), family = ' weights ' , ) self . w_1 = Parameter( shape = self . shape[ 1 : 3 ], init_func = lambda s: np . zeros(s), family = ' weights ' , ) self . w_2 = Parameter( shape = self . shape[ 2 : 4 ], init_func = lambda s: np . zeros(s), family = ' weights ' , ) # - Specify and add submodules # These will be the neurons in our layer, to receive the weighted # input signals. This sub-module will be automatically configured # internally, to specify the required state and parameters self . neurons1 = Rate( self . shape[ 1 ], dt = dt) . timed() self . neurons2 = Rate( self . shape[ 2 ], dt = dt) . timed() # - The ` evolve ` method contains the internal logic of your module (continues on ne xt page) 10.4. Building a network using nested TimedModule s 79 Rockpool, Release 3.0.3 (continued from pre vious pag e) # ` evolve ` takes care of passing data in and out of the module, # and between sub-modules if present. def evolve ( self , ts_input: TimeSeries, duration: float , num_timesteps: int , record: bool = False , * args, ** kwargs): # - Pass input data through the input weights x = ts_input @ self . w_0 # - Pass the signals through the neurons and hidden weights x ,_ ,_ = self . neurons1(x) x = x @ self . w_1 x ,_ ,_ = self . neurons2(x) x = x @ self . w_2 # - Return the module output return x, {}, {} [6]: Nin = 3 Nout = 1 dt = 1 tmod = ffwd_tmod_net((Nin, 5 , 7 , Nout), dt = dt) print (tmod) ffwd_tmod_net with shape ( 3 , 5 , 7 , 1 ) { TimedModuleWrapper ' neurons1 ' with shape ( 5 , 5 ) { Rate ' _module ' with shape ( 5 , ) } TimedModuleWrapper ' neurons2 ' with shape ( 7 , 7 ) { Rate ' _module ' with shape ( 7 , ) } } [7]: # - Examine the nested parameters of the TimedModule print (tmod . parameters()) { ' w_0 ' : array ([[ 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 . ]]) , ' w_1 ' : array ([[ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]]) , ' w_2 ' : array ([[ 0 . ] , [ 0 . ] , [ 0 . ] , (continues on ne xt page) 80 Chapter 10. High-lev el TimedModule API Rockpool, Release 3.0.3 (continued from pre vious pag e) [ 0 . ] , [ 0 . ] , [ 0 . ] , [ 0 . ]]) , ' neurons1 ' : { ' _module ' : { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) } } , ' neurons2 ' : { ' _module ' : { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) } } } [8]: # - Set the weights to something non-zero p = tmod . parameters() p[ ' w_0 ' ] = np . random . normal(size = p[ ' w_0 ' ] . shape) + 1 p[ ' w_1 ' ] = np . random . normal(size = p[ ' w_1 ' ] . shape) + 1 p[ ' w_2 ' ] = np . random . normal(size = p[ ' w_2 ' ] . shape) + 1 # - Assign the new parameters tmod . set_attributes(p) print ( ' Weights: ' , tmod . parameters()) Weights: { ' w_0 ' : array ([[ 1.73722151 , 1.30356212 , 0.55156482 , 2.8019148 , 1.09027286 ] , [ 1.97210989 , 1.88721147 , 1.29312512 , -0.28583698 , 1.45047169 ] , [ 0.04967296 , -0.94915265 , 1.37862991 , 1.29235999 , 0.050853 ]]) , ' w_1 ' : array ([[ 1.52490803 , 0.53006083 , 3.11873417 , -0.15332358 , 1.51820289 , 2.88070423 , -0.55447475 ] , [ 0.03845312 , 0.76734798 , 0.77595556 , 2.23176957 , -0.90711186 , 1.39216172 , 0.132243 ] , [ 0.47959018 , 0.99225231 , 0.54699617 , 0.91157126 , 0.61972611 , 3.7774115 , 2.23284232 ] , [ 0.56585036 , 2.65117266 , 0.4449058 , 0.0736041 , 0.02211706 , -0.34763718 , -0.79899013 ] , [ 1.51294292 , 0.81017138 , 1.07416865 , -1.13288234 , 0.92521921 , 1.6997927 , 1.38329005 ]]) , ' w_2 ' : array ([[ -0.01196603 ] , [ 1.74152837 ] , [ -0.35683857 ] , [ -0.29947419 ] , [ 0.38918974 ] , [ 1.76011316 ] , (continues on ne xt page) 10.4. Building a network using nested TimedModule s 81 Rockpool, Release 3.0.3 (continued from pre vious pag e) [ 0.96554928 ]]) , ' neurons1 ' : { ' _module ' : { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 . ]) } } , ' neurons2 ' : { ' _module ' : { ' tau ' : array ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' threshold ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) } } } [9]: # - Generate some white noise input ts_input = TSContinuous . from_clocked(np . random . rand( 1000 , Nin), name = ' White noise ' , dt = dt, periodic = True , ) print ( ' Input: ' , ts_input) ts_input . plot() # - Evolve the TimedModule with the input ts_output, _, rec = tmod(ts_input) # - Display the output print ( ' Output: ' , ts_output) plt . figure() ts_output . plot(); Input: periodic TSContinuous object ` White noise ` from t = 0.0 to 1000.0 . Samples: 1000 . ␣ ˓ → Channels: 3 Output: non-periodic TSContinuous object ` Output samples '' neurons2 ''` from t = 0.0 to ␣ ˓ → 1000.0 . Samples: 1000 . Channels: 1 82 Chapter 10. High-lev el TimedModule API Rockpool, Release 3.0.3 10.5 Building a netw ork using internal Module s Sometimes the modules y ou w ant to use to build a netw ork are not a vailable as TimedModule s. In this case y ou can use the lo w -le v el Module API inter nall y , and still present a high-le v el TimedModule API inter f ace to the outside w or ld. T o do so, TimedModule pro vides a number of con v enience methods f or wrapping, un wrapping and ras ter ising time ser ies data: time_base, input_raster, num_timesteps = self._repare_input( ts_input: TimeSeries, duration: float, num_timesteps: int ) ts_output = self._gen_timeseries(output_raster, **kwargs) The e xample belo w illustrates ho w to use these methods to build a similar netw ork architecture to the one abo v e, but using lo w -le v el Module API submodules. 10.5. Building a network using internal Module s 83 Rockpool, Release 3.0.3 (continued from pre vious pag e) [ 0.05882018 0.09205481 0.15850767 ] [ 0.08602361 0.12148949 0.16591538 ] [ 0.10797263 0.11710763 0.18505278 ] [ 0.13634151 0.13502166 0.20233528 ] [ 0.15542242 0.15269144 0.20374872 ] [ 0.16027772 0.15767853 0.19997193 ]]] [4]: print ( "new_state:" , new_state) new_state: { ' x ' : DeviceArray ([ 0.16027772 , 0.15767853 , 0.19997193 ] , dtype = float32 ) , ' rng_key ' : DeviceArray ([ 2469880657 , 3700232383 ] , dtype = uint32 ) } So f ar so good. The issue with jax is that jit -compiled modules and functions cannot ha v e side-effects. For R oc kpool, e v olution almost alw ays has side-effects, in terms of updating the inter nal state v ar iables of each module. In the case of the e v olution abo v e, w e can see that the inter nal state w as not updated dur ing ev olution: [5]: print ( "mod.state:" , mod . state()) print (mod . state()[ "x" ], " != " , new_state[ "x" ]) mod.state: { ' rng_key ' : DeviceArray ([ 237268104 , 2681681569 ] , dtype = uint32 ) , ' x ' : DeviceArray ([ 0 ., 0 ., 0 . ] , dtype = float32 ) } [ 0 . 0 . 0 . ] != [ 0.16027772 0.15767853 0.19997193 ] The cor rect resolution to this is to assign new_state to the module atf er each e v olution: [6]: mod = mod . set_attributes(new_state) print (mod . state()[ "x" ], " == " , new_state[ "x" ]) [ 0.16027772 0.15767853 0.19997193 ] == [ 0.16027772 0.15767853 0.19997193 ] Y ou will ha v e noticed the functional f or m of the call to set_attributes() abo v e. This is addressed in the ne xt section. 11.2 F unctional state and attribute setting Direct attr ibute assignment w orks at the top le v el, using standard Python syntax: [7]: new_tau = mod . tau * 0.4 mod . tau = new_tau print (new_tau, " == " , mod . tau) 90 Chapter 11. [] Low-le vel functional API Rockpool, Release 3.0.3 [ 0.008 0.008 0.008 ] == [ 0.008 0.008 0.008 ] A functional f or m is also suppor ted, via the set_attributes() method. Here a cop y of the module (and submodules) is retur ned, to replace the “old” module with one with updated attr ibutes: [8]: params = mod . parameters() params[ "tau" ] = params[ "tau" ] * 3.0 # - Note the functional calling style mod = mod . set_attributes(params) # - check that the attribute was set print (params[ "tau" ], " == " , mod . tau) [ 0.024 0.024 0.024 ] == [ 0.024 0.024 0.024 ] 11.3 F unctional module reset R esetting the module s tate and parameters also must be done using a functional f or m: [9]: # - Reset the module state mod = mod . reset_state() # - Reset the module parameters mod = mod . reset_parameters() 11.4 Jax flattening JaxModule pro vides the methods tree_flatten() and tree_unflatten() , which are required to serialise and deser ialise modules f or Jax compilation and e x ecution. If y ou wr ite a JaxModule subclass, it will be automatically regis tered with Jax as a pytree . Y ou shouldn ’ t need to o v er r ide tree_flatten() or tree_unflatten() in y our modules. Flattening and unflattening requires that y our __init__() method mus t be callable with onl y a shape as in put, which should be sufficient to specify the netw ork architecture of y our module and all submodules. If that isn ’ t the case, then y ou ma y need to o v er r ide tree_flatten() and tree_unflatten() . 11.3. Functional module reset 91 Rockpool, Release 3.0.3 92 Chapter 11. [] Low-le vel functional API CHAPTER T WEL VE TRAINING A ROCKPOOL NET W ORK WITH J AX Jax is a Python pac kag e f or differentiable programming. With a con v enient numpy -lik e inter face, Jax will automagi- call y compute the gradients of y our code. This is a hug e boon f or optimisation, especiall y f or neural netw orks. In this o v er view w e sho w ho w to build and optimise a simple Jax-based netw ork in R ockpool. W e sho w ho w to wr ite a task Dataset, ho w to wr ite a loss function, and ho w to wr ite a training loop to per f orm the optimisation. W e also illustrate some adv anced topics, such as pro viding parameter bounds during optimisation. [1]: # -- Some useful imports # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Rich printing try : from rich import print except : pass # - Numpy import numpy as np # - Import and configure matplotlib for plotting import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 12.1 Jax considerations Jax is a functional programming librar y — ideally e v er ything should be wr itten as side-effect-free functions. F or this pur pose, R oc kpool pro vides the functional API (see [] Low-lev el functional API ). R ockpool tries to make using Jax -bac k ed modules as straightf or ward as possible, via the JaxModule base class. Jax pro vides a v er y conv enient nump y -compatible inter f ace, via the jax.numpy packag e. If y ou need to do an y numer ic computation inter f aced with R oc kpool/Jax, then y ou must use jax.numpy . Y ou ’ll receiv e an er ror if y ou don ’ t. 93 Rockpool, Release 3.0.3 12.2 Loss functions 12.2.1 Loss function com ponents pro vided by R ockpool R ockpool pro vides some useful training utilities under training . training.jax_loss includes se v eral components f or building y our o wn loss (or cost) functions. F unction Use mse() Mean-squared error (basic loss). Ensures that tw o signals become more similar (e.g. tar g et signal and netw ork output). l2sqr_norm() L2-squared norm, f or parameter regular isation. Keeps parameter v alues closer to zero. l0_norm_appro x() Smooth and differentiable L0 nor m appro ximation. Encourag es parameter sparsity (i.e. man y zero entr ies in a parameter v ector). bounds_cost() Pro vide a cost function component that enf orces minimum and/or maximum parameter bounds. make_bounds() Con v enience function to cons tr uct a template set of bounds f or use in an optimisation problem. softmax() Compute the softmax function. Useful f or incor poration in the readout la y ers of deep netw orks, dur ing training. logsoftmax() Compute the log softmax function. Used in readouts when training with cross-entopy loss functions. Y ou can use these components b y impor ting rockpool.training.jax_loss : [2]: # - Import the loss components for use from rockpool.training import jax_loss as l 12.2.2 W riting y our own loss function F or later use in optimising a netw ork, a loss function mus t be differentiable with respect to the netw ork parameters. A con v enient w a y to achie v e this is illutrated here, where we define a loss function that accepts the parameters, the netw ork object, the inputs f or this batc h, and the cor responding targ et signals. The loss function is then responsible f or initialising the netw ork — remember R ockpool netw orks usuall y ha v e s tate, and this needs to be tak en into account dur ing training – setting the netw ork parameters, e v ol ving the netw ork and computing the loss f or this batch. This f or m is con v enient since y ou can compute gradients o v er the entire function. By def ault, jax computes gradients o v er the first argument to a function — in this case, the set of netw ork parameters. R emember , if y ou w ant to do an y other arbitrar y calculations, y ou must use jax.numpy instead of numpy . [3]: def loss_mse (parameters, net, inputs, target): # - Handle the network state — randomise or reset net = net . reset_state() # - Assign the provided parameters to the network net = net . set_attributes(parameters) # - Evolve the network to get the ouput output, _, _ = net(inputs) # - Compute a loss value w.r.t. the target output return l . mse(output, target) 94 Chapter 12. T raining a Rockpool network with Jax Rockpool, Release 3.0.3 12.3 Defining a task dataset W e will define a simple random regression task, where random frozen input noise is mapped to randoml y chosen smooth output signals. W e implement this using a Dataset -compatible class, implementing the __len__() and __getitem__() methods. [4]: # - Define a dataset class implementing the indexing interface class MultiClassRandomSinMapping : def __init__ ( self , num_classes: int = 2 , sample_length: int = 100 , input_channels: int = 50 , target_channels: int = 2 , ): # - Record task parameters self . _num_classes = num_classes self . _sample_length = sample_length # - Draw random input signals self . _inputs = np . random . randn(num_classes, sample_length, input_channels) + 1.0 # - Draw random sinusoidal target parameters self . _target_phase = np . random . rand(num_classes, 1 , target_channels) * 2 * np . pi self . _target_omega = ( np . random . rand(num_classes, 1 , target_channels) * sample_length / 50 ) # - Generate target output signals time_base = np . atleast_2d(np . arange(sample_length) / sample_length) . T self . _targets = np . sin( 2 * np . pi * self . _target_omega * time_base + self . _target_phase ) def __len__ ( self ): # - Return the total size of this dataset return self . _num_classes def __getitem__ ( self , i): # - Return the indexed dataset sample return self . _inputs[i], self . _targets[i] [5]: # - Instantiate a dataset Nin = 2000 Nout = 2 num_classes = 3 T = 100 ds = MultiClassRandomSinMapping( num_classes = num_classes, input_channels = Nin, target_channels = Nout, sample_length = T, ) (continues on ne xt page) 12.3. Defining a task dat aset 95 Rockpool, Release 3.0.3 (continued from pre vious pag e) # Display the dataset classes plt . figure() for i, sample in enumerate (ds): plt . subplot( 2 , len (ds), i + 1 ) plt . imshow(sample[ 0 ] . T, aspect = "auto" ) plt . title( f"Input class { i } " ) plt . subplot( 2 , len (ds), i + len (ds) + 1 ) plt . plot(sample[ 1 ]) plt . xlabel( f"Target class { i } " ) 12.4 Building a Jax netw ork W e ’ll define a v er y simple netw ork to sol v e the regression task, which will in f act not be s tateful — w e ’ll use LinearJax modules to wrap w eight matr ices, and the InstantJax module to add a non-linear ity . W e ’ll build an MLP -lik e netw ork with one hidden la y er incor porating a tanh nonlinear ity . The Sequential combi- nator is used to con v enientl y stac k the modules tog ether . [6]: # - Import the Rockpool modules and Sequential combinator from rockpool.nn.modules import LinearJax, InstantJax from rockpool.nn.combinators import Sequential import jax import jax.numpy as jnp Nhidden = 8 net = Sequential( LinearJax((Nin, Nhidden)), InstantJax(Nhidden, jnp . tanh), LinearJax((Nhidden, Nout)), ) print (net) 96 Chapter 12. T raining a Rockpool network with Jax Rockpool, Release 3.0.3 JaxSequential with shape ( 2000 , 2 ) { LinearJax ' 0_LinearJax ' with shape ( 2000 , 8 ) InstantJax ' 1_InstantJax ' with shape ( 8 , ) LinearJax ' 2_LinearJax ' with shape ( 8 , 2 ) } 12.5 W riting a training loop In the e xample here w e use an implementation of the A dam optimiser pro vided b y Jax . See the Jax documentation on ho w to use their optimiser inter f ace. The jax.value_and_grad() transf or m accepts our loss function, and con v er ts it automaticall y into a function that computes the gradient of the loss with respect to the netwrork parameters (as w ell as the loss v alue itself). W e mak e use of jax.jit() to compile the optimiser and loss gradient functions, so the y are computed efficientl y on the GPU or CPU . W e k eep trac k of the loss v alue o v er tr ials, so w e can obser v e the training process. [7]: # - Useful imports from tqdm .autonotebook import tqdm from copy import deepcopy from itertools import count # -- Import an optimiser to use and initalise it import jax from jax.example_libraries.optimizers import adam, sgd # - Get the optimiser functions init_fun, update_fun, get_params = adam( 1e-4 ) # - Initialise the optimiser with the initial parameters params0 = deepcopy(net . parameters()) opt_state = init_fun(params0) # - Get a compiled value-and-gradient function loss_vgf = jax . jit(jax . value_and_grad(loss_mse)) # - Compile the optimiser update function update_fun = jax . jit(update_fun) # - Record the loss values over training iterations loss_t = [] grad_t = [] num_epochs = 1000 # - Loop over iterations i_trial = count() for _ in tqdm( range (num_epochs)): for sample in ds: # - Get an input / target sample (continues on ne xt page) 12.5. Writing a training loop 97 Rockpool, Release 3.0.3 (continued from pre vious pag e) input , target = sample[ 0 ], sample[ 1 ] # - Get parameters for this iteration params = get_params(opt_state) # - Get the loss value and gradients for this iteration loss_val, grads = loss_vgf(params, net, input , target) # - Update the optimiser opt_state = update_fun( next (i_trial), grads, opt_state) # - Keep track of the loss loss_t . append(loss_val) 100%|| 1000/1000 [00:03<00:00, 278.95it/s] W e can visualise the loss to see that w e are indeed lear ning to match the desired netw ork output. [8]: # - Plot the loss over iterations plt . plot(np . array(loss_t)) plt . yscale( "log" ) plt . xlabel( "Iteration" ) plt . ylabel( "Loss" ) plt . title( "Training loss" ); As a sanity chec k w e can e valuate the netw ork f or each class, and plot the netw ork output v ersus the targ et signals: [9]: # - Apply the trained parameters to the network params_hat = get_params(opt_state) net = net . set_attributes(params_hat) # - Evaluate classes for i_class, sample in enumerate (ds): input , target = sample # - Evaluate network net = net . reset_state() (continues on ne xt page) 98 Chapter 12. T raining a Rockpool network with Jax Rockpool, Release 3.0.3 (continued from pre vious pag e) output, _, _ = net( input , record = True ) # - Plot output and target plt . figure() plt . plot(output[ 0 ], "k-" ) plt . plot(sample[ 1 ], "--" ) plt . xlabel( "Time (steps)" ) plt . ylabel( "Value" ) plt . legend( [ "Output $y_0$" , "Output $y_1$" , "Target $\hat {y} _0$" , "Target $\hat {y} _1$" , ] ) plt . title( f"Class { i_class } " ) 12.5. Writing a training loop 99 Rockpool, Release 3.0.3 And if w e visualise the time constants, w e should see that none of them violate the lo w er bounds w e imposed: [17]: # - Get the final set of optimised parameters and apply them params = get_params(opt_state) net_stateful = net_stateful . set_attributes(params) # - Visualise the time constants plt . hist(net_stateful[ 1 ] . tau * 1e3 , 20 ) plt . xlabel( "Time constants $ \\ tau$ (ms)" ) plt . ylabel( "Count" ) plt . title( "Trained time constants" ); 106 Chapter 13. Adv anced Jax training topics CHAPTER F OURTEEN BUILDING ROCKPOOL MODULES WITH T ORCH R ockpool pro vides torc h-bac k ed modules with standard dynamics, f or simple integ ration with other torch-pro vided modules from torch.nn . Class Description RateTorc h A la y er of non-spiking fir ing-rate neurons, with trainable time constants, thresholds and biases per neu- ron; optinall y suppor ting recur rent connectivity LIFTorch A la y er of leaky integrate-and-fire spiking neurons, optionally supporting recur rent connectivity . T rain- inable with sur rogate gradient descent, with trainable time cons tants, biases, thresholds per neuron ExpSynTo rch Exponential synapses, with trainable time constants LinearTo rch Equiv alent to a standard trainable linear w eights la y er , but full y suppor ting the R ockpool APIs InstantT orch W rap an arbitrar y function as a R ockpool module 14.1 Use the Rockpool T orch-bac k ed classes The classes abo v e can be used directly to build netw ork arc hitectures in R ockpool, including mixing classes from torch.nn . Here w e build a simple f eed-f or w ard dynamical rate netw ork, including a dropout la y er from torch. [22]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Rich printing try : from rich import print except : pass # - Import and configure matplotlib for plotting import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 # - Torch imports (continues on ne xt page) 107 Rockpool, Release 3.0.3 (continued from pre vious pag e) import torch import torch.nn as nn import torch.nn.functional as F [69]: from rockpool.nn.modules import RateTorch, LinearTorch from rockpool.nn.combinators import Sequential Nin = 2 Nhidden = 5 Nout = 2 # Define a simple feed-forward network using the Torch backend net = Sequential( LinearTorch((Nin, Nhidden)), RateTorch((Nhidden,)), nn . Dropout2d( 0.25 ), LinearTorch((Nhidden, Nout)), RateTorch((Nout,)), ) net [69]: TorchSequential with shape (2, 2) { LinearTorch ' 0_LinearTorch ' with shape (2, 5) RateTorch ' 1_RateTorch ' with shape (5,) Dropout2d ' 2_Dropout2d ' with shape (None,) LinearTorch ' 3_LinearTorch ' with shape (5, 2) RateTorch ' 4_RateTorch ' with shape (2,) } [70]: # - Evolve the network on random data and plot data = torch . rand(( 1 , 100 , Nin)) out, _, _ = net(data) plt . plot(out[ 0 ] . detach()); [71]: # - Recording internal signals also works out, _, rd = net(data, record = True ) print ( list (rd . keys())) 108 Chapter 14. Building Rockpool modules with T orch Rockpool, Release 3.0.3 [ ' 0_LinearTorch ' , ' 0_LinearTorch_output ' , ' 1_RateTorch ' , ' 1_RateTorch_output ' , ' 2_Dropout2d ' , ' 2_Dropout2d_output ' , ' 3_LinearTorch ' , ' 3_LinearTorch_output ' , ' 4_RateTorch ' , ' 4_RateTorch_output ' ] 14.2 Con ver t an e xisting T orch torch.nn.module for use in Rockpool T orch modules implemented using torch.nn.Module can be con v er ted directl y to the R ockpool API using the method TorchModule.from_torch() . This method returns an object adher ing to the R oc kpool lo w-le v el API, con v er ting T orch calls and attr ibutes into R ockpool calls and regis tered attr ibutes. Here w e sho w an e xample of a simple T orch module co v er ted to a R oc kpool object. [72]: # - Torch imports import torch import torch.nn as nn import torch.nn.functional as F # - Rockpool imports from rockpool.nn.modules import TorchModule # - Implement a Torch class class TorchNet (torch . nn . Module): def __init__ ( self , * args, ** kwargs): super () . __init__ ( * args, ** kwargs) # - Build some convolutional layers self . conv1 = nn . Conv2d( 1 , 2 , 3 , 1 ) # - Add a dropout layer self . dropout1 = nn . Dropout2d( 0.25 ) # - Fully-connected layer self . fc1 = nn . Linear( 338 , 10 ) # - Register an example buffer self . register_buffer( "test_buf" , torch . zeros( 3 , 4 )) def forward ( self , x): x = self . conv1(x) x = F . relu(x) x = F . max_pool2d(x, 2 ) (continues on ne xt page) 14.2. Conv ert an existing T orch torch.nn.module for use in Rockpool 109 Rockpool, Release 3.0.3 (continued from pre vious pag e) x = self . dropout1(x) x = torch . flatten(x, 1 ) x = self . fc1(x) x = F . relu(x) output = F . log_softmax(x, dim = 1 ) return output [73]: # - Instantiate the network and test the Torch API # Equates to one random 28x28 image random_data = torch . rand(( 1 , 1 , 28 , 28 )) # - Generate torch module and test evaluation mod = TorchNet() result = mod(random_data) [74]: # - Convert object to Rockpool API, in-place TorchModule . from_torch(mod) print (mod) TorchNet ' TorchModulePatch ' with shape ( None , ) { Conv2d ' TorchModulePatch ' with shape ( None , ) Dropout2d ' TorchModulePatch ' with shape ( None , ) Linear ' TorchModulePatch ' with shape ( None , ) } [75]: # - Use the Rockpool API to evolve the module output, _, _ = mod(random_data) print (output) tensor ([[ -2.2458 , -1.9656 , -2.3617 , -2.3617 , -2.3617 , -2.3617 , -2.3617 , -2.3617 , -2.3617 , -2.3617 ]] , grad_fn = < LogSoftmaxBackward0 >) The module attr ibutes can be accessed using the R ockpool API via parameters() , state() and simulationparameters() methods. The attr ibute dictionar ies retur ned b y these methods suppor t an addi- tional method astorch() , whic h con v er ts the attr ibute dictionar y to a g enerator retur ning ra w Tensor s. Doing so is equiv alent to calling the Torch.nn.Module.parameters() method. [76]: # - Use the Rockpool API to access parameters print ( "Parameters: " , mod . parameters()) print ( "State: " , mod . state()) Parameters: { ' conv1 ' : { ' weight ' : Parameter containing: tensor ([[[[ -0.1636 , -0.3071 , 0.0886 ] , [ 0.1826 , -0.0988 , -0.2805 ] , (continues on ne xt page) 110 Chapter 14. Building Rockpool modules with T orch Rockpool, Release 3.0.3 (continued from pre vious pag e) [ 0.1841 , -0.1396 , -0.0389 ]]] , [[[ 0.2814 , -0.2359 , -0.0974 ] , [ -0.2386 , 0.3125 , 0.1958 ] , [ 0.3165 , 0.0791 , -0.0173 ]]]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ -0.1282 , 0.0541 ] , requires_grad = True ) } , ' dropout1 ' : {} , ' fc1 ' : { ' weight ' : Parameter containing: tensor ([[ 0.0261 , -0.0336 , -0.0126 , ... , -0.0055 , 0.0495 , -0.0540 ] , [ 0.0469 , 0.0163 , -0.0500 , ... , -0.0408 , -0.0364 , -0.0121 ] , [ -0.0284 , 0.0247 , 0.0290 , ... , -0.0128 , 0.0444 , 0.0534 ] , ... , [ 0.0184 , 0.0500 , 0.0326 , ... , -0.0116 , -0.0092 , 0.0071 ] , [ 0.0190 , 0.0424 , -0.0505 , ... , -0.0379 , -0.0238 , 0.0469 ] , [ 0.0119 , 0.0063 , 0.0538 , ... , -0.0211 , -0.0373 , 0.0374 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ -0.0106 , -0.0069 , -0.0463 , 0.0346 , 0.0390 , -0.0147 , -0.0386 , -0.0023 , 0.0171 , -0.0368 ] , requires_grad = True ) } } State: { ' test_buf ' : tensor ([[ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ]]) , ' conv1 ' : {} , ' dropout1 ' : {} , ' fc1 ' : {} } [77]: # - Convert the parameter dictionary to torch parameters print ( "Parameters.astorch(): " , list (mod . parameters() . astorch())) Parameters.astorch () : [ Parameter containing: tensor ([[[[ -0.1636 , -0.3071 , 0.0886 ] , [ 0.1826 , -0.0988 , -0.2805 ] , [ 0.1841 , -0.1396 , -0.0389 ]]] , [[[ 0.2814 , -0.2359 , -0.0974 ] , [ -0.2386 , 0.3125 , 0.1958 ] , [ 0.3165 , 0.0791 , -0.0173 ]]]] , requires_grad = True ) , Parameter containing: (continues on ne xt page) 14.2. Conv ert an existing T orch torch.nn.module for use in Rockpool 111 Rockpool, Release 3.0.3 (continued from pre vious pag e) tensor ([ -0.1282 , 0.0541 ] , requires_grad = True ) , Parameter containing: tensor ([[ 0.0261 , -0.0336 , -0.0126 , ... , -0.0055 , 0.0495 , -0.0540 ] , [ 0.0469 , 0.0163 , -0.0500 , ... , -0.0408 , -0.0364 , -0.0121 ] , [ -0.0284 , 0.0247 , 0.0290 , ... , -0.0128 , 0.0444 , 0.0534 ] , ... , [ 0.0184 , 0.0500 , 0.0326 , ... , -0.0116 , -0.0092 , 0.0071 ] , [ 0.0190 , 0.0424 , -0.0505 , ... , -0.0379 , -0.0238 , 0.0469 ] , [ 0.0119 , 0.0063 , 0.0538 , ... , -0.0211 , -0.0373 , 0.0374 ]] , requires_grad = True ) , Parameter containing: tensor ([ -0.0106 , -0.0069 , -0.0463 , 0.0346 , 0.0390 , -0.0147 , -0.0386 , -0.0023 , 0.0171 , -0.0368 ] , requires_grad = True ) ] 14.3 W rite a nativ e Rockpool/T orch module using TorchModule Y ou can also use TorchModule directl y as a base class, in place of torch.nn.Module . Usuall y this will be a drop-in replacement, without modifying the initialisation or e v aluation code. The e xample here mimics the netw ork abo v e — onl y the inherited base class has been chang ed. [78]: # - Implement a Rockpool class using the TorchModule base class class RockpoolNet (TorchModule): def __init__ ( self , * args, ** kwargs): super () . __init__ ( * args, ** kwargs) # - Build some convolutional layers self . conv1 = nn . Conv2d( 1 , 2 , 3 , 1 ) # - Add a dropout layer self . dropout1 = nn . Dropout2d( 0.25 ) # - Fully-connected layer self . fc1 = nn . Linear( 338 , 10 ) # - Register an example buffer self . register_buffer( "test_buf" , torch . zeros( 3 , 4 )) def forward ( self , x): x = self . conv1(x) x = F . relu(x) x = F . max_pool2d(x, 2 ) x = self . dropout1(x) x = torch . flatten(x, 1 ) x = self . fc1(x) x = F . relu(x) (continues on ne xt page) 112 Chapter 14. Building Rockpool modules with T orch Rockpool, Release 3.0.3 (continued from pre vious pag e) output = F . log_softmax(x, dim = 1 ) return output [79]: # - Instantiate the Rockpool class directly rmod = RockpoolNet() print (rmod) RockpoolNet with shape ( None , ) { Conv2d ' conv1 ' with shape ( None , ) Conv2d ' conv1 ' with shape ( None , ) Dropout2d ' dropout1 ' with shape ( None , ) Dropout2d ' dropout1 ' with shape ( None , ) Linear ' fc1 ' with shape ( None , ) Linear ' fc1 ' with shape ( None , ) } [80]: # - Evaluate the module using the Rockpool API output, _, _ = rmod(random_data) print (output) tensor ([[ -2.3283 , -2.3283 , -2.3283 , -2.3283 , -2.3283 , -2.2359 , -2.2393 , -2.2600 , -2.3283 , -2.3283 ]] , grad_fn = < LogSoftmaxBackward0 >) [81]: # - Access parameters using the Rockpool API print ( "Parameters: " , rmod . parameters()) print ( "State: " , rmod . state()) Parameters: { ' conv1 ' : { ' weight ' : Parameter containing: tensor ([[[[ 0.2828 , -0.2562 , 0.0924 ] , [ 0.1476 , 0.0577 , -0.0121 ] , [ 0.0073 , 0.2832 , -0.2923 ]]] , [[[ 0.2786 , 0.0777 , -0.0933 ] , [ -0.1199 , 0.0570 , -0.2343 ] , [ 0.2991 , 0.0064 , -0.2985 ]]]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ -0.2721 , -0.2435 ] , requires_grad = True ) } , ' dropout1 ' : {} , ' fc1 ' : { ' weight ' : Parameter containing: tensor ([[ -0.0320 , 0.0249 , -0.0329 , ... , 0.0196 , 0.0241 , -0.0021 ] , [ -0.0400 , 0.0448 , -0.0266 , ... , -0.0056 , -0.0111 , 0.0318 ] , [ 0.0204 , 0.0127 , -0.0184 , ... , 0.0482 , -0.0074 , 0.0258 ] , ... , [ 0.0328 , -0.0477 , 0.0297 , ... , -0.0182 , -0.0296 , 0.0217 ] , (continues on ne xt page) 14.3. Writ e a native R ockpool/T orch module using TorchModule 113 Rockpool, Release 3.0.3 (continued from pre vious pag e) [ -0.0037 , 0.0421 , 0.0048 , ... , 0.0057 , -0.0409 , -0.0112 ] , [ -0.0214 , -0.0465 , -0.0319 , ... , 0.0304 , -0.0220 , -0.0306 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0.0059 , 0.0042 , -0.0443 , 0.0200 , 0.0258 , 0.0406 , 0.0433 , -0.0303 , 0.0038 , 0.0473 ] , requires_grad = True ) } } State: { ' test_buf ' : tensor ([[ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ]]) , ' conv1 ' : {} , ' dropout1 ' : {} , ' fc1 ' : {} } 14.4 Con ver ting from Rockpool/torch t o pur e torch Sometimes y ou ma y want to use the R oc kpool pro vided TorchModule der iv ed classes with other software that e xpects pure torch (e.g. MLFlo w or Pytorc h Lightning). In that case y ou can use the to_torch() method to e xpose a pure torch inter f ace. Here w e sho w ho w that w orks, using the class RockpoolNet defined abo v e. 14.4.1 Rockpool API [82]: # - Instantiate the Rockpool class net = RockpoolNet() print (net) RockpoolNet with shape ( None , ) { Conv2d ' conv1 ' with shape ( None , ) Conv2d ' conv1 ' with shape ( None , ) Dropout2d ' dropout1 ' with shape ( None , ) Dropout2d ' dropout1 ' with shape ( None , ) Linear ' fc1 ' with shape ( None , ) Linear ' fc1 ' with shape ( None , ) } [83]: # - Rockpool dictionary-based parameter API print ( "Parameters:" , net . parameters()) Parameters: { ' conv1 ' : { ' weight ' : Parameter containing: (continues on ne xt page) 114 Chapter 14. Building Rockpool modules with T orch Rockpool, Release 3.0.3 (continued from pre vious pag e) tensor ([[[[ -0.2905 , 0.1654 , -0.1893 ] , [ -0.0804 , 0.1523 , -0.3320 ] , [ 0.0346 , 0.1020 , 0.0288 ]]] , [[[ -0.1374 , 0.3117 , 0.0558 ] , [ -0.3279 , 0.1651 , 0.3008 ] , [ 0.0011 , 0.1701 , -0.1425 ]]]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ -0.1574 , -0.1525 ] , requires_grad = True ) } , ' dropout1 ' : {} , ' fc1 ' : { ' weight ' : Parameter containing: tensor ([[ 0.0269 , -0.0310 , 0.0103 , ... , -0.0044 , -0.0114 , -0.0388 ] , [ 0.0409 , -0.0286 , 0.0256 , ... , 0.0166 , 0.0072 , -0.0476 ] , [ -0.0356 , 0.0108 , -0.0136 , ... , -0.0108 , 0.0268 , 0.0322 ] , ... , [ -0.0467 , -0.0277 , 0.0084 , ... , -0.0023 , 0.0034 , -0.0107 ] , [ -0.0191 , 0.0524 , -0.0005 , ... , 0.0305 , 0.0221 , -0.0304 ] , [ -0.0199 , 0.0537 , 0.0393 , ... , -0.0248 , 0.0081 , 0.0438 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0.0454 , 0.0186 , -0.0167 , -0.0339 , 0.0249 , 0.0274 , -0.0339 , -0.0019 , -0.0250 , -0.0271 ] , requires_grad = True ) } } [84]: # Evaluate one random 28x28 image random_data = torch . rand(( 1 , 1 , 28 , 28 )) # - Rockpool standard calling semantics print (net(random_data)) ( tensor ([[ -2.3239 , -2.3239 , -2.2561 , -2.2330 , -2.3239 , -2.3239 , -2.3239 , -2.3239 , -2.2750 , -2.3239 ]] , grad_fn = < LogSoftmaxBackward0 >) , { ' test_buf ' : tensor ([[ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ] , [ 0 ., 0 ., 0 ., 0 . ]]) , ' conv1 ' : {} , ' dropout1 ' : {} , ' fc1 ' : {} } , {} ) 14.4. Conv erting from Rockpool/torch to pure torch 115 Rockpool, Release 3.0.3 122 Chapter 15. T raining a Rockpool network with T orch CHAPTER SIXTEEN HO W TO: C ONFIGURE AND PERF ORM CONS TRAINED OPTIMIZA TION IN ROCKPOOL Spiking Neural N etw orks present a more comple x optimisation problem than standard DNNs. This is due not onl y to the comple x dynamics of spiking neurons but also to the additional classes of parameters present in SNNs. DNNs usuall y optimise linear w eights and bias parameters, all of which share a common scale and which can adopt uncons trained finite values. SNNs, on the other hand, contain v ar i- ous time-constant parameters of v ar ious f or mulations, which can onl y v alidl y adopt a con- strained rang e of v alues. F or e xample, time constants in the f or m of synaptic and membrane 𝜏 ‘ 𝑠𝑚𝑢𝑠𝑡𝑏𝑒𝑝𝑜𝑠𝑖𝑡𝑖𝑣 𝑒.𝐷 𝑒𝑐𝑎𝑦 𝑓 𝑜𝑟 𝑚𝑢𝑙 𝑎𝑡𝑖𝑜𝑛𝑠𝑓 𝑜𝑟 𝑠𝑦 𝑛𝑎𝑝𝑠𝑒𝑎𝑛𝑑𝑚𝑒𝑚𝑏𝑟 𝑎𝑛𝑒𝑡𝑖𝑚𝑒𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡𝑠𝑚𝑢𝑠𝑡𝑟 𝑎𝑛𝑔 𝑒 “(0 , 1)‘ . Fir - ing thresholds are usuall y also s tr ictl y positiv e v alues. Because of this need, R ockpool pro vides con v enient wa y s to access individual classes of parameters in a comple x netw ork via the Module.parameters() inter face and a set of tools f or easil y configur ing and imposing boundar y constraints during optimisation. This Ho w T o guide sho ws y ou ho w to use the training.torch_loss and training.jax_loss packag es and the f eatures of the utilities.tree_utils mini-librar y to set up constrained optimisation problems. [1]: # - Make sure additional required packages are installed import sys !{ sys.executable } -m pip install --quiet rich torch jax optax from rich import print import matplotlib.pyplot as plt plt . rcParams[ ' figure.figsize ' ] = [ 12 , 4 ] plt . rcParams[ ' figure.dpi ' ] = 300 import numpy as np import torch , jax , optax [ notice ] A new release of pip available: 22.2.2 -> 23.0.1 [ notice ] To update, run: pip install --upgrade pip /Users/Shared/anaconda3/envs/py38/lib/python3.8/site-packages/chex/_src/pytypes.py:37: ␣ ˓ → FutureWarning: jax.tree_structure is deprecated, and will be removed in a future ␣ ˓ → release. Use jax.tree_util.tree_structure instead. PyTreeDef = type(jax.tree_structure(None)) 123 Rockpool, Release 3.0.3 16.1 torch inter face for constrained optimization R ockpool supports both torch and jax optimisation back ends, with a common API f or setting up constrained optimi- sation. Here, w e demonstrate the torch inter f ace to set parameter constraints f or a single LIF module. [2]: # - Import the LIF module we will use from rockpool.nn.modules import LIFTorch # - Create a single LIF module net = LIFTorch( 1 ) print ( ' Module: ' , net) print ( ' Parameters: ' , net . parameters()) Module: LIFTorch with shape ( 1 , 1 ) Parameters: { ' tau_mem ' : Parameter containing: tensor ([ 0.0200 ] , requires_grad = True ) , ' tau_syn ' : Parameter containing: tensor ([[ 0.0200 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0 . ] , requires_grad = True ) , ' threshold ' : Parameter containing: tensor ([ 1 . ] , requires_grad = True ) } Ev en this single spiking neuron has f our parameters — tw o time constants f or synapse and membrane ( tau_syn and tau_mem ), which mus t be positiv e; a bias parameter , which can adopt an y value; and a threshold parameter which should also be positiv e. Suppose either time constant becomes neg ativ e during training. In that case, the dynamics of the module will be undefined and most lik el y unstable, leading to a breakdo wn of both netw ork dynamics and training. R ockpool pro vides a cos t function bounds_cost() , which imposes bounded parameter cons traints. W e also pro vide a helper function make_bounds() , which helps y ou build specifications f or which parameters should be cons trained and ho w . Belo w w e sho w ho w the cost function beha v es as a parameter approac hes and violates a constraint (0, 1). [3]: # - Import the ` make_bounds ` and ` bounds_cost ` helper functions from rockpool.training.torch_loss import make_bounds, bounds_cost xs = np . linspace( - 1 , 2 , 1001 ) cost = [bounds_cost({ ' x ' : torch . tensor(x)}, { ' x ' : 0. }, { ' x ' : 1. }) for x in xs] plt . figure() plt . plot(xs, cost) plt . plot([ 0 , 0 ], [ 0 , 3 ], ' r: ' ) plt . plot([ 1 , 1 ], [ 0 , 3 ], ' r: ' ) plt . xlabel( ' Parameter value ' ) plt . ylabel( ' Cost ' ); 124 Chap ter 16. How T o: Configure and per form constrained optimization in Rockpool Rockpool, Release 3.0.3 When no bounds are violated, the cost e v aluates to zero. A t the bounds, a cost of 1 is imposed, which increases f or increasing violations. N o w let ’ s see ho w to create and appl y bounds to the parameters of a LIF module. The training.torch_loss.make_bounds() function tak es the parameters of a R ockpool netw ork and g enerates lo w er - and upper -bounds configuration dictionaries. These dictionar ies mimic the structure of the netw ork parameters. [4]: # - Call ` make_bounds ` on the parameters of the module lb, ub = make_bounds(net . parameters()) print (lb, ub) { ' tau_mem ' : -inf, ' tau_syn ' : -inf, ' bias ' : -inf, ' threshold ' : -inf } { ' tau_mem ' : inf, ' tau_syn ' : inf, ' bias ' : inf, ' threshold ' : inf } By def ault, no parameters are constrained — the lo w er and upper bounds are set to neg ativ e and positiv e infinity , respectiv el y . W e set bounds b y changing the v alues to finite lo w er and upper bounds. Let ’ s use (0ms, 200ms) as the constraints f or tau_mem . [5]: lb[ ' tau_syn ' ] = 0. ub[ ' tau_syn ' ] = 200e-3 print (lb, ub) { ' tau_mem ' : -inf, ' tau_syn ' : 0.0 , ' bias ' : -inf, ' threshold ' : -inf } { ' tau_mem ' : inf, ' tau_syn ' : 0.2 , ' bias ' : inf, ' threshold ' : inf } [6]: # - Evaluate the boundary constraint cost print (bounds_cost(net . parameters(), lb, ub)) tensor ( 0 ., grad_fn = < SumBackward0 >) In this case, no bounds are violated, so the cost is zero. N o w let ’ s look at an e xample of a comple x netw ork with man y la y ers and module nes ting. W e’ll define the netw ork to use tw o different classes of leak parameters, needing different constraints on eac h class. [7]: from rockpool.nn.modules import LinearTorch from rockpool.nn.combinators import Sequential, Residual (continues on ne xt page) 16.1. torch interface for constrained optimization 125 Rockpool, Release 3.0.3 (continued from pre vious pag e) net = Sequential( LinearTorch(( 2 , 3 )), LIFTorch( 3 , leak_mode = "decays" ), Residual( LinearTorch(( 3 , 3 )), LIFTorch( 3 , leak_mode = "decays" ), ), LinearTorch(( 3 , 5 )), LIFTorch( 5 , leak_mode = "taus" ), ) print ( ' Network: ' , net) print ( ' Parameters: ' , net . parameters()) Network: TorchSequential with shape ( 2 , 5 ) { LinearTorch ' 0_LinearTorch ' with shape ( 2 , 3 ) LIFTorch ' 1_LIFTorch ' with shape ( 3 , 3 ) TorchResidual ' 2_TorchResidual ' with shape ( 3 , 3 ) { LinearTorch ' 0_LinearTorch ' with shape ( 3 , 3 ) LIFTorch ' 1_LIFTorch ' with shape ( 3 , 3 ) } LinearTorch ' 3_LinearTorch ' with shape ( 3 , 5 ) LIFTorch ' 4_LIFTorch ' with shape ( 5 , 5 ) } Parameters: { ' 0_LinearTorch ' : { ' weight ' : Parameter containing: tensor ([[ -1.7288 , 0.3565 , 0.7264 ] , [ -0.9882 , -1.2147 , 1.2812 ]] , requires_grad = True ) } , ' 1_LIFTorch ' : { ' alpha ' : Parameter containing: tensor ([ 0.5000 , 0.5000 , 0.5000 ] , requires_grad = True ) , ' beta ' : Parameter containing: tensor ([[ 0.5000 ] , [ 0.5000 ] , [ 0.5000 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0 ., 0 ., 0 . ] , requires_grad = True ) , ' threshold ' : Parameter containing: tensor ([ 1 ., 1 ., 1 . ] , requires_grad = True ) } , ' 2_TorchResidual ' : { ' 0_LinearTorch ' : { ' weight ' : Parameter containing: tensor ([[ 0.7141 , -1.3781 , -0.7695 ] , [ -0.8757 , -0.6188 , -0.4058 ] , [ -0.8914 , 0.4774 , 0.1480 ]] , requires_grad = True ) (continues on ne xt page) 126 Chap ter 16. How T o: Configure and per form constrained optimization in Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) } , ' 1_LIFTorch ' : { ' alpha ' : Parameter containing: tensor ([ 0.5000 , 0.5000 , 0.5000 ] , requires_grad = True ) , ' beta ' : Parameter containing: tensor ([[ 0.5000 ] , [ 0.5000 ] , [ 0.5000 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0 ., 0 ., 0 . ] , requires_grad = True ) , ' threshold ' : Parameter containing: tensor ([ 1 ., 1 ., 1 . ] , requires_grad = True ) } } , ' 3_LinearTorch ' : { ' weight ' : Parameter containing: tensor ([[ 0.4936 , 1.3029 , -1.0018 , 1.1015 , -1.4031 ] , [ 0.1428 , 1.2207 , 1.1742 , 0.8693 , -0.3616 ] , [ 0.1114 , -1.3645 , -1.3504 , -0.6766 , -0.4882 ]] , requires_grad = True ) } , ' 4_LIFTorch ' : { ' tau_mem ' : Parameter containing: tensor ([ 0.0200 , 0.0200 , 0.0200 , 0.0200 , 0.0200 ] , requires_grad = True ) , ' tau_syn ' : Parameter containing: tensor ([[ 0.0200 ] , [ 0.0200 ] , [ 0.0200 ] , [ 0.0200 ] , [ 0.0200 ]] , requires_grad = True ) , ' bias ' : Parameter containing: tensor ([ 0 ., 0 ., 0 ., 0 ., 0 . ] , requires_grad = True ) , ' threshold ' : Parameter containing: tensor ([ 1 ., 1 ., 1 ., 1 ., 1 . ] , requires_grad = True ) } } This is a deepl y nes ted netw ork with a comple x set of parameters. Luc kily , R oc kpool pro vides se v eral con v enient tools that mak e it easy to build constraints e v en f or comple x netw orks. The Module.parameters() method allo w s y ou to easil y e xtract families of parameters, helping y ou identify all time constants, f or e xample. The Module.attributes_named() method allo ws y ou to specify par ticular named parame- ters. The mini-librar y tree_utils helps y ou easil y manipulate the parameter and cons traint dictionar ies to set chosen bounds. Here w e ’ll use the tree_utils.set_matching() function to set bounds f or chosen parameter sets. [8]: # - Import the tree utilities library import rockpool.utilities.tree_utils as tu # - Make template lower and upper bounds lb, ub = make_bounds(net . parameters()) (continues on ne xt page) 16.1. torch interface for constrained optimization 127 Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Set lower bounds on "decays" and "taus" family parameters lb = tu . set_matching(lb, net . parameters( ' decays ' ), 0.5 ) lb = tu . set_matching(lb, net . parameters( ' taus ' ), 0. ) # - Set upper bounds on "decays" family parameters ub = tu . set_matching(ub, net . parameters( ' decays ' ), 1. ) # - Set an upper bound on a specific parameter name ub = tu . set_matching(ub, net . attributes_named( ' tau_syn ' ), 500e-3 ) print (lb, ub) { ' 0_LinearTorch ' : { ' weight ' : -inf } , ' 1_LIFTorch ' : { ' alpha ' : 0.5 , ' beta ' : 0.5 , ' bias ' : -inf, ' threshold ' : -inf } , ' 2_TorchResidual ' : { ' 0_LinearTorch ' : { ' weight ' : -inf } , ' 1_LIFTorch ' : { ' alpha ' : 0.5 , ' beta ' : 0.5 , ' bias ' : -inf, ' threshold ' : -inf } } , ' 3_LinearTorch ' : { ' weight ' : -inf } , ' 4_LIFTorch ' : { ' tau_mem ' : 0.0 , ' tau_syn ' : 0.0 , ' bias ' : -inf, ' threshold ' : -inf } } { ' 0_LinearTorch ' : { ' weight ' : inf } , ' 1_LIFTorch ' : { ' alpha ' : 1.0 , ' beta ' : 1.0 , ' bias ' : inf, ' threshold ' : inf } , ' 2_TorchResidual ' : { ' 0_LinearTorch ' : { ' weight ' : inf } , ' 1_LIFTorch ' : { ' alpha ' : 1.0 , ' beta ' : 1.0 , ' bias ' : inf, ' threshold ' : inf } } , ' 3_LinearTorch ' : { ' weight ' : inf } , ' 4_LIFTorch ' : { ' tau_mem ' : inf, ' tau_syn ' : 0.5 , ' bias ' : inf, ' threshold ' : inf } } [9]: # - Evaluate the boundary constraint cost for the full set of network parameters print (bounds_cost(net . parameters(), lb, ub)) tensor ( 0 ., grad_fn = < SumBackward0 >) Defining and e v aluating boundar y losses f or cons trained optimisation is made simple, e v en f or comple x netw orks! Imposing the constraints is as simple as including torch_loss.bounds_cost() as a f actor of the loss function dur ing training, as in the e xample belo w . [10]: from torch.optim import Adam from torch.nn import CrossEntropyLoss # - Initialise the optimiser optimizer = Adam(net . parameters() . astorch(), lr = 1e-3 ) func_loss = CrossEntropyLoss() # - Dummy dataset dataset = [(torch . tensor(np . random . rand( 1 , 1 , 2 ), dtype = torch . float), torch . tensor(np . (continues on ne xt page) 128 Chap ter 16. How T o: Configure and per form constrained optimization in Rockpool Rockpool, Release 3.0.3 (continued from pre vious pag e) ˓ → random . rand( 1 , 1 , 5 ), dtype = torch . float))] # - Optimiser loop over dataset for input , target in dataset: optimizer . zero_grad() output, _, _ = net( input ) # - Evaluate the functional and constraints losses loss = func_loss(output, target) + bounds_cost(net . parameters(), lb, ub) # - Perform the backward step loss . backward() optimizer . step() 16.2 jax inter face for constrained optimization The jax inter f ace f or constrained optimisation is identical to the torch interface. Here w e demonstrate a similar constrained optimisation problem as abo v e. [11]: # - Import the Rockpool NN modules from rockpool.nn.modules import LIFJax, LinearJax from rockpool.nn.combinators import Sequential # - Import tools from `` jax_loss `` instead of `` torch_loss `` from rockpool.training.jax_loss import make_bounds, bounds_cost # - Import the tree utility package from rockpool.utilities import tree_utils as tu [12]: # - Set up a simple network net = Sequential( LinearJax(( 2 , 3 )), LIFJax( 3 ), LinearJax(( 3 , 5 )), LIFJax( 5 ), ) print ( ' Network: ' , net) print ( ' Parameters: ' , net . parameters()) Network: JaxSequential with shape ( 2 , 5 ) { LinearJax ' 0_LinearJax ' with shape ( 2 , 3 ) LIFJax ' 1_LIFJax ' with shape ( 3 , 3 ) LinearJax ' 2_LinearJax ' with shape ( 3 , 5 ) LIFJax ' 3_LIFJax ' with shape ( 5 , 5 ) } Parameters: { ' 0_LinearJax ' : { ' weight ' : array ([[ 1.45853284 , -1.09957019 , -0.31666267 ] , [ 0.63834515 , 1.47955107 , 1.03358989 ]]) (continues on ne xt page) 16.2. jax interface for constrained optimization 129 Rockpool, Release 3.0.3 (continued from pre vious pag e) } , ' 1_LIFJax ' : { ' tau_mem ' : DeviceArray ([ 0.02 , 0.02 , 0.02 ] , dtype = float32 ) , ' tau_syn ' : DeviceArray ([[ 0.02 ] , [ 0.02 ] , [ 0.02 ]] , dtype = float32 ) , ' bias ' : DeviceArray ([ 0 ., 0 ., 0 . ] , dtype = float32 ) , ' threshold ' : DeviceArray ([ 1 ., 1 ., 1 . ] , dtype = float32 ) } , ' 2_LinearJax ' : { ' weight ' : array ([[ 0.3648217 , 0.34105733 , 1.23947428 , -0.42756732 , 0.6361447 ␣ ˓ → ] , [ -0.19837451 , 0.61290813 , 1.25214626 , 1.206278 , 0.70346237 ] , [ 0.73964399 , -1.02753273 , -0.28541291 , -1.10618743 , 0.78135608 ]]) } , ' 3_LIFJax ' : { ' tau_mem ' : DeviceArray ([ 0.02 , 0.02 , 0.02 , 0.02 , 0.02 ] , dtype = float32 ) , ' tau_syn ' : DeviceArray ([[ 0.02 ] , [ 0.02 ] , [ 0.02 ] , [ 0.02 ] , [ 0.02 ]] , dtype = float32 ) , ' bias ' : DeviceArray ([ 0 ., 0 ., 0 ., 0 ., 0 . ] , dtype = float32 ) , ' threshold ' : DeviceArray ([ 1 ., 1 ., 1 ., 1 ., 1 . ] , dtype = float32 ) } } W e ag ain use training.jax_loss.make_bounds() to build a template configuration f or constrained optimisation. W e use the tree handling librar y and Module.parameters() , to set lo w er -bounds cons traints on time constants. [13]: # - Build a template configuration lb, ub = make_bounds(net . parameters()) print (lb, ub) { ' 0_LinearJax ' : { ' weight ' : -inf } , ' 1_LIFJax ' : { ' bias ' : -inf, ' tau_mem ' : -inf, ' tau_syn ' : -inf, ' threshold ' : -inf } , ' 2_LinearJax ' : { ' weight ' : -inf } , ' 3_LIFJax ' : { ' bias ' : -inf, ' tau_mem ' : -inf, ' tau_syn ' : -inf, ' threshold ' : -inf } } { ' 0_LinearJax ' : { ' weight ' : inf } , ' 1_LIFJax ' : { ' bias ' : inf, ' tau_mem ' : inf, ' tau_syn ' : inf, ' threshold ' : inf } , ' 2_LinearJax ' : { ' weight ' : inf } , ' 3_LIFJax ' : { ' bias ' : inf, ' tau_mem ' : inf, ' tau_syn ' : inf, ' threshold ' : inf } } [14]: # - Set lower bounds for time constants lb = tu . set_matching(lb, net . parameters( ' taus ' ), 0. ) print (lb) 130 Chap ter 16. How T o: Configure and per form constrained optimization in Rockpool Rockpool, Release 3.0.3 { ' 0_LinearJax ' : { ' weight ' : -inf } , ' 1_LIFJax ' : { ' bias ' : -inf, ' tau_mem ' : 0.0 , ' tau_syn ' : 0.0 , ' threshold ' : -inf } , ' 2_LinearJax ' : { ' weight ' : -inf } , ' 3_LIFJax ' : { ' bias ' : -inf, ' tau_mem ' : 0.0 , ' tau_syn ' : 0.0 , ' threshold ' : -inf } } [15]: # - Evaluate the boundary constraint cost print (bounds_cost(net . parameters(), lb, ub)) 0.0 Theref ore, the R ockpool-pro vided inter f ace f or setting bounds is almost identical betw een torch and jax . Belo w w e sho w a v er y simple jax optimisation loop that incor porates the boundar y constraints during optimisation. [16]: # - Initialise the Adam optimiser with the initial network parameters optimizer = optax . adam( 1e-4 ) params = net . parameters() opt_state = optimizer . init(params) # - Use an MSE loss func_loss = lambda o, t: jax . numpy . mean((o - t) ** 2 ) # - Network evaluation and loss function def eval_loss (params, net, input , target): output, _, _ = net( input ) loss = func_loss(output, target) + bounds_cost(params, lb, ub) return loss # - Dummy dataset dataset = [(np . random . rand( 1 , 1 , 2 ), np . random . rand( 1 , 1 , 5 ))] # - Loop over dataset, evaluating loss and applying updates for input , target in dataset: loss_value, grads = jax . value_and_grad(eval_loss)(params, net, input , target) updates, opt_state = optimizer . update(grads, opt_state, params) params = optax . apply_updates(params, updates) 16.3 N e xt steps See T r aining a R oc kpool netw ork with Jax f or a jax training e xample that includes constraints. 16.3. N ext steps 131 Rockpool, Release 3.0.3 (continued from pre vious pag e) for _ in tqdm( range (num_epochs)): # - Get parameters for this iteration params = get_params(opt_state) # - Get the loss value and gradients for this iteration loss_val, grads = loss_vgf(params, modRNN . module, input_t, target_t) # - Update the optimiser opt_state = update_fun( next (i_trial), grads, opt_state) # - Keep track of the loss loss_t . append(loss_val) 0%| | 0/6000 [00:00<?, ?it/s] [9]: # - Plot the loss curve over training plt . plot(loss_t) plt . yscale( "log" ) plt . xlabel( "Iteration" ) plt . ylabel( "Loss" ) plt . title( "Training progress" ); Ok, the loss has decreased to a lo w v alue. Let ’ s see what the netw ork has lear ned! [10]: # - Plot the output of the trained reservoir # - Simulate with trained parameters modRNN . _module = modRNN . _module . set_attributes(get_params(opt_state)) modRNN . reset_all() ts_output, _, record_dict = modRNN(ts_input) # - Compare the output to the target ts_output . plot() ts_target . plot(ts_output . times, ls = "--" , lw = 2 ) plt . legend() plt . title( "Output vs target" ); 138 Chapter 17. Gradient descent tr aining of a rate-based recurrent network Rockpool, Release 3.0.3 If all has gone according to plan, the output of the reser v oir should closel y match the tar get signal. W e can see the effect of training b y e xamining the dis tr ibution of netw ork parameters belo w . [11]: TSContinuous . from_clocked( record_dict[ "1_RateJax" ][ "x" ][ 0 ], dt = dt, name = "Neuron state" ) . plot(skip = 20 , stagger = 1 ); [12]: # - Plot the network time constants plt . figure() plt . hist(modRNN . _module[ 1 ] . tau / 1e-3 , 21 ) plt . legend([ "Trained time constants" ]) plt . xlabel( "Time constant (ms)" ) plt . ylabel( "Count" ) plt . title( "Distribution of time constants" ); 17.3. N etwork model 139 Rockpool, Release 3.0.3 [13]: # - Plot the recurrent layer biases plt . figure() plt . stem(modRNN . _module[ 1 ] . bias) plt . title( "Distribution of trained biases" ) plt . xlabel( "Unit" ) plt . ylabel( "Bias" ); W e can e xamine something of the computational proper ties of the netw ork by finding the eig enspectr um of the Jacobian of the recur rent la y er . The Jacobian ˆ 𝐽 is giv en b y ˆ 𝐽 = ( ˆ 𝑊 𝑟 − 𝐼 ) ./ ˆ 𝑇 where 𝐼 is the identity matr ix, ./ denotes element-wise division, and 𝑇 is the matr ix composed of all time constants ˆ 𝜏 of the recur rent la y er . Belo w w e plot the eig en v alues 𝜆 of 𝐽 . In m y training result, sev eral comple x eig en values 𝜆 with real parts g reater than zero are present in the trained eig enspectr um. These cor respond to oscillator y modes, which are ob viously useful in g enerating the c hir p output. [14]: # - Plot the recurrent layer eigenspectrum J = modRNN . _module[ 1 ] . w_rec - np . identity(nResSize) J = J / modRNN . _module[ 1 ] . tau (continues on ne xt page) 140 Chapter 17. Gradient descent tr aining of a rate-based recurrent network Rockpool, Release 3.0.3 (continued from pre vious pag e) J0 = w_rec0 - np . identity(nResSize) J0 = J0 / tau0 plt . figure() eigs = np . linalg . eigvals(J) eigs0 = np . linalg . eigvals(J0) plt . plot(np . real(eigs0), np . imag(eigs0), "." ) plt . plot(np . real(eigs), np . imag(eigs), "o" ) plt . plot([ 0 , 0 ], [ - 100 , 100 ], "--" ) plt . legend( ( "Initial eigenspectrum" , "Trained eigenspectrum" , ) ) plt . xlim([ - 350 , 250 ]) plt . ylim([ - 100 , 100 ]) plt . title( "Eigenspectrum of recurrent weights" ) plt . xlabel( "Re($\lambda$)" ) plt . ylabel( "Im($\lambda$)" ); The Kernel crashed while executing code in the the current cell or a previous cell. ␣ ˓ → Please review the code in the cell(s) to identify a possible cause of the failure. ␣ ˓ → Click <a href= ' https://aka.ms/vscodeJupyterKernelCrash ' >here</a> for more info. View ␣ ˓ → Jupyter <a href= ' command:jupyter.viewOutput ' >log</a> for further details. 17.4 Summar y Gradient descent training does a good job of optimmising a dynamic recur rent netw ork f or a difficult task requir ing significant temporal memor y . jax pro vides a computationally efficient bac k -end as well as automatic differentiation of the recur rent la y er . The combination in R ockpool allo w s us to optimise not jus t the w eights and biases of a netw ork, but also to adapt the neuron dynamics to a desired task. 17.4. Summar y 141 Rockpool, Release 3.0.3 142 Chapter 17. Gradient descent tr aining of a rate-based recurrent network CHAPTER EIGHTEEN TRAINING A SPIKIN G NET W ORK WITH J AX This tutor ial demonstrates using R oc kpool and a Jax -accelerated LIF f eed-f or ward neuron la y er to per f or m g radient descent training of all netw ork parameters. The result is a trained spiking la y er whic h can g enerate a pre-defined signal from a noisy spiking input. 18.1 Requirements and housekeeping This e xample requires the R oc kpool pac kag e from SynSense, as well as jax and its dependencies. [1]: import jax [2]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) # - Rockpool imports from rockpool import TSEvent, TSContinuous from rockpool.nn.modules import LIFJax, LinearJax, ExpSynJax from rockpool.nn.modules.jax.jax_lif_ode import LIFODEJax from rockpool.nn.combinators import Sequential from rockpool.parameters import Constant # - Typing from typing import Callable, Dict, Tuple import types # - Numpy import numpy as np import copy # - Pretty printing try : from rich import print except : pass # TQDM from tqdm .autonotebook import tqdm (continues on ne xt page) 143 Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Plotting imports and config import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 18.2 Signal gener ation from frozen noise task W e will use a single f eed-f or ward la y er of spiking neurons to con v er t a chosen patter n of random in put spik es o v er time, into a pre-defined temporal signal with comple x dynamics. The netw ork architecture is s trictly f eedf or w ard, but the spiking neurons nev er theless contain temporal dynamics in their synaptic and membrane signals, with e xplicit time constants. Some number of input c hannels Nin will contain independent Poisson spik es at some rate spiking_prob/dt . A single output channel should g enerate a chirp signal with increasing frequency , up to a maximum of chirp_freq_factor . Y ou can pla y with these parameters belo w . [3]: # - Define input and target Nin = 200 dt = 1e-3 chirp_freq_factor = 10 dur_input = 1000e-3 # - Generate a time base T = int (np . round(dur_input / dt)) timebase = np . linspace( 0 ,( T - 1 ) * dt, T) # - Generate a chirp signal as a target chirp = np . atleast_2d(np . sin(timebase * 2 * np . pi * (timebase * chirp_freq_factor))) . T target_ts = TSContinuous(timebase, chirp, periodic = True , name = "Target chirp" ) # - Generate a Poisson frozen random spike train spiking_prob = 0.01 input_sp_raster = np . random . rand(T, Nin) < spiking_prob input_sp_ts = TSEvent . from_raster( input_sp_raster, name = "Input spikes" , periodic = True , dt = dt ) # - Plot the input and target signals plt . figure() input_sp_ts . plot(s = 4 ) (target_ts * Nin / 2 + Nin / 2 ) . plot(color = "orange" , lw = 2 ) plt . legend() plt . title( "Input and target" ); 144 Chapt er 18. T raining a spiking network with Jax Rockpool, Release 3.0.3 18.3 LIF neuron The spiking neuron w e will use is a leaky integ rate-and-fire spiking neuron (“LIF” neuron). This neuron rece vies input spik e trains 𝑆 𝑖𝑛 ( 𝑡 ) = ∑︀ 𝑗 𝛿 ( 𝑡 − 𝑡 𝑗 ) , which are integrated via w eighted e xponential synapses. Synaptic cur rents are then integrated into a neuron state (“membrane potential”) 𝑉 𝑚𝑒𝑚 . The neuron obe ys the dynamics 𝜏 𝑚𝑒𝑚 · ˙ 𝑉 𝑚𝑒𝑚 + 𝑉 𝑚𝑒𝑚 = 𝐼 𝑠𝑦 𝑛 + 𝐼 𝑏𝑖𝑎𝑠 + 𝜎 𝜁 ( 𝑡 ) 𝜏 𝑠𝑦 𝑛 · ˙ 𝐼 𝑠𝑦 𝑛 + 𝐼 𝑠𝑦 𝑛 = 0 𝐼 𝑠𝑦 𝑛 + = 𝑊 𝑖𝑛 · 𝑆 𝑖𝑛 ( 𝑡 ) Where 𝜏 𝑚𝑒𝑚 and 𝜏 𝑠𝑦 𝑛 are membrane and synaptic time constants; 𝐼 𝑏𝑖𝑎𝑠 is a constant bias current f or each neuron; 𝜎 𝜁 ( 𝑡 ) is a white noise process with std. de v . 𝜎 . Output spik es are g enerated when 𝑉 𝑚𝑒𝑚 crosses the firing threshold 𝑉 𝑡ℎ = 0 . This process g enerates a spik e train 𝑆 ( 𝑡 ) as a ser ies of delta functions, and causes a subtractiv e reset of 𝑉 𝑚𝑒𝑚 : 𝑉 𝑚𝑒𝑚 > 𝑉 𝑡ℎ → 𝑆 ( 𝑡 ) = 𝐻 ( 𝑉 𝑚𝑒𝑚 ( 𝑡 )) , 𝑉 𝑚𝑒𝑚 = 𝑉 𝑚𝑒𝑚 − 1 The analog output signal is g enerated using a sur rog ate 𝑈 ( 𝑡 ) = tanh( 𝑥 + 1) / 2+0 . 5 The output of the netw ork 𝑜 ( 𝑡 ) is theref ore giv en b y 𝑜 ( 𝑡 ) = 𝑊 𝑜𝑢𝑡 · 𝑆 ( 𝑡 ) F or more detail, see the documentation f or the Jax module LIFJax . 18.4 Build a netw ork The netw ork architecture is a single f eedf or w ard la y er , with w eighted spiking in puts and outputs. Spiking is g enerated via a function that pro vides a sur rogate gradient in the bac kw ards pass. This per mits propagation of an error g radient through the la y er , making gradient-descent training possible. F or this regression task w e will also use an e xponential synapse la y er to per fpr m temporal smoothing of the output. R egressing to a smooth signal is much easier with a continuous output signal, than using the spik e deltas alone. 18.3. LIF neuron 145 Rockpool, Release 3.0.3 [4]: # - Network size N = 50 Nout = 1 input_scale = 1. [5]: # - Generate a network using the sequential combinator modFFwd = Sequential( LinearJax((Nin, N)), LIFJax(N, dt = dt), ExpSynJax(N), LinearJax((N, Nout)), ) print (modFFwd) JaxSequential with shape ( 200 , 1 ) { LinearJax ' 0_LinearJax ' with shape ( 200 , 50 ) LIFJax ' 1_LIFJax ' with shape ( 50 , 50 ) ExpSynJax ' 2_ExpSynJax ' with shape ( 50 , ) LinearJax ' 3_LinearJax ' with shape ( 50 , 1 ) } 18.5 Simulate initial state of netw ork If w e simulate the untrained netw ork with our random input spik es, w e don ’ t e xpect an ything sensible to come out. Let ’ s do this, and take a look at ho w the netw ork beha v es. [6]: # - Randomise the network state modFFwd . reset_state() # - Evolve with the frozen noise spiking input tsOutput, new_state, record_dict = modFFwd(input_sp_raster * input_scale, record = True ) # - Plot the analog output plt . figure() plt . plot(tsOutput[ 0 ]) [6]: [<matplotlib.lines.Line2D at 0x3186ab730>] 146 Chapt er 18. T raining a spiking network with Jax Rockpool, Release 3.0.3 W e can also e xamine the inter nal s tate of the netw ork, b y inter rogating record_dict : [7]: # - Make a function that converts `` record_dict `` def plot_record_dict (rd): Isyn_ts = TSContinuous . from_clocked( rd[ "1_LIFJax" ][ "isyn" ][ 0 ,: ,: , 0 ], dt, name = "Synaptic currents $I_ {syn} $" ) Vmem_ts = TSContinuous . from_clocked( rd[ "1_LIFJax" ][ "vmem" ][ 0 ], dt, name = "Membrane potential $V_ {mem} $" ) spikes_ts = TSEvent . from_raster( rd[ "1_LIFJax" ][ "spikes" ][ 0 ], dt, name = "LIF layer spikes" ) # - Plot the internal activity of selected neurons plt . figure() Isyn_ts . plot(stagger = 1.1 , skip = 5 ) plt . figure() Vmem_ts . plot(stagger = 1.1 , skip = 5 ) plt . figure() spikes_ts . plot(s = 4 ) plot_record_dict(record_dict) 18.5. Simulate initial state of network 147 Rockpool, Release 3.0.3 [6]: # - Scale down recurrent weights for stability net[ 2 ][ 1 ] . w_rec . data = net[ 2 ][ 1 ] . w_rec / 10. 27.3 Step 2: Extract the com putational gr aph for the ne tw ork T o obtain a g raph descr ibing the entire netw ork, whic h contains the computational flo w of inf or mation through the netw ork as w ell as all parameters, w e simply use the as_graph() method. F or more inf or mation, see Computational g r aphs in R oc kpool . [7]: print (net . as_graph()) GraphHolder "TorchSequential__11277641184" with 2 input nodes -> 2 output nodes 27.4 Step 3: Map the network t o a hardware specification W e no w need to chec k that the netw ork can be suppor ted b y the X y lo hardw are, and assign hardware resources to the v arious aspects of the netw ork architecture. F or e xample, each neuron in the netw ork mus t be assigned to a hardw are neuron. Each non-input w eight element in the netw ork must be assigned to a global hidden w eight matr ix f or X y lo. Output neurons in the final la y er mus t be assigned to output channels. The X y lo f amily includes se vral devices with differing HW blocks and suppor t. These are suppor ted b y independent subpac kag es under rockpool.devices.xylo , and named after the chip ID in y our HDK. R ockpool can detect this automaticall y f or y ou, and impor t the cor rect packag e, b y connecting to the HDK. If y ou do not ha v e a X y lo HDK, then y ou can use the rockpool.devices.xylo.syns61201 pac kag e suppor ting X y lo-A udio 2. [8]: # - Import the Xylo HDK detection function from rockpool.devices.xylo import find_xylo_hdks # - Detect a connected HDK and import the required support package connected_hdks, support_modules, chip_versions = find_xylo_hdks() found_xylo = len (connected_hdks) > 0 if found_xylo: hdk = connected_hdks[ 0 ] x = support_modules[ 0 ] else : assert False , ' This tutorial requires a connected Xylo HDK to run. ' The connected Xylo HDK contains a Xylo Audio v2 (SYNS61201). Importing ` rockpool.devices. ˓ → xylo.syns61201 ` T o conv er t the computational graph to a X ylo specfication, w e use the mapper() function. In order to retain floating- point representations f or parameters, w e can specifiy weight_dtype = float . See the documentation f or mapper() f or fur ther details. [9]: # - Call the Xylo mapper on the extracted computational graph spec = x . mapper(net . as_graph(), weight_dtype = ' float ' ) 250 Chapter 27. Quick-start with X ylo ™ SNN core Rockpool, Release 3.0.3 [10]: print (spec) { ' mapped_graph ' : GraphHolder "TorchSequential__11277641184" with 2 input nodes -> 2 ␣ ˓ → output nodes, ' weights_in ' : array ([[ -0.77903908 , 1.18929613 , 1.16114461 , 0.22493351 , 0 . ␣ ˓ → , 0 . , 0 . , 0 . ] , [ 1.32337248 , -1.16171956 , -1.33154929 , 0.3293041 , 0 . , 0 . , 0 . , 0 . ]]) , ' weights_out ' : array ([[ 0 . , 0 . ] , [ 0 . , 0 . ] , [ 0 . , 0 . ] , [ 0 . , 0 . ] , [ -0.84590757 , -1.19352996 ] , [ -0.10860074 , 0.6373018 ] , [ -0.16565597 , -0.92963493 ] , [ -0.2262736 , 0.42454898 ]]) , ' weights_rec ' : array ([[ 0 . , 0 . , 0 . , 0 . , 0. ˓ → 10326695 , -0.14892304 , 1.01041162 , -0.20021439 ] , [ 0 . , 0 . , 0 . , 0 . , 0.03674126 , -0.49414289 , -0.6105172 , 0.60044777 ] , [ 0 . , 0 . , 0 . , 0 . , 0.34814227 , -1.09550583 , 0.31232011 , -0.12054205 ] , [ 0 . , 0 . , 0 . , 0 . , -0.83039075 , 0.96797431 , -0.54736292 , 0.51089108 ] , [ 0 . , 0 . , 0 . , 0 . , -0.08836855 , 0.05024868 , 0.09914371 , 0.0179751 ] , [ 0 . , 0 . , 0 . , 0 . , -0.04342629 , 0.07340087 , 0.05501813 , 0.11716658 ] , [ 0 . , 0 . , 0 . , 0 . , 0.0911395 , 0.1181069 , 0.00574297 , 0.06705973 ] , [ 0 . , 0 . , 0 . , 0 . , -0.03105988 , -0.00630997 , 0.07385659 , -0.03389113 ]]) , ' dash_mem ' : array ([ 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 ]) , ' dash_mem_out ' : array ([ 4.32192802 , 4.32192802 ]) , ' dash_syn ' : array ([ 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 , 4.32192802 ]) , ' dash_syn_2 ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' dash_syn_out ' : array ([ 4.32192802 , 4.32192802 ]) , ' threshold ' : array ([ 1 ., 1 ., 1 ., 1 ., 10 ., 10 ., 10 ., 10 . ]) , ' threshold_out ' : array ([ 1 ., 1 . ]) , ' bias ' : array ([ 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 ., 0 . ]) , ' bias_out ' : array ([ 0 ., 0 . ]) , ' weight_shift_in ' : 0 , ' weight_shift_rec ' : 0 , ' weight_shift_out ' : 0 , ' aliases ' : [[ 4 ] , [ 5 ] , [ 6 ] , [ 7 ] , [] , [] , [] , []] , ' dt ' : 0.001 } 27.4. Step 3: Map the netw ork to a hardware specification 251 Rockpool, Release 3.0.3 27.5 Step 4: Quantize the specfication for the X ylo integer logic R ockpool pro vides a number of functions f or quantizing specifications f or X y lo, in the packag e rockpool.transform. quantize_methods . Here w e will use global_quantize() to automatically find a good shared representation of the netw ork parameters, that is compatible with the integ er logic on X y lo. [11]: from rockpool.transform import quantize_methods as q # - Quantize the specification spec . update(q . global_quantize( ** spec)) print (spec) { ' mapped_graph ' : GraphHolder "TorchSequential__11277641184" with 2 input nodes -> 2 ␣ ˓ → output nodes, ' weights_in ' : array ([[ -74 , 113 , 111 , 21 , 0 , 0 , 0 , 0 ] , [ 126 , -111 , -127 , 31 , 0 , 0 , 0 , 0 ]]) , ' weights_out ' : array ([[ 0 , 0 ] , [ 0 , 0 ] , [ 0 , 0 ] , [ 0 , 0 ] , [ -90 , -127 ] , [ -12 , 68 ] , [ -18 , -99 ] , [ -24 , 45 ]]) , ' weights_rec ' : array ([[ 0 , 0 , 0 , 0 , 10 , -14 , 96 , -19 ] , [ 0 , 0 , 0 , 0 , 4 , -47 , -58 , 57 ] , [ 0 , 0 , 0 , 0 , 33 , -104 , 30 , -11 ] , [ 0 , 0 , 0 , 0 , -79 , 92 , -52 , 49 ] , [ 0 , 0 , 0 , 0 , -8 , 5 , 9 , 2 ] , [ 0 , 0 , 0 , 0 , -4 , 7 , 5 , 11 ] , [ 0 , 0 , 0 , 0 , 9 , 11 , 1 , 6 ] , [ 0 , 0 , 0 , 0 , -3 , -1 , 7 , -3 ]]) , ' dash_mem ' : array ([ 4 , 4 , 4 , 4 , 4 , 4 , 4 , 4 ]) , ' dash_mem_out ' : array ([ 4 , 4 ]) , ' dash_syn ' : array ([ 4 , 4 , 4 , 4 , 4 , 4 , 4 , 4 ]) , ' dash_syn_2 ' : array ([ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ]) , ' dash_syn_out ' : array ([ 4 , 4 ]) , ' threshold ' : array ([ 95 , 95 , 95 , 95 , 954 , 954 , 954 , 954 ]) , ' threshold_out ' : array ([ 106 , 106 ]) , ' bias ' : array ([ 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 ]) , ' bias_out ' : array ([ 0 , 0 ]) , ' weight_shift_in ' : 0 , ' weight_shift_rec ' : 0 , ' weight_shift_out ' : 0 , ' aliases ' : [[ 4 ] , [ 5 ] , [ 6 ] , [ 7 ] , [] , [] , [] , []] , ' dt ' : 0.001 } 252 Chapter 27. Quick-start with X ylo ™ SNN core Rockpool, Release 3.0.3 27.6 Step 5: Con v er t the specification to a hardware configuration W e no w con v er t the netw ork specification to a hardw are configuration object f or the X y lo HDK. [12]: # - Use rockpool.devices.xylo.config_from_specification config, is_valid, msg = x . config_from_specification( ** spec) if not is_valid: print (msg) 27.7 Step 6: Deplo y the configuration to the X ylo HDK [13]: # - Use rockpool.devices.xylo.XyloSamna to deploy to the HDK if found_xylo: modSamna = x . XyloSamna(hdk, config, dt = dt) print (modSamna) XyloSamna with shape ( 2 , 8 , 2 ) 27.8 Step 7: Ev olv e the network on the X ylo HDK N o w w e will g enerate a random P oisson input, and e v ol v e the netw ork on the X y lo HDK using this input. Using R ockpool w e can record all inter nal states on the X ylo HDK during e v olution. N ot e that r ecor ding internal stat e slow s ev olution belo w r eal-time. [14]: # - Generate some Poisson input T = 100 f = 0.1 input_spikes = np . random . rand(T, Nin) < f TSEvent . from_raster(input_spikes, dt, name = ' Poisson input events ' ) . plot(); [15]: # - Evolve the network on the Xylo HDK if found_xylo: out, _, r_d = modSamna(input_spikes, record = True ) (continues on ne xt page) 27.6. Step 5: Con v er t the specification to a hardware configuration 253 Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Show the internal state variables recorded print (r_d . keys()) dict_keys ([ ' Vmem ' , ' Isyn ' , ' Isyn2 ' , ' Spikes ' , ' Vmem_out ' , ' Isyn_out ' , ' times ' ]) [16]: # - Plot some internal state variables if found_xylo: plt . figure() plt . imshow(r_d[ ' Spikes ' ] . T, aspect = ' auto ' , origin = ' lower ' ) plt . title( ' Hidden spikes ' ) plt . ylabel( ' Channel ' ) plt . figure() TSContinuous(r_d[ ' times ' ], r_d[ ' Vmem ' ], name = ' Hidden membrane potentials ' ) . ˓ → plot(stagger = 127 ) plt . figure() TSContinuous(r_d[ ' times ' ], r_d[ ' Isyn ' ], name = ' Hidden synaptic currents ' ) . ˓ → plot(stagger = 127 ) 254 Chapter 27. Quick-start with X ylo ™ SNN core Rockpool, Release 3.0.3 27.9 Step 8: Simulate the HDK using a bit-precise simulator R ockpool comes bundled with X y loSim , a bit-precise simulator of the X y lo SNN core. The inter face to e v ol v e and record from a XyloSim object is identical to other R ockpool modules. W e create a XyloSim object using the same hardw are configuration that w e used to deplo y the netw ork to the X y lo HDK. [17]: modSim = x . XyloSim . from_config(config) print (modSim) XyloSim with shape ( 16 , 1000 , 8 ) [18]: # - Evolve the input over the network, in simulation out, _, r_d = modSim(input_spikes, record = True ) # - Show the internal state variables recorded print (r_d . keys()) dict_keys ([ ' Vmem ' , ' Isyn ' , ' Isyn2 ' , ' Spikes ' , ' Vmem_out ' , ' Isyn_out ' ]) [19]: # - Plot some internal state variables plt . figure() plt . imshow(r_d[ ' Spikes ' ] . T, aspect = ' auto ' , origin = ' lower ' ) plt . title( ' Hidden spikes ' ) plt . ylabel( ' Channel ' ) plt . figure() TSContinuous . from_clocked(r_d[ ' Vmem ' ], dt, name = ' Hidden membrane potentials ' ) . ˓ → plot(stagger = 127 ) plt . figure() TSContinuous . from_clocked(r_d[ ' Isyn ' ], dt, name = ' Hidden synaptic currents ' ) . ˓ → plot(stagger = 127 ); 27.9. Step 8: Simulate the HDK using a bit-precise simulator 255 Rockpool, Release 3.0.3 If y ou compare the traces here with those recorded from the X y lo HDK abo v e, y ou will see the y are identical. XyloSim pro vides a w a y to quickl y v er ify a netw ork configuration without requiring the X ylo HDK hardw are. 256 Chapter 27. Quick-start with X ylo ™ SNN core Rockpool, Release 3.0.3 27.10 Summar y The flo w -chart belo w summar ises the s teps in taking a R ockpool netw ork from a high-le v el definition to deplo yment on the X y lo HDK. [20]: Image(filename = ' xylo-pipeline.png ' ) [20]: 27.10. Summar y 257 Rockpool, Release 3.0.3 258 Chapter 27. Quick-start with X ylo ™ SNN core CHAPTER T WENT YEIGHT TRAINING A SPIKIN G NET W ORK T O DEPL O Y T O THE XYL O ™ DIGIT AL SNN [1]: # -- Some useful imports # - Time-series data from rockpool import TSContinuous, TSEvent # - Rich printing try : from rich import print except : pass # - Numpy import numpy as np # - Import and configure matplotlib for plotting import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ "figure.figsize" ] = [ 12 , 4 ] plt . rcParams[ "figure.dpi" ] = 300 28.1 Considerations This notebook sho w s ho w to define a spiking model using the LIFTorch class and train it on a simple task. F or simplicity , the used task is just to spik e at cer tain times giv en random but fix ed in put. In this case w e targ et the X y lo hardw are and hav e to consider the hardw are constraints. I.e. the hardware has to receiv e spiking input, has onl y 16 input c hannels and 1000 recur rentl y connected neurons. W e will use Samna to create and v alidate a model definiton f or X y lo and then use XyloSim to simulate it. No te Both Samna and X y loSim are required f or this tutorial. 259 Rockpool, Release 3.0.3 – 16 bit membrane potential – 16 bit synaptic cur rent – 8 bit w eights – 4 bit bitshift deca y f or membrane and synapses – special: possibl y 2 synapses per neuron • Simulation – Integration method: ∗ F or w ard euler , resolution betw een 1 ms and 10 ms These requirements can differ from X y lo device to de vice. See Ov er view of the X ylo ™ f amily f or more inf or mation. In order to r un our model on X y lo, w e ha v e to fulfill those requirements. The SNN in this e xample already fulfills the connectivtiy and neuron requirements. So the main difficulty is to bring the parameters to the cor rect f ormat and also to quantize the w eights and thresholds to 8 and 16 bits respectiv ely . F or that w e use the g raph representation of our netw ork and use the mapper() function to get the specs f or X y lo. [8]: from rockpool.devices.xylo.syns61201 import mapper spec = mapper(net . as_graph(), weight_dtype = "float" ) print (spec[ "weights_in" ]) [[ -0.35482565 -0.37593043 0.52281165 ... 0 . 0 . 0 . ] [ 0.34481534 0.31385767 -0.11158055 ... 0 . 0 . 0 . ] [ -0.30120176 -0.52312404 0.43335626 ... 0 . 0 . 0 . ] ... [ -0.62413222 0.59378749 0.70466232 ... 0 . 0 . 0 . ] [ -0.26751989 -0.16315432 0.23857698 ... 0 . 0 . 0 . ] [ 0.17672117 -0.08555783 0.45087662 ... 0 . 0 . 0 . ]] [9]: w_inp_float = spec[ "weights_in" ] w_rec_float = spec[ "weights_rec" ] w_out_float = spec[ "weights_out" ] aliases = spec[ "aliases" ] fig = plt . figure(figsize = ( 16 , 3 )) ax = fig . add_subplot( 141 ) ax . set_title( "w_inp float" ) im = ax . imshow(w_inp_float[:, :], aspect = "auto" , interpolation = "None" ) plt . colorbar(im) ax = fig . add_subplot( 142 ) ax . set_title( "w_rec float" ) im = ax . imshow(w_rec_float[:, :], aspect = "auto" , interpolation = "None" ) plt . colorbar(im) (continues on ne xt page) 266 Chapter 28. T raining a spiking network t o deplo y to the X ylo ™ digital SNN Rockpool, Release 3.0.3 (continued from pre vious pag e) ax = fig . add_subplot( 143 ) ax . set_title( "aliases" ) aliases_ = np . zeros_like(w_rec_float[:, :]) for i, a in enumerate (aliases): if len (a) > 0 : aliases_[i, a[ 0 ]] = 1 im = ax . imshow(aliases_, aspect = "auto" , interpolation = "None" ) ax = fig . add_subplot( 144 ) ax . set_title( "w_out float" ) im = ax . imshow(w_out_float, aspect = "auto" , interpolation = "None" ) plt . colorbar(im); As can be seen, the input w eight matr ix has a dimension of (16, 63) and contains the w eights of the firs t linear la y er of the model. The recur rent w eight matrix is nonzero in the top-r ight quar ter , reflecting that the first 63 neurons connect to the ne xt 63 neurons (filled with the w eights on the second linear la y er of the model). N ote that there are no recur rent connections as the diagonal bloc ks are all zero. In the model w e defined a skip connection skipping the hidden la y er . In T orch that ’ s easil y done b y the + operator , adding the activ ations of one la y er to another and hence skipping the la y ers in betw een (the dimensions mus t be compatible). W e g et the same beha vior in X y lo using aliases. If neuron ‘a ’ has alias ‘b’ means that the spike count of ‘b’ is increased b y the spike count of ‘a ’ . In our case, w e aliased the input neurons to the hidden neurons to add up their activ ations. The output w eight matr ix connects the second population to the output, hence there are non-zero values f or the las t 63 neurons to the tw o output neurons. The time-constants f or the e xponential deca y are less e x citing; the y are all 2 ms. But this timeconstant has to be translated to v alues f or bit-shift deca y (dash). The equation f or that is: 𝑑𝑎𝑠ℎ = [log 2 ( 𝜏 /𝑑𝑡 )] Hence, a timeconstant of 2ms with a simulation resolution of 1 ms is 1. But what is done with this bitshift of 1? Let ’ s ha v e an e xample and compare e xponential deca y to bitshift deca y . [10]: tau = 20e-3 # 20 ms dt_ = 1e-3 # 1 ms resolution dash = np . round(np . log2(tau / dt_)) . astype( int ) exp_propagator = np . exp( - dt_ / tau) simtime = 0.2 # 1 sec t_ = 0 (continues on ne xt page) 28.5. Deploying this netw ork to X yloSim 267 Rockpool, Release 3.0.3 (continued from pre vious pag e) v_tau = [ 1000 ] v_dash = [ 1000 ] while t_ < simtime: v_tau . append(v_tau[ - 1 ] * exp_propagator) if (v_dash[ - 1 ] >> dash) > 0 : v_dash . append(v_dash[ - 1 ] - (v_dash[ - 1 ] >> dash)) else : v_dash . append( int (np . clip(v_dash[ - 1 ] - 1 , 0 , np . inf))) t_ += dt_ plt . plot(np . arange( 0 , len (v_tau)) * dt_, v_tau, label = "exponential decay" ) plt . plot(np . arange( 0 , len (v_dash)) * dt_, v_dash, label = "bitshift decay" ) plt . legend() plt . xlabel( "Time (s)" ); As clear l y visible, the bitshift deca y beha v es similar to an e xponential deca y but poses limitations on time-constants and resolution. Ev en w orse, SNNs are v ery sensitiv e on the time-constants and hence e v en those ‘small’ differences can chang e the dynamics in the netw ork drasticall y . BUT bitshifts are e xtremely c heap to compute and put into hardware which is the main reason w e are using them on X y lo. N o w , let ’ s in v estig ate the quantization of w eights. W e use the f ollo wing function: [11]: from rockpool.transform import quantize_methods as q quant_spec = spec . copy() del quant_spec[ "mapped_graph" ] del quant_spec[ "dt" ] quant_spec . update(q . global_quantize( ** quant_spec)) print (quant_spec[ "weights_in" ]) [[ -43 -45 63 ... 000 ] [ 42 38 -13 ... 000 ] [ -36 -63 52 ... 000 ] ... [ -75 72 85 ... 000 ] [ -32 -20 29 ... 000 ] [ 21 -10 54 ... 000 ]] 268 Chapter 28. T raining a spiking network t o deplo y to the X ylo ™ digital SNN Rockpool, Release 3.0.3 N o w . let ’ s visualize the quantized w eight in compar ison to the floating point w eights. [12]: fig = plt . figure(figsize = ( 16 , 10 )) ax = fig . add_subplot( 321 ) ax . set_title( "w_inp float" ) ax . hist(np . ravel(w_inp_float[w_inp_float != 0 ]), bins = 2 ** 8 ) ax = fig . add_subplot( 322 ) ax . set_title( "w_inp quant" ) ax . hist(np . ravel(quant_spec[ "weights_in" ][quant_spec[ "weights_in" ] != 0 ]), bins = 2 ** 8 ) ax = fig . add_subplot( 323 ) ax . set_title( "w_rec float" ) ax . hist(np . ravel(w_rec_float[w_rec_float != 0 ]), bins = 2 ** 8 ) ax = fig . add_subplot( 324 ) ax . set_title( "w_rec quant" ) ax . hist( np . ravel(quant_spec[ "weights_rec" ][quant_spec[ "weights_rec" ] != 0 ]), bins = 2 ** 8 ) ax = fig . add_subplot( 325 ) ax . set_title( "w_out float" ) ax . hist(np . ravel(w_out_float[w_out_float != 0 ]), bins = 2 ** 8 ) ax = fig . add_subplot( 326 ) ax . set_title( "w_out quant" ) ax . hist( np . ravel(quant_spec[ "weights_out" ][quant_spec[ "weights_out" ] != 0 ]), bins = 2 ** 8 ); 28.5. Deploying this netw ork to X yloSim 269 Rockpool, Release 3.0.3 As y ou can see, the floating point w eights all belo w 0 while the quantized w eights got scaled to be between -128 and +127. In this case, w e are lucky and the dis tr ibution of w eights look v ery similar . U nluc ky cases w ould be: • W eight distribution is not centered around 0 • A f e w v er y strong w eights • N on-flat distribution In all those ‘unluc ky’ cases, the quantized w eights could not make use of the complete rang e and might the resulting netw ork beha vior might be v er y different compared with the floating point netw ork. There might be ‘tr ic ks ’ to a v oid those situations. In this case, though, e v er ything looks fine. So let ’ s go to the ne xt step and create a xy lo configuration and simulate that netw ork. [13]: from rockpool.devices.xylo.syns61201 import config_from_specification xylo_conf, is_valid, message = config_from_specification( ** quant_spec) print ( "Valid config: " , is_valid) Valid config: True [14]: from rockpool.devices.xylo.syns61201 import XyloSim from rockpool.timeseries import TSEvent [15]: sim = XyloSim . from_config(xylo_conf, dt = dt) . timed(add_events = True ) 270 Chapter 28. T raining a spiking network t o deplo y to the X ylo ™ digital SNN Rockpool, Release 3.0.3 [16]: # - Evaluate classes for i_class, [ input , target] in enumerate (ds): sim . reset_state() sim . reset_time() inp = TSEvent . from_raster( input . cpu() . numpy(), dt = dt, t_start = 0 ) output, _, rec_sim = sim(inp, record = True ) # - Plot output and target plt . figure() output . plot() plt . plot(np . arange( len (target)) * dt, target, "--" ) plt . xlabel( "Time (s)" ) plt . ylabel( "Value" ) plt . ylim([ 0 , target . max()]) plt . title( f"Class { i_class } " ) As y ou can see, the quantized model s till spik es at the same times as the floating point model (see figure after training). Let ’ s br iefly compare the inter nal dynamics. [17]: fig = plt . figure(figsize = ( 16 , 25 )) (continues on ne xt page) 28.5. Deploying this netw ork to X yloSim 271 Rockpool, Release 3.0.3 (continued from pre vious pag e) ax = fig . add_subplot( 5 , 2 , 1 ) plt . plot(rec_float[ "1_LIFBitshiftTorch" ][ "isyn" ] . squeeze( 0 ) . squeeze( - 1 ) . detach()) plt . plot( rec_float[ "2_TorchResidual" ][ "1_LIFBitshiftTorch" ][ "isyn" ] . squeeze( 0 ) . squeeze( - 1 ) . detach() ) ax . set_title( "Isyn inp / hidden float" ) ax = fig . add_subplot( 5 , 2 , 2 ) rec_sim[ "Isyn" ] . plot() ax . set_title( "Isyn recurrent layer quant" ) ax = fig . add_subplot( 5 , 2 , 3 ) plt . plot(rec_float[ "1_LIFBitshiftTorch" ][ "vmem" ] . squeeze( 0 ) . squeeze( - 1 ) . detach()) plt . plot( rec_float[ "2_TorchResidual" ][ "1_LIFBitshiftTorch" ][ "vmem" ] . squeeze( 0 ) . squeeze( - 1 ) . detach() ) ax . set_title( "Vmem inp / hidden float" ) ax = fig . add_subplot( 5 , 2 , 4 ) rec_sim[ "Vmem" ] . plot() ax . set_title( "Vmem recurrent layer quant" ) ax = fig . add_subplot( 5 , 2 , 5 ) spk_float = np . vstack( [ rec_float[ "1_LIFBitshiftTorch" ][ "spikes" ] . squeeze( 0 ) . detach() . T, rec_float[ "2_TorchResidual" ][ "1_LIFBitshiftTorch" ][ "spikes" ] . squeeze( 0 ) . detach() . T, ] ) nids, times = np . where(spk_float) plt . scatter(times, nids, s = 1 ) ax . set_title( "spikes inp / hidden float" ) ax = fig . add_subplot( 5 , 2 , 6 ) rec_sim[ "Spikes" ] . plot(s = 1 ) plt . scatter(times, nids, s = 1 ) ax . set_title( "spikes recurrent layer quant" ) ax = fig . add_subplot( 5 , 2 , 7 ) plt . plot(rec_float[ "4_LIFBitshiftTorch" ][ "isyn" ] . squeeze( 0 ) . squeeze( - 1 ) . detach()) ax . set_title( "Isyn output float" ) (continues on ne xt page) 272 Chapter 28. T raining a spiking network t o deplo y to the X ylo ™ digital SNN Rockpool, Release 3.0.3 (continued from pre vious pag e) ax = fig . add_subplot( 5 , 2 , 8 ) rec_sim[ "Isyn_out" ] . plot() ax . set_title( "Isyn output quant" ) ax = fig . add_subplot( 5 , 2 , 9 ) plt . plot(rec_float[ "4_LIFBitshiftTorch" ][ "vmem" ] . squeeze( 0 ) . squeeze( - 1 ) . detach()) ax . set_title( "Vmem output float" ) ax = fig . add_subplot( 5 , 2 , 10 ) rec_sim[ "Vmem_out" ] . plot() ax . set_title( "Vmem output quant" ); 28.5. Deploying this netw ork to X yloSim 273 Rockpool, Release 3.0.3 274 Chapter 28. T raining a spiking network t o deplo y to the X ylo ™ digital SNN Rockpool, Release 3.0.3 Although it ’ s probably hard to see as there are so man y neurons, the dynamics within the netw orks look v er y similar . 28.6 Deplo y to X ylo In the ne xt step, w e want to tak e this trained netw ork and deplo y it on the X y lo hardw are. From our X y loSim simulation, w e kno w already that the netw ork will r un on X y lo and should produce the same results. Hence, the transf er is easy and straight f or ward. Firs t step is to plug in the hardw are using the USB-C cable and find the board. [33]: from rockpool.devices.xylo import find_xylo_hdks xylo_hdk_nodes, mods, vers = find_xylo_hdks() print (xylo_hdk_nodes) The connected Xylo HDK contains a Xylo Audio v2 (SYNS61201). Importing ` rockpool.devices. ˓ → xylo.syns61201 ` [< samna.xyloA2TestBoard.XyloA2TestBoard object at 0x2d8044970 >] N o w , w e can use the same configuration f or X ylo as w e did bef ore f or X yloSim. [35]: if len (xylo_hdk_nodes) > 0 : x = mods[ 0 ] modSamna = x . XyloSamna(xylo_hdk_nodes[ 0 ], xylo_conf, dt = dt) . timed() else : modSamna = None print ( "No Xylo board found!" ) W e can also r un the netw ork on X y lo in the same wa y and see that X y lo produces the same output as X y loSim did. [36]: # - Evaluate classes if modSamna is not None : for i_class, [ input , target] in enumerate (ds): modSamna . reset_state() modSamna . reset_time() inp = TSEvent . from_raster( input . cpu() . numpy(), dt = dt, t_start = 0 ) output, _, rec_sim = modSamna(inp, record = True ) # - Plot output and target plt . figure() output . plot() plt . plot(np . arange( len (target)) * dt, target, "--" ) plt . xlabel( "Time (s)" ) plt . ylabel( "Value" ) plt . ylim([ 0 , target . max()]) plt . title( f"Class { i_class } " ) else : print ( "No Xylo board found!" ) 28.6. Deploy t o X ylo 275 Rockpool, Release 3.0.3 (continued from pre vious pag e) ax . set_xticks([]) ax . set_xlabel( "" ) ax = fig . add_subplot( 514 ) rec[ "rect" ] . plot() ax . set_xticks([]) ax . set_xlabel( "" ) ax = fig . add_subplot( 515 ) plt . imshow( filt_spikes . raster(dt = 10e-3 , add_events = True ) . T, aspect = "auto" , origin = "lower" ) ax . set_xlabel( "Time (s)" ) ax . set_ylabel( "Output channel" ) ax . set_title( "Spiking output" ); 282 Chapter 29. Using the analog frontend model Rockpool, Release 3.0.3 In this pre vious plot w e can see man y things. Let ’ s go through it piece b y piece. 29.3.1 Input The first panel is simpl y the ra w input signal. 29.3.2 LNA The lo w -noise amplifier pro vides nor malisation and pre-amplification of the input signal, and simulates the distortion and nonlinear ity present in the HW . 29.3. Input g eneration 283 Rockpool, Release 3.0.3 29.3.3 BPF The bandpass filter g et activ e in the sequence of their center frequency . Y ou can also see that their center frequency is log ar ithmicall y dis tr ibuted. The center frequency is calculated using this eq uation: 𝑓 𝑐 𝑖 = 𝑓 𝑐 𝑖 − 1 𝑓 𝑖 factor + 𝑓 𝑐 mismatc h Also, the width of the filter is scaled with its center frequency . This can be manipulated using the Q factor . 29.3.4 Rectification The rectification cor responds to an abs operation but is also subject to noise. 29.3.5 Spike con version As can be seen in the last panel, the different channels emit spik es cor responding to the center frequencies of their band-pass filters. The spik e con v ersion is done b y charging a capacitor with a current cor responding to the output of the full-w a v e recifier . If the capacitor reaches a threshold, a spike is produced. As the capacitor is rather small on the hardware, the spik e rate can g et v er y high. The solution w as to use a digital counter . The digital counter allo w s onl y e v er y n th spik e to pass and drops the rest. n can be set using the digital_counter parameter The capacitor is subject to leak, which can be set with the leakage parameter . It can be used to lo w er the impact of the noise floor . If the leak is high, small bac kground noise does not lead to threshold crossing. T r y it, if the leak is reduced, the noise g enerated b y the AFE g et ’ s visible in the spiking output. 284 Chapter 29. Using the analog frontend model CHAPTER THIRT Y QUICK ST ART WITH XYL O ™ A UDIO 3 NOTE : X y loA udio 3 requires samna==0.39.6 or higher This tutor ial will re vie w the steps and tools req uired to build and train audio applications f or deplo yment on X y lo ™ A udio 3. A typical pipeline to build and deplo y an audio application f or X y lo ™ A udio 3 contains the f ollo wing steps: • preprocessing: con v er ting audio signals to spik e trains • designing and training an SNN model with spik e-encoded audio data • deplo ying the trained model in X y lo ™ A udio 3 HDK The f ollo wing diag ram illustrates these s teps along with the required tools from R ockpool, highlighted in green: • AFESimPDM and AFESimExternal are tools in R ockpool that simulate the A udio Front End (AFE) of X y lo ™ A udio 3, used as a preprocessing step to con v er t audio signals to spik e trains. • XyloSamna and XyloMonitor are R ockpool APIs that interface the user and X ylo ™ A udio 3 HDK. These APIs are designed with user -fr iendliness in mind, making the interaction with X y loA udio 3 a seamless e xperience. Please see: Using AFESim as an audio tr ansf orm where w e elaborated on using AFESimPDM and AFESimExternal as an audio transf or m to g enerate spik e-encoded samples. In the In Depth section of the R ockpool documentation at left, y ou can also see more details about the process of designing and training a model in R ockpool. 30.1 Ov er view of training and deplo yment flow f or X ylo ™ A udio 3 [1]: from IPython.display import Image Image( "figures/tools_diagram-150.png" ) 285 Rockpool, Release 3.0.3 [1]: 286 Chapter 30. Quick st ar t with X ylo ™ Audio 3 CHAPTER THIRT Y ONE OPERA TION T YPES OF XYL O A UDIO 3 X y loA udio 3 HDK has tw o different operation modes: • A cceler at ed time • Real-time T w o different deplo yments can be per f or med depending on the operation mode. Both require mapping the model’ s configuration into the X y lo SNN core and reading the output spik es. 1. Deployment wit h liv e audio fr om micr ophone : T rained models can be tested with liv e audio pla y ed to the micro- phone. This type of deplo yment r uns in Real-time mode. 2. Deployment by bypassing t he micr ophone path : T rained models can be tes ted with pre-g enerated spike-encoded audios. This type of deplo yment r uns in A cceler at ed time mode. W e use R ockpool’ s XyloSamna and XyloMonitor APIs to manage the deplo yment pipelines in deplo yment types 1 and 2, respectiv el y .. F or more details and an e xample of each deplo yment, please see the tutor ial Using X yloSamna and X yloMonit or to deploy a model on X yloAudio 3 HDK 287 Rockpool, Release 3.0.3 288 Chapter 31. Operation types of X yloAudio 3 CHAPTER THIRT Y T W O USING AFESIM AS AN A UDIO TRANSFORM R ockpool contains a simulation of the A udio Front End (AFE) of X y loA udio 3, which is used as a pre-processing s tep to con v er t audio signals to spik e trains. The con v er ted v ersion of audio can be used in the f ollo wing scenar ios: • As a training sample to train an SNN model in R ockpool • As a test sample to tes t a model on the X y loA udio 3 SNN core f or debugging pur poses. This is done b y b ypassing the microphone and AFE in the HDK. See the r elated tutorial f or more inf or mation. In this tutor ial, w e will ref er to the AFE simulator in X y loA udio 3 as AFESim3 and will go through an e xample of ho w to configure and use AFESim3 as an audio transf or m f or a train or test pipeline. There are tw o main modes in the AFESim3 module in R ockpool: • AFESimExternal – This mode of AFESim3 is independent of the microphone type. It bypasses the microphone path and passes an e xter nal audio (14-bit QU ANTIZED signal) into the filterbank and divisiv e nor malization module • AFESimPDM – In this mode, audio samples are passed through a preprocessing chain composed of PDM microphone model, filter bank and divisiv e nor malization module Using AFESimExternal is recommended f or de v eloping applications, while AFESimPDM is more suitable f or advanced debugging tasks. 32.1 AFESimExternal As illustrated in the diagram belo w , AFESimExternal receiv es input audio as an arra y , resamples and quantizes it to 14-bit f or mat, and passes it to the filter bank (which co v ers 16 frequency bands betw een 100 Hz and 17 KHz). Depending on the mode selected f or spike_gen_mode , fix ed or adaptiv e thresholds will be applied to filter output channels to g enerate a spik e train. spike_gen_mode is b y def ault set to ' divisive_norm ' , and changing it and related parameters ( low_pass_averaging_window , rate_scale_factor , dn_EPS ) is not recommended. The Divisiv e Normalization (DN) module regulates the noise sensitivity of different frequency bands of the filter bank b y applying adaptiv e thresholds. If the a v erag e po w er of a filter in a specific time windo w is less than 𝜖 , that filter’ s threshold will be adapted to g enerate f e w er spikes. The user can deactiv ate Divisiv e Normalization onl y f or debugging pur poses b y c hoosing spike_gen_mode = ' threshold ' and passing fixed_threshold_vec . The spik e train is rasterized with a giv en dt , which should the time s tep used in y our SNN model. [1]: import warnings from IPython.display import Image (continues on ne xt page) 289 Rockpool, Release 3.0.3 (continued from pre vious pag e) warnings . filterwarnings( "ignore" ) Image( "figures/afesim_external.png" ) [1]: The f ollo wing transf or m can conv er t audio: np.ndarray samples to spike trains: [2]: import numpy as np import matplotlib.pyplot as plt plt . rcParams[ ' figure.dpi ' ] = 300 from typing import Union, Optional, Tuple from rockpool.devices.xylo.syns65302 import AFESimExternal dt_s = 0.009994 # AFESimExternal afesim_external = AFESimExternal . from_specification(spike_gen_mode = "divisive_norm" , fixed_threshold_vec = None , rate_scale_factor = 63 , low_pass_averaging_window = 84e-3 , dn_EPS = 32 , dt = dt_s, ) WARNING:root: ` dn_rate_scale_bitshift ` = (6, 0) is obtained given the target ` rate_scale_ ˓ → factor ` = 63, with diff = 0.000000e+00 WARNING:root: ` dn_low_pass_bitshift ` = 12 is obtained given the target ` low_pass_ ˓ → averaging_window ` = 0.084, with diff = 1.139200e-04 WARNING:root: ` down_sampling_factor ` = 488 is obtained given the target ` dt ` = 0.009994, ␣ ˓ → with diff = -2.400000e-07 32.2 AFESimPDM The diagram belo w illus trates the difference betw een AFESimPDM and AFESimExternal . AFESimPDM includes inter - nall y a simulation of a digital microphone model, composed of a sigma-delta modulator , and pol yphase lo wpass filter to con v er t the PDM signal to 14-bit quantized data. This module can be used when debugging the PDM modules on X y loA udio 3. [3]: Image( "figures/afesimpdm.png" ) 290 Chapter 32. Using AFESim as an audio transform Rockpool, Release 3.0.3 [3]: The f ollo wing transf or m can conv er t audio: np.ndarray samples to spike trains: [4]: from rockpool.devices.xylo.syns65302 import AFESimPDM # AFESimPDM afesim_pdm = AFESimPDM . from_specification(spike_gen_mode = "divisive_norm" , fixed_threshold_vec = None , rate_scale_factor = 63 , low_pass_averaging_window = 84e-3 , dn_EPS = 32 , dt = dt_s, ) WARNING:root: ` dn_rate_scale_bitshift ` = (6, 0) is obtained given the target ` rate_scale_ ˓ → factor ` = 63, with diff = 0.000000e+00 WARNING:root: ` dn_low_pass_bitshift ` = 12 is obtained given the target ` low_pass_ ˓ → averaging_window ` = 0.084, with diff = 1.139200e-04 WARNING:root: ` down_sampling_factor ` = 488 is obtained given the target ` dt ` = 0.009994, ␣ ˓ → with diff = -2.400000e-07 32.3 Applying AFESim3 transform W e appl y a tes t audio (a 3 second sample) to both introduced AFESim3 transf or ms to g enerate our pre-recorded data [5]: ! pip install --quiet librosa import librosa audio_path = ' audio_sample/scream_sample.wav ' test_sample, sr = librosa . load(audio_path, sr = None ) test_sample = np . expand_dims(test_sample, axis = 0 )[ 0 ] [6]: # Use AFESimExternal Transform out_external,_,_ = afesim_external((test_sample, sr)) # Use AFESimPDM Transform out_pdm,_,_ = afesim_pdm((test_sample, sr)) WARNING:2025-04-14 17:53:25,513:jax._src.xla_bridge:969: An NVIDIA GPU may be present on ␣ ˓ → this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu. WARNING:jax._src.xla_bridge:An NVIDIA GPU may be present on this machine, but a CUDA- ˓ → enabled jaxlib is not installed. Falling back to cpu. 32.3. Applying AFESim3 transform 291 Rockpool, Release 3.0.3 (continued from pre vious pag e) import simpleaudio as sa import numpy as np def get_wave_object (test_file): sample_rate, data = wavfile . read(test_file) duration = int ( len (data) / sample_rate) # in seconds n = data . ndim if data . dtype == np . int8: bytes_per_sample = 1 elif data . dtype == np . int16: bytes_per_sample = 2 elif data . dtype == np . float32: bytes_per_sample = 4 else : raise ValueError ( "recorded audio should have 1 or 2 bytes per sample!" ) wave_obj = sa . WaveObject( audio_data = data, num_channels = data . ndim, bytes_per_sample = bytes_per_sample, sample_rate = sample_rate ) return duration,wave_obj Requirement already satisfied: simpleaudio in /home/vleite/.pyenv/versions/3.11.11/envs/ ˓ → rockpool311/lib/python3.11/site-packages (1.0.4) [ notice ] A new release of pip is available: 24.0 -> 25.3 [ notice ] To update, run: python -m pip install --upgrade pip W arning : The ne xt cell will pla y an audio sample f or each class (glass-break, gun-shot, scream, back ground). If y ou w ould like to tes t the detection using the XYloAudio de v elopment board, make sure the microphone on the board is close to y our PC speak er , at a reasonable v olume. [7]: for sound in rare_sounds: print ( f ' Testing { sound } sound ' ) # Load audio sample test_audio = f ' audio_sample/ { sound } _sample.wav ' duration, wave_obj = get_wave_object(test_audio) T = int (duration / model_dt) # timesteps # Instantiate XyloMonitor, deploy network to device xylo_monitor = XyloMonitor(device = xa3, config = xylo_conf, dt = model_dt, output_mode = ˓ → ' Spike ' , dn_active = True , main_clk_rate = 12.5 ) # Evolve XyloMonitor object play_obj = wave_obj . play() out, state, rec = xylo_monitor . evolve(input_data = np . zeros([T, Nin]), record_ ˓ → power = True ) play_obj . wait_done() (continues on ne xt page) 298 Chapter 33. Using X yloSamna and X yloMonitor t o deplo y a model on X yloAudio 3 HDK Rockpool, Release 3.0.3 (continued from pre vious pag e) # Prediction prediction = rare_sounds[np . argmax(np . sum(out, axis = 0 ))] print ( f ' Detected sound: { prediction } \n ' ) Testing glass sound Detected sound: glass Testing gun sound Detected sound: gun Testing scream sound Detected sound: scream Testing background sound Detected sound: background 33.3.2 Po wer consumption W e can record the po w er consumption b y setting “record 𝑝 𝑜𝑤 𝑒𝑟 = 𝑇 𝑟 𝑢𝑒 “ 𝑤 ℎ𝑖𝑙 𝑒𝑒𝑣 𝑜𝑙 𝑣 𝑖𝑛𝑔 𝑤 𝑖𝑡ℎ : 𝑝𝑦 : 𝑐𝑙 𝑎𝑠𝑠 : ‘ .𝑠𝑦 𝑛𝑠 65302 .𝑋 𝑦 𝑙 𝑜𝑀 𝑜𝑛𝑖𝑡𝑜𝑟 ‘ .𝑇 ℎ𝑖𝑠𝑤 𝑖𝑙𝑙 𝑟 𝑒𝑐𝑜𝑟 𝑑𝑡ℎ𝑒𝑝𝑜𝑤 𝑒𝑟 𝑖𝑛𝑡ℎ𝑒𝑡ℎ𝑟 𝑒𝑒𝑐ℎ𝑎𝑛𝑛𝑒𝑙 𝑠 : “ 𝑖𝑜 “ , “ 𝑎𝑛𝑎𝑙 𝑜𝑔 “ 𝑎𝑛𝑑 “ 𝑑𝑖𝑔 𝑖𝑡𝑎𝑙 “ . N ote: w e can configure the chip to run with a slo wer cloc k to sa v e energy . N ote, ho w e v er , that w e need to use a cloc k speed that is f as t enough to finish the data processing in one step. [8]: io_power = np . mean(rec[ ' io_power ' ]) analog = np . mean(rec[ ' analog_power ' ]) digital = np . mean(rec[ ' digital_power ' ]) print ( f ' XyloAudio 3 \n io: \t { np . ceil(io_power * 1e6 ) : .0f } µ W \t AFE core: \t { np . ceil(analog * ␣ ˓ → 1e6 ) : .0f } µ W \t DFE+SNN core: \t { np . ceil(digital * 1e6 ) : .0f } µ W \n ' ) XyloAudio 3 io: 456 µ W AFE core: 13 µ W DFE+SNN core: 588 µ W [ ]: 33.3. Using X yloMonitor in Real Time mode 299 Rockpool, Release 3.0.3 300 Chapter 33. Using X yloSamna and X yloMonitor t o deplo y a model on X yloAudio 3 HDK CHAPTER THIRT YF OUR B ANDP ASS FIL TERING IN A UDIO FRONT END (AFE) OF XYL O ™ A UDIO 3 The k e y f eature e xtraction module in X ylo ™ A udio 3 is the collection of Mel-filterbanks, which compute the time- frequency transf orm of the input audio signal. In this tutor ial, w e will revie w the main f eatures of the filter bank and pro vide instructions on ho w to reconfigure it if needed. The central frequency and all filterbank parameters ha v e been designed f or efficient audio processing, co v er ing the rang e of 100 𝐻 𝑧 up to around 17 𝐾 𝐻 𝑧 audio spectr um. Main f eatures of the filterbank include: • Log ar ithmicall y spaced, matc hed to human perception mechanism, with the f ollo wing freq uency scaling 𝛼 and quality f actor 𝑄 . 𝛼 = ( 17000 100 ) 1 / (16 − 1) = 170 1 15 = 1 . 4083 and 𝑄 ≈ 6 • Designed as order -1 Butter w or th filters with digital transf er function: 𝐻 ( 𝑧 ) = 𝑏 ( 𝑧 ) 𝑎 ( 𝑧 ) = 𝑏 0 + 𝑏 1 𝑧 + 𝑏 2 𝑧 2 𝑎 0 + 𝑎 1 𝑧 + 𝑎 2 𝑧 2 . • Implemented as a cascaded of AR (auto-regressiv e filter) 𝐻 1 ( 𝑧 ) = 1 𝑎 ( 𝑧 ) f ollo w ed b y the MA (mo ving a v erag e) filter 𝐻 2 ( 𝑧 ) = 𝑏 ( 𝑧 ) , as illustrated in F igure 1. In R ockpool, the bandpass filtering is implemented as a quantized digital filterbank with hard-coded parameters (see AFESimExternal or AFESimPDM ). Def ault central frequencies in X y lo ™ A udio 3 are: [1]: import numpy as np f0 = 100 f15 = 17000 N_filters = 16 # number of filters alpha = np . power(f15 / f0, 1 / (N_filters - 1 )) freqs = [f0] for i in range ( 1 ,N_filters): freqs . append( int (alpha * freqs[ - 1 ])) print ( f ' designed filter centers are: { freqs } ' ) designed filter centers are: [100, 140, 197, 277, 390, 549, 773, 1088, 1532, 2157, 3037, ␣ ˓ → 4277, 6023, 8482, 11945, 16822] After parameter quantization, there will be a shift in the central frequency of filters. Theref ore, actual v alues f or centers of filters in AFESim and HDK are: [105 , 136 , 201 , 279 , 390 , 542 , 762 , 1070 , 1503 , 2110 , 2963 , 4161 , 5843 , 8204 , 11582 , 16611] 301 Rockpool, Release 3.0.3 Let ’ s instantiate AFESimExternal , as an e xample: [2]: from rockpool.devices.xylo.syns65302 import AFESimExternal import warnings warnings . filterwarnings( ' ignore ' ) dt_s = 0.009994 afesim_external = AFESimExternal . from_specification(spike_gen_mode = "divisive_norm" , fixed_threshold_vec = None , rate_scale_factor = 63 , low_pass_averaging_window = 84e-3 , dn_EPS = 32 , dt = dt_s,) WARNING:root: ` dn_rate_scale_bitshift ` = (6, 0) is obtained given the target ` rate_scale_ ˓ → factor ` = 63, with diff = 0.000000e+00 WARNING:root: ` dn_low_pass_bitshift ` = 12 is obtained given the target ` low_pass_ ˓ → averaging_window ` = 0.084, with diff = 1.139200e-04 WARNING:root: ` down_sampling_factor ` = 488 is obtained given the target ` dt ` = 0.009994, ␣ ˓ → with diff = -2.400000e-07 Let ’ s demonstrate the response of filters in the simulator b y passing monotone sinusoids as input (firs t and fifth filters with central frequencies 105 𝐻 𝑧 and 390 𝐻 𝑧 ). [9]: from rockpool.devices.xylo.syns65302.afe import ChipButterworth import matplotlib.pyplot as plt test_freqs = [ 105 , 390 ] #input frequencies fs = 48000 #sampling rate N = 5 #input duration (seconds) time = np . linspace( 0 ,N, int (N * fs)) EPS = 0.00001 fb = ChipButterworth() B_in = fb . bd_list[ 0 ] . B_in + 4 # quantization range for input signal plt . figure(figsize = ( 11 , 4 )) for i,f in enumerate (test_freqs): sig_in = np . sin( 2 * np . pi * f * time) sig_in = sig_in / np . max(np . abs(sig_in)) * ( 1 + EPS) * 2 ** (B_in - 1 ) # quantize the ␣ ˓ → sinal q_sig_in = sig_in . astype(np . int64) output,_, _ = afesim_external((q_sig_in,fs)) ax = plt . subplot( 1 , 2 ,i + 1 ) plt . imshow(output . T, aspect = ' auto ' ); plt . grid( True ); plt . xlabel( ' Time(sec) ' ); plt . ylabel( ' Freq(Hz) ' ) ax . set_xticks( range ( 0 , 500 , 100 )); ax . set_yticks( range (N_filters)) ax . set_xticklabels( [ int (t * np . round(dt_s, decimals = 2 )) for t in ax . get_xticks()]) ax . set_yticklabels(freqs); plt . colorbar() 302 Chapter 34. Bandpass Filtering In Audio F ront End (AFE) Of X ylo ™ Audio 3 Rockpool, Release 3.0.3 34.1 Reconfiguration of filter parameters [4]: from IPython.display import Image, display, Markdown display(Image( "figures/block_diagram.png" )) display(Markdown( "**Figure 1:** Overview of the implemented filter architecture." )) Figure 1: Ov er vie w of the implemented filter architecture. As mentioned abo v e, the design v alues of the filter parameters are suitable f or efficient audio processing. The y hav e been carefull y c hosen to ensure good co v erag e of the range betw een 100 𝐻 𝑧 and 17 𝑘 𝐻 𝑧 , as w ell as numer ical stability . Changing these parameters is not r ecommended f or audio applications. Ho w e v er , the filter parameters can be modified b y e xper t users in the simulator and also reconfigured in the rele v ant registers in hardw are, f or shifting the filter centers to desired v alues (f or special non-audio use cases). In g eneral IIR filters are sensitiv e to quantization and coefficient rounding and w e advise users to approach filter modification with caution, especially reg arding stability and numerical precision. The main parameters to modify are 𝑎 1 and 𝑎 2 in 𝐻 ( 𝑧 ) . Using scip y .signal.iir peak(), w e can compute the transf er 34.1. Reconfiguration of filter parameters 303 Rockpool, Release 3.0.3 function parameters giv en the sampling frequency , quality f actor ( 𝑄 ), and center frequency . The f ollo wing code blocks can be used to calculate the transf er function of a desired filter (i.e., the required 𝑎 1 and 𝑎 2 parameters). The e xample demons trates ho w to modify the filters to co v er a rang e betw een 300 𝐻 𝑧 and 20 𝐾 𝐻 𝑧 . [5]: import numpy as np f0 = 300 f15 = 20000 N_filters = 16 # number of filters alpha = np . power(f15 / f0, 1 / (N_filters - 1 )) new_freqs = [f0] for i in range ( 1 ,N_filters): new_freqs . append( int (alpha * new_freqs[ - 1 ])) print ( f ' modified filter centers are: { new_freqs } ' ) modified filter centers are: [300, 396, 523, 691, 914, 1209, 1599, 2115, 2798, 3702, ␣ ˓ → 4898, 6480, 8573, 11342, 15006, 19854] [6]: from scipy import signal Q = 6 #Qfactor new_params = [] for f in new_freqs: # Get digital filter coefficients (z-domain) filter_params = signal . iirpeak(f, Q, fs = fs) new_params . append(filter_params) b ,a = new_params[ 0 ] #parameters of the first filter a 1,a 2 = a[ 1 ], a[ 2 ] 𝑎 1 and 𝑎 2 parameters need to be quantized as f ollo w s: (In AFESim, the filters are quantized to match X y lo ™ A udio 3 specifications.) • ˜ 𝑎 1 = [2 𝐵 𝑏 2 𝐵 𝑎,𝑓 𝑎 1 ] ˜ 𝑎 2 = [2 𝐵 𝑏 2 𝐵 𝑎,𝑓 𝑎 2 ] where: • 𝐵 𝑏 : bits needed f or scaling b0 • 𝐵 𝑎𝑓 : bits needed f or encoding the fractional par ts of taps Without going into details, 𝐵 𝑏 and 𝐵 𝑎𝑓 are integ er v alues defining the precision of eac h parameter in the filter . The y ha v e been hardcoded in AFESim (see AFESimExternal or AFESimPDM ) f or each of the 16 filters and should not be modified. ˜ 𝑎 1 and ˜ 𝑎 2 can be calculated as f ollo w s [7]: a1_hats, a2_hats = [],[] for i, params in enumerate (new_params): b ,a = params a 1,a 2 = a[ 1 ], a[ 2 ] B_b = fb . bd_list[i] . B_b B_af = fb . bd_list[i] . B_af q_scale = ( 2 ** B_b) * ( 2 ** B_af) a1_hat, a2_hat = int (q_scale * a1), int (q_scale * a2) a1_hats . append(a1_hat) a2_hats . append(a2_hat) and replaced in ~.syns65302.ChipButterworth module f or each filter . 304 Chapter 34. Bandpass Filtering In Audio F ront End (AFE) Of X ylo ™ Audio 3 Rockpool, Release 3.0.3 Equiv alentl y in HDK, ˜ 𝑎 1 and ˜ 𝑎 2 parameters can be modified b y setting bpf_a1_values and bpf_a2_values on the chip configuration: F or e xample, in the tutor ial Using X yloSamna and X yloMonit or to deplo y a model on X y loAudio 3 HDK , a chip con- figuration is retur ned after mapping a netw ork: spec = mapper(model.as_graph(), weight_dtype= ' float ' , threshold_dtype= ' float ' , dash_ ˓ → dtype= ' float ' ) spec.update(q.channel_quantize(**spec)) xylo_conf, is_valid, msg = config_from_specification(**spec) T o set the new ˜ 𝑎 1 and ˜ 𝑎 2 parameters, modify the returned configuration as f ollo w s: xylo_conf.digital_frontend.filter_bank.bpf_a1_values = a1_hats xylo_conf.digital_frontend.filter_bank.bpf_a2_values = a2_hats 34.1. Reconfiguration of filter parameters 305 Rockpool, Release 3.0.3 306 Chapter 34. Bandpass Filtering In Audio F ront End (AFE) Of X ylo ™ Audio 3 CHAPTER THIRT YFIVE INTRODUCTION T O XYL O ™ IMU X y loIMU is a platf or m f or motion sensor y processing, combining an IMU accelerometer sensor inter f ace with a lo w- po w er SNN inf erence core. X yloIMU is designed f or sub-mW motion processing, in alwa y s-on applications. This notebook giv es y ou an o v er vie w of interfacing from R oc kpool to the v ar ious cores of X y lo-IMU . See also Quic k-s tart with X ylo ™ SNN cor e and The X ylo ™ IMU pr epr ocessing interface . [1]: # - General installation and imports import sys !{ sys.executable } -m pip install --quiet matplotlib rich "rockpool[xylo, torch]" # - Import numpy import numpy as np # - Configure matplotlib import matplotlib.pyplot as plt % matplotlib inline plt . rcParams[ ' figure.figsize ' ] = [ 12 , 4 ] plt . rcParams[ ' figure.dpi ' ] = 300 # - Display images from IPython.display import Image import warnings warnings . filterwarnings( ' ignore ' ) # - Nice printing from rich import print 35.1 Ov er view of the X ylo ™ IMU design X y loIMU receiv es direct input from a MEMSIC MC3632 IMU sensor , and is designed to enable motion classification applications in real-time at lo w po wer . The bloc k -le v el outline of the chip is sho wn belo w . A dedicated IMU inter f ace connects to the IMU sensor via a mas ter SPI bus, and con v er ts motion data into streams of e v ents. Ev ent-encoded motion data is transf er red to a digital SNN core f or inf erence. The results of inf erence, encoded as output e v ent streams, are transmitted o v er an inter r upt bus or sla v e SPI bus to an e xter nal microcontroller . A bloc k of control logic manag es configuration and communication with the cores. Configuration and readout from X y loIMU is per f or med o v er a sla v e SPI bus. 307 Rockpool, Release 3.0.3 35.4 P ART II: The IMU encoding inter face X y loIMU incor porates an efficient e v ent encoding block f or IMU motion data, whic h includes remo v al / cor rection of rotation b y detection the g ra vity-associated f orce v ector . F or a more detailed descr iption of the inter f ace, see The X y lo ™ IMU pr eprocessing int er f ace . The bloc k diagram belo w sho w s the main steps in preprocessing IMU data to e v ents. Three channels of IMU input (x, y , z) are con v er ted to 15 e v ent channels (5 per sensor c hannel), which are then sent to the X ylo SNN inf erence core. R ockpool includes a bit-accurate simulation of the IMU con v ersion inter face, in IMUIFSim . This simulator allo w s y ou to encode IMU motion data as e v ents in the same w a y as liv e IMU data on X y lo-IMU, f or use in training and testing applications. [17]: Image( ' IMU-IF-block-level.png ' ) 314 Chapter 35. Introduction to X ylo ™ IMU Rockpool, Release 3.0.3 [17]: 35.4.1 Using the IMU IF Simulation module See also The X ylo ™ IMU pr epr ocessing int er f ace and Specifying IMU pr eprocessing par amet ers [18]: from rockpool.devices.xylo.syns63300 import IMUIFSim # - Load a dummy IMU sample input_data = np . load( ' data.npy ' ) # - Create an IMUIFSim module mod = IMUIFSim() print (mod) IMUIFSim with shape ( 3 , 15 ) { ModSequential ' model ' with shape ( 3 , 15 ) { RotationRemoval ' 0_RotationRemoval ' with shape ( 3 , 3 ) { ModSequential ' sub_estimate ' with shape ( 3 , 9 ) { SubSpace ' 0_SubSpace ' with shape ( 3 , 9 ) SampleAndHold ' 1_SampleAndHold ' with shape ( 9 , 9 ) } } FilterBank ' 1_FilterBank ' with shape ( 3 , 15 ) ScaleSpikeEncoder ' 2_ScaleSpikeEncoder ' with shape ( 15 , 15 ) } } IMUIFSim is a standard R oc kpool module accepting raw IMU data, and returning encoded e v ents. [19]: # - Quantize the raw IMU data from rockpool.devices.xylo.syns63300 import Quantizer quantizer = Quantizer(shape = 3 , scale = 0.49 , num_bits = 16 ) Q_data, _, _ = quantizer(input_data) # - Test evolution and plot input and output out, _, r_d = mod(Q_data, record = True ) times = np . arange( 0 , input_data . shape[ 0 ]) * mod . dt ax = plt . subplot( 2 , 1 , 1 ) ax . plot(times, input_data) ax . set_xlim( 0 , times[ - 1 ]) ax . set_xticklabels([]) (continues on ne xt page) 35.4. P ART II: The IMU encoding inter face 315 Rockpool, Release 3.0.3 (continued from pre vious pag e) ax . set_title( ' Raw IMU Input ' ) ax . set_ylabel( ' Accel. ' ) ax = plt . subplot( 2 , 1 , 2 ) ax . imshow(out[ 0 ] . astype( int ) . T, aspect = ' auto ' , interpolation = ' none ' , origin = ' lower ' ) ax . set_xticklabels(np . array(ax . get_xticks() * mod . dt) . astype( int )) ax . set_title( ' Encoded IMU as events ' ) ax . set_xlabel( ' Time (s) ' ) ax . set_ylabel( ' Channel ' ); Using the export_config() method, y ou can obtain a hardware configuration object with whic h y ou can configure the encoding bloc k on X y lo IMU . [20]: if_config = mod . export_config() print (if_config) xyloImu::configuration:: InputInterfaceConfig ( enable = 1 , configuration_timeout = 3.000000 , ␣ ˓ → estimator_k_setting = 4 , select_iaf_output = 0 , bypass_jsvd = 0 , update_matrix_threshold = 10 , delay_threshold = 500 , bpf_ ˓ → bb_values = { 6543265 43265432 } , bpf_bwf_values = { 888888888888888 } , bpf_baf_values = { 9 ␣ ˓ → 10 11 12 13 9 10 11 12 13 9 10 11 12 13 } , bpf_a1_values = { -64458 -63288 -60684 -54529 -39246 -64458 -63288 -60684 - ˓ → 54529 -39246 -64458 -63288 -60684 -54529 -39246 } , bpf_a2_values = { 31754 30771 28888 25417 19378 31754 30771 28888 ␣ ˓ → 25417 19378 31754 30771 28888 25417 19378 } , scale_values = { 555555555555555 } , iaf_threshold_ ˓ → values = { 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 }) IMUIFSim suppor ts all optional f eatures of the IMU encoding inter face. See Specifying IMU pr epr ocessing parame t ers f or more detail on ho w to specify the parameters of the IMU IF . 316 Chapter 35. Introduction to X ylo ™ IMU Rockpool, Release 3.0.3 35.4.2 Accessing the IMU encoding interface on HW R ockpool also pro vides a module IMUIFSamna to record encoded data from the IMU interface on X yloIMU . The IMU sensor data can come either from the on-board IMU chip, or from pre-recorded data s treamed from the host PC. W arning IMUIFSamna cur rentl y uses a w ork -around to record encoded IMU data as ev ents. It is only capable of recording one e v ent per time-step per c hannel from the IMU inter face. Ho w e v er , the IMU inter face is capable of producing multiple e v ents per time-step per c hannel. As a result, data recorded using IMUIFSamna ma y differ from that trans- mitted to the SNN core on X y lo IMU . Y ou can alter nativ ely use IMUData to record ra w IMU data, and IMUIFSim to encode that data, suppor ting multiple e v ents per time-step per c hannel. This ma y result in more accurate encoding of data. [21]: Image( ' IMUIFSamna.png ' , width = 400 ) [21]: [22]: # - Import the required Rockpool module from rockpool.devices.xylo.syns63300 import IMUIFSamna # - Instantiate the module, connected to an HDK ` db ` imu_if = IMUIFSamna(db, prerecorded_imu_input = False ) # - You can also supply pre-recorded IMU data, and specify the IMU IF parameters # IMUIFSamna(db, prerecorded_imu_input = True, interface_params={}) [23]: # - Record for 1000 time steps, by providing zero data out, _, _ = imu_if(np . zeros(( 1000 , 3 ))) [24]: # times, channels = np.where(out) # plt.scatter(times, channels) plt . imshow(out . T, aspect = ' auto ' , interpolation = ' none ' ) [24]: <matplotlib.image.AxesImage at 0x386214310> 35.4. P ART II: The IMU encoding inter face 317 Rockpool, Release 3.0.3 Delete the module to release access to the IMU sensor . [25]: del imu_if [2024-06-07 13:03:24.926] [Graph] [warning] Graph is destroyed while running! Note: ␣ ˓ → Filter nodes constructed by ` sequential ` method won ' t work after corresponding graph ␣ ˓ → is destroyed and please manually stop the graph after use. 35.5 P ART III: Recording live IMU data R ockpool pro vides a module IMUData to record ra w IMU data from the MEMSIC IMU Sensor on the X y loIMU HDK. This can be used to record liv e IMU input, or to record IMU data to build a training dataset. [26]: Image( ' IMUData.png ' , width = 400 ) [26]: [27]: # - Import and instantiate and IMUData module from rockpool.devices.xylo.syns63300 import IMUData mod = IMUData(db) print (mod) IMUData with shape ( 0 , 3 ) IMUData is a standard R oc kpool module, whic h retur ns real-time samples recorded from the IMU de vice on a X y lo IMU HDK. Use a zero-element numpy ar ra y to specify ho w man y samples to record. [28]: dt = mod . dt # By default Xylo IMU operates at 200 Hz T = int ( 5. / dt) # Record for 5 seconds (continues on ne xt page) 318 Chapter 35. Introduction to X ylo ™ IMU Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Record live IMU sensor data data, _, _ = mod(np . zeros(( 0 ,T , 0 ))) # - Plot the data samples times = np . arange( 0 ,T ) * mod . dt plt . plot(times, data) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Acceleration ' ); 35.6 P ART IV : Real-time streaming mode The streaming mode combines the IMU inter face and SNN inf erence core modules tog ether in real time using the class XyloIMUMonitor . In this mode y ou onl y read the output e v ents from the SNN core, with input either from the liv e IMU sensor , or streamed from a PC. [29]: Image( ' XyloIMUMonitor.png ' , width = 400 ) [29]: 35.6.1 Step 1: Build a network in rockpool and con ver t it to a hardware configura- tion [30]: # - Define the size of the network layers Nin = 15 (continues on ne xt page) 35.6. P ART IV : Real-time streaming mode 319 Rockpool, Release 3.0.3 (continued from pre vious pag e) Nhidden = 4 Nout = 2 dt = 1. / 200 [31]: # - Define the network architecture using combinators and modules net = Sequential( LinearTorch((Nin, Nhidden), has_bias = False ), LIFTorch(Nhidden, dt = dt), Residual( LinearTorch((Nhidden, Nhidden), has_bias = False ), LIFTorch(Nhidden, has_rec = True , threshold = 1. ,d t = dt), ), LinearTorch((Nhidden, Nout), has_bias = False ), LIFTorch(Nout, dt = dt), ) print (net) TorchSequential with shape ( 15 , 2 ) { LinearTorch ' 0_LinearTorch ' with shape ( 15 , 4 ) LIFTorch ' 1_LIFTorch ' with shape ( 4 , 4 ) TorchResidual ' 2_TorchResidual ' with shape ( 4 , 4 ) { LinearTorch ' 0_LinearTorch ' with shape ( 4 , 4 ) LIFTorch ' 1_LIFTorch ' with shape ( 4 , 4 ) } LinearTorch ' 3_LinearTorch ' with shape ( 4 , 2 ) LIFTorch ' 4_LIFTorch ' with shape ( 2 , 2 ) } [32]: # - Call the Xylo mapper on the extracted computational graph spec = mapper(net . as_graph(), weight_dtype = ' float ' , threshold_dtype = ' float ' , dash_dtype = ˓ → ' float ' ) # - Quantize the specification spec . update(q . global_quantize( ** spec)) # # you can also try channel-wise quantization # spec.update(q.channel_quantize(**spec)) # print(spec) # - Use rockpool.devices.xylo.config_from_specification to convert it to a hardware ␣ ˓ → configuration config, is_valid, msg = config_from_specification( ** spec) if not is_valid: print (msg) 320 Chapter 35. Introduction to X ylo ™ IMU Rockpool, Release 3.0.3 35.6.2 Step 2: Deploy the ne twork on chip and run simulation [36]: # - Find and connect to a Xylo IMU HDK xylo_hdk_nodes, _, vers = find_xylo_hdks() print (xylo_hdk_nodes, vers) if len (xylo_hdk_nodes) == 0 or vers[ 0 ] != ' syns63300 ' : assert False , ' This tutorial requires a connected Xylo IMU HDK to demonstrate. ' else : db = xylo_hdk_nodes[ 0 ] The connected Xylo HDK contains a Xylo IMU. Importing ` rockpool.devices.xylo.syns63300 ` [< samna.xyloImuBoards.XyloImuTestBoard object at 0x3508e55b0 >] [ ' syns63300 ' ] [37]: # - Use the ` XyloIMUMonitor ` module to deploy to the HDK, running in real time from rockpool.devices.xylo.syns63300 import XyloIMUMonitor output_mode = "Vmem" modMonitor = XyloIMUMonitor(device = db, config = config, dt = dt, output_mode = output_mode) [38]: # - A resultList stack to store the results from IPython.display import clear_output class ResultList ( object ): def __init__ ( self , max_len = 100 ): self . _list = [] self . max_len = max_len def reset ( self ): self . _list = [] def append ( self , num): if len ( self . _list) < self . max_len: self . _list . append(num) else : self . _list[: self . max_len - 1 ] = self . _list[ 1 :] self . _list[ self . max_len - 1 ] = num def is_full ( self ): if len ( self . _list) == self . max_len: return True else : return False def counts ( self , features = []): count = 0 for _ in self . _list: if _ in features: count += 1 return count (continues on ne xt page) 35.6. P ART IV : Real-time streaming mode 321 Rockpool, Release 3.0.3 (continued from pre vious pag e) def __len__ ( self ): return len ( self . _list) @property def list ( self ): return self . _list [39]: # - Draw a real time image for output channels, you can shake the Xylo IMU device while ␣ ˓ → running lines = [ResultList(max_len = 10 ) for _ in range (Nout)] time_base = ResultList(max_len = 10 ) tt = 0 T = 200 t_inference = 10. from time import time t_start = time() while (time() - t_start) < t_inference: # - Perform inference on the Xylo IMU HDK output, _, _ = modMonitor(input_data = np . zeros((T, 3 ))) if output is not None : output = np . max(output, axis = 0 ) for i in range (Nout): lines[i] . append(output[i]) time_base . append(tt) tt += 0.1 ax_time = time_base . list for i in range (Nout): plt . plot(ax_time, lines[i] . list, label = f"class { i } " ) plt . xlabel( ' time ' ) plt . ylabel( ' Vmem ' ) plt . legend() plt . pause( 0.1 ) clear_output(wait = True ) 322 Chapter 35. Introduction to X ylo ™ IMU Rockpool, Release 3.0.3 35.7 Par t V : Measuring power on the X ylo HDK XyloSamna pro vides an inter f ace to real-time po w er measurements on the X y lo ™ IMU HDK. Cur rent on se v eral po w er nets on the chip can be sampled async hronously , while the device is in operation. The evolve() method pro vides a po wer measurement interf ace while the de vice is in inf erence mode, and the XyloSamna module can be used also to measure idle po w er . [42]: # - Find and connect to a Xylo IMU HDK xylo_hdk_nodes, modules, versions = find_xylo_hdks() print (xylo_hdk_nodes, versions) if len (xylo_hdk_nodes) == 0 or versions[ 0 ] is not ' syns63300 ' : assert False , ' This tutorial requires a connected Xylo IMU HDK to demonstrate. ' else : db = xylo_hdk_nodes[ 0 ] x = modules[ 0 ] The connected Xylo HDK contains a Xylo IMU. Importing ` rockpool.devices.xylo.syns63300 ` [< samna.xyloImuBoards.XyloImuTestBoard object at 0x3508e55b0 >] [ ' syns63300 ' ] On instantiation, XyloSamna can specify a po w er sampling frequency . By def ault, po w er is measured at 5 Hz. Belo w w e increase this to 20 Hz. [43]: # - Set a low clock frequency for the Xylo device print ( f ' Setting Xylo main clock to { x . xylo_imu_devkit_utils . set_xylo_core_clock_freq(db, ␣ ˓ → 6.25 ) } MHz ' ) # - Use XyloSamna to deploy to the HDK modSamna = x . XyloSamna(db, config, dt = 10e-3 , power_frequency = 20. ) print (modSamna) Setting Xylo main clock to 6.25 MHz XyloSamna with shape ( 15 , 4 , 2 ) 35.7. Par t V : Measuring power on the X ylo HDK 323 Rockpool, Release 3.0.3 [11]: Ra w IMU data is obtained at 200 Hz from the IMU sensor , pro viding quantized three-c hannel (x, y , z) samples. Depending on the applicaiton use-case, an IMU sensor might ha v e an arbitrar y or changing orientation in the real w or ld. The IMU inter f ace contains a set of algorithms f or estimating and correcting f or sensor orientation , such that the gra vitity v ector is alw a ys aligned with the -Z axis. This rotation correction module can be bypassed. The IMU data is anal y sed b y a tunable bank of bandpass filters , which can separate the data frequencies with central frequencies betw een 0..20Hz. U p to fiv e filters can be used f or each (x, y , z) channel of the IMU data. T w o alter nativ e strategies are a vailable f or e v ent encoding. The first quantizes and scales the filtered data to a maximum of 15 e v ents per time-step per c hannel. The second uses a digital integrate-and-fire neur on to encode e v ents. A full simulation of the IMU encoding inter f ace is a v ailable in R ockpool, via the IMUIFSim module and the devices. xylo.syns63300.imuif pac kag e. [12]: # - Import Rockpool modules from rockpool.devices.xylo.syns63300 import Quantizer, IMUIFSim from samna.xyloImu.configuration import InputInterfaceConfig [13]: Image( ' imu-ifsim-module.png ' ) [13]: The high-le v el module IMUIFSim encapsulates the entire preprocessing toolchain, and per mits configuration of the encoding approach. Inter nally the se v eral blocks in the encoding c hain also ha v e cor responding R ockpool modules (compare the tw o figures abo v e). It is also possible to access the simulations of individual blocks, using the R oc kpool modules RotationRemoval , FilterBank , ScaleSpikeEncoder and IAFSpikeEncoder . The IMU IF simulation e xpects integ er -quantized data with 16 bits per sample per c hannel (i.e. -32’768..32’767). This 330 Chapter 36. The X ylo ™ IMU preprocessing inter face Rockpool, Release 3.0.3 can con v enientl y be simulated using the Quantizer module. [14]: ## Load and quantize an IMU sample record with open ( "data.npy" , "rb" ) as f: data = np . load(f) quantizer = Quantizer(scale = 0.49 , num_bits = 16 ) data_quantized, _, _ = quantizer(data) data_quantized . shape plt . figure() plt . plot(data_quantized[ 0 ]) plt . title( f"Quantized IMU Sample Record" ) plt . show() 36.2 Using the high-le vel simulation interface The R ockpool module IMUIFSim uses def ault configuration parameters co v er ing a reasonable set of use cases. By def ault rotation remo v al is switc hed on, and the ScaleSpikeEncoder is used. Y ou create and interact with the IMU IF simulation module identicall y to an y other R ockpool module: [15]: # - Generate an IMU IF simulation with default parameters mod_IMUIF = IMUIFSim() print (mod_IMUIF) IMUIFSim with shape ( 3 , 15 ) { ModSequential ' model ' with shape ( 3 , 15 ) { RotationRemoval ' 0_RotationRemoval ' with shape ( 3 , 3 ) { ModSequential ' sub_estimate ' with shape ( 3 , 9 ) { SubSpace ' 0_SubSpace ' with shape ( 3 , 9 ) SampleAndHold ' 1_SampleAndHold ' with shape ( 9 , 9 ) } } FilterBank ' 1_FilterBank ' with shape ( 3 , 15 ) (continues on ne xt page) 36.2. Using the high-lev el simulation interface 331 Rockpool, Release 3.0.3 (continued from pre vious pag e) ScaleSpikeEncoder ' 2_ScaleSpikeEncoder ' with shape ( 15 , 15 ) } } [16]: # - Pass an IMU sample through the simulation result, _, _ = mod_IMUIF(data_quantized) # - Display the encoded result plt . figure() plt . imshow(result[ 0 ] . astype( ' float ' ) . T, aspect = ' auto ' , origin = ' lower ' ) plt . title( f"Preprocessed IMU Sample Record" ) plt . show() Expor t the hardw are configuration, f or use in configur ing a X y lo IMU chip, using the export_config() method. [17]: config = mod_IMUIF . export_config() print (config) xyloImu::configuration:: InputInterfaceConfig ( enable = 1 , configuration_timeout = 3.000000 , ␣ ˓ → estimator_k_setting = 4 , select_iaf_output = 0 , bypass_jsvd = 0 , update_matrix_threshold = 10 , delay_threshold = 500 , bpf_ ˓ → bb_values = { 6543265 43265432 } , bpf_bwf_values = { 888888888888888 } , bpf_baf_values = { 9 ␣ ˓ → 10 11 12 13 9 10 11 12 13 9 10 11 12 13 } , bpf_a1_values = { -64458 -63288 -60684 -54529 -39246 -64458 -63288 -60684 - ˓ → 54529 -39246 -64458 -63288 -60684 -54529 -39246 } , bpf_a2_values = { 31754 30771 28888 25417 19378 31754 30771 28888 ␣ ˓ → 25417 19378 31754 30771 28888 25417 19378 } , scale_values = { 555555555555555 } , iaf_threshold_ ˓ → values = { 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 1024 }) Giv en a sa v ed hardw are configuration, y ou can also g enerate an IMU IF simulation based on that config, using the from_config() class method: 332 Chapter 36. The X ylo ™ IMU preprocessing inter face Rockpool, Release 3.0.3 [18]: mod = IMUIFSim . from_config(config) print (mod) IMUIFSim with shape ( 3 , 15 ) { ModSequential ' model ' with shape ( 3 , 15 ) { RotationRemoval ' 0_RotationRemoval ' with shape ( 3 , 3 ) { ModSequential ' sub_estimate ' with shape ( 3 , 9 ) { SubSpace ' 0_SubSpace ' with shape ( 3 , 9 ) SampleAndHold ' 1_SampleAndHold ' with shape ( 9 , 9 ) } } FilterBank ' 1_FilterBank ' with shape ( 3 , 15 ) ScaleSpikeEncoder ' 2_ScaleSpikeEncoder ' with shape ( 15 , 15 ) } } 36.3 N e xt steps See Specifying IMU pr eprocessing par amet er s f or details of ho w to configure the preprocessing and encoding modules. 36.3. N ext steps 333 Rockpool, Release 3.0.3 334 Chapter 36. The X ylo ™ IMU preprocessing inter face CHAPTER THIRT Y SEVEN SPECIFYING IMU PREPROCESSIN G P ARAMETERS X y lo ™ IMU includes a custom pre-processing and e v ent encoding inter face designed to w ork with IMU data. When designing an IMU signal processing application with X y lo ™ IMU , the parameters of the IMU inter face should be set appropr iatel y f or the giv en application. This notebook e xplains the parameters that can be modified b y the application de v eloper , and demonstrates ho w to configure them. [14]: ## - Imports and configuration import numpy as np # - Plotting and config import matplotlib.pyplot as plt plt . rcParams[ "figure.figsize" ] = [ 9.6 , 3.6 ] plt . rcParams[ "figure.dpi" ] = 1200 plt . rcParams[ "font.size" ] = 12 try : from rich import print except : pass from IPython.display import Image import warnings warnings . filterwarnings( ' ignore ' ) from rockpool.devices.xylo.syns63300 import Quantizer 37.1 Ov er view of preprocessing chain modules [15]: Image( ' imu-ifsim-module.png ' ) 335 Rockpool, Release 3.0.3 [15]: As descr ibed in The X ylo ™ IMU pr epr ocessing interface , the IMU IF simulation and configuration is encapsulated in the R ockpool modules RotationRemoval , FilterBank , ScaleSpikeEncoder and IAFSpikeEncoder . This notebook sho w s ho w to configure the various logical blocks of the preprocessing c hain f or desired beha viour . 37.2 Specifying rotation correction parameters The rotation remo val bloc k in the IMU inter face continuousl y estimates the sensor orientation in 3-D space, based on the (x, y, z) in put signal. This estimation is smoothed o v er time, with a configurable a v eraging windo w f or the estimation. It then computes a compensating transf or mation f or the (x, y, z) data which cor rects f or the sensor or ientation, such that the gra vity v ector is aligned with the -Z axis. The simulation of this bloc k is embodied b y the RotationRemoval class. Onl y a 3-channel in put and 3-channel output shape is suppor ted. The bloc k offers tw o configurable parameters: • A v eraging windo w f or estimating the sensor orientation (paramter num_avg_bitshift ) • Sample-and-hold duration deter mining ho w often to update the sensor or ientation cor rection (parameter sampling_period ) num_avg_bitshift specifies a lo w-pass filter smoothing f or estimating or ientation, with an appro ximate windo w length of 2 ** num_avg_bitshift time steps. This paramter def aults to 4 (i.e. 16 time steps). sampling_period specifies a sample-and-hold duration f or the or ientation cor rection, such that the cor rection inf or - mation is updated onl y e v er y sampling_period time steps. This parameter def aults to 10 time steps. T o tune these parameters y ou should in v estig ate the dynamics of IMU data under y our use case. Ho w often and ho w quic kly do y ou e xpect a sensor rotation? Ho w impor tant is the inf or mation ar ising from sensor rotation f or y our appli- cation? These aspects will influence ho w slo w l y y ou should cor rect f or sensor rotation, or whether y ou should do so at all. [16]: from rockpool.devices.xylo.syns63300.imuif import RotationRemoval rr = RotationRemoval( num_avg_bitshift = 6 , sampling_period = 50 , ) print (rr) 336 Chapter 37. Specifying IMU preprocessing par ameters Rockpool, Release 3.0.3 RotationRemoval with shape ( 3 , 3 ) { ModSequential ' sub_estimate ' with shape ( 3 , 9 ) { SubSpace ' 0_SubSpace ' with shape ( 3 , 9 ) SampleAndHold ' 1_SampleAndHold ' with shape ( 9 , 9 ) } } [17]: ## - Demonstrate using an IMU data sample # - Load a data sample and ensure it has the correct scale with open ( "data.npy" , "rb" ) as f: data = np . load(f) quantizer = Quantizer( 3 , scale = 0.49 , num_bits = 16 ) data_quantized, _, _ = quantizer(data) # - Apply the rotation removal correction data_corrected, _, _ = rr(data_quantized) # - Plot the IMU sample and rotation corrected sample times = np . arange( len (data)) / 200. plt . figure() plt . plot(times, data_quantized[ 0 ]) plt . plot(times[[ 0 , - 1 ]], [ 0 , 0 ], ' k: ' ) plt . yticks([]) plt . title( f"Quantized IMU Sample Record" ) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Amplitude ' ) plt . figure() plt . plot(times, data_corrected[ 0 ]) plt . plot(times[[ 0 , - 1 ]], [ 0 , 0 ], ' k: ' ) plt . yticks([]) plt . title( f"Rotation Removed IMU Sample Record" ) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Amplitude ' ); 37.2. Specifying rotation correction parameters 337 Rockpool, Release 3.0.3 37.3 Specifying filter -bank par ameters 37.3.1 Specifying a single band-pass filter The classes FilterBank and BandPassFilter are used to specify and simulate a bank of filters f or IMU data. Indi- vidual filters are specified b y pro viding a lo w -pass and high-cut frequency , as w ell as a sampling frequency . On X y lo IMU , data is designed to be sampled at 200~Hz. Here w e will sho w ho w to specify a band-pass filter with a pass-band betw een 5 and 10 Hz, at the def ault sampling fre- quency of 200 Hz. In the code belo w we specify the sampling freq uency e xplicitl y , but to use the default on X ylo ™ IMU y ou can skip this argument. [18]: from rockpool.devices.xylo.syns63300.imuif import FilterBank, BandPassFilter (continues on ne xt page) 338 Chapter 37. Specifying IMU preprocessing par ameters Rockpool, Release 3.0.3 (continued from pre vious pag e) # - Specify a band-pass from 5..10 Hz, sampling at 200 Hz low_cut = 5. # Hz high_cut = 10. # Hz Fs = 200. # Hz # - Instantiate an object for the band-pass filter bpf = BandPassFilter . from_specification(low_cut, high_cut, Fs) print (bpf) BandPassFilter ( B_b = 3 , B_wf = 8 , B_af = 12 , a1 = -59258 , a2 = 27986 , scale_out = 0.5836772581461334 ) W arning N ote that BandPassFilter is not a R oc kpool module, but simpl y a utility class simulating filters. W e can in v estig ate the transf er response of the filter b y sho wing the result of filter ing a chir p signal. T o accomplish this w e use the chirp() function from scipy.signal . W e pass the linear chirp through the single band-pass filter twice, to remo v e any filter onset transients. W e then plot the chirp response and estimate the filter po w er spectrum. [19]: ## - Generate and plot a chirp response # - Import utility functions import numpy as np import scipy.signal as sig # - Generate a 10-second chirp signal ranging 40..0..40 Hz T = 5. times = np . arange( 0 , int (T * Fs)) / Fs chirp = np . array(sig . chirp(times, 0 ,T , 40 ) * 100 , dtype = np . int64) . astype( object ) freq = np . linspace( 0 , 40 , len (times)) times2 = np . concatenate([times, times + times[ - 1 ] + 1 / Fs]) chirp2 = np . concatenate([chirp, np . flip(chirp)]) freq2 = np . concatenate([freq, np . flip(freq)]) # - Pass the signal through the filter resp = bpf(chirp2) # - Plot the chirp and the filter response plt . figure() plt . plot(times, chirp, label = ' Chirp ' ) plt . plot(times, np . flip(resp[ len (times) - 1 : - 1 ]), label = ' Response ' ) plt . legend() plt . title( ' Filter chirp response ' ) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Amplitude ' ); 37.3. Specifying filter -bank par ameters 339 Rockpool, Release 3.0.3 (continued from pre vious pag e) from rockpool.devices.xylo.syns63300 import IMUIFSim # - Instantiate the IMU IF simulation module imuif = IMUIFSim() # - Apply the module to the IMU sample data encoded, _, _ = imuif(data_quantized) # - Plot the result plt . imshow(encoded[ 0 ] . astype( float ) . T, aspect = ' auto ' , interpolation = ' none ' , origin = ˓ → ' lower ' ) xticks, _ = plt . xticks() xlim = plt . xlim() plt . xticks(xticks, xticks / Fs) plt . xlim(xlim) plt . xlabel( ' Time (s) ' ) plt . ylabel( ' Channel ' ) plt . title( ' Encoded IMU signal ' ); 346 Chapter 37. Specifying IMU preprocessing par ameters CHAPTER THIRT YEIGHT O VER VIEW OF D YNAP-SE2 This tutor ial pro vides an o v er vie w of Dynap-SE2 mix ed signal architecture. If y ou ’ re familiar with the c hip and looking f or a hands-on tutor ial, please see Dynap-SE2 Quic k S tar t tutor ial. Other wise, let ’ s deep div e into the c hip! 38.1 Introduction Dynap-SE2, (D Ynamic Neuromor phic Async hronous Processor - ScalablE 2) inher its the e v ent-dr iv en nature of the D YN AP f amil y . The mix ed-signal chip uses analog spiking neurons and analog synapses as the computing units, which directl y emulates biological beha vior . T ransis tors of the neural cores operate in the subthreshold region, which results in po w er consumption of about one-thousandth to one-millionth of the s tate-of-the-ar t digital neuromor phic chips, belo w mW . Each c hip f eatures: • 1024 A dExpIF (adaptiv e e xponential integrate-and-fire) analog ultra-lo w -po w er spiking neurons, • 64 synapses per neuron with configurable dela y , w eight, and shor t-ter m plasticity . [9]: from IPython.display import Image Image( "images/dynapse2.jpeg" ) 347 Rockpool, Release 3.0.3 [9]: In a broad perspectiv e, Dynap-SE g rants a hardw are infrastructure to f acilitate reconfigurable, g eneral-pur pose, real- time analog spiking neural netw ork applications. The no v el e v ent-routing technology of Dynap-SE mak es it possible to de v elop ultra-lo w-po w er and ultra-lo w latency solutions f or edg e computing applications. Also, the advanced, con- figurable neuron and synapse circuitr y pro vides a basis to simulate SNNs with comple x dynamical character is tics in analog hardw are. The figure belo w displa ys an abs tract arc hitecture of the mix ed signal chip. 38.2 Architecture Ov er view [10]: Image( "images/dynapse_architecture.png" ) 348 Chapter 38. Overview of Dynap-SE2 Rockpool, Release 3.0.3 [10]: The neural computation unit is the main building bloc k creating the dynamics. Each neural core pieces tog ether 256 analog neurons shar ing the same parameter set. C AM and SRAM are the digital memor y blocks holding the transmit- ting and receiving e v ent configurations. Analog computation tak es place in the synapses and the neuron soma. F our different synapses: AMP A, G AB A, NMD A, and SHUNT , integrate the incoming ev ents and inject cur rent into the membrane. While AMP A and NMD A produce e x citator y post-synaptic potentiation, G AB A and SHUNT synapses produce inhibitor y post-synaptic potentiation. In other w ords, AMP A and NMD A activ ation increase the chance that the neuron fires; GAB A and SHUNT activ ation decrease the fir ing probability . The listening e v ent setting stored in the C AM ref ers to a synapse type. Theref ore, each of the 64 connections of a neuron can specify its synaptic processing unit. Neuron soma integrates the injection cur rents and holds a temporal state. Charging and disc harging capacitors in 38.2. Architecture Overview 349 Rockpool, Release 3.0.3 configurable paths designates the temporal beha vior . A secondar y reading on the membrane capacitance, the membrane cur rent, functions as the temporal state v ar iable. U pon membrane cur rent reaching the firing threshold, the neuron ’ s reset mechanism s teps in and tr iggers the e v ent sensing units. The e v ent is packag ed in AER f or mat and is broadcasted to indicated locations. In this w a y , the neuron computes the dynamics using analog sub-threshold circuits but conv e y s the resulting outputs using a digital routing mechanism. 38.3 Parameter Handling Each neural core holds a parameter group to set the neuronal and synaptic parameters f or its 256 neurons and their pre-synaptic synapses. The neurons in the same core share the same parameter v alues, including time constants, refrac- tor y per iods, synaptic connection strengths and etc. Special digital to analog con v eters, bias g enerators denominated as BG , set these parameter cur rent v alues. In total there are 70 parameters setting different beha vioral attr ibutes of the neurons and synapses; including time constants, pulse widths, amplifier g ain ratios and synaptic w eight s trengths. [11]: Image( "images/param_conversion.png" ) [11]: F or details of the circuits please chec k: • Synapse Circuitr y – C. Bar tolozzi and G. Indiv er i, “Synaptic Dynamics in Analog VLSI, ” in Neural Computation, v ol. 19, no. 10, pp. 2581-2603, Oct. 2007, doi: 10.1162/neco.2007.19.10.2581. • Neuron Membrane Circuitry – P . Livi and G. Indiv er i, “ A cur rent-mode conductance-based silicon neuron f or address-ev ent neuromor - phic sy s tems, ” 2009 IEEE Inter national Symposium on Circuits and Sys tems, 2009, pp. 2898-2901, doi: 10.1109/ISC AS.2009.5118408. • Bias Generator Circuitr y – T . Delbr uck, R. Berner , P . Lichts teiner and C. Dualibe, “32-bit Configurable bias cur rent g enerator with sub-off-cur rent capability , ” Proceedings of 2010 IEEE Inter national Symposium on Circuits and Sy s- tems, 2010, pp. 1647-1650, doi: 10.1109/ISC AS.2010.5537475. 350 Chapter 38. Overview of Dynap-SE2 Rockpool, Release 3.0.3 38.4 Routing Each Dynap-SE2 c hip has 1024 neurons distr ibuted o v er 4 individuall y configurable neural cores, connected b y a patented hierarchical routing grid. The tag-based routing infrastructure pro vides direct communication from one c hip to 15 × 15 sur rounding chips (7 s teps w est, 7 s teps nor th, 7 steps eas t, 7 steps south), connecting up to 230k neurons. • Each neuron has 64 f an-in (neuron) and 4 f an-out (neural core) capacity . • Synapses are not uniquel y addressed, instead vir tual addresses named tags are used to specify connections. – 11-bit globall y mutliple x ed locally uniq ue/mutliple x ed tags F or details please chec k: • S. Moradi, N . Qiao, F . Stef anini and G. Indiv er i, “ A Scalable Multicore Architecture With Heterog eneous Memor y S tr uctures f or Dynamic Neuromorphic Asynchronous Processors (D YN APs), ” in IEEE T ransac- tions on Biomedical Circuits and Sy s tems, v ol. 12, no. 1, pp. 106-122, F eb. 2018, doi: 10.1109/TB- C AS.2017.2759700 [12]: Image( "images/router.png" ) [12]: The neural computation unit is the main building bloc k creating the dynamics. Each neural core pieces tog ether 256 analog neurons shar ing the same parameter set. C AM and SRAM are the digital memor y blocks holding the transmit- ting and receiving e v ent configurations. Analog computation tak es place in the synapses and the neuron soma. F our different synapses: AMP A, G AB A, NMD A, and SHUNT , integrate the incoming ev ents and inject cur rent into the membrane. While AMP A and NMD A produce e x citator y post-synaptic potentiation, G AB A and SHUNT synapses produce inhibitor y post-synaptic potentiation. In other w ords, AMP A and NMD A activ ation increase the chance that the neuron fires; GAB A and SHUNT activ ation decrease the fir ing probability . The listening e v ent setting stored in the C AM ref ers to a synapse type. Theref ore, each of the 64 connections of a neuron can specify its synaptic processing unit. Neuron soma integrates the injection cur rents and holds a temporal state. Charging and disc harging capacitors in configurable paths designates the temporal beha vior . A secondar y reading on the membrane capacitance, the membrane cur rent, functions as the temporal state v ar iable. U pon membrane cur rent reaching the firing threshold, the neuron ’ s reset mechanism s teps in and tr iggers the e v ent sensing units. The e v ent is packag ed in AER f or mat and is broadcasted to indicated locations. In this w a y , the neuron computes the dynamics using analog sub-threshold circuits but conv e y s the resulting outputs using a digital routing mechanism. 38.4. Routing 351 Rockpool, Release 3.0.3 38.5 Simulation Dynap-SE simulator uses the anal ytical transf er functions of the analog VLSI neuron&synapse implementations and sol v es them in time. It uses Forward-Euler method, which is one of the oldes t and simplest algor ithms to sol v e first- order ordinar y differential equations giv en an initial v alue. This wa y , giv en the cur rent time step s tate, it predicts the ne xt time step s tate iterativ el y . [13]: Image( "images/spiking_neuron_operation.png" ) [13]: The simulator DynapSim , pro vides an abs tract Dynap-SE machine that operates in the same parameter space as Dynap- SE f amil y processors. DynapSim does not simulate the hardware numericall y precisel y but e x ecutes a f ast and an appro ximate simulation. It uses f or ward Euler updates to predict the time-dependent dynamics and sol v es the charac- ter istic circuit transf er functions in time. In principle, the de vice and the simulator do not react e xactl y the same to the same input. Due to the nature of the analog de vice mismatc h problem, tw o ph ysical c hips w ould not react the same as w ell, it should not create a problem in application de v elopment. The netw orks to be deplo y ed to a chip should be robust ag ainst parameter v ar iations. T o be able to stress this more, DynapSim pro vides an integrated mismatch simulation f eature that can alter the parameter projection. Operating an appro ximate simulation that has the potential to g eneralize Dynap-SE processors has adv antages o v er a bulky realistic simulation. First, it runs a lot f aster than a transis tor -lev el accurate simulator . It mak es the optimization pipeline ter minate in f easible time windo ws. Second, the DynapSim methodology does not rely on e xact v alues; it f ocuses on reproducing a beha vior minimizing the dependence on v alues. The values here ref er to the e xact timing of the spik es, the v oltag e & cur rent amplitudes, and ev en the time cons tants. This w a y , abstract mac hine w ould ha v e a g eneralization capacity ins tead of o v er fitting to a specific chip la y out. Onl y a simulator with g eneralization capability could br ing neuromor phic mix ed-signal applications to our dail y liv es. 38.6 N e xt steps • Simulate a Dynap-SE2 netw ork and deplo y that netw ork to a Dynap-SE2 HDK, see Quick S tar t with Dynap-SE2 • In v es tig ate the neuron model implementation, see DynapSim N euron Model • T rain an SNN that is aimed to be deplo y ed to a Dynap-SE2 HDK, see T r aining a spiking netw ork t o deplo y to Dynap-SE2 352 Chapter 38. Overview of Dynap-SE2 CHAPTER THIRT YNINE QUICK ST ART WITH D YNAP-SE2 This notebook giv es y ou a quic k o v erview of taking a netw ork from a high-le v el Python simulator through to deplo yment on Dynap-SE2 chip. The mater ial is designed to be independent of the rest of the Dynap-SE2 tutorials. If y ou ha v e heard of R ockpool and Dynap-SE2 chip bef ore, y ou ’ re good to go! This one f ocus on hands-on e xper ience without going into too much detail. P ar ticularl y this notebook sho ws ho w to: 1. Build a Dynap-SE2 compatible netw ork with R ockpool , using DynapSim and LinearJax la y ers. 2. Extract the computational graph f or that netw ork containing all parameters needed to specify the c hip config- uration. 3. Map the computational graph to Dynap-SE2 hardware, allocating hardw are resources wisel y . 4. Quantize the netw ork parameters to make sure that the y fit. 5. Connect and Configure the chip, deplo y the netw ork. 6. Simulate on Dynap-SE2 and read the results. 7. Simulate on CPU and see ho w the netw ork affected b y all those transf or mations and quantizations affected the netw ork beha viour . 39.1 Impor ts [1]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) import numpy as np # - Rockpool imports from rockpool.nn.modules.jax import LinearJax from rockpool.nn.combinators import Sequential from rockpool.timeseries import TSEvent, TSContinuous # - Dynap-SE imports from rockpool.devices.dynapse import ( DynapSim, mapper, autoencoder_quantization, config_from_specification, (continues on ne xt page) 353 Rockpool, Release 3.0.3 (continued from pre vious pag e) find_dynapse_boards, DynapseSamna, dynapsim_net_from_config, ) # - Tutorial Utils from doc_utils import ( FrozenNoiseDataset, poisson_spike_train, visualize_device_sim, plot_Ix, plot_model_response, plot_model_response_histogram, ) # - Plotting and config import sys !{ sys.executable } -m pip install --quiet matplotlib import matplotlib.pyplot as plt plt . rcParams[ "figure.figsize" ] = [ 9.6 , 3.6 ] plt . rcParams[ "figure.dpi" ] = 1200 plt . rcParams[ "font.size" ] = 12 39.2 T utorial Utils [2]: % psource FrozenNoiseDataset class FrozenNoiseDataset : """ FrozenNoise is a synthetic dataset implementation for testing DynapSE-2 simulation ␣ ˓ → pipeline. It generates possion spike train rasters :param n_samples: number of samples included in the dataset :type n_samples: int :param n_channels: number of spiking channels (input neurons), defaults to 60 :type n_channels: int, optional :param duration: the duration of each synthetic recording, defaults to 500e-3 :type duration: float, optional :param dt: The discrete time resolution of the recording, defaults to 1e-3, defaults ␣ ˓ → to 1e-3 :type dt: float, optional :param rate: mean firing rate in Hz (applies to all channels), defaults to 50 :type rate: float, optional :param seed: random number generator seed, defaults to 2022 :type seed: Optional[float], optional """ def __init__ ( (continues on ne xt page) 354 Chapter 39. Quick Star t with Dynap-SE2 Rockpool, Release 3.0.3 (continued from pre vious pag e) self , n_samples : int , n_channels : int = 60 , duration : float = 500e-3 , dt : float = 1e-3 , rate : float = 50 , seed : Optional [ float ]= 2022 , )- > None : """__init__ parameters explained in class header""" self . n_in = n_channels self . n_out = n_samples self . dt = dt self . input_raster = poisson_spike_train ( n_channels , duration , rate , dt , batch_size = n_samples , seed = seed ) # One hot encoded target labels self . labels = np . expand_dims ( np . identity ( n_samples , dtype = float ), 1 ) def __getitem__ ( self , index : int )- > Tuple [ np . ndarray ]: """ __getitem__ [] getter implementation :param index: the sample index :type index: int :return: data, label :data: a single sample, raster :label: one hot encoded class of the sample :rtype: Tuple[np.ndarray] """ return self . input_raster [ index ], self . labels [ index ] @ property def full_batch ( self )- > Tuple [ np . ndarray ]: return self [:, :, :] def __len__ ( self )- > int : """__len__ returns the number of samples stored""" return len ( self . input_raster ) def plot_samples ( self , idx : Optional [ List [ int ] ]= None , adjust_size : bool = False )- > None : """ plot_samples visualizes the samples indicated by the idx list, stored in dataset :param idx: the index list of samples to be visualized, defaults to None :type idx: Optional[List[int]], optional :param adjust_size: adjust the size of the resulting plot accordingly or not, ␣ ˓ → defaults to False :type adjust_size: bool, optional """ (continues on ne xt page) 39.2. T utorial Utils 355 Rockpool, Release 3.0.3 39.4.2 Step 2.2 : Run N o w let ’ s r un the model and compare the responses in ter ms of fir ing rate ratio. The firing rate ratio ( FRR ) is the ratio betw een the super ior and inf er ior mean fir ing rates read from the decision neurons. FRR is calculated as f ollo ws: 𝐹 𝑅 𝑅 = 𝑟 superior 𝑟 inf erior If y ou ha v e visited the last tutorial, y ou probably realized that the FRR readings are w orse in this case, e v en though w e ha v e used the same trained w eights. Y ou ’ re right! That ’ s because w e initiated the simulator with a mismatch parameter in this case and it de viated the netw ork parameters a bit. It should be OK as long as it does not c hang e the high lev el beha viour of the netw ork. That is one neuron is firing at a clear l y higher rate than the other . W e’ll f ace this issue using different chips as w ell. [11]: plot_model_response(net, train_data, dt) It ’ s seen that neuron 0 fires at a significantly higher rate than the neuron 1 giv en the frozen noise sample 0. F or frozen noise 1, neuron 1 beats neuron 0. Let ’ s chec k the tes t samples. [12]: plot_model_response_histogram(net, test_data) Histogram: 100 iteration [00:17, 5.85 iteration/s] 362 Chapter 39. Quick Star t with Dynap-SE2 Rockpool, Release 3.0.3 This histogram sho w s us that the fir ing rates betw een neurons are much more greater than 1 onl y in the case that the training sample 1 or the training sample 2 pro vided. In all other cases, the FRR is close to 1, meaning that the netw ork does not e xpress a clear decision on the 39.5 Step 3 : Extract the gr aph and Map In this step, w e e xtract the computational g raph of the entire netw ork and tr y to allocate hardw are resources. The chip has multiple cores that w e can allocate neurons. The mapper finds the mos t suitable cores f or neurons depending on neurons ’ parameters. Each neural core has a different parameter configuration. T o obtain a g raph descr ibing the entire netw ork, whic h contains the computational flo w of inf or mation through the netw ork as w ell as all parameters, w e simply use the :p y :meth:‘ .Module.as 𝑔 𝑟 𝑎𝑝ℎ ‘ 𝑚𝑒𝑡ℎ𝑜𝑑. F or more inf or mation, see :ref:‘/adv anced/graph 𝑜 𝑣 𝑒𝑟 𝑣 𝑖𝑒𝑤 .𝑖𝑝𝑦 𝑛𝑏 ‘ . [13]: spec = mapper(net . as_graph()) # The behavior of the circuit depends on the temprature. For now, just scale the Iscale ␣ ˓ → parameter manually to make the network fire more or less. # In future, we ' ll omit this! spec[ "Iscale" ] *= 50 39.6 Step 4 : Quantize W eights The w eight matr ices stored inside the la y ers are allo w ed to g et any v alue in the simulation. Ho w ev er , the hardw are does not ha v e a free-of-choice w eight setting f eature. Only a 4-bit res tricted connection-specific weight assignment is possible. While deplo ying a netw ork to Dynap-SE2, w eight matr ices can be con v er ted to a device configuration through a quantization phase. F or Dynap-SE2, netw ork w eight configuration is a tw o-f old process. 4 base w eight parameters define the basis of the inner product space. Connection-specific digital memory cells store the 4-bit binary w eight masks, producing the e xact w eight cur rent that the synapse uses. The object of quantization is to find a base w eight cur rent v ector and a binar y bit-mask matr ix such that the y together reconstruct the desired w eight matrix with minimum deviation. A popular unsuper vised machine learning method, A utoEncoder str ucture, is used f or this. In this approach, intermediate code representation indicates the base weight cur rents, and the decoder w eight matrix giv es binar y bit-masks. 39.5. Step 3 : Extract the graph and Map 363 Rockpool, Release 3.0.3 [14]: from IPython.display import Image Image( "images/weight_quantization.png" , width = 960 ) [14]: The objectiv e of the unsuper vised A utoEncoder training is to find a hardw are configuration that reproduces the targ et w eight matr ix with minimum deviation. Mean square er ror approac h is used to calculate the matr ix reconstruction loss. The reconstruction loss simpl y computes the difference betw een absolute values of the original w eight matrix and the reconstructed v ersion. It takes the sq uares of differences of each cell and returns the mean v alue. 𝑓 𝑀 𝑆 𝐸 ( 𝑊 𝑄 , 𝑊 ) = 1 𝑁 · 𝑀 𝑁 ∑︁ 𝑖 =0 𝑀 ∑︁ 𝑗 =0 ‖ 𝑊 𝑄 [ 𝑖, 𝑗 ] − 𝑊 [ 𝑖, 𝑗 ] ‖ 2 [15]: spec . update(autoencoder_quantization( ** spec)) 39.7 Step 5 : Obtain Hardware Config W e no w con v er t the netw ork specification to a hardw are configuration object. This is fundementall y a samna object that encapsulates the hardw are configuration parameters. The input c hannel map pro vides the mapping betw een in put timeser ies channels and the des tinations which will helps us to map in fur ther stag es. Deplo ying a netw ork to the chip requires translating the neuron and synapse parameters and finding a netw ork connec- tivity configuration. The parameter translation means finding a bias g enerator setting that e xpresses the cur rent values closel y . The cur rent v alues used in the simulator are translated to coarse and fine values using bias g enerator look up tables. It requires finding the closes t possible cur rent v alue that the bias g enerator could pro vide with a vailable coarse and fine v alues. The optimized w eight matr ices ref er to a connectivity configuration but could not be applied to the c hip directl y . The quantization procedure finds the bes t possible base weight currents and w eight masks recons tr ucting the w eight matr i- ces. The w eight masks are dispatched to digital memory blocks embedded in individual neural units. The base weights are translated to coarse-fine v alues similar to other bias parameters. 364 Chapter 39. Quick Star t with Dynap-SE2 Rockpool, Release 3.0.3 The w eight values are allo w ed to be negativ e or positiv e in the simulation. Ho w ev er , the sign does not cor respond to an actual attr ibute in the hardw are. The sign is accepted to represent the synapse ’ s e x citation or inhibition beha vior and is used to deter mine the synapse type. If the value is neg ativ e, the configuration pipeline chooses the inhibitory GAB A synapse; if the v alue is positiv e, it sets up an e x citatory AMP A synapse. In this wa y , the neg ativ e and positiv e w eights produce the desired effect on the post-synaptic membrane. [16]: config = config_from_specification( ** spec) 39.8 Step 6 : Connect to De vice [17]: ## Connect to device se2_devices = find_dynapse_boards() found_se2 = len (se2_devices) > 0 if not found_se2: raise IOError ( "This tutorial requires a connected Dynap-SE2 Stack Board HDK to demonstrate." ) else : se2 = DynapseSamna(se2_devices[ 0 ], ** config) 39.9 Step 7 : Run the Simulation on Dynap-SE2 The netw ork parameters and quantized w eights are used to configure a hardw are netw ork on the Dynap-SE2 chip. The frozen noise patter ns are con v er ted to real-time AER sequences. Each e v ent sho wn in the noise patter n is con v er ted to a data pac kag e encapsulating the time and the address. In this wa y , the time ser ies can dr iv e a real-time de vice emulation. An on-board FPG A circuit con v er ts AER e v ents to digital pulses that stimulate the synaptic in put g ates of the neurons. Analog neurons process the in puts and produce spik es cor respondingl y . Whenev er a neuron fires, digital circuits on FPG A senses the e v ent and encapsulate the timestamp and the source address as an AER e v ent. These output AER e v ents are temporar il y s tored in buffers implemented inside FPG A, w aiting f or real-time reading. The output of the emulation is recorded as AER e v ent sequences and visualized in a similar w a y that the other e v ent sequences are visualized. The figures sho w us that the netw ork’ s response matches the simulated e xpectations. [18]: out, state, rec = se2(train_data[ 0 ][ 0 ], record = True ) visualize_device_sim(train_data[ 0 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> 39.8. Step 6 : Connect to De vice 365 Rockpool, Release 3.0.3 [19]: out, state, rec = se2(train_data[ 1 ][ 0 ], record = True ) visualize_device_sim(train_data[ 1 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> 39.9.1 Step 7.1 : N egativ e T ests Let ’ s f eed the tes t data to the de vice and see that there is no significant difference betw een neurons ’ output fir ing patter ns. [20]: out, state, rec = se2(test_data[ 0 ][ 0 ], record = True ) visualize_device_sim(test_data[ 0 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> 366 Chapter 39. Quick Star t with Dynap-SE2 Rockpool, Release 3.0.3 [21]: out, state, rec = se2(test_data[ 1 ][ 0 ], record = True ) visualize_device_sim(test_data[ 1 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> [22]: out, state, rec = se2(test_data[ 2 ][ 0 ], record = True ) visualize_device_sim(test_data[ 2 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> 39.9. Step 7 : Run the Simulation on Dynap-SE2 367 Rockpool, Release 3.0.3 [23]: out, state, rec = se2(test_data[ 3 ][ 0 ], record = True ) visualize_device_sim(test_data[ 3 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> [24]: out, state, rec = se2(test_data[ 4 ][ 0 ], record = True ) visualize_device_sim(test_data[ 4 ][ 0 ], out, rec, config[ "input_channel_map" ], se2 . dt); <Figure size 11520x4320 with 0 Axes> 368 Chapter 39. Quick Star t with Dynap-SE2 Rockpool, Release 3.0.3 39.10 Step 8 : Simulate the Quantized N etwork It ’ s also possible to reconstruct the simulator from a configuration object and run the simulation to obser v e the effects of the quantization and parameter translation. S teps here are no different than the simulating a rockpool netw ork. [25]: net_quantized = dynapsim_net_from_config( ** config) # Remember, we scaled the Iscale previously, now we should take this back! net_quantized[ 1 ] . Iscale /= 50 [26]: plot_model_response(net_quantized, train_data, dt) 39.10. Step 8 : Simulate the Quantized Network 369 Rockpool, Release 3.0.3 [27]: plot_model_response_histogram(net_quantized, test_data) Histogram: 100 iteration [00:17, 5.70 iteration/s] 370 Chapter 39. Quick Star t with Dynap-SE2 CHAPTER F ORT Y D YNAPSIM NEURON MODEL This tutor ial pro vides an o v er vie w of Dynap-SE2 neuron & synapse simulation and sur rog ate gradient function imple- mentation. W e will in v es tig ate: 1. A theoretical o v er view of silicon synapse and membrane circuitry . 2. Ho w to build an SNN with DynapSim . 3. A single neuron response to an e x poisson spike train s timulation 4. The sur rogate gradient implementation, whic h can be used to train a netw ork with backpropag ation. [1]: from IPython.display import Image 40.1 Section 1 : N euron Model In this chapter , a behavioral in v estig ation of silicon neuron and silicon synapse circuits presented. F or detailed anal y ses of the circuits, please ref er to: • Synapse Circuitr y – C. Bar tolozzi and G. Indiv er i, “Synaptic Dynamics in Analog VLSI, ” in Neural Computation, v ol. 19, no. 10, pp. 2581-2603, Oct. 2007, doi: 10.1162/neco.2007.19.10.2581. • Neuron Membrane Circuitry – P . Livi and G. Indiv er i, “ A cur rent-mode conductance-based silicon neuron f or address-ev ent neuromor - phic sy s tems, ” 2009 IEEE Inter national Symposium on Circuits and Sys tems, 2009, pp. 2898-2901, doi: 10.1109/ISC AS.2009.5118408. • Anal y sis of the Silicon Neuron and Synapse Circuits – E. Chicca, F . S tef anini, C. Bar tolozzi and G. Indiv er i, “Neuromorphic Electronic Circuits f or Building A utonomous Cognitiv e Sy stems, ” in Proceedings of the IEEE, v ol. 102, no. 9, pp. 1367-1388, Sept. 2014, doi: 10.1109/JPR OC.2014.2313954. Silicon neuron and synapse implementations f or m a basis f or the realization of computational neural models through adjusting some bias v oltag es and cur rents. The analy sis presented in this chapter sho w s ho w the higher -le v el h yper - parameters relate to lo w -le v el de vice v oltages and cur rents. The simulator’ s job is to translate the beha vioral dynamics of a computational neural setting into the VLSI parameters of the respectiv e circuits. 371 Rockpool, Release 3.0.3 (continued from pre vious pag e) raster = np . random . poisson ( rate * dt , ( batch_size , steps , n_channels )) # Check if raster has at least one spike if not any ( raster . flatten ()): raise ValueError ( "No spike generated at all due to low firing rate or short simulation time ␣ ˓ → duration!" ) spike_tensor = np . array ( raster , dtype = float ) return spike_tensor [25]: % psource plot_Ix def plot_Ix ( Ix_record : np . ndarray , Ithr : Optional [ Union [ float , np . ndarray ] ]= None , dt : float = 1e-3 , name : Optional [ str ]= None , idx_map : Optional [ Dict [ int , NeuronKey ] ]= None , margin : Optional [ float ]= 0.2 , ax : Optional [ matplotlib . axes . Axes ]= None , line_ratio : float = 0.3 , ylabel : str = "Current (A)", * args , ** kwargs , )- > TSContinuous : """ plot_Ix converts an ` Ix_record ` current measurements/recordings obtained from the ␣ ˓ → record dictionary to a ` TSContinuous ` object and plot :param Ix_record: Membrane or synapse currents of the neurons recorded with respect ␣ ˓ → to time (T,N) :type Ix_record: np.ndarray :param Ithr: Spike threshold or any other upper threshold for neurons. Both a single ␣ ˓ → float number for global spike threshold and an array of numbers for neuron-specific ␣ ˓ → thresholds can be provided. Plotted with dashed lines if provided, defaults to None :type Ithr: Optional[float], optional :param dt: The discrete time resolution of the recording, defaults to 1e-3 :type dt: float, optional :param name: title of the figure, name of the ` TSContinuous ` object, defaults to None :type name: str, optional :param idx_map: a dictionary of the mapping between matrix indexes of the neurons ␣ ˓ → and their global unique neuron keys, defaults to None :type idx_map: Optional[Dict[int, NeuronKey]], optional :param margin: The margin between the edges of the figure and edges of the lines, ␣ ˓ → defaults to 0.2 :type margin: Optional[float], optional :param ax: The sub-plot axis to plot the figure, defaults to None :type ax: Optional[matplotlib.axes.Axes], optional :param line_ratio: the ratio between Imem lines and the Ispkthr lines, defaults to 0. ˓ → 3 (continues on ne xt page) 378 Chapter 40. DynapSim N euron Model Rockpool, Release 3.0.3 (continued from pre vious pag e) :type line_ratio: float, optional :param ylabel: ylabel value to be printed :type ylabel: str, optional :return: Imem current in ` TSContinuous ` object format :rtype: TSContinuous """ f_margin = 1.0 + margin if margin is not None else 1.0 if ax is not None : plt . sca ( ax ) # Convert and plot Ix = TSContinuous . from_clocked ( Ix_record , dt = dt , name = name ) _lines = Ix . plot ( stagger = np . float32 ( Ix . max * f_margin ), * args , ** kwargs ) plt . ylabel ( ylabel ) if idx_map is not None : ax = plt . gca () handles , labels = ax . get_legend_handles_labels () ax . legend ( handles [::- 1 ], [f"n[{n_key}]" for n_key in idx_map . values ()][::- 1 ], bbox_to_anchor =( 1.05 , 1.05 ), ) plt . tight_layout () # Upper threshold lines if Ithr is not None : linewidth = _lines [ 0 ]. _linewidth * line_ratio Ithr = np . ones_like ( Ix_record )* Ithr Ithr = TSContinuous . from_clocked ( Ithr , dt = dt ) Ithr . plot ( stagger = np . float32 ( Ix . max * f_margin ), linestyle ="dashed", linewidth = linewidth , ) return Ix [26]: % psource split_yaxis def split_yaxis ( top_ax : matplotlib . axes . Axes , bottom_ax : matplotlib . axes . Axes , top_bottom_ratio : Tuple [ float ], )- > None : """ split_yaxis arrange ylimits such that two different plots can share the same y axis ␣ ˓ → without any intersection :param top_ax: the axis to place on top (continues on ne xt page) 40.2. Section 2 : Simulation 379 Rockpool, Release 3.0.3 (continued from pre vious pag e) :type top_ax: matplotlib.axes.Axes :param bottom_ax: the axis to place on bottom :type bottom_ax: matplotlib.axes.Axes :param top_bottom_ratio: the ratio between top and bottom axes :type top_bottom_ratio: Tuple[float] """ def arrange_ylim ( ax : matplotlib . axes . Axes , place_top : bool , factor : float )- > None : """ arrange_ylim helper function to arrange y_limits :param ax: the axis to change the limits :type ax: matplotlib.axes.Axes :param place_top: place the axis of interest to top or bottom :type place_top: bool :param factor: the factor to multiply the y-range and allocate space to the ␣ ˓ → other plot :type factor: float """ bottom , top = ax . get_ylim () if place_top : bottom = bottom - factor * ( top - bottom ) else : top = top + factor * ( top - bottom ) ax . set_ylim ( top = top , bottom = bottom ) f_top = top_bottom_ratio [ 1 ]/ top_bottom_ratio [ 0 ] f_bottom = top_bottom_ratio [ 0 ]/ top_bottom_ratio [ 1 ] arrange_ylim ( top_ax , 1 , f_top ) arrange_ylim ( bottom_ax , 0 , f_bottom ) 40.2.3 Step 1 : N etw ork Configuration Configure an SNN, no different than configur ing an y other netw ork in R ockpool. T o recall, please ref er to : Getting S tar ting with R ockpool. [27]: model = Sequential( LinearJax(shape = ( 1 , 1 ), weight = np . array([ 0.1 ]), has_bias = False ), DynapSim(( 1 , 1 ), has_rec = False ), ) model [27]: JaxSequential with shape (1, 1) { LinearJax ' 0_LinearJax ' with shape (1, 1) DynapSim ' 1_DynapSim ' with shape (1, 1) } 380 Chapter 40. DynapSim N euron Model Rockpool, Release 3.0.3 40.2.4 Step 2 : In put Spik e T rain Generate a random poisson spik e train with mean frequency 20 Hz [28]: dt = 1e-3 spike_raster = poisson_spike_train( n_channels = 1 , duration = 1.0 , rate = 20.0 , dt = dt, seed = 2022 ) . astype( bool ) spike_ts_in = TSEvent . from_raster(spike_raster[ 0 ], dt = dt, name = "Input Spike Train" ) spike_ts_in . plot() plt . tight_layout() 40.2.5 Step 3 : Run Simulate the netw ork while recording the inter mediate steps [29]: model . reset_state() out, state, record_dict = model(spike_raster, record = True ) 40.2.6 Step 4 : Analyze the r esults Get the synapse and membrane responses from the record dictionar y , plot the figure. [30]: spike_ts_out = TSEvent . from_raster(out[ 0 ], dt = dt, name = "Output Spike Train" ) spike_ts_out . plot() plt . tight_layout() 40.2. Section 2 : Simulation 381 Rockpool, Release 3.0.3 Plot the membrane potential reading and synaptic injection cur rent on the figure. [32]: # Plot the vmem and isyn on the same figure fig, ax1 = plt . subplots() ax2 = ax1 . twinx() plot_Ix( record_dict[ "1_DynapSim" ][ "vmem" ][ 0 ], ylabel = "Voltage (V)" , ax = ax1, label = "$V_ {mem} $" , ) plot_Ix(record_dict[ "1_DynapSim" ][ "isyn" ][ 0 ], ax = ax2, color = "red" , label = "$I_ {syn} $" ) split_yaxis(ax1, ax2, ( 2 , 1 )) plt . tight_layout() A t first glance, the response is similar to the response of the Leaky Integrate and Fire neuron response presented in the simple introduction tutor ial. Indeed, with small dispar ities, the y are pretty close to each other . In both cases, the synaptic cur rent instantl y increases when a spike arr iv es and leaks consis tentl y . One significant difference in DPI response is the shor t-ter m f acilitation; the jump amount depends on the synaptic state. If the cur rent v alue 𝐼 𝑠𝑦 𝑛 is sufficientl y greater than 𝐼 𝑔 𝑎𝑖𝑛 , the jump is more prominent, and else it ’ s depressed. Also, w e can see the effect of positiv e f eedback circuitry that the more the membrane capacitor is char g ed, the bigger the jumps w e obser v e. 382 Chapter 40. DynapSim N euron Model Rockpool, Release 3.0.3 F or this specific e x ecution, the la y out parameters: capacitance, ther mal v oltag e, and subthreshold slope f actor , are all k ept at their def ault v alues. The default v alues of the parameters pro vide a smooth operation region. 40.3 Section 3 : Spike Generation Logic R emember that the common f eature of e v er y spiking neuron is declared as that the y hold the temporal s tate inf or mation and produce a spik e when the state threshold cons traints are satisfied. As a computatioanl neuron model, the Dynap- SE neuron synapse simulation bloc k, stores the temporal state in 𝐼 𝑚𝑒𝑚 current and compare the 𝐼 𝑚𝑒𝑚 cur rent with spik e threshold cur rent 𝐼 𝑠𝑝𝑘 𝑡ℎ𝑟 to produce a spike. Ho w e v er , the naiv e implementation of this conditional logic mak es it troublesome to optimize a spiking neural netw ork using gradient based optimization mechanisms. Instead, a Hea viside- step function with cus tom g radient r ules is implemented to decide on spik e g eneration at eac h time step. [33]: % psource step_pwl @ custom_jvp def step_pwl ( imem : jnp . DeviceArray , Ispkthr : jnp . DeviceArray , Ireset : jnp . DeviceArray , max_spikes_per_dt : int = jnp . inf , )- > float : """ step_pwl implements heaviside step function with piece-wise linear derivative to use ␣ ˓ → as spike-generation surrogate :param imem: Input current to be compared for firing :type imem: jnp.DeviceArray :param Ispkthr: Spiking threshold current in Amperes :type Ispkthr: jnp.DeviceArray :param Ireset: Reset current after spike generation in Amperes :type Ireset: jnp.DeviceArray :return: number of spikes produced :rtype: float """ spikes = jnp . ceil ( jnp . log ( imem / Ispkthr )) n_spikes = jnp . clip ( spikes , 0.0 , max_spikes_per_dt ) return n_spikes 40.3.1 Section 3.1 Set the parameters Spiking threshold cur rent and reset cur rent [34]: # Currents in Amperes Ispkthr = 1e-6 Ireset = 5e-13 40.3.2 Section 3.2 Scan the Parameter Space Generate a log ar itmic space and record the output of the spik e g eneration function step_pwl at each time s tep 40.3. Section 3 : Spike Generation Logic 383 Rockpool, Release 3.0.3 [35]: Imem_space = np . logspace(start =- 14 , stop =- 3 , num = 10000 ) n_spikes_trace = [step_pwl(Imem, Ispkthr, Ireset) for Imem in Imem_space] [36]: # Plotting fig = plt . figure() plt . semilogx(Imem_space, n_spikes_trace, label = "# spikes" ) plt . axvline(Ispkthr, linestyle = "dashed" , color = "red" , label = "$I_ {spkthr} $" ) # Labeling plt . xlabel( "$I_ {mem} $" ) plt . ylabel( "Number of Spikes" ) plt . legend() plt . tight_layout() The x axis represents the membrane cur rent and the y axis represents the number of spikes produced. The actual circuit produces a spik e compar ing the membrane potential 𝑉 𝑚𝑒𝑚 and the spik e threshold parameter set 𝑉 𝑠𝑝𝑘𝑡ℎ𝑟 . So, although it ’ s ph y sicall y not possible, if 𝑉 𝑚𝑒𝑚 doubles the 𝑉 𝑠𝑝𝑘𝑡ℎ𝑟 then tw o spik es at the same time w ould ha v e to be produced. In the simulation, there is no har m to go be y ond the ph y sical limitations being a ware the conseq uences. Easing off the restrictions makes a broader parameter space visible. The phy sical reality could be compelled using regular ization techniq ues when necessary . In the subthreshold operation region, the relation betw een cur rent v alues and respectiv e base v oltages is e xponential. Theref ore, doubling the potential cor responds to squar ing the cur rent. R especting this, the step-function that is imple- mented requires that the current should be one order of magnitude higher than the spike threshold current in order f or multi-spik e production. A ccordingl y , the wa v ef or m pro vided in the figure resembles a linear s taircase in the log-scaling. The e xact equation producing this thresholding mec hanism is giv en as f ollo ws: num spik es = ⌈︂ ln (︂ 𝐼 𝑚𝑒𝑚 𝐼 𝑠𝑝𝑘 𝑡ℎ𝑟 )︂⌉︂ 40.4 Section 4 : Surrog ate Gradient F unction Although the functionality introduced up to this point is sufficient to build and e x ecute spiking neural netw orks using the Dynap-SE neuron model, it ’ s not enough to optimize a netw ork efficientl y . In order to run a g radient-based optimization algor ithm, this neuron model requires a surrogate function. Chec k these ref erences f or more about sur rogate gradient approach: 384 Chapter 40. DynapSim N euron Model Rockpool, Release 3.0.3 • J. Lee, T . Delbr ück, and M. Pf eiffer . T raining deep spiking neural netw orks using backpropag ation. Frontiers in Neuroscience, 10, 2016 • E. O. N eftci, H. Mostaf a, and F . Zenk e. Sur rog ate gradient lear ning in spiking neural netw orks: Br inging the po w er of gradient-based optimization to spiking neural netw orks. IEEE Signal Processing Mag azine, 36(6):51–63, 2019 Broadl y speaking, it addresses the problem that the spiking neurons deliv er discrete outputs using an indifferentiable threshold function, which mak es it impossible to bac kpropag ate the er ror . In the con v entional bac kpropag ation ap- proach, the c hain r ule is applied f or er ror credit assignment. In spiking neurons, the output is a spike train and bac k - propag ating the er ror to pre vious la y er requires taking the deriv ativ e of the threshold function. In Dynap-SE neuron implementation, taking the der iv ativ e of the output spike train with respect to a parameter that specify the membrane cur rent dynamics looks like the f ollo wing. 𝜕 𝑆 𝑜𝑢𝑡 ( 𝑡 ) 𝜕 𝑃 = 𝜕 Θ( 𝐼 𝑚𝑒𝑚 , 𝐼 𝑠𝑝𝑘 𝑡ℎ𝑟 ) 𝜕 𝐼 𝑚𝑒𝑚 · 𝜕 𝐼 𝑚𝑒𝑚 𝜕 𝑃 Here, Θ stands f or the Heaviside s tep function, and the parameter P can be an ything that chang es the membrane dynam- ics lik e a leakag e cur rent, or a gain current, and etc. In order to find ho w a small fraction of c hang e in the parameter 𝑃 affects the output spik e train, the Θ function should be differentiable. Ho we v er , the der ivativ e of the spik e generation function is almost alw a ys zero since the surface is mos tly flat. When the der iv ativ e is not zero, it ’ s infinite because of the sudden jumps. As a solution, an appro ximate continuous function that is able to substitute the e xact spik e generation function is used as a sur rogate in the bac kw ard pass. Implementation of the custom gradient r ule is giv en belo w . [37]: from rockpool.devices.dynapse.simulation.surrogate import step_pwl_jvp % psource step_pwl_jvp @ step_pwl . defjvp def step_pwl_jvp ( primals : Tuple [ jnp . DeviceArray ], tangents : Tuple [ jnp . DeviceArray ] )- > Tuple [ jnp . DeviceArray ]: """ step_pwl_jvp custom jvp function defining the custom gradient rule of the step pwl ␣ ˓ → function :param primals: the primary variables passed as the input to the ` step_pwl ` function :type primals: Tuple[jnp.DeviceArray] :param tangents: the first order gradient values of the primal variables :type tangents: Tuple[jnp.DeviceArray] :return: modified forward pass output and the gradient values :rtype: Tuple[jnp.DeviceArray] """ imem , Ispkthr , Ireset , max_spikes_per_dt = primals imem_dot , Ispkthr_dot , Ireset_dot , max_spikes_per_dt_dot = tangents primal_out = step_pwl (* primals ) tangent_out = jnp . clip ( jnp . ceil ( imem - Ireset ), 0 , 1 )* imem_dot return primal_out , tangent_out 40.4.1 Section 4.1 Gr adient Compuation N o w , re-use the logaritmic space, but this time store the gradient of the step function step_pwl computed independentl y at each point in the parameter space. [38]: # Scan the parameter space Imem_nabla = [jax . grad(step_pwl)(Imem, Ispkthr, Ireset) for Imem in Imem_space] (continues on ne xt page) 40.4. Section 4 : Surrogate Gradient F unction 385 Rockpool, Release 3.0.3 (continued from pre vious pag e) # Plot fig = plt . figure() plt . xlabel( "$I_ {mem} $" ) plt . ylabel( "$ \\ nabla I_ {mem} $" ) plt . semilogx(Imem_space, Imem_nabla) plt . tight_layout() The gradient value is eq ual to 1 pro vided that the 𝐼 𝑚𝑒𝑚 is g reater than 𝐼 𝑟 𝑒𝑠𝑒𝑡 v alue. W ith this properl y scaled gradient, bac kpropag ation or an y other g radient based method can be applied to computational Dynap-SE neuron model. N ote that, ha ving a cons tant v alue abo v e a threshold, the sur rog ate function resembles the gradient of the f amous ReLU function :). 40.4.2 Section 4.2 : Restoring Surrogate F unction Computing the integral of the g radient v alues, restores the surrogate function. Since only the cus tom g radient imple- mented, that is the onl y w a y to visualize the beha vior of the vir tual bac kw ard pass function. [39]: # Integrate diff_space = np . diff(np . concatenate(([Imem_space[ 0 ] / 2 ], Imem_space))) surrogate = np . cumsum(np . array(Imem_nabla) * diff_space) [40]: # Plot fig = plt . figure() plt . loglog(Imem_space, surrogate, label = "$I_ {surrogate} $" ) plt . axvline(Ireset, linestyle = "dashed" , color = "red" , label = "$I_ {reset} $" ) # Label plt . xlabel( "$I_ {mem} $" ) plt . ylabel( "$I_ {surrogate} $" ) plt . legend() plt . tight_layout() 386 Chapter 40. DynapSim N euron Model Rockpool, Release 3.0.3 In the sur rogate counterpar t, again the x axis represents the membrane current, but the y axis here is not the number of spik es. Instead, the w a v ef or m seen can be reg arded as a smoothed out v ersion of the staircase outlook of the actual function. Also, in order to ensure that the membrane cur rent is differentiable in the full operation rang e, the cut-off v alue is not 𝐼 𝑠𝑝𝑘𝑡ℎ𝑟 , but it ’ s 𝐼 𝑟 𝑒𝑠𝑒𝑡 . 40.5 N e xt Steps The f ollo wing tutor ial will co v er netw ork optimization, and post-optimization s tag es required to deplo y a netw ork succesfull y to the de vice. • F or gradient based Dynap-SE2 SNN optimization see T raining a spiking ne twor k to deploy t o Dynap-SE2 • F or post-optimization s teps see Quic k Start with Dynap-SE2 40.5. N ext Steps 387 Rockpool, Release 3.0.3 • Büchel, J., Zendr ik o v , D., Solinas, S. et al. Super vised training of spiking neural netw orks f or robust deplo yment on mix ed-signal neuromor phic processors. Sci R ep 11, 23376 (2021). • Zendr ik o v , D., Solinas, S., & Indiv er i, G. (2022). Brain-inspired methods f or ac hie ving robus t computation in heterog eneous mix ed-signal neuromor phic processing sys tems. bioRxiv . De vice mismatc h simulation de viates the parameters with a g aussian dis tr ibution, pinning the initial v alues as the mean v alues. In this w a y , the simulator can e xpose the reality better . [7]: Nin = train_data . n_in Nrec = train_data . n_out dt = 1e-3 [8]: net = Sequential( LinearJax(shape = (Nin, Nrec), has_bias = False ), DynapSim((Nrec, Nrec), has_rec = True , percent_mismatch = 0.05 , dt = dt), ) net WARNING:absl:No GPU/TPU found, falling back to CPU. (Set TF_CPP_MIN_LOG_LEVEL=0 and ␣ ˓ → rerun for more info.) [8]: JaxSequential with shape (60, 2) { LinearJax ' 0_LinearJax ' with shape (60, 2) DynapSim ' 1_DynapSim ' with shape (2, 2) } 41.4.1 Step 2.1 : The initial Results of F rozen Noise 0 [9]: plot_model_response(net, train_data, dt) 394 Chapter 41. T raining a spiking network t o deplo y to Dynap-SE2 Rockpool, Release 3.0.3 41.5 Step 3 : Response Analysis The figures abo v e sho w s that the netw ork reacts similar l y to giv en different noise patter ns. In order to put a figure on the netw ork’ s response, neurons mean fir ing rates in the giv en duration are computed b y summing up all the spik es g enerated and dividing b y the number of timesteps. 𝑟 = 1 𝑑𝑡 · 𝑁 · 𝑁 ∑︁ 𝑖 =0 𝑆 [ 𝑖 ] On top of that, to compare the neurons ’ capability to distinguish the frozen noise patterns, the ratio betw een neurons ’ mean fir ing rate is used. The firing rate ratio ( FRR ) is the ratio betw een the super ior and inf er ior mean fir ing rates read from the decision neurons. FRR is calculated as f ollo ws: 𝐹 𝑅 𝑅 = 𝑟 superior 𝑟 inf erior It ’ s seen that the outputs of the neurons are almost identical receiving the frozen noise patterns of similar discrete time ser ies of e v ents. The number is pretty close to eac h others and to 1. This sho ws that the randoml y initialized DynapSim netw ork is insensitiv e to the noise patter ns. Belo w , a collection of fir ing rate responses of the initial netw ork to the test data is giv en. [10]: plot_model_response_histogram(net, test_data) Histogram: 100 iteration [00:16, 5.95 iteration/s] 41.5. Step 3 : Response Analy sis 395 Rockpool, Release 3.0.3 In order to teach the netw ork to sense the temporal nuances hidden in these time series, a g radient-based optimization procedure will be e x ecuted ne xt. The f ollo wing sections present this procedure, star ting from introducing the objectiv e function. 41.6 Step 4 : Optimization The objectiv e of the optimization is to make one of the neurons fire at a noticeably higher rate upon receiving a specific frozen noise. F or e xample, if frozen noise 1 is dispatched to the de vice then neuron 1 should fire dominantl y , and if frozen noise 2 is dispatched, neuron 2 fir ing should dominate the output reading. In order to ac hie v e this beha vior , mean square error (MSE) loss is e xploited. 41.6.1 Step 4.1 Loss F unction 𝑓 MSE ( 𝑦 out , 𝑦 target ) = 1 𝑁 𝑁 ∑︁ 𝑖 =0 ( 𝑦 out [ 𝑖 ] − 𝑦 targ et [ 𝑖 ]) 2 The targ et train is a unif or m spike train that has an e v ent at ev ery time step. [11]: # - Loss function @jax . jit @jax . value_and_grad def loss_vgf (params, net, input , target): net = net . set_attributes(params) net = net . reset_state() output, _, _ = net( input ) return jl . mse(output, target) Ev en though, the training objectiv e is to mak e the neuron fire constantl y , the actual neuron model is not capable of producing e xactl y the same ideal spike train. The refractor y per iods and spike freq uency adaptation mechanism pre v ent the neuron from fir ing incessantl y . Theref ore, it ’ s impossible to g et zero er ror in an y case. The optimizer pushes the neurons to do their best to con v erg e to the ideal spiking regime. The e xpected training beha vior is that the er ror will start high and then g raduall y drop do wn to a le v el that is definitel y abo v e zero. 41.6.2 Step 4.2 Optimizer T o update the weight matrices such that the netw ork w ould achie v e the leas t possible mean square error , a g radient- based optimization procedure is applied. In this e xper iment, a popular g radient descent v ar iation, A daptiv e Moment Estimation, A dam , is used. This method pro vides a firs t-order gradient-based optimization of stoc hastic objectiv e functions based on adaptiv e estimates of lo w er -order moments. [12]: # - Initialise optimiser init_fun, update_fun, get_params = adam(step_size = 1e-3 ) opt_state = init_fun(net . parameters()) update_fun = jax . jit(update_fun) 41.6.3 Step 4.3 Mismatch Generator (Optional) This step is not necessary to train a DynapSim netw ork, but highly sugg ested. As e xplained abo v e, the hardware suffers from de vice mismatc h, and w e already initiated the model with a frozen mismatc h. On top of it, de viating parameter v alues during training, w ould giv e us a better , more robust parameter space. [13]: # Obtain the prototoype and the random number generator keys rng_key = jnp . array([ 2021 , 2022 ], dtype = jnp . uint32) (continues on ne xt page) 396 Chapter 41. T raining a spiking network t o deplo y to Dynap-SE2 Rockpool, Release 3.0.3 (continued from pre vious pag e) mismatch_prototype = dynamic_mismatch_prototype(net) # Get the mismatch generator function regenerate_mismatch = mismatch_generator(mismatch_prototype, percent_deviation = 0.30 , ␣ ˓ → sigma_rule = 3.0 ) regenerate_mismatch = jax . jit(regenerate_mismatch) 41.6.4 Step 4.5 Run the trainin loop Each training s tep, or epoch, includes a f or w ard and a bac kw ard pass. The f or ward pass simulates the neural dynamics in time and produces the spik e trains. The bac kw ard pass bac kpropag ates the er ror in time, assigning the credits to w eight values. Since the f or w ard computation introduces indifferentiable functions, a sur rogate function appro ximation is used in the bac kw ard pass. (see neuron model tutor ial f or more) As a result, the w eight values g et small updates in each time s tep fixing the beha vior slightl y in the appro ximate continuous space. So, this will tak e time.. Y ou can w ait or y ou can use the pre-trained w eights f or later stag es. [14]: # - Configure learning num_epochs = int ( 1e6 ) apply_mismatch = 100 loss_t = [] # - Get input and target batches batch_input, batch_target = train_data . full_batch # - Training loop t = tqdm( range (num_epochs), desc = ' Training ' , unit = ' Epoch ' , total = num_epochs) for epoch in t: # - Get parameters opt_parameters = get_params(opt_state) # - Regenerate mismatch once in a while if epoch % apply_mismatch == 0: rng_key, _ = rand . split(rng_key) new_params = regenerate_mismatch(net, rng_key = rng_key) net = net . set_attributes(new_params) # - Compute loss and gradient l ,g = loss_vgf(opt_parameters, net, batch_input, batch_target) loss_t . append(l . item()) t . set_postfix({ ' loss ' : l . item()}, refresh = False ) # - Update optimiser opt_state = update_fun(epoch, g, opt_state) Training: 100%|| 1000000/1000000 [06:41<00:00, 2491.61Epoch/s, loss=0.453] 41.6. Step 4 : Optimization 397 Rockpool, Release 3.0.3 41.7 Step 5 : Results W e ha v e trained our netw ork to recognize some giv en frozen noise patter ns. No w it ’ s time to chec k it ’ s per f or mance 41.7.1 Step 5.1. Loss Plot It ’ s seen in the figure that the loss decreased from 0.5 to a non-zero value o v er man y epochs (as e xpected). Ho w e v er , e v en this much drop creates a hug e difference in beha vior , and the netw ork obtains the ability to classify tw o similar frozen noise samples. [15]: plt . semilogy(loss_t) plt . xlabel( ' Epochs ' ) plt . ylabel( ' Loss Value ' ) plt . title( "MSE Loss" ) plt . tight_layout() 41.7.2 Section 5.2 Get the Optimized N etwork As a result of training, the netw ork must ha v e lear ned to sense the difference betw een frozen noise patter ns. In order to obser v e the beha vior of the netw ork and compare with the initial v ersion, the optimized netw ork is simulated. [16]: net_optimized = net . set_attributes(get_params(opt_state)) plot_model_response(net_optimized, train_data, dt) 398 Chapter 41. T raining a spiking network t o deplo y to Dynap-SE2 Rockpool, Release 3.0.3 41.7.3 Section 5.3 : N egative T est Optimization results sho w that this tin y recur rent spiking neural netw ork of tw o DynapSim neurons can dis tinguish one frozen noise from another . If the optimization requirements w ere satisfied, then the netw ork should respond clearl y to recognized noises and react randoml y to an ything else. In other w ords, decision neurons should fire tog ether or sta y silent upon receiving a non-recognized signal. If one of them fires and the other one s ta ys silent, then it sho w s that the netw ork can be deceiv ed and the decisions are not reliable. N o w , w e will use the test set data to sho w that our netw ork recognize onl y the the training samples. In order to test this, 1000 frozen noise patter ns with the same mean frequency and length are g enerated using the same process used in training data g eneration. T o pro v e the rate ratio is dramatically c hang ed only f or recognized signals, the FRRs betw een decision neurons are recorded. [17]: plot_model_response(net_optimized, test_data, dt, range ( 5 )) 41.7. Step 5 : Results 399 Rockpool, Release 3.0.3 400 Chapter 41. T raining a spiking network t o deplo y to Dynap-SE2 Rockpool, Release 3.0.3 It will be a long pag e if we plot 100 samples one after another , ho w ev er , w e can giv e an histogram sho wing FRR readings of all 100 test samples. [18]: plot_model_response_histogram(net_optimized, test_data) Histogram: 100 iteration [00:15, 6.27 iteration/s] 41.7.4 Section 5.4 : Sa ve the parameters for the ne xt steps It ’ s g reat that w e hav e a netw ork that can recognize the frozen noise patter ns using onl y 2 analog neurons! In the ne xt steps w e will map this netw ork to the hardware f ollo wing some steps similar to q uic k s tar t tutor ial. Let ’ s sa v e the netw ork w eights at this s tag e bef ore doing an ything else. [19]: p0, p1 = get_params(opt_state) . values() with open ( "data/w_in_optimized.npy" , "wb" ) as f: np . save(f, p0[ "weight" ]) with open ( "data/w_rec_optimized.npy" , "wb" ) as f: np . save(f, p1[ "w_rec" ]) 41.7. Step 5 : Results 401 Rockpool, Release 3.0.3 41.8 N e xt steps Please continue with the post-training tutorial to see ho w this netw ork is deplo y ed to the hardware. Quick S tar t with Dynap-SE2 402 Chapter 41. T raining a spiking network t o deplo y to Dynap-SE2 CHAPTER F ORT Y T W O COMPUT A TIONAL GRAPHS IN ROCKPOOL Graphs is R ockpool are used to con v er t the structure of an arbitrar y netw ork to deplo y on neuromor phic hardware. The graphs cannot be used f or simulation, and are g enerally more cons trained than netw orks used f or training and simulation. [1]: # - Switch off warnings import warnings warnings . filterwarnings( "ignore" ) 42.1 Graph base classes Graphs consist of modules (deriv ed from the class graph.GraphModule ), connected o v er nodes (der iv ed from the class graph.GraphNode ). graph.GraphModule s are units of computation. For e xample, a set of linear w eights; a population of spiking neurons. graph.GraphModule s contain links to sets of in put nodes and output nodes, which define the c hannels of inf or mation flo wing in and out of a module. These nodes are graph.GraphNode objects. [2]: from IPython.display import Image from pathlib import Path Image(Path( "images" , "GraphModule.png" )) [2]: The figure abo v e sho w s the conceptual contents of a graph.GraphModule . input_nodes and output_nodes are ordered lists containing ref erences to zero or more graph.GraphNode objects. graph.GraphModule subclasses define specific computational units. The y contain all additional parameters needed to define the configuration of the computational unit. For e x ample, see the graph.LIFNeuronWithSynsRealValue class, which contains the configuration parameters needed to define a standard LIF neuron with e xponential synapses. A graph.GraphNode is an object that holds connections betw een graph.GraphModule s. Eac h node can ha v e mul- tiple sources and sinks of inf or mation. 403 Rockpool, Release 3.0.3 rockpool.nn.modules.torch.torch_module Pro vide a base class f or build T orch-compatible modules F unctions to_float_tensor (x) Classes TorchModule (*args, **kw args) Base class f or modules that are compatible with both T orch and R oc kpool TorchModuleParameters A dict subclass that suppor ts con v ersion to ra w values rockpool.nn.modules.torch.updown_t orch F eedf or w ard lay er that con v erts each analogue input c hannel to one spiking up and one do wn channel Classes StepPWL (*args, **kw args) Hea viside step function with piece-wise linear derivativ e to use as spik e-generation surrogate UpDownTorch (*args, **kw args) F eedf or w ard lay er that con v erts each analogue input channel to one spiking up and one spiking do wn c hannel. rockpool.nn.network s Defines classes f or encapsulating and g enerating netw orks of la y ers Modules synnet Implements the SynNet arc hitecture f or temporal signal processing wavesense Implements the W av eSense arc hitecture f or temporal signal processing rockpool.nn.network s.synnet Implements the SynNet arc hitecture f or temporal signal processing The SynNet arc hitecture is descr ibed in Bos & Muir 2022 [ https:// ar xiv .org/ abs/ 2208.12991 ] and Bos & Muir 2024 [ https:// ar xiv .or g/ abs/ 2406.15112 ] Classes SynNet (*args, **kw ar gs) Define a SynNet architecture netw ork 48.1. Package structure summar y 495 Rockpool, Release 3.0.3 rockpool.nn.network s.wav esense Implements the W av eSense arc hitecture f or temporal signal processing The W av eSense arc hitecture is descr ibed in W eidel et al 2021 [ https:// ar xiv .org/ abs/ 2111.01456 ] See Also W aveSense: T raining a Spiking N eur al N etw or k with T emporal Conv olutions Classes WaveBlock (n_channels_res, n_c hannels_skip, ...) WaveNet ([n_classes, n_channels_in, ...]) WaveSenseBlock (*args, **kw ar gs) Implements a single W av eSenseBloc k WaveSenseNet (*args, **kw ar gs) Implement a W av eSense netw ork rockpool.parameters Classes to manag e registered Module attributes in R ockpool F unctions Constant (obj) Identify an initialisation argument as a cons tant (non- trainable) parameter Classes Parameter (data, f amily , init_func, ...) R epresent a module parameter ParameterBase (data, f amily , init_func, ...) Base class f or R oc kpool registered attributes RP_Constant () R epresent a concrete initialisation v alue as a constant pa- rameter , whic h should not be trained SimulationParameter (data, f amily , init_func, ...) R epresent a module simulation parameter State (data, f amily , init_func, ...) R epresent a module state rockpool.timeseries Classes to manag e time ser ies data F unctions full_nan (shape) Build an all-N aN ar ra y get_global_ts_plotting_backend () R etur n a string representing the cur rent plotting back end load_ts_from_file (path[, e xpected_type]) Load a timeser ies object from an npz file set_global_ts_plotting_backend (bac kend[, ...]) Set the plotting bac kend f or use b y TimeSeries classes 496 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 Classes TSContinuous ([times, samples, num_channels, ...]) R epresents a continuously -sampled time series. TSDictOnDisk ([data]) Beha v es like a dict. TSEvent ([times, channels, per iodic, ...]) R epresents a discrete time ser ies, composed of binar y e v ents (present or absent). TimeSeries ([times, per iodic, t_s tar t, ...]) Base class to represent a continuous or e v ent-based time ser ies. rockpool.training Contains pac kages f or assis ting with NN training Modules adversarial_jax Functions to implement adv ersarial training approaches using Jax ctc_loss Implementations of the CTC loss, in nump y , Jax and T orch jax_debug Utilities f or debugging Jax training loops jax_loss Jax functions useful f or training netw orks using Jax Modules. torch_loss T orch loss functions and regularizers useful f or training netw orks using T orch Modules. rockpool.training.adversarial_jax Functions to implement adv ersarial training approaches using Jax See also Adv ersarial tr aining illustrates ho w to use the functions in this module to implement adv ersarial attacks on the parameters of a netw ork dur ing training. F unctions adversarial_loss (parameters, net, inputs, ...) Implement a h ybr id task / adv ersar ial robus tness loss pga_attack (params_flattened, net, rng_ke y , ...) P er f or ms the PG A (projected gradient ascent) based at- tac k on the parameters of the netw ork giv en inputs. rockpool.training.ctc_loss Implementations of the CTC loss, in nump y , Jax and T orch F unctions ctc_loss_jax (label, log_prob, seq_length[, ...]) Jax-compatible implementation of the CTC loss ctc_loss_numpy (label, log_prob, seq_length) N ump y implementation of the CTC loss continues on ne xt page 48.1. Package structure summar y 497 Rockpool, Release 3.0.3 T able 219 – continued from pre vious page ctc_loss_torch (label, log_prob, seq_length) T orch implementation of the CTC loss rockpool.training.jax_debug Utilities f or debugging Jax training loops F unctions debug_evolution (jmod, state, parameters, input) Debug and repor t the presence of N aNs in netw ork state / output debug_optimisation (jmod, parameters, input, ...) Debug an optimisation step, repor ting the presence of N aNs in loss and g radients flatten (g ener ic_collection[, sep]) Flattens a g ener ic collection of collections into an or - dered dictionar y . rockpool.training.jax_loss Jax functions useful f or training netw orks using Jax Modules. See also See T raining a R oc kpool netw ork wit h Jax f or an introduction to training netw orks using Jax-bac ked modules in R oc kpool, including the functions in jax_loss . F unctions bounds_clip (params, lo w er_bounds, upper_bounds) bounds_cost (params, lo wer_bounds, upper_bounds) Impose a cost on parameters that violate bounds con- straints l0_norm_approx (params[, sigma]) Compute a smooth differentiable appro ximation to the L0-nor m l2sqr_norm (params) Compute the mean L2-squared-norm of the set of param- eters logsoftmax (x[, temperature]) Efficient implementation of the log softmax function make_bounds (params) Con v enience function to build a bounds template f or a problem mse (output, targ et) Compute the mean-squared error betw een output and tar - g et softmax (x[, temperature]) Implements the softmax function sse (output, targ et) Compute the sum-squared error between output and tar - g et rockpool.training.torch_loss T orch loss functions and regularizers useful f or training netw orks using T orch Modules. 498 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 F unctions bounds_cost (params, lo wer_bounds, upper_bounds) Impose a cost on parameters that violate bounds con- straints make_bounds (params) Con v enience function to build a bounds template f or a problem summed_exp_boundary_loss (data[, ...]) Compute the summed e xponential er ror of boundar y vi- olations of an input. Classes ParameterBoundaryRegularizer (*args, **kw args) Class wrapper f or the summed e xponential er ror of boundar y violations of an in put. rockpool.transform Contains pac kages f or transf or ming parameters and netw orks Modules dropout Pro vide a Dropout parameter transf ormation mismatch Analog de vice mismatch transf ormation (jax) imple- mentation param_transformer Pro vide a Module wrapper that transf orms parameters bef ore e v olution quantize Pro vide a stoc hastic q uantization parameter transf orma- tion module quantize_methods Quantisation methods f or X y lo torch_transform Defines the parameter and activ ation transf or mation-in- training pipeline f or TorchModule s rockpool.transform.dropout Pro vide a Dropout parameter transf ormation F unctions Dropout (mod, *args, **kw ar gs) Classes JaxDropout (*args, **kw ar gs) ModDropout (*args, **kw ar gs) 48.1. Package structure summar y 499 Rockpool, Release 3.0.3 rockpool.transform.mismatch Analog de vice mismatch transf ormation (jax) implementation F unctions mismatch_generator (prototype[, ...]) mismatch_g enerator returns a function which simulates the analog de vice mismatch effect. module_registery (module) module_registery traces all the nested module and registered parameters of the JaxModule base giv en and retur ns a tree, whose lea v es includes onl y the parameters.SimulationParameters and parameters.Parameters rockpool.transform.param_transformer Pro vide a Module wrapper that transf orms parameters bef ore ev olution F unctions deep_update (source, o v er r ides) U pdate a nested dictionary or similar mapping. Classes JaxParameterTransformerMixin (module[, ...]) ParameterTransformerMixin (module[, ...]) rockpool.transform.quantize Pro vide a stoc hastic q uantization parameter transf or mation module F unctions StochasticQuantize (mod, *args, **kw ar gs) Classes JaxStochasticQuantize (*args, **kw ar gs) ModStochasticQuantize (*args, **kw ar gs) 500 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 rockpool.transform.quantize_methods Quantisation methods f or X y lo Defines the post-training q uasntization methods global_quantize() and channel_quantize() . F unctions channel_quantize (w eights_in, w eights_rec, ...) Quantize a X y lo model f or deplo yment, using per - channel parameter scaling global_quantize (w eights_in, w eights_rec, ...) Quantize a X y lo model f or deplo yment, using global pa- rameter scaling validate_weights_to_2d (data) validate_weights_to_3d (data) rockpool.transform.torch_transform Defines the parameter and activ ation transf or mation-in-training pipeline f or TorchModule s See also /adv anced/QuantT orch.ip ynb Examples Construct a netw ork, and patch it to round eac h w eight parameter: >>> net = Sequential( ... ) >>> T_fn = lambda p: stochastic_rounding(p, num_levels = 2 ** num_bits) >>> T_config = make_param_T_config(net, T_fn, ' weights ' ) >>> T_net = make_param_T_network(net, T_config) T rain here. T o bur n-in and remo v e the transf ormations: >>> burned_in_net = apply_T(T_net) >>> unpatched_net = remove_T_net(burned_in_net) F unctions apply_T (T_net[, inplace]) "Bur n in" a set of parameter transf ormations, appl y- ing each transf or mation and storing the resulting trans- f ormed parameters deterministic_rounding (v alue[, input_rang e, ...]) Quantise v alues b y shifting them to the closest quantisa- tion le v el dropout (param[, dropout_prob]) Randomly set v alues of a tensor to 0. , with a defined probability int_quant (v alue[, maintain_zero, ...]) T ransf or ms a tensor to a quantized space with a rang e of integ er values defined b y n_bits continues on ne xt page 48.1. Package structure summar y 501 Rockpool, Release 3.0.3 T able 233 – continued from pre vious page make_act_T_config (net[, T_fn, ModuleClass]) Create an activity transf ormation configuration f or a net- w ork make_act_T_network (net, act_T_config[, inplace]) P atch a R oc kpool netw ork with activity transf or mers make_backward_passthrough (function) W rap a function to pass the gradient directl y through in the bac kward pass make_param_T_config (net, T_fn[, param_famil y]) Helper function to build parameter transf or mation con- figuration trees make_param_T_network (net, T_config_tree[, ...]) P atch a R oc kpool netw ork to appl y parameter transf or - mations in the f orward pass remove_T_net (T_net[, inplace]) U n-patch a transf ormed-patched netw ork stochastic_channel_rounding (v alue, out- put_rang e) P er f or m s tochastic rounding of a matrix, but with the in- put rang e defined automatically f or each column inde- pendentl y stochastic_rounding (v alue[, input_rang e, ...]) P er f or m floating-point s tochastic rounding on a tensor , with detailed control o v er quantisation le v els t_decay (deca y[, dt]) quantizes deca y factor (e xp (-dt/tau)) of LIF neurons: al- pha and beta respectiv el y f or Vmem and Isyn the quanti- zation is done based one con v er ting the deca y to bitshoft subtraction and reconstructing deca y . Classes ActWrapper (*args, **kw args) A wrapper module that applies an output activity trans- f or mation after e v olution TWrapper (*args, **kw args) A wrapper f or a R ockpool T orc hModule, implementing a parameter transf or mation in the f or w ard pass class_calc_q_decay (dt) function used to calculate bitshift equiv alent of deca y (e xp(-dt/tau)) rockpool.typehints Module to pro vide useful types f or R ockpool See Roc kpool P ar ameter handling f or more inf or mation on the a vailable types. Module A ttributes P_float A P arameter or a float P_int A P arameter or an int P_str A P arameter or a str ing P_bool A P arameter or a boolean P_Callable A P arameter or a Callable P_ndarray A P arameter or a numpy arra y Tree A Python tree-lik e object Leaf A leaf node in a tree Value The v alue in a tree leaf node Node A node in a tree P_tree A P arameter or a T ree P_tensor A P arameter or a torch tensor FloatVector A float scalar or a float v ector continues on ne xt page 502 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 T able 235 – continued from pre vious page IntVector An int scalar or an int v ector JaxRNGKey A Jax RN G ke y JaxTreeDef A Jax tree definition TreeDef A Jax-lik e tree definition Ex ceptions DRCError An Er ror class representing a Design-R ule Check viola- tion DRCWarning A W ar ning / Er ror class representing a Design-Rule Chec k warning rockpool.utilities General utilities Modules backend_management Utility functionality f or managing back ends benchmarking Utilities f or benchmarking neuron la y ers jax_tree_utils Utility functions f or w orking with trees. property_arrays proper ty_ar ra y s.py - Collection of arra y classes to be used as proper ties to control timedarray_shift Implementation of TimedArray . tree_utils T ree manipulation utilities with no e xter nal dependen- cies type_handling type_handling.p y - Con v enience functions f or chec king and con v er ting object types rockpool.utilities.backend_management Utility functionality f or managing back ends T o chec k a standard bac kend, use backend_available() . T o chec k a non-standard bac kend specification, use check_backend() . T o build a shim class that raises an er ror on instantiation, f or when a required bac kend is not a vailable, use missing_backend_shim() . F unctions backend_available (*bac kend_names) R epor t if a back end is a vailable f or use check_backend (bac kend_name[, ...]) Chec k if a back end is a v ailable, and register it in a list of a v ailable back ends check_samna_available () chec k_samna_a v ailable controls if samna pack - ag e is "installed" and "usable" The def ault backend_available() operation cannot cor rectl y identifies the samna a v ailability . continues on ne xt page 48.1. Package structure summar y 503 Rockpool, Release 3.0.3 Examples >>> data, (state0, state1, state2) = self . _auto_batch(data, ( self . state0, self . ˓ → state1, self . state2)) This will v er ify that data has the cor rect final dimension (i.e. self.size_in ). If data has onl y tw o dimensions (T, Nin) , then it will be augmented to (1, T, Nin) . The individ- ual states will be replicated out from shape (a, b, c, ...) to (n_batches, a, b, c, ...) and re- tur ned. If data has onl y a single dimension (T,) , it will be e xpanded to (1, T, self.size_in) . state0 , state1 , state2 will be replicated out along the batch dimension. >>> data, (state0,) = self . _auto_batch(data, ( self . state0,), (( 10 , - 1 , self . ˓ → size_in),)) A ttempt to replicate state0 to a specified size (10, -1, self.size_in) . P arameters • data ( np.ndarray ) – Input data tensor . Either (batches, T, Nin) or (T, Nin) • states ( Tuple ) – T uple of s tate variables. Each will be replicated out o v er batches b y prepending a batch dimension • target_shapes ( Tuple ) – A tuple of tar get size tuples, eac h cor responding to each s tate argument. The individual states will be replicated out to matc h the cor responding targ et sizes. If not pro vided (the default), then s tates will be only replicated along batc hes. R eturns (np.ndarra y , T uple[np.ndarra y]) data, states R eturn type T uple [ ndarray , T uple [ ndarr ay ]] _force_set_attributes (bool) If True , do not sanity-c heck attributes when setting. _get_attribute_family ( type_name: str , family : str | T uple | List = N one ) → dict Search f or attr ibutes of this module and submodules that match a giv en famil y This method can be used to con v enientl y get all w eights f or a netw ork; or all time constants; or an y other f amily of parameters. Parameter f amilies are defined simply b y a string: "weights" f or w eights; "taus" f or time constants, etc. These strings are arbitrar y , but if y ou f ollo w the con v entions then future dev elopers will thank y ou (that includes y ou in six month ’ s time). P arameters • type_name ( str ) – The class of parameters to search f or . Must be one of ["Parameter", "SimulationParameter", "State"] or another future subclass of ParameterBase • family ( Union [ str , Tuple [ str ]] ) – A string or list or tuple of s tr ings, that define one or more attr ibute f amilies to search f or R eturns A nested dictionary of attr ibutes that match the pro vided type_name and family R eturn type dict 510 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 _get_attribute_registry () → T uple[Dict, Dict] R etur n or initialise the attr ibute registry f or this module R eturns registered_attributes, registered_modules R eturn type (tuple) _has_registered_attribute ( name: str ) → bool Chec k if the module has a registered attr ibute P arameters name ( str ) – The name of the attr ibute to chec k R eturns True if the attr ibute name is in the attribute registr y , False other wise. R eturn type bool _in_Module_init (bool) If e xists and True , indicates that the module is in the __init__ chain. _name: str | None N ame of this module, if assigned _register_attribute ( name: str , val: P arame terBase ) R ecord an attr ibute in the attr ibute registry P arameters • name ( str ) – The name of the attr ibute to register • val ( ParameterBase ) – The ParameterBase subclass object to register . e.g. Parameter , SimulationParameter or State . _register_module ( name: str , mod: ModuleBase ) R egister a sub-module in the module regis tr y P arameters • name ( str ) – The name of the module to register • mod ( ModuleBase ) – The ModuleBase object to register _reset_attribute ( name: str ) → ModuleBase R eset an attr ibute to its initialisation v alue P arameters name ( str ) – The name of the attr ibute to reset R eturns F or compatibility with the functional API R eturn type self ( Module ) _shape The shape of this module 48.2. Base classes 511 Rockpool, Release 3.0.3 _spiking_input: bool Whether this module receiv es spiking input _spiking_output: bool Whether this module produces spiking output _submodulenames: List[str] R egistry of sub-module names _wrap_recorded_state ( r ecor ded_dict : dict , t_start : float ) → Dict[s tr , TimeSeries] Con v er t a recorded dictionar y to a TimeSeries representation This method is optional, and is pro vided to mak e the timed() conv ersion to a TimedModule w ork better . Y ou should o v er r ide this method in y our custom Module , to wrap each element of y our recorded state dictionar y as a TimeSeries P arameters • state_dict ( dict ) – A recorded state dictionary as retur ned b y evolve() • t_start ( float ) – The initial time of the recorded state, to use as the s tar ting point of the time ser ies • recorded_dict ( dict ) R eturns The mapped recorded state dictionary , wrapped as TimeSeries objects R eturn type Dict[str , TimeSeries ] as_graph () → GraphModuleBase Con v er t this module to a computational graph R eturns The computational graph cor responding to this module R eturn type Gr aphModuleBase Raises NotImplementedError – If as_graph() is not implemented f or this subclass attributes_named ( name: T uple[str] | Lis t[str] | str ) → dict Search f or attr ibutes of this or submodules b y time P arameters name ( Union [ str , Tuple [ str ] ) – The name of the attr ibute to searc h f or R eturns A nested dictionary of attr ibutes that match name R eturn type dict property class_name: str Class name of self T ype str 512 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 abstractmethod evolve ( input_data , r ecor d: bool = F alse ) → T uple[An y , Any , Any] Ev ol v e the state of this module o v er input data N OTE: THIS MODULE CLASS DOES NOT PR O VIDE DOCUMENT A TION FOR ITS EV OL VE METHOD. PLEASE UPD A TE THE DOCUMENT A TION FOR THIS MODULE. P arameters • input_data – The input data with shape (T, size_in) to e v ol v e with • record ( bool ) – If True , the module should record inter nal state during ev olution and retur n the record. If False , no recording is required. Def ault: False . R eturns (output, ne w_state, record) output (np.ndarra y): The output response of this module with shape (T, size_out) ne w_state (dict): A dictionar y containing the updated state of this and all submodules after e v olution record (dict): A dictionar y containing recorded s tate of this and all submodules, if reques ted using the record argument R eturn type tuple property full_name: str The full name of this module (class plus module name) T ype str modules () → Dict R etur n a dictionar y of all sub-modules of this module R eturns A dictionar y containing all sub-modules. Each item will be named with the sub-module name. R eturn type dict property name: str The name of this module, or an empty string if None T ype str parameters ( f amily : str | T uple | List = N one ) → Dict R etur n a nested dictionary of module and submodule Parameters Use this method to inspect the P arameters from this and all submodules. The optional ar gument family allo w s y ou to search f or Parameters in a particular famil y — f or e xample "weights" f or all w eights of this module and nested submodules. Although the family argument is an arbitrary str ing, reasonable choises are "weights" , "taus" f or time constants, "biases" f or biases. . . Examples Obtain a dictionar y of all P arameters f or this module (including submodules): >>> mod . parameters() dict{ ... } 48.2. Base classes 513 Rockpool, Release 3.0.3 Obtain a dictionar y of P arameters from a par ticular famil y: >>> mod . parameters( "weights" ) dict{ ... } P arameters family ( str ) – The famil y of P arameters to search f or . Def ault: None ; retur n all parameters. R eturns A nested dictionary of Parameters of this module and all submodules R eturn type dict reset_parameters () R eset all parameters in this module R eturns The updated module is retur ned f or compatibility with the functional API R eturn type Module reset_state () → ModuleBase R eset the state of this module R eturns The updated module is retur ned f or compatibility with the functional API R eturn type Module set_attributes ( new_attributes: dict ) → ModuleBase Set the attr ibutes and sub-module attributes from a dictionar y This method can be used with the dictionar y retur ned from module e v olution to set the ne w state of the module. It can also be used to set multiple parameters of a module and submodules. Examples Use the functional API to e v ol v e, obtain new s tates, and set those states: >>> _, new_state, _ = mod( input ) >>> mod = mod . set_attributes(new_state) Obtain a parameter dictionar y , modify it, then set the parameters bac k: >>> params = mod . parameters() >>> params[ ' w_input ' ] *= 0. >>> mod . set_attributes(params) P arameters new_attributes ( dict ) – A nested dictionary containing parameters of this module and sub-modules. R eturn type ModuleBase 514 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 property shape: tuple The shape of this module T ype tuple simulation_parameters ( f amily : str | T uple | List = N one ) → Dict R etur n a nested dictionary of module and submodule SimulationParameters Use this method to inspect the SimulationP arameters from this and all submodules. The optional argument family allo w s y ou to search f or SimulationP arameters in a par ticular f amily . Examples Obtain a dictionar y of all SimulationP arameters f or this module (including submodules): >>> mod . simulation_parameters() dict{ ... } P arameters family ( str ) – The famil y of SimulationP arameters to search f or . Def ault: None ; retur n all SimulationP arameter attr ibutes. R eturns A nested dictionary of SimulationParameters of this module and all submodules R eturn type dict property size: int (DEPREC A TED) The output size of this module T ype int property size_in: int The input size of this module T ype int property size_out: int The output size of this module T ype int property spiking_input: bool If True , this module receiv es spiking input. If False , this module e xpects continuous input. T ype bool property spiking_output If True , this module sends spiking output. If False , this module sends continuous output. T ype bool 48.2. Base classes 515 Rockpool, Release 3.0.3 state ( f amily : str | T uple | List = N one ) → Dict R etur n a nested dictionary of module and submodule States Use this method to inspect the S tates from this and all submodules. The optional argument family allo w s y ou to search f or States in a particular famil y . Examples Obtain a dictionar y of all S tates f or this module (including submodules): >>> mod . state() dict{ ... } P arameters family ( str ) – The famil y of S tates to search f or . Default: None ; retur n all S tate attr ibutes. R eturns A nested dictionary of States of this module and all submodules R eturn type dict timed ( output_num: int = 0 , dt: float = N one , add_ev ents: bool = F alse ) Con v er t this module to a TimedModule P arameters • output_num ( int ) – Specify which output of the module to tak e, if the module retur ns multiple output ser ies. Default: 0 , tak e the first (or onl y) output. • dt ( float ) – Used to pro vide a time-step f or this module, if the module does not already ha v e one. If self already defines a time-step, then self.dt will be used. Def ault: None • add_events ( bool ) – Iff True , the TimedModule will add e v ents occur r ing on a single timestep on in put and output. Def ault: False , don ’ t add time steps. R etur ns: TimedModule : A timed module that wraps this module 48.2.2 nn.modules.TimedModule class nn.modules. TimedModule ( *ar gs , **kw args ) Bases: ModuleBase The R oc kpool base class f or all TimedModule modules TimedModule pro vides functionality f or Module s to understand time ser ies data, and to con v eniently e v olv e, handle and retur n time series data from modules. The evolve() method pro vided b y TimedModule can accept TimeSeries objects nativ el y as input, or can accept cloc ked / ras ter ised input data. See also TimedModule pro vides the useful methods _prepare_input() and _gen_timeseries() to help y ou in rasterising data f or y our o wn TimedModule subclasses. F or more inf or mation on ho w to used the TimedModule API f or R ockpool, see High-lev el TimedModule API . 516 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 A ttributes ov er view class_name Class name of self full_name The full name of this module (class plus module name) input_type The TimeSeries class accepted b y this module name The name of this module, or an empty string if None output_type The TimeSeries class retur ned b y this module shape The shape of this module size (DEPREC A TED) The output size of this module size_in The input size of this module size_out The output size of this module spiking_input If True , this module receiv es spiking input. spiking_output If True , this module sends spiking output. t The cur rent e v olution time of this la y er , in seconds dt The simulation and input ras ter isation timestep f or this TimedModule Methods o verview __init__ (dt[, spiking_input, ...]) Initialise this TimedModule object as_graph () Con v er t this module to a computational graph attributes_named (name) Search f or attr ibutes of this or submodules b y time evolve ([ts_in put, duration, num_timesteps, ...]) Ev ol v e the state of this module o v er time modules () R etur n a dictionar y of all sub-modules of this module parameters ([f amily]) R etur n a nested dictionary of module and submodule P arameters reset_all () R eset the inter nal state and time of this module and all sub-modules reset_parameters () R eset all parameters in this module reset_state () R eset the state of this module reset_time () R eset the inter nal time of this module and all sub- modules to zero set_attributes (ne w_attr ibutes) Set the attr ibutes and sub-module attributes from a dictionar y simulation_parameters ([f amily]) Return a nested dictionary of module and submodule SimulationP arameters state ([f amil y]) R etur n a nested dictionary of module and submodule S tates __in_TimedModule_init: bool = False A flag indicating that this TimedModule is cur rentl y being initialised __init__ ( dt : float | SimulationP ar amet er , spiking_input : bool = F alse , spiking_output : bool = F alse , add_ev ents: bool = T r ue , *ar gs , **kwar gs ) Initialise this TimedModule object When initialised, the TimedModule will hav e a dt attr ibute assigned, as well as initialising the internal module _timestep , _parent_dt_factor and _is_child . The subclass evolve() method will be wrapped to update the inter nal times tamp clock. P arameters 48.2. Base classes 517 Rockpool, Release 3.0.3 • dt ( float ) – The duration of a single time step f or this module, in seconds • spiking_input ( bool ) – If True , this module accepts TSEvent e v ent time ser ies objects as input. If False (def ault), this module accepts TSContinuous continuous time ser ies objects as input. • spiking_output ( bool ) – If True , this module sends TSEvent e v ent time ser ies objects as output. If False (def ault), this module sends TSContinuous continuous time ser ies objects as output. • *args – A dditional positional arguments • **kwargs – A dditional ke yw ord arguments • add_events ( bool ) _abc_impl = <_abc._abc_data object> _determine_timesteps ( ts_input : TimeSeries | N one = None , dur ation: float | None = N one , num_timest eps: int | N one = None ) → int Deter mine ho w man y time steps to e v ol v e with the giv en input specification P arameters • ts_input ( Optional [ TimeSeries ] ) – TxM or Tx1 time ser ies of in put signals f or this la y er • duration ( Optional [ float ] ) – Duration of the desired ev olution, in seconds. If not pro vided, num_timesteps or the duration of ts_input will be used to determine ev olu- tion time • num_timesteps ( Optional [ int ] ) – Number of e v olution time steps, in units of dt . If not pro vided, duration or the duration of ts_input will be used to deter mine e v olution time R eturn int num_timesteps: N umber of ev olution time steps R eturn type int _evolve_wrapper ( ts_input=N one , duration=N one , num_timesteps=N one , kwar gs_timeseries=N one , recor d: bool = F alse , *ar gs , **kwar gs ) → T uple[TimeSeries, Dict, Dict] W rap a call to evolve() to update the inter nal time-steps count See evolve() f or calling syntax. P arameters record ( bool ) R eturn type T uple [ TimeSeries , Dict , Dict ] _force_set_attributes (bool) If True , do not sanity-c heck attributes when setting. _gen_time_trace ( t_start : float , num_timest eps: int ) → ndar ra y Generate a time trace starting at t_start , of length num_timesteps with time step dt P arameters • t_start ( float ) – Start time, in seconds • num_timesteps ( int ) – Number of time s teps to g enerate, in units of dt 518 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 R eturn ndarra y Generated time trace R eturn type ndarr ay _gen_timeseries ( output : ndarr ay , **kw args ) → T imeSer ies W rap a clock ed / rasterised output ar ra y into a TimeSeries object Output TimeSeries will be of the appropr iate subclass, and will be named nicel y . P arameters • output ( np.ndarray ) – The cloc k ed or raster ised output data (T, N) • **kwargs – A dditional ke yw ord arguments to TimeSeries R eturns The data in output wrapped into a TimeSeries object R eturn type TimeSeries _gen_tscontinuous ( output : ndarr ay , dt : float | N one = N one , t_start : float | None = N one , name: str | N one = N one , periodic: bool = F alse , interp_kind: str = ' pr evious ' ) → TSContinuous W rap a rasterised output ar ra y as a TSContinuous object to present as output f or this module Output TSContinuous s will be named nicel y , with cor rect s tar t times, durations, etc. Sev eral attr ibutes of the TSContinuous object can be set as arguments here. P arameters • output ( np.ndarray ) – A cloc k ed time ser ies data ar ra y (T, N) • dt ( Optional [ float ] ) – The time-step of the cloc ked ar ra y output . If not pro vided, the module dt will be used • t_start ( Optional [ float ] ) – The star t time of the output TSContinuous object, in seconds. If not pro vided, the module time bef ore ev olution will be used • name ( Optional [ str ] ) – The desired name of the TSContinuous object. If not pro vided, the object will be named nicel y according to the module name • periodic ( bool ) – Flag to indicate whether the retur ned TSContinuous should be per i- odic. Def ault: False , the TSContinuous will not be per iodic • interp_kind ( str ) – The sty le of inter polation to appl y to the retur ned TSContinuous object. Def ault: "previous" R eturns The wrapped output data as a TSContinuous object R eturn type TSContinuous _gen_tsevent ( output : ndarr ay , dt : float | N one = None , t_s tar t : float | None = N one , name: str | N one = N one , periodic: bool = F alse , num_c hannels: int | N one = None , spikes_at_bin_s tar t : bool = F alse ) → TSEv ent W rap a rasterised output ar ra y as a TSEvent object to present as output f or this module Output TSEvent s will be named nicel y , with cor rect start timesm durations, etc. Se v eral attr ibutes of the TSEvent object can be set as arguments here. P arameters 48.2. Base classes 519 Rockpool, Release 3.0.3 R eturn type tuple property full_name: str The full name of this module (class plus module name) T ype str property input_type: type The TimeSeries class accepted b y this module T ype type modules () → Dict R etur n a dictionar y of all sub-modules of this module R eturns A dictionar y containing all sub-modules. Each item will be named with the sub-module name. R eturn type dict property name: str The name of this module, or an empty string if None T ype str property output_type: type The TimeSeries class retur ned b y this module T ype type parameters ( f amily : str | T uple | List = N one ) → Dict R etur n a nested dictionary of module and submodule Parameters Use this method to inspect the P arameters from this and all submodules. The optional ar gument family allo w s y ou to search f or Parameters in a particular famil y — f or e xample "weights" f or all w eights of this module and nested submodules. Although the family argument is an arbitrary str ing, reasonable choises are "weights" , "taus" f or time constants, "biases" f or biases. . . Examples Obtain a dictionar y of all P arameters f or this module (including submodules): >>> mod . parameters() dict{ ... } Obtain a dictionar y of P arameters from a par ticular famil y: >>> mod . parameters( "weights" ) dict{ ... } P arameters family ( str ) – The famil y of P arameters to search f or . Def ault: None ; retur n all parameters. 526 Chapter 48. F ull API summar y for Rockpool Rockpool, Release 3.0.3 R eturns A nested dictionary of Parameters of this module and all submodules R eturn type dict reset_all () → N one R eset the inter nal state and time of this module and all sub-modules R eturn type N one reset_parameters () R eset all parameters in this module R eturns The updated module is retur ned f or compatibility with the functional API R eturn type Module reset_state () → ModuleBase R eset the state of this module R eturns The updated module is retur ned f or compatibility with the functional API R eturn type Module reset_time () → N one R eset the inter nal time of this module and all sub-modules to zero R eturn type N one set_attributes ( new_attributes: dict ) → ModuleBase Set the attr ibutes and sub-module attributes from a dictionar y This method can be used with the dictionar y retur ned from module e v olution to set the ne w state of the module. It can also be used to set multiple parameters of a module and submodules. Examples Use the functional API to e v ol v e, obtain new s tates, and set those states: >>> _, new_state, _ = mod( input ) >>> mod = mod . set_attributes(new_state) Obtain a parameter dictionar y , modify it, then set the parameters bac k: >>> params = mod . parameters() >>> params[ ' w_input ' ] *= 0. >>> mod . set_attributes(params) P arameters new_attributes ( dict ) – A nested dictionary containing parameters of this module and sub-modules. 48.2. Base classes 527 [Document text truncated for crawler view.]