Design and Implementation of a SNTP Client on FPGA

Design and Implemen a ion o a SNTP Clien on FPGA J. Viejo, J. Juan, M. J. Bellido, E. Os ua, A. Millan, P. Ruiz-de-Cla ijo, A. Mu˜ noz, and D. Gue e o G upo ID2 (In es igacion y Desa ollo Digi al) Depa amen o de Tecnologia Elec onica-Uni e sidad de Se illa E. T. S. Ing. In o ma ica, Campus Uni e si a io Reina Me cedes 41012 Se illa (SPAIN) Email: [email p o ec ed], [email p o ec ed], [email p o ec ed], [email p o ec ed], [email p o ec ed], [email p o ec ed], am i[email p o ec ed], [email p o ec ed] Abs ac — This con ibu ion p esen s he design and imple- men a ion o a SNTP clien module sui able o IEC 61850 en i onmen s ully done in ha dwa e. The module is able o p o ide synch oniza ion and accu a e ime e e ence wi hin a mi- c osecond wi h espec o a SNTP se e , in a ex emely compac , cos -e ec i e and low powe de ice comple ely implemen ed in a low g ade FPGA chip. The e o e i can be an ideal eplacemen o expensi e compu e -based solu ions o dedica ed GPS ecei e s in a wide ange o indus ial applica ions. This SNTP clien is pa o a common echnological pla o m o implemen ing Remo e Te minal Uni s (RTUs) unde IEC 61850. I. INTRODUCTION Time s amping is a c i ical ask in many indus ial con ol sys ems. Da a acquisi ion by Remo e Te minal Uni s (RTUs) being a ypical example. In his sense, he indus y no m IEC 61850 [1] de ines he Simple Ne wo k Time P o ocol (SNTP) [2] o e E he ne as a s anda d way o synch onize a se o subs a ions wi h a ime se e . SNTP is a simpli ied e sion o he mo e gene al Ne wo k Time P o ocol (NTP) [3] ha is commonly used in In e ne se e s and ou e s. Bo h SNTP and NTP sha e he same communica ion p o ocol and da a o ma , he main di e ence being ha NTP uses sophis ica ed algo i hms ha ensu es a co ec synch oniza ion wi h mul iple se e s unde highly a iable la ency da a links, which is common in a wo ld wide ne wo k like he In e ne . On he con a y, SNTP co e s he synch oniza ion wi h a single se e and uses a simpli ied s a eless algo i hm, which makes i sui able o embedded sys ems in a con olled indus ial en i onmen . Ne e heless, a SNTP clien may communica e wi h ei he a SNTP se e o a ull NTP se e . In his scena io, a ime se e will ypically ga he ac- cu a e ime in o ma ion om an absolu e e e ence like an accu a e clock o a GPS ecei e . SNTP clien s loca ed a he subs a ions will synch onize wi h he se e h ough he Local A ea Ne wo k (LAN) making i unnecessa y o ins all absolu e e e ences a he subs a ions. SNTP clien s will hen p o ide he nea by elec onic equipmen wi h he necessa y ime in o ma ion. This con ibu ion is pa o a p ojec suppo ed by he Minis y o Indus y o Spain and leaded by he Tel en company [4] which inali y is o de elop a Common Tech- nological Pla o m (PTC) o acili a e he implemen a ion o he unc ionali y ypically ound in Remo e Te minal Uni s (RTU) used o con ol he public powe g id. A key poin in he p ojec is o assu e he synch oniza ion o elec onic equipmen wi hin he ange o a ew mic oseconds. This synch oniza ion is achie ed by he use o SNTP clien s and se e s ully implemen ed in ha dwa e. SNTP clien and se e design is di ided in h ee main phases: 1) Basic p o ocol s ack implemen a ion and in eg a ion wi h E he ne con olle . A leas he ollowing p o ocols a e needed: IP, ARP, UDP, and NTP. 2) NTP clien implemen a ion: issuing o eques s, answe p ocessing and local clock synch oniza ion. 3) NTP se e implemen a ion: synch oniza ion wi h an ex e nal e e ence and eques p ocessing. This pape desc ibes he cu en s a us o he clien imple- men a ion ha includes phase 1 and pa o phase 2. The es o his con ibu ion is o ganized as ollows: a b ie in oduc ion o NTP and SNTP is included in sec ion II, sec ion III enume a es he equi emen s and gene al speci ica- ions o he sys em, sec ion IV gi es he de ails o he design and implemen a ion o he ha dwa e SNTP clien , sec ion V includes some implemen a ion esul s and sec ion VI discusses some conclusions. II. NTP/SNTP PROTOCOL BASIC OPERATION The ope a ion o he NTP/SNTP p o ocol is e y simple (Fig. 1). The clien sends a eques o he se e by issuing an UDP da a packe whe e he ime o i s local clock (T1) is included. When he eques is ecei ed a he se e a new ime s amp T2 is gene a ed wi h he ecep ion ime as gi en by he se e ’s local clock. A e p ocessing he eques , he se e issues a eply including he ime a which he eply lea es he se e (T3). When he clien ecei es he eply, he a i al ime (T4) is also anno a ed. Wi h his se o imes amps he clien can calcula e he ound ip ime ( d) and he ime o se be ween he clien ’s and se e ’s clocks ( o se ). Assuming a symme ic connec ion i gi es: d = (T4−T1)−(T3−T2) o se =(T2−T1)+(T3−T4) 2(1) Fig. 1. Ope a ion o he NTP/SNTP p o ocol. Using he calcula ed o se , he clien can co ec i s local clock o ma ch he se e ime. So wa e implemen a ions o NTP ipically achie e ime synch oniza ion wi hin a mil- lisecond wi h espec o he se e [3]. The e a e wo main sou ces o e o . The i s one is he asymme y in he ne wo k communica ion when he ime spen by he clien ’s eques o each he se e is di e en o he ime spen by he answe o each he clien . This is due o unp edic able la ency in ne wo k equipmen , specially when collisions ake place and he numbe o he de ices in ol ed inc eases. The second main sou ce o e o is due o he a iable ime gap be ween he ins an he ime s amp is egis e ed in he da ag am and he eal ins an he da ag am lea es o eaches he hos . In ypical so wa e implemen a ion, hese ime s amps a e egis e ed by clien /se e so wa e unning as a use le el applica ion (Fig. 2) so he ime s amp e o will depend on he ime spen p ocessing he da ag am as i goes h ough he p o ocol s ack and so wa e laye s. This e o will la gely depend on sys em load, de ailed so wa e implemen a ion, e c. The p ecision o he NTP synch oniza ion can be la gely imp o ed by doing he ime-s amping ope a ion in lowe laye s [5], he e o e some ope a ing sys em ke nels like Linux o F eeBSD suppo NTP p ocessing in he ke nel [6]. This way, p ecision may each some ens o mic oseconds. The highes p ecision in he ime-s amping ope a ion is only achie able i done by he E he ne de ice ha dwa e as soon as he packe s a i e o lea e he in e ace. III. SYSTEM SPECIFICATION The main objec i e o his con ibu ion is o build cos - e ec i e, au onomous, compac and highly accu a e SNTP clien and se e modules sui able o , bu no limi ed o, IEC 61850 en i onmen s. The SNTP se e will use a s anda d GPS ecei e as a ime e e ence. I will use he PPS (Pulse Pe Second) signal and NMEA da a om he GPS o synch onize i s in e nal clock. The SNTP clien will synch onize wi h he se e h ough he local a ea ne wo k using he NTP p o o- col, and will p o ide a PPS signal and NMEA in o ma ion Fig. 2. Laye s whe e NTP can be implemen ed. h ough a se ial in e ace hus emula ing a GPS ecei e . A ypical scena io is depic ed in Fig. 3 whe e he se e (H- SNTPD) ga he s he ime om he GPS ecei e and clien s (H-SNTPC) p o ide ime and synch oniza ion in o ma ion o emo e e minal uni s (RTU). Mo e speci ically, he SNTP clien should mee he ollow- ing c i e ia: •The clien will ope a e in a s anda d 10/100/1000 MHz E he ne LAN. •The clien will be con igu ed au oma ically using he BOOTP [7] p o ocol so ha he con igu a ion o all he clien s can be cen alized in a single BOOTP se e . •The p ecision o he local clock a he clien should be wi hin 10 µs wi h espec o he se e ’s clock in op imal condi ions: ha dwa e ime-s amping in he se e and di ec LAN connec ion wi hou swi ches. In ypical condi ions (so wa e se e and s anda d swi ch connec- ion) p ecision should be always wi hin 1 ms. •The whole clien design should i in a single, low densi y FPGA chip and should need no addi ional ha dwa e, so ha cos o sys em pa s will be unde $20 be o e mass p oduc ion. •Low powe . Implemen ed in a low densi y, low equency FPGA, he clien will consume unde 1 W o a e age powe which is much lowe han a compu e -based im- plemen a ion ha would consume abou 100 W. IV. DESIGN AND IMPLEMENTATION In his sec ion, he mos impo an aspec s o design and implemen a ion a e commen ed. A diag am o he modules ha o m he SNTP clien is shown in Fig. 4. We can dis inguish he ollowing pa s: con ol uni , E he ne MAC con olle , SNTP clien module, and PPS gene a ion and RMC ame ansmission module. Nex , we will b ie ly explain he unc ionali y o each one o hese subsys ems. The con ol uni is in cha ge o a bi a ing he ope a ion o he es o he modules in o de o pe o m he adequa e ask in each momen . The module has been modeled as a Fini e S a e Machine using Ve ilog coding acco ding o he s uc u e Fig. 3. Typical scena io o deploying ha dwa e SNTP clien and se e s. Fig. 4. Block diag am o he SNTP clien . desc ibed in [8]. The con ol uni de ines wo main ope a ing modes: 1) Con igu a ion. When he SNTP clien s a s o ope a e o a e a sys em ese , an au oma ic con igu a ion p o- cess is pe o med acco ding o he Boo s ap P o ocol (BOOTP) [7]. This p ocess consis s o inding he SNTP clien MAC add ess and a se ies o con igu a ion pa- ame e s like SNTP clien and se e IP add esses and he RS-232 se ial po baud a e. We ha e used BOOTP because o i s simplici y compa ed o DHCP [9] which makes i mo e sui able o be implemen ed in ha dwa e, while he ex ended capabili ies o DHCP a e no use ul o he in ended applica ion and would only in oduce ex a de elopmen and esou ce cos s. 2) No mal ope a ion. Once he con igu a ion p ocess has inished, he SNTP clien begins o wo k in o he no mal ope a ion mode. In his mode, he de ice ca ies ou di e en asks which we a e going o summa ize nex . Fi s ly, he SNTP clien needs o know he SNTP se e MAC add ess, so he clien includes a simple implemen- a ion o he Add ess Resolu ion P o ocol (ARP) [10]; as well, he clien mus be able o send an ARP eply packe whene e ano he de ice eques s i s MAC add ess. Secondly, he clien mus ansmi a ime eques packe (SNTP message) a secondly in e als. Finally, when he SNTP clien ecei es he ime eply packe , he imes amps ob ained a e egis e ed so ha he SNTP clien module can synch onize he local clock. The E he ne MAC con olle is in cha ge o con olling a s anda d Fas E he ne PHY de ice, allowing us o ansmi and ecei e E he ne ames acco ding o IEEE 802.3 speci ica ion. The implemen a ion o his module has been ca ied ou using he T i-mode E he ne MAC IP-co e a ailable om he OpenCo es p ojec in i s web po al openco es.o g. Mo eo e , his IP-co e has a FIFO in e ace o use applica ions which acili a es he SNTP clien design. The use in e ace has been de eloped acco ding o he spec- i ica ion documen [11]. This in e ace has been implemen ed as a ini e s a e machine coded in Ve ilog, and is o med by a ansmi e module and a ecei e module. So, on he one hand, he ansmi e module is able o ansmi h ee di e en E he ne ames: BOOTP Reques , ARP Reques /Reply and ime eques packe s, which a e s o ed in a RAM. This module also includes a memo y upda ing componen ha is in cha ge o upda ing he di e en packe ields be o e ansmi ing hem. On he o he hand, he ecei e module is able o iden i y he ollowing ames: BOOTP Reply, ARP Reques /Reply and ime eply packe s, uling ou he es o E he ne ames. The SNTP clien module is in cha ge o calcula ing he clock o se using he imes amps and synch onizing he local ime. Addi ionally, a d i con ol is ca ied ou in o de o imp o e he local clock accu acy. A his momen , his componen is being de eloped using he Sys em Le el ool Sys em Gene a o o DSP acco ding o he me hodology p esen ed in [12]. Finally, he PPS gene a ion and RMC ame ansmission module is in cha ge o gene a ing a synch oniza ion signal (PPS+NMEA) which will be sen h ough he se ial po o a Remo e Te minal Uni . To implemen his module he same me hodology used o build he SNTP clien module will be employed. A he ime o w i ing, he sys em is ully speci ied and he mos c i ical and complex pa s ha e been implemen ed: he con ol uni , he use in e ace and pa o he SNTP clien unc ions. V. RESULTS In his sec ion, simula ion and ha dwa e implemen a ion esul s a e desc ibed in some de ail. A. Simula ion esul s In o de o check ha he designs wo ks co ec ly, he ollowing simula ion p ocess has been ca ied ou . A he i s s age, he design has been e i ied using Simulink and ModelSim. Fo he gene a ion o he inpu s imuli, he Sou ce Blockse o Simulink has been employed. A he second s age, we ha e used he Xilinx ool ChipScope P o o pe o m he on-chip e i ica ion o he clien . In his way, we ha e e i ied TABLE I HARDWARE IMPLEMENTATION RESULTS ON SPARTAN-3E XC3S500E. Figu e o me i Usage (%) Slices 1,122 (24%) Slice Flip Flops 1,287 (13%) 4 inpu LUTs 1,303 (13%) Bonded IOBs 35 (15%) Block RAMs 10 (50%) GCLKs 6 (25%) Maximum ope a ion equency 96 MHz he co ec ansmission and ecep ion o he di e en packe ypes: BOOTP, ARP, and SNTP messages. B. Ha dwa e implemen a ion esul s In his subsec ion, ha dwa e implemen a ion esul s will be p esen ed. Speci ically, wo igu es o me i will be analyzed: ha dwa e esou ces and maximum ope a ion equency. The design has been implemen ed on a Spa an-3E XC3S500E FPGA. The Table I shows he design esul s in he cu en de elopmen s age. I is wo h no no e ha al hough he implemen a ion is no inished, he mos esou ce consuming blocks a e implemen ed, which a e he use in e ace and he con ol logic. Conside ing his and he ac ha he e is s ill plen y o oom o op imiza ion in he al eady implemen ed ha dwa e, i is expec ed ha he inal esou ce equi emen s will be e y close o he da a in Table I. VI. CONCLUSION The design o a SNTP clien comple ely done in ha dwa e has been p esen ed. By using a high le el me hodology and s anda d FPGA echnology i is possible o p oduce a high accu a e, cos -e ec i e and lexible solu ion o accu a e ime dis ibu ion and ime s amping in indus ial en i onmen s ha ag ees wi h in e na ional s anda ds. Fi s p o o ypes a e expec ed o p o ide synch oniza ion in he ange o he mi- c osecond in a compac and cheap de ice ha would sub- s i u e expensi e compu e -based solu ions o dedica ed GPS ecei e s. ACKNOWLEDGMENT This wo k has been pa ially suppo ed by he PROFIT- MITC PTC FIT-330100-2006-60 p ojec and he Andalusian Regional Go e nmen ’s EXC-2005-TIC-1023 p ojec . REFERENCES [1] IEC 61850 Communica ion Ne wo ks and Sys ems In Subs a ions, Tech- nical Commi ee 57. In e na ional Elec o echnical Commission. [2] D. L. Mills, Simple Ne wo k Time P o ocol (SNTP) Ve sion 4 o IP 4, IP 6 and OSI, RFC 4330, Ca ego y: In o ma ional. Janua y 2006. [3] D. L. Mills, Ne wo k Time P o ocol (Ve sion 3) Speci ica ion, Implemen- a ion and Analysis, RFC 1305, S a us: D a S anda d. Ma ch 2006. [4] Tel en Company Web Po al. h p://www. el en .com. [5] T. Skeie, S. Johannessen, and Ø. Holmeide, Highly Accu a e Time Synch oniza ion o e Swi ched E he ne , 8 h IEEE In e na ional Con e ence on Eme ging Technologies and Fac o y Au oma ion, ETFA 2001, An ibes-Juan les Pins, F ance Oc obe 2001. [6] D. L. Mills and P. H. Kamp. The nanoke nel, P oc. P ecision Time and Time In e al (PTTI) Applica ions and Planning Mee ing Res on VA, No embe 2000. [7] B. C o and J. Gilmo e, Boo s ap P o ocol (BOOTP), RFC 951, Sep embe 1985. [8] C. E. Cummings, The Fundamen als o E icien Syn hesizable Fini e S a e Machine Design using NC-Ve ilog and BuildGa es, In e na ional Cadence Use g oup con e ence, ICU 2002, San Jose, Cali o nia, Sep embe 2002. [9] R. D oms, Dynamic Hos Con igu a ion P o ocol, RFC 2131, Ma ch 1997. [10] D. C. Plumme , An E he ne Add ess Resolu ion P o ocol, RFC 826, a.k.a. STD 37, No embe 1982. [11] J. Gao, 10 100 1000 Mbps T i-mode E he ne MAC Speci ica ion, OPENCORES.ORG, Janua y 2006. [12] J. Viejo, M. J. Bellido, A. Millan, E. Os ua, J. Juan, P. Ruiz-de- Cla ijo, and D. Gue e o, E icien design and implemen a ion on FPGA o a Mic oBlaze pe iphe al o p ocessing di ec elec ical ne wo ks measu emen s, Fi s IEEE Symposium on Indus ial Embedded Sys ems, IES 2006, An ibes-Juan les Pins, F ance, Oc obe 2006.