Anomaly-based network intrusion detection methods

INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER Anomaly-based Ne wo k In usion De ec ion Me hods Pa el NEVLUD, Mi osla BURES, Lukas KAPICAK, Ja osla ZDRALEK Depa men o Telecommunica ions, Facul y o Elec ical Enginee ing and Compu e Science, VSB–Technical Uni e si y o Os a a, 17. lis opadu 15, 708 33 Os a a-Po uba, Czech Republic pa [email p o ec ed], mi osla[email p o ec ed], lukas.k[email p o ec ed], ja osla .[email p o ec ed] Abs ac . The a icle deals wi h de ec ion o ne wo k anomalies. Ne wo k anomalies include e e y hing ha is qui e di e en om he no mal ope a ion. Fo de- ec ion o anomalies we e used machine lea ning sys- ems. Machine lea ning can be conside ed as a suppo o a limi ed ype o a i icial in elligence. A machine lea ning sys em usually s a s wi h some knowledge and a co esponding knowledge o ganiza ion so ha i can in e p e , analyse, and es he knowledge acqui ed. The e a e se e al machine lea ning echniques a ail- able. We es ed Decision ee lea ning and Bayesian ne wo ks. The open sou ce da a-mining amewo k WEKA was he ool we used o es ing he classi y, clus e , associa ion algo i hms and o isualiza ion o ou esul s. The WEKA is a collec ion o machine lea ning algo i hms o da a mining asks. Keywo ds Anomaly-based de ec ion, a ack, bayesian ne - wo ks, WEKA. 1. In oduc ion Nowadays, compu e ne wo k is a equen a ge o a acks in o de o ob ain con iden ial da a, o un- a ailabili y o ne wo k se ices. To de ec and p e en hese a acks, he e a e a la ge numbe o so wa e o ha dwa e solu ions such as IDS (In usion De ec ion Sys ems), i ewalls and moni o ing sys ems. These a acks inc eased no mal ne wo k a ic ha appea s as some hing undesi able, wha would no oc- cu in he ne wo k. Such de ia ions om no mal op- e a ion a e called as ne wo k anomalies. Be ween ne - wo k anomalies include e e y hing ha is qui e di e - en om he no mal ope a ion o he ne wo k [1]. Anomalies a e alues in a s a is ical sample which does no i a pa e n ha desc ibes mos o he da a Fig. 1: A simple example o anomalies. poin s. Figu e 1 illus a es anomalies in a simple 2- dimensional da a se . The da a has one no mal e- gions, since mos obse a ions lie in his egion. Th ee poin s ha a e su icien ly a away om he egions a e anomalies. One o hese poin s is bo de poin ha can be de ec ed as anomaly. 2. De ec ion o Ne wo k Anomalies Ne wo k anomalies can be de ec ed in se e al ways. Each me hod has i s ad an ages and disad an ages, bu in p ac ice he e a e h ee commonly used me h- ods. Them oge he hey can de elop sys ems such as IDS so wa e. 2.1. Compa ing Signa u es The p inciple o his me hod is he compa ison o ne - wo k da a wi h a da abase o signa u es. Signa u e da abase con ains pa e ns o da a anomalies. Da a anomaly pa e n is ac ually a desc ip ion o a ypi- cal da a sequence ha cha ac e izes he anomaly. The c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 468 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER p inciple can be seen in Figu e 2. I used he same p inciple as in he an i- i us p og ams. Fig. 2: Compa ing signa u es. The e ec i eness o anomaly de ec ion using signa- u e ecogni ion is highly dependen on he quali y o he da abase o signa u es. The big disad an age is al- mos no de ec ion o new ypes o a acks called Ze o day a ack, because i is no in he da abase signa u e pa e n o his ype o anomaly [4]. 2.2. S a e ul P o ocol Analysis S a e ul p o ocol analysis assumes ha each p o ocol used o ne wo k communica ion is speci ied, such as RFC. Thanks o p ecise speci ica ions, all connec ions using p o ocols de ined s a e. Each e en mus occu a he igh momen , he s a e. This makes i possible o desc ibe he p o ocol as a s a e machine. Figu e 3 illus a es an example o s a e ul machine. Fig. 3: S a e ul p o ocol analysis. The ad an age o his me hod is less equen up- da es. The s a e ul analysis needs upda e only a e he change o p o ocol o he ins alla ion o a new one [3]. 2.3. Beha io al Analysis The me hod o beha io al analysis is based on he as- sump ion ha he eme gence o anomalies can be de- ec ed by he de ia ion om he no mal o expec ed ne wo k beha io . Model o no mally o an icipa ed beha io o he ne wo k is c ea ed based on ne wo k moni o ing and collec ing e e ence in o ma ion. The e e ence in o ma ion is compiled model no mal beha io and ne wo k a ic is subsequen ly compa ed wi h his model. Any de ia ion om such a lea ned model is au oma ically conside ed an anomaly. The p inciple can be seen in Fig. 4. Fo beha io al analysis and c ea e ne wo k’s model can be used MLS (Machine Lea ning Sys ems). Fig. 4: Beha io al analysis. The disad an age o his me hod is p ecisely he ine de ec ion. Any de ia ion om he no mal model is de ec ed e en hough i is no an a ack o h ea . I is due o he ac ha he c ea ion o he model can no cap u e all ypes o ne wo k a ic and use ac i i y on he ne wo k. This model is c ea ed o some ex en dis o ed. On he o he hand, beha io al analysis p o ides an ad an age in e ms o de ec ion o comple ely new ypes o h ea s, o example, by compa ing de ec ion signa u es did no eac a all. 3. Machine Lea ning Sys ems I we wan o be able o sol e he compu e p oblem, some in elligence is needed. Machine lea ning can be conside ed as a suppo o a limi ed ype o a i icial in elligence. Algo i hms MLS can mo e on wi h he de elopmen o compu e s. This means ha compu e s a e no longe jus a da abase compa ing se s o da a. A machine lea ning sys em usually s a s wi h some knowledge and a co esponding knowledge o ganiza- ion so ha i can in e p e , analyze, and es he knowledge acqui ed. The p inciple can be seen in Fig. 5. c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 469 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER Fig. 5: P inciple o machine lea ning sys em. T aining is he p ocess o making he sys em able o lea n. I may consis o andomly selec ed examples ha include a a ie y o ac s and de ails including i ele an da a. The lea ning echniques can be cha - ac e ized as a sea ch h ough a space o possible hy- po heses o solu ions. Backg ound knowledge can be used o make lea ning mo e e icien by educing he sea ch space. The success o machine lea ning sys em also de- pends on he algo i hms. These algo i hms con ol he sea ch o ind and build he knowledge s uc u es. The algo i hms should ex ac use ul in o ma ion om aining examples. The e a e se e al machine lea ning echniques a ailable [2]. Among he bes -known machine lea ning algo i hms include: •Decision ee lea ning. •A i icial neu al ne wo ks. •Gene ic p og amming. •Clus e ing. •Bayesian ne wo ks. •Rep esen a ion lea ning. 4. Decision T ee Lea ning Decision ee lea ning is ‘a me hod o app oxima ing disc e e alued unc ions ha is obus o noisy da a and capable o lea ning disjunc i e exp essions’ acco d- ing o [5]. Ross Quinlan has p oduced se e al wo king decision ee induc ion me hods ha ha e been implemen ed in his p og ams, ID3, C4.5 and C5. Decision ee induc- ion akes a se o known da a and induces a decision ee om ha da a. The ee can hen be used as a ule se o p edic ing he ou come om known a ibu es. The ini ial da a se om which he ee is induced is known as he aining se . The decision ee akes he op-down o m. A he op is he i s a ibu e and i s alues, om his nex b anch leads o ei he an a - ibu e o an ou come. E e y possible lea o he ee e en ually leads o an ou come. 4.1. Decision T ees – C4.5 C4.5 is an algo i hm de eloped by Ross Quinlan ha gene a es Decision T ees (DT), which can be used o classi ica ion p oblems. I imp o es (ex ends) he ID3 algo i hm by dealing wi h bo h con inuous and dis- c e e a ibu es, missing alues and p uning ees a e cons uc ion. I s comme cial successo is C5.0/See5, a lo as e ha C4.5, mo e memo y e icien and used o building smalle decision ees. J48 is an open sou ce Ja a implemen a ion o he C4.5 algo i hm in he WEKA da a mining ool. Algo i hm 1 C4.5(D) Inpu : an a ibu e- alued da ase D 1: T ee = {} 2: i Dis ”pu e” OR o he s opping c i e ia me hen 3: e mina e 4: end i 5: o all a ibu e a ∈Ddo 6: Compu e in o ma ion- heo e ic c i e ia i we spli a 7: end o 8: abes = Bes a ibu e acco ding o abo e compu ed c i e ia 9: T ee = C ea e a decision node ha es s abes in he oo 10: D = Induced sub-da ase s om Dbased on abes 11: o all D do 12: T ee = C4.5(D ) 13: A ache T ee o he co esponding b anch o T ee 14: end o 15: e u n T ee The gene ic desc ip ion o how C4.5 wo ks is shown in Algo i hm 1. A decision ee is buil op-down om a oo node and in ol es pa i ioning he da a in o subse s ha con ain ins ances wi h simila alues (ho- mogenous). Decision ee algo i hm uses en opy o calcula e he homogenei y o a sample. I he sample is comple ely homogeneous he en opy is ze o and i he sample is an equally di ided i has en opy o one. The en opy o class andom a iable ha akes on c alues wi h p obabili ies p1, p2, . . . , pcis gi en by: En opy(S) = c X i=1 −pilog2pi.(1) Figu e 6 shows he o m o he en opy unc ion el- a i e o a bina y classi ica ion. Fig. 6: En opy unc ion. c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 470 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER (a) pingsweep (b) po sweep Fig. 7: J48 classi ie esul s. (a) pingsweep (b) po sweep Fig. 8: BayesNe classi ie esul s. The es ima ion c i e ion in he decision ee algo- i hm is he selec ion o an a ibu e o es a each decision node in he ee. The goal is o selec he a ibu e ha is mos use ul o classi ying examples. A good quan i a i e measu e o he wo h o an a - ibu e is a s a is ical p ope y called in o ma ion gain ha measu es how well a gi en a ibu e sepa a es he aining examples acco ding o hei a ge classi ica- ion. This measu e is used o selec among he can- dida e a ibu es a each s ep while g owing he ee. The in o ma ion gain is based on he dec ease in en- opy a e a da ase is spli on an a ibu e. Con- s uc ing a decision ee is all abou inding a ibu e ha e u ns he highes in o ma ion gain (i.e. he mos homogeneous b anches). Gain(S, A) = En opy(S)−X ∈V alues(A) |S | |S|En opy(S ),(2) whe e V alues(A) is he se o all possible alues o a - ibu e A, and S is he subse o S o which a ibu e Ahas alue (i.e. S ={sˆ I S|A(s) = }). The i s e m in he equa ion o in o ma ion gain is jus he en opy o he o iginal collec ion Sand he second e m is he expec ed alue o he en opy a e S is pa i ioned using a ibu e A. The expec ed en opy desc ibed by his second e m is simply he sum o he en opies o each subse S , weigh ed by he ac ion o examples |S |/|S| ha belong o S .Gain(S, A) is he e o e he expec ed educ ion in en opy caused by knowing he alue o a ibu e A. Pu ano he way, Gain(S, A) is he in o ma ion p o ided abou he a - ge a ibu e alue, gi en he alue o some o he a - ibu e A. The alue o Gain(S, A) is he numbe o bi s sa ed when encoding he a ge alue o an a bi- a y membe o S, by knowing he alue o a ibu e A. 5. Bayesian Ne wo ks Bayesian ne wo ks a e g aphical ep esen a ion o he ela ionship be ween a iables. G aphical ep esen a- c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 471 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER ion o Bayesian ne wo ks a e di ec ed acyclic g aphs wi h nodes and edges. Nodes ep esen a iables, pa- ame e s o hypo heses and edges ep esen condi ional dependencies. 5.1. Algo i hm o Nai e Bayesian The Nai e Bayesian classi ie is based on Bayes’ he- o em wi h independence assump ions be ween p edic- o s. Bayes heo em p o ides a way o calcula ing he pos e io p obabili y, P(c|x), om P(c), P(x), and P(x|c). Nai e Bayes classi ie assumes ha he e ec o he alue o a p edic o (x) on a gi en class (c) is independen o he alues o o he p edic o s. This as- sump ion is called class condi ional independence. P(c|x) = P(x|c)P(c) P(x),(3) P(c|x) = P(x1|c)×P(x2|c)×· · ·×P(xn|c)×P(c),(4) whe e P(c|x) is he pos e io p obabili y o class ( a - ge ) gi en p edic o (a ibu e), P(c) is he p io p ob- abili y o class, P(x|c) is he likelihood which is he p obabili y o p edic o gi en class and P(x) is he p io p obabili y o p edic o . 6. Expe imen al Resul s Fo da a mining pla o m was chosen open sou ce p ojec WEKA [6]. WEKA is a collec ion o machine Fig. 9: Visualize esul s o pingsweep. lea ning algo i hms o da a mining asks. The algo- i hms can ei he be applied di ec ly o a da ase o called om you own Ja a code. WEKA con ains ools o da a p e-p ocessing, clas- si ica ion, eg ession, clus e ing, associa ion ules, and isualiza ion. I is also well-sui ed o de eloping new machine lea ning schemes. Tes ed ac i i ies we e cap u ed by he ne wo k a ic collec o and sa ed as pcap iles. These pcap iles a e bina y iles and can’ be ead di ec ly in o mos da a mining applica ions. These pcap iles we e con e ed in o cs iles wi h sc ip s using sha k [7]. WEKA accep s *.cs iles, *.a iles o a connec ion o a da abase. Fo his e- sea ch we e used con e ed cs iles ha was opened in WEKA. As pa o he p ep ocessing s ep, in o ma ion da a we e injec ed in o he da a which we e use ul o ain- ing o he da a mining algo i hm. Addi ionally insigni - ican da a we e elimina ed i i we e no se ing he o e all p ocess. To begin unning his da a h ough he algo i hms i was opened in WEKA explo e . Fo he pu pose o hese ini ial uns we selec ed all o he a ibu es. Fig. 10: Visualize esul s o po sweep. Figu e 7(a) shows he esul s in WEKA o a J48 Classi ie aining un on da a acqui ed du ing ping sweep o he a ge ne wo k. The esul s desc ibed he e we e de i ed om single nmap a ge scan, whe e an a ack compu e scanned a ic im ne wo k, p obing o ac i e IP add esses. Also Fig. 7(b) shows he esul s in WEKA o a J48 Classi ie aining un on da a acqui ed du ing po c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 472 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER sweep o he a ge compu e . The esul s desc ibed he e we e de i ed om single nmap a ge scan, whe e an a ack compu e scanned a ic im compu e , p ob- ing o open po s. The e we e chosen only 3 a ibu es o isualize ex- pe imen al esul s. These a ibu es we e IP sou ce add ess, IP des ina ion add ess and P o ocol. Figu e 9 and Fig. 10 show ela ion be ween hese a ibu es. Figu e 8(a) shows he esul s in WEKA o a Nai e- Bayes Classi ie aining un on da a acqui ed du ing ping sweep o he a ge ne wo k. Also Fig. 8(b) shows he esul s in WEKA o a Nai eBayes Classi ie ain- ing un on da a acqui ed du ing po sweep o he a - ge compu e . 7. Conclusion and Fu u e Wo k In his pape , we ha e p esen ed de ec ion o ne wo k anomalies by using machine lea ning sys ems. Machine lea ning sys em usually s a s wi h some knowledge and du ing he ounds can imp o e i s knowledge. The e we e es ed some a acks in egula ne wo k a ic. As he i s a ack was used ping sweep o sub ne wo k a ge o ge in o ma ion abou ac i e IP ad- d esses. The po sweep was used as second a ack o scanning open po s a he a ge ic im compu e . Fi s , we cap u ed ne wo k a ic by he ne wo k collec o and sa ed da a as pcap o ma ile. Nex , we con e ed collec ed da a om pcap ile in o cs o - ma ile. We used some sc ip s by means o sha k o con e da a om pcap o cs ile o ma . Nex con- e ed cs da a was inse ed in he WEKA so wa e o use classi ica ion o da a. Finally classi ied da a was isualize by WEKA so wa e. Fu u e wo k expec s o use mo e a ibu es ha will be ge om pcap iles. We also assume he use o o he classi ica ion me hods and o he da a mining al- go i hms. Acknowledgmen The esea ch leading o hese esul s has ecei ed und- ing om he Eu opean Communi y’s Se en h F ame- wo k P og amme (FP7/2007-2013) unde g an ag ee- men no. 218086. Re e ences [1] FOWLER, Ch. A. and R. J. HAMMELL II. Build- ing Baseline P ep ocessed Common Da a Se s o Mul iple Follow-on Da a Mining Algo i hms. In: P oceedings o he Con e ence on In o ma ion Sys- ems Applied Resea ch 2012. New O leans: ED- SIG, 2012, pp. 1–17. ISSN 2167-1508. [2] FARRAPOSO, F., P OWEZARSKI and E. MONTEIRO. NADA–Ne wo k Anomaly De ec- ion Algo i hm. In: 18 h IFIP/IEEE In e na- ional Wo kshop on Dis ibu ed Sys ems: Ope - a ions and Managemen , DSOM 2007. San Jose: Sp inge Ve lag, 2007, ol. 4785, pp 191—194, ISBN 978-3-540-75694-1. [3] DAS, K. P o ocol Anomaly De ec ion o Ne wo k-based In usion De ec ion. The SANS Ins i u e [online]. 2002. A ailable a : h p://www.sans.o g/ eading_ oom/ whi epape s/de ec ion/p o ocol_anomaly_ de ec ion_ o _ne wo kbased_in usion_ de ec ion_349?show=349.php&ca =de ec ion. [4] RICHARD, M. In usion De ec ion FAQ: A e he e limi a ions o In usion Signa u es?. The SANS Ins i u e [online]. 2001. A ailable a : h p://www.sans.o g/ esou ces/id aq/ limi a ions.php. [5] MITCHELL, Tom M. Machine lea ning. Bos on: McG aw-Hill, 1997. ISBN 00-704-2807-7. [6] WEKA 3. Da a Mining So wa e in Ja a [online]. 2013. A ailable a : h p://www.cs.waika o.ac. nz/ml/weka/ [7] TSha k [online]. 2013. A ailable a : h p://www. wi esha k.o g/docs/man-pages/ sha k.h ml Abou Au ho s Pa el NEVLUD ecei ed his M.Sc. deg ee in elecommunica ion enginee ing om VSB–Technical Uni e si y o Os a a, Czech Republic in 1995. Since his yea he has been holding posi ion as an assis an p o esso a he Depa men o Telecommunica ions, VSB–Technical Uni e si y o Os a a. The opics o his esea ch in e es s a e communica ion echnologies, ne wo king and secu i y. Mi osla BURES ecei ed his M.Sc. deg ee in elecommunica ions om VSB–Technical Uni e si y o Os a a, Czech Republic in 2011. Since 2011 has been s udying Ph.D. deg ee a he same uni e si y. His esea ch is ocused on ne wo king, analysis o ne wo k’s da a and secu i y. c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 473 INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER Lukas KAPICAK ecei ed his M.Sc. deg ee in elecommunica ions om VSB–Technical Uni e si y o Os a a, Czech Republic in 2007. Since 2007 has been s udying Ph.D. deg ee a he same uni e si y. His esea ch is ocused on wi eless ansmission and da a low analysis, simula ion and op imiza ion. Ja osla ZDRALEK holds posi ion as an associa e p o esso wi h Depa men o Telecommunica ions, VSB–Technical Uni e si y o Os a a, Czech Republic. He ecei ed his M.Sc. deg ee in Compu e Science om Slo ak Technical Uni e si y o B a isla a, Slo akia in 1977. He ecei ed his Ph.D. deg ee om VSB–Technical Uni e si y o Os a a in 2002, disse a ion hesis ”Diagnos ic sys em wi hou disman ling o locomo i e con olle ”. His esea ch is ocused on aul ole an sys em and communica ion echnologies. c 2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 474