scieee Science in your language
[en] (orig)

Anomaly-based network intrusion detection methods

Abstract

The article deals with detection of network anomalies. Network anomalies include everything that is quite different from the normal operation. For detection of anomalies were used machine learning systems. Machine learning can be considered as a support or a limited type of artificial intelligence. A machine learning system usually starts with some knowledge and a corresponding knowledge organization so that it can interpret, analyse, and test the knowledge acquired. There are several machine learning techniques available. We tested Decision tree learning and Bayesian networks. The open source data-mining framework WEKA was the tool we used for testing the classify, cluster, association algorithms and for visualization of our results. The WEKA is a collection of machine learning algorithms for data mining tasks.

Read accessible full text

Anomaly-based network intrusion detection methods

Author: Nevlud, Pavel
Publisher: Vysoká škola báňská - Technická univerzita Ostrava
Year: 2013
Source: https://dspace.vsb.cz/bitstreams/9883fbeb-b3af-40bb-a0a9-501c2fad0122/download
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
Anomaly-based Ne wo k In usion De ec ion
Me hods
Pa el NEVLUD, Mi osla BURES, Lukas KAPICAK, Ja osla ZDRALEK
Depa men o Telecommunica ions, Facul y o Elec ical Enginee ing and Compu e Science,
VSB–Technical Uni e si y o Os a a, 17. lis opadu 15, 708 33 Os a a-Po uba, Czech Republic
pa [email p o ec ed], mi osla[email p o ec ed], lukas.k[email p o ec ed], ja osla .[email p o ec ed]
Abs ac . The a icle deals wi h de ec ion o ne wo k
anomalies. Ne wo k anomalies include e e y hing ha
is qui e di e en om he no mal ope a ion. Fo de-
ec ion o anomalies we e used machine lea ning sys-
ems. Machine lea ning can be conside ed as a suppo
o a limi ed ype o a i icial in elligence. A machine
lea ning sys em usually s a s wi h some knowledge and
a co esponding knowledge o ganiza ion so ha i can
in e p e , analyse, and es he knowledge acqui ed.
The e a e se e al machine lea ning echniques a ail-
able. We es ed Decision ee lea ning and Bayesian
ne wo ks. The open sou ce da a-mining amewo k
WEKA was he ool we used o es ing he classi y,
clus e , associa ion algo i hms and o isualiza ion o
ou esul s. The WEKA is a collec ion o machine
lea ning algo i hms o da a mining asks.
Keywo ds
Anomaly-based de ec ion, a ack, bayesian ne -
wo ks, WEKA.
1. In oduc ion
Nowadays, compu e ne wo k is a equen a ge o
a acks in o de o ob ain con iden ial da a, o un-
a ailabili y o ne wo k se ices. To de ec and p e en
hese a acks, he e a e a la ge numbe o so wa e o
ha dwa e solu ions such as IDS (In usion De ec ion
Sys ems), i ewalls and moni o ing sys ems.
These a acks inc eased no mal ne wo k a ic ha
appea s as some hing undesi able, wha would no oc-
cu in he ne wo k. Such de ia ions om no mal op-
e a ion a e called as ne wo k anomalies. Be ween ne -
wo k anomalies include e e y hing ha is qui e di e -
en om he no mal ope a ion o he ne wo k [1].
Anomalies a e alues in a s a is ical sample which
does no i a pa e n ha desc ibes mos o he da a
Fig. 1: A simple example o anomalies.
poin s. Figu e 1 illus a es anomalies in a simple 2-
dimensional da a se . The da a has one no mal e-
gions, since mos obse a ions lie in his egion. Th ee
poin s ha a e su icien ly a away om he egions
a e anomalies. One o hese poin s is bo de poin ha
can be de ec ed as anomaly.
2. De ec ion o Ne wo k
Anomalies
Ne wo k anomalies can be de ec ed in se e al ways.
Each me hod has i s ad an ages and disad an ages,
bu in p ac ice he e a e h ee commonly used me h-
ods. Them oge he hey can de elop sys ems such as
IDS so wa e.
2.1. Compa ing Signa u es
The p inciple o his me hod is he compa ison o ne -
wo k da a wi h a da abase o signa u es. Signa u e
da abase con ains pa e ns o da a anomalies. Da a
anomaly pa e n is ac ually a desc ip ion o a ypi-
cal da a sequence ha cha ac e izes he anomaly. The
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 468
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
p inciple can be seen in Figu e 2. I used he same
p inciple as in he an i- i us p og ams.
Fig. 2: Compa ing signa u es.
The e ec i eness o anomaly de ec ion using signa-
u e ecogni ion is highly dependen on he quali y o
he da abase o signa u es. The big disad an age is al-
mos no de ec ion o new ypes o a acks called Ze o
day a ack, because i is no in he da abase signa u e
pa e n o his ype o anomaly [4].
2.2. S a e ul P o ocol Analysis
S a e ul p o ocol analysis assumes ha each p o ocol
used o ne wo k communica ion is speci ied, such as
RFC. Thanks o p ecise speci ica ions, all connec ions
using p o ocols de ined s a e. Each e en mus occu
a he igh momen , he s a e. This makes i possible
o desc ibe he p o ocol as a s a e machine. Figu e 3
illus a es an example o s a e ul machine.
Fig. 3: S a e ul p o ocol analysis.
The ad an age o his me hod is less equen up-
da es. The s a e ul analysis needs upda e only a e
he change o p o ocol o he ins alla ion o a new one
[3].
2.3. Beha io al Analysis
The me hod o beha io al analysis is based on he as-
sump ion ha he eme gence o anomalies can be de-
ec ed by he de ia ion om he no mal o expec ed
ne wo k beha io . Model o no mally o an icipa ed
beha io o he ne wo k is c ea ed based on ne wo k
moni o ing and collec ing e e ence in o ma ion.
The e e ence in o ma ion is compiled model no mal
beha io and ne wo k a ic is subsequen ly compa ed
wi h his model. Any de ia ion om such a lea ned
model is au oma ically conside ed an anomaly. The
p inciple can be seen in Fig. 4. Fo beha io al analysis
and c ea e ne wo k’s model can be used MLS (Machine
Lea ning Sys ems).
Fig. 4: Beha io al analysis.
The disad an age o his me hod is p ecisely he ine
de ec ion. Any de ia ion om he no mal model is
de ec ed e en hough i is no an a ack o h ea . I
is due o he ac ha he c ea ion o he model can
no cap u e all ypes o ne wo k a ic and use ac i i y
on he ne wo k. This model is c ea ed o some ex en
dis o ed.
On he o he hand, beha io al analysis p o ides an
ad an age in e ms o de ec ion o comple ely new
ypes o h ea s, o example, by compa ing de ec ion
signa u es did no eac a all.
3. Machine Lea ning Sys ems
I we wan o be able o sol e he compu e p oblem,
some in elligence is needed. Machine lea ning can be
conside ed as a suppo o a limi ed ype o a i icial
in elligence. Algo i hms MLS can mo e on wi h he
de elopmen o compu e s. This means ha compu e s
a e no longe jus a da abase compa ing se s o da a.
A machine lea ning sys em usually s a s wi h some
knowledge and a co esponding knowledge o ganiza-
ion so ha i can in e p e , analyze, and es he
knowledge acqui ed. The p inciple can be seen in
Fig. 5.
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 469
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
Fig. 5: P inciple o machine lea ning sys em.
T aining is he p ocess o making he sys em able o
lea n. I may consis o andomly selec ed examples
ha include a a ie y o ac s and de ails including
i ele an da a. The lea ning echniques can be cha -
ac e ized as a sea ch h ough a space o possible hy-
po heses o solu ions. Backg ound knowledge can be
used o make lea ning mo e e icien by educing he
sea ch space.
The success o machine lea ning sys em also de-
pends on he algo i hms. These algo i hms con ol
he sea ch o ind and build he knowledge s uc u es.
The algo i hms should ex ac use ul in o ma ion om
aining examples. The e a e se e al machine lea ning
echniques a ailable [2].
Among he bes -known machine lea ning
algo i hms include:
•Decision ee lea ning.
•A i icial neu al ne wo ks.
•Gene ic p og amming.
•Clus e ing.
•Bayesian ne wo ks.
•Rep esen a ion lea ning.
4. Decision T ee Lea ning
Decision ee lea ning is ‘a me hod o app oxima ing
disc e e alued unc ions ha is obus o noisy da a
and capable o lea ning disjunc i e exp essions’ acco d-
ing o [5].
Ross Quinlan has p oduced se e al wo king decision
ee induc ion me hods ha ha e been implemen ed in
his p og ams, ID3, C4.5 and C5. Decision ee induc-
ion akes a se o known da a and induces a decision
ee om ha da a. The ee can hen be used as a ule
se o p edic ing he ou come om known a ibu es.
The ini ial da a se om which he ee is induced is
known as he aining se . The decision ee akes he
op-down o m. A he op is he i s a ibu e and
i s alues, om his nex b anch leads o ei he an a -
ibu e o an ou come. E e y possible lea o he ee
e en ually leads o an ou come.
4.1. Decision T ees – C4.5
C4.5 is an algo i hm de eloped by Ross Quinlan ha
gene a es Decision T ees (DT), which can be used o
classi ica ion p oblems. I imp o es (ex ends) he ID3
algo i hm by dealing wi h bo h con inuous and dis-
c e e a ibu es, missing alues and p uning ees a e
cons uc ion. I s comme cial successo is C5.0/See5, a
lo as e ha C4.5, mo e memo y e icien and used
o building smalle decision ees. J48 is an open
sou ce Ja a implemen a ion o he C4.5 algo i hm in
he WEKA da a mining ool.
Algo i hm 1 C4.5(D)
Inpu : an a ibu e- alued da ase D
1: T ee = {}
2: i Dis ”pu e” OR o he s opping c i e ia me hen
3: e mina e
4: end i
5: o all a ibu e a ∈Ddo
6: Compu e in o ma ion- heo e ic c i e ia i we spli a
7: end o
8: abes = Bes a ibu e acco ding o abo e compu ed c i e ia
9: T ee = C ea e a decision node ha es s abes in he oo
10: D = Induced sub-da ase s om Dbased on abes
11: o all D do
12: T ee = C4.5(D )
13: A ache T ee o he co esponding b anch o T ee
14: end o
15: e u n T ee
The gene ic desc ip ion o how C4.5 wo ks is shown
in Algo i hm 1. A decision ee is buil op-down om
a oo node and in ol es pa i ioning he da a in o
subse s ha con ain ins ances wi h simila alues (ho-
mogenous). Decision ee algo i hm uses en opy o
calcula e he homogenei y o a sample. I he sample
is comple ely homogeneous he en opy is ze o and i
he sample is an equally di ided i has en opy o one.
The en opy o class andom a iable ha akes on c
alues wi h p obabili ies p1, p2, . . . , pcis gi en by:
En opy(S) =
c
X
i=1
−pilog2pi.(1)
Figu e 6 shows he o m o he en opy unc ion el-
a i e o a bina y classi ica ion.
Fig. 6: En opy unc ion.
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 470
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
(a) pingsweep (b) po sweep
Fig. 7: J48 classi ie esul s.
(a) pingsweep (b) po sweep
Fig. 8: BayesNe classi ie esul s.
The es ima ion c i e ion in he decision ee algo-
i hm is he selec ion o an a ibu e o es a each
decision node in he ee. The goal is o selec he
a ibu e ha is mos use ul o classi ying examples.
A good quan i a i e measu e o he wo h o an a -
ibu e is a s a is ical p ope y called in o ma ion gain
ha measu es how well a gi en a ibu e sepa a es he
aining examples acco ding o hei a ge classi ica-
ion. This measu e is used o selec among he can-
dida e a ibu es a each s ep while g owing he ee.
The in o ma ion gain is based on he dec ease in en-
opy a e a da ase is spli on an a ibu e. Con-
s uc ing a decision ee is all abou inding a ibu e
ha e u ns he highes in o ma ion gain (i.e. he mos
homogeneous b anches).
Gain(S, A) = En opy(S)−X
∈V alues(A)
|S |
|S|En opy(S ),(2)
whe e V alues(A) is he se o all possible alues o a -
ibu e A, and S is he subse o S o which a ibu e
Ahas alue (i.e. S ={sˆ
I S|A(s) = }).
The i s e m in he equa ion o in o ma ion gain
is jus he en opy o he o iginal collec ion Sand he
second e m is he expec ed alue o he en opy a e S
is pa i ioned using a ibu e A. The expec ed en opy
desc ibed by his second e m is simply he sum o he
en opies o each subse S , weigh ed by he ac ion
o examples |S |/|S| ha belong o S .Gain(S, A) is
he e o e he expec ed educ ion in en opy caused by
knowing he alue o a ibu e A. Pu ano he way,
Gain(S, A) is he in o ma ion p o ided abou he a -
ge a ibu e alue, gi en he alue o some o he a -
ibu e A. The alue o Gain(S, A) is he numbe o
bi s sa ed when encoding he a ge alue o an a bi-
a y membe o S, by knowing he alue o a ibu e
A.
5. Bayesian Ne wo ks
Bayesian ne wo ks a e g aphical ep esen a ion o he
ela ionship be ween a iables. G aphical ep esen a-
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 471
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
ion o Bayesian ne wo ks a e di ec ed acyclic g aphs
wi h nodes and edges. Nodes ep esen a iables, pa-
ame e s o hypo heses and edges ep esen condi ional
dependencies.
5.1. Algo i hm o Nai e Bayesian
The Nai e Bayesian classi ie is based on Bayes’ he-
o em wi h independence assump ions be ween p edic-
o s. Bayes heo em p o ides a way o calcula ing he
pos e io p obabili y, P(c|x), om P(c), P(x), and
P(x|c). Nai e Bayes classi ie assumes ha he e ec
o he alue o a p edic o (x) on a gi en class (c) is
independen o he alues o o he p edic o s. This as-
sump ion is called class condi ional independence.
P(c|x) = P(x|c)P(c)
P(x),(3)
P(c|x) = P(x1|c)×P(x2|c)×· · ·×P(xn|c)×P(c),(4)
whe e P(c|x) is he pos e io p obabili y o class ( a -
ge ) gi en p edic o (a ibu e), P(c) is he p io p ob-
abili y o class, P(x|c) is he likelihood which is he
p obabili y o p edic o gi en class and P(x) is he
p io p obabili y o p edic o .
6. Expe imen al Resul s
Fo da a mining pla o m was chosen open sou ce
p ojec WEKA [6]. WEKA is a collec ion o machine
Fig. 9: Visualize esul s o pingsweep.
lea ning algo i hms o da a mining asks. The algo-
i hms can ei he be applied di ec ly o a da ase o
called om you own Ja a code.
WEKA con ains ools o da a p e-p ocessing, clas-
si ica ion, eg ession, clus e ing, associa ion ules, and
isualiza ion. I is also well-sui ed o de eloping new
machine lea ning schemes.
Tes ed ac i i ies we e cap u ed by he ne wo k a ic
collec o and sa ed as pcap iles. These pcap iles a e
bina y iles and can’ be ead di ec ly in o mos da a
mining applica ions.
These pcap iles we e con e ed in o cs iles wi h
sc ip s using sha k [7]. WEKA accep s *.cs iles,
*.a iles o a connec ion o a da abase. Fo his e-
sea ch we e used con e ed cs iles ha was opened
in WEKA.
As pa o he p ep ocessing s ep, in o ma ion da a
we e injec ed in o he da a which we e use ul o ain-
ing o he da a mining algo i hm. Addi ionally insigni -
ican da a we e elimina ed i i we e no se ing he
o e all p ocess. To begin unning his da a h ough
he algo i hms i was opened in WEKA explo e . Fo
he pu pose o hese ini ial uns we selec ed all o he
a ibu es.
Fig. 10: Visualize esul s o po sweep.
Figu e 7(a) shows he esul s in WEKA o a J48
Classi ie aining un on da a acqui ed du ing ping
sweep o he a ge ne wo k. The esul s desc ibed
he e we e de i ed om single nmap a ge scan, whe e
an a ack compu e scanned a ic im ne wo k, p obing
o ac i e IP add esses.
Also Fig. 7(b) shows he esul s in WEKA o a J48
Classi ie aining un on da a acqui ed du ing po
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 472

INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
sweep o he a ge compu e . The esul s desc ibed
he e we e de i ed om single nmap a ge scan, whe e
an a ack compu e scanned a ic im compu e , p ob-
ing o open po s.
The e we e chosen only 3 a ibu es o isualize ex-
pe imen al esul s. These a ibu es we e IP sou ce
add ess, IP des ina ion add ess and P o ocol. Figu e 9
and Fig. 10 show ela ion be ween hese a ibu es.
Figu e 8(a) shows he esul s in WEKA o a Nai e-
Bayes Classi ie aining un on da a acqui ed du ing
ping sweep o he a ge ne wo k. Also Fig. 8(b) shows
he esul s in WEKA o a Nai eBayes Classi ie ain-
ing un on da a acqui ed du ing po sweep o he a -
ge compu e .
7. Conclusion and Fu u e
Wo k
In his pape , we ha e p esen ed de ec ion o ne wo k
anomalies by using machine lea ning sys ems. Machine
lea ning sys em usually s a s wi h some knowledge
and du ing he ounds can imp o e i s knowledge.
The e we e es ed some a acks in egula ne wo k
a ic. As he i s a ack was used ping sweep o sub
ne wo k a ge o ge in o ma ion abou ac i e IP ad-
d esses. The po sweep was used as second a ack o
scanning open po s a he a ge ic im compu e .
Fi s , we cap u ed ne wo k a ic by he ne wo k
collec o and sa ed da a as pcap o ma ile. Nex , we
con e ed collec ed da a om pcap ile in o cs o -
ma ile. We used some sc ip s by means o sha k o
con e da a om pcap o cs ile o ma . Nex con-
e ed cs da a was inse ed in he WEKA so wa e o
use classi ica ion o da a. Finally classi ied da a was
isualize by WEKA so wa e.
Fu u e wo k expec s o use mo e a ibu es ha will
be ge om pcap iles. We also assume he use o
o he classi ica ion me hods and o he da a mining al-
go i hms.
Acknowledgmen
The esea ch leading o hese esul s has ecei ed und-
ing om he Eu opean Communi y’s Se en h F ame-
wo k P og amme (FP7/2007-2013) unde g an ag ee-
men no. 218086.
Re e ences
[1] FOWLER, Ch. A. and R. J. HAMMELL II. Build-
ing Baseline P ep ocessed Common Da a Se s o
Mul iple Follow-on Da a Mining Algo i hms. In:
P oceedings o he Con e ence on In o ma ion Sys-
ems Applied Resea ch 2012. New O leans: ED-
SIG, 2012, pp. 1–17. ISSN 2167-1508.
[2] FARRAPOSO, F., P OWEZARSKI and E.
MONTEIRO. NADA–Ne wo k Anomaly De ec-
ion Algo i hm. In: 18 h IFIP/IEEE In e na-
ional Wo kshop on Dis ibu ed Sys ems: Ope -
a ions and Managemen , DSOM 2007. San Jose:
Sp inge Ve lag, 2007, ol. 4785, pp 191—194,
ISBN 978-3-540-75694-1.
[3] DAS, K. P o ocol Anomaly De ec ion o
Ne wo k-based In usion De ec ion. The
SANS Ins i u e [online]. 2002. A ailable
a : h p://www.sans.o g/ eading_ oom/
whi epape s/de ec ion/p o ocol_anomaly_
de ec ion_ o _ne wo kbased_in usion_
de ec ion_349?show=349.php&ca =de ec ion.
[4] RICHARD, M. In usion De ec ion FAQ: A e
he e limi a ions o In usion Signa u es?.
The SANS Ins i u e [online]. 2001. A ailable
a : h p://www.sans.o g/ esou ces/id aq/
limi a ions.php.
[5] MITCHELL, Tom M. Machine lea ning. Bos on:
McG aw-Hill, 1997. ISBN 00-704-2807-7.
[6] WEKA 3. Da a Mining So wa e in Ja a [online].
2013. A ailable a : h p://www.cs.waika o.ac.
nz/ml/weka/
[7] TSha k [online]. 2013. A ailable a : h p://www.
wi esha k.o g/docs/man-pages/ sha k.h ml
Abou Au ho s
Pa el NEVLUD ecei ed his M.Sc. deg ee in
elecommunica ion enginee ing om VSB–Technical
Uni e si y o Os a a, Czech Republic in 1995. Since
his yea he has been holding posi ion as an assis an
p o esso a he Depa men o Telecommunica ions,
VSB–Technical Uni e si y o Os a a. The opics o
his esea ch in e es s a e communica ion echnologies,
ne wo king and secu i y.
Mi osla BURES ecei ed his M.Sc. deg ee in
elecommunica ions om VSB–Technical Uni e si y
o Os a a, Czech Republic in 2011. Since 2011 has
been s udying Ph.D. deg ee a he same uni e si y.
His esea ch is ocused on ne wo king, analysis o
ne wo k’s da a and secu i y.
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 473
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 11 |NUMBER: 6 |2013 |DECEMBER
Lukas KAPICAK ecei ed his M.Sc. deg ee
in elecommunica ions om VSB–Technical Uni e si y
o Os a a, Czech Republic in 2007. Since 2007 has
been s udying Ph.D. deg ee a he same uni e si y.
His esea ch is ocused on wi eless ansmission and
da a low analysis, simula ion and op imiza ion.
Ja osla ZDRALEK holds posi ion as
an associa e p o esso wi h Depa men
o Telecommunica ions, VSB–Technical Uni e si y
o Os a a, Czech Republic. He ecei ed his M.Sc.
deg ee in Compu e Science om Slo ak Technical
Uni e si y o B a isla a, Slo akia in 1977. He ecei ed
his Ph.D. deg ee om VSB–Technical Uni e si y
o Os a a in 2002, disse a ion hesis ”Diagnos ic
sys em wi hou disman ling o locomo i e con olle ”.
His esea ch is ocused on aul ole an sys em and
communica ion echnologies.
c
2013 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 474