A SVM and k-NN Res ic ed S acking o
Imp o e Land Use and Land Co e Classi ica ion
Jo ge Ga cia-Gu ie ez, Daniel Ma eos-Ga cia, and Jose C. Riquelme-San os
Depa men o Compu e Science
E.T.S.I.I. - Uni e si y o Se ille, Spain
{jga cia,ma eos, iquelme}@lsi.us.es
Abs ac . Land use and land co e (LULC) maps a e emo e sensing
p oduc s ha a e used o classi y a eas in o diffe en landscapes. The
newes echniques ha e been applied o imp o e he final LULC classifi-
ca ion and mos o hem a e based on SVM classifie s. In his pape , a
new me hod based on a mul iple classifie s ensemble o imp o e LULC
map accu acy is shown. The me hod builds a s a is ical as e om LI-
DAR and image usion da a ollowing a pixel-o ien ed s a egy. Then, he
pixels om a aining a ea a e used o build a SVM and k-NN es ic ed
s acking aking in o accoun he special cha ac e is ics o spa ial da a.
A compa ison be ween a SVM and he es ic ed s acking is ca ied ou .
The esul s o he es s show ha ou app oach imp o es he esul s in
he con ex o he eal da a om a ipa ian a ea o Huel a (Spain).
1 In oduc ion
Remo e sensing has become a e y impo an ool o ca y ou many diffe en
asks o he Na u al En i onmen . In his way, emo e sensing has success ully
been applied o impo an ac i i ies like flood con ol, o es al in en o ies o
in asi e species con ol in p o ec ed o specially in e es ing a eas.
Al hough emo e sensing usually wo ks wi h images exclusi ely, da a usion
has been o high in e es since he appea ance o new ac i e senso s (i.e., da a
is p oduced as a esponse o a s imulus which is no he sola ligh ). They
complemen images and o e come some o hei limi a ions, e.g., he p oblems
associa ed o shadows. These limi a ions cause usion o senso s can be ound
as a p ope echnique specially in e es ing o imp o e he esul s o he classical
emo e sensing app oaches. One o he mos ac i e esea ch lines has been based
on LIDAR (LIgh De ec ion And Ranging) echnology. This echnology is able o
egis e objec heigh s and i is specially ecommended o be applied on complex
landscapes like ipa ian zones. Thus, Ve els e al.[1] use LIDAR o s udy ege al
species communi ies and An ono akis e al.[2] de elop a new me hodology o
iden i y diffe en ypes o comme cial wood in ipa ian zones using only LIDAR.
An au oma ic pixel classifica ion which is gene ally supe ised is usually he
fi s s ep o ex ac knowledge om emo e sensing da a. Se e al echniques
om machine lea ning ha e been used wi h sa is ac o y esul s hough suppo
ec o machines (SVM) a e he p edominan echnique o ob ain he bes esul s
E.S. Co chado Rod iguez e al. (Eds.): HAIS 2010, Pa II, LNAI 6077, pp. 493–500, 2010.
494 J. Ga cia-Gu ie ez, D. Ma eos-Ga cia, and J.C. Riquelme-San os
in mos cases [3]. Despi e he SVM’s high accu acy, imp o emen is needed o
each he s anda ds o p oduc s like land uses and land co e (LULC) maps [4].
LULC maps a e emo e sensing p oduc s ha a e used o classi y a eas in o
diffe en landscapes subjec o hei own cha ac e is ics o unc ionali y. The
newes echniques ha e been applied o imp o e he final classifica ion o de elop
LULC maps. Fau el e al. [5] apply an SVM o classi y he pixels depending on
mo phologic and hype spec al da a. In Mi akis e al. [6], a neu onal ne wo k
wi h weigh s de e mined by a gene ic algo i hm ob ains he final classifica ion
using usion ope a o s and uzzy logic. I is impo an o unde line ha ensembles
a e one o he mos powe ul ools in machine lea ning and so hey a e in emo e
sensing whe e hey ha e also been applied p o usely. A e y clea example can
be seen in [7] whe e an s acking o se e al SVM’s and a andom o es is used
o ca y ou he pixel classifica ion.
This wo k explo es he applica ion o ensembles on emo e sensing aking
ad an age o con ex ual in o ma ion [8] om mul i-sou ce (LIDAR and ae ial
images) da a. Thus, a no el supe ised me hod called R-STACK (based on a
s acking o a SVM and mul iple NN classifie s) is shown wi h wo pu poses:
–Show an easy way o imp o e he quali y o models when in elligen ech-
niques a e applied on LIDAR and image y usion da a.
–Imp o e he gene al accu acy o an au oma ically gene a ed LULC map.
The es o he pape is o ganized as ollows. Sec ion 2 p o ides a desc ip ion o
he da a used in his wo k. Sec ion 3 desc ibes he me hodology used, highligh -
ing he ea u e se and he model ex ac ion p ocess. The esul s achie ed a e
shown in sec ion 4 and, finally, sec ion 5 is de o ed o summa y he conclusions
and o discuss u u e lines o wo k.
2Da aDesc ip ion
A LIDAR sys em is an op ical senso echnology ha measu es p ope ies o
sca e ed ligh (usually lase ) o find ange and/o o he in o ma ion o a dis an
a ge . The whole p ocess s a s wi h he emission o pola ized ligh , ypically, in
he ul a iole isible o nea in a ed. Then, LIDAR ca ches he eflec ed signal
om he opog aphic su ace and measu es he ime employed o each e u n
o es ablish he dis ance be ween he emi e and he objec ha p oduced he
e u n. This p ocess is helped by a global posi ioning sys em (GPS) o gi e ise
o a cloud poin da abase in which o e e y poin , i is possible o find: spa ial
posi ion(i.e., x, y and z coo dina es), in ensi y o e u n, numbe o he e u n in
a sequence (i a pulse caused mul iple impac s), e c. This ea u es and he RGB
alues in an o hopho o a e used in his wo k o ob ain s a is ical measu es on
which he me hod is based and hey will be explained in sec ion 3.
The LIDAR da a was aken in coas al a eas o he p o ince o Huel a. The
pulses we e geo- e e enced and co ec ly alida ed by he dis ibu o o he da a
and ha ing 1,384,875 eco ds o an a ea o 1.5 km2. The epo ed p ecision
indica es a maximum e o o 0.5 m in he x-y posi ions, and o 0.15 m in he
A SVM and k-NN Res ic ed S acking o Imp o e LULC Classifica ion 495
z posi ion. Along wi h he LIDAR fligh , he ae ial pho og aphs we e aken o
he a ea wi h a esolu ion o 0.5 m2.
The s udy a ea is si ua ed in he sou h o Spain, in he mou h o he Tin o
and Odiel i e s. This a ea is nea he ci y o Huel a and p esen s a mix o u ban
and na u al a eas. The na u al a eas can be classified in fi e subclasses: wa e ed
zones, ma shland and ege a ion (low, middle and high). The high ege a ion is
o med by sca ce ees o he genus eucalyp us in hea ea.Themiddle ege a ion
is o med by diffe en ypes o Medi e anean bushes ha p incipally su ound
oads and u ban a eas. Pas u es a e classified as low ege a ion and include ba e
ea h a eas. In addi ion, he u ban a eas a e also classified in fi e subclasses:
oads and ailways, buildings, coal deposi s, dumps and mixed a eas.
3Me hod
Ou LULC de elopmen me hod (see Figu e 3) ollows a pixel-o ien ed s a egy
which obliges us o c ea e a ma ix o as e whe e each elemen is a pixel. Each
pixel ep esen s an a ea in unc ion o he esolu ion. The alue o esolu ion
mus be p o ided by he use as a me hod pa ame e o de e mine he a ea
wi hin each pixel. The esolu ion depends on he LIDAR poin densi y and he
o hopho o esolu ion. In ou case, he selec ed esolu ion is se a 3 m2. Lesse
esolu ion could damage he smalles classes ( oads) classifica ion and bigge
esolu ion canno be possible due o he LIDAR esolu ion (0.5 poin s/m2).
Apa om he esolu ion, i is necessa y o supply a digi al ele a ion model
(DEM) o ex ac he ac ual heigh s o he LIDAR e u ns. In ou case, his
p ocess is ca ied ou by an adap a i e mo phologic fil e [9]. In addi ion, expe
knowledge has been applied o manually classi y o e a 2% o o al da a (7399
ins ances). Expe knowledge leaned on he pho og aphs aken in he same fligh
as LIDAR da a and p e ious in o ma ion om he Regional Minis y o Andalu-
sia (LULC map om 2003) was collec ed by an ope a o o build he aining
se .
The nex s ep (s ep 2 in Figu e 3) is o calcula e a se o a iables om image
RGB alues, LIDAR in ensi y, heigh s and dis ibu ion o he LIDAR e u ns o
each pixel (a o al o 500,000 pixels). In his manne , six y-one diffe en measu es
we e calcula ed o e e y pixel. Mos o a iables used ha e been ex ac ed om
li e a u e [10][2]. In Table 1, a summa y o hese ea u es can be seen. Specially
in e es ing is he case o he no malized diffe ence ege a ion index (NDVI). The
classical NDVI is gene a ed om nea in a ed band (NIR) and he ed band (R)
as can be seen in Equa ion 1. In ou case, i canno be calcula ed since he NIR
band is no a ailable in LIDAR o o hopho og aphy. Thus, he new a ibu e
SNDVI has been used o simula e he NDVI using he in ensi y (I) om LIDAR
(Equa ion 2) as nea -in a ed alue which app oxima es he eal NIR alue.
NDV I =NIR −R
NIR +R(1)
SNDV I =I−R
I+R(2)
496 J. Ga cia-Gu ie ez, D. Ma eos-Ga cia, and J.C. Riquelme-San os
A new me hod called R-STACK based on a modified s acking o wo well-known
classifie s (SVM and k-NN) has been de eloped o he model gene a ion. The
Weka [11] implemen a ion o SOM and a ad-hoc k-NN implemen a ion we e
used o each classifie espec i ely. Mo eo e , he gene al scheme o s acking
has been modified o adap i o geog aphic da a. In his way, he fi s le el
(s eps 5 and 6) consis s in a SVM which akes e e y ea u e om he pixels in
he aining a ea o build an ini ial model which classifies e e y pixel om he
s udy zone. A ha poin , a classical SVM applica ion on images is esul ed.
Inpu
l: LIDAR da a
o: O hopho og ahy da a
Ou pu
m: LULC map
Begin
1. De elop a ma ix as e in which e e y cell in ol es a physical
posi ion
2. Add he co esponding s a is ics om land o o each pixel in as e
3. Selec a aining se om as e , called ain
4. Label each pixel in ain using expe knowledge
5. Build a SVM model, s m, om ain
6. Use s m o classi y e e y pixel in as e
7. Fo each pixel pin as e
7.1. Collec he neigbou hood o pin a se s
7.2. Build a k-NN model, knn, oms
7.3. Use knn o classi y p
8. Re u n a map mwi h e e y pixel spa ial posi ion and i s label
End
Fig. 1. The LULC classifica ion me hod based on a R-STACK algo i hm (s eps 6 o 8)
The no el y o he R-STACK me hod se les on he second le el (s ep 7).
Pa icula ly, on he applica ion o se e al classifie s (k-NN) and he way hey
a e ained. Thus, a k-NN is build o each pixel aking i s neighbou s in he
as e as aining se which in ol es a s ong ela ion (physical dependence)
among he aining pixels and he cu en pixel o classi y. Fo he s udy a ea,
we wo k wi h k= 3 and an 8-adjacency ha is, each 3-NN is de eloped wi h
jus 8 ins ances o he pixel su ounding a ea. Fo his eason, he p ocess can be
ackled om he poin o iew o efficiency and complexi y. In he end, he k-NN
classifies he cu en pixel again ha ing used he model buil by i s neighbou s.
In his way, possible inconsis ences and non-desi ed effec s can be emo ed. I is
impo an o poin ou ha i is necessa y o make a as e copy be o e his las
p ocess and whils he classes in he o iginal as e a e modified, e e y k-NN has
o be build aking he neighbou s om he as e copy in o de o a oid colla e al
A SVM and k-NN Res ic ed S acking o Imp o e LULC Classifica ion 497
Table 1. Six y-one candida e a iables. Va iables wi h (*) a e calcula ed o each band
o a pixel: Heigh (H), In ensi y(I), Red(R), G een(G) and Blue(B).
Va iable Desc ip ion Va iable Desc ip ion
SNDVIMIN SNDVI minimum ICV In ensi y coefficien o a ia ion
SNDVIMAX SNDVI maximum HCV Heigh coefficien o a ia ion
SNDVISTD SNDVI S anda d de ia ion SLP Slope
SNDVIAVG SNDVI a e age CRR Canopy elie a io
MIN(*) Minimum PEC Pene a ion coefficien
MAX(*) Maximum TOTALR To al o e u ns
STD(*) S anda d de ia ion PCTN1 Unique e u n pe cen age
AVG(*) A e age PCTN2 Double e u n pe cen age
VAR(*) Va iance PCTN3 Th ee o mo e e u ns pe cen age
SKEW(*) Skewness PCTR1 Fi s e u n pe cen age
KURT(*) Ku osis PCTR2 Second e u n pe cen age
RANGE(*) Range PCTR3 Thi d o la e e u n pe cen age
NOTFIRST Second o la e e u n PCTR31 PCTR3 o e PCTR1
EMP Emp y neighbou s PCTR21 PCTR2 o e PCTR1
PCTR32 PCTR3 o e PCTR2
effec s. O he wise, he new classifica ion sequence would affec he esul o he
emaining pixels.
4Resul s
Two kinds o es ing ha e been ca ied ou o compa e he efficiency o ou ap-
p oach agains a classical SVM. The fi s es is based on s a is ical echniques.
Since emo e sensing da a is expensi e o gene a e, he compa ison has o es on
an a ificial da a spli . In ou case, 100 spli s a e c ea ed om he o iginal da a
so ha each spli con ains abou 740 ins ances. Then, a 10- old-c oss- alida ion
p ocess is made o e e y spli . The esul s a e egis e ed o he ollowing com-
pa ison p ocess.
We ha e used he p ocedu e sugges ed in se e al wo ks [12] o obus ly com-
pa ing classifie s ac oss mul iple da ase s in o de o e alua e he s a is ical sig-
nificance o he measu ed diffe ences in algo i hm anks. The chosen p ocedu e
in ol es he use o a s a is ical es o compa e classifie s one each o he . Ou
objec i e was o compa e a classical SVM o ou app oach in e ms o accu acy.
Thus, he Wilcoxon p ocedu e was selec ed as he app op ia e es .
A ai compa ison o he algo i hms is ob ained by a e age anks and in his
case, a e he p e ious 100 10- old-c oss- alida ion esul s, ou app oach anks
fi s . Wi h he measu ed a e age anks, he Wilcoxon es checks whe he he
a e age anks a e significan ly diffe en om he mean ank = 1.5 expec ed
unde he null hypo hesis. Leaning on a s a is ical package (MATLAB), p alue
o he Wilcoxon es ha e esul ed on a alue less han 5.72e−06 so he null

498 J. Ga cia-Gu ie ez, D. Ma eos-Ga cia, and J.C. Riquelme-San os
Table 2. A summing up o he hold-ou es o he SVM classical app oach
Use class Wa e Ma sh Roads o Low Middle High Buildings Coal Dumps Mixed
sample ailways Veg. Veg. Veg. deposi s a eas
Wa e 178 5 0 0 0 0 0 0 0 2
Ma shland 0 100 1 2 2 1 0 2 0 1
Roads o
ailways 0 4 69 0 6 0 0 0 1 0
Low Veg. 0 4 2 50 1 0 0 0 0 0
MiddleVeg.0 9 2 2213 0 0 0 0
High
Veg. 0 0 0 0 0 26 0 0 0 0
Buildings 0 2 3 0 2 1 31 0 0 0
Coal
deposi s01 0 410 0 10 0 0
Dumps10 0 000 0 0 219
Mixed a eas 0 0 17 0 0 0 0 0 1 2
TP Ra e 0.962 0.917 0.863 0.877 0.568 1.0 0.795 0.625 0.677 0.1
FP Ra e 0.002 0.051 0.048 0.015 0.021 0.009 0 0.003 0.004 0.021
P ecision 0.994 0.8 0.734 0.862 0.636 0.839 1 0.833 0.913 0.143
KIA 0.815
Co ec ly
classified 0.846
Table 3. A summing up o he hold-ou es o he SVM + k-NN es ic ed s acking
Use class Wa e Ma sh Roads o Low Middle High Buildings Coal Dumps Mixed
sample ailways Veg. Veg. Veg. deposi s a eas
Wa e 181 3 0 0 0 0 0 0 0 1
Ma shland 1 98 1 5 1 1 0 0 0 2
Roads o
ailways 0 4 72 0 2 0 0 0 0 2
Low Veg. 0 2 2 53 0 0 0 0 0 0
MiddleVeg.0 4 0 5253 0 0 0 0
High Veg. 0 0 0 0 0 26 0 0 0 0
Buildings 0 2 2 0 2 2 31 0 0 0
Coal
deposi s01 1 400 0 10 0 0
Dumps10 0 000 0 0 246
Mixed a eas 0 0 15 1 0 0 0 0 0 4
TP Ra e 0.978 0.899 0.9 0.93 0.676 1.0 0.795 0.625 0.774 0.2
FP Ra e 0.005 0.033 0.04 0.028 0.009 0.01 0 0 0 0.019
P ecision 0.989 0.86 0.774 0.779 0.833 0.813 1 1 1 0.267
KIA 0.847
Co ec ly
classified 0.873
A SVM and k-NN Res ic ed S acking o Imp o e LULC Classifica ion 499
hypo hesis is ejec ed. Ha ing ound ha he measu ed a e age anks a e sig-
nifican ly diffe en (a α= 0.05), ou analysis based on anks e eals ha he
accu acy o classical SVM is significan ly wo se han ha o ou app oach o
his kind o da a.
The second ype o es ing is a hold-ou p ocess wi h da a p e iously classified.
This is he common es ing in emo e sensing. The es da a se (600 ins ances)
was selec ed om he o iginal da a se because o i s special difficul y o be
classified and i is no pa o he aining se . In Table 3 and Table 2, esul s o
ou app oach and classic SVM a e shown when he hold-ou p ocess is ca ied
ou . The gene al imp o emen is a 3% which is a e y impo an ad ance.
5 Conclusions
In his pape , a new me hod based on a mul iple classifie s ensemble was used o
imp o e LULC map accu acy. The me hod buil a s a is ical as e om LIDAR
and image usion da a ollowing a pixel-o ien ed s a egy. Then, he pixels om
a aining a ea we e used o ain a SVM and k-NN es ic ed s acking (called
R-STACK) aking in o accoun he special cha ac e is ics o spa ial da a. A
compa ison be ween a SVM and he R-STACK me hod was ca ied ou . The
esul s in a ipa ian a ea o Huel a (Spain) showed a global accu acy o 84.6%
o he classical SVM and 87.6% o he new app oach which means a significan
ad ance.
E en hough esul s a e sa is ac o y, he e a e s ill se e al p oblems o fix.
Some o hem a e ela ed o shadows om images and i s weigh on he final
classifica ion which has o be aken in o accoun . Hence, a con ol o weigh s
o each ea u e has o be implemen ed in o de o a oid hei misclassifica ion
effec s. Gene ic algo i hms could be a e y sui able ool o sol e his p oblem.
In addi ion, dependence on he aining se can be a mo e impo an p oblem.
Some imes, he aining se can be incomple e o no enough o desc ibe he eal
space. These p oblems a e ha de o fix. Despi e he ac ha a semi-supe ised
app oach seems o be mo e sui able o so ou his kind o p oblems, e y ew
semi-supe ised p oposals can be ound ye and mo e esea ch is needed in o de
o de elop hem wi h he equi ed accu acy. Finally, some p oblems a e inhe en
in pixel-o ien ed app oaches such as he de ec ion o pa ial a ificial s uc u es.
In he u u e, i would be e y in e es ing o apply a p io phase in which a
low addi ion o he compu a ional cos , an objec -o ien ed segmen a ion and
classifica ion could be ca ied ou o ex ac he mos difficul s uc u es o
classi y, using isual ecogni ion echniques om he compu e ision wo ld.
Acknowledgmen s. We would like o hank he Regional Minis y o Andalu-
sia o all he suppo ecei ed in he de elopmen o his wo k and especially,
o hank I ene Ca pin e o, Juan Jos´e Vales and Daniel Laguna o hei e y
app ecia ed commen s. We would also like o hank F ancisco Ma ´ınez-´
Al a ez
and Luis Gon¸cal es-Seco o all he ime hey in es ed ha allowed his wo k o
be comple ed.
500 J. Ga cia-Gu ie ez, D. Ma eos-Ga cia, and J.C. Riquelme-San os
Re e ences
1. Ve els , J., Gee ling, G., Syko a, K., Cle e s, J.: Mapping o agg ega ed floodplain
plan communi ies using image usion o casi and lida da a. In e na ional Jou nal
o Applied Ea h Obse a ion and Geoin o ma ion (11), 83–94 (2009)
2. An ona akis, A., Richa ds, K., B asing on, J.: Objec -based land co e classifi-
ca ion using ai bo ne LIDAR. Remo e Sensing o En i onmen (112), 2988–2998
(2008)
3. Dalpon e, M., B uzzone, L., Vesco o, L., Damiano, G.: The ole o spec al es-
olu ion and classifie complexi y in he analysis o hype spec al images o o es
a eas. Remo e Sensing o En i onmen (113), 2345–2355 (2009)
4. Shao, G., Wu, J.: On he accu acy o landscape pa e n analysis using emo e
sensing da a. Landscape Ecology (23), 505–511 (2008)
5. Fau el, M., Benedik sson, J., Chanusso , J., S einsson, J.: Spec al and spa ial
classifica ion o hype spec al da a using SVMs and mo phological p ofiles. IEEE
T ansac ions on Geoscience and Remo e Sensing 46(11), 3804–3814 (2008)
6. Mi akis, N., Topaloglou, C., Alexand idis, T., Theocha is, J., Zalidis, G.: Decision
usion o GA sel -o ganizing neu o- uzzy mul ilaye ed classifie s o land co e clas-
sifica ion using ex u al and spec al ea u es. IEEE T ansac ions on Geoscience
and Remo e Sensing 46(7), 2137–2152 (2008)
7. Waske, B., an de Linden, S.: Classi ying mul ile el image y om sa and op-
ical senso s by decision usion. IEEE T ansac ions on Geoscience and Remo e
Sensing 46(5), 1457–1466 (2008)
8. Co ijo, F.J., Blanca, N.P.D.L.: Imp o ing classical con ex ual classifica ions. In-
e na ional Jou nal o Remo e Sensing 19(8) (1998)
9. Goncal es-Seco, L., Mi anda, D., C ecen e, R., Fa o, J.: Digi al e ain model
gene a ion using ai bo ne LIDAR in flo es ed a ea o Galicia. In: P oceedings o 7 h
In e na ional Symposium on Spa ial Accu acy Assessmen in Na u al Resou ces
and En i onmen al Sciences, Spain, pp. 169–180 (2006)
10. Hudak, A.T., C ooks on, N.L., E ans, J.S., Halls, D.E., Falkowski, M.J.: Nea es
neighbo impu a ion o species-le el, plo -scale o es s uc u e a ibu es om
lida da a. Remo e Sensing o En i onmen 112, 2232–2245 (2008)
11. Hall, M., F ank, E., Holmes, G., P ah inge , B., Reu emann, P., Wi en, I.H.: The
WEKA da a mining so wa e: An upda e. SIGKDD Explo a ions 11(1) (2009)
12. Ga cia, S., He e a, F.: An ex ension on s a is ical compa isons o classifie s o e
mul iple da a se s o all pai wise compa isons. Jou nal o Machine Lea ning Re-
sea ch 9, 2677–2694 (2008)