Palm tree image classification : a convolutional and machine learning approach

Robe o Jose Luna Oli a es PALM TREE IMAGE CLASSIFICATION A con olu ional and machine lea ning app oach PALM TREE IMAGE CLASSIFICATION A con olu ional neu al ne wo k and machine lea ning app oach Disse a ion supe ised by Joel Dinis Bap is a Fe ei a da Sil a, PhD Ins i u o Supe io de Es a ís ica e Ges ão de In o mação, Uni e sidade No a de Lisboa Lisbon, Po ugal Disse a ion co-supe ised by Ped o da Cos a B i o Cab al, PhD Ins i u o Supe io de Es a ís ica e Ges ão de In o mação, Uni e sidade No a de Lisboa Lisbon, Po ugal Disse a ion co-supe ised by Ignacio Gue e o, PhD Ins i u e o New Imaging Technologies, Uni e si a Jaume I Cas ellón de la Plana, Spain Feb ua y 2019 i Acknowledgemen s I would like o hanks my supe iso Joel Sil a, o sha ing his knowledge, encou - agemen and guidance, made his esea ch p ojec enjoyable and ul illing. To my co - supe iso s Ped o Cab al and Ignacio Gue e o o hei eedback and cons uc i e c i icism. Thanks o he effo o all he p o esso ha helped h ough he mas e p og ams. Special hanks o p o esso s Ma co Painho and Ch is oph B ox o hei guidance and ad ice o he du a ion o he p og am.To all my classma e ha became my amily du ing his pe iod. To he E asmus Mundus p og am o gi ing me he oppo uni y o comple e his mas e p og ams and ha e an un o ge able expe ience. To my amily o always suppo ing me and showing me o belie e in mysel . ii I don’ belie e i . P o e i o me and I s ill won’ belie e i Douglas Adams PALM TREE IMAGE CLASSIFICATION A con olu ional and machine lea ning app oach Abs ac Con olu ional neu al ne wo ks ha e p o en o excel a image classi ica ion asks, do o his hey ha e being inco po a ed in o he emo e sensing ield, ini ial hu dles in hei applica ion like he need o la ge da a se s o hea y compu a ional bu den, ha e being sol e wi h se e al app oaches. In his pape he ans e lea ning app oach is es ed o classi ica ion o a e y high esolu ion images o a palm oil plan a ion. This app oach uses a p e ained con olu ional neu al ne wo k o ex ac ea u es om an image, and label hem wi h he aid o machine lea ning models. The esul s p esen ed in his s udy show ha he ea u es ex ac ed a e a iable op ion o image classi ica ion wi h he aid o machine lea ning models. An o e all accu acy o 97% in image classi ica ion was ob ained wi h he suppo ec o machine model. i KEYWORDS Con olu ional Neu al Ne wo k Machine Lea ning Unnamed Ae ial Vehicle Image Classi ica ion T ans e Lea ning O e Fea ACRONYMS GIS - Geog aphical In o ma ion Sys ems GEOBIA - Geog aphic Objec -Based Image Analysis CNN - Con olu ional Neu al Ne wo k UAV - Unmanned Ae ial Vehicle FVP - Fi s Pe son View RC - Remo e Con ol CMOS - Complemen a y Me al-Oxide Semiconduc o CHDK - Canon Hack De elopmen Ki RF - Random Fo es LR - Logis ic Reg ession DT - Decision T ee GBC - G adien Boos Classi ie KNN - K nea es neighbo SVM - Suppo Vec o Machine TP - T ue Posi i es TN - T ue Nega i es FP - False Posi i es FN - False Nega i es i INDEX OF THE TEXT INDEX OF FIGURES x INDEX OF TABLES xi 1 In oduc ion 1 1.1 An o e iew o he wo k .......................... 1 1.2 Aim and Objec i es ............................. 3 1.3 Documen o ganiza ion ........................... 3 2 Li e a u e e iew 4 2.1 Remo e sensing and UAV applica ions .................. 4 2.1.1 Remo e sensing ........................... 4 2.1.2 UAV applica ions .......................... 5 2.2 Machine lea ning echniques in Remo e Sensing ............. 6 2.3 Con olu ional neu al-ne wo ks ...................... 7 2.4 Applica ion o CNN in Remo e Sensing .................. 9 2.5 Fundamen als o CNN ........................... 10 2.6 O e Fea .................................. 11 3 Da a and Me hods 13 3.1 Desc ip ion o s udy a ea .......................... 13 3.2 UAV Sys em ................................. 13 3.3 Ae ial su ey ................................. 15 3.4 Da a se .................................... 15 3.5 Me hodology ................................. 17 3.5.1 Tools ................................. 17 3.6 Da a P ep ocessing ............................. 18 3.7 O e Fea and Fea u e Ex ac ion .................... 19 3.8 Machine Lea ning Models ......................... 19 3.9 Visualiza ion ................................. 20 3.10 Accu acy assessmen ............................ 20 4 Resul s 21 ii INDEX OF THE TEXT 4.1 Resul o e iew ............................... 21 4.2 Image p ep ocessing ............................ 21 4.3 Fea u e ex ac ion wi h O e Fea .................... 22 4.4 Machine lea ning models classi ica ion esul s .............. 23 4.4.1 SVM classi ica ion esul s ..................... 25 4.5 Image Classi ica ion ............................. 26 5 Discussion and Conclusion 31 5.1 Discussion .................................. 31 5.2 Conclusion .................................. 34 Bibliog aphic Re e ences 35 I Annex 42 I.1 Example o ea u es ex ac ed using O e Fea .............. 42 I.2 Image classi ica ion esul s ......................... 42 I.3 ROC g aphs ................................. 43 I.4 Con usion ma ix .............................. 45 II Annex 2 47 II.1 C opping Images .............................. 48 II.2 Ex ac ing ea u es wi h O e ea ..................... 48 II.3 Machine lea ning Models .......................... 50 II.4 Fine uning Machine lea ning Models ................... 51 II.4.1 Suppo Vec o Machine ...................... 51 iii 2.1. REMOTE SENSING AND UAV APPLICATIONS equipmen o small adio con ol ai c a , he e o e his e iew will ocus on he de i- ni ion p oposed by Wa s e al. (2012), as i i s in o he ype o sys em ha was used o collec ing he da a o his s udy. Wa s p oposed de ini ion conside s h ee main pa ame e s: al i ude, endu ance and ligh capabili ies, depending on hese cha ac e - is ics a UAV can i in o an speci ic ca ego y, such as Low Al i ude Sho -Endu ance (LASE) sys ems. These pla o ms ypically ha e a payload in he ange o 2-5 kg, wi h a wingspan smalle han 3 me e s, up o 2 hou s o ligh ime and can ope a e in a ange o 5 km wi h a g ound s a ion. In o he wo ds, his would be conside ed a “small UAV", and a e he sys ems mos commonly used and widesp ead in esea ch p ojec s Zecha e al. (2013). Se e al ac o s can be a ibu ed o he UAV h i ing in emo e sensing, echnology has played a p ominen ole wi h b eak h ough in posi ional and na iga ion sys ems Zecha e al. (2013), minia u iza ion o ha dwa e and senso s Casag ande e al. (2018) and Salamí e al. (2014), ha made i echnically and economically easible o c ea e sys ems capable o au onomously ollowing a ligh pa h and ga he emo e sensed da a. This end also sp ead o da a p ocessing ha dwa e and so wa e, ex ac ing in o ma ion became efficien , ela i ely cheap and accessible o he gene al public. O he impo an ac o s o be conside ed a e he lexibili y a UAV p o ides ega d- ing empo al, spa ial and spec al esolu ion. The modula na u e o hese sys ems allows o seamlessly in e changeable senso s Zecha e al. (2013), as well as he pos- sibili y o selec ing he bes condi ion and ime o su eying an a ea Salamí e al. (2014). This cha ac e is ics and lexibili y allows his sys ems o se es as a comple- men o e en subs i u e o adi ional pla o ms like space bo ne o pilo ed ai c a s Salamí e al. (2014). In some cases UAV is he only iable op ion o accessing o ob aining in o ma ion o emo e and dange ous a eas o humans Bolla d-B een e al. (2015) and E e ae s (2008) and inally, depending on he con ex and scale o he applica ion, hey can be a cheape al e n i e o adi ional me hods Iizuka e al. (2018) and Wa s e al. (2012). In conclusion echnological ad ancemen made i so, ha i ’s no longe necessa y o ha e highly ained pe sonnel and equipmen o build a UAV, ob ain da a, p ocess i and ex ac use ul in o ma ion. 2.1.2 UAV applica ions The e a e many applica ions o UAV da a in emo e sensing, ca og aphy p oduc- ion is one o he p inciple examples, Be eška and Ruzgien ˙ e (2013) ob ained images and p oduced digi al ele a ions models (DEM), ha ul ill he equi emen s o opo- g aphical and GIS applica ions. Resea ched done by Mesas-Ca ascosa e al. (2014) assessed he quali y o o hopho os gene a ed by UAV, he spa ial accu acy was e al- ua ed using echniques employed by mapping agencies, wi h posi i e esul s. A new ield whe e UAVs a e gaining ac ion is ai quali y measu emen , as he e iew by 5 CHAPTER 2. LITERATURE REVIEW Villa e al. (2016) shows a g owing in e es and applica ions o measu ing ai quali y, a mosphe ic composi ion and ai pollu an moni o ing. Geomo phology su ey a e also being done wi h he help o UAVs, as p o ed by Wang e al. (2016), wi h soil e osion moni o ing in Sancha i e basin in No heas China. El ne e al. (2016) p esen s an o e iew in he use o high esolu ion UAV im- ages o ec ea ing physical objec s, his me hods ha e being used in he econs uc ion o 3 dimensional objec s in a ious con ex and esea ch p ojec s. Vege a ion s udies ha e been one o he p ominen ocus o emo e sensing, be i o ag icul u al o o es y applica ions. In he ield o o es managemen Miki a e al. (2016) used ae ial and g ound pho og amme y o es ima ing heigh and diame e a b eas heigh (DBH), Yuan and Hu (2016) applied andom o es models and objec - based classi ica ion o iden i ying pes and moni o ing heal h o o es . Ve y high esolu ion da a can be used o single weed mapping, o si e speci ic plan p o ec ion in whea ields P lanz e al. (2018), Luna and Lobo (2016) iden i ied and mapped gaps in suga cane plan a ions o op imizing eplan ing me hods, Gago e al. (2015) analyzed he bene i s and gaps o wa e s ess moni o ing. Mos o he applica ions lis ed abo e ely on iden i ying objec s, de ining hei ea u es and bounda ies. Many me hods ha e being de eloped o his pu pose, wi h mos o hem ocused on au oma ing his ask. In ea ly s ages he ocus was on iden i- ying la ge objec s do o esolu ion es ains o senso s. Mos o he me hods de elop whe e based on s a is ical analysis o pixels bu his mean ha unde lying spa ial pa e ns whe e no conside ed Casag ande e al. (2018). Wi h he inc easing quali y o senso s and he widesp ead use o UAVs iden i y- ing smalle objec s became easible, he a ailabili y o new da a pa ed he way o a pa adigm shi . The co e analysis based on s a is ics we e complemen ed wi h ob- jec classi ica ion, his led o he c ea ion o new concep s, ela ionships be ween he da a and me hods o analysis. Fo his concep s Hay and Cas illa (2008) p oposed he e m geog aphic objec -based image analysis (GEOBIA), hey conside ed o be a a sub- discipline o geog aphic in o ma ion science, ha ocus is c ea ing au oma ed me hods o segmen ing emo ely sensed images in o meaning ul objec s and analyzing hei spa ial, spec al and empo al cha ac e is ics, o c ea e new geog aphic in o ma ion in GIS- eady o ma Hay and Cas illa (2008). This ield has bene i ed om he de el- opmen and ad ancemen o machine lea ning me hods and specially con olu ional neu al ne wo ks (CNN), due o hei capaci y o sol e complex image classi ica ion p oblems Chen e al. (2018). 2.2 Machine lea ning echniques in Remo e Sensing Machine lea ning (ML) echniques a e app oaches o sol ing classi ica ion and eg es- sion p oblems, hey a e conside ed "uni e sal app oxima o s", hey can lea n he inne wo kings o a sys em om a la ge se o da a wi hou p io knowledge La y e al. 6 2.3. CONVOLUTIONAL NEURAL-NETWORKS (2016). Being able o model complex sys ems, capaci y o using a wide ange o inpu and no assuming no mal dis ibu ion o he da a a e some o he main eason hey ha e become popula . These cha ac e is ics a e also common among emo e sensed da a Maxwell e al. (2018), he e o e his me hods ha e being widely used and applied by esea che s and specialis in he ield. Some o he mos common ML algo i hms a e decision ees (DT), sel-o ganizing map (SOM), andom o es s (RF), suppo ec- o machines (SVM) and a i icial neu al ne wo ks (ANN), he la e wo ha e being widely applied in he ield o geoscience La y (2010). Selec ing he bes algo i hm can be a challenging ask, as p o en by he compa ison done by Maxwell e al. (2018), in which he e’s no clea algo i hm ha cons an ly ou pe o ms he es . This migh be caused by de e ing me hodological app oaches and speci ic cha ac e is ics o he da a, he e o e he bes algo i hm is case speci ic and hus i ’s ecommended ha he analysis should include mul iple classi ie s Law ence and Mo an (2015). The scien i ic li e a u e ega ding he use o hese me hods has being s eadily inc easing, Belgiu and D ăguţ(2016) p o ides and o e iew o s udies using RF classi- ie , Moun akis e al. (2011) p esen s a simila e iew bu o SVM and Maxwell e al. (2018) compa es he accu acy o algo i hms in a emo e sensing con ex . Recen ly in e es and inno a ion has g a i a ed owa ds he de elopmen o deep neu al ne - wo k, a o m o a i icial neu al ne wo k, ha ou pe o m adi ional me hods Zhang e al. (2016). Figu e 2.2 exempli ies a basic a i icial neu al ne wo k a chi ec u e, i has h ee neu ons in he inpu laye , ha eed neu ons in o a hidden laye , which in e ms ac i a e he ou pu laye . Deep neu al ne wo ks a e se e al neu al ne wo ks s acked oge he , esea ches ound ha adding mo e laye s o he ne wo k would inc ease he accu acy o he model. This has being p o en in se e al s udies whe e con olu ional nei al ne wo ks pe o m image classi ica ion ask, sec ion 2.3 will ocus on his models and hei use in emo e sensing con ex . 2.3 Con olu ional neu al-ne wo ks The i s ins ance o a con olu ional neu al ne wo k is usually c edi ed o he esea ch pape Neocogni on: A Sel -o ganizing Neu al Ne wo k Model o a Mechanism o Pa e n Recogni ion Una ec ed by Shi in Posi ion Fukushima and Miyake (1982), his was a sys em designed o isual pa e ecogni ion and was inspi ed on he ind- ings o Hubel and Wiesel (1962), on how a ca ’s isual co ex is s uc u ed and how i wo ks o iden i y objec s. LeCun e al. (1998) me hod o ecognizing hand w i en digi s was also an ea ly applica ion o CNN and se ed as basis o o he esea ch and models. The yea 2012 is a benchma k yea , as K izhes ky, Su ske e and Hi on published ImageNe Classi ica ion wi h Deep Con olu ional Neu al Ne wo ks K izhe sky e al. 7 CHAPTER 2. LITERATURE REVIEW Figu e 2.1: A i icial neu al ne wo k a chi ec u e Image Sou ce: Glosse .ca (2013) (2012). Conside ed o be a seminal wo k in he ield o deep lea ning, i was he i s ins ance ha a CNN was used o winning he LSVRC (ImageNe La ge-Scale Visual Recogni ion Challenge), a yea ly compe i ion in he compu e ision ield, which e- qui es he compe i o s o de elop algo i hms ha can au oma ically anno a ed images om a da a se Russako sky e al. (2015). The model p oposed by K izhe sky e al. (2012) called AlexNe , had he lowes e o a e in he his o y o he compe i ion un il ha poin . The au ho s explained ha he limi a ions in hei model was due o ech- nological cons ains as well as amoun o da a a ailable, and p edic ed ha wi h mo e da a and echnological ad ancemen his ype o models would inc ease in accu acy. F om ha poin on CNN, we e a mains ay in compu e ision compe i ions and hei popula i y s a ed o pe mea e in o o he ields. In he coming yea s modi ica- ions whe e made o he o iginal model, om ine uning i ’s pa ame e s Zeile and Fe gus (2013), o simpli ying he s uc u e o a chi ec u e Simonyan and Zisse man (2014), o c ea ing new and complex model Szegedy e al. (2015). A endency was es- ablished, each new i e a ion o he CNN model su passed i s p edecesso in accu acy. The ully con olu ional ne wo k wo k Long e al. (2015) was ano he impo an benchma k in seman ic segmen a ion, i achie ed he goal o assigning each pixel in an image o a class. Pa allel o he adjus men made o he CNN a chi ec u e and accu acy, o he s eps we e aken in o making he p ocess as e and adding ea u es, such as de ining he bounda ies o objec s Gi shick e al. (2014), his me hod combined image classi ica ion and de ec ion and s a ed a new end in he ield. Subsequen imp o emen s we e made o his app oach, in i s speed and accu acy Gi shick (2015) and simpli ying i s s uc u e Ren e al. (2015). As hese me hods ma u ed and p o ed success ul, hey became he model o choice 8 2.4. APPLICATION OF CNN IN REMOTE SENSING o many applica ions, including emo e sensing. Ne e heless, emo e sensed da a in gene al p esen s some pa icula i ies ha need o be conside ed: • A i s co e emo e sensed da a has an impo an geospa ial componen ha needs o be conside , ha nessing hese cha ac e is ics can led o be e analysis. • In gene al, emo ely sensed da a comes om wide ange o senso s, wi h diffe en spec al and spa ial esolu ion, as well as diffe en ypes o da a. • The la ge amoun o da a p oduced wi h high empo al esolu ion is c ea ing a shi owa ds analysis o ime se ies ins ead o single scene. • Remo e sensing has adi ional app oach o de ec ing and quan i ying a phe- nomenon, using models and expe knowledge. On he o he hand, deep lea ning app oaches ocus on ully au oma ed expe - ee knowledge solu ions. 2.4 Applica ion o CNN in Remo e Sensing Despi e his con ex emo e sensing scien is , ha e manage o exploi he po en ial o deep lea ning o a a ying ange o applica ions. Salbe g (2015) p oposed a me hod o de ec ing seal pups, om ae ial emo ely sensed images. The me hod uses a p e- ained deep con olu ional neu al ne wo k o ea u e ex ac ion and suppo ec o machine o classi ica ion. Resul s show he me hod was success ul and implies ha he me hodolgy can be gene alized and applied o objec ecogni ion in diffe en sce- na ios, bu equi es u he imp o emen s and ine uning. Chen e al. (2017), used deep lea ning echniques o coun ui s in a ee, i s iden i ying po en ial egions o in e es ( he loca ion o he ui s), hen compa ing i o g ound u h da a and inally applying linea eg ession o es ima ing a inal coun , he au ho s p opose using his me hodology o plan pheno ype iden i ica ion, coun ing plan s and moni o plan disease ha p esen isual symp oms. Hung e al. (2014), p oposed a me hod o ob- aining he bes ligh al i ude o a UAV, o achi ing he bes classi ying pe o mance o de ec ing weeds in c op ields, Zhu e al. (2017) p o ides a e iew o applica ions o CNN, showcasing s a e o he a me hods. A speci ic deep lea ning wo k low o de ec ing and coun ing palm ee was p o- posed by Li e al. (2017) hei me hod was es ed agains manually labeled ees, showing good esul s, a limi a ion hey encoun e , was he size o hei da a se . O he me hod ied o gene alize he classi ica ion p ocess o diffe en o es ypes, using a cascade neu al ne wo k, i showed p omising esul s, bu he au ho s in end o be e hei me hod applying a con olu ional neu al ne wo k a chi ec u e Tianyang e al. (2018). 9 CHAPTER 2. LITERATURE REVIEW 2.5 Fundamen als o CNN A CNN, is a machine lea ning algo i hm, ha can lea n by analyzing la ge quan i ies o da a. I s o igin s ems om a i icial neu al ne wo k models, which a e ma hema ical app oxima ion o how a human b ain wo ks. Thei gene al p ocess includes analyzing inpu s ough a “neu on” o a laye o ma hema ical ope a ions and classi ying he ou pu depending on i s alue. This p ocess is achie ed by an ac i a ion ucn ion, his unc ion se es as a h eshold o de ining he class o he ou pu . Fo image ecogni ion asks he inpu a e pixel alues. This models pa icula i y good o image analysis, because hey can deal wi h he la ge amoun o da a by educing hei dimension, hus easing he p ocessing bu den. They ha e p o en o excel a ex ac ing ea u es om images, pe o ming classi ica ion K izhe sky e al. (2012) and Szegedy e al. (2015), seman ic segmen a ion Long e al. (2015) and objec de ec ion Gi shick e al. (2014), Ren e al. (2015), and Zhu e al. (2017). CNN ollow a basic s uc u e and i s main componen s can be classi ied depending on hei unc ions, nex a desc ip ion o he main componen s is desc ibed. This sec ion is based he lec u e no es om Ka pa hy (2016). • Con olu ion laye : This laye ecei es he inpu as an a ay o pixels, each one has h ee alues associa ed wi h hem, hey a e wid h, heigh and dep h, which ela es o he numbe o bands ha he image has. This laye ans oms all his in o ma ion in o a unc ion ha will desc ibe he pixel, in ma hema ical e ms his is called con olu ion, hus he name. This ope a ion is pe o med on he whole image by sliding a il e o ke nel. The ou pu s a e alues om ha summa ize each pixel. • Pooling laye : his pa is esponsible o educing he dimension o he con- ol ed laye , one o he mos common me hods o doing his is called max pool- ing, which consis on e u ning he maximum alue om each o he con olu ion ope a ions wi hin a de ined window. • Ac i a ion laye : They a e in cha ge o de ining which neu ons o laye s om he ne wo k should be ac i a ed. • Fully connec ed laye : is he laye ha is in cha ge o classi ying he ea u es ex ac ed om he p e ious laye s and gene a es an ou pu . As he name sugges his laye s connec all he inpu laye s o he ou pu s. In summa y when an image is ed o a CNN, he con olu ional and pooling laye s ans o m he pixel da a, his phase is also called ea u e ex ac ion, his is when he CNN will ecognize geome ies in an image, o example iden i y he s uc u e o a palm ee o a di oad. The classi ica ion o he image is done by he ully connec ed 10 2.6. OVERFEAT laye s, hei ou pu is usually a p obabili y sco e o wha he image is. Figu e 2.2 shows he a chi ec u e o Alexne K izhe sky e al. (2012), i s s uc u e se e as ounda ion o many o he models Se mane e al. (2013) and Zeile and Fe gus (2014). Figu e 2.2: A chi ec u e o a Con olu ional Neu al Newo k Image Sou ce: K izhe sky e al. (2012) Despi e excelling in image classi ica ion asks, hese models ha e some d awbacks such as he need o la ge da a se s o labeled images as well he ime i akes o ain a model om sc a ch, none heless he e a e expe ience o using p e ained models as ea u e ex ac o wi h posi i e esul s Ma manis e al. (2016), Noguei a e al. (2017), and O hman e al. (2016). This app oach is called ans e lea ning, i elies on using a p e ained CNN and emo ing he ully connec ed laye (laye ha p o ide classi ica ion esul s), and ex ac ing ea u es using he es o he a chi ec u e Ka pa hy (2016). This me hod p o ides he ma hema ical desc ip ion o geome ies such as edges, cu es, size, shapes and angles ound in he image. This desc ip ion a e he ea u es om he image and hey can be used o ain a line classi ie like SVM Zhang e al. (2015). This app oach was also applied in his s udy, using O e Fea Se mane e al. (2013), which is a p e ained CNN ha was used as a ea u e ex ac o , his ea u es we e la e used o ain se e al machine lea ning classi ie s. 2.6 O e Fea The O e Fea ne wo k is based on Alexne K izhe sky e al. (2012) wi h some mino modi ica ions. I has 6 con olu ional laye s, hey ha e a a ying numbe o neu ons om 96 o 1024, he ke nel in hese laye s a y in size om 3x3 o 7x7 and a max pool- ing laye wi h ke nel size anging om 3x3 and 5x5, ounded up by 3 ully connec ed laye s. This model in oduced a no el way o inc easing classi ica ion accu acy by ain- ing a CNN o classi y and de ec objec s simul aneously. This amewo k achie ed he bes esul in he localiza ion ask o he 2013 ImageNe La ge Scale Visual Recogni ion Challenge (ILSVRC) and 4 h place in he classi ica ion ask. O e Fea was ained 11 CHAPTER 2. LITERATURE REVIEW using ImageNe 2012 aining se , which has o e a million images sp ead ou in 1000 classes Deng e al. (2009). This da a se con ains images ha a e mainly cen e ed and ee om image occlusion and clu e Sha i Raza ian e al. (2014). The bes i e a ion o his model se ed as base o , O e Fea ea u e ex ac o , ha was used in his s udy. The e a e se e al ins ances o he use o O e Fea , Noguei a e al. (2017) compa ed h ee diffe en app oaches o using CNN in emo e sensing. In he s udy hey buil and ain om sc a ch a CNN, ine une a p e- ained CNN and use O e Fea o ex ac ea u es and classi y images. Ma manis e al. (2016) p oposed a wo s ep classi ica ion p ocess whe e hey used he p e- ained O e Fea model coupled wi h ano he CNN in cha ge o classi ying he ea u es ex ac ed. Sha i Raza- ian e al. (2014) used O e Fea o a ious image ecogni ion ask, wi h diffe en da a se ob aining esul s ha compe e wi h highly sophis ica ed and highly uned s a e o he a me hods. In his s udy he O e Fea ne wo k was used as i has yield good esul s o classi ica ion ask, also he au ho p o ide a e sion o his model as a eady o use so wa e (h ps://gi hub.com/se mane /O e Fea ) ha can pe o m ea u e ex ac ion on ou da a se wi h ou he need o high compu a ional cos . 12 C h a p e 3 Da a and Me hods This sec ion will desc ibe he con ex o he s udy a ea, how he da a was ga he and he mos impo an cha ac e is ics. 3.1 Desc ip ion o s udy a ea The da a was ga he on Sep embe 27 2014, nea Loma de Mico illage 12 ° 11’ 45.54" N, 83 ° 49’ 48.04"W in he Municipali y o Kuk a Hill, which is pa o he Au onomous egion o he Ca ibbean sou h coas (RACCs) o Nica agua (Figu e 3.1). Loma de Mico is conside ed o ha e a opical monsoon wea he , we season ex ends o 10 mon hs, his ansla es o 2000 - 3000 mm o ain all. A e age empe a u e anges om 24 - 27 ° C. Rolling hills wi h slopes be ween 20 - 30% a e s aples o he elie , wi h he highes poin ising 192 me e abo e sea le el. The ae ial su ey was done o e a palm ee (Elaeis guineensis) plan a ion wi h an a ea o 45 ha. In i se e al ypes o land co e could be dis inguished, mos p ominen ly p esen we e ma u e palm ees, palm ees in de eloping s ages, di oads, pa ches o na u al o es , g asslands and sca e in as uc u e. 3.2 UAV Sys em The sys em used o he ae ial su ey was a ixed wing glide UAV (Figu e 3.2). The ai plane is a skywalke model 1800 om 2014, which is o iginally manu ac u ed as a emo e con ol plane wi h ocus on p o iding he use s i s pe son iew (FPV) capabili ies. The RC ai plane was modi ied o i la ge payloads (came a, ba e ies) and wi h an au opilo , senso s and na iga ion sys em making i capable o au onomous 13 CHAPTER 3. DATA AND METHODS Figu e 3.1: S udy a ea in Loma de Mico, Eas Nica agua ligh , his was done by he consul ing i m EVOLO. Nex a de ailed desc ip ion o he model is gi en. The ai ame o body was made om EPO (Expanded PolyOle in) oam, had a o al weigh o 5 kg, a wingspan o 1800 mm and an endu ance o 35 minu es a a speed o 40 km/h. The sys em used an A duPilo Mega au opilo APM 2.5, capable o au onomous s abiliza ion and way poin na iga ion, coupled wi h compass, GPS, ba ome ic p essu e senso and eleme y communica ion, he sys em as a whole is capable o comple ing a p e loaded ligh plan au onomously. The g ound con ol s a ion was a lap op unning Mission Planne so wa e e sion 1.3.10 connec ed o a adio ecei e , his allowed eal ime communica ion wi h he UAV, du ing he mission i ansmi ed posi ion (X,Y), heigh abo e g ound le el, yaw, pi ch, oll angles, speed and ba e y le els. Images we e collec ed using a Canon Powe Sho S100 came a (Canon Inc., Tokyo, Japan) his model has a complemen a y me al-oxide semiconduc o (CMOS) senso size 7.53 x 5.64 mm and an app oxima e a ea o 42.5 mm which i s 12,100,000 pixels. Cus om i mwa e CHDK e sion 1.2 (Canon Hack De elopmen Ki ) was se up on he came a. This ee so wa e was de eloped by Canon PoweSho use s wi h he pu pose o adding ea u es and enhancing he capabili ies o he "poin and shoo " came a. In his s udy i was used o p og am he came a o ake pic u es e e y second by inc easing he shu e speed, his would no be possible using he de aul so wa e p o ided by he manu ac u e . 14 C h a p e 4 Resul s 4.1 Resul o e iew Resul s a e p esen ed in his chap e , e y high esolu ion images we e used as inpu o ex ac ing ea u es using a p e ained CNN, he ea u es we e used o ain ma- chine lea ning models and o classi y he images, he bes model (SVM) achie ed a accu acy o 97%. 4.2 Image p ep ocessing The c opping p ocess o each image p oduced 192 iles, gene a ing a o al o 2112 iles. Each ile was gi en a name based on he numbe o he image and he numbe o ile. The lis wi h names was used o labeling each ile depending on he p esence o absence o palms, in o al iles labeled as "1" we e 1001, meaning iles wi h palms. The emaining 1111 we e labeled as "0" meaning no palms. Table 4.1 p o ides an example o he name gi en o a ile, he image i belongs o, he numbe o he ile unde "c op" column and he label gi en. This able con ains six eco ds as an example o he esul s om he c opping p ocess and he nomencla u e used o each ile. name numbe c op label IMG_2361_250x250_000.jpg 2631 0 1 IMG_2367_250x250_100.jpg 2367 100 0 IMG_2371_250x250_021.jpg 2371 21 1 IMG_2374_250x250_029.jpg 2374 29 0 IMG_2376_250x250_031.jpg 2376 31 0 IMG_2392_250x250_020.jpg 2392 20 1 Table 4.1: C opped iles and labels gi en based on p esence o palms 21 CHAPTER 4. RESULTS A de ailed look o he objec s p esen in he iles is p esen ed in Figu e 4.1, i shows examples o he mos ep esen a i e objec s o he s udy a ea, such as palm in de elopmen s ages and ma u e, di oad, o es , human made in as uc u e and g assland. I ems (a), (b) and (c) a e iles classi ied as 1, on he o he hand i ems (d), (e) and ( ) a e classi ied as 0. (a) (b) (c) (d) (e) ( ) Figu e 4.1: Examples o objec s in he iles (a) Ma u e Palms (b) Palms in de elopmen s ages (c) Di oad (d) In as uc u e (e) G assland ( ) Fo es 4.3 Fea u e ex ac ion wi h O e Fea The c opped images we e p ocessed wi h O e Fea , he ou pu o his p ocess was a ile con aining ea u es om each ile. This is a ec o o 4096 dimensions also called CNN codes Ka pa hy (2016), speci ically i s a ex ile con aining a lis o alues ha ep esen in a ma hema ical o m he geome ies ound in he iles. This da a was me ged wi h he ile gene a ed in sec ion 4.2 and made i possible o ela e labels o he ex ac ed ea u es ile. Examples o he ex ac ed ea u es o iles labeled as "1" and "0" can be ound in Annex I. 22 4.4. MACHINE LEARNING MODELS CLASSIFICATION RESULTS 4.4 Machine lea ning models classi ica ion esul s The da a se gene a e in sec ion 4.3 was used as inpu o aining machine lea ning models, he da a was andomly spli 70% o aining and 30% es ing, he models we e ine uned and he bes pa ame e s we e selec ed and hen es ed. Figu e 4.2 summa- izes he o e all accu acy o each model, DT model achie ed he lowes accu acy wi h 86.88%±1.11% , he second lowes was RF model wi h 90.50%±0.71% , he nex h ee models had simila pe o mance, GBC 92.66%±0.79% , KNN 94.18%±0.63% and LR 95.88%±0.60% , inally he SVM model had he highes accu acy wi h 97.73%±0.51% . Figu e 4.2: O e all accu acy and s anda d de ia ion o models Table 4.2 p o ides a simila desc ip ion o he pe o mance om he classi ie s, bu also includes alues o sensi i i y, speci ici y and p ecision. Wi h ega ds o sensi i i y he op h ee alues ollows he same o de as obse ed in he o e all accu acy, he o h highes alue co espond o he RF model ollowed by GBC, his is he in e se o de compa ed o o e all accu acy. The lowes alue co esponds o DT model, as is he case wi h o e all accu acy. Values o speci ici y ollow he same pa e n as he o e all accu acy, while he p ecision mi o s he o de ound in he sensi i i y. Gene ally he e’s a endency ha he DT model p o ides he lowes alues ega dless i is sensi i i y, speci ici y, p ecision and he e o e o e all accu acy, andom o es 23 CHAPTER 4. RESULTS pe o mance be e han DT, bu no as good as he clus e o m by GBC, KNN and LR, hese h ee models ha e a simila pe o mance bu om he h ee GBC pe o ms wo s and LR achie ing he bes pe o mance. Finally he SVM has he gene al highes pe o mance om all he models. Classi ie Sensi i i y Speci ici y P ecision OA STD SVM 97.30 98.11 97.89 97.73 0.51% LR 96.30 95.50 96.88 95.88 0.60% KNN 95.80 92.71 96.38 94.18 0.63% GBC 93.31 92.08 93.87 92.66 0.79% RF 94.71 86.68 95.28 90.48 0.71% DT 85.61 88.03 86.13 86.88 1.11% Table 4.2: Accu acy assessmen o Classi ica ion models Pe o mance o he models a e p esen ed in a mo e angible ma e in Table 4.3, Table 4.4 and Table 4.5. These con usion ma ices ep esen he classi ica ion esul s o DT, GBC and SVM models, hese h ee we e compa ed because hey ep esen he ange o pe o mances, he models p esen s he lowes and highes accu acy (DT, SVM) as well a model which pe o med in he middle g ound be ween he o ex emes (GBC). The ables ha e wo ows and wo columns, he heade o he columns a e "0" and "1" each ep esen ing he labels. The combina ion o ows and columns p oduce he alues o ue posi i es (TP) "1 1", iles wi h palms co ec ly classi ied, ue nega i es (TN) "0 0", iles wi h no palms co ec ly classi ied, alse posi i es (FP)"0 1", iles wi h no palms classi ied as ha ing palms and "1 0" alse nega i es (FN), iles wi h palms, ha we e no ecognized by he model. Re e ence DT 0 1 To al Resul P edic ed 0 978 144 1122 1 133 857 990 To al Resul 1111 1001 2112 Table 4.3: Con usion ma ix DT Table 4.3 p esen s he esul s o DT, he model wi h he lowes pe o mance. Tiles classi ied as ue posi i es amoun 857 and ue nega i es 978, lea ing a o al o alse posi i es o 144 and alse nega i es o 133. Re e ence GBC 0 1 To al Resul P edic ed 0 1023 67 1090 1 88 934 1022 To al Resul 1111 1001 2112 Table 4.4: Con usion ma ix GBC 24 4.4. MACHINE LEARNING MODELS CLASSIFICATION RESULTS Table 4.4 de ails he esul s o om g adien boos classi ie , i ep esen s he mid- dle g ound o pe o mance, i co ec ly classi ied 934 iles wi h palms ( ue posi i es) and 1023 wi hou palms ( ue nega i es). False posi i es iles sum a o al o 67 and alse nega i es 88. Re e ence SVM 0 1 To al Resul P edic ed 0 1090 27 1117 1 21 974 995 To al Resul 1111 1001 2112 Table 4.5: Con usion ma ix SVM Table 4.5 is ocus on SVM, i classi ied co ec ly 974 om 1001 (T ue Posi i es) iles wi h palm ees, in he case o iles wi h no palm i classi ied co ec ly 1090 om 1111 (T ue Nega i es). I classi ied 21 iles as alse nega i es and 27 as alse posi i es. The esul s p esen ed in his sec ion poin s o he SVM as being he bes model, conside ing his endency he image classi ica ion esul s will ocus on he ou pu p o ided by he SVM model. 4.4.1 SVM classi ica ion esul s The SVM model used a linea ke nel and he cos pa ame e was se o "C=1000". The linea ke nel is ecommend o da a se s ha can be linea ly sepa a ed, also is less complex compa ed o he adial base ke nel, hus i has less pa ame e s ha need ine uning. The cos pa ame e was ob ained du ing he ine uning p ocess. In Table 4.6 a b eakdown o he classi ica ion esul s pe image, o example images 2376 and 2377 ha e no alse nega i es, his is in acco dance o he desc ip ion gi en in sec ion 3.4, as bo h his images con ain a low pe cen age o palms, 22% and 1% espec i ely. Images 2371 and 2378 each ha e one ile classi ied as alse nega i e. The image wi h he mos FN iles is 2395 wi h 6 ollowed by 2389 wi h 5 and 2367 wi h 4, in hese images palm ep esen s a leas 60% o he image, mixed wi h o he ea u es and land co e ypes such as o es and g asslands, di oads The case wi h mo e FP iles co espond o image 2371 wi h 5 cases, his image is composed by ma u e palm plan a ion, g assland and o es . Images 2376 and 2389 ha e 4 cases each o FP, his images a e con as ing, as he i s one is mainly na u al o es and palm only ep esen s 22% o he image and he la he is mainly palm ee plan a ion mix wi h o es , g assland and in as uc u e. Is impo an o no ice ha image 2389 p esen s he mos iles w ongly classi ied wi h a o al o 11, on he con a y image 2377 has no w ongly classi ied iles. These wo images a e e y diffe en om each o he , he desc ip ion p o ided in sec ion 3.4 shows ha 2389 con ains, se e al land co e ypes, while image 2377 is mainly one land co e ype co esponding o na u al o es . 25 CHAPTER 4. RESULTS image Name TP TN FP FN 2361 171 19 0 2 2367 85 102 1 4 2371 109 77 5 1 2374 43 145 1 3 2376 9 179 4 0 2377 7 185 0 0 2378 27 162 2 1 2389 118 65 4 5 2392 170 19 0 3 2395 172 13 1 6 2402 63 124 3 2 To al 974 1090 21 27 Table 4.6: Classi ica ion esul s om SVM model o each image Adding o he pe o mance analysis, Figu e 4.3 shows he ecei e ope a ing cha - ac e is ics (ROC) cu e. This analysis p o ides an insigh in o how well he model sepa a es da a in o iles con aining palm and no palm. The ho izon al axis con ains he alues o he alse posi i e a e and in he e ical axis he ue posi i e a e, his alues a e closely linked o he speci ici y and sensi i i y espec i ely. The cu e has an upwa d end and le els ou , nea he op o g aph, whe e he alues o ue posi i e a e is close o 1. The cu e p oduces a high alue o he me ic a ea unde he cu e, in his case 95 % o he a ea o he g aph is unde he cu e, his means ha he model is good a sepa a ing he classes. As a e e ence he ed do ed line ha ans e se he g aph ep esen s a 50% p obabili y o co ec ly classi ying he da a o andom chance. The u he away he blue cu e is om he ed do ed line means ha he model is be e classi ie compa ed o andom chance. 4.5 Image Classi ica ion Following he esul s om sec ion 4.4, classi ica ion o he images was done based on he SVM model. P edic ions we e made on he da a ex ac ed in sec ion 4.3, and compa ed o he labels manually gi en. A shape ile wi h he classi ica ion esul s was c ea ed and used o aid isualiza ion. This was done by s acking he shape ile o e he image ile and ca ego izing he iles by colo . FP and FN iles we e gi en magen a and cyan espec i ely and a wide bo de o make hen s and ou . TP and TN nega i es we e gi en blue and ed colo s, less ocused was gi en o his iles as hey we e co ec ly classi ied. An example o he classi ica ion is p esen ed in Figu e 4.4, emphasis is gi en o his sub se o images as he con ain mos o miss classi ied iles. The comple e esul s o all he images a e p esen ed in Annex I. I em (a) is he classi ica ion esul s o image 2367, i s a mix o palms and na u al o es , i p esen s 4 cases o FP and 1 o FN. I em 26 4.5. IMAGE CLASSIFICATION Figu e 4.3: SVM classi ie ROC cu e (b) co esponds o image 2371 which is a mix be ween palms and g asslands,i has he mos cases o FN iles om his sub se wi h 5 and only 1 case o FP. Image 2374 is showed in i em (c) is simila in con en o i em (a) i p esen s 3 cases o FP and 1 o FN, i em (d) is image 2378 is manly na u al o es and a small pa ch o palms i has 2 cases o FN and 1 FP. Image 2389 co esponds o i em (e) i has he mos cases o miss classi ica ion, 4 FN and 5 FP, he image has a pa ch o na u al o es palms, di oads and human made in as uc u e. I em ( ) is mainly palm ees and a bi o na u al o es , i ep esen s image 2395 and has 1 case o FN bu he mos FP cases om his sub se wi h 6. I em (g) is image 2402, i has a simila s uc u e o i em (a) and has 3 FN and 2 FP. In gene al ea u es p esen ed in his igu e sha e he cha ac e is ics o o ha ing se e al land co e ypes, meaning ha he images con ain ansi ions a eas, whe e is no clea ly Figu e 4.5 p esen s examples o FP iles, hey we e gi en label "0" bu he model ga e i a inco ec label o "1". In gene al his iles can be ound in ansi ions e- gions, o example i em (a) is he ansi ion zone be ween a pa ch o o es and palm plan a ion. I ems (b) and (d) a e g asslands bu ha ha e bush like plan s ha sha e esemblance wi h palm in de eloping s ages, i em (c) is o es land co e wi h se e al ypes o ees, i p esen s a complex s uc u e ha con used he classi ie . To ha e a close look and analyze iles classi ied as FN Figu e 4.6 was c ea ed. Examples in he igu e show palms ha we e no ecognized by he model, i em (a) con ains ma u e palms and a di oad in e sec ion, i em (b) is showing some sca e palms no ye ully de elop, i em (c) a e palms nea he beginning o a pa ch o o es and i em (e) a e se e al ma u e palms and one small palm. In gene al his iles show 27 CHAPTER 4. RESULTS (a) (b) (c) (d) (e) ( ) (g) (h) (a) image 2367 (b) image 2371 (c) image 2374 (d) image 2378 (e) image 2389 ( ) image 2395 (g) image 2402 (h) Legend Figu e 4.4: Image classi ica ion esul s 28 4.5. IMAGE CLASSIFICATION (a) (b) (c) (d) Figu e 4.5: Examples o iles classi ied as False Posi i es (a) image 2389 (b) image 2371 (c) image 2374 (d) image 2389 palms mixed wi h o he objec s, close o ansi ion zones, g ainy images o wi h mo ion blu and palms wi h diffe en de elopmen s ages in he same ile. The esul s p esen ed in Figu e 4.5 and Figu e 4.6 pain a gene al pic u e o whe e he model has p oblems wi h classi ica ion. In gene al e ms his a e a eas wi h a lo o diffe en objec s such as palms wi h di oads o o es . Figu e 4.5 is a close up o examples o images classi ied as alse posi i es. The igu e p esen some common cha ac e is ics as hey a e low quali y images, whe e de ec ing palms can be difficul e en o humans, also some palms a e in de eloping s ages and as such don’ ha e he cha ac e is ic ma u e palm shape, o he examples include isola ed palms and ansi ions zones whe e he land co e changes om plan a ion o some hing diffe en . A b ie desc ip ion o he iles is p esen ed nex : (a), image 2389, he e’s a small bush ha esembles a palm, (b) image 2371, g assland wi h and bushes, (c) image 2374, se e al ees o ming a complex s uc u e and (d) image 2389, he image is blu y and has plan s ha a e difficul o dis inguish o con iden ly classi y as palms. Figu e 4.6 29 CHAPTER 4. RESULTS sha e some o he same cha ac e is ics, e o s whe e loca ed nea ansi ions zones, nea he bounda ies o he plan a ion o iles wi h diffe en land co e s. I was also no ed ha iles whe e he e is a mix o palms in diffe en de elopmen s ages a e p oblema ic, palms in de eloping s age ha e a one o colo ha makes hen simila o bushes, hus making hem difficul o de ec e en o he human eye. The iles p esen ed in he igu e belong o (a) image 2392, i has ma u e palms and a di oad, (b) image 2389, small palms and g assland,(c) image 2374, small palms nex o o es and (d) image 2395, ma u e palms mixed wi h small palms. (a) (b) (c) (d) Figu e 4.6: Examples o iles classi ied as False Nega i es (a) image 2392 (b) image 2389 (c) image 2374 (d) image 2395 30 BIBLIOGRAPHIC REFERENCES Hubel, D. H. and T. N. Wiesel (1962). “Recep i e ields, binocula in e ac ion and unc ional a chi ec u e in he ca ’s isual co ex.” In: The Jou nal o physiology 160.1, pp. 106–154. Hung, C., Z. Xu, and S. Sukka ieh (2014). “Fea u e lea ning based app oach o weed classi ica ion using high esolu ion ae ial images om a digi al came a moun ed on a UAV.” In: Remo e Sensing 6.12, pp. 12037–12054. Iizuka, K., M. I oh, S. Shiode a, T. Ma suba a, M. Doha , and K. Wa anabe (2018). “Ad- an ages o unmanned ae ial ehicle (UAV) pho og amme y o landscape analysis compa ed wi h sa elli e da a: A case s udy o pos mining si es in Indonesia.” In: Cogen Geoscience 4.1, pp. 1–15. In e na ional Ci il A ia ion O ganiza ion (2011). Unmanned Ai c a Sys ems (UAS). 1, pp. 1–54. Ka pa hy, A. (2016). Lec u e no es Con olu ional Neu al Ne wo ks o Visual Recogni ion. K izhe sky, A., I. Su ske e , and G. E. Hin on (2012). “ImageNe Classi ica ion wi h Deep Con olu ional Neu al Ne wo ks.” In: Ad ances in Neu al In o ma ion P ocess- ing Sys ems 25. Ed. by F. Pe ei a, C. J. C. Bu ges, L. Bo ou, and K. Q. Weinbe ge . Cu an Associa es, Inc., pp. 1097–1105. u l:h p://pape s.nips.cc/pape /4824- imagene -classi ica ion-wi h-deep-con olu ional-neu al-ne wo ks.pd . La y, D. J., A. H. Ala i, A. H. Gandomi, and A. L. Walke (2016). “Machine lea ning in geosciences and emo e sensing.” In: Geoscience F on ie s 7.1, pp. 3–10. La y, D. J. (2010). “A i icial in elligence in geoscience and emo e sensing.” In: Geo- science and Remo e Sensing New Achie emen s. InTech. Law ence, R. L. and C. J. Mo an (2015). “The Ame icaView classi ica ion me hods accu acy compa ison p ojec : A igo ous app oach o model selec ion.” In: LeCun, Y., L. Bo ou, Y. Bengio, and P. Haffne (1998). “G adien -based lea ning ap- plied o documen ecogni ion.” In: P oceedings o he IEEE 86.11, pp. 2278–2324. Li, W., H. Fu, L. Yu, and A. C acknell (2017). “Deep Lea ning Based Oil Palm T ee De ec ion and Coun ing o High-Resolu ion Remo e Sensing Images.” In: Remo e Sensing 9.1. issn: 2072-4292. doi:10.3390/ s9010022.u l:h p://www.mdpi. com/2072-4292/9/1/22. Lillesand, T., R. W. Kie e , and J. Chipman (2014). Remo e sensing and image in e p e a- ion. John Wiley & Sons. Long, J., E. Shelhame , and T. Da ell (2015). “Fully con olu ional ne wo ks o se- man ic segmen a ion.” In: P oceedings o he IEEE con e ence on compu e ision and pa e n ecogni ion, pp. 3431–3440. Luna, I. and A. Lobo (2016). “Mapping c op plan ing quali y in suga cane om UAV image y: a pilo s udy in Nica agua.” In: Remo e Sensing 8.6, p. 500. Ma manis, D., M. Da cu, T. Esch, and U. S illa (2016). “Deep lea ning ea h obse a ion classi ica ion using ImageNe p e ained ne wo ks.” In: IEEE Geoscience and Remo e Sensing Le e s 13.1, pp. 105–109. 37 BIBLIOGRAPHIC REFERENCES Masini, N., F. Gizzi, M. Biscione, V. Fundone, M. Sedile, M. Sileo, A. Pecci, B. Laco a a, and R. Lasapona a (2018). “Medie al A chaeology Unde he Canopy wi h LiDAR. The (Re) Disco e y o a Medie al Fo i ied Se lemen in Sou he n I aly.” In: Remo e Sensing 10.10, p. 1598. Maxwell, A. E., T. A. Wa ne , and F. Fang (2018). “Implemen a ion o machine-lea ning classi ica ion in emo e sensing: An applied e iew.” In: In e na ional jou nal o emo e sensing 39.9, pp. 2784–2817. Megahed, Y., P. Cab al, J. Sil a, and M. Cae ano (2015). “Land co e mapping analysis and u ban g ow h modelling using emo e sensing echniques in g ea e Cai o egion—Egyp .” In: ISPRS In e na ional Jou nal o Geo-In o ma ion 4.3, pp. 1750– 1769. Mesas-Ca ascosa, F. J., I. C. Rumbao, J. A. B. Be ocal, and A. G.-F. Po as (2014). “Posi ional quali y assessmen o o hopho os ob ained om senso s onboa d mul i- o o UAV pla o ms.” In: Senso s 14.12, pp. 22394–22407. Miki a, T., P. Jana a, and P. Su o ` y (2016). “Fo es s and in en o y based on combined ae ial and e es ial close- ange pho og amme y.” In: Fo es s 7.8, p. 165. Moun akis, G., J. Im, and C. Ogole (2011). “Suppo ec o machines in emo e sensing: A e iew.” In: ISPRS Jou nal o Pho og amme y and Remo e Sensing 66.3, pp. 247–259. Na algund, R. R., V Jaya aman, and P. Roy (2007). “Remo e sensing applica ions: An o e iew.” In: Cu en Science (00113891) 93.12. Noguei a, K., O. A. Pena i, and J. A. dos San os (2017). “Towa ds be e exploi ing con olu ional neu al ne wo ks o emo e sensing scene classi ica ion.” In: Pa e n Recogni ion 61, pp. 539–556. O hman, E., Y. Bazi, N. Alajlan, H. Alhich i, and F. Melgani (2016). “Using con o- lu ional ea u es and a spa se au oencode o land-use scene classi ica ion.” In: In e na ional Jou nal o Remo e Sensing 37.10, pp. 2149–2167. Paja es, G. (2015). “O e iew and cu en s a us o emo e sensing applica ions based on unmanned ae ial ehicles (UAVs).” In: Pho og amme ic Enginee ing & Remo e Sensing 81.4, pp. 281–330. Pal, M. and P. M. Ma he (2003). “An assessmen o he effec i eness o decision ee me hods o land co e classi ica ion.” In: Remo e sensing o en i onmen 86.4, pp. 554–565. Ped egosa, F., G. Va oquaux, A. G am o , V. Michel, B. Thi ion, O. G isel, M. Blondel, P. P e enho e , R. Weiss, V. Dubou g, J. Vande plas, A. Passos, D. Cou napeau, M. B uche , M. Pe o , and E. Duchesnay (2011). “Sciki -lea n: Machine Lea ning in Py hon.” In: Jou nal o Machine Lea ning Resea ch 12, pp. 2825–2830. Pena i, O. A. B., K. Noguei a, and J. A. dos San os (2015). “Do Deep Fea u es Gene - alize F om E e yday Objec s o Remo e Sensing and Ae ial Scenes Domains?” In: The IEEE Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR) Wo kshops. 38 BIBLIOGRAPHIC REFERENCES Pe opoulos, G. P., C. Kalai zidis, and K. P. Vad e u (2012). “Suppo ec o machines and objec -based classi ica ion o ob aining land-use/co e ca og aphy om Hy- pe ion hype spec al image y.” In: Compu e s & Geosciences 41, pp. 99–107. P lanz, M., H. No dmeye , and M. Schi mann (2018). “Weed Mapping wi h UAS Image y and a Bag o Visual Wo ds Based Image Classi ie .” In: Remo e Sensing 10.10, p. 1530. Ren, S., K. He, R. Gi shick, and J. Sun (2015). “Fas e R-CNN: Towa ds Real-Time Ob- jec De ec ion wi h Region P oposal Ne wo ks.” In: Ad ances in Neu al In o ma ion P ocessing Sys ems 28. Ed. by C. Co es, N. D. Law ence, D. D. Lee, M. Sugiyama, and R. Ga ne . Cu an Associa es, Inc., pp. 91–99. u l:h p://pape s.nips. cc/pape /5638- as e - -cnn- owa ds- eal- ime-objec -de ec ion-wi h- egion- p oposal-ne wo ks.pd . Rich e , G. M., F. Agos ini, A. Ba ke , D. Cos omi is, and A. Qi (2016). “Assessing on- a m p oduc i i y o Miscan hus c ops by combining soil mapping, yield modelling and emo e sensing.” In: Biomass and Bioene gy 85, pp. 252–261. Rogan, J., J. F anklin, D. S ow, J. Mille , C. Woodcock, and D. Robe s (2008). “Map- ping land-co e modi ica ions o e la ge a eas: A compa ison o machine lea ning algo i hms.” In: Remo e Sensing o En i onmen 112.5, pp. 2272–2283. Russako sky, O., J. Deng, H. Su, J. K ause, S. Sa heesh, S. Ma, Z. Huang, A. Ka pa hy, A. Khosla, M. Be ns ein, e al. (2015). “Imagene la ge scale isual ecogni ion challenge.” In: In e na ional Jou nal o Compu e Vision 115.3, pp. 211–252. Salamí, E., C. Ba ado, and E. Pas o (2014). “UAV ligh expe imen s applied o he emo e sensing o ege a ed a eas.” In: Remo e Sensing 6.11, pp. 11051–11081. Salbe g, A.-B. (2015). “De ec ion o seals in emo e sensing images using ea u es ex ac ed om deep con olu ional neu al ne wo ks.” In: Geoscience and Remo e Sensing Symposium (IGARSS), 2015 IEEE In e na ional. IEEE, pp. 1893–1896. Se mane , P., D. Eigen, X. Zhang, M. Ma hieu, R. Fe gus, and Y. LeCun (2013). “O e - ea : In eg a ed ecogni ion, localiza ion and de ec ion using con olu ional ne - wo ks.” In: a Xi p ep in a Xi :1312.6229. Sha i Raza ian, A., H. Azizpou , J. Sulli an, and S. Ca lsson (2014). “CNN ea u es off- he-shel : an as ounding baseline o ecogni ion.” In: P oceedings o he IEEE con e ence on compu e ision and pa e n ecogni ion wo kshops, pp. 806–813. Sil a, J. D.B. F. da (2016). “Con ibu ions o he imp o emen o speci ic class map- ping.” Doc o al disse a ion. Uni e sidade No a de Lisboa In o ma ion Manage- men School. Simonyan, K. and A. Zisse man (2014). Ve y Deep Con olu ional Ne wo ks o La ge- Scale Image Recogni ion. a Xi : 1409.1556 [cs.CV]. Suma hi, S, S. Chai, and A. Mohamed (2008). “U iliza ion o oil palm as a sou ce o enewable ene gy in Malaysia.” In: Renewable and Sus ainable Ene gy Re iews 12.9, pp. 2404–2421. 39 BIBLIOGRAPHIC REFERENCES Szegedy, C., W. Liu, Y. Jia, P. Se mane , S. Reed, D. Anguelo , D. E han, V. Vanhoucke, and A. Rabino ich (2015). “Going Deepe Wi h Con olu ions.” In: The IEEE Con- e ence on Compu e Vision and Pa e n Recogni ion (CVPR). Tianyang, D., Z. Jian, G. Sibin, S. Ying, and F. Jing (2018). “Single-T ee De ec ion in High-Resolu ion Remo e-Sensing Images Based on a Cascade Neu al Ne wo k.” In: ISPRS In e na ional Jou nal o Geo-In o ma ion 7.9, p. 367. Villa, T. F., F. Gonzalez, B. Miljie ic, Z. D. Ris o ski, and L. Mo awska (2016). “An o e iew o small unmanned ae ial ehicles o ai quali y measu emen s: P esen applica ions and u u e p ospec i es.” In: Senso s 16.7, p. 1072. Wang, R., S. Zhang, L. Pu, J. Yang, C. Yang, J. Chen, C. Guan, Q. Wang, D. Chen, B. Fu, e al. (2016). “Gully e osion mapping and moni o ing a mul iple scales based on mul i-sou ce emo e sensing da a o he Sancha Ri e Ca chmen , No heas China.” In: ISPRS In e na ional Jou nal o Geo-In o ma ion 5.11, p. 200. Waske, B., J. A. Benedik sson, and J. R. S einsson (2009). “Classi ying emo e sensing da a wi h suppo ec o machines and imbalanced aining da a.” In: In e na ional Wo kshop on Mul iple Classi ie Sys ems. Sp inge , pp. 375–384. Wa s, A. C., V. G. Amb osia, and E. A. Hinkley (2012). “Unmanned ai c a sys ems in emo e sensing and scien i ic esea ch: Classi ica ion and conside a ions o use.” In: Remo e Sensing 4.6, pp. 1671–1692. Yuan, Y. and X. Hu (2016). “Random o es and objec ed-based classi ica ion o o es pes ex ac ion om UAV ae ial image y.” In: The In e na ional A chi es o Pho og amme y, Remo e Sensing and Spa ial In o ma ion Sciences 41, p. 1093. Zecha, C., J Link, and W Claupein (2013). “Mobile senso pla o ms: Ca ego isa ion and esea ch applica ions in p ecision a ming.” In: Jou nal o Senso s and Senso Sys ems 2.1, pp. 51–72. Zeile , M. D. and R. Fe gus (2013). S ochas ic Pooling o Regula iza ion o Deep Con o- lu ional Neu al Ne wo ks. a Xi : 1301.3557 [cs.LG]. Zeile , M. D. and R. Fe gus (2014). “Visualizing and unde s anding con olu ional ne wo ks.” In: Eu opean con e ence on compu e ision. Sp inge , pp. 818–833. Zhang, L., L. Zhang, and B. Du (2016). “Deep lea ning o emo e sensing da a: A echnical u o ial on he s a e o he a .” In: IEEE Geoscience and Remo e Sensing Magazine 4.2, pp. 22–40. Zhang, L., Z. Shi, and J. Wu (2015). “A hie a chical oil ank de ec o wi h deep su - ounding ea u es o high- esolu ion op ical sa elli e image y.” In: IEEE Jou nal o Selec ed Topics in Applied Ea h Obse a ions and Remo e Sensing 8.10, pp. 4895– 4909. Zhou, P., G. Cheng, Z. Liu, S. Bu, and X. Hu (2016). “Weakly supe ised a ge de- ec ion in emo e sensing images based on ans e ed deep ea u es and nega i e boo s apping.” In: Mul idimensional Sys ems and Signal P ocessing 27.4, pp. 925– 944. 40 BIBLIOGRAPHIC REFERENCES Zhu, X. X., D. Tuia, L. Mou, G.-S. Xia, L. Zhang, F. Xu, and F. F aundo e (2017). “Deep lea ning in emo e sensing: a comp ehensi e e iew and lis o esou ces.” In: IEEE Geoscience and Remo e Sensing Magazine 5.4, pp. 8–36. Zo ea, M., M. Ne y, B. Ruga, L. B. Ca alho, and A. C. Bas os (2018). “Oil-Palm T ee De ec ion in Ae ial Images Combining Deep Lea ning Classi ie s.” In: IGARSS 2018-2018 IEEE In e na ional Geoscience and Remo e Sensing Symposium. IEEE, pp. 657–660. 41 A n n e x I Annex This sec ion con ains in o ma ion and esul s ha we e no p esen ed in he main ex bu a e complemen a y o he esul s I.1 Example o ea u es ex ac ed using O e Fea Only a po ion o he ea u es a e p esen ed since each ea u e has 4096 a iables. The i s numbe indica es he label, ollowed by he alues o he ea u es. • Example o ea u e ex ac ed wi h O e Fea o a ile label as "1" 1 -0.977721 -0.254899 -0.350732 -2.20161 0.167981 0.032499 -1.24036 -0.368897 0.0988937 -0.26209 -0.931044 0.121792 -0.182609 -1.45426 -1.85707 -0.254737 -1.17318 -0.861829 -2.39979 -2.0163 -0.294961 • Example o ea u e ex ac ed wi h O e Fea o a ile label as "0" 0 -2.67815 -1.83985 -0.732881 -3.21898 -1.22884 -1.39543 -2.83617 -1.52186 -2.04997 -0.404118 -1.40013 -1.29628 -0.978714 -2.79572 -0.372814 -1.82225 -1.96431 -1.44416 -3.20762 -2.65322 -0.602951 -2.64013 I.2 Image classi ica ion esul s Image classi ica ion esul s om SVM model, all he image a e p esen ed. 42 I.3. ROC GRAPHS Figu e I.1: Image classi ica ion esul s (a) (b) (c) (d) (e) ( ) (g) (h) (i) (j) (k) (a) Image 2389 (b) Image 2402 (c) Image 2395 (d) Image 2367 (e) Image 2361 ( ) Image 2371 (g) Image 2374 (h) Image 2376 (i) Image 2377 (j) Image 2378 (k) Image 2392 I.3 ROC g aphs ROC and a ea unde he cu e g aphs o each model. 43 ANNEX I. ANNEX Figu e I.2: ROC esul s o ML models (a) (b) (c) (d) (c) (d) (a) DT (b) LR (c) SVM (d) KNN (e) RF (e) GBC 44 I.4. CONFUSION MATRIX I.4 Con usion ma ix Con usion ma ices o he classi ie s used in he s udy Table I.1: Con usion ma ix SVM Re e ence SVM 0 1 To al Resul P edic ed 0 978 144 1117 1 133 857 995 To al Resul 1111 1001 2112 Table I.2: Con usion ma ix DT Re e ence DT 0 1 To al Resul P edic ed 0 978 144 1122 1 133 857 990 To al Resul 1111 1001 2112 Table I.3: Con usion ma ix LR Re e ence LR 0 1 To al Resul P edic ed 0 1061 37 1098 1 50 964 1014 To al Resul 1111 1001 2112 Table I.4: Con usion ma ix GBC Re e ence GBC 0 1 To al Resul P edic ed 0 1023 67 1090 1 88 934 1022 To al Resul 1111 1001 2112 45 ANNEX I. ANNEX Table I.5: Con usion ma ix KNN Re e ence KNN 0 1 To al Resul P edic ed 0 1030 42 1072 1 81 959 1040 To al Resul 1111 1001 2112 Table I.6: Con usion ma ix RF Re e ence RF 0 1 To al Resul P edic ed 0 963 53 1016 1 148 948 1096 To al Resul 1111 1001 2112 46 II.4. FINE TUNING MACHINE LEARNING MODELS 13 y = da a[:, 0] 14 X = da a[:, 1:] 15 16 ke nels = [’linea ’,’ b ’,’poly’] 17 o ke nel in ke nels: 18 s c = s m.SVC(ke nel=ke nel). i (X,y) 19 plo SVC(’ke nel=’ +s (ke nel)) 20 21 de plo SVC( i le): 22 #c ea e a mesh o plo in‘ 23 x_min, x_max = X[:,0].min() −1, X[:,0].max() + 1 24 y_min, y_max = X[:,1].min() −1, X[:,1].max() + 1 25 h = (x_max / x_min)/100 26 xx,yy = np.meshg id(np.a ange(x_min, x_max, h), np.a ange(y_min, y_max, h)) 27 pl .subplo (1, 1, 1) 28 Z = s c.p edic (np.c_[xx. a el(), yy. a el()]) 29 Z = Z. eshape(xx.shape) 30 pl .con ou (xx, yy, z, cmap=pl .cm.Pai ed, alpha=0.8) 31 pl .sca e (X[:, 1:], c=y, cmap=pl .cm.Pai ed) 32 pl .xlabel(’ ec o ’) 33 pl .ylabel(’label’) 34 pl .xlim(xx.min(), xx.max()) 35 pl . i le( i le) 36 pl .show() Lis ing II.5: SVM Fine Tuning 53 da eyea UNDEFINED i le UNDEFINED au ho sho name UNDEFINED PhD