Fuzzy - based method of detecting the enviroment character for UAV optical stabilization

INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER Fuzzy - Based Me hod o De ec ing he En i omen Cha ac e o UAV Op ical S abiliza ion Da id NOVAK1, Pe CERMAK2 1Ins i u e o Compu e Science, Facul y o Philosophy and Science, Silesian Uni e si y, Bez uc Sq. 13, 746 01, Opa a, Czech Republic 2Resea ch Ins i u e o he IT4Inno a ions Cen e o Excellence, In i u e o Compu e Science, Facul y o Philosophy and Science, Silesian Uni e si y, Bez uc Sq. 13, 746 01, Opa a, Czech Republic oln[email p o ec ed], [email p o ec ed] DOI: 10.15598/aeee. 13i3.1299 Abs ac . An op ical s abiliza ion o UAV (UAS) is a e y impo an pa o a s uc u e in hei con ol sys- ems. No only as a backup s abiliza ion sys em in a case o IMU ailu e, bu also as a main sys em, used o s abiliza ion o na iga ion. In his pape he con- cep o a sys em o en i onmen cha ac e de ec ion is p esen ed. The sys em can classi y a su ounding en- i onmen depending on chosen cha ac e is ics. Such sys em can be used o a be e ho izon de ec ion due o swi ching o a co ec ho izon de ec ion algo i hm, which can be used o de e mining he posi ion o UAV. The sys em is based on Takagi - Sugeno uzzy in e ence sys em and uzzy a i icial neu al ne wo ks. An ea lie wo k on his subjec was p esen ed las yea , bu con- cep o he sys em was edesigned wi h a usage o uzzy a i icial neu al ne wo k o a mo e p ecisi e ou pu s and au oma ic de e mina ion o cha ac e is ics o uzzy se s on inpu . Keywo ds Fuzzy, IMU, in e ence sys em, ne wo k, neu al, op ical s abiliza ion, UAV. 1. IMU and hei Weaknesses 1.1. Elec omagne ic Noise and GPS Denied En i onmen An ine ial measu emen uni (IMU) is an elec onic de ice ha measu es and epo s a c a eloci y, o i- en a ion, and g a i a ional o ces, using a combina ion o accele ome e s and gy oscopes, some imes also mag- ne ome e s. Cu en ly, he use o magne ome e s in IMUs, is mo e common. Magne ome e s using changes in he Ea h’s magne ic ield o mome emen de e - mina ion. Magne ic ields a e ec o quan i ies cha - ac e ized by bo h s eng h and di ec ion. Using hose alues we can e alua e speed and accele a ion in 3D space same way, like using accele ome e s. In a case o an elec omagne ic in e ences, magne ome e p o ides inco ec da a so e alua ions o posi ion a e inco ec . Such IMU ailu e may cause an inco ec eac ion o UAV con ol sys em and as esul c ash o damage o a ehicle. Elec omagne ic in e ences lead o an uncon olled mo emen s o UAV in a case o algo i hm ailu e ( o example posi ion hold). In such case, GPS signal can- no be used o co ec ion due o low accu acy (usually in me e s). D i o UAV, o example small helicop e , can lead o PID con olle oscilla ions which can cause an uncon olled mo emen s in a ea o i e me e s and mo e adius. The mos common sou ces o elec omagne ic in e - ence a e: •UAV elec ic pa s - o example elec ic mo o s, ansmi e . •Magne ic anomalies - such as a s o m, elec ic wi ing, elec ic ains. •Elec omagne ic pulse, in en ional in e e ences. Fo he easons men ioned abo e, he need o a backup sys em is ob ious, especially o UAV. Unlike no mal ai planes and helicop e s, which can be con- olled by a human pilo in a case o senso ailu e, he au onomous UAV do no ha e such an op ion. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 255 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER 1.2. Op ical S abiliza ion o he UAV The mos common al e na i e o IMU is an op ical s a- biliza ion. Usually op ical low, co ne and ea u e de- ec ion algo i hms a e used. Those algo i hms ha e limi a ions especially due o he low esolu ion o on- boa d came as. E en high esolu ion came as canno p o ide enough usable da a in high al i udes. In such al i udes, small changes and image shi ing, which a e needed o op ical low de e mina ion, canno be de- ec ed. High al i udes o UAV and bad a mosphe ic condi- ions a e easons o a ea u e de ec ion algo i hm ail- u es. In such condi ions, image ea u es usually canno be epea edly de ec ed. Fo applicable UAV op ical s abiliza ion we need some ixed poin s (clus e s) o de ec ed in o ma ion in cap u ed image. Such poin o in o ma ion should be p esen in mos cap u ed images so his in o ma ion can be used o de e mina ions o mo emen s. The ques ion is, how many poin s o in o ma ion is needed o UAV s abiliza ion in 6 deg ees o eedom (DOF). In ideal condi ion we ha e one de ec ed ea u e o e e y deg ee o eedom. Fo backup sys em o IMU i is possible o de e mine a comp omise and de ine minimum numbe o such ea u es which we can use o sho ime eme gency s abiliza ion. 2. Ho izon Based UAV S abiliza ion As a ho izon we can usually unde s and a di iding line be ween sky and g ound. Usually we canno be su e ha sky and g ound a e always isible, so le s use he de ini ion o ho izon as he line, pa allel wi h wa e su ace. Posi ion and angle o ho izon can be de e - mined by human eye in mos o scanned images. The e a e many de ec ion algo i hms o a ious condi ions. P oblem occu s wi h only one onboa d UAV came a. Usage o only one came a canno ensu e ha i will al- ways cap u e he scene in which he ho izon is loca ed, o he posi ion o he ho izon ha can be de e mined. Such p oblem can be pa ially sol ed by usage o mo e han one onboa d came a o wi h usage o pano amic onboa d came a. In a case o bad a mosphe ic con- di ions, which educes isibili y, we can use he mal ision, UV o HR came a. De ec ed ho izon p o ides in o ma ion, which can be used o eme gency UAV s abiliza ion. F om ho izon posi ion and angle we can calcula e in o ma ion abou changes o pi ch and oll o UAV. In o ma ion abou al i ude, o a ion and d i ing on ho izon al plane can- no be de e mined. F om hose easons we need o add an auxilia y image de ec ion algo i hms such as op ical low and ea u e de ec ion. The p oblem wi h low image esolu ion in high al i- udes canno be e ec i ely sol ed. Bu p oposed s abi- liza ion sys em will wo k as backup eme gency sys em, so his p oblem may no be signi ican . In a case o eme gency si ua ion we can p esume, ha scanned im- age esolu ion will be g ea e when UAV is close o he g ound o obs acle, so s abiliza ion sys em will ha e mo e p ecise ou pu . Such ea u e can a oid collisions in eme gency si ua ions. F om in o ma ion abo e we can unde s and ha o e icien op ical s abiliza ion, based on he ho izon de- ec ion, we need o p o ide ce ain y o ho izon de- ec ion in e e y si ua ion and en i onmen in which UAV will ope a e. Such ea u e can p o ide p oposed me hod o au oma ic en i onmen de ec ion. 2.1. Usual P oblems wi h he Ho izon De ec ion Algo i hms Cu en ly, he e a e some gene al ad ances in ho izon de ec ion. Each o hem cande ec he ho izon unde ce ain condi ions om he de ec ion o ho izon us- ing Hough ans o ma ion o mo e complex me hods. The e is an o e iew o some o he echniques and p oblems depending on he condi ions in Tab. 1. In- en ionally we do no men ion he de ec ion ho izon in u ban a eas he e, because in such condi ions all lis ed p ocedu es ail i ually. The e a e a numbe o ho izons de ec ion solu ions wi h di e en pe cen age success o he de ec ion. Fo small UAVs, which each maximum heigh s o hund eds o me e s, he ho izon de ec ion is no a i ial ask hough. A his poin i is necessa y o de ine he di - e ence be ween he ho izon and he isible ho izon. We conside as he ho izon he di iding line be ween sky and ea h, ha is, a ho izon al line ha is le el o he g ound. We unde s and he isible ho izon as he eal di iding line be ween sky and ea h, which is isi- ble om a gi en loca ion in he scene. Visible ho izon may he e o e no always be ho izon al wi h he g ound plane. Ano he p oblem a ises na u ally in u ban a - eas, whe e he ho izon mus be de ec ed o example om anishing poin s o de ec ed building ou lines. The ideal objec i e is o ind a uni e sal app oach how o de ec he ac ual posi ion o he ho izon and any en i onmen . Such p ocedu e would no ail in any a ea, which he UAV can ge in. To de ec ho izon in u ban a eas he e a e se e al me hods ha a e based on he moni o ing o main lines in he image and he de ec ion o he anishing poin s, om which he likely posi ion o ho izon and i s angle can be calcula ed. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 256 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER Tab. 1: Some p oblems o s anda d ho izon de ec o s depending on condi ions. The me hod The bes esul s The possible weaknesses Hough ans o ma ion, edge de ec o s. G ea heigh s, s aigh ho izon. Fails a low al i udes and inden ed a eas Sea ching o he cen e s o g a i y o he g ound and sky. Equally con ou ed ho izon In he inden ed su oundings and a highe cloudiness Op ical low. G ea heigh s, a li le con ou ed ho izon. May ail a low al i udes and inden ed su oundings. Algo i hms ha wo k wi h edges howe e ail in open landscape, because he e is no co ec de ec ion o ho - izon al lines and he subsequen de i a ion o he an- ishing poin s because hey a e i ually absen in he scene. 3. The De ec ion o he En i onmen Fo he p oblems desc ibed in he p e ious chap e s a simple solu ion can be used, based on he idea o de- ec ing he en i onmen , in which he UAV de ice is mo ing, and he subsequen co ec choice o he al- go i hm o de ec ing he ho izon. A his poin , i is i ele an wha kind o algo i hms o ho izon ecogni- ion we will use; he impo an hing is o choose a leas one o landscape wi hou build-up a ea, one o u ban a eas and one o indoo s. The ollowing pseudo- code ep esen s he basic algo i hm o en i onmen de ec ion. Algo i hm 1 1: o e e do 2: i (de ec ed.en i onmen . a iable) < u ban hen 3: e u n ho izon de ec ion algo i hm o open en i onmen 4: else 5: i (u ban.en i onmen . a iable) < ou doo s hen 6: e u n ho izon de ec ion algo i hm o u - ban en i onmen 7: else 8: e u n ho izon de ec ion algo i hm o in- doo s en i onmen 9: end i 10: end i 11: end o Howe e , he choice o co ec algo i hm which co - ec ly calcula es he decision a iables based on he de ec ed pa ame e s, which will lead o he co ec en- i onmen iden i ica ion is s ill a p oblem. Fo co ec decision is be e o use some so -compu ing me hod. We decide o use uzzy - based app oach. Basic p ob- lem is, o cou se, selec ing he igh inpu pa ame e s. 3.1. The Choice o De ec ion Pa ame e s The in o ma ion lis ed in he ex abo e implies ha one o he key pa ame e s o he decision-making al- go i hm will be s aigh lines de ec ion and hei cha - ac e is ics. Quan i y o de ec ed s aigh lines wi h speci ic leng hs is qui e di e en in open scene y and u ban de elopmen . Howe e , his may no always be decisi e. The s aigh lines, which can p o ide alse in o ma ion, such as loca ion o he anishing poin (powe lines, ences, oads), can be de ec ed e en in open landscape. The e o e, i is necessa y o p o ide addi ional inpu pa ame e s. We mus include only he lines ha ha e a ce ain leng h o he calcula ions, hough. I is necessa y o exclude sho lines, which may occu in bo h ypes o en i onmen s. We can se- lec minimum leng h o s aigh lines using o example he size o he p ocessed (see equa ion bellow), clong is a cons an , which de ines he minimum size o he de- ec ed lines. seg =imagewid h +imageheigh clong .(1) I is a good idea o use he di ec ion o de ec ed lines o calcula e he nex pa ame e . I is speci ically he numbe o lines ha ha e he same di ec ion as his a - ibu e co esponds o a high deg ee o u ban en i on- men . I is app op ia e o use he numbe o de ec ed ec angles is he las pa ame e . I is essen ial o use an algo i hm ha is able o de ec ec angles wi h pe - spec i e [1]. Rep esen a ion o ec angles in he images om u ban en i onmen is usually high. Because we de ec he s aigh lines and hei di ec ions, he e is no p oblem o ind he lines ha o m he edges o de- ec ed ec angles. Fo de ec ion o en i onmen i is necessa y o coun wi h he ac ha quad angles a e also de ec ed ou side u ban a eas, bu hei numbe will be low. Fo de e mina ion o a di e ence be ween indoo s and ou doo s en i onmen we can use posi ion o de ec ed anishing poin s. Le s sepa a e inpu se o anishing poin s o wo g oups, inside image and ou side. The e o e we ha e ou inpu pa ame e s: c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 257 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER •numbe o line segmen s, •numbe o pa allel lines clus e s, •numbe o de ec ed ec angles, •numbe o anishing poin s inside o image. Ou pu pa ame e s o decision making a e h ee: • a iable o u ban en i onmen , • a iable o non u ban en i onmen , •indoo a iable. The inal ou pu om decision p ocess is classi ied en i onmen (indoo , ou doo , ci y). 3.2. The P oposed Me hod Fo sol ing his p oblem we can choose om many possible so -compu ing app oaches. Fi s expe imen s we e made wi h usage o uzzy Takagi - Sugeno in e - ence sys em. Du ing implemen a ion o such sys em some impo an p oblems occu ed. The bigges p ob- lem was wi h uzzi ica ion o inpu and choosing a igh uzzy se s o linguis ic a iables. uzzy se s had o be speci ied manually, and ine uning also had o be made by he p og amme . Also ules beha iou was no easy o p edic . F om hose easons we decided o use an a i icial neu al ne wo k wi h supe ised lea ning. Ou sys em should be able o lea n om gi en examples o en i onmen s. As he bes app oach seems o be us- age o uzzy classi ica ion neu al ne wo k (NFC), which app oxima es beha iou o Takagi - Sugeno in e ence sys em [5]. Scheme o p oposed me hod can be seen in Fig. 1. Images om onboa d UAV came a is p ocessed by im- age ecognizing p ocedu es and numbe s o lines, clus- e s, ec angles and anishing poin s a e de e mined. Those alues a e used as an inpu o uzzy classi ica- ion neu al ne wo k. Based on NFC ou pu , he co - ec ho izon ecogni ion algo i hm is chosen. De ec ed ho izon is used o u he op ical s abiliza ion o UAV. Fi s s ep was o c ea e an Takagi - Sugeno based uzzy in e ence sys em. A ypical ule in a Sugeno uzzy model has he o m [3]: Algo i hm 2 1: i Inpu 1 = x and Inpu 2 = y, hen 2: Ou pu is z = ax + by + ci 3: end i Fo a ze o-o de Sugeno model, he ou pu le el zis a cons an (a=b= 0). The ou pu le el zio each Fig. 1: Scheme o p oposed sys em o en i onmen de ec ion and s abiliza ion. ule is weigh ed by he i ing s eng h wio he ule. Fo example, o an AND ule wi h Inpu 1=xand Inpu 2=y, he i ing s eng h is: wi=AndMe hod (F1(x)F2(y)) ,(2) whe e F1() and F2() a e he membe ship unc ions o Inpu 1and Inpu 2. The inal ou pu o he sys em can be he weigh ed a e age o all ule ou pu s. FinalOu pu =Pn i=1 wizi Pn i=1 wi .(3) Mo e abou he Sugeno uzzy in e ence sys ems can be ound in [3] o [4]. Le us deno e he inpu and ou pu pa ame e s as: •SEG numbe o line segmen s, •CL numbe o pa allel lines clus e s, •RC numbe o de ec ed ec angles, •VPI numbe o anishing poin s inside image. Ini ial inpu uzzy membe ship unc ion we e se as we can see on Fig. 2, we ha e chosen Gaussian unc ion. Ou pu o in e ence sys em should be h ee a iables, one o he u ban en i onmen , one o non-u ban en- i onmen and one o indoo s. Resul s om an ea ly expe imen s wi h he uzzy IS can be seen on Fig. 3 and Fig. 4. We used he inpu pa- ame e s wi hou anishing poin s de ec ion.URB and NURB alues a e he uzzy IS ou pu s om ange <0,1> and ep esen s de ec ed en i onmen membe - ship. As we can see, he e we e p oblems wi h de- ec ion o e lap, so he esul s we e no usable in eal condi ions. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 258 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER Fig. 2: Gaussian inpu uzzy membe ship unc ion o p oposed sys em. 0 5 10 15 20 25 30 35 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 ou pu alue Ou doo s URB NURB Fig. 3: Fuzzy IS o en i onmen de ec ion es ou pu (Ou - doo s en i onmen ). Many con igu a ions we e es ed, bu o eal wo ld use we need mo e obus and eliable solu ion. F om hose easons we choose o ap oxima e he uzzy IS wi h a uzzy a i icial neu al ne wo k. Nex s ep was o app oxima e uzzy IS wi h usage o uzzy neu al ne wo k o classi ica ion (FNNC). 4. The Fuzzy Neu al Ne wo k o En i onmen Classi ica ion Scheme o used FNNC can be seen in Fig. 5. As men- ioned abo e, o i h ule, s h sample and j h ea u e we a e using Gaussian membe ship unc ion wi h cen e c, 0 5 10 15 20 25 30 35 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 ou pu alue Ci y URB NURB Fig. 4: Fuzzy IS o en i onmen de ec ion es ou pu (Ci y en i onmen ). which is de ined as ollows [5]: µij(xsj ) = exp −(xsj −cij)2 2σ2 ij !.(4) Fo uzzy ica ion we a e using ollowing ule, whe e Nis numbe o ea u es: αis = N Y j=1 µij(xsj ).(5) De uzzy ica ion is done wi h usage o ollowing ule: βsk = M X i=1 αiswik.(6) Weigh w ep esen ing deg ee o belonging o k h class. Finally, de e mining o ou pu class C o s h inpu is done simply by choosing he maximum: Cs= max {osk}, k = 1,2, ..., K. (7) Fo supe ised aining o he uzzy classi ica ion neu al ne wo k we can use a ie y o lea ning algo- i hms. Du ing he neu al ne wo k aining a cho- sen algo i hm should upda e he alues o membe ship unc ion cen e s and wid hs, also all weigh s on connec- ion om uzzy ica ion o de uzzy ica ion laye . Usu- ally a ian s o he SGG algo i hm is ecommended [5] o aining howe e in his case we ha e p oblem wi h a iable numbe o inpu s. Also he lea ning algo i hm mus be able o wo k online. The main p inciple o supe ised lea ning is o manual swi ching o co ec en i onmen , which causes o gene a e designa ed ou - pu . UAV now can collec da a and lea n om gi en examples. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 259 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER Fig. 5: Scheme o uzzy neu al ne wo k o en i onmen classi- ica ion. A p esen , he online lea ning algo i hm o ou pu - pose is he subjec o esea ch. Fo es ing unc ionali y o p oposed sys em we a e using empo a y app oach, which conside collec ing da a and hen make o line lea ning. Those alues a e used as ini ial weigh s o NFC. Nex s ep is o use modi ied online lea ning al- go i hm, based on e o g adien descen p inciples. 5. Conclusion and Fu u e Wo k The abili y o de ec he cha ac e o he en i onmen in which au onomous en i y ope a es should be an im- po an ea u e o u u e au onomous sys ems, espe- cially o UAVs. Usage o he uzzy classi ica ion neu al ne wo k o such de ec ion is p obably an app op ia e Fig. 6: En i onmen de ec ion applica ion sc een sho . app oach as i co esponds o he cha ac e o he in- pu pa ame e s ha ha e a high deg ee o unce ain y. Also, his app oach seems o be app op ia e o he possibili y o using he amoun o o line and online lea ning algo i hms o uzzy neu al ne wo k. Ou ecen esea ch indica es ha he me hod p e- sen ed in his a icle is applicable o p ac ical use in UAVs s abiliza ion sys ems. In he nea u u e we ex- pec he publica ion o compa a i e esul s using di - e en ypes o online o lea ning algo i hms and neu al ne wo k con igu a ions. Acknowledgmen This pape has been elabo a ed in he amewo k o he IT4Inno a ions Cen e o Excellence p ojec , eg. No. CZ.1.05/1.1.00/02.0070 suppo ed by Ope a ional P og am ’Resea ch and De elopmen o Inno a ions’ unded by S uc u al Funds o he Eu opean Union and s a e budge o he Czech Republic. We also like o hank SGS/8/2014 - Expe im- age analysis and decision making in medicine and UAS/UAV sys ems. Re e ences [1] SHAW, D. and N. BARNES. Pe spec i e ec an- gle de ec ion. In: P oceedings o he Wo kshop o he Applica ion o Compu e Vision, in conjunc- ion wi h ECCV 2006. G az: Sp inge , 2006, pp. 119–127. ISBN 978-3540338345. [2] NOVAK, D. and P. CERMAK. Ad an ages o using mul i-agen p inciples in FAIL - SAFE UAV sys em design. In: 16 h In e na ional Con- e ence on Me hods and Models in Au oma ion and Robo ics (MMAR). Miedzyzd oje: IEEE, 2011, pp. 162–167. ISBN 978-1-4577-0912-8. DOI: 10.1109/MMAR.2011.6031337. [3] PASSINO, K. M. and S. YURKOVICH. Fuzzy Con ol. 1s ed. Columbus: Addison Wesley Pub- lishing Company, 1997. ISBN 978-0201180749. [4] TAKAGI, T. and M. SUGENO. Fuzzy iden i ica- ion o sys ems and i s applica ions o modeling and con ol. Sys ems, Man and Cybe ne ics. 1985, ol. SMC-15, no. 1, pp. 116–132. ISSN 0018-9472. DOI: 10.1109/TSMC.1985.6313399. [5] DO, Q. H. and J.-F. CHEN. A Neu o-Fuzzy Ap- p oach in he Classi ica ion o S uden s’ Aca- demic Pe o mance. Compu a ional In elligence and Neu oscience. 2013, ol. 2013, no. 6, pp. 1– 7. ISSN 1687-5273. DOI: 10.1155/2013/179097. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 260 INDUSTRIAL APPLICATIONS VOLUME: 13 |NUMBER: 3 |2015 |SEPTEMBER Abou Au ho s Da id NOVAK ecei ed his M.Sc. om Facul y o Philosophy and Science, Silesian Uni e si y in 2010. His esea ch in e es s include UAV/UAS au onomous sys ems. He cu en ly wo ks owa ds his Ph.D. deg ee unde he supe ision o Doc. Ing. Pe Ce mak, Ph.D. Pe CERMAK was bo n in 1970 in Opa a, Czech Republic. He g adua ed om he VSB–Technical Uni e si y Os a a, ea ned his Ph.D. in Technical Cybe ne ics. Associa e P o es- so in he ield o Technical Cybe ne ics. In his ime in Resea ch Ins i u e o he IT4Inno a ions Cen e o Excellence, Silesian Uni e si y in Opa a and Di ec o o Robo ic labo a o y. Main in e es s in Sys em Mod- eling wi h using A i icial In elligence, Fuzzy-Neu al Ne wo ks, Robo ics, Digi al Image P ocessing and Analysis. Hobby in e es s in RC Helicop e , RC ai planes and pho og aphy o na u e. c 2015 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 261