Development of additional sensor capabilities for use in unmanned aerial vehicles under CIDIFA

Parcelas, Rafael

Development of additional sensor capabilities for use in unmanned aerial vehicles under CIDIFA

Parcelas, Rafael

Published: 2018
Publisher: Academia da Força Aérea
Repository: comum.rcaap.pt
Language: en

Abstract

Veículos aéreos não tripulados (VANTs) originalmente foram desenvolvidos para aplicações militares que, pela sua natureza não transportam operadores humanos a bordo. Esta dissertação foi realizada no seguimento do projeto Seagull. Este originou da necessidade da geração de um conhecimento situacional marítimo rigoroso e representa um dos esforços da Força Aérea Portuguesa para o cumprimento da sua missão, de acordo com os objetivos definidos pelo Conceito Estratégico de Defesa Nacional. Esta tese tem por objectivo expandir as capacidades já desenvolvidas no projeto Seagull. O trabalho apresentado foi realizado sobre a premissa que o ambiento marítimo é o principal cenário de operação. Assim sendo, um objetivo importante é a capacidade de localizar embarcações a partir dos dados visuais provenientes da aeronave. Cameras equipadas com lentes de parâmetros variáveis revelam-se mais úteis do que as com lentes de parâmetros fixos. As lentes de parâmetros variáveis permitem a um operador obter imagens com melhor qualidade através de ajustes na configuração das lentes, contudo estas não são usualmente usadas em visão computacional devido a dificuldade inerente em modelar variaçães continuas das configurações da camera. Esta tese apresenta uma metodologia para calibração de cameras com zoom variável e a sua utilização numa metodologia de localização geográfica. A metodologia de calibração de cameras apresentada é baseada na técnica de Zhang. Os modelos obtidos dos parâmetros intrínsecos da camera foram testados num equipamento com lentes de zoom numa experiência à escala reduzida do problema da localização geográfica. Utilizando a hipótese da terra-plana, o objetivo desta experiência foi validar tanto os modelos obtidos da calibração como a performance da metodologia para localização de alvos. Identificando a localização em pixeis de um alvo junto com a posição e orientação da camera, as coordenadas de um alvo são determinadas no ”referencial do mundo”. Uma experiência a escala real do problema de localização geográfica foi conduzida na Academia da Força Aérea Portuguesa, onde um conjunto de locais foram selecionados para testar a metodologia desenvolvida. O objetivo desta experiência foi avaliar o trabalho desenvolvido através da determinação das coordenadas geográficas de um alvo nâo cooperativo no solo sob condições reais. Através da comparação de resultados entre as estimativas providenciadas pela metodologia de localização geográfica proposta e as coordenadas GPS dos alvos foi possível identificar fatores e causas de erro possíveis de serem mitigadas. Os resultados obtidos das experiências realizadas revelam uma interdependência entre os parâmetros intrínsecos da camera utilizada e o seu zoom. A metodologia de localização geográfica apresenta resultados promissores para ser utilizada em ambiente marítimo. Contudo, os erros experimentais observados foram categorizados em três fatores. O primeiro fator verificado foi as limitações dos modelos obtidos da calibração da camera. O segundo fator foi a diferença de alturas entre a altitude acima do solo da camera e a altitude acima do alvo. O terceiro e último fator é a influência dos fatores ambientais no ruído da imagem e a sua sensibilidade à prespectiva da imagem para a captura do alvo.

Full text

PORTUGUESE AIR FORCE ACADEMY De elopmen o addi ional senso capabili ies o use in unmanned ae ial ehicles unde CIDIFA Ra ael Ba alha Pa celas Al e es Aluno Engenhei o Elec o écnico 138104-A Thesis o ob ain he Mas e o Science Deg ee in Elec o echnic Enginee ing Supe iso (s): P o . Dou o Rica do Ad iano Ribei o Cap . Gonçalo Cha e s San os C uz Examina ion Commi ee Chai pe son: B igadie Gene al José Augus o Nunes Vicen e Passos Mo gado Supe iso : Doc o Rica do Ad iano Ribei o Membe o he Commi ee: Doc o José An ónio da C uz Pin o Gaspa Sin a, Feb ua y 2018 ii To he Po uguese Ai Fo ce Academy... iii i Acknowledgmen s I would like o exp ess my g a i ude o he Po uguese Ai Fo ce Academy o gi ing me he oppo - uni y o pu sue my academic s udies. This p es igious ins i u ion suppo ed all o my s udies du ing he pas six yea s and, du ing his disse a ion p o ided he equipmen , ma e ial, logis ical and human condi ions, helping me in he de elopmen o his esea ch wo k. This hesis would no ha e been possible wi hou he suppo o many people. I would like o exp ess my deepes g a i ude o my supe iso P o esso Rica do Ribei o o his guidance, suppo , encou age- men , and pa ience h oughou my esea ch pe iod and expe imen al esul s analysis. His insigh and guidance ha e shaped his hesis, and i emain deeply g a e ul o all ha he has augh me. I also would like o hank my co-supe iso Cap . Gonc¸alo C uz o his cons an suppo , guidance, a ailabili y and cons uc i e sugges ions, which we e de e minan o he accomplishmen and success o he wo k p esen ed in his hesis. Finally, would like o hank my gi l iend Ana Cl´ audia Lopes and my amily, Domingos Pa celas, Am´ elia Ba alha and Jo˜ ao Dua e Pa celas, o all o hei encou agemen and suppo . i Resumo Ve´ ıculos a´ e eos n˜ ao ipulados (VANTs) o iginalmen e o am desen ol idos pa a aplicac¸ ˜ oes mili a es que, pela sua na u eza n˜ ao anspo am ope ado es humanos a bo do. Es a disse ac¸ ˜ ao oi ealizada no seguimen o do p oje o Seagull. Es e o iginou da necessidade da ge ac¸ ˜ ao de um conhecimen o si ua- cional ma ´ ı imo igo oso e ep esen a um dos es o c¸os da Fo c¸a A´ e ea Po uguesa pa a o cump imen o da sua miss˜ ao, de aco do com os obje i os de inidos pelo Concei o Es a ´ egico de De esa Nacional. Es a ese em po objec i o expandi as capacidades ja desen ol idas no p oje o Seagull. O abalho ap esen ado oi ealizado sob e a p emissa que o ambien o ma ´ ı imo ´ e o p incipal cen´ a io de ope ac¸ ˜ ao. Assim sendo, um obje i o impo an e ´ e a capacidade de localiza emba cac¸ ˜ oes a pa i dos dados isuais p o enien es da ae ona e. Came as equipadas com len es de pa ˆ ame os a i´ a eis e elam-se mais ´ u eis do que as com len es de pa ˆ ame os ixos. As len es de pa ˆ ame os a i´ a eis pe mi em a um ope ado ob e imagens com melho qualidade a a ´ es de ajus es na con igu ac¸ ˜ ao das len es, con udo es as n˜ ao s˜ ao usualmen e usadas em is˜ ao compu acional de ido a di iculdade ine en e em modela a iac¸ ˜ oes con inuas das con igu ac¸ ˜ oes da came a. Es a ese ap esen a uma me odologia pa a calib ac¸ ˜ ao de came as com zoom a i´ a el e a sua u ilizac¸ ˜ ao numa me odologia de localizac¸ ˜ ao geog ´ a ica. A me odologia de calib ac¸ ˜ ao de came as ap e- sen ada ´ e baseada na ´ ecnica de Zhang. Os modelos ob idos dos pa ˆ ame os in ´ ınsecos da came a o am es ados num equipamen o com len es de zoom numa expe iˆ encia ` a escala eduzida do p ob- lema da localizac¸ ˜ ao geog ´ a ica. U ilizando a hip´ o ese da e a-plana, o obje i o des a expe iˆ encia oi alida an o os modelos ob idos da calib ac¸ ˜ ao como a pe o mance da me odologia pa a localizac¸ ˜ ao de al os. Iden i icando a localizac¸ ˜ ao em pixeis de um al o jun o com a posic¸ ˜ ao e o ien ac¸ ˜ ao da came a, as coo denadas de um al o s˜ ao de e minadas no ” e e encial do mundo”. Uma expe iˆ encia a escala eal do p oblema de localizac¸ ˜ ao geog ´ a ica oi conduzida na Academia da Fo c¸a A´ e ea Po uguesa, onde um conjun o de locais o am selecionados pa a es a a me odologia desen ol ida. O obje i o des a expe iˆ encia oi a alia o abalho desen ol ido a a ´ es da de e minac¸ ˜ ao das coo denadas geog ´ a icas de um al o n˜ ao coope a i o no solo sob condic¸ ˜ oes eais. A a ´ es da compa ac¸ ˜ ao de esul ados en e as es ima i as p o idenciadas pela me odologia de localizac¸ ˜ ao ge- og ´ a ica p opos a e as coo denadas GPS dos al os oi poss´ ı el iden i ica a o es e causas de e o poss´ ı eis de se em mi igadas. Os esul ados ob idos das expe iˆ encias ealizadas e elam uma in e dependˆ encia en e os pa ˆ ame os in ´ ınsecos da came a u ilizada e o seu zoom. A me odologia de localizac¸ ˜ ao geog ´ a ica ap esen a e- sul ados p omisso es pa a se u ilizada em ambien e ma ´ ı imo. Con udo, os e os expe imen ais obse - ados o am ca ego izados em ˆ es a o es. O p imei o a o e i icado oi as limi ac¸ ˜ oes dos modelos ob idos da calib ac¸ ˜ ao da came a. O segundo a o oi a di e enc¸a de al u as en e a al i ude acima do solo da came a e a al i ude acima do al o. O e cei o e ul imo a o ´ e a in luˆ encia dos a o es ambien ais no u´ ıdo da imagem e a sua sensibilidade ` a p espec i a da imagem pa a a cap u a do al o. Pala as-cha e: calib ac¸ ˜ ao de cˆ ame as, localizac¸ ˜ ao geog ´ a ica, VANTs ii iii Abs ac Unmanned ae ial ehicles (UAVs) we e o iginally designed o mili a y applica ions, which by na u e, did no necessa ily equi ed human ope a o s on-boa d. This hesis was de eloped ollowing he Seagull P ojec , which ep esen s one o he Po uguese Ai Fo ce a emp s o ul ill i s mission equi emen s in compliance wi h he guidelines o he S a egic Na ional De ense Concep . De eloped by he Po uguese Ai Fo ce Academy, i o igina ed om a need o gene a e a mo e accu a e ma i ime si ua ion awa eness o he Po uguese ma i ime e i o y. This hesis aims o ex end he capabili ies p e iously de eloped in he Seagull p ojec , which one o he main in e es s was he abili y o geo-loca e iden i ied a ge essels om he isual da a o he ai c a , whe e he use o came a de ices wi h a iable-pa ame e lenses is mo e use ul han hose wi h ixed-pa ame e lenses. The a iable-pa ame e lenses enable an ope a o o ob ain be e images by adjus ing he came a’s lenses o he p esen condi ions o a scene. Howe e , a iable pa ame e lenses a e no commonly used in compu e ision because hey a e di icul o model o con inuous a ia ions o he lenses con igu a ion. This hesis p esen s a came a calib a ion me hodology o de ices wi h a iable zoom and i s em- ploymen in a ision-based a ge geo-loca ion me hod. The came a calib a ion me hod p esen ed in his wo k is based on Zhang’s echnique o came a calib a ion, we e a model o he came a in insic pa ame e s is ob ained by explo ing he in e dependence be ween he came a zoom and i s pa ame e s. A small-scale expe imen o he geo-loca ion p oblem was conduc ed in o de o alida e bo h he came a in insic pa ame e s models and he p oposed geo-loca ion me hodology. In his expe imen by iden i ying he pixel loca ion o a a ge in an image and he measu emen s o he came a posi ion and pose, he wo ld coo dina es o he a ge a e de e mined. Using he same me hodology a ull-scale expe imen o he geo-loca ion p oblem was de ised. In his a se o loca ion ac oss he Po uguese Ai Fo ce Academy Campus we e selec ed o employ he me hodology de ised. The objec i e o his expe imen was o assess he accu acy o he wo k p esen ed in his hesis unde eal-wo ld condi ions and he abili y o localize an uncoope a i e g ound a ge using he UAV ision senso . By compa ing he esul s o he a ge es ima ed posi ion ob ained om he geo-loca ion me hodology wi h he ac ual GPS coo dina es o he selec ed loca ions o iden i y ac o s and/o e o sou ces whe e iden i ied, which can be mi iga ed. The geo-loca ion me hodology de eloped is a p oo o concep ha empi ically as shown encou aging p omises o be employed in a ma i ime en i onmen . The e o s e i ied in he expe imen al p ocedu es we e mainly caused by h ee ac o s. Fi s he p ecision o he came a a iable-pa ame e s models ob ained, hese a e es ima es o he eal beha io o he lenses and as such a e conside ed empi ical app oxima ions. Second he heigh di e ence be ween he came a al i ude abo e g ound and al i ude abo e a ge , which is linked o he la -ea h hypo hesis, as such e o s can occu om he ela i e heigh di e ence. Thi d he en i onmen al ac o s which induce image noise, which wi h he inc easing obliqui y o he cap u ed image his e o s can be exace ba ed as demons a ed in his wo k. Keywo ds: came a calib a ion; geo-loca ion; a iable-pa ame e lenses; UAVs. ix 4.11 Tangen ial dis o ion pa ame e p1 h ough he zoom ange. . . . . . . . . . . . . . . . . . 55 4.12 Tangen ial dis o ion pa ame e p2 h ough he zoom ange. . . . . . . . . . . . . . . . . . 55 4.13 Focal leng h polynomial models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.14 Radial dis o ion pa ame e s k1polynomialmodels....................... 59 4.15Top iew.............................................. 60 4.16Side iew ............................................. 60 4.17 Labo a o y expe imen al se up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.18 Expe imen al so wa e diag am. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.19 Pe spec i e cen e poin de e mina ion s eps. . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.20 Pe spec i e cen e poin ma kings in he came a moun . . . . . . . . . . . . . . . . . . . . 63 4.21 SEP so wa e a chi ec u e wi h geo-loca ion module. . . . . . . . . . . . . . . . . . . . . . 68 4.22Geo-loca ionmodule. ...................................... 69 4.23 Tes a ge o he ull scale expe imen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.24 Came a se up in he wa e owe o he Po uguese Ai Fo ce Academy. . . . . . . . . . . 71 4.25 Tes pa ame e s acco ding o a ge loca ions. . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.26 Came a se up in he wa e owe o he Po uguese Ai Fo ce Academy. . . . . . . . . . . 72 4.27 Image noise sample in loca ion 10................................ 73 4.28Tes scena iosp oposed. .................................... 74 4.29 E o o each loca ion acco ding he p oposed scena io o analysis. . . . . . . . . . . . . . 75 4.30 Geo-loca ion e o as a unc ion o he il angle. . . . . . . . . . . . . . . . . . . . . . . . . 77 B.1 Geo-loca ion e o o Scena io 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 B.2 Geo-loca ion e o o Scena io 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 B.3 Geo-loca ion e o o Scena io 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 C.1 Assuming a la ea h model, he g ound dis ance o a came a image can be compu ed by knowledge o he il angle λ he lens e ical and ho izon al ield o iew espec i ely τ and η,and heal i udeh. .................................... 91 x i Glossa y AGL Abo e g ound le el. AIS Au oma ic Iden i ica ion Sys em. CIAFA Cen o de In es igac¸ ˜ ao da Academia da Fo c¸a A´ e ea. DEM Digi al Ele a ion Model. DIAC Dual image o he absolu e conic. ECEF Ea h Cen e ed Ea h Fixed. ETP Payload G ound S a ion. GPS Global Posi ioning Sys em. ISR/IST Ins i u e o Sys ems and Robo ics/Ins i u o Su- pe io T´ ecnico. Ma link Mic o Ai Vehicle Communica ion P o ocol. ROS Robo Ope a ing Sys em. SEC2 On-boa d Command and Con ol Uni . SEP On-boa d Payload Con ol Uni . SFM S uc u e om Mo ion. UAV Unmanned Ai c a Vehicle. WSG84 Wo ld Geode ic Sys em 1984. x ii x iii Chap e 1 In oduc ion 1.1 Mo i a ion The Po uguese con inen al e i o y is loca ed in he occiden al pe iphe y o Eu ope bo de ing Spain, co e ing an a ea o 89 000 Km2. The Po uguese e i o y also includes he a chipelagos o Madei a and Ac¸o es loca ed in he No h A lan ic wi h a combined 3000 Km2a ea [1]. In o al, he coun y occupies a land a ea o 92 000 Km2 Po ugal has small land mass e i o y wi h sca ce na u al esou ces. Howe e , conside ing i s ma - i ime e i o y, i is one o he la ges ma i ime na ions in he wo ld. Se e al o he mos impo an ma i ime shipping ou es in he A lan ic ocean c oss he Po uguese na ional wa e s, due i s unique ma - i ime geopoli ical posi ion o Po ugal he na ion aces mul iple challenges in main aining he con ol and su eillance o he na ional ma i ime space. The conside able size o he Po uguese ma i ime e i o y, o igina es unique su eillance challenges which he Po uguese S a e is con on ed. The main challenges a e ela ed wi h he ul illmen o i s in e - na ional obliga ions and esponsibili ies in i s so e eign e i o y bo h in he ma i ime exclusi e economic zone and he sea ch and escue a ea. These obliga ions include: o e sigh and con ol o ma i ime ela ed ac i i ies, en i onmen al con ol, sea ch and escue o shipw ecked essels and medical suppo o hei c ews and ope a ions in e i o ial wa e s. The Po uguese S a egic Na ional De ense Concep [2] es ablishes he guidelines ha he go e n- men mus add ess o ul ill he esponsibili ies in i s so e eign ma i ime e i o y. The de elopmen o hese guidelines in o policies alls unde ins i u ions wi h p ope ju isdic ion. As such, he ma i ime e - i o y secu i y, sa e y and su eillance mus be in eg a ed in a b oad scale pe spec i e wi h all pa ne s in ol ed. The S a egic Na ional De ense Concep [2] es ablishes ha he capabili y o p ope su eillance and con ol o he ma i ime space mus be main ained, wi h ocus on employing he mos cos e ec i e means. This documen also emphasizes he op imiza ion and coo dina ion be ween he na ional ins i u- ions; he esponsible use o comba means in c iminal ac i i ies; and he p e en ion and p epa a ion o p o ocols in o de o espond o na u al disas e s h ough an e ec i e obse a ion and ale sys em [2]. 1 Up un il now he Po uguese Ai Fo ce has been commi ed esponding o hese guidelines by using manned ai pla o ms. Howe e , unmanned ai c a pla o ms (in con as o manned ai c a ), cha ac- e ized by hei low cos s in bo h main enance and ope a ion, eme ge in his con ex as an appealing al e na i e o educe bo h he man powe employed; he pla o ms main enance cos s; and he ai c a acquisi ion cos s. The Seagull p ojec was conduc ed o es and alida e solu ions o mee immedia e and u u e needs o he managemen and in o ma ion ga he ing o he ma i ime si ua ion [3]. The Seagull p ojec s a ed in July 2013 and ended in July 2015. The p ojec was led by C i ical So wa e wi h he pa icipa ion o o he pa ne s such as: CIAFA (Cen o de In es igac¸ ˜ ao da Academia da Fo c¸a A´ e ea), ISR/IST (Ins i u e o Sys ems and Robo ics/Ins i u o Supe io T´ ecnico), FEUP (Faculdade de Engenha ia da Uni e sidade do Po o) and he Cen o de In es igac¸ ˜ ao Na al. The Seagull p ojec aimed o de elop an in elligen in eg a ed sys em o UAV’s and i s pu pose was o gene a e a mo e accu a e ma i ime si ua ion awa eness knowledge base. Ma i ime si ua ion awa eness is a la ge opic and he Seagull p ojec add essed some o he poin s p esen in he S a egic Na ional De ense Concep wi h conc e e solu ions such as: a ge de ec ion, classi ica ion and acking, iden i ica ion o suspicious essel beha io and moni o ing en i onmen al indica o s [3]. In o de o accomplish hese objec i es he ai bo ne pla o m was equipped wi h an a ay o senso s and came as in a mul i ude o ligh spec ums (in a ed, mul i-spec al, isible ligh ). The dimension o his p ojec p o ided an unique oppo uni y o he de elopmen o ision based algo i hms an so wa e a chi ec u e design [3]. This disse a ion is aligned wi h he con inui y o he p ojec . By wo king wi hin he de eloped so wa e and wi h he al eady employed senso s in an a emp o expand and p ope ly alida e he concep s suppo ing he ex ended capabili ies o he ai bo ne pla o m. 1.2 P oblem Fo mula ion One o he p ima y applica ion o UAVs has been sho ange ae ial econnaissance, whe e hey ha e been p ima ily esponsible o collec ing in o ma ion and elaying i o an ex e nal ope a o on he g ound. The employmen o UAV pla o ms in ope a ional scena ios is an inc easing eali y and he de elopmen o he Seagull p ojec ep esen s one o he Po uguese Ai Fo ce a emp s o de elop new echnologies and capabili ies usable in he o ganiza ion daily ope a ions. One impo an senso ha many UAVs ha e is a came a de ise. Wi h a came a, a ehicle can ob ain a a lo o in o ma ion abou i s su oundings and possible a ge s o in e es . In he Seagull p ojec , one o he mos impo an opics de eloped was he capabili y o de ec ing a ge s in ma i ime su eillance en i onmen , speci ically he de ec ion o essels using an op ical colo senso moun ed on an ai bo ne pla o m. This opic ep esen s a e y complex p oblem due o he ma i ime en i onmen whe e he de ec ion algo i hm mus be able o deal wi h a iabili y o he essels appea ance, wa e c es s, p esence o sun gla e, change in he ligh ing condi ions, clouds and pla o m mo emen [4]. The ex ension o his capabili y is he geo-loca ion o essels once hey a e iden i ied in an im- age. Geo-loca ion is de ined as he iden i ica ion o es ima ion o he eal-wo ld geog aphic loca ion o 2 an objec . This in o ma ion allows he na ional ins i u ions o ake immedia e ac ions in ela ed o he su eillance and con ol o he ma i ime scena io. In his wo k, objec ecogni ion was no a majo conce n, a he he ocus was o ind a a ge eal wo ld coo dina es wi h a a iable-pa ame e s lenses came a in o de o es a geo-loca ion p oo o con- cep me hodology. The solu ion p oposed was designed in o de o wo k as independen ly as possible om he objec ecogni ion algo i hm and o he sys ems o de ec ion. 1.2.1 P oblem Desc ip ion The p oblem p esen ed in his hesis consis s o ob aining eal wo ld coo dina es (la i ude, longi ude and al i ude) o a a ge iden i ied in a image. To accomplish his goal, se e al challenges a ise as esul o i s complexi y. The i s challenge is he ma hema ical cha ac e iza ion and modeling o he came a equipmen , whe e a calib a ion p ocedu e mus be employed. The second challenge is e alua ing he co ec ness o he de i ed models om calib a ion. Once a a ge is iden i ied in an image and he came a de ice ha cap u ed such image is co ec ly modeled he hi d challenge is he dep h o ange es ima ion o he a ge ega ding he ai bo ne pla o m. Monocula came as such as he one used in his wo k, do no p o ide his in o ma ion he e o e, an al e na i e solu ion mus be employed. All o hese challenges a e pa o he ision based geo-loca ion p oblem p oposed in his hesis. In o de o o e come hem, se e al objec i es we e ou lined. 1.2.2 Objec i es The esea ch p esen ed in his hesis a emp s o ul ill he ollowing 5 objec i es: 1. The de elopmen o a me hod o calib a e came as wi h a iable se ings; 2. Modula ion o came a in insic pa ame e s by he zoom o he came a’s lens; 3. The de elopmen o a geo-loca ion algo i hm; 4. The analysis o he came a calib a ion model and geo-loca ion algo i hm in expe imen al and eal wo ld scena io; 5. In eg a ion wi h he so wa e de eloped in he Seagull So wa e A chi ec u e; 3 1.3 Li e a u e Re iew 1.3.1 Came a Calib a ion A came a is a de ice designed o cap u ing a pho og aphic image o eco ding a ideo (s eam o images). The calib a ion o such de ices is a p ocess om which i is possible o ob ain he came a in insic pa ame e s ( ocal leng h, lens dis o ion, op ical image cen e , model pa ame e s) and ex insic pa ame e s (loca ion and pose). The came a in insic pa ame e s a e unique o each and e e y single de ice and do no change when he came a is mo ed and i s lens con igu a ion is no al e ed. The s a e-o - he-a came a calib a ion echniques sol e he in insic and ex insic came a pa am- e e s using a physical calib a ion pa e n (such as a plana checke boa d). Se e al images om he pa e n a e cap u ed in di e en poses, and he came a model pa ame e s a e op imized in o de o co espond image coo dina es o known coo dina es o he calib a ion pa e n. The s a e-o - he-a me hods a e Tsai’s [5], Heikkil¨ a’s [6] and Zhang’s [7] me hods. These me hods mainly di e in how he came a in insic pa ame e s a e de e mined and how he he calib a ion pa e ns a e use . Sal i [8] aces a compa ison o he calib a ion me hods de eloped be ween 1982 and 1998 and Tsai’s me hod, shows he bes pe o mance despi e equi ing a highe p ecision in he inpu da a. The widely used me hod p oposed by Tsai, which is based on a wo-s ep echnique modeling only adial lens dis o ion a e conside ed [5]. This me hod ep esen s a con en ional app oach ha is based on he adial alignmen cons ain and equi es accu a e 3D wo ld- e e enced coo dina es measu emen wi h espec o a ixed e e ence (plana pa e n). Heikkil¨ a’s me hod, is also wo ld- e e ence based, al hough no included in Sal i’s wo k, employs he mo e gene al di ec linea ans o ma ion echnique by making use o he p io knowledge o in insic pa ame e s. I also in ol es a mo e comple e came a model ega ding he came a lenses dis o ions[6]. Zhang’s me hod ep esen s a new leap in came a calib a ion echniques. This me hod is a di e en special case o Heikkil¨ a’s o mula ion, whe e i combines he bene i s o wo ld- e e ence based and au o-calib a ion app oaches. This enables he linea es ima ion o all in insic pa ame e s. Au o-calib a ion o sel -calib a ion app oaches de e mine came a pa ame e s di ec ly om mul iple non-calib a ed iews o a scene by ma ching co esponding ea u es be ween iews, despi e unknown came a mo ion and e en changes in some o he in insic pa ame e s. This me hod in oduces a lexible app oach in which ei he he came a o he plana pa e n can be mo ed eely and he calib a ion p ocedu e is easily epea able wi hou edoing any measu emen s. By examining bo h Tsai’s and Zhang’s me hods, on he one hand Tsai’s e eals o be mo e p ecise, i he inpu da a is no co up ed howe e , e y accu a e measu emen s o he calib a ion pa e n a e needed. On he o he hand Zhang’s me hod does no equi e nei he special measu emen s o he plana pa e n no om he expe imen al se up. Acco ding o Sun [9] be e esul s a e ob ain wi h Zhang’s me hod and he algo i hm sensibili y o e o in he measu emen s can be imp o ed by inc easing he numbe o co ne s in he checke pa e n used. Taking in accoun he esul s om Sal i [8] and Sun [9], Zhang’s me hod is s a ed as a s a e-o - he- 4 a e e ence o came a calib a ion. A e sion o his me hod uses a pa e n o ci cles o a oid e o s in he coo dina es measu emen s in he pa e n al hough he e is no a consensus i i imp o es he esul s om a checke boa d pa e n. 1.3.2 Geo-loca ion Vision-based geo-loca ion is an ex ensi ely explo ed opic and has been a well documen ed p oblem. F om he se e al app oaches de eloped in he academic communi y he e a e ou di e en ypes which a e impo an o e iew[10]: 1. The Pape -Map-Based Me hod 2. Line-o -sigh il e ing 3. S uc u e F om Mo ion (SFM) 4. Geo- e e enced image y When using ai bo ne pla o ms o UAV’s he es ima ion o he a ge g ound al i ude o he a ge ange o he ai c a , is an essen ial pa o sol ing he g ound objec geo-loca ion p oblem. The pay- load es ic ions in his ype o pla o ms ep esen s a majo p oblem which can make his ask challeng- ing. Gi en he ac ha his ype o pla o ms canno ca y a hea y ange senso capable o measu ing dis ances o hund eds o me e s, mos o he me hods ely on assuming a la ea h model. This as- sump ion implies ha he e is no heigh di e ence a a g ound le el be ween he ai c a and he a ge loca ions. In a ma i ime en i onmen his is a ela i ely sa e assump ion because ei he a GPS ecei e o a calib a ed ba ome ic senso espec i ely assume a mean sea le el al i ude o an a e age sea le el p essu e o ope a e. In his wo k he mean sea le el co esponds o he g ound al i ude o he a ge . The Pape -Map-Based Me hod This is mos s aigh o wa d a ge ing me hod and, assumes ha he a ge is cen e ed in a came a image, and he a ge geog aphic loca ion is de e mined as he in e sec ion be ween he came a poin ing axis and he g ound plane[10]. The came a loca ion and a i ude mus be known and he a ge mus be in line o sigh . In his me hod a Digi al Ele a ion Model (DEM) o he e ain o he la wo ld assump ion hypo hesis mus be employed o he calcula ion. This me hod accu acy elies s ongly on he p ecision o he measu emen s o he ai c a pi ch, oll and heading, and he came a il and pan as well as he came a calib a ion. E o s can also a ise om he inaccu acy o he DEM as well as senso bias and noise. Typical senso s ins alled on mos UAV pla o ms ha e low accu acy due o limi ed payload capabili ies. This can o igina e e o s om a single a ge measu emen sample om 20-40 me e s wi h he pla o m lying a 100 me e s [10, 11]. The sys ema ic e o s e i ied a e usually biases in he es ima ion o he came a o ien a ion. They can be he consequence o biases in he UAV a i ude and o ien a ion, poo calib a ion o he magne ic compass senso o he misalignmen o he se e al ine ial measu emen uni s in ei he he came a uni 5 o he ai c a . These e o s can be mi iga ed my selec ing a ligh pa h ha enables mo e accu a e a ge posi ion es ima es. In [11] i is shown ha by lying in a ci cula loi e ing pa e n and applying a leas squa e il e o mul iple obse a ions o he a ge , he e o s om he geo-loca ion dec ease om 20-40 me e s o less han 5 me e s. The la wo ld assump ion was employed in he expe imen and he esul s we e ob ained using o line ligh da a. In [12], esul s show an app oxima e 4 me e s e o o he a ge loca ion, These esul s we e ob ained by using a combina ion o a DEM model o he ligh a ea and a a ge ec o owa ds a a ge o in e es in he image ame. Line-o -sigh Fil e ing This me hod is simila o he Pape -Map-Based Me hod howe e , i does no ely in a DEM o he la ea h hypo hesis. The idea behind his me hod is ei he o ind he in e sec ion poin o mul iple ays om mul iple obse a ions passing h ough he came a and he a ge o using s e eo came as. This in e sec ion iden i ies he 3D coo dina es o he a ge . Due o senso bias he ays migh ne e p oduce a poin o in e sec ion and he e o e he p oblem is o iden i y he closes poin be ween such ays. A Kalman il e can be used o emo e possible senso noise and bias bu mul iple samples o he a ge a e s ill equi ed. In [10], his me hod is implemen ed and p oduces a a ge ing accu acy o 10 me e s a an al i ude o 100 me e s. These esul s we e ob ained by using a Ra en-B pla o m. S uc u e F om Mo ion (SFM) The p incipal sou ce o unce ain y in he line-o -sigh il e ing me hod is likely o be an e o in he supposed o ien a ion o he ehicle and hus he came a. As an example, a an al i ude o 100 me e s wi h a came a pi ched down 30o, a wind-induced 1.5opi ch e o p oduces 10 me e o a ge ing e o , and e en i his is ze o-mean noise, i will damp slowly o e many ames o il e ing [10]. This me hod elies on he 2D image coo dina es o mul iple a ge s seen in images aken om mul iple came a posi- ions and sol es o he 3D loca ions o each a ge . The me hod can be simpli ied and use ewe a ge s and/o images i some o he 3D posi ions o o ien a ions a e gi en as inpu [10]. The ma hema ical p emise o SFM is desc ibed in de ail in[13, 14]. Geo- e e enced image y This me hod consis s in egis e ing a on boa d ideo and compa e i o geo- e e enced images, hus ex ac ing he a ge coo dina es di ec ly om he e e enced image. This me hod is independen o he came a pose accu acy. The senso in o ma ion on boa d can be used o a i s es ima e o he a ge posi ion, bu he inal posi ion is compu ed a e he egis a ion p ocess. This egis a ion p ocess is a limi a ion o his echnique, as well as i s dependence on he a ailabili y o a e e ence image y o a speci ic egion. On 6 he o he hand, he apid de elopmen o image y ools such as Google Ea h makes his me hod e y p omising. These me hods ha e been es ed in [15, 16] on la ge pla o ms wi h ex emely accu a e ins umen a- ion and ha e demons a ed a high deg ee o accu acy. Howe e , hese me hods ha e ye o be es ed on smalle pla o ms wi h hei ela i ely inaccu a e ins umen a ion. Fu he mo e, hese me hods equi e a known map o he wo ld wi h e e ence image y p ecisely aligned o geo-coo dina es. This me hod and his cons ain a e no de ailed in he p esen documen . 1.4 P oposed App oach The p oposed app oach in his hesis o he geo-loca ion o essels in a ma i ime en i onmen is sepa a ed in o ou di e en s eps. The i s s ep is he cha ac e iza ion o a came a in insic pa ame e s beha io in i s zoom ange. To accomplish his he OpenC ool box was used, which elies on Zang’s [7] came a calib a ion echnique, in which he came a lenses con igu a ion a e no al e ed du ing he p ocedu e. In his hesis he used op ical senso is a gimbal came a, u he de ailed in he nex chap e . Gimbal came as inhe en ly can change hei se ings and he e o e i is p oposed a me hod o calib a e he came a by changing he lenses con igu a ion and de e mining a he in insic pa ame e s each ime a change occu s. This modeling app oach p o ides a way o de ine he came a pa ame e s du ing i s ope a ion. The second s ep is o implemen a geo-loca ion algo i hm based on he Pape -Map-Based me hod wi h he assump ion o he la wo ld model. This ep esen s he mos easonable solu ion, due o he scena io o ope a ion (ma i ime en i onmen ), ha ing a accu a e DEM o applying he Geo-Re e enced Image y me hod e eals o be imp ac ical because o he en i onmen e e changing na u e and he impossibili y o mapping he sea su ace accu a ely. The S uc u e F om Mo ion me hod is no sui able, because his me hod expec s ha he ai bo ne pla o m lies a a conside able a iable dis ances om he a ge essel. In his si ua ion being able o acqui e se e al images o a a ge in se e al signi ican ly di e en posi ions is no possible. In he con ex o his wo k, ob aining a geo-loca ion solu ion based in a single sampled image is mo e ele an . The hi d s ep is o me ge Zang’s came a calib a ion me hodology and he geo-loca ion Pape -Map- Based me hod in labo a o y simula ions wi h a ge s in known loca ions and compa e esul s wi h he loca ions p o ided by he p oposed me hodology p o iding an assessmen o he concep de ised in his documen . The ou h and inal s ep is a ull scene y simula ion in o de o alida e ha he designed algo i hm and came a calib a ion p o ide a desi able solu ion o he p oposed p oblem. This simula ion is con- duc ed unde he expec ed ci cums ances o he ai c a ope a ion, howe e some ligh pa ame e s o his expe imen we e pu posely main ained cons an in o de o ensu e a con olled scena io. 7 In p ac ice his ype o communica ion can be seen as a da a bus whe e p og ams subsc ibed o a opic can ead published messages and publishe p og ams can w i e messages o he opics. This mechanism o communica ion is anspa en o all compu e s in a ne wo k and p o ides he backbone o he communica ion be ween he SEC2 and SEP compu e s, elimina ing he need o any o he communica ion mechanism. The E he ne connec ion be ween SEC2 and SEP is p edic able and e icien . The ROS communica ion p o ocol p o ides a solu ion ha is bo h e sa ile and allows he de elopmen and in eg a ion o o he p ocesses in he a chi ec u e. The ROS middlewa e has wo o he majo ea u es: he osbag and he abili y o in eg a e mul iple compu e languages. The i s allows he sa ing o ROS messages in memo y du ing he execu ion o a mission. La e his messages can be eplayed and used o ec ea e he mission and s udied o access he p ocesses pe o mance and he ou pu esul s [19]. The second allows o in eg a e mul iple compu e languages. As long as hey sha e in o ma ion abou he messages s uc u e, mul iple p og ams o p ocesses in di e en languages can communica e. 2.1.4 On-Boa d Command and Con ol Sys em The on-boa d Command and Con ol Sys em (SEC2) uses a Linux Ope a ing Sys em. I con ains a d i e esponsible o he in e ac ions wi h he au opilo , communica ion elay and con ol supe iso logic uni s all connec ed ia ROS middlewa e, as p esen ed in Figu e 2.6. Figu e 2.6: On-Boa d Command and Con ol A chi ec u e (adap ed om Figu e 6 in [19]). The Au opilo D i e connec s he Piccolo au opilo and he SEC2 sys em. I is he only componen which di ec ly communica es wi h he Piccolo au opilo wi h knowledge o i s speci ic Applica ion P o- 14 g amming In e ace. By using a single so wa e componen o communica e be ween he Piccolo and he SEC2 he on-boa d so wa e is independen o he au opilo d i e . In case o modi ica ions in he so wa e a chi ec u e he only componen o eplace would be he Piccolo uni . This modula agg ega- ion allows o a single de ice o p o ide na iga ion eleme y (e.g. GPS posi ioning, ai c a heading) and o ecei e o send da a ough he ”payload s eam” message made a ailable by he Piccolo. The Communica ion Relay module, is asked wi h managing he communica ion logic o and om he UAV pla o m. The module ensu es ha all communica ion needs o he pla o m a e mee (in cases whe e he Piccolo does no p o ide) speci ically he insu ance o message deli e ies, he sending o la ge da a packages and he co ec ion o he da a. This module ecei es and s eamlines he in o ma- ion as an a ay o by es being comple ely agnos ic in ega d o he messages con en s. This ype o communica ion is pa icula ly use ul due o i s gene aliza ion capabili y whe e any kind o da a can be ansmi ed wi hou e ealing nei he he channel cha ac e is ics no he p o ocol used o communica e. When a message which equi es acknowledgmen o ecep ion is sen , he module uses a mes- sage iden i ie o check i he message is ecei ed by wai ing o a acknowledgmen message wi h ha same iden i ie . In cases whe e he acknowledgmen is no ecei ed i esends he message. Due o his me hod, i is possible ha a same message is ecei ed se e al imes. In his case, he iden i ie p e iously sen ensu es ha he message is ead and only in e p e ed once. The Con ol Supe iso module is esponsible o he ai c a con ol. Unde no mal condi ions, all ai c a con ol is done ia he Piccolo module. Howe e , unde wo condi ions he Con ol Supe iso assumes con ol o he ai c a : when he Sense and A oid (S&A) module wa ns o a possible collision and when a de ec ed a ge is engaged in pu sue. The Ta ge T acking module uses he a ge and he ai c a posi ion p o ided espec i ely by he SEP and Au opilo D i e o ensu e he ai c a lies in o a ci cula loi e ing pa e n a ound he a ge . The Sense and A oid (S&A) module wa ns o possible collisions wi h neighbo ing ai c a pla o ms p esen in he a ea o ope a ion and ins uc s commands o e ou e he ai c a in case o collision. Two e ou ing s a egies a e p o ided, ei he by non linea con ol o by a mixed app oach based in coope a i e/ad e sa y game heo y beha io om pla o ms in ou e o a possible collision. When a possible collision is de ec ed he collision a oidance algo i hm p o ides he bes na iga ion solu ion and ale s he Con ol Supe iso module. The la e au ho izes a high p io i y na iga ion decision o a oid collisions. 15 2.1.5 On-Boa d Payload Sys em The on-boa d compu e sys em uses a Linux ope a ing sys em. I con ains he Image Acquisi ion and Au oma ic Iden i ica ion Sys em (AIS) d i e , Seagull Manage , De ec ion Module and Seagull Ac ua o s modules all connec ed ia he ROS middlewa e as p esen ed in Figu e 2.7. Figu e 2.7: On-Boa d Payload A chi ec u e (adap ed om Figu e 10 in [19]). The Image Acquisi ion modules a e esponsible o da a acquisi ion om he op ical senso s em- ployed in he mission. Fo each came a de ice in he payload wi h suppo o Video4Linux (a ailable in he sys ems ke nel) and wi h he ROS OpenC , an image message is pe iodically published in speci ic ROS opics dedica ed o each senso . This allows he De ec ion module o subsc ibe o he dedica ed opics o access he images The Au oma ic Iden i ica ion Sys em (AIS) is used by ma i ime pla o ms, moni o ing se ices in o de o loca e pla o ms. This sys em is manda o y by law o be p esen in essels bigge hen 15 me e s long and ansmi s in o ma ion ega ding he essel iden i ica ion, posi ion, heading and eloci y. The da a a e ansmission a ies acco dingly o he essel eloci y aking anywhe e om 20 seconds o 3 minu es and he ma i ime moni o ing se ices ga he he in o ma ion o all essels sailing along he cos o ia sa elli e. The AIS D i e equipped in he Seagull a chi ec u e enables he UAV pla o m o ecei e he same in o ma ion ha he moni o ing se ices (in he scena io o ope a ion o he ai c a ) and communica es i o he SEP uni . The Seagull Manage is he module esponsible o synch onizing he so wa e communica ion be- ween he SEP and he ETP. The communica ed da a is hen sen o he Communica ion Relay module in SEC2 which allows gene ic communica ion in he o ma o by e a ays. The Seagull Manage will encode/decode he by e a ays and sen o ecei e da a acco dingly o he Ma link (Mic o Ai Vehicle 16 Communica ion P o ocol). This p o ocol allows he easy s uc u ing o messages and au oma ic ea u es o encode and decode messages. Some o he p incipal in e ac ions be ween he Seagull Manage and he ETP a e: •Mission planning: The ETP will communica e a mission plan o he Seagull Manage wi h he mission ype and a ea o ope a ion. The Seagull Manage uses his in o ma ion o con igu e he De ec ion module acco dingly; •De ec ion no i ica ion: The Seagull Manage sends a no i ica ion o he ETP each ime a a ge is de ec ed by he De ec ion Module; •O de o (o no o) ollow a a ge : The ETP can eques SEP he ac i a ion o deac i a ion o he a ge ollowing ea u e. By execu ing his eques he Seagull Manage will ac i a e/deac i a e such ea u e in he Con ol Supe iso o he SEC2; •Image Reques : The ETP can eques SEP o a speci ic a ge image. The Seagull Manage will edi ec ha eques o he De ec ion Module. Once an image is acqui ed i is sen back o he ETP. 2.1.6 De ec ion Module The De ec ion module in eg a es all he compu e ision, il e ing, es ima ion and decision algo i hms. The module is i sel an agg ega e o o he sub-modules as p esen ed in Figu e 2.8 and is implemen ed in C++. Each in e nal sub-module is implemen ed as a class. This a chi ec u e has been chosen due o pe o mance issues. Some o his pe o mance issues a e ela ed wi h he ansmission o images as a ROS message, which can be di icul p ocess pa icula ly i mul iple messages a e ansmi ed. P oblems ela ed o message lagging can occu ei he om he publishe o subsc ibe , w ong message encoding om he pa s in communica ion and memo y usage a e jus some examples o common issues, which a e exace ba ed i mul iple images and/o came as a e used. The Acquisi ion sub-model ecei es images om he se e al came a d i e s and is esponsible o he co ec ing he sampling o images, we e i does some p e-p ocessing o he ecei ed images (e.g. image o ma con e sion). The eleme y da a can be ob ained om se e al sou ces ei he om he au opilo uni o he TASE 150 gimbal. Due o he ac ha he De ec ion module ecei es di e en in o ma ion a di e en imes ins ances om di e en sou ces, he Teleme y Da a Agg ega ion sub-module analyzes all o his in- o ma ion and synch onizes i wi h he images ecei ed o ob ain a s eam o images and espec i e eleme y a he ime o he cap u e. The Vision sub-module employs all a ge de ec ion algo i hms. This sub-module uses he images collec ed and he de ec ion con igu a ion pa ame e s co esponden wi h he mission ype. I p o ides desc ip o s o he a ge s iden i ied, including no jus he dimension and posi ion o he a ge essels in he image plane bu also a quali y quan i ie pa ame e o he de ec ion ound. 17 Figu e 2.8: De ec ion Module (adap ed om Figu e 29 in [18]). 18 The Fil e ing and Associa ion sub-module is asked wi h wo ope a ions. Fi s i il e s he a ge ’s po- si ion, helping he de ec ion o a ge s in in e mi en da a and inc eases he de ec ion quali y h oughou he mission, consequen ly, inc easing he de ec ion quali y quan i ie o consis en ly de ec ed a ge s. Second, i de e mines which de ec ion co espond o each a ge essel. This associa ion ask is e- qui ed because he Vision sub-module e u ns a bulk o de ec ed a ge s wi hou any in o ma ion o which a ge essel o igina e each de ec ion. The Objec Da a Base sub-module s o es he loca ion and he images o p e iously de ec ed a ge s. This in o ma ion is upda ed wi h he da a ob ained om he Fil e ing and Associa ion module. Whene e new a ge a e de ec ed, a new en y is added o he da a base. The Decision sub-module ecei es all o he in o ma ion and de e mines i he a ge should be pu - sued o no , acco ding o he cu en mission equi emen s. The Decision module can use he loca ion da a om he AIS da abase by compa ing i wi h he a ge s posi ions. This module sends he equi ed in o ma ion o con olling he gimbal came a h ough he espec i e con ol module. The ou pu o he Decision sub-module is a lis o de ec ed a ge s and hei ele ance o he mission. The AIS Da abase module compiles he da a ansmi ed om he essels. Because his da a has a slow upda e a e i is necessa y o s o e i o la e use. The TASE Con ol sub-module ensu es he mo emen s pan, il and zoom o he TASE150 gimbal as long as his payload equipmen is p esen . This sub-module calcula es he pa ame e s necessa y o he ope a ion o he Seagull Ac ua o s module and publishes hem in he a ROS opic “ ase command in”. 19 2.1.7 Seagull Ac ua o s The module Seagull Ac ua o s con ols he TASE 150 gimbal. This is a payload module wi h ac i e con ol o e he TASE 150 pan, il and zoom, allowing he a ge objec o be ollowed. The pan, il and zoom o he came a can be con olled manually o au oma ically main aining ixed o a ion a es o angles, keeping a a ge objec in image. This module is di ided in o wo sub-modules: TASE D i e and TASE Comms. Figu e 2.9: Comple e Seagull Sys ems (Figu e 32 in [18]). The TASE D i e sub-module manages all he communica ion wi h he TASE Comms. The aw da a is ecei ed in his node, decoded and agg ega ed in he ROS messages “TASETeleme y”, “TASEAn- gles” o “TASERa es”. The TASE Con ol sub-module in he De ec ion module p esen ed p e iously is esponsible o publishing he ROS message ”TASECommand”. The TASE Comms sub-module manages all communica ion be ween he TASE, he Au opilo D i e module and he SEP. The connec ion o he Au opilo D i e module enables a g ound ope a o o manu- ally con ol he TASE150. The connec ion o he SEP allows he De ec ion module o ope a e he gimbal au oma ically. This wo connec ions a e mu ually exclusi e. 2.2 TASE Fea u es Du ing he expe imen al p ocedu es p esen ed in his disse a ion, u he desc ibed in la e chap- e s, a new module named TASE Fea u es was inse ed in o he Seagull so wa e a chi ec u e. This module is used o he expe imen al p ocedu es only wi h he objec i e o isola ing he gimbal con ol pa o he emaining componen s o he Seagull p ojec so wa e (ei he he De ec ion module o he au opilo ). The module Tase ea u es ensu es a mo e di ec con ol o he TASE 150 gimbal, whe e he se e al con igu a ions o he came a can be con olled (pan, il , zoom, ocus, shu e ape u e among 20 o he s), wi hou he in e e ence o o he modules. This module is a new ea u e (no coded in he o igi- nal so wa e a chi ec u e o he Seagull p ojec ). I was p og ammed and inse ed in he SEP uni o he came a calib a ion es s only, being la e eplaced by he Geo-loca ion module (see sec ion 4.3). Figu e 2.10 p esen s how he Tase ea u es module is in eg a ed in he SEP so wa e a chi ec u e. Figu e 2.10: SEP expe imen al so wa e a chi ec u e. The TASE Fea u es module uses he same in o ma ion as he De ec ion module om he SEP com- pu e iewpoin and communica es wi h he Seagull Ac ua o s module ia he same ROS opics (which he messages s uc u e is p esen ed in Tables 2.1 and 2.2). Fu he mo e, he De ec ion module ou pu (a lis o de ec ed a ge s) is no conside ed. This p o isional a chi ec u e is sel su icien o de ec he ea u es necessa y o he came a calib a ion p ocedu es and ensu es ha all ROS communica ions ecei ed o sen by his module a e con ained in he in e nal SEP a chi ec u e, speci ically he ROS messages TASECommand and TASETeleme y. Table 2.1 suma izes all pa ame e s s uc u ed wi hin a ” ase command in” ROS opic and he pub- lished message TASECommand. This message es ablishes angula posi ions o angula speeds com- mands as well as he s abiliza ion mode and zoom a es. I is ansmi ed by he Tase Fea u es module and does no equi e acknowledgmen o ecep ion. Table 2.2 summa izes all pa ame e s s uc u ed wi hin a ” ase eleme y” ROS opic and he pub- lished message TASETeleme y. This message allows o ob ain senso da a o he TASE 150 gimbal and he Sony FCB-EX1000 came a. This in o ma ion will la e p o ide he necessa y pa ame e s o implemen he geo-loca ion algo i hm. 21 Message Fields Fo ma Uni s Desc ip ion heade S d msgs N/A Heade wi h addi ional in o ma ion pan loa 32 deg ees o deg ees/second Agula posi ion o angula speed o pan came a mo emen . il loa 32 deg ees o deg ees/second Agula posi ion o angula speed o il came a mo emen . zoom in 8 deg ees o deg ees/second Came a Zoom mo emen . F om -8 o 8 whe e nega i e numbe s ep esen zoom ou and posi i e numbe s ep esen zoom in. zoom imeou uin 8 10 milliseconds Speci ica ion o he zoom mo emen ime o execu ion. lags uin 8 N/A 0-3 Rese ed modes o ope a ion. 4 No applicable. 5 Angula Speed and image s abiliza ion command u n o . 6 Angula Speed and image s abiliza ion command u n on. 7 Posi ion and image s abiliza ion command u n o . 8 Posi ion and image s abiliza ion command u n on. 9 impulse and image s abiliza ion command u n o . 10 impulse and image s abiliza ion command u n on. impulse ime uin 8 10 milliseconds Time o espond o a impulse command. Table 2.1: ase command in opic message ields (Table 28 in [18]). 22 Message Fields Fo ma Uni s Desc ip ion heade S d msgs N/A Heade wi h addi ional in o ma ion lags uin 16 N/A Indica o o da a package mode uin 8 N/A Came a ope a ion mode came a uin 8 N/A Came a bi s in o ma ion ime uin 32 m/s Time since las ese la i ude loa 32 deg ees Came a GPS la i ude coo dina es longi ude loa 32 deg ees Came a GPS longi ude coo dina es al i ude loa 32 me e s Came a al i ude no h loa 32 me e s/second No h componen speed ec o eas loa 32 me e s/second Eas componen speed ec o down loa 32 me e s/second Down componen speed ec o moun oll loa 32 deg ees Roll angle o he gimbal body moun pi ch loa 32 deg ees Pi ch angle o he gimbal body moun yaw loa 32 deg ees Yaw angle o he gimbal body pan loa 32 deg ees/second Pan angle o he came a in ela ion o he gimbal body il loa 32 deg ees/second Til angle o he came a in ela ion o he gimbal body oll loa 32 deg ees/second Roll angle o he came a in ela ion o he gimbal body h o loa 32 deg ees Ho izon al ield o iew in deg ees o loa 32 deg ees Ve ical ield o iew in deg ees ocus uin 16 N/A Focus posi ion boa d emp in 8 ◦C Tempe a u e zoom a e uin 16 X Zoom a io ocus mode uin 8 N/A Came a ocus mode Table 2.2: ase eleme y message ields (Table 32 in [18]). 23 Figu e 3.1: Pinhole came a model ep esen a ion. The ”pin-hole” came a model is de ined as sm =AhR| iM, (3.1) o , s     u 1      =     x0cx 0 ycy 0 0 1           11 12 13 1 21 22 23 2 31 32 33 3              X Y Z 1         , (3.2) whe e: •(u , ) - Image coo dina es o a p ojec ed poin ; •A - Came a ma ix, o ma ix o in insic pa ame e s; •( x y) - Focal leng hs in he xand yaxis; •(cx, cy) - P incipal poin coo dina es; •hR| i- Join o a ion- ansla ion ma ix (ex insic pa ame e s); •[X Y Z 1]T- Coo dina es o a 3D poin in he wo ld coo dina e space; •s - Scale ac o . 30 The ma ix o in insic pa ame e s does no depend on he scene iewed, he e o e once he in insic pa ame e s a e de e mined, hey can be e-used as long as he ocal leng h is ixed [22]. The join o a ion- ansla ion ma ix o he ex insic pa ame e s is used o desc ibe he came a mo ion a ound a s a ic scene, o igid mo ion o an objec in on o a s ill came a. When Z6= 0 he equa ion 3.2 can be ew i en [22] x0=X/Z, (3.3) y0=Y/Z, (3.4) u= xx0+cx, (3.5) = yy0+cy, (3.6) s     u 1      =     x0cx 0 ycy 0 0 1           R     X Y Z      +      . (3.7) 3.1.2 Came a Calib a ion Came a calib a ion is a necessa y p ocedu e in o de o de e mine he ma ix o in insic pa ame e s. This s ep is undamen al in i-dimensional compu e ision and allows o ex ac me ic in o ma ion om bi-dimensional images. Zhang [7] p oposes a echnique which equi es only ha a came a obse e a plana pa e n (o model poin s) shown in se e al di e en poses, whe e ei he he came a o he plana pa e n can be mo ed by hand. The p oposed app oach, which uses bi-dimensional me ic in o ma ion, lies be- ween pho og amme ic calib a ion, which uses a explici i-dimensional pa e n measu emen s, and sel -calib a ion, which uses mo ion igidi y o equi alen ly implici i-dimensional in o ma ion. Assuming ha he obse ed pa e n is on (Z= 0) o he wo ld coo dina e sys em, equa ion 3.2 is e-de ined as sm =A     11 12 13 1 21 22 23 2 31 32 33 3              X Y 0 1         =A     11 12 1 21 22 2 31 32 3           X Y 1      , (3.8) he e o e, a poin Mand i s image p ojec ion ma e ela ed by a p ojec i e homog aphy ma ix Hwhich maps poin s be ween π∞and he image plane: sm =H M, (3.9) 31 H=λA      11 12 1 21 22 2 31 32 3      , (3.10) as a simpli ica ion, deno ing he i h column o he join o a ion- ansla ion ma ix by Ri. The homog aphy ma ix His deno ed as H=hh1h2h3i=λA h 1 2 i=λA Ri, (3.11) his homog aphy mapping me hod is independen o he came a posi ion and depends only on he cam- e a in e nal calib a ion and he model, whe e λis an a bi a y scala [7]. The absolu e conic is in a ian unde Euclidean ans o ma ions, he e o e i s ela i e posi ion o a mo ing pa e n is cons an . Fo ixed in insic came a pa ame e s he image o he absolu e conic will also be cons an . Since he absolu e conic Ω∞is on π∞i is possible o compu e i s image p ojec ion unde he homog aphy ma ix H. The image o he conic C= Ω∞=Iin π∞[13] is de ined by w= (ARi)−TΩ∞(ARi)−T= (ARi)−TI(ARi)−T=A−TRiR−1 iA−1=A−TA−1. (3.12) The exp ession w=A−TA−1desc ibes he image o he absolu e conic w(IAC). Simila ly o Ω∞ he conic wis an imagina y poin conic wi h no eal poin s. Using he duali y p inciple we can ex apola e ha he dual image o he absolu e conic (DIAC) is w∗=w−1=AAT, (3.13) his is a line conic, whe eas wis a poin conic (al hough i does no con ain any eal poin s as s a ed be o e), he conic w∗is he image o Q∗ ∞. This esul s poin s ha once w(o he equi alen w∗) is iden i ied in an image hen he in insic pa ame e s ma ix Ais also de e mined. The Absolu e conic can be pe cei ed as a calib a ion objec p esen in all scenes. Once de e mined, i can be used o econs uc he me ic scene cap u ed by he came a. I is, howe e , no always so simple o ind he absolu e conic in he econs uc ed space. 32 Figu e 3.2: The absolu e conic Ω∞and i s image p ojec ion w o di e en came a posi ions iand j. The model plane (o model pa e n) unde he assump ion o he simpli ied no a ion in oduced in equa ion 3.11 [13], is de ined as   3 T 3   T        X Y Z 0         = 0, (3.14) in e sec ing he plane π∞a a line de ined by wo pa icula poin s M1= [ 10] and M2= [ 20], by in e sec ing his line wi h he absolu e conic, we ob ain: M∞=  1±i 2 0 . (3.15) The esul ing a pai o complex conjuga e poin s e eals ha unde Euclidean ans o ma ion hey a e in a ian and he e o e hei p ojec ion on he image plane can be de i ed as m∞=A( 1±i 2)=(h1±i h2), (3.16) and conside ing ha a poin on he image o he absolu e conic (w) mus e i y equa ion 2.7, p esen ed in he p e ious chap e , esul s (h1±i h2)Tw(h1±i h2) = (h1±i h2)TA−TA−1(h1±i h2)=0, (3.17) whe e bo h eal and imagina y pa s mus be ze o. This esul s in wo cons ain s o e he in insic came a pa ame e s o one gi en homog aphy: 33 (hT 1A−TA−1h2= 0,(3.18a) hT 1A−TA−1h1=hT 2A−TA−1h2.(3.18b) De ining was a gene ic ma ix w=A−TA−1=     w11 w12 w13 w21 w22 w23 w31 w32 w33      , (3.19) i holds ha wis symme ic and de ined by a 6D ec o [7]: W=hw11 w12 w22 w13 w23 w33iT . (3.20) By de ining he i h column ec o o Hby hi=hhi1hi2hi3iT , we ob ain hT iwhj= T ijW, (3.21) we e he poin s o he model pa e n obse ed in a image a e desc ibed as ij =hhi1hj1hi1hj2+hi2hj1hi3hj3hi3hj1+hi1hj3hi3hj2+hi2hj3hi3hj3iT . (3.22) he e o e he cons ain s p o ided by equa ions 3.18a and 3.18b, om a gi en homog aphy, can be ew i en as 2 homogeneous equa ions in w[7]:   T 12 ( 11 − 22)T W= 0. (3.23) Once nimages o poin s a e obse ed, by s aking nequa ions 3.23 we ob ain [7]: V W = 0. (3.24) In his equa ion Vis de ined has a 2nX6ma ix. I n > 3we will ha e in gene al a unique solu ion W de ined up o a scale ac o . The solu ion o3.23 is well known as he eigen ec o o VTVassocia ed o he smalles eigen alue (equi alen ly, he igh singula ec o o Vassocia ed o he smalles singula alue). Once Wis es ima ed, we can compu e all came a in insic ma ix A[7]. 34 3.1.3 Came a Dis o ion So a in his wo k i has been assumed ha he pin-hole model desc ibing he came a pe spec i e ans o ma ions o i-dimensional en i ies o bi-dimensional image objec s is op imal. Howe e , dis o ion pa ame e s a e p esen in any came a lenses and mus be aken in o accoun . A dis o ion can be e e ed as a lens abe a ion in which a de ia ion om ec ilinea p ojec ion occu s. A lens ha exhibi s dis o ion p oduces sligh ly cu ed images o all hose lines ha do no pass h ough he cen e o he image. This ac o o igina es mainly om he lens manu ac u ing p ocess o assembling and is some hing ha can’ be o e looked. F om a geome ic iewpoin , he lens dis o ion ela es o he posi ion o image poin s in he image plane. I a poin posi ion in an image is no accu a e he esul s om ei he he calib a ion p ocedu es o compu e ision algo i hms ha depend on i s pixel coo dina es will be e oneous. The e o e, conside ing he lens dis o ion e ec in he ”pin-hole” came a model [23] p esen ed in equa ion 3.7, he amoun o dis o ion p esen in a pixel loca ion can be de ined by      u0=u+δu(u, ) 0= +δ (u, ) , (3.25) uand a e he o iginal, dis o ion- ee image coo dina es and u0and 0a e he co esponding coo di- na es wi h dis o ion obse ed in a image. Equa ion 3.25 indica es ha he e o in each pixel coo dina e pai (u, )is dependen on he posi ion o he poin . Two di e en componen s o dis o ion a e consid- e ed: adial and angen ial dis o ion. Radial dis o ion is he symme ic dis o ion caused by he lens due o impe ec ions in cu a u e when he lens was manu ac u ed. This ype o dis o ion is s ic ly symme ic and causes ei he an inwa d o ou wa d shi o a gi en image poin om i s ideal loca ion [23]. A nega i e adial displacemen o he image poin s is commonly e e ed o as ba el dis o ion. I causes ou e poin s o be p ojec ed in a image plane inc easingly oge he and he scale o dec ease. On he o he hand posi i e adial displacemen is e e ed o as pincushion dis o ion and he opposi e occu s causing ou e poin s o sp ead in he p ojec ed image and he scale o inc ease. Ac ual op ical sys ems a e subjec o a ious deg ees o decen e ing, ha is, he op ical cen e s o lenses a e no co-linea . This de ec in oduces wha is called decen e ing dis o ion. This dis o ion has bo h adial and angen ial componen s Thin p ism dis o ion a ises om impe ec ion in lens design and manu ac u ing as well as came a assembly ( o example, sligh il o some lens elemen s o he image sensing a ay). This ype o dis o ion can be adequa ely modeled by adding a hin p ism o he op ical sys em, causing addi ional adial and angen ial dis o ions. Figu es 3.3 and 3.4 illus a e how his wo componen s adial and angen ial dis o ion espec i ely can a ec a image uniquely ough he lens design and manu ac u ing. 35 Figu e 3.3: E ec o adial dis o ion. Solid line - no dis o ion; dashed line: adial dis o ion (a: nega i e, b:posi i e). Figu e 3.4: E ec o angen ial dis o ion. Solid lines - no dis o ions; dashed lines - angen ial dis o ion. Conside ing his wo dis o ions componen s, he pin-hole came a model wi h he assump ions o he equa ions 3.5 and 3.6 has o be e- amed o ake in o accoun he dis o ion, whe e u= xx00 +cx(3.26) and = yy00 +cy. (3.27) The dis o ed coo dina es x00 and y00 a e ede ined acco dingly o [22] (conside ing ha 2= (x02+y02)) as: x00 =x01 + k1 2+k2 4+k3 6 1 + k4 2+k5 4+k6 6+p2( 2+ 2x02)+2p1x0y0, (3.28) y00 =y01 + k1 2+k2 4+k3 6 1 + k4 2+k5 4+k6 6+p1( 2+ 2y02)+2p2x0y0, (3.29) whe e k1, k2, k3, k4, k5and k6 ep esen he adial dis o ion coe icien s and p1and p2a e angen ial dis o ion coe icien s. 36 3.2 Came a, Gimbal and Body F ames This sec ion desc ibes he coo dina e ame ans o ma ions ha a e p esen ed in a ai bo ne pla - o m, which uses a op ical senso o geo-loca e a a ge . Assuming ha he o igins o he gimbal and came a ame a e loca ed in cen e o mass o he UAV he e a e i e di e en ames o conside : came a ame, gimbal ame, body ame, ehicle ame and ine ial ame. Coo dina e ame ans o ma ions a e desc ibed by wo basic ope a ions: o a ion and ansla ion. Es ablishing ha all coo dina e ames a e cen e ed in he ai c a cen e o mass, ansla ion ma ices a e no conside ed, and only o a ion ans o ma ions a e needed. Any o a ion ope a ion can be achie ed by composing h ee elemen al o a ions abou he ca esian axis. This h ee di e en o a ions a e ob ained ough h ee di e en angles commonly named Eule angles. The Eule angles a e used o desc ibe he o ien a ion o a igid body wi h espec o a known coo dina e ame. Ro a ions can be de ined as h ee sepa a e ma ices acco dingly o he coo dina e ames axis x, y and z, conside ing a gene ic angle α, he o a ion ma ices a e: Rx(α) =      1 0 0 0 cos α−sin α 0 sin αcos α      Ry(α) =      cos α0 sin α 0 1 0 −sin α0 cos α      Rz(α) =      cos α−sin α0 sin αcos α0 0 0 1      . Figu e 3.5 ep esen s a schema ic iew he geome ic ela ions be ween he se e al coo dina e ames. The came a, gimbal, body, ehicle and ine ial coo dina e ames a e deno ed espec i ely by: FC= (xC, yC, zC),FG= (xG, yG, zG),FB= (xB, yB, zB),FV= (xV, yV, zV)and FI= (xI, yI, zI). The gimbal and body coo dina e ames a e ela ed by he angles αaz and αel, which co espond o he pan and il mo emen s o he gimbal came a. The body and ine ial coo dina e ames a e ela ed by he con en ional angles o ai c a mo emen pi ch (θ), oll (φ) and yaw (ψ). 37 (a) (b) Figu e 3.5: Ai c a and gimbal coo dian e ames ep esen a ion (adap ed om [21]). (a)- op iew. (b)- side iew 38 3.2.1 Came a F ame The o igin o he came a coo dina e ame is a he op ical cen e o he came a p esen ed in Figu e 3.5 in blue. The came a axis xCpoin s o he igh o he image plane, yCpoin s downwa d on he image plane and zCpoin s in he di ec ion o he op ical axis o he came a. The coo dina e ame o a ion om he came a o he gimbal coo dina e ame is gi en by RG C=     001 100 010      . (3.30) The opposi e o a ion, om he gimbal o he came a coo dina e ame is gi en by: RC G=     010 001 100      . (3.31) The ans o ma ions om he came a ame o he gimbal came a and ice- e sa ep esen a simple axis swap as illus a ed in Figu e 3.5, whe e xC,yCand zCco espond espec i ely o he gimbal axis yG,zGand xG. 3.2.2 Gimbal F ame The gimbal coo dina e ame azimu h and ele a ion angles a e espec i ely he pan and il mo e- men s o he TASE 150 gimbal, his allows i o o a e in u n o he yGand zGaxis. The axis xGpoin s in he came a lens di ec ion and he axis yGand zGha e he same di ec ion ha yBezBas Figu e 3.5 illus a e. The il o ele a ion angle (αel) is de ined as a o a ion a ound he axis yG. The pan o azimu h angle (αaz) is de ined as a o a ion a ound he axis zG. Using he no a ion o he Eule angles he o a ion om he gimbal o he body coo dina e ame is de ined by RG B=Rz(−αaz)Ry(−αel) =      cos(αel) cos(αaz)−sin(αaz) sin(αel) cos(αaz) sin(αaz) cos(αel) cos(αaz) sin(αaz) sin(αel) −sin(αel) 0 cos(αel)      , (3.32) whe eas he opposi e o a ion, om he body o he gimbal coo dina e ame is gi en by RB G=Ry(αel)Rz(αaz) =      cos(αel) cos(αaz) cos(αel) sin(αaz)−sin(αel) −sin(αaz) cos(αaz) 0 sin(αel) cos(αaz) sin(αel) sin(αaz) cos(αel)      . (3.33) 39 4.1 Came a Calib a ion 4.1.1 Expe imen P ocedu e The came a calib a ion expe imen was conduc ed using he so wa e a chi ec u e desc ibed in Chap e 2, Sec ion 2.2. This so wa e a chi ec u e allowed o acqui e se e al ideos we e a checke pa e n was ilmed. The calib a ion p ocedu e was done o line (by sampling he ames o he acqui ed ideos) wi h a dedica ed py hon sc ip using he open sou ce lib a y OpenC [22]. The calib a ion p ocedu e was employed acco ding o he s eps de ined by Zhang [7] as ollows: 1. A p in ed pa e n was a ach o a plana su ace; 2. A ew sample images o he model plane we e collec ed unde di e en plane o ien a ions; 3. The OpenC lib a y and py hon sc ip de ec ed he ea u e poin s in he images; 4. The in insic pa ame e s and all he ex insic pa ame e s we e es ima ed; 5. The coe icien s o he adial and angen ial dis o ion we e de e mined. The s eps de ined by Zhang [7] a e alid o ixed came a se ings, howe e in he wo k o S um [25] he hypo hesis o in e dependence be ween he came a in insic pa ame e s and i s se ings is explo ed. The e o e in his expe imen , he heo e ical assump ions o S um we e aken in o accoun o ob ain he came a in insic pa ame e s as a unc ion o i s se ings (zoom and ocus). To accomplish a ull a iable came a se ings calib a ion, se e al images samples we e collec ed ac oss he came a zoom band wi h a ixed ocus (a ”in ini y”) and wi h au o- ocus. Sepa a ing he wo ocus modes allows he e i ica ion o he heo e ical asse ions o S um [25], despi e he ac ha in his expe imen al p ocedu es he ocus se ing is no conside ed. Figu e 4.1: Au o- ocus samples o calib a ion. 46 Figu e 4.2: Fixed ocus samples o calib a ion. Figu es 4.1 and 4.2 illus a e se e al samples o bo h au o- ocus and he ixed ocus scena io. In bo h cases, i is possible o ob ain he came a in insic pa ame e s om he checke pa e n ea u es. Howe e , in he case o he ixed ocus scena io, wi h inc easing zoom i becomes inc easingly di icul o ob ain image samples o ob ain a calib a ion. As he zoom inc eases he image blu becomes mo e e iden and he de ec ion o ea u es in he checke pa e n becomes nea ly impossible. 4.1.2 Expe imen P epa a ions The p ocess o collec ing image samples o bo h scena ios illus a ed in Figu es 4.1 and 4.2 ac oss he en i e zoom band is a e y ime-consuming ask [25]. The e o e i is impo an o assess how he amoun o samples a ec he compu a ion ime o he calib a ion p ocess in o de o make he en i e p ocess as much ime e icien as possible. Figu e 4.3 illus a es how he equi ed compu ing ime inc eases in ela ion o he numbe o samples p o ided o he calib a ion algo i hm. In he con a y, he esul s o he calib a ion algo i hm s a o se le a ound he 35 samples pe calib a ion. This assessmen illus a es he ac ha e en i 35 o mo e samples a e used he calib a ion esul s will no be mo e accu a e. Conside ing ha he esul o he calib a ion algo i hm only depends on he samples p o ided and he compu ing ime inc eases wi h he numbe o samples, no leading o a mo e p ecise esul , a mode a e alue o 40 image samples pe calib a ion was selec ed. This ini ial assessmen es shows ha he accu acy o he came a in insic esul s a e di ec ly linked o he image samples used. Ac oss he zoom band, a o al o 10555 image samples o a ixed ocus and 50802 image samples o au o- ocus we e ob ained. The disc epancy in he numbe s o image samples e i ied be ween he wo scena ios is due o image blu when using a ixed ocus se ing, as obse ed in Figu e 4.2. In his case, some o he collec ed samples p esen a high le el o blu , making i e y di icul o ex ac he ea u e poin s o he checke pa e n and leading o a smalle amoun o image samples 47 usable o calib a ion. Gi en he numbe o image samples collec ed o bo h scena ios and he ac ha he calib a ion only depends on he images gi en o he calib a ion algo i hm, a s a is ical app oach was aken in o conside a ion. This s a is ical in e p e a ion o he came a in insic pa ame e s gi en by he calib a ion p ocedu e has wo dis inc objec i es: minimize he in luence o he image samples in he calib a ion and access he pe o mance o he calib a ion algo i hm. The i s objec i e is accomplished by p o iding di e en image samples o each zoom le el se e al imes o he calib a ion algo i hm. This ensu es ha a la ge a ie y o images a e es ed and minimizes he in luence o a single image (o g oup o images) in he calib a ion esul s. The second objec i e is o e i y he calib a ion algo i hm pe o mance a ies when, o a same scena io, di e en g oups o images a e p o ided and he esul s o he di e en alues ob ained o he in insic pa ame e s a e compa ed. Gi en he ac ha all came a se ings a e he same and only he images samples a e di e en , simila esul s mus be expec ed ega dless o he image samples p o ided. Fo each zoom le el, 50 sepa a e calib a ions we e conduc ed using 40 andomly selec ed images samples o he co esponding zoom le el. This p o ided a da abase o alues ac oss he en i e zoom band wi h mo e han one alue o each in insic pa ame e om a mul i ude o combina ions o image samples. The da a o he da abase was collec ed o bo h ixed and au o- ocus. Figu es 4.4 and 4.5 a e he esul s o he calib a ions conduc ed wi h a ixed ocus scena io. In Figu e 4.4 i is possible o e i y ha he samples a e age p incipal poin coo dina es cxand cy, ega dless o he zoom le el, a e mo e o less consis en wi h he image cen e being loca ed a he pixel coo dina es (u, ) = (320,240). In Figu e 4.5 i is possible o e i y ha he ocal leng hs xand yg adually inc eases wi h he zoom le el, howe e i appea s ha a a io o 1:1 is p esen be ween xand y. Figu e 4.3: Calib a ion ime (seconds) o he ocal leng h a 0% zoom le el. 48 Figu e 4.4: P incipal poin (cxand cy) a ia ion h ough he zoom ange. Rega dless o he zoom le el, he a e age o he samples collec ed p esen alues o he p incipal close o he image cen e . Figu e 4.5: Focal leng h xand y a ia ion h ough he zoom ange. The ocal leng h inc ease wi h he zoom le el (lowe le co ne is a = 0% zoom and op igh co ne is a 100% zoom. 49 Figu es 4.4 and 4.5 e eal some in e es ing esul s espec i ely ega ding he loca ion o he p incipal poin and he ocal leng h which enable he possibili y o using wo calib a ion lags om he OpenC oolbox. The lags o he OpenC lib a y can be se when ce ain ela ions be ween some o he in insic pa ame e s a e obse ed and help o educe he compu ing ime o he calib a ion algo i hm. Gi en ha he p incipal poin does no exhibi signi ican changes, a he luc ua es i s posi ion a he image cen e and he ocal leng h p esen a a io o 1:1, in an a emp o dec ease he o al compu ing ime wo OpenC oolbox lags we e selec ed: CV CALIB FIX PRINCIPAL POINT, which ixes he cen- al poin a (u, ) = (320,240) and CV CALIB FIX ASPECT RATIO, which o ces a a io be ween he ocal leng hs du ing he calib a ion. This ini ial assessmen p o ided a mo e ime e icien app oach. I no conduc ed, each calib a ion would ake a ound 37.5 seconds which could lead o upwa ds o 30 minu es o calib a e each zoom le el. Conside ing ha a o al o 62 zoom le els we e es ed, his amoun s o app oxima ely 32 hou s o compu ing ime jus o one scena io o he wo conside ed. Using he p oposed lags, he en i e calib a- ion p ocedu e o each scena io in ques ion, ac oss he en i e zoom band, was done in app oxima ely 24 hou s. 50 4.1.3 Expe imen al Resul s One o he indica o s o a came a calib a ion quali y is he e-p ojec ion e o . De ined as he geo- me ic e o co esponding o he image dis ance be ween a p ojec ed poin in he image plane and he measu ed one Figu e 4.6: Re-p ojec ion RMS e o h ough he zoom ange. Es ablishing ha he poin s in he calib a ion p ocedu e a e pe ec ly measu ed in a i s image and, conside ing only he e o s in a second image. The mo e app op ia e way o analyze he e-p ojec ion e o is o minimize he ans e unc ion gi en by: X i d(x0 i, H ¯xi)2, (4.1) his ep esen s he Euclidean image dis ance in he measu ed poin s x0 iand he co esponding alue o he poin s H¯xico esponding o he i s image. In a cases whe e image measu emen e o occu in bo h images, i is ideal o minimize hose e o s in bo h images. One way o ob aining a mo e ealis ic e o unc ion is by: X i d(xi, H−1x0 i)2+d(x0 i, Hxi)2, (4.2) whe e i is conside ed he o wa d (H) and backwa d (H2) ans o ma ion and he sum o he geome ic e o s in he i s and second image, espec i ely he e ms o he sum. 51 A mo e simplis ic app oach is o concep ualize he e-p ojec ion oo mean squa e e o as he sum o squa ed dis ances be ween he obse ed p ojec ions o he image poin s and he p ojec ed pa e n poin s in o he image plane. In Figu e 4.6 he esul s o he calib a ion o he en i e zoom band a e p esen ed, whe e i is possible o e i y he di e en beha io s o he wo scena ios conside ed. In Figu e 4.6 when compa ing he au o- ocus scena io wi h ixed ocus scena io i is possible o e i y ha i exhibi s a be e RMS e o h oughou he zoom band, i is also possible o e i y ha he alue o he samples o he calib a ion p esen s a smalle s anda d de ia ion. This ac is in e ed om he sample alues dispe sion a ound hei espec i e a e age. One o he ac ha can be poin ed is he ac ha he ixed ocus scena io p esen s an inc ease in he RMS e-p ojec ion e o as he zoom le el inc eases, e en ually s abilizing. This is due o he ac ha image samples collec ed a a highe zoom le el su e om an inc easing blu . Due o he deg aded image quali y, he calib a ion algo i hm pe o mance is a ec ed leading o highe e-p ojec ion e o s. Figu e 4.7: Focal leng h h ough he zoom ange. Figu e 4.7 ep esen s he ocal leng h in pixel uni s ac oss he zoom band. I is possible o obse e ha he highe zoom le els he highe he ocal leng h. Be ween he wo scena ios s udied, we see ha he alues o he ocal leng h s a e y simila ly in bo h case. Howe e , as he zoom inc eases and passes he 50% ma k, he ixed ocus scena io s a s o exhibi highe alues. This once again be jus i ied by he image quali y o his scena io, whe e he image blu induces noise in he esul s om he calib a ion algo i hm. Ano he indica o o he noise in he esul s is he di e ence in he dispe sion o he alues be ween he conside ed scena ios. As in he RMS e-p ojec ion e o , he ixed ocus scena io shows a highe s anda d de ia ion o he samples when compa ed o he au o- ocus. 52 Figu e 4.8: Radial dis o ion pa ame e k1ac oss he zoom ange. Figu e 4.9: Radial dis o ion pa ame e k2ac oss he zoom ange. Figu es 4.8, 4.9 and 4.10 illus a e se e al adial dis o ion pa ame e s o he came a lenses. F om he da a collec ed i is possible o e i y ha in Figu e 4.8 he pa ame e k1exhibi s a e y de ined beha io we e he highe he zoom le el he highe i s alue is. Simila ly o he ocal leng h, he da a collec ed be ween he wo scena ios conside ed p esen s a highe dispe sion o alues in he ixed ocus 53 Figu e 4.10: Radial dis o ion pa ame e k3ac oss he zoom ange. scena io when compa ing o he au o- ocus. The a e age alues also s a e y simila , howe e as he zoom inc eases he a e age esul s di e ge be ween he scena ios. Pa ame e s k2and k3, espec i ely Figu es 4.9 and 4.10 p esen a mo e cons an alues h oughou he i s hal o he zoom band. Howe e , bo h show e a ic esul s in he ange o 80%-100% and 60%-80%, espec i ely. The da a sugges s ha he calib a ion algo i hm is o e loaded, ying o ob ain accu a e alues o e y small changes and leading o some unp edic abili y. This esul s a e e ealed o be e y ha d o model as a con inuous unc ion in he anges a ec ed by he sudden changes in alue. Figu es 4.11 and 4.12 p esen he angen ial dis o ion pa ame e s. These pa ame e s show a con- s an alue close o ze o in bo h cases. Once again he s anda d de ia ion o he samples in he case o he ixed ocus is conside ably highe han he au o- ocus scena io, and he alues o he a e age a e no so well es ablished as ze o, a he hey luc ua e a ound he ze o alue. 54 Figu e 4.11: Tangen ial dis o ion pa ame e p1 h ough he zoom ange. Figu e 4.12: Tangen ial dis o ion pa ame e p2 h ough he zoom ange. 55 4.2.2 Expe imen P epa a ions The i s p epa a ions conce n he geome ic loca ion o he pe spec i e cen e o he came a. This is he poin whe e no pa allax e o s occu . Depending on he lenses design, he geome ic pe spec i e cen e poin loca ed on he op ical axis may be behind, wi hin o in on o he lens sys em and e en a an in ini e dis ance om he lens in he case o elecen ic sys ems. The pe spec i e cen e poin ep esen s he poin in a lens o which when o a ing he came a o he pano amic head, image s i chings do no appea in sequenced cap u ed images. The e o e, o he pu poses o his expe imen , i can be in e p e ed as he geome ic cen e o he lenses assembly, co esponding o he ocus poin o he ”pin-hole” came a model p e iously. To de e mine he pe spec i e cen e poin posi ion wo objec s mus be placed e ically in on o he came a. In he labo a o y en i onmen , he legs o a able whe e used o his pu pose. Looking a he images cap u ed in Figu e 4.19, he came a was o a ed so ha he wo objec s we e loca ed a he igh o he image ame. A e wa ds, he came a was o a ed so ha he objec s a e on he le side o he image. Repea ing his p ocedu e while ansla ing he came a back and o wa d ela i e o he o a ion axis he e is a poin in which he wo objec s selec ed (legs o he able) become pe ec ly aligned in he image ega dless o he came a o ien a ion. Figu e 4.19: Pe spec i e cen e poin de e mina ion s eps. The pe spec i e cen e was ma ked in he came a moun as Figu e 4.20 p esen s. This poin he e o wa d is assumed as he o igin o he ine ial e e ence ame conce ning his expe imen , illus a ed as he poin (0,0,0) in Figu es 4.15 and 4.16. 62 Figu e 4.20: Pe spec i e cen e poin ma kings in he came a moun . 4.2.3 Expe imen al Resul s The expe imen s conduc ed co e se e al zoom le els, as well as di e se came a, poses. Table 4.3 summa izes he es condi ions o each es ealized. These condi ions we e pu posely a ied wi h he objec i e o p o iding a se o di e en scena ios and compa ing hem o he se e al polynomial app oaches p oposed. Shee 9 Page 1 Tes Scena io Tes 0 Tes 1 Tes 2 Tes 3 Tes 4 Tes 5 Uni s Tes Pa ame e s Pan -180 -180 -180 -255 -180 -180 º (Deg ees) Til -25 -20 -20 -20 -20 -20 º (Deg ees) Heading 180 180 180 105 180 180 º (Deg ees) Pi ch 0 5 7 10 7 5.2 º (Deg ees) Roll 3.5 3.5 3.5 2.1 2 3.5 º (Deg ees) Zoom 0 5 10 20 50 95 % (Pe cen age) Table 4.3: Tes s pa ame e s summa y Table. To compa e he known posi ion o he a ge s and hei es ima ed posi ions he a ge coo dina es a e gi en by Tand he loca ion es ima es a e desc ibed as E. The loca ion e o is gi en by: ρ=T−E=hx y z i−hxeyezei=hx y zi, (4.9) he mean e o and oo mean squa e e o o he esul ing employmen o he geo-loca ion algo i hm a e espec i ely: µe=1 N n X i=1 qρ2 x+ρ2 y+ρ2 z, (4.10) 63 RMSE o = u u u N P i=1 (qρ2 x+ρ2 y+ρ2 z) N, (4.11) In Tables 4.4, 4.5 and 4.6 he alue o |ρ|is p esen ed acco ding o he es and a ge conside ed. The mean e o , mean RMS e o and s anda d de ia ion o he collec ed samples is also p esen ed. The samples collec ed a e analyzed om a s a is ical iewpoin due o he se e al luc ua ions induced by he de ec ion algo i hm and he angles senso s in o ma ion o he gimbal came a. k14 Page 6 Ta ge Tes 0 Tes 1 Tes 2 Tes 3 Tes 4 Tes 5 Mean E o RMS E o 10.0269 0.0310 0.0420 NA NA NA 0.0333 0.0339 20.0510 0.0506 0.0475 NA NA NA 0.0497 0.0497 30.0180 0.0233 0.0278 NA NA NA 0.0230 0.0234 40.0447 0.0428 0.0517 0.0270 NA NA 0.0415 0.0425 50.0676 0.0490 0.0478 0.0520 NA NA 0.0541 0.0547 60.0407 0.0564 0.0514 0.0324 NA NA 0.0452 0.0462 70.1052 0.0730 0.1052 0.0839 0.0839 NA 0.0903 0.0912 80.1012 0.0956 0.0994 0.0843 0.0885 NA 0.0938 0.0940 90.0742 0.1032 0.1030 0.0858 0.0837 NA 0.0900 0.0907 10 0.1531 0.1764 0.1420 0.1570 0.1547 NA 0.1566 0.1570 11 0.1732 0.1281 0.1890 0.1692 0.1771 0.1904 0.1712 0.1724 12 0.1559 0.1934 0.1725 0.1606 0.1591 NA 0.1683 0.1689 Table 4.4: E o ec o module (|ρ|) when using ocal leng h 14 h deg ee model and a 16 h deg ee app oxima ion o k1. K10 Page 7 Ta ge Tes 0 Tes 1 Tes 2 Tes 3 Tes 4 Tes 5 Mean E o RMS E o 10.0270 0.0308 0.0422 NA NA NA 0.0334 0.0340 20.0524 0.0509 0.0479 NA NA NA 0.0504 0.0504 30.0187 0.0241 0.0283 NA NA NA 0.0237 0.0240 40.0449 0.0430 0.0519 0.0270 NA NA 0.0417 0.0427 50.0672 0.0492 0.0472 0.0525 NA NA 0.0540 0.0546 60.0411 0.0565 0.0515 0.0325 NA NA 0.0454 0.0463 70.1053 0.0736 0.1054 0.0841 0.0840 NA 0.0905 0.0914 80.1013 0.0954 0.0993 0.0842 0.0883 NA 0.0937 0.0939 90.0744 0.1033 0.1032 0.0859 0.0836 NA 0.0901 0.0908 10 0.1528 0.1761 0.1422 0.1572 0.1545 NA 0.1566 0.1569 11 0.1734 0.1278 0.1891 0.1690 0.1773 0.1914 0.1713 0.1726 12 0.1561 0.1960 0.1735 0.1616 0.1571 NA 0.1688 0.1695 Table 4.5: E o ec o module (|ρ|) when using ocal leng h 12 h deg ee model and a 14 h deg ee app oxima ion o k1. 64 k8 Page 8 Ta ge Tes 0 Tes 1 Tes 2 Tes 3 Tes 4 Tes 5 Mean E o RMS E o 10.0270 0.0311 0.0421 NA NA NA 0.0334 0.0340 20.0509 0.0508 0.0474 NA NA NA 0.0497 0.0497 30.0183 0.0236 0.0280 NA NA NA 0.0233 0.0237 40.0448 0.0428 0.0516 0.0273 NA NA 0.0416 0.0426 50.0677 0.0492 0.0482 0.0522 NA NA 0.0543 0.0549 60.0409 0.0562 0.0504 0.0327 NA NA 0.0450 0.0459 70.1055 0.0736 0.1053 0.0834 0.0845 NA 0.0905 0.0914 80.1013 0.0958 0.0999 0.0863 0.0889 NA 0.0944 0.0946 90.0751 0.1037 0.1033 0.0863 0.0868 NA 0.0910 0.0917 10 0.1533 0.1764 0.1430 0.1573 0.1543 NA 0.1568 0.1572 11 0.1738 0.1283 0.1892 0.1695 0.1767 0.1884 0.1710 0.1722 12 0.1560 0.1936 0.1772 0.1610 0.1594 NA 0.1695 0.1700 Table 4.6: E o ec o module (|ρ|) when using ocal leng h 10 h deg ee model and a 12 h deg ee app oxima ion o k1. Analyzing he esul s in Tables 4.4, 4.5 and 4.6 i is possible o conclude ha he combina ion o models ega ding Table 4.4 p o ide he bes esul s o he p oposed expe imen . Compa ing bo h he mean and RMS e o s o he da a he combina ion o he ocal leng h 14 h deg ee model and k116 h deg ee p o ide he smalle posi ion e o es ima e. The da a p esen ed also e eals wo indica ions. Fi s , he dis ance inc easing o he a ge s wi h ega d o he ine ial ame o igin ( he poin cen e o p ojec ion) esul s in a highe e o . The e o e, he u he away a a ge is loca ed he highe he e o and i s co esponding posi ion es ima e is u - he away om he eal posi ion. Secondly, he da a sugges s ha o di e en zoom le els he a ge coo dina e e o s a e simila , and possibly independen o he zoom used. The i s indica ion e eals some sys ema ic e o s. These can occu ei he om he came a and gimbal senso s measu emen s o he models ob ained om he calib a ion o he ocal leng h and dis o ion pa ame e s. The modula ion o he came a in insic pa ame e s is no an exac alue o he pa ame e s, as he manu ac u e is he only one who migh know his in o ma ion and i is no disclosed in he equipmen manual. The e o e, he models ob ained should be iewed has an es ima e a no has he p ecise, accu a e, and exac models, which desc ibe he in e dependence be ween he came a lenses and i s cha ac e is ic pa ame e s. The second indica ion allows o es ablish he cohe ence o he models p esen ed in Figu es 4.13 and 4.14. Th oughou he zoom le els, he esul s ob ained using he es ima ed models a e e y simila and so a e he esul s as p esen ed in Tables 4.4, 4.5 and 4.6, a es ing o hei alidi y. The expe imen al esul s show ha , independen ly o he zoom le el used, he a ge es ima es e o s a e e y simila , alida ing he hypo hesis o using models o emula e he came a pa ame e s. 65 4.2.4 Discussion o Resul s The expe imen esul s e eal a p omising ou come o he employmen o he geo-loca ion algo i hm in a ull-scale scena io. Depending on he used polynomial model he esul s a e encou aging. Ve i ying Figu es 4.13 and 4.14 he model unc ions do no encompass pe ec ly he in insic pa ame e s. How- e e , hey p o ide he bes possible app oxima ion o hei espec i e pa ame e s beha io . Because o he oscilla ions e i ied (unde /o e -shoo ing o hese models), he senso bias o he equipmen and de ec ion algo i hm, an e o in he geo-loca ion is ine i ably p esen . Based on he obse ed esul s (Tables 4.4, 4.5 and 4.6) and he Figu es 4.13 and 4.14 wi h he polynomial models, he combina ion o models which p esen he smalles ange ec o e o is he 14 h deg ee polynomial model o he ocal leng h and he 16 h deg ee polynomial model o he adial dis o ion pa ame e k1. Shee 1 Page 4 Ta ge Ta ge Coo dina es Tes 0 Tes 1 Tes 2 1x: 1.1 y: -0.4 z: 1.04 x: 1.09 y: -0.38 z: 1.04 x: 1.092 y: -0.37 z: 1.04 x: 1.099 y: -0.36 z: 1.04 2x: 1.1 y: 0 z: 1.04 x: 1.09 y: 0.05 z: 1.04 x: 1.116 y: 0.048 z: 1.04 x: 1.115 y: 0.045 z: 1.04 3x: 1.1 y: 0.4 z: 1.04 x: 1.1 y: 0.418 z: 1.04 x: 1.11 y: 0.421 z: 1.04 x: 1.114 y: 0.424 z: 1.04 4x: 1.4 y: -0.4 z: 1.04 x: 1.434 y: -0.37 z: 1.04 x: 1.434 y: -0.37 z: 1.04 x: 1.438 y: -0.37 z: 1.04 5x: 1.4 y: 0 z: 1.04 x: 1.433 y: 0.059 z: 1.04 x: 1.431 y: 0.038 z: 1.04 x: 1.429 y: 0.038 z: 1.04 6x: 1.4 y: 0.4 z: 1.04 x: 1.436 y: 0.419 z: 1.04 x: 1.451 y: 0.424 z: 1.04 x: 1.441 y: 0.431 z: 1.04 7x: 1.7 y: -0.4 z: 1.04 x: 1.796 y: -0.36 z: 1.04 x: 1.767 y: -0.37 z: 1.04 x: 1.796 y: -0.36 z: 1.04 8x: 1.7 y: 0 z: 1.04 x: 1.78 y: 0.062 z: 1.04 x: 1.789 y: 0.035 z: 1.04 x: 1.791 y: 0.04 z: 1.04 9x: 1.7 y: 0.4 z: 1.04 x: 1.772 y: 0.418 z: 1.04 x: 1.801 y: 0.421 z: 1.04 x: 1.801 y: 0.42 z: 1.04 10 x: 2 y: -0.4 z: 1.04 x: 2.146 y: -0.35 z: 1.04 x: 2.175 y: -0.38 z: 1.04 x: 2.139 y: -0.37 z: 1.04 11 x: 2 y: 0 z: 1.04 x: 2.163 y: 0.059 z: 1.04 x: 2.124 y: 0.032 z: 1.04 x: 2.186 y: 0.034 z: 1.04 12 x: 2 y: 0.4 z: 1.04 x: 2.149 y: 0.446 z: 1.04 x: 2.189 y: 0.441 z: 1.04 x: 2.168 y: 0.439 z: 1.04 Table 4.7: Real and es ima ed coo dina es ob ained by using a 14 h deg ee polynomial model o he ocal leng h and a 16 h deg ee polynomial model o k1 esul s o es s 0 o 2a e shown. Shee 2 Page 2 Ta ge Ta ge Coo dina es Tes 3 Tes 4 Tes 5 1x: 1.1 y: -0.4 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA x: NA y: NA z: NA 2x: 1.1 y: 0 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA x: NA y: NA z: NA 3x: 1.1 y: 0.4 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA x: NA y: NA z: NA 4x: 1.4 y: -0.4 z: 1.04 x: 1.379 y: -0.38 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA 5x: 1.4 y: 0 z: 1.04 x: 1.374 y: 0.045 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA 6x: 1.4 y: 0.4 z: 1.04 x: 1.368 y: 0.405 z: 1.04 x: NA y: NA z: NA x: NA y: NA z: NA 7x: 1.7 y: -0.4 z: 1.04 x: 1.781 y: -0.38 z: 1.04 x: 1.781 y: -0.38 z: 1.04 x: NA y: NA z: NA 8x: 1.7 y: 0 z: 1.04 x: 1.77 y: 0.047 z: 1.04 x: 1.775 y: 0.047 z: 1.04 x: NA y: NA z: NA 9x: 1.7 y: 0.4 z: 1.04 x: 1.78 y: 0.431 z: 1.04 x: 1.781 y: 0.421 z: 1.04 x: NA y: NA z: NA 10 x: 2 y: -0.4 z: 1.04 x: 2.151 y: -0.36 z: 1.04 x: 2.148 y: -0.36 z: 1.04 x: NA y: NA z: NA 11 x: 2 y: 0 z: 1.04 x: 2.162 y: 0.049 z: 1.04 x: 2.171 y: 0.046 z: 1.04 x: 2.185 y: 0.045 z: 1.04 12 x: 2 y: 0.4 z: 1.04 x: 2.158 y: 0.429 z: 1.04 x: 2.156 y: 0.431 z: 1.04 x: NA y: NA z: NA Table 4.8: Real and es ima ed coo dina es ob ained by using a 14 h deg ee polynomial model o he ocal leng h and a 16 h deg ee polynomial model o k1 esul s o es s 3 o 5a e shown. 66 Tables 4.7 and 4.8 p esen he de ailed esul s o he expe imen de ailing each measu ed coo dina e and es conduc ed o he 14 h deg ee polynomial model o he ocal leng h and he 16 h deg ee poly- nomial model o he adial dis o ion pa ame e k1. The ac ha he dis o ion pa ame e s k2,k3,p1,p2 we e disca ded appa en ly do no exhibi a a signi ican in luence in e o s ob ained in his expe imen . The ac i was assumed ha he disca ded pa ame e s we e no signi ican would lead o belie e ha a highe e o would be e iden howe e a he scale o his expe imen , he esul s show o he wise. The ocus se ing, howe e , is a came a se ing wi h a small bu e iden p esence. In e e y es and a ge conside ed i appea s ha a small de ia ion o he igh in he yaxis is p esen . The de ia ion can be a ibu ed o he non-modula ion o he ocus se ing. As s a ed by S um [25] his would cause a ansla ion o he cen al poin (e en i mainly due o he zoom se ing). Accoun ing o he small change in he cen al poin due o he ocus change could lead o a mo e a o able esul and possibly elimina e he de ia ion e i ied in he yaxis. In summa y, he esul s o his expe imen allowed o e i y he alidi y o he calib a ion p ocedu e p oposed as well as he geo-loca ion algo i hm and de e mining he mo e a o able model o he cam- e a pa ame e s o employmen in a ull-scale scena io. The esul s e ealed o be encou aging and, al hough e o s we e e i ied, hei p esence is ine i able due o he choices and assump ions made o he came a in insic pa ame e s modula ion. 67 4.3 Geo-loca ion Module in he SEP So wa e A chi ec u e The p oposed in eg a ion o he so wa e de eloped in his disse a ion can be illus a ed by Fig- u e 4.21, we e he geo-loca ion module is in eg a ed in o he sep a chi ec u e using he ROS middle- wa e. Figu e 4.21: SEP so wa e a chi ec u e wi h geo-loca ion module. The p oposed so wa e a chi ec u e is illus a ed in Figu e 4.21. In his a chi ec u e, in con as o he expe imen al a chi ec u e, all he SEP modules a e enabled and he Tase Fea u es module is eplaced wi h he Geo-loca ion module. This module is de ined as a ROS node and accesses o he memo y o load he came a models p e-de e mined in he came a calib a ion p ocedu e. The geo-loca ion module ecei es a lis o a ge s and hei pixel coo dina es om he de ec ion module as well as he ai c a and came a eleme y and ou pu s a lis o he coo dina es o he iden i ied a ge s. The me hodology employed was al eady ex ensi ely de ailed in chap e 3. Figu e 4.22 summa izes he geo-loca ion algo i hm and implemen a ion in he con ex o he so wa e a chi ec u e. The came a models s o ed in memo y de e mined in he calib a ion p ocedu es a e combined wi h a a ge lis in he Came a F ame sub-module, his de ines he a ge s de ec ed in he came a coo dina e ame de ined by he coo dina es (u, , ), espec i ely he pixel ho izon al and e ical coo dina es and he ocal leng h gi en by he model o he co esponding zoom. In his sub-module, a dis o ion co ec ion is applied by using he k1 adial dis o ion pa ame e model. The eleme y da a o he ai c a pi ch, oll, yaw, al i ude abo e g ound le el (AGL) and GPS posi ion oge he wi h he angles (pan and il ) o he gimbal came a a e combined o ob ain he ECEF coo di- na es o he a ge (in me e s). Howe e o an ope a o hey a e no e y use ul. The e o e, hey a e ans o med in o geode ic coo dina es la i ude, longi ude, and al i ude be o e being published in a ROS opic. 68 Figu e 4.22: Geo-loca ion module. 69 4.4 Geo-loca ion in Real-Wo ld Condi ions 4.4.1 Expe imen P ocedu e This expe imen es s he geo-loca ion algo i hm in a ull-scale eal-wo ld scena io. The objec i es o his expe imen a e: inal alida ion o he p oposed me hodology o calib a ing a came a wi h a iable se ings and es he geo-loca ion algo i hm unde eal-wo ld condi ions and cons ain s. To accomplish he p oposed objec i es he 14 h deg ee polynomial model o he ocal leng h and he 16 h deg ee polynomial model o he adial dis o ion pa ame e k1we e used. In his expe imen , a es a ge wi h 2.8 me e s in leng h by 2 me e s in wid h was buil (p esen ed in Figu e 4.23) and placed in se e al loca ions h oughou he Po uguese Ai Fo ce Academy campus. The gimbal came a se up was placed on he op o he wa e owe building as p esen ed by Figu e 4.24. This loca ion was chosen o ensu e a su icien heigh o app oxima e, as close as possible, he ai c a al i ude o ope a ion. Figu e 4.23: Tes a ge o he ull scale expe imen . Figu e 4.26 p esen s he gimbal came a loca ion as well as he a ge s loca ions. In each loca ion, se e al es s we e conduc ed by a ying he pan, il o zoom gimbal pa ame e s. Du ing he es s, he gimbal came a was main ained in a s a ic posi ion and he loca ion o he a ge s was chosen o accommoda e a a ie y o anges. The a ge s a e numbe ed om 1 o 10 acco ding o hei inc easing dis ance o he came a loca ion. Each a ge GPS coo dina es we e ob ained wi h a ecei e o compa e he eal a ge coo dina es wi h he es ima ions made by he geo-loca ion algo i hm. In his expe imen al se up, en i onmen al ac o s such as wind and sun gla e we e p esen and expec ed o in luence he esul s, in con as wi h he labo a o y simula ion om he p e ious expe imen p esen ed in Sec ion 4.2. The came a in his expe imen was main ained in s a ic posi ion wi h a ixed heading, pi ch, and oll (heading = 150o, pi ch=3.15o, oll=0.95o). The a ge was cap u ed in se e al es s wi h a iable pan, il o zoom summa ized in Table 4.25 (a) and (b), which p esen s he alues o he a iable pa ame e s acco dingly o he es and loca ions illus a ed in Figu e 4.26. 70 Figu e 4.24: Came a se up in he wa e owe o he Po uguese Ai Fo ce Academy. Shee 1 Page 1 Ta ge Loca ion Tes Tes Pa ame e s Ta ge Loca ion Tes Tes Pa ame e s Pan (deg ees) Til (deg ees) Zoom (%) Pan (deg ees) Til (deg ees) Zoom (%) 11 124,996 -30,005 89,5 6 1 176,036 -9,992 98,784778 2 124,985 -29,994 98,9873 2 -179,983 -11,012 85,36403 2 1 125,518 -21,990 98,902036 3 -179,983 -11,012 85,364034 2 125,518 -22,002 100 4 177,015 -9,964 85,364033 3 126,518 -22,002 95,742138 5 175,033 -12,015 85,364033 3 1 -175,496 -23,990 98,934 6 175,984 -9,987 98,742138 2 -175,486 -23,990 88,06 7 1 135,487 -5,002 100 3 -175,485 -25,990 80,06097 2 135,487 -5,002 100 4 -174,018 -24,488 72,263 3 136,014 -4,503 100 5 -173,978 -23,067 80,7376 4 134,954 -4,503 100 4 1 -133,975 -18,982 92,8579 5 134,954 -5,500 100 2 -136,977 -18,982 92,8579 8 1 151,020 -5,013 95,832 3 -136,977 -19,034 92,8579 2 151,794 -5,202 100 4 -134,026 -22,036 92,8579 3 151,450 -4,303 100 5 -135,000 -20,993 99,2324 9 1 156,486 -4,303 100 6 -135,000 -20,500 100 2 155,953 -3,999 100 5 1 -118,000 -16,003 84,127493 3 156,486 -4,303 100 2 -114,030 -16,002 84,12749 4 155,466 -3,501 100 3 -115,900 -18,002 84,12749 10 1 -174,047 -5,001 100 4 -116,992 -19,990 93,68937 2 -174,547 -5,002 100 5 -116,992 -19,990 93,6893 3 -174,534 -5,012 100 6 -114,042 -19,990 93,68937 4 -174,002 -5,500 100 (a) Ta ge loca ions 1 o 5. Shee 1 Page 1 Ta ge Loca ion Tes Tes Pa ame e s Ta ge Loca ion Tes Tes Pa ame e s Pan (deg ees) Til (deg ees) Zoom (%) Pan (deg ees) Til (deg ees) Zoom (%) 11 124,996 -30,005 89,5 6 1 176,036 -9,992 98,784778 2 124,985 -29,994 98,9873 2 -179,983 -11,012 85,36403 2 1 125,518 -21,990 98,902036 3 -179,983 -11,012 85,364034 2 125,518 -22,002 100 4 177,015 -9,964 85,364033 3 126,518 -22,002 95,742138 5 175,033 -12,015 85,364033 3 1 -175,496 -23,990 98,934 6 175,984 -9,987 98,742138 2 -175,486 -23,990 88,06 7 1 135,487 -5,002 100 3 -175,485 -25,990 80,06097 2 135,487 -5,002 100 4 -174,018 -24,488 72,263 3 136,014 -4,503 100 5 -173,978 -23,067 80,7376 4 134,954 -4,503 100 4 1 -133,975 -18,982 92,8579 5 134,954 -5,500 100 2 -136,977 -18,982 92,8579 8 1 151,020 -5,013 95,832 3 -136,977 -19,034 92,8579 2 151,794 -5,202 100 4 -134,026 -22,036 92,8579 3 151,450 -4,303 100 5 -135,000 -20,993 99,2324 9 1 156,486 -4,303 100 6 -135,000 -20,500 100 2 155,953 -3,999 100 5 1 -118,000 -16,003 84,127493 3 156,486 -4,303 100 2 -114,030 -16,002 84,12749 4 155,466 -3,501 100 3 -115,900 -18,002 84,12749 10 1 -174,047 -5,001 100 4 -116,992 -19,990 93,68937 2 -174,547 -5,002 100 5 -116,992 -19,990 93,6893 3 -174,534 -5,012 100 6 -114,042 -19,990 93,68937 4 -174,002 -5,500 100 (b) Ta ge loca ions 6 o 10. Figu e 4.25: Tes pa ame e s acco ding o a ge loca ions. 71 4.4.3 Discussion o Resul s The expe imen p ocedu e conduc ed and consequen da a analysis poin s o se e al ac o s which in luence he geo-loca ion me hod p oposed. The i s ac o is he came a in insic pa ame e models ob ained. The models used a e app oxima ions o he in insic pa ame e s beha io as unc ion o he zoom se ing, hey’ e bound o hei ela i e accu a eness and he e o e he e o s made du ing he model calcula ions a e p opaga ed o his expe imen . The ac ha , in he small-scale scena io o his expe imen , he dis ega d o some dis o ion pa ame e s did no p esen a g ea in luence in he esul s, a a la ge scale hey undoub edly in luence he esul s. The second ac o which a ec s he geo-loca ion algo i hm is he heigh di e ence be ween he al i- ude abo e a ge and he al i ude abo e g ound. When hese al i udes a e simila he e a e no signi - ican e o s. Howe e , when he heigh di e ence inc eases, he posi ioning e o s inc ease, meaning ha he p oposed geo-loca ion solu ion is no sui able o be employed in ugged e ain we e signi ican geological o ma ion such as moun ains and alleys would esul in a signi ican loss o accu acy. The hi d ac o which in luences he geo-loca ion algo i hm pe o mance is he a ge dis ance in ela ion o he came a, whe e o highe dis ances he e o is consequen ly highe . The inc easing a ge dis ance esul s in a inc easing image obliqueness, whe e images cap u ed a a mo e oblique posi ion co e a bigge a ea. The e o e, he u he a a ge is loca ed he mo e in luence he pixel loca ion o he a ge has in he inal es ima e o a a ge coo dina es. The esul s ob ained a e encou aging. Howe e , hey poin some ac o s which much be aken in o conside a ion o u u e es s. The ac ha he scena io o ope a ion o he geo-loca ion me hod de- eloped is a ma i ime en i onmen , whe e he heigh di e ence be ween he a ge al i ude and he g ound (mean sea le el) al i ude is negligible, mi iga es he e ec ha he heigh di e ence in luences he p oposed me hod. The a ge dis ance a which he a ge is poin s o a ca e ul selec ion o he ligh pa h o he ai c a . In his, he lowe he mo e oblique an image is he mo e p omp he loca ion e o s a e, meaning ha an al i ude/dis ance comp omise mus be e i ied which minimizes he e o . 78 Chap e 5 Conclusions In conclusion o his hesis, his chap e desc ibes he main achie emen s, sugges ions o opics o u he esea ch and he conclusions d awn om he esea ch conduc ed. 5.1 Achie emen s The came a in insic and dis o ion pa ame e s h oughou i s zoom band we e ob ained using a s a e-o - he-a came a calib a ion echnique, whe e he assump ion ha he came a cha ac e is ic pa- ame e s a e in e dependen wi h lenses con igu a ion was alida ed. The beha io o he came a pa- ame e s was app oxima ed in se e al polynomial app oaches and alida ed in a con olled en i onmen . Using he came a models which shown mo e p omising esul s in labo a o y en i onmen , a ull- scale expe ience o he geo-loca ion algo i hm was conduc ed. In his expe imen a a ge was placed in se e al loca ions ac oss he Po uguese Ai Fo ce Academy campus. The analyses o he esul s o his expe imen we e done conside ing a numbe o scena ios which in luence he es ima es o a a ge posi ion. A geo-loca ion algo i hm was designed acco ding o he scena io o employmen o he p ojec . The me hod de ised o geo-loca e a ge s is a s and-alone, ins an and online solu ion o de e mine an image a ge geode ic coo dina es. This geo-loca ion me hod only depends on he knowledge o he came a pa ame e s model, he came a and ai c a on-boa d senso s. The geo-loca ion algo i hm was design so ha i s in eg a ion in he Seagull so wa e a chi ec u e could be done wi hou any majo modi ica ions o upg ades o i s s uc u e. 79 5.2 Fu u e Wo k Fu u e wo k ha would imp o e he esea ch p esen ed in his hesis and imp o e he capabili ies o he Seagull p ojec include: 1. Design o a il e ing solu ion o minimize he esul s dispe sion o he geo-loca ion algo i hm; 2. S udy o he ligh pa e n which minimizes he geo-loca ion e o ; 3. Accu a e e alua ion o he a ge geo-loca ion using a s e eo came a se -up; 4. Gimbal con ol loop o au oma ic a ge acking; 5. De elopmen o a coope a i e geo-loca ion sys em allowing mul iple pla o ms o simul aneous loca e a a ge . 5.3 Conclusions In summa y, his hesis has p esen ed a me hodology o geo-loca ing an unknown, uncoope a i e a ge objec in he ma i ime en i onmen . In he con ex o he Seagull p ojec , his wo k ep esen s an ex ension o he capabili ies al eady in exis ence. This hesis p esen s a came a calib a ion me hodology, based on s a e-o - he-a echniques, o came as wi h a iable zoom. Expe imen s ha e p o en he alidi y o he me hod p oposed howe e , a majo d awback is ha his me hod is a ime-consuming p ocess. I was no e alua ed i he e ec- i eness o he p oposed me hod is alid o came as wi h mo e complex lenses assembly o came a de ices han he one used in his wo k. The ocus se ing was b ie ly s udied and he da a p o ed he possibili y o neglec ing i , wi hou losing igo in he s udy. Howe e , he e ec s o he ocus se ing can be inco po a ed bu , wi h di e en in e dependence models. The wo k p esen ed in his documen shown expe imen al esul s which allowed o demons a e an in e dependence be ween he came a zoom and i s in insic and dis o ion pa ame e s, whe e i was possible o de ine se e al polynomial models o he pa ame e s h ough he zoom band. The expe i- men s esul s lead o he conclusion ha a 14 h deg ee polynomial app oach o he ocal dis ances and a16 h deg ee polynomial app oach o he dis o ion pa ame e held he bes esul s. The p oposed geo-loca ion me hod was de ised acco ding o he en i onmen o ope a ion o he ai c a . Tes s in a con olled en i onmen e eal he sus ainabili y and alidi y o he algo i hm imple- men ed. Howe e , he loca ion e o s inc eased wi h he dis ance o he a ge loca ion in ela ion o he came a posi ion. An expe imen in eal wo ld condi ions also e i ied his ac , we e he e o s we e consequen ly highe gi en he a ge dis ances conside ed. In a eal wo ld scena io, se e al sou ces o e o we e iden i ied: wind, he inaccu a e measu e- men o he al i ude abo e a ge and he a ge ho izon al dis ance o he came a. The wind p oduced oscilla ions in he came a moun leading o image noise (in some cases unusable images). The di e - ence be ween he al i ude abo e a ge and he al i ude abo e g ound ha e shown o play an impo an 80 ole and e eal ha he solu ion p oposed is no sui able o be employed in i egula e ain. The a ge g ound dis ance o he came a is one o he ac o which in luences he esul s. The highe he dis ance a a ge is loca ed, he highe he image obliqui y and he mo e p omp he a ge es ima es a e o loca ion e o s. In conclusion, he wo k p esen ed in his disse a ion de ails a possible me hodology o geo-loca e a ge s wi h a calib a ed monocula came a, which shows p omising esul s. The came a calib a ion me hod employed held good esul s as well as he geo-loca ion algo i hm, unde a con olled en i on- men . Unde eal-wo ld condi ions, he me hod p oposed in his hesis held sa is ac o y esul s as a s and-alone ins an online solu ion. Howe e , ce ain sou ces o e o whe e e i ied and highligh ed because o hei e ec in a ec ing he esul s. The esul s ob ained in his wo k a e good indica o s ha he p oposed app oach is sui able o be mo e ca e ully s udy, and de eloped o inc ease he Seagull p ojec capabili ies. This in u n can help he Ai Fo ce Academy in de eloping a sus ainable solu ion o Ai Fo ce obliga ions o : con ol and su eillance in ma i ime en i onmen . 81 82 Bibliog aphy [1] Es a ´ egia nacional pa a o ma 2013-2020. Technical epo , Go e no de Po ugal, 2013. [2] Concei o es a ´ egico de de esa nacional. Technical epo , Go e no de Po ugal, 2013. [3] A. Be na dino, N. Fe ei a, C. Dias, F. Nunes, G. C uz, and J. Viegas. SEAGULL - Rela ´ o io Es ado da A e. Technical epo , C i ical So wa e S.A., 2013. [4] G. C uz and A. Be na dino. Ae ial de ec ion in ma i ime scena ios using con olu ional neu al ne - wo ks. In In e na ional Con e ence on Ad anced Concep s o In elligen Vision Sys ems, 2016. Blanc-Talon J. and Dis an e C. and Philips W. and Popescu D. and Scheunde s P. (eds) Ad anced Concep s o In elligen Vision Sys ems ACIVS 2016. Lec u e No es in Compu e Science, ol 10016, Sp inge , Cham. [5] R. Tsai. e sa ile came a calib a ion echnique o high-accu acy 3d machine ision me ology using o - he-shel came as and lens. In IEEE Jou nal o Robo ics and Au oma ion, 1987. [6] J. Heikkil¨ a. Geome ic came a calib a ion using ci cula con ol poin s. In EEE T ans. Pa e n Anal. Machine In ell, page 1066–1077. [7] Z. Zhang. Flexible came a calib a ion by iewing a plane om unknown o ien a ions. In The P o- ceedings o he Se en h IEEE In e na ional Con e ence on Compu e Vision, 1999. [8] J. Sal i, X. A mangue, and J. Ba lle. A compa a i e e iew o came a calib a ing me hods wi h accu acy e alua ion. In Pa e n Recogni ion, pages 1617–1635, 2002. [9] W.Sun and J. R. Coope s ock. An empi ical e alua ion o ac o s in luencing came a calib a ion ac- cu acy using h ee publicly a ailable echniques. In Machine Vision and Applica ions, page 51–67, 2006. [10] R. Madison, P. DeBi e o, A. Olean, and M. Peebles. Ta ge geoloca ion om a small unmanned ai c a sys em. In IEEE Ae ospace Con e ence, Big Sky, Mon ana, Ma ch 2008. [11] D. B. Ba be , J. D. Redding, T. W. McLain, R. W. Bea d, and C. N. Taylo . Vision-based a ge geo-loca ion using a ixed-wing minia u e ai ehicle. In Jou nal o In elligen and Robo ic Sys ems, olume 47, pages 361–382, Decembe 2006. 83 [12] J. A. Ross, B. R. Geige , G. L.Sinsley, J. F. Ho n, L. N. Long, and A. F. Niessne . Vision-based a ge geoloca ion and op imal su eillance on an unmanned ae ial ehicle. In T. P. S. Uni e si y, edi o , AIAA Guidance, Na iga ion, and Con ol Con e ence, Haway, Augus 2008. [13] R. Ha ley and A. Zisse man. Mul iple View Geome e y in compu e ision. Camb idge Uni e si y P ess, 2003. [14] D. Fo sy h and J. Ponce. Compu e Vision: A Mode n App oach. NJ: P en ice Hall, 2003. [15] H. Schul z, A. Hanson, E. Riseman, F. S olle, and Z. Zhu. A sys em o eal- ime gene a ion o geo- e e enced e ain models. In SPIE Enabling Technologies o Law En o cemen , Bos on, 2000. [16] R. Kuma , S. Sama aseke a, S. Hsu, and K. Hanna. Regis a ion o highly-oblique and zoomed in ae ial ideo o e e ence image y. In P oceedings o he IEEE Compu e Socie y Compu e Vision and Pa e n Recogni ion Con e ence, 2000. [17] A. Be na dino, R. Ba is a, G. C uz, T. Oli ei a, and R. Ribei o. SEAGULL - Algo i mos de Payload e Documen ac¸ ˜ ao Cien ´ ı ica. Technical epo , C i ical So wa e S.A, 2013. [18] D. Salguei o, G. C uz, J. Viegas, P. Sil a, R. Ba is a, T. Oli ei a, and R. Ribei o. SEAGULL - Especi icac¸ ˜ ao do So wa e Emba cado. Technical epo , C i ical So wa e S.A, 2014. [19] A. Be na dino, G. C uz, J. Viegas, N. Fe ei a, P. Sil a, R. Ba is a, and S. Fe ei a. SEAGULL - Especi icac¸ ˜ ao da A qui ec u a do So wa e. Technical epo , C i ical So wa e S.A, 2013. [20] Abou os, 2017. URL h p://www. os.o g/abou - os/. [21] R. W. Bea d and T. W. McLain. Small Unmanned Ai c a Theo y and P ac ice. P ince on Uni e si y P ess, 1nd edi ion, 2011. ISBN:978-0-691-14921-9. [22] Open C . Came a Calib a ion and 3D Recons uc ion, 2017. URL h p://docs.openc .o g/2.4/ modules/calib3d/doc/came a_calib a ion_and_3d_ econs uc ion.h ml. [23] M. H. Juayang Weng, Paul Cohen. Came a calib a ion wi h dis o ion models and accu acy e alu- a ion. IEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1992. [24] M. Heikkinen. Cgeschlossene o meln zu be echnung aumliche geoda ische koo dina en aus ech winkligen koo dina en. Zei sch i u Ve messungswesen, 1982. [25] P. S u m. Sel -calib a ion o a mo ing zoom-lens came a by p e-calib a ion. Image and Vision Compu ing, 1997. [26] Open C - u o ial. Came a Calib a ion and 3D Recons uc ion, 2017. URL h ps://docs.openc . o g/3.1.0/dc/dbb/ u o ial_py_calib a ion.h ml. [27] Ma lab. Came a Calib a ion Toolbox o Ma lab, 2017. URL h p://www. ision.cal ech.edu/ bougue j/calib_doc/. 84 Appendix A Came a Calib a ion Resul s me ged Page 9 Zoom Au o-Focus Fixed Focus Au o-Focus Fixed Focus Au o-Focus Fixed Focus RMS s d RMS s d x s d x s d k1 s d k1 s d 0,00 0,96 0,24 0,96 0,24 641,59 5,66 657,49 298,08 -0,32 0,05 0,71 1,85 2,41 0,71 0,13 0,72 0,14 676,33 4,63 782,63 317,70 -0,29 0,04 1,33 2,13 3,14 0,81 0,07 0,81 0,07 647,80 5,23 671,01 282,96 -0,37 0,02 1,62 1,99 3,44 0,70 0,05 0,72 0,22 678,88 4,45 722,67 313,06 0,08 2,42 1,65 3,07 5,44 0,83 0,07 0,85 0,19 674,10 5,70 779,49 271,84 -0,18 0,77 1,07 1,84 6,81 0,63 0,05 0,65 0,14 697,50 2,56 718,11 303,82 0,08 1,98 1,47 2,74 7,48 0,63 0,10 0,64 0,13 688,66 5,16 785,11 313,27 -0,28 0,07 1,10 1,68 8,55 0,91 0,11 0,92 0,13 694,67 4,28 750,27 288,44 -0,27 0,08 0,91 1,61 12,95 0,62 0,10 0,62 0,10 729,44 3,09 874,82 307,18 -0,27 0,03 1,45 2,10 13,93 0,78 0,07 0,80 0,16 740,13 6,12 784,80 306,48 -0,12 0,91 0,91 2,02 14,48 0,68 0,07 0,70 0,15 733,16 3,20 819,99 266,06 -0,18 0,09 0,98 2,09 17,00 0,75 0,07 1,10 0,16 756,24 5,06 882,24 316,87 -0,22 0,04 2,06 2,60 18,02 0,64 0,09 1,01 0,17 761,04 3,61 825,54 327,67 -0,18 0,15 2,61 2,61 19,91 0,73 0,05 1,10 0,12 769,29 2,36 851,39 295,74 -0,13 0,04 2,53 2,47 22,42 0,64 0,05 1,01 0,10 799,32 2,92 902,80 58,55 -0,14 0,04 2,87 2,60 22,93 0,77 0,06 1,13 0,17 793,13 2,24 883,54 55,02 1,10 8,82 3,96 8,87 23,43 0,72 0,06 1,08 0,09 806,18 2,95 906,60 59,67 -0,20 0,03 2,55 2,59 23,93 0,66 0,06 1,02 0,13 811,50 1,96 912,48 61,23 -0,17 0,04 2,38 2,71 26,68 0,63 0,08 0,98 0,12 829,63 2,22 928,46 58,56 -0,12 0,06 2,76 2,83 28,38 0,81 0,06 1,16 0,11 848,15 2,85 948,12 47,09 -0,16 0,08 2,28 2,88 29,57 0,81 0,06 1,18 0,14 867,19 2,05 966,62 57,12 -0,01 0,06 2,72 2,40 33,91 0,66 0,05 0,99 0,11 910,36 4,61 996,63 55,20 -0,13 0,06 2,65 2,71 35,19 0,88 0,08 1,25 0,14 943,21 4,25 1043,48 63,23 0,34 2,87 3,48 3,40 38,89 0,91 0,06 1,26 0,10 989,51 3,23 1088,42 58,13 -0,20 0,06 2,82 2,71 42,22 0,72 0,06 1,08 0,15 1040,49 3,23 1136,62 56,09 0,00 0,07 3,51 2,42 42,93 0,72 0,07 1,07 0,11 1046,09 5,63 1158,44 53,87 -0,16 0,09 2,38 2,39 44,96 1,07 0,07 1,42 0,10 1096,60 5,48 1199,84 51,18 -0,14 0,28 2,15 2,68 47,89 0,74 0,04 1,08 0,11 1170,92 6,17 1267,67 55,84 0,10 0,12 3,52 2,45 Table A.1: Came a calib a ion expe imen esul s. 85 me ged Page 10 Zoom Au o-Focus Fixed Focus Au o-Focus Fixed Focus Au o-Focus Fixed Focus RMS s d RMS s d x s d x s d k1 s d k1 s d 51,35 0,87 0,05 1,67 0,21 1227,62 8,59 1816,11 177,42 -0,18 0,12 4,84 3,41 55,19 0,57 0,04 1,46 0,21 1276,39 4,56 1874,81 176,18 0,38 0,60 2,94 3,29 56,49 0,86 0,03 1,65 0,20 1368,00 9,95 1932,40 184,86 0,40 0,44 4,63 3,83 58,52 0,79 0,14 1,65 0,24 1408,63 6,09 2012,75 173,33 0,36 0,38 3,97 3,92 59,48 0,76 0,04 1,62 0,23 1496,96 10,76 2145,06 159,78 0,04 0,26 3,59 4,15 62,96 0,95 0,06 1,78 0,24 1603,92 11,54 2195,94 167,78 0,35 0,22 3,92 3,78 63,86 0,82 0,08 1,69 0,24 1609,58 10,52 2196,97 195,87 0,45 0,14 4,25 3,95 64,81 0,85 0,04 1,72 0,23 1672,83 8,89 2305,37 169,76 0,41 0,21 4,94 3,87 66,79 0,92 0,03 1,75 0,24 1753,88 12,83 2371,41 166,40 0,23 0,23 3,98 3,76 69,00 0,97 0,05 1,76 0,23 1819,94 10,18 2461,24 161,19 0,74 0,39 3,60 3,83 70,95 0,69 0,04 1,57 0,23 1968,26 12,49 2885,91 200,67 0,55 0,52 5,35 4,24 72,70 0,95 0,06 1,85 0,23 2095,07 11,66 2962,36 209,03 0,45 0,39 4,27 3,90 74,28 0,88 0,04 1,75 0,23 2277,68 20,86 3228,16 217,25 1,49 0,38 6,84 4,36 75,93 0,76 0,06 1,61 0,20 2370,59 20,29 3271,41 211,49 0,35 0,36 4,95 3,81 77,73 0,79 0,06 1,61 0,20 2525,98 18,47 3404,98 206,86 0,71 0,28 4,97 4,05 79,31 0,72 0,04 1,56 0,21 2742,69 42,84 3650,03 228,76 1,25 0,42 6,45 3,55 80,97 0,78 0,03 1,60 0,26 2903,41 36,16 3838,18 206,87 0,91 0,31 5,24 3,51 83,66 0,85 0,05 1,68 0,25 3619,63 74,04 4529,44 227,36 1,52 0,43 6,21 3,67 85,03 0,72 0,05 1,59 0,24 3703,18 68,23 4585,97 220,23 3,09 0,84 6,62 3,93 86,78 0,66 0,03 1,53 0,22 3873,30 95,98 4773,75 212,14 3,54 1,33 7,88 4,05 88,23 0,65 0,03 1,47 0,25 4236,55 75,96 5121,67 232,77 3,70 0,78 7,68 4,45 89,05 0,87 0,05 1,72 0,21 4171,98 72,21 5072,86 194,08 2,20 0,59 6,74 3,74 90,15 0,73 0,07 1,61 0,21 4446,28 111,26 5369,65 221,96 3,65 1,03 7,35 3,99 91,09 0,97 0,05 1,81 0,24 5010,19 47,92 5901,55 223,48 4,63 1,07 8,31 4,17 92,34 0,87 0,05 1,75 0,25 5420,80 161,09 6310,01 204,38 3,03 1,18 8,17 3,65 93,37 0,86 0,05 1,66 0,22 5647,84 110,81 6565,24 239,91 4,15 1,38 8,93 3,87 94,23 0,80 0,06 1,59 0,20 6238,81 215,59 7162,70 327,60 5,26 1,53 8,64 3,56 96,07 0,86 0,05 1,71 0,21 7479,63 199,67 8384,72 290,97 7,06 1,30 11,38 4,25 96,76 1,04 0,07 1,87 0,22 8306,09 194,80 9217,11 306,22 12,40 1,66 16,39 4,18 97,34 0,74 0,05 1,59 0,21 8453,49 222,27 9336,53 324,90 13,88 2,92 18,33 4,73 97,88 0,80 0,05 1,63 0,26 10562,62 687,79 11410,57 721,43 13,80 2,44 19,02 4,25 98,45 0,89 0,06 1,71 0,20 10306,27 291,06 11155,54 342,70 21,22 3,41 24,68 4,56 98,62 0,81 0,05 1,65 0,19 11227,40 541,65 12110,01 576,29 21,62 3,13 26,30 5,12 100,00 1,03 0,07 1,90 0,20 15146,08 1060,28 16070,25 1029,85 61,02 10,99 65,59 12,10 Table A.2: Came a calib a ion expe imen esul s (con inua ion). 86 Appendix B Geo-loca ion E o s Figu e B.1: Came a and a ge GPS coo dina es ixed. Es ima ion using al i ude abo e a ge o each loca ion. 87