Repositorio Institucional de Documentos

Proyecciones C´onicas de Rectas en Sistemas Catadi´optricos para Percepci´on Visual en Entornos Construidos por el Hombre Jes´us Berm´udez Cameo Director: Jos´e Jes´us Guerrero Campo M´aster en Ingenier´ıa de Sistemas e Inform´atica Departamento de Inform´atica e Ingenier´ıa de Sistemas Centro Polit´ecnico Superior Universidad de Zaragoza Septiembre de 2011 Resumen Los sistemas de visi´on omnidireccional son dispositivos que permiten la adquisici´on de im´agenes con un campo de vista de 360oen un eje y superior 180oen el otro. La necesidad de integrar estas c´amaras en sistemas de visi´on por computador ha impulsado la investigaci´on en este campo profundizando en los modelos matem´aticos y la base te´orica necesaria que permite la implementaci´on de aplicaciones. Existen diversas tecnolog´ıas para obtener im´agenes omnidireccionales. Los sistemas catadi´optricos son aquellos que consiguen aumentar el campo de vista utilizando espejos. Entre estos, encontramos los sistemas hipercatadi´optricos que son aquellos que utilizan una c´amara perspectiva y un espejo hiperb´olico. La geometr´ıa hiperb´olica del espejo garantiza que el sistema sea central. En estos sistemas adquieren una especial relevancia las rectas del espacio, en la medida en que, rectas largas son completamente visibles en ´unica imagen. La recta es una forma geom´etrica abundante en entornos construidos por el hombre que adem´as acostumbra a ordenarse seg´un direcciones dominantes. Salvo construcciones singulares, la fuerza de la gravedad fija una direcci´on vertical que puede utilizarse como referencia en el c´alculo de la orientaci´on del sistema. Sin embargo el uso de rectas en sistemas catadi´optricos implica la dificultad a˜nadida de trabajar con un modelo proyectivo no lineal en el que las rectas 3d son proyectadas en c´onicas. Este TFM recoge el trabajo que se presenta en el art´ıculo ”Significant Conics on Catadioptric Images for 3D Orientation and Image Rectification” que pretendemos enviar a ”Robotics and Autonomous Systems”. En ´el se presenta un m´etodo para calcular la orientaci´on de un sistema hiper-catadi´optrico utilizando las c´onicas que son proyecciones de rectas 3D. El m´etodo calcula la orientaci´on respecto del sistema de referencia absoluto definido por el conjunto de puntos de fuga en un entorno en que existan direcciones dominantes. Inicialmente se presenta un nuevo enfoque para extraer c´onicas significantes en la imagen cruda que corresponden a proyecciones de rectas en la escena. Usando la calibraci´on interna y dos puntos de la imagen se pueden calcular anal´ıticamente estas c´onicas. El enfoque propuesto trabaja directamente con c´onicas en la imagen cruda a diferencia de otros m´etodos existentes [16, 19, 9, 4] que realizan el ajuste en la esfera unitaria alrededor del origen del sistema de referencia. Esto permite plantear criterios de decisi´on con umbrales definidos en pixeles. Un ejemplo es la distancia m´etrica propuesta para decidir si un punto pertenece a una c´onica en el algoritmo RANSAC de extracci´on. Tambi´en se ha desarrollado un exhaustivo an´alisis de los elementos que pueden afectar a la precisi´on en la extracci´on de estas c´onicas. El m´etodo se ha validado mostrando la influencia de cada uno de los par´ametros de calibraci´on y de la longitud de la c´onica en la definici´on de la misma. Una vez que las c´onicas significantes han sido extra´ıdas, se explota la informaci´on que contienen para calcular la orientaci´on del sistema. En concreto, se aprovecha la presencia de rectas paralelas en entornos fabricados por el hombre para calcular los puntos de fuga dominantes en la imagen omnidireccional. Para obtener la intersecci´on de dos de estas c´onicas significantes se analiza el triangulo com´un auto polar. Con la informaci´on contenida en los puntos de fuga se puede obtener la orientaci´on tridimensional del sistema catadi´optrico. El m´etodo se ha validado experimentalmente utilizando im´agenes sint´eticas y reales. Para la validaci´on con im´agenes reales el sistema catadi´optrico se ha acoplado a una cabeza rotatoria de precisi´on goniom´etrica. La lectura del goni´ometro se ha comparado con los resultados del algoritmo observando un error absoluto medio de orientaci´on inferior al grado. La validaci´on experimental se ha completado con la utilizaci´on del rectificaci´on de secuencias de im´agenes. El resultado de esta rectificaci´on es una secuencia de im´agenes en la que ´unicamente se observa traslaci´on conservando la orientaci´on fija respecto de un sistema de referencia absoluto. Adicionalmente se ha comparado la orientaci´on calculada en estas secuencias con una IMU. El presente Trabajo Fin de M´aster es el resultado de mi trayectoria investigadora en el DIIS desde la finalizaci´on de mi proyecto fin de carrera en el a˜no 2009. En el a˜no 2008 inici´e mi actividad investigadora en el departamento gracias a una beca de colaboraci´on durante el ´ultimo curso de la titulaci´on de Ingenier´ıa Industrial colaborando en la realizaci´on de experimentos de calibraci´on y en la redacci´on del art´ıculo ” Calibration of Omnidirectional Cameras in practice. A Comparison of Methods. ”[13], que ha sido 6 recientemente aceptado en la revista ”Computer Vision and Image Understanding”. En Junio de 2009 present´e mi PFC titulado ”Calibraci´on de Sistemas Catadi´optricos de Visi´on Omnidirecional” . Continu´e la l´ınea de trabajo durante el verano de 2009 y parte del curso lectivo siguiente iniciando los estudios de M´aster en Ingenier´ıa de Sistemas e Inform´atica. El resultado de esta colaboraci´on acaba reflej´andose en un art´ıculo aceptado en el ICRA 2010 ”Self-orientation of a hand-held catadioptric systems in manmade environments”[12]. Posteriormente, me incorporo como becario en el Centro de Dise˜no de Producto Mecatr´onico del Instituto Tecnol´ogico de Arag´on centrando mi actividad en el desarrollo de software y electr´onica en sistemas empotrados para redes de sensores. Desde entonces, he compaginado mi labor de desarrollo electr´onico en el ITA con la investigaci´on en sistemas omnidireccionales en el DIIS y el I3A. Los resultados de esta etapa se concretan en el trabajo presentado en el 10th Omnivis ”Line extraction in central hyper-catadioptric systems”[5] , que se celebr´o en el RSS 2010, y en el trabajo posterior que concluye con el art´ıculo que sirve de referencia en este TFM titulado ”Significant Conics on Catadioptric Images for 3D Orientation and Image Rectification” que pretendemos enviar pr´oximamente a la revista Robotics and Autonomous Systems. Contents 1 Introduction 9 1.1 Contributions............................................ 10 2 Projections of Lines in Central Catadioptric Systems 11 3 Catadioptric Image Lines Computing 13 3.1 Significant conic definition using two points . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3.2 Distance from a point to a conic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.3 Catadioptric Line Images Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4 Influence of calibration and observed length on the extraction of CILs 19 4.1 Influenceofcalibration....................................... 19 4.2 Influence of observed length of the CIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 5 Vanishing Points and Image Rectification 23 5.1 Intersection of Two CILs Using the Common Self-polar Triangle . . . . . . . . . . . . . . . 23 5.2 Vertical Vanishing Point (VVP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.3 Horizontal Vanishing Point (HVP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.4 Computing the Orientation from VPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 6 Experiments 27 6.1 OrientationAccuracy ....................................... 27 6.2 Rectification of Image Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 7 Conclusions 35 7 8CONTENTS Chapter 1 Introduction Omnidirectional cameras are devices designed to capture images with a wide field of view. This characteristic introduces a new approach in computer vision minimizing the possibility of fatal occlusions and helping the tracking of features. Among these cameras we find the catadioptric systems, which are a combination of a mirror and a camera. Many of these systems conserve the central single view property which helps to obtain a geometric reconstruction based on the triangulation of the light rays from multiple views. In [1] an analysis of central catadioptric systems is presented and it describes when they have the single view point property. Among these we have the hyper-catadioptric system which is composed of a hyperbolic mirror and a perspective camera. In robotics when a central catadioptric system is used it is commonly observed that it has a vertical orientation. This is because most robotic platforms used are wheel-based. Under this configuration planar-motion and 1D image geometry are assumed to simplify the problem. In applications where line tracking or line matching is performed [11, 7, 14] this assumption is useful because vertical lines become straight radial lines in the image. When this assumption can not be satisfied all lines present in the scene become conics in catadioptric images. This general situation require the development of new algorithms which allow to deal with these conic projections and to interact with the environment. One of the advantages of omnidirectional catadioptric systems is the visible length of straight lines projected on the image as a consequence of their wide field of view. When we use catadioptric systems in man-made environments we can observe sets of parallel and orthogonal lines. These sets of lines encapsulate geometrical information which we exploite in this work. In particular, vanishing points contains the orientation of the camera with respect to the coordinate reference system defined by the main directions of the environment. However, dealing with line projections in catadioptric images becomes extraction of conics. In general, five points are required to determine uniquely a conic. When the internal calibration of the central catadioptric system is known, only two points are needed to compute these significant conics, which we particularly call catadioptric image lines (CILs). Catadipotric image lines and other significant conics have previously studied for many purposes. In [20, 17] conic projections of lines are used to estimate the intrinsic calibration. In [18] significant conics which are projections of spheres are also used to obtain the camera calibration. However, main efforts dealing with conics in catadioptric images have been focused on line extraction. In [16], the space of the equivalent sphere which is the unified domain of central catadioptric sensors combined with the Hough transform is used. In [19] the authors also use the Hough transform and two parameters on the unitary sphere to detect the image lines. The accuracy on the detection of these two approaches depends on the resolution of the Hough transform. The higher the accuracy, the more difficult it is to compute the lines. In [9] the randomized Hough transform is used to overcome the singularity present in [16, 19], which speeds up the extraction of the conics. This scheme is compared in converge mapping to a RANSAC approach. In [4] an scheme of split and merge is proposed to extract the image lines present in a connected component. The connected components are computed in two steps. The first step consists of detecting the edges using 9 16 CHAPTER 3. CATADIOPTRIC IMAGE LINES COMPUTING number of points in the component are smaller than a threshold. In Fig. 3.2 we can observe the three main steps to extract the CILs. A pseudo-code version is presented in Algorithm 1. Algorithm 1 2-point significant conic extraction algorithm Require: Image Ensure: Ωarray edges =Canny(image) boundaries =extractboundaries(edges) for k= 1 to nBoundaries do x=boundaries(k) j= 0 while remaining points/total points > T do xnplane =H−1 cx for i= 1 to nAttemps do xrandom =rand(xnP lane,2) Ωimg(i) = twoP oints2Conic(xrandom) d=dist2conic(Ωimg(i),x) xvote(i) = inliers(x, d, T hreshold) votes(i) = size(xinliers(i)) end for indmax =MaximumV oted(Ωimg , votes) Ω = Ωimg(indmax) xinliers =xvote(indmax) j=j+ 1 Ωarray(j) = Ω x=UpdateRemainingBoundary(x,xinliers) end while end for Five points vs. Two points For illustration purposes we show in Fig. 3.3 the extraction of three significant conics corresponding to vertical lines. We use conic fitting with five points and the extraction of the same CILs using our 2-point approach. We also show the vertical vanishing point which is used as a measure of quality. All significant conics corresponding to vertical lines must intersect this point. When the generic five point approach is used, the shape of the CILs changes depending on the points used to compute it. It can be either a hyperbola or an ellipse and it can easily change from one to the other. This has a great effect on the accuracy of the feature, specially at points out the extension of the observed points. For example, using conics showed in Fig. 3.3(a) we can not compute the vertical vanishing point by intersection. In the case of the proposed 2-point algorithm significant conics cover with accuracy the points used to compute it, and describe the edge from which the points were extracted. Therefore the three conics intersect in the vanishing point. Another advantage of the 2-point approach is the number of iterations performed inside the voting approach. For instance, using a probability p= 99% of not failing in the random search and 50% of outliers (ε) just 17 iterations are needed to get a result using the proposed approach (2-point) and 146 using the general five point approach. The number of iterations nris given by nr=log(1−p) log(1−(1−ε)k). Discrimination of significant conics Since lines present in the scene become conics in hyper-catadioptric images, one may think that these conics may represent projections of circles or other conic shapes. Our approach is able to distinguish between 3.3. CATADIOPTRIC LINE IMAGES EXTRACTION 17 (a) (b) Figure 3.3: Computing a CIL with (a) using the five point approach. (b) using our approach with only two close points. The central blue point corresponds to the vertical vanishing point. significant conics, which are projections of 3D straight lines, and the rest of the image conics. To illustrate this, we show in Fig. 3.4 the behavior of our approach extracting significant conics in hyper-catadioptric images. We observe a circular contour whose boundary points were given manually. We applied our RANSAC extraction algorithm on this connected component and assuming the contour is composed of significant conics, we extract all the CILs present. We can observe how the algorithm is not able to fit the whole circle with a single significant conic. With our approach we can directly test if a conic corresponds to a significant conic. The definition presented in section 3.1 is not limited to two points. From (3.5) we observe that each point gives a row of a linear homogeneous system. For n points we have M=     ¯x1¯y1−1 β1 ¯x2¯y2−1 β2 . . .. . .. . . ¯xn¯yn −1 βn      .(3.15) This situation allows to fit a significant conic using more than two points but at the same time it allows to distinguish significant conics from other conics. Notice that the rank of the homogeneous matrix Mis two when it represents a significant conic. We count the number of total CILs extracted and the number of CILs that correctly represent lines in the 3D scene in order to validate this approach. For example, in Fig. 3.5(a) we have a total of 144 extracted CILs from which 8 are extracted from the person in the scene and only other 8 CILS are not supported by a 3D straight line although they were detected and connected by Canny. In Fig. 3.5(b) a second image is shown with 125 CILs extracted, where only 4 CILS have not a 3D straight line support. 18 CHAPTER 3. CATADIOPTRIC IMAGE LINES COMPUTING Figure 3.4: Extraction of CILs over a circular contour. Blue points give the connected component. Red points correspond to the CILs detected by RANSAC. The corresponding conics to each segment CIL are shown in green . (a) (b) Figure 3.5: Extraction of CILs from two hyper-catadioptric images. Chapter 4 Influence of calibration and observed length on the extraction of CILs In this section we present an exhaustive analysis of the influence of the calibration parameters of an hypercatadioptric system on the extraction of CILs. We also analyze the influence of the length of the observed segment. This is evaluated considering the position of the two extreme points of a significant conic segment. We use a hyper-catadioptric system2acquiring omnidirectional images with a resolution of 800×600 pixels and calibrated using [10]. 4.1 Influence of calibration In this experiment we consider the following calibration parameters, focal lengths (γx, γy), principal point (u0, v0) and mirror parameter ξ. We modify each parameter independently inside a certain range from which we select 1000 samples. Then, we compute a specific CIL for each value using its two extreme points. We perform this experiment for two different types of CILs, one of which is nearly perpendicular to the optical axis and the other which is nearly parallel (see Fig. 4.1). In all graphics the horizontal axis represent the variation of the calibration parameter and the vertical axis the mean error in pixels of the conic fitting. •Focal length: In Fig. 4.2(a),(b) we show the corresponding conics when the focal lengths (γx, γy) are modified. We observe that these parameters affect their corresponding coordinate of influence xand y. The plot shows that the effect is the inverse for both axis. However, the magnitude is bigger in the case of γy. Lines with a more horizontal component are more affected by the change on the focal length. •Principal point: In Fig. 4.2(c),(d) we observe the influence of the principal point (u0, v0) on the computation of the CILs. When these two parameters are modified we observe a displacement proportional to the distortion added to the corresponding parameter on the corresponding axis. •Mirror parameter: In Fig. 4.2(e) we observe the effect of the mirror parameter ξon the computation of CILs. We observe that both lines approximate a straight line when the value of ξapproximates to zero. This is explained because ξ= 0 represents the pin-hole model which projects lines to lines. Therefore, the closer a significant conic is to a straight line, the less the mirror parameter ξhas an influence. 2http://www.neovision.cz/ 19 20CHAPTER 4. INFLUENCE OF CALIBRATION AND OBSERVED LENGTH ON THE EXTRACTION OF CILS (a) (b) Figure 4.1: Two different types of CILs. (a) Nearly perpendicular to the optical axis and (b) Nearly parallel to the optical axis.. We observe that the fitting error is bigger for the perpendicular line than for the parallel line, except for errors in v0. This is explained since the perpendicular lines are transformed in conics with a higher curvature than the parallel ones, which are mapped to straighter lines. 4.2 Influence of observed length of the CIL In the extraction process of the CILs using RANSAC, two points in a connected component are randomly selected and a CIL is computed. Then it is used to obtain all the points that belong to such CIL. In this experiment we want to observe the influence of the distance between these two defining points on the accuracy of the extracted CIL. As in previous experiment we observe that the influence of the calibration is stronger in perpendicular lines to the optical axis we perform some simulations using this type of line. We try different line lengths, from a few degrees to the longest theoretical line which has an observed extension of 180◦. We also modify the mirror parameter to observe the behavior when we are close to a parabolic mirror. As we expected, we observe that the longer the line is, the better the extraction of the CIL is. In Fig. 4.3(a) we observe such behavior with a mirror parameter of ξ= 0.75. We also observe that when we approach the maximum observed lenght of the line (180◦) the error starts increasing, showing the limit of the longest line that should be used to extract the corresponding CIL. In Fig. 4.3(b) we repeat the experiment, this time using a mirror parameter of ξ= 0.95 close to a parabolic mirror (ξ= 1). We observe that the error increase a lot when we are closer to both the longest line and the parabolic mirror. The para-catadioptric system case has been studied in [2] and it is simpler than the hyper-catadioptric analyzed here. With the information obtained from this experiment we can infer that we should select the two more distant points to extract their corresponding CIL. However, there exist a superior limit that depends on the length of the CIL and the mirror parameter. In practice it is very unlikely to observe a line of such length. Since we use a RANSAC approach, whose principle is to search in the space of solutions minimizing the error, this problematic situation is automatically avoided. 4.2. INFLUENCE OF OBSERVED LENGTH OF THE CIL 21 (a) (b) (c) (d) (e) Figure 4.2: Mean error in pixels of the conic fitting varying the calibration parameters.(a,b) Focal length, γx,γy.(c,d) Principal point, u0,v0.(e) Mirror parameter ξ. 22CHAPTER 4. INFLUENCE OF CALIBRATION AND OBSERVED LENGTH ON THE EXTRACTION OF CILS (a) (b) Figure 4.3: Maximum error in pixels as a function of the extension (in radians) of the observed CIL for mirror parameters (a) ξ= 0.75 and (b) ξ= 0.95. Chapter 5 Vanishing Points and Image Rectification The vanishing points indicate the intersection of image lines corresponding to parallel lines in the scene. In vertical aligned catadioptric systems, vertical lines are radial lines in the catadioptric image. Their intersection point, the vertical vanishing point (VVP), is located close to the image center. When the camera is not vertically aligned, the vertical lines become conic curves as we explained before. One consequence is that the VVP displaces from the image center. Its new location contains important information about the orientation of the camera with respect to the scene. 5.1 Intersection of Two CILs Using the Common Self-polar Triangle In a general configuration, two conics intersect in four points (Fig .5.1). The union of couples of these points define three distinct pair of lines. The intersection of these lines represent the vertices of the selfpolar triangle common to a pair of conics [3]. We have studied the particular case where two CILs intersect, which is a degenerate configuration, since they intersect in two points. As we observe in Fig. 5.2, there exist a line lthat intersects these two points and the origin of the normalized plane. Our goal is to compute this line and from it to extract the two intersections of the conics that correspond to the two points P+ and P−. Let n1= (nx1, ny1, nz1)Tand n2= (nx2, ny2, nz2)Ttwo normal vectors representing the projection of two lines in the scene and ¯ Ω1and ¯ Ω2two conics representing the image lines in the normalized plane. The vertices of the self-polar triangle associated to the pencil ¯ Ω(λ) = ¯ Ω1+λ¯ Ω2satisfy the constraint det(¯ Ω1+λ¯ Ω2) = 0.(5.1) If we develop this constraint we obtain a third order polynomial where just one of the solutions is real and it corresponds to λ1=−n2 z1/n2 z2. So, the null-space of ¯ Ω(λ1) = ¯ Ω1+λ1¯ Ω2is the line l, expressed in a parametric way as l=µ·v=µvx vy=µn2 z2ny1nz1−n2 z1ny2nz2 n2 z1nx2nz2−n2 z2nx1nz1.(5.2) The intersection of this line to both ¯ Ω1and ¯ Ω2gives the two points P+and P−. To obtain them we solve for µin the following equation µ2(c1v2 x+c2vxvy+c3v2 y) + µ(c4vx+c5vy) + c6= 0 (5.3) 23 24 CHAPTER 5. VANISHING POINTS AND IMAGE RECTIFICATION N1 N2 N3 P+- P-- P++ P-+ V2 V1 Figure 5.1: Intersection of two generic conics. V1 V2 l(µ) P+ P- Figure 5.2: Intersection of two significant conics in the normalized plane. and substitute in (5.2). 5.2 Vertical Vanishing Point (VVP) We use a classic algorithm to detect the VVP. Let mbe the number of putative vertical CILs detected in the omnidirectional image and let nitheir corresponding representation in the normalized plane. For every pair of CILs (there is a total of m(m−1)/2 pairs), we compute their intersection as explained above. Then for each line niwe compute the distance to these points. If the line is parallel to that pair of CILs the distance is smaller than a threshold and then that line votes that possible VVP. The most voted point is considered the VVP. A refinement of the estimation can be performed using the plines that voted for the VVP. This refinement can be performed using singular value decomposition to solve a linear system, followed by an optimization process to improve the accuracy. As these steps also increase the computational cost, we decide to avoid them in our final fast implementation. 5.3 Horizontal Vanishing Point (HVP) Once the VVP is extracted we can exploit several properties to compute the horizontal vanishing point (HVP). The VVP ¯xVVP =¯xvx,¯xvydefines a separation between vertical lines and potential horizontal lines. The VVP defines the plane of the horizon, when this plane intersects the unitary sphere it is projected into the horizon conic, which is defined as: Ωv=   χ2¯x2 vx1−ξ2−(χ−ξ)2ξ2χ2¯xvx¯xvy1−ξ2χ¯xvx(χ−ξ) χ2¯xvx¯xvy1−ξ2χ2¯x2 vy1−ξ2−(χ−ξ)2ξ2χ¯xvy(χ−ξ) χ¯xvx(χ−ξ)χ¯xvy(χ−ξ) (χ−ξ)2  ,(5.4) where χ= ξ+r1+(1−ξ2)¯x2 vx+¯x2 vy ¯x2 vx+¯x2 vy+1 . Therefore the HVP must lie in this conic. 5.4. COMPUTING THE ORIENTATION FROM VPS 25 By introducing these constraints we reduce the search space and we impose the main directions to be perpendicular. Notice that using a single image without any assumptions it is impossible to distinguish between the VVP and the HVP. However, some prior knowledge is easy to have in practice. 5.4 Computing the Orientation from VPs Here we explain the relation between the VVP computed in the normalized plane and the orientation of the catadioptric system. Having an absolute reference system defined by the main directions, the orientation of the camera with respect to this system is described by three angles (φ ψ α). Two of them are obtained from the VVP. Writing the VVP in polar coordinates ¯ xvp = (ρv, θv)T(Fig. 5.3(a)) we observe that there exist a relation between the angle θvand the angle ψrepresenting the rotation of the catadioptric system around the z-axis (5.5). The negative angle is produced by the mirror effect which inverts the catadioptric image. ψ=−θv(5.5) We observe that the component ρvis intrinsically related to the angle φand the mirror parameter ξ of the catadioptric system. Since angles φand ψare independent, we consider the case where ψ= 0 (Fig. 5.3(b)). Using (5.2) and (5.3) with a pair of parallel CILs in polar coordinates we compute the following relationship ρv=−sin φ cos φ±ξ,(5.6) selecting geometrically compatible solutions, φcan be isolated resulting in: φ= atan2 (1, ρv) + arccos −ρvξ pρ2 v+ 1!.(5.7) With φand ψangles any image can be transformed to a vertical reference in which VVP lies on the image center (Fig 5.3(c)). We perform the vertical rectification in two steps. The first step undoes a horizontal rotation according to angle ψ(−θvin the image) to an arbitrary central reference (see Fig. 5.3(b)). The second step consist of translating the VVP to the image center through a rotation around the vertical axis by the angle φ, which is computed 5.7 from ρv(in the image) and the mirror parameter ξ(see Fig. 5.3(c)). This procedure is performed by the following equation ¯x0 h=~rotz(ψ)rotx(−ϕ)rotz(−ψ)~−1(¯xh)(5.8) Once we have rectified the HVP to this reference we can compute the full orientation and perform the full rectification of the catadioptric image. We compute its polar coordinates from which we only require the angle component α. With this angle we translate it to a central position. Since a horizontal rotation was performed in the previous step, the position of the HVP has been modified and we have to undo such transformation. From these angles we construct a rotation matrix Rwhich relates the absolute reference system with the camera reference system Xcam =RXabs where R=rotz(ψ)rotx(φ)rotz(α−ψ).(5.9) 32 CHAPTER 6. EXPERIMENTS (a) (b) (c) (d) Figure 6.7: Example of full image rectification. (a) Frame 1 of the sequence 2. (b) Conic extraction using our approach. (c) Putative vertical and horizontal vanishing points. The yellow circles represent the putative vertical vanishing points. The blue ones the putative horizontal vanishing points and the green ones are the intersections points that cannot be consider either vertical or horizontal vanishing points. The white square is the estimated HVP and th black one is the VVP. (d) Full-rectified image. The vertical CILs are shown in white and the horizontal ones in red. See color version. Frame 107 Frame 242 Figure 6.8: Panoramic representation of several full-rectified frames. Vertical lines are shown in white and horizontal ones in red. The horizontal vanishing point is aligned to the image center. 6.2. RECTIFICATION OF IMAGE SEQUENCES 33 (a) (b) (c) (d) (e) Figure 6.9: Image sequence acquired with the system on helmet camera: (a) Example of omnidirectional image, (b) Vanishing Point Extraction, (c) Rectified image using orientation, (d) Acquisition helmet, (e) Comparison between angle φgiven by the accelerometer and the algorithm. Figure 6.10: Example of rectified and unwrapped image using extracted orientation.Helmet and mirror introduce artefacts on occluded zones. 34 CHAPTER 6. EXPERIMENTS Chapter 7 Conclusions We have presented an analysis on the extraction of significant conics in omnidirectional images generated by a calibrated catadioptric system. We use a new approach that requires just two points and the calibration of the system. Working on the image allow us to use an approximation of the metric distance from point to conic measured in pixels. We study the behaviour of the algorithm and how to discriminate between significant conics from others. We show the influence of each particular calibration parameter on the CIL extraction. We also show how the length of the significant conic on the image increase the accuracy of the CIL extraction. We develop the common self-polar triangle approach to the particular case of significant conics intersection. We use a voting approach to select the VPs from the intersections of the significant conics. Orientation of the catadioptric system is estimated relating the location of the vanishing points on the catadioptric image with the orientation angles. To show the effectiveness of this method we perform experiments with synthetic and real images. In particular we test the accuracy of the orientation estimation by using static images, which are acquired using a goniometer. The information given by this device is used as ground truth. We also show the behaviour of the method dealing with image sequences. We show a comparison between the algorithm results and the orientation date extracted from an accelerometer. Using the orientation we perform the vertical rectification of the sequence in order to create images where applications that require the vertical constraint can be used. 35 36 CHAPTER 7. CONCLUSIONS Bibliography [1] S. Baker and S. Nayar. A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vision, 35(2):175–196, 1999. [2] J. P. Barreto and Helder Araujo. Geometric properties of central catadioptric line images and their application in calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8):1327– 1333, 2005. [3] Joao Barreto. General Central Projection Systems: Modeling, Calibration and Visual Servoing. PhD thesis, 2003. [4] Jean Charles Bazin, C´edric Demonceaux, and Pascal Vasseur. Fast central catadioptric line extraction. In IbPRIA ’07: Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II, pages 25–32, 2007. [5] J. Berm´udez, L. Puig, and J. J. Guerrero. Line extraction in central hyper-catadioptric systems. OMNIVIS - 10th Workshop on Omnidirectional Vision, Camera Networks and Nonclassical Cameras , pp8:1-7,June, 2010, 2010. [6] Christopher Geyer and Konstantinos Daniilidis. A unifying theory for central panoramic systems and practical applications. In ECCV (2), pages 445–461, 2000. [7] J. J. Guerrero, A. C. Murillo, and C. Sag¨ues. Localization and matching using the planar trifocal tensor with bearing-only data. IEEE Transactions on Robotics, 24(2):494–501, 2008. [8] B. Magnier, F. Comby, O. Strauss, J. Triboulet, and C. Demonceaux. Highly specific pose estimation with a catadioptric omnidirectional camera. In In IEEE Int. Conf. on Imaging Systems and Techniques (IST’10), Thessaloniki, Grece, July 2010. [9] C. Mei and E. Malis. Fast central catadioptric line extraction, estimation, tracking and structure from motion. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pages 4774–4779, Oct. 2006. [10] C. Mei and P. Rives. Single viewpoint omnidirectional camera calibration from planar grids. In ICPR, pages 3945–3950, 2007. [11] Y. Mezouar, H.H. Abdelkader, P. Martinet, and F. Chaumette. Central catadioptric visual servoing from 3d straight lines. In Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on, volume 1, pages 343–348, Sept.-2 Oct. 2004. [12] L. Puig, J. Berm´udez, and J. J. Guerrero. Self-orientation of a hand-held catadioptric system in man-made environments. 2010 IEEE International Conference on Robotics and Automation (ICRA) pp:2549-2555, Anchorage, Alaska May, 2010. 37 38 BIBLIOGRAPHY [13] L. Puig, J. Bermudez, P. Sturm, and J.J. Guerrero. Calibration of omnidirectional cameras in practice. a comparison of methods. Computer Vision and Image Understanding, 2011. [14] D Scaramuzza, A Martinelli, and R Siegwart. A robust descriptor for tracking vertical lines in omnidirectional images and its use in mobile robotics. International Journal of Robotics Research, 2009. Special Issue on Field and Service Robotics. [15] Peter Sturm and Pau Gargallo. Conic fitting using the geometric distance. In Proceedings of the Asian Conference on Computer Vision, Tokyo, Japan. Springer, 2007. [16] P. Vasseur and E. M. Mouaddib. Central catadioptric line detection. In British Machine Vision Conference, September 2004. [17] Fuchao Wu, Fuqing Duan, Zhanyi Hu, and Yihong Wu. A new linear algorithm for calibrating central catadioptric cameras. Pattern Recognition, 41(10):3166–3172, 2008. [18] Xianghua Ying and Zhanyi Hu. Catadioptric camera calibration using geometric invariants. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10):1260–1271, Oct. 2004. [19] Xianghua Ying and Zhanyi Hu. Catadioptric line features detection using hough transform. In ICPR 2004. Proceedings of the 17th International Conference on, volume 4, pages 839–842, Aug. 2004. [20] Xianghua Ying and Hongbin Zha. Simultaneously calibrating catadioptric camera and detecting line features using hough transform. IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005, IROS 2005., pages 412–417, Aug. 2005.