Novel Aerial Manipulator for Accurate and Robust Industrial NDT Contact Inspection: A New Tool for the Oil and Gas Inspection Industry

sensors Article Novel Aerial Manipulator for Accurate and Robust Industrial NDT Contact Inspection: A New Tool for the Oil and Gas Inspection Industry Miguel Ángel Trujillo 1,*, José Ramiro Martínez-de Dios 2, Carlos Martín 3, Antidio Viguria 1and Aníbal Ollero 2 1 CATEC (Advanced Center for Aerospace Technologies), C/ Wilbur y Orville Wright 19, 41309 La Rinconada, Seville, Spain; [email protected] 2School of Engineering, University of Seville, Avda. Camino de los Descubrimientos, 41092 Seville, Spain; [email protected] (J.R.M.-d.D.); [email protected] (A.O.) 3La Línea Vertical, Camino de la Ermita 10, 11300 La Línea de la Concepción, Cádiz, Spain; [email protected] *Correspondence: matr[email protected] Received: 15 February 2019; Accepted: 10 March 2019; Published: 15 March 2019   Abstract: There is a strong demand in the oil and gas industry to develop alternatives to manual inspection. This paper presents AeroX, a novel aerial robotic manipulator that provides physical contact inspection with unprecedented capabilities. AeroX has a semi-autonomous operation, which provides interesting advantages in contact inspection. In the free-flight mode, the pilot guides the robot until performing contact with its end-effector on the surface to be inspected. During contact, AeroX is in its fully-autonomous global navigation satellite system (GNSS)-free contact–flight mode, in which the robot keeps its relative position w.r.t. the surface contact point using only its internal sensors. During autonomous flight, the inspector can move—with uninterrupted contact—the end-effector on the surface for accurately selecting the points where to perform A-scan measurements or continuous B-scan or C-scan inspections. AeroX adopts an eight-tilted rotor configuration and a simple and efficient design, which provides high stability, maneuverability, and robustness to rotor failure. It can perform contact inspection on surfaces at any orientation, including vertical, inclined, horizontal-top or horizontal-bottom, and its operation can be easily integrated into current maintenance operations in many industries. It has been extensively validated in outdoor experiments including a refinery and has been awarded the EU Innovation Radar Prize 2017. Keywords: aerial; robot; manipulator; contact; inspection; NDT; ultrasonic; oil and gas; UAV 1. Introduction The oil and gas industry has a large variety of procedures to ensure the safety of its facilities and personnel, and at the same time tightly controlcosts. These procedures involve extensive inspections, many of which should be performed at height and using sensors that require to be in contact with the surfaces being inspected. Contact inspection is traditionally performed by technicians accessing to the specific inspection points using man-lifts, cranes or scaffolds, or rope-access techniques. Plant contact inspections involve very high costs for the oil and gas sector. Even a small refinery requires inspecting several thousands of points each year and at least 30% of them are in areas of very difficult access. There is a strong demand in the oil and gas industry to develop alternatives to manual inspection. Sensors deployed at selected locations in the plant, such as in [ 1 , 2 ], provide punctual measurements at a high rate, which is suitable for intensive monitoring of few or small critical components but is not ideal for covering/inspecting a full plant. Crawler robots with magnetic wheels have been Sensors 2019,19, 1305; doi:10.3390/s19061305 www.mdpi.com/journal/sensors Sensors 2019,19, 1305 2 of 24 developed to move on the outer or inner surfaces of pipes or structures [ 3 ]. But they are valid only for ferromagnetic surfaces and often have low accessibility for locations at height or isolated from nearby structures. The aerial robotics solutions for the oil and gas industrial inspections at height have been analyzed in the H2020 EU project AEROARMS (https://aeroarms-project.eu/). Aerial vehicles equipped with robotic arms, which have been called aerial manipulation systems, have demonstrated their capability to grasp and deploy objects, turn valves or assemble structures [ 4 – 6 ]. One of the main research challenges in aerial robotics manipulation is to keep stability during physical contact [ 7 , 8 ]. The aforementioned aerial manipulation robots are very promising but are still not ready or robust for their everyday use in industrial conditions. This paper presents AeroX, a novel aerial robotic manipulator solution for physical contact inspection. It is composed of a robotic vehicle, a six degree-of-freedom (DoF) robotic arm, and a robotic end-effector equipped with wheels and inspection sensors. AeroX provides unprecedented inspection capabilities. Firstly, it can perform contact inspection on surfaces with any position and orientation, including vertical, inclined, horizontal-top or horizontal-bottom. Secondly, the AeroX controller is able to efficiently compensate perturbations thanks to its design, which transmits the surface contact forces and perturbations to the robot center of mass and allows small movements of the aerial part of the robot in every DoF to absorb other perturbations such as wind. AeroX adopts an eight-tilted rotor configuration, which provides high maneuverability, agility, and robustness to rotor failure. Finally but not least, in contrast to existing systems, AeroX has a semi-autonomous operation. During free-flight, the pilot guides the robot and, after contacting the surface with its end-effector, switches to fully-autonomous global navigation satellite system (GNSS)-free contact flight. During contact flight, AeroX flies keeping its position steady w.r.t. the surface contact point with uninterrupted contact, so that when the inspection operator moves the wheels of the end-effector, the aerial robot follows that motions. As a result, AeroX can be easily operated by personnel with low training in robotics and integrated into current maintenance operations in many industries. This paper analyzes the inspection requirements for its use in the oil and gas sector, presents the solution and evaluates its robustness and performance in experiments performed in realistic outdoor scenarios and in a refinery in Germany, see Figure 1. A video of these experiments can be seen at https://youtu.be/Hy45WQ3GLcI . The proposed aerial manipulator has been patented in [ 9 ] and has recently been awarded the EU Innovation Radar Prize 2017 (https://www.innoradar.eu/ innoradarprize). Figure 1. AeroX performing an ultrasonic thickness measurement in a refinery located in Germany in 18 July. The exact location cannot be disclosed due to confidentiality issues. This paper is structured as follows. Section 2summarizes the state of the art of technological solutions for non-destructive testing (NDT) inspection in the oil and gas sector focusing on aerial Sensors 2019,19, 1305 3 of 24 manipulators. The proposed aerial manipulator solution is described in Sections 3and 4. The operation during a typical inspection process is summarized in Section 5. The experimental validation and robustness analysis are presented in Section 6. Section 7summarizes the conclusions and future work. 2. Related Work Periodic inspections are required to ensure suitable and safe operation in many industrial plants. The scope and types of inspections are well described in the plant inspection plan, where requirements are based on the detection and evaluation of the different types of defects and their size. The detection and measuring of defects are needed due to legal requirements and to ensure the plant safety. These defect measurements are analyzed also to obtain trends and predictions, which are used as guidelines for maintenance plans or even plant modifications. The inspection of pipelines and tanks is a very relevant use case in oil and gas plants maintenance. Corrosion is the most common failure in pipes and it could occur inside a pipe, caused by the transported medium, or it could be found outside the pipe due to the surrounded environment, or atmospheric conditions, like rain and humidity. Two of the major problems produced by corrosion are the decrease in wall thickness and the cracking of the pipe. These types of defects are being measured nowadays using sensors that require contact. Ultrasonic inspection is a well-established and standardized method for corrosion detection, wall thickness measurement, and sizing of the defect. Eddy current testing is also applied [ 10 ]. Ultrasonic testing (UT) requires that the sensor is coupled to the component being inspected through a couplant. Eddy current testing does not require the couplant, but it needs to be attached to the scanned surface in order to omit lift-off effects. Plant inspection has been traditionally performed by technicians accessing to the specific inspection points. The inspection points are often located at inaccessible heights and the operator needs to use man-lifts, cranes or scaffolds to have access to their locations. The use of rope-access techniques has been generalized in the oil and gas sector [ 11 ] after the success of the first projects in offshore platforms in the North Sea in the 1970s. In onshore plants, rope-access allows reducing inspection costs by up to 90% when compared to traditional methods (scaffolding, lifting platforms, etc.). In offshore plants, rope-access is the only technically and economically viable solution. Also, placing the inspection transducer at certain positions and scanning along the pipes omitting flanges, nozzles, holding gaskets is costly and time-consuming. It is estimated that a small-sized refinery with one crude unit requires inspecting by ultrasound technique more than 15,000 points each year and at least 30% of them are in areas of very difficult access. Furthermore, the pipes or surrounding objects could have high-temperature surfaces (up to 400 ◦ C) or contain potentially dangerous materials or fluids, substantially increasing the risk of falls or other incidents. There is a strong demand in the oil and gas industry to find alternatives to manual inspection in order to reduce risks and costs. One of the first technologies adopted is based on nodes—with sensors, wireless communications, and integrated battery—deployed at selected locations in the plant [ 1 , 2 ]. These sensors can gather measurements very frequently and are suitable for intensive monitoring of critical components. They operate in an unattended way for long periods of time under potentially bad conditions, originating frequent sensor failures. Manual recalibration or substitution of sensors located at inaccessible locations is dangerous and costly. Some methods, such as [ 12 ], detect the consistency of the measurements of a sensor comparing its measurements to those from nearby sensors and detect the sensor it is faulty or badly calibrated. Sensor-based technology provides punctual measurements and is suitable for intensive monitoring of few or small critical components but it is not ideal for covering a full oil and gas plant. Recent solutions using robots with magnetic wheels have been applied [ 3 ]. They can solve some inspection problems, but usually, have low accessibility to pipes at height. These robots need heavy magnetic wheels, which are difficult to be detached and are valid only for pipes with ferromagnetic surfaces. Other solutions are robots navigating inside the pipes [ 13 ]. They have similar accessibility problems and require having the pipe out of service during the inspection. Sensors 2019,19, 1305 4 of 24 Moreover, robot malfunctioning may result in pipe blockages. Accessibility is the main problem of robot technologies particularly for inspecting locations at a certain altitude and isolated from nearby structures. Aerial robot manipulation is growing quickly in the last years towards a stronger position in robotics. Two DoF robotic arms installed onboard aerial vehicles have performed grasping and deployment of objects [ 4 ] or turning valves [ 5 ]. Multirotor and helicopter aerial robots with onboard six DoFs robotic arms have been developed [ 14 , 15 ]. Aerial manipulators were also been used for assembly of 3D structures [6]. One of the main research challenges in aerial robotics manipulation is to keep stability during physical contact. In [ 7 ] it is shown how keeping contact when the robot is below the contact point improves stability. Compliant articulated robotic arms mounted on board multirotors have been analyzed [ 8 ]. In [ 16 ] Delta manipulators were integrated on a side of the aerial robot and the stability problem was studied, making the interaction with the environment compliant. The aforementioned robots are capable to contact at points located either under or at the front of the robot. In [ 17 ] an aerial vehicle that operates both at the bottom and front is presented. In [ 18 ] aerial manipulators capable to operate in any direction were presented. Stable contact operations have been achieved by a pushing trirotor in [ 19 ] or, a quadrotor in [ 20 ]. In [ 21 ] a long rigid tool exerted forces against a surface. A compliant robot that preserved stability during collision is presented in [ 22 ]. Also, new aerial robots capable of 6D physical interaction [ 23 ] and performing collaborative transportation [ 24 ] were also presented. The great majority of the above robotic solutions were tested only in the laboratory or under very controlled conditions. Although many advances and RnD activities have been performed in aerial manipulator systems in the last years, in general, these systems do not have the necessary robustness and are still far from their everyday application in industrial conditions. Some drones are conceived as robust solutions for the industry, such as [ 25 ] or [ 26 ], but they are not able to perform contact inspections. The first aerial vehicles for contact inspections in industrial plants were announced in 2018. Two good examples are [ 27 , 28 ]. Existing solutions can contact only vertical surfaces—to the best of our knowledge only [ 27 ] can contact also inclined surfaces. In existing systems, the pilot guides the UAV to the surface until it contacts the surface. Once the contact has been established, the contact point cannot be moved by the pilot. Hence, if the pilot did not contact exactly at the desired location, he has to repeat the contact operation. Hence, these systems have very low accuracy selecting the inspection points—it depends on the pilot expertise, and cannot perform continuous inspections such as B-scan or C-scan, which are measuring techniques related to the capture of data while moving continuously over the surface [29]. The main advantages of AeroX over existing systems can be summarized as: • It can perform contact inspection on surfaces at any orientation including vertical, inclined, horizontal-top, and horizontal-bottom. • During contact it has fully-autonomous GNSS-free navigation. It flies with uninterrupted contact keeping steady its position w.r.t. the surface contact point using only its internal sensors. The inspector can move the wheels of the end-effector with uninterrupted contact enabling accurately selecting the inspection point. • It can be operated very easily by two persons; the pilot and the inspector, who do not need to be aware of the aerial part of the system while operating the movement and coupling of the contact sensors. • AeroX has robust and stable flight capabilities. The aerial robot controller is able to efficiently compensate perturbations thanks to its design, which transmits the contacting forces to the robot center of mass and allows small movements of the aerial part of the robot in every DoF to absorb disturbances such as wind. AeroX has an eight-tilted rotor configuration which provides high maneuverability, agility, and robustness to rotor failure. • Its end-effector can be easily interchanged, giving new inspection capabilities to the robotic system. Sensors 2019,19, 1305 5 of 24 3. The Aerial Manipulator Solution The design and development of AeroX were based on requirements driven by the industry inspection end-user needs for giving successful solutions to the industry. The oil and gas industry imposes very tough requirements. The operation of the robotic solution should be robust under nominal working conditions at heights with potential strong wind gusts. Besides, the proposed aerial robotic system should be integrated with the current maintenance operations in many industries and should be easily operated by the personnel that is currently involved in industrial plant inspections, which often have low training in robotics. The main requirements considered in the design of AeroX can be summarized in the following: • It should keep a sensor in steady physical contact at a point on the surface where a measurement is going to be taken. •It should mount a variety of physical contact sensors such as Eddy-current sensors or ultrasonic sensors that measure thickness and corrosion, among others. • The aerial platform should have fast reactivity and controllability in order to fly close to obstacles in constrained environments with wind gusts. • It should be easy to be operated for the personnel currently involved in industry inspection with low training. • Its operation should be easily adaptable to the specific inspection procedures currently used in industries. • The robotic system should be very robust and reliable for everyday operations in industrial settings. The proposed aerial robotic system—the AeroX robot—has three main components, see Figure 2: • The aerial platform. Its design enables applying on the surface the contact forces required for physical inspections. The aerial platform is endowed with tilted rotors, which, as will be described later, increases its stability and reactivity to compensate perturbations. • The robotic contact device. It is responsible for providing the capability of steady contact with surfaces. It is a mechatronic device with six DoF and has the robotic end-effector at its end. Due to its efficient design, surface contact forces are transmitted to the center of mass (CoM) of the aerial robot which enables their efficient and effective compensation. • The robotic end-effector. It is located at the end of the robotic contact device and is endowed with wheels for moving on the surface under inspection. It integrates the sensors to be used for inspection and also additional sensors to facilitate operation, e.g., a camera for helping the operator identify the defects under inspection. A variety of robotic end-effectors can be mounted on the robotic contact device in order to use different sensors, change the motion on the surface of the robotic end-effector or even change the type of docking to the surface. An end-effector with four wheels, an ultrasound sensor, and a gel injection device is presented in this paper. The operation of AeroX has two main modes. In the free-flight mode, the pilot guides the aerial robot (in manual or assisted way) to the element to be inspected and moves the robotic contact device to the selected inspection point until the robotic end-effector is in contact with the surface. As soon as the contact has been performed, the contact-flight mode starts, the pilot activates the fully-autonomous stabilization mode, which keeps the aerial vehicle steady w.r.t. the surface contact point with uninterrupted contact using only the measurements from the robot internal sensors, see Section 4.1. The inspector teleoperates the movement of the wheels of the robotic end-effector on the surface. As a result, when the inspector moves the wheels of the end-effector, the surface contact point changes and the aerial vehicle moves to keep its position steady w.r.t. the surface contact point. Hence, the aerial robot follows the end-effector commanded by the inspector. Some videos of the operation of AeroX can be found in Section 6.3. Sensors 2019,19, 1305 6 of 24 Figure 2. The AeroX robot in physical inspection of a pipe during a validation experiment. 4. Main AeroX Components This section summarizes the main components of AeroX including the aerial platform, the robotic contact device, and the robotic end-effector. 4.1. Robotic Contact Device One of the main components of the proposed aerial robotic system is its novel robotic contact device, whose patent has been granted at the beginning of 2018 [ 9 ]. Due to its mechanical design and integration to the aerial platform, all the surface contact forces are transmitted to the CoM of the aerial robot, which enables simplifying the stabilization and control of the aerial vehicle. With the robotic contact device, the aerial robot is capable of absorbing perturbations, keeping a robust and stable contact perpendicular to the contacted surface. The main characteristics of the robotic contact device are: • Its mechanical design and integration with the aerial platform enable inspecting surfaces with different positions and orientations such as vertical, horizontal and inclined surfaces, see pictures from outdoor experiments in Figure 3. • The robotic contact device transmits all forces directly to the CoM of the aerial vehicle, simplifying its stabilization and control. • All the joints of the robotic contact device are equipped with shock absorbers, which increase perturbation rejection and surface compliance during contact. • The joints of the robotic contact device are equipped with internal sensors. During contact-flight their measurements are used to estimate and control the relative position of the aerial robot w.r.t. the contact point using only its internal sensors and without any external positioning system or sensor. Figure 4b shows a scheme of AeroX with the six joints of the robotic contact device, which provides six DoFs. The batteries of the robot are located (as a counterweight) at the other side of the robotic device in order to equilibrate the torque produced by the weight of the end-effector. The first three joints of the robotic contact device ( θ1 , θ2 and d3 ) follow a polar robot configuration. The first two joints are rotations around the center of the aerial vehicle. Giving positive angles to θ1 would move the end-effector to the top of the body of the aerial vehicle. The second, θ2 , is a rotation that would move the end-effector to the left side of the aerial robot body. The third joint produces a linear motion Sensors 2019,19, 1305 7 of 24 which axis goes through the center of mass (CoM) of the aerial vehicle, which is located exactly at the geometrical center of the aerial robot body, see Figure 4b. The joint variable d3 is defined as the distance between the robot CoM and the axis intersection of the last three joints θ4,θ5and θ6. Figure 3. AeroX in contact inspection of surfaces with different positions orientations in different outdoor experiments and with different end-effectors: inspecting pipes ( bottom-left and bottom-right ), horizontal ceilings (top-right) and vertical surfaces (top-left). The last three joints are rotations located close to the robotic end-effector and enable orientating it perpendicularly to the contact surface. The rotation axes of these last joints coincide at the same point, as shown in Figure 4b. The distance between this point and the surface contact point is denoted as lc. Each of the three first joints of the robotic contact device is equipped with a sensor. During contact, their measurements are used to estimate at a high rate (100 Hz) the position of the aerial robot w.r.t. the surface contact point. This relative position together with the robot orientation measured with the inertial measurement unit (IMU) is used for high-frequency stabilization and hovering, enabling fully-autonomous manipulation operations without relying on any external positioning system. During contact-flights, all joints of the robotic contact device turn into a passive mode (zero torque mode) enabling the aerial vehicle to move freely without friction coming from the joints. Hence, the CoM of the robot receives only a linear force from the surface contact and does not receive torques that could destabilize the aerial robot. The integration of the robotic contact device with the aerial platform is such that during contact-flights the robot can change its pose w.r.t. the robotic end effector a few centimeters in any direction and a few degrees in any orientation, 20 cm and 10 degrees in the prototype used in the experiments. This provides the aerial vehicle capability to reject flight perturbations (e.g., due to wind gusts) during the contact-flight without changing the contact point. Using the joint sensors to estimate the aerial robot pose has clear practical advantages for industrial inspections. Firstly, it enables very high-rate and accurate pose estimations using robust and simple mechanical joints and onboard sensors. The accuracy of the resulting position is similar to that from common high-precision commercial off-the-shelf (COTS) real-time kinematic (RTK)-GPS but Sensors 2019,19, 1305 8 of 24 (a) (b) (c) Figure 4. Different schemes of the proposed aerial robot showing its kinematic representation. ( a ) Drawing of AeroX. Its size is 170 × 230 cm. ( b ) Scheme of the robotic contact device showing its six joints and the reference systems for each DoF. ( c ) Drawing of the aerial robot. The tilted-rotor configuration can be noticed. Each rotor is alternatively tilted a fixed angle β= 1 / 6 rad around its motor boom. Sensors 2019,19, 1305 9 of 24 with significantly higher rates that enable real-time control and also the advantage that it is valid in environments with poor or non-existent GNSS coverage. In the following, we obtain the expressions used to compute the position of the aerial robot w.r.t. the contact point using only its internal sensors. The CoM of the aerial vehicle is located at the geometrical center of the aerial robot body, where the axes θ1 and θ2 coincide, see Figure 4b. A frame is defined for each joint of the robotic contact device, see Figure 4b. The aerial robot coordinate frame, whose origin is at the robot CoM, is denoted as frame 0. The relative pose of the surface contact point w.r.t. frame 0 is expressed as: 0 6T=0 1T1 2T. . . 5 6T, (1) where X YTis the transformation matrix from frame Y to frame X. In many cases lc can be considered negligible due to the low size of the end-effector, so in this case the origin of the frame of joint 3, see Figure 4b, can be taken as the surface contact point. The transformation between the origin of the frame 3 and the aerial robot frame can be expressed as: 0 3T=      cθ1cθ2−cθ1sθ2−sθ1cθ1cθ2d3 sθ2cθ20sθ2d3 sθ1cθ2−sθ1sθ2cθ1sθ1cθ2d3 0 0 0 1      , (2) where sθ stands for sin(θ) and cθ stands for cos(θ) . All operations in each component of 0 3T are trigonometric, hence all involved forces and torques are transmitted directly to the CoM. The first three elements in the last column of 0 3T represent the relative position of the surface contact point w.r.t. the aerial robot frame expressed in the robot frame. It will be called SCPlocal . During the flight, the orientation of the aerial robot (and of its reference frame) might be different from the Earth reference frame. Hence, we need to correct SCPlocal in order to express this translation in Earth axes. The orientation of the aerial robot reference frame w.r.t. the Earth axes can be measured accurately and with a high rate (200 Hz) with the robot IMU and magnetometer. With their measurements, we compute the direct cosine matrix ( DCM ), a rotation matrix that transforms from the aerial robot frame to a frame whose origin is the robot CoM but has Earth axes. Hence, it is easy to note that the relative position of the robot CoM in Earth axes w.r.t. the surface contact point can be computed very efficiently as follows: posRobot =−DCM SCPlocal. (3) Notice that the pose of the aerial robot w.r.t. the surface contact point is fully defined by posRobot and DCM . Hence, the adopted method computes very efficiently and at a rate of 100 Hz the robot pose w.r.t. the surface contact point using only internal sensors. This provides high autonomy, which is of interest for operation in unstructured scenarios such as factories. This robot pose estimation is used for fully-autonomous robot stabilization and control during contact-flights, as described in Section 5.1. 4.2. The Aerial Platform The controllability and reactivity to fly close to obstacles and withstand wind gusts, maneuverability and agility to fly in constrained industrial environments and the robustness for everyday operation have been the main characteristics in the design of the aerial platform. The proposed aerial vehicle has eight 2 kW motors (KDE5215XF-220), providing a maximum take-off weight of 25 kg. The motors are set such that the robotic contact device can rotate around the vehicle CoM without colliding with the propellers. Each set of motor-propellers is alternatively tilted 30 degrees around the motor boom, see Figure 4c. The tilting of the motors are fixed and cannot be controlled. The actuation variables that can be controlled are each of the eight motor desired forces. Tilted rotors enable independent control of the six DoF of the vehicle adding the capability to control Sensors 2019,19, 1305 16 of 24 Figure 8. Picture of the on-board camera used for the visual-based relative localization system. The green rectangles are used to mark the SURF features in FSt that were correctly matched—after executing RANSAC—with features in FSt−1 ,—marked with white rectangles. The matchings (of the end-effector) filtered out by RANSAC are marked with red rectangles. A small white arrow illustrates the relative displacement between the end-effector and the pipe between t−1 and t. 6. Experimental Validation Different sets of experiments were performed to validate the performance of AeroX. Two types of experiments are presented. First, some experiments in outdoor realistic scenarios—including a refinery in Germany—are described to illustrate the operation and performance of the system. Additionally, its robustness against external perturbations was analyzed in laboratory experiments. Finally, Section 6.3 provides links to different videos showing AeroX validation and robustness analysis experiments. 6.1. Validation in Realistic Environments A total of more than 200 different experiments have been performed in outdoor realistic scenarios. Some experiments were performed in a real oil and gas facility in Germany, see Figure 1and Section 6.3. Others, were carried out at a testing scenario near Seville with a real pipe of 6 m of longitude and 0.6 m of diameter that was placed 4 m above the ground level, see Figures 2and 9. In all these experiments AeroX was operated only with a pilot and an inspector non-qualified in robotics. The objective of these experiments was to: (1) validate AeroX as a suitable inspection tool for industry and; (2) validate the AeroX inspection procedure presented in Section 5.2. Below an experiment performed at the testing scenario near Seville in October 2018 is described as an example of all the different experiments that have been performed in outdoor realistic scenarios. A summary video of that experiment is available at https://youtu.be/nvw0I5ux19U. The inspection procedure described in Section 5.2 is followed. The pilot manually performed the take-off of the aircraft and guided it to the pipe. The contact was performed at time 0:28 s of the video. At that time the pilot switched to contact-flight mode and AeroX kept steady its position w.r.t. the contact point using its internal sensors and not the GNSS measurements. At time 0:35 the inspector moved the end-effector to the right. The images captured by the onboard camera can be seen at time 0:37. At time 0:48 the inspector moved the end-effector to the left. The images taken by the onboard camera can be seen at time 0:59. At 1:32 the pilot deactivated the contact-flight mode, took control of AeroX and landed at 1:43 s. Sensors 2019,19, 1305 17 of 24 Figure 9. AeroX during contact flight in an experiment performed in Seville in October 2018. AeroX was flying in autonomous mode and the end-effector was teleoperated by the inspector. Images at different times show the motion of the end-effector along the pipe while keeping the relative position of the aerial vehicle. The resulting telemetry in the above experiment—which took 200 s—is shown in Figure 10. The video only shows a summary of the experiment. Figure 10-left shows the control performance of the aerial robot in terms of position and attitude. Figure 10-left-bottom shows the AeroX positioning error using the proposed controller and the robot internal sensor-based position estimation method developed. The robot RMS positioning control error was of 4.6 cm along the experiment. The maximum instantaneous positioning error measured was 13.8 cm. These results validate the control architecture of AeroX. Figure 10-right-top shows the computed lateral forces ( Fx , Fy and Fz ). It can be noticed that the robot is exerting a lateral force ( Fx ) of about 20 N against the surface under inspection. Figure 10-right-center shows the readings of the sensors in the three first joints of the robotic contact device. Figure 10-right-bottom shows the displacements of the end-effector, first along and then around the pipe, estimated by the method described in Section 5.3. The wind measured during this experiment changed between 0 and 3 m per second. The flight of AeroX was unaffected by these wind speeds. Section 6.2 shows that the operation of AeroX is rather robust against significant external perturbations. Also, preliminary outdoor experiments were performed at the outdoor testing scenario to validate the capability of AeroX to make contact—and inspect—vertical and horizontal surfaces. AeroX was also validated in a real oil and gas plant in Germany in July 2018, whose location is left undisclosed due to confidentiality issues. The main objective was to validate its operation from the inspector and end-user perspectives. The operation of the end-effector was extensively tested on several surfaces with different conditions of dirt, curvature, heights, and accessibility. For all of them, the end-effector was correctly guided by the inspector and UT measurements were taken ( see Figures 5and 7 ) at all the positions selected by the inspector. The performance of the gel dispenser system of the UT probe was validated. The status of the coupling between the UT sensor and the surface under inspection was also checked using both, the small cameras onboard the end-effector and the real-time A-scan measurements that were visualized in the operator screens. These aforementioned experiments validated the operation of AeroX and its inspection procedure in industrial scenarios. AeroX can easily fly in constrained industrial scenarios and the inspector can easily select the contact points to be analyzed. AeroX can fly in a fully-autonomous way during contact without any GNSS system or external sensor. The inspection procedure benefits from the expertise of the pilot and of the inspector and clearly separates their roles and operation preventing that they disturb one another. Sensors 2019,19, 1305 18 of 24 Figure 10. AeroX telemetry during contact-flight in one outdoor experiment performed in the outdoor testing setting near Seville in October 2018. The control responses for attitude and position (blue = reference, green = measured data) as well as the 3D estimated positioning error are shown at the left. The computed linear forces, torques, values of the first joint variables of the robotic contact device are plotted at the right. Linear force Fx is in blue while Fy is in green. In the torque graph, the torques around axes X,Yand Zare respectively in blue, green and red color. The relative displacement of the robotic end-effector along and around the pipe is presented at the left-bottom. 6.2. Robustness Analysis The operation of AeroX should be robust under nominal working conditions at heights with potential strong wind gusts. A series of experiments have been performed to assess the stability and robustness of Aerox to strong flight perturbations. The experiments were performed in the CATEC indoor testbed, a 15 × 15 × 5 m room equipped with a 20-camera VICON motion capture system [ 33 ] capable of providing <1 mm localization accuracy. The VICON system was not used at all in any AeroX component or module, it was used only as ground truth to evaluate its performance. In these experiments, external perturbations were injected in AeroX during contact in fully autonomous flights. The perturbations were rather high and the experiments were very risky. For safety reasons, a safety rope with nearly zero tension (like a safety belt) was attached on the top of AeroX. This safety rope was not involved at all in AeroX flight: the operation of AeroX and its control system was exactly as described above. In these experiments, AeroX performed exactly the aforementioned inspection procedure. The pilot manually guided AeroX to the surface—vertical in Sensors 2019,19, 1305 19 of 24 these experiments—in free-flight mode. When the contact was performed, he switched to contact-flight mode and AeroX kept steady its pose w.r.t. the contact point using its internal sensors and not the GPS or the VICON measurements. The perturbations were injected by pulling a rope attached to the robot landing gear with an approximate force of 100 N, see video at https://youtu.be/mp4UAuhNHWc. Notice that these forces are significantly higher than the perturbations usuallly originated by wind gusts or by the irregularities in the surface contact. Two pictures in this experiment taken from the video are shown in Figure 11. AeroX had clearly different attitudes in both pictures. Figure 11. Two images from the video at https://youtu.be/mp4UAuhNHWc taken during the robustness analysis experiments. The safety rope with negligible tension is shown at the top. The telemetry of AeroX during this experiment is shown in Figure 12. The perturbations induced sudden changes in the attitude of AeroX in axis X, which can be clearly noticed in Figure 12-top-left. Figure 12-left shows the torques and linear forces computed by the AeroX controller. To compensate the perturbations the controller actuated with high torques in Xand lateral forces in Y. Figure 12-right shows the relative position XYZ of the robot w.r.t. the surface contact point computed by the developed method and also with the ground truth distance measured with VICON. Two conclusions can be reached. First, the robot position estimation error in all axes was lower than 1 cm. The 3D position estimation errors were lower than 4 cm, see Figure 12-right-bottom. The RMS position estimation error in all the robustness analysis experiments was 0.92 cm, which validated the robot relative position estimation method developed. The second conclusion is that the developed controller stabilizes Aerox with a very low error even with high external perturbations. In this experiment the robot positioning error w.r.t. the reference value was lower than 6 cm in each axis. Sensors 2019,19, 1305 20 of 24 Figure 12. Telemetry of AeroX in fully-autonomous flight during contact in a robustness analysis experiments. The 100 N perturbations injected generated the response of the developed controller, which can be noticed in the robot attitude and torque graphs. On the right, the robot relative position w.r.t. the surface contact point is shown together with the ground truth and the reference values for each axis. The robot position error w.r.t. the reference value in each axis was lower than 6 cm. 6.3. Video Attachments Three videos were produced to summarize the experiments performed with the AeroX robot: • Validation in an oil and gas refinery (https://youtu.be/Hy45WQ3GLcI). This video shows a brief summary of the many hours of operation of AeroX during its validation in an oil and gas refinery in Germany. The inspector guided the end-effector along many different surfaces and took hundreds of ultrasonic thickness measurements at inaccessible elements of the plant. •Outdoor testing experiments (https://youtu.be/nvw0I5ux19U). This video shows the complete operation of AeroX—from take-off to landing—in our outdoor testing setting in Seville. AeroX was manually controlled in assisted mode during free-flight mode. During contact-flight mode, AeroX flew in an autonomous way without any pilot intervention and the inspector teleoperated the end-effector. • Robustness analysis experiments (https://youtu.be/mp4UAuhNHWc). This video shows an experiment in the CATEC Aerial Robot testbed. The first part of the video shows how AeroX performs the contact operation on a static surface. During the contact-flight mode, Aerox was in fully autonomous mode. Flight perturbations were injected with a rope attached to the robot landing gear. The second part of the video shows preliminary outdoor tests in which AeroX contacts a pipe in different orientations. Although in these preliminary tests AeroX flew in a fully autonomous way, it was attached to a rope for safety reasons. Sensors 2019,19, 1305 21 of 24 7. Conclusions The procedures of the oil and gas industry involve extensive inspections, many of them should be performed at height and using sensors that require to be in contact with the elements to be inspected. The high costs and risks of traditional manual contact inspection using man-lifts, cranes, scaffolds or rope-access have generated in the oil and gas sector a strong need for alternative solutions. This paper presents AeroX, a novel aerial robotic manipulator that provides physical contact inspection with unprecedented capabilities. AeroX has a semi-autonomous operation. The pilot first guides the robot until performing contact. After contact, the pilot switches to autonomous GNSS-free contact flight and AeroX flies in fully autonomous mode with uninterrupted contact keeping steady w.r.t. the surface contact point. During the contact flight, the inspection operator can move the end-effector wheels on the surface and the aerial robot will follow the motion keeping uninterrupted contact. This can be used for precisely selecting the points to perform A-scan, B-scan, or C-scan inspections. Its operation can be easily integrated into current maintenance programs in many industries. AeroX is operated only by the pilot and by the inspector, each with clear and different roles. The inspector is focused only on the inspection process and does not require any training in robotics. Besides, AeroX has a simple, efficient and robust design that enables its everyday use in fully working conditions. Its tilted-rotor configuration provides high maneuverability, agility to reject flight perturbations and robustness. Its design enables performing contact inspection on surfaces with any orientation, including vertical, inclined, horizontal-top or horizontal-bottom. AeroX has been extensively tested and validated. Performance analysis experiments carried out in the CATEC indoor testbed shown that AeroX—both in fully autonomous flight as well as flight guided by the pilot—is rather robust to external perturbing forces of up to 100 N, which are significantly higher than potential flight perturbations originated by wind gusts or irregularities in the surface contact. Besides, extensive validation experiments (>200 flights) in different conditions were performed in an outdoor testing scenario and in a real refinery. All these experiments were fully successful and validated the proposed robot and its operating procedure as useful tools for contact inspection in real settings. Certified inspectors together with certified pilots confirmed the suitability of the proposed inspection solution during the refinery experiments. The extension of the developed aerial robot to other industries using other sensors in the end-effector and the development and integration of end-user functionalities to autonomously implement B-scan and C-scan inspections using the automatic vision-based relative localization assistant described in Section 5.3 are the object of current development. 8. Patents The mechanical configuration of the aircraft was patented in ES-2-614-994-B1 with the title “Aeronave con dispositivo de contacto” (in English: Aircraft with contact device) and was obtained the 23rd of February of 2018. The authors are, in order: Trujillo Soto, Miguel Ángel; Viguria Jiménez, Antidio; Márquez Font, José Carlos; Petrus Moreno, Ángel Luis; Jiménez Bellido, Antonio; García Freire, Juan Jesús and Ollero Baturone, Aníbal. Author Contributions: M.Á.T. is the technical leader of the prototype, designed the aerial robot, wrote all the control and high-level software, conducted all the experiments and is the main writer of the paper. J.R.M.-d.D. developed the vision-based relative localization assistant of the system, prepared that subsection and extensively reviewed the writing and approach of the paper. A.V. contributed in the patent and leading the Avionics and System division of CATEC that made this development possible. C.M. contributed as an inspection expert in the introduction and current work section of the paper and giving specialized NDT inspection support during the experiments. A.O. reviewed the paper and coordinates the AEROARMS project, which has been a fundamental pillar for the development of the presented aerial robot. Acknowledgments: This work was supported by European Commission H2020 Project AEROARMS under Grant 644271. The authors would like to thank all the CATEC working team that made possible the current implementation and the presented experimentation with the aerial robot. In particular, we would like to mention Sensors 2019,19, 1305 22 of 24 J. C. Marquez, A. L. Petrus, A. Jimenez, J. J. Garcia, and M. Barbero. Furthermore, we would like to thank the AEROARMS TÜV NORD team for their support during the Germany experiments. Conflicts of Interest: The authors declare no conflict of interest. Abbreviations The following abbreviations are used in this manuscript: COTS Commercial off-the-shelf DCM Direct cosine matrix DoF Degree of freedom EU European Union GNSS Global navigation satellite system GPS Global positioning system IMU Inertial measurement unit KW Kilowatt NDT Non-destructive testing RANSAC Random sample consensus RMS Root mean square ROI Region of interest RTK Real-time kinematic SURF Speeded-up robust features UAV Unmanned aerial vehicle UT Ultrasonic testing References 1. Akhondi, M.R.; Talevski, A.; Carlsen, S.; Petersen, S. Applications of Wireless Sensor Networks in the Oil, Gas and Resources Industries. In Proceedings of the 2010 24th IEEE International Conference on Advanced Information Networking and Applications, Perth, Australia, 20–23 April 2010; pp. 941–948. 2. Savazzi, S.; Guardiano, S.; Spagnolini, U. Wireless Sensor Network Modeling and Deployment Challenges in Oil and Gas Refinery Plants. Int. J. Distrib. Sens. Netw. 2013,9, 383168. [CrossRef] 3. Inspection Robotics. Available online: http://inspection-robotics.com/ (accessed on 10 February 2019). 4. Kim, S.; Choi, S.; Kim, H.J. Aerial manipulation using a quadrotor with a two DOF robotic arm. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Tokyo, Japan, 3–7 November 2013. 5. Korpela, C.; Orsag, M.; Oh, P. Towards valve turning using a dual-arm aerial manipulator. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014. 6. Lippiello, V.; Cacace, J.; Santamaria-Navarro, A.; Andrade-Cetto, J.; Trujillo, M.A.; Esteves, Y.R.; Viguria, A. Hybrid Visual Servoing With Hierarchical Task Composition for Aerial Manipulation. IEEE Robot. Autom. Lett. 2016,1, 259–266. [CrossRef] 7. Jimenez-Cano, A.E.; Braga, J.; Heredia, G.; Ollero, A. Aerial manipulator for structure inspection by contact from the underside. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Hamburg, Germany, 28 September–2 October 2015; Volume 2015, pp. 1879–1884. 8. Suarez, A.; Heredia, G.; Ollero, A. Lightweight compliant arm with compliant finger for aerial manipulation and inspection. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea, 9–14 October 2016; Volume 2016, pp. 4449–4454. 9. Trujillo, M.A.; Viguria, A.; Marquez, J.C.; Petrus, A.L.; Jimenez, A.; Garcia, J.J.; Ollero, A. Aeronave con Dispositivo de Contacto. Spain Patent ES 2 614 994 B1, 2 March 2018. 10. Birring, A.; Marshall, G. Eddy Current Testing in the Petrochemical Industry. Mater. Eval. 2003 ,61, 1190–1195. 11. Industrial Rope Access Trade Association. Available online: https://irata.org/page/history (accessed on 10 February 2019). Sensors 2019,19, 1305 23 of 24 12. Paneque, J.L.; Martinez-deDios, J.R.; Ollero, A. Robust Decentralized Context-Aware Sensor Fault Detection with In-Place Self-Calibration. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3130–3136. 13. Honeybee Robotics. Available online: https://www.honeybeerobotics.com/portfolio/pipe-inspection- robot/ (accessed on 10 February 2019). 14. Ruggiero, F.; Trujillo, M.A.; Cano, R.; Ascorbe, H.; Viguria, A.; Perez, C.; Lippiello, V.; Ollero, A.; Siciliano, B. A multilayer control for multirotor UAVs equipped with a servo robot arm. In Proceedings of the IEEE International Conference on Robotics and Automation, Hong Kong, China, 31 May–7 June 2015. 15. Kondak, K.; Huber, F.; Schwarzbach, M.; Laiacker, M.; Sommer, D.; Bejar, M.; Ollero, A. Aerial manipulation robot composed of an autonomous helicopter and a 7 degrees of freedom industrial manipulator. In Proceedings of the IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2014; pp. 2107–2112. 16. Fumagalli, M.; Stramigioli, S.; Carloni, R. Mechatronic design of a robotic manipulator for Unmanned Aerial Vehicles. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea, 9–14 October 2016. 17. Kamel, M.; Alexis, K.; Siegwart, R. Design and modeling of dexterous aerial manipulator. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea, 9–14 October 2016; Volume 2016, pp. 4870–4876. 18. Park, S.; Her, J.; Kim, J.; Lee, D. Design, modeling and control of omni-directional aerial robot. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Daejeon, Korea, 9–14 October 2016. 19. Papachristos, C.; Alexis, K.; Tzes, A. Efficient force exertion for aerial robotic manipulation: Exploiting the thrust-vectoring authority of a tri-tiltrotor UAV. In Proceedings of the IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2014. 20. Darivianakis, G.; Alexis, K.; Burri, M.; Siegwart, R. Hybrid predictive control for aerial robotic physical interaction towards inspection operations. In Proceedings of the IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2014. 21. Gioioso, G.; Ryll, M.; Prattichizzo, D.; Bulthoff, H.H.; Franchi, A. Turning a near-hovering controlled quadrotor into a 3D force effector. In Proceedings of the EEE International Conference on Robotics and Automation, Karlsruhe, Germany, 6–10 May 2014. 22. Bartelds, T.; Capra, A.; Hamaza, S.; Stramigioli, S.; Fumagalli, M. Compliant Aerial Manipulators: Toward a New Generation of Aerial Robotic Workers. IEEE Robot. Autom. Lett. 2016,1, 477–483. [CrossRef] 23. Ryll, M.; Muscio, G.; Pierri, F.; Cataldi, E.; Antonelli, G.; Caccavale, F.; Franchi, A. 6D Physical Interaction with a Fully Actuated Aerial Robot. In Proceedings of the IEEE International Conference on Robotics and Automation, Singapore, 29 May–3 June 2017; pp. 1–6. 24. Staub, N.; Mohammadi, M.; Bicego, D.; Prattichizzo, D.; Franchi, A. Towards Robotic MAGMaS : Multiple Aerial-Ground Manipulator Systems. In Proceedings of the IEEE International Conference on Robotics and Automation, Singapore, 29 May–3 June 2017; pp. 1307–1312. 25. Madhavan, R.; Thévoz, P.; Briod, A. The World Cup of Drones Flyability Wins the UAE Drones for Good Award. IEEE Robot. Autom. Maga. 2015,22, 108–110. [CrossRef] 26. Intel Falcon 8+ Drone Transforms Inspections Conducted in the Oil and Gas Industry. Available online: https://www.youtube.com/watch?v=IZ8yaP9FWEc (accessed on 10 February 2019). 27. Apellix. Available online: https://www.apellix.com/ (accessed on 10 February 2019). 28. Ronik Inpectioneering. Available online: http://www.inspectioneering.eu/ (accessed on 10 February 2019). 29. Ultrasonic Data Presentation. NDT Resource Center. Available online: https://www.nde-ed. org/EducationResources/CommunityCollege/Ultrasonics/EquipmentTrans/DataPres.htm (accessed on 10 February 2019). 30. Michieletto, G.; Ryll, M.; Franchi, A. Fundamental Actuation Properties of Multirotors: Force-Moment Decoupling and Fail-Safe Robustness. IEEE Trans. Robot. 2018,PP, 1–14. [CrossRef] 31. Bay, H.; Tuytelaars, T.; Van Gool, L. SURF: Speeded Up Robust Features. In Proceedings of the Computer Vision—ECCV 2006, Graz, Austria, 7–13 May 2006; Leonardis, A., Bischof, H., Pinz, A., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 404–417. Sensors 2019,19, 1305 24 of 24 32. Fischler, M.A.; Bolles, R.C. Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. Commun. ACM 1981,24, 381–395. [CrossRef] 33. Vicon Motion Systems Ltd UK. Available online: https://www.vicon.com/ (accessed on 10 February 2019). c 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).