Design and Implementation of Vision Based Leader Follower Tracking Using PID Control

UNIVERSITY OF TECHNOLOGY (YATANARPON CYBER CITY) FACULTY OF ELECTRONIC ENGINEERING DESIGN AND IMPLEMENTATION OF VISION-BASED LEADER-FOLLOWER TRACKING USING PID CONTROL BY HSU MYAT THEIN LWIN M.E. THESIS OCTOBER, 2019 PYIN OO LWIN UNIVERSITY OF TECHNOLOGY (YATANARPON CYBER CITY) FACULTY OF ELECTRONIC ENGINEERING DESIGN AND IMPLEMENTATION OF VISION-BASED LEADER-FOLLOWER TRACKING USING PID CONTROL BY HSU MYAT THEIN LWIN A THESIS SUBMITTED TO THE FACULTY OF ELECTRONIC ENGINEERING IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ENGINEERING (ELECTRONICS) OCTOBER, 2019 PYIN OO LWIN UNIVERSITY OF TECHNOLOGY (YATANARPON CYBER CITY) FACULTY OF ELECTRONIC ENGINEERING We certify that we have examined, and recommend to the University Steering Committee for Postgraduate Studies for acceptance of the M.E. thesis entitled: “DESIGN AND IMPLEMENTATION OF VISION-BASED LEADERFOLLOWER TRACKING USING PID CONTROL” submitted by Ma Hsu Myat Thein Lwin, Roll No. M.E. EcE-5 (October, 2019) to the Faculty of Electronic Engineering in partial fulfillment of the requirements for the degree of M.E. (Electronics). Board of Examiners: 1. Dr. Htin Kyaw Oo Professor and Head Faculty of Electronic Engineering ………………………. University of Technology (Yatanarpon Cyber City) (Chairman) 2. Dr. Cho Thet Paing Professor Faculty of Electronic Engineering ……………………. University of Technology (Yatanarpon Cyber City) (Supervisor) 3. Dr. Thae Su Aye Professor Faculty of Electronic Engineering ……………………. University of Technology (Yatanarpon Cyber City) (Co-Supervisor) 4. Daw Nu Nu Ye Lecturer Faculty of Electronic Engineering ……………………. University of Technology (Yatanarpon Cyber City) (Member) 5. Dr. Myo Myint Professor Department of Electronic Engineering …..…………………. Technological University (Thanlyin) (External Examiner) i ACKNOWLEDGEMENTS Firstly, I thank my lovely parents, who let me to attend M.E course and support fund although I am a graduated person. I specially thank to my teacher, Dr. Htin Kyaw Oo, Head of Faculty of Electronic Engineering for his motivation and encouragement in my interested field throughout the years of not only undergraduate student but also master student. I greatly thank to my Supervisor, Dr. Cho Thet Paing, for her kind guidance and encouragement at the time of this thesis. Without her passionate supporting, this thesis could not be conducted successfully. Moreover, I also especially thank to my Co-Supervisor, Dr. Thae Su Aye form Microelectronics and Embedded Technology Division, Faculty of Electronic Engineering, who supported components and instruments for my research. In addition, I thank to Dr. Myo Myint, my external examiner from Thanlyin Technological University who greatly supported ways and guidance to be successful finished all requirements of my research. Also, I especially thank to my teachers U Khin Zaw Oo and Daw Nu Nu Ye, who especially encouraged motivation when I face difficulties throughout the time of my research. I also thank to my teacher, Daw Yi Yi Aung from Department of English, who supported to be conducted my research paper with English language correctly and systematically. Finally and the most important is that I thank my friends for their kindness, motivation, strength and encouragement throughout the research time. I am very glad for the happiest time of my student life with my teachers and my friends, and it is sure that this time will be unforgettable memories throughout my life. ii ABSTRACT This research investigates the control of a group of robots consisting of a leader and a follower to maintain a desired geometric formation. The architecture of this research mainly contributes vision system and control algorithm. Vision system is performed by the single front camera using image processing to estimate information between leader and follower robots. Control algorithm is designed for the follower robot to track the desired position by maintaining predetermined formation between the two robots. The follower robot used in this research is differential drive mobile vehicle. Black-box type modelling is performed to generate dynamic transfer functions for the two DC motors of mobile robot through System Identification Toolbox in Matlab without the need to know the mechanical concept of a DC motor. Proportional-Integral-Differential (PID) controllers are designed for speed and position control of the vehicle to maintain the desired performance and reducing inaccuracies of the system because of systematic error such as modelling faults. Generalized forward kinematics of differential drive mobile robot is used to transform a desired position in lateral and longitudinal motion of the vehicle. The performance of the system is analysed in both simulation and implementation results. iii TABLE OF CONTENTS Page ACKNOWLEDGEMENTS i ABSTRACT ii TABLE OF CONTENTS iii LIST OF FIGURES vi LIST OF TABLES ix CHAPTER TITLE 1 INTRODUCTION 1 1.1. Multi-Robot Systems 1 1.2. Motivation 1 1.3. Aim and Objectives 2 1.4. Related Works 2 1.5. System Description 3 1.6. System Scope 4 1.7. Organization 4 2 COMPONENETS AND THEORETICAL BACKGROUNDS 5 2.1. Hardware and Software Components 5 2.1.1. Optical Encoder 5 2.1.2. Arduino 7 2.1.3. DC Motor 8 2.1.4. L298N Dual Motor Controller Module 9 2.1.5. Raspberry Pi 10 2.1.6. Webcam 11 2.1.7. Matlab 12 2.2. Theoretical Backgrounds 12 2.2.1. Control Systems 13 2.2.1.1. Open loop and closed loop control system 13 2.2.1.2. Systematic and non-systematic error in control systems 14 2.2.1.3. Linear and non-linear controllers 15 2.2.2. Proportional-Integral-Differential (PID) Control Method 16 iv 2.2.3. Vision System 17 2.2.3.1. Digital image processing 17 2.2.3.2. Colour based object detection 19 2.2.3.2.1. Colour spaces 19 2.2.3.3. Morphological transformation 22 2.2.3.3.1. Erosion 22 2.2.3.3.2. Dilation 22 2.2.3.3.3. Opening 23 2.2.3.3.4. Closing 23 3 SYSTEM DESIGN CONSIDERATION AND SIMULATION 24 3.1. Vision System 24 3.1.1. Target Detection 24 3.1.1.1. Captured video 25 3.1.1.2. HSV thresholding 25 3.1.1.3. Morphological transformation 26 3.1.1.4. Rectangle shape detection 26 3.1.2. Position Estimation 26 3.1.2.1. Distance estimation 27 3.1.2.2. Angle estimation 27 3.2. Modelling, Control and Simulation of Differential Drive Mobile Robot 28 3.2.1. Modelling 28 3.2.1.1. DC motor model 29 3.2.1.2. Black-box modelling of DC motors using system identification 31 3.2.2. Open Loop Analysis 35 3.2.3. Motion Planning of Differential Drive Mobile Robot 36 3.2.3.1. Mathematical model 36 3.2.3.2. Kinematic equations 37 3.2.4. Design and Simulation of PID controllers 38 3.2.4.1. PID controllers for speed control of DC motors 38 3.2.4.2. PID controllers for position control of mobile robot 41 v 3.2.4.2.1. Lateral PID controller 41 3.2.4.2.1. Longitudinal PID controller 43 3.2.4.2.3. Simulation results of position control 44 4 EXPERIMENTS AND IMPLEMENTATION RESULTS OF LEADER-FOLLOWER SYSTEM 48 4.1. Experimental Setup 48 4.2. Experiments of Vision System 49 4.2.1. Target Leader Detection 49 4.2.2. Leader Position Estimation 52 4.3. Implementation Leader Follower System 55 5 DISCUSSION, CONCLUSION, LIMITATION, AND FURTHER EXTENSION 63 5.1. Discussion and Conclusion 63 5.2. Limitation and Further Extension 64 REFERENCES 66 LIST OF PUBLICATIONS 69 3 system identification is described in reference paper [11Lee]. This reference paper provides about black-box modelling intended for the electrical engineers without the knowledge of mechanical concepts. The importance of low-level stability control of the development of the leader-following robotic application is presented in thesis paper [12Cri]. The kinematic and motion planning of differential-drive mobile robot system is provided in paper [16Kot]. 1.5. System Description There are two main portions in this research. The first part is vision system using image processing, and another one contains modelling and control algorithm. Distance and angle information between leader and follower mobile robots is obtained by the dimension of the rectangle blob behind the leader through image processing using Python programming language in Raspberry Pi. Black-box type modelling using System Identification Toolbox in Matlab is performed to estimate the transfer functions of the two DC motors that are implemented on the left and right sides of the follower robot. PID controllers are designed for the speed and position control of the robot. Arduino IDE is supported as the main platform to implement the PID controller for this research. Arduino and Raspberry Pi are connected using USB serial communication. Image processing estimates the desired position (distance and angle values) for the robot to travel. These values are the desired set-point values for the position control of the follower robot. Figure 1.1. Schematic Representation of Leader-Follower System Left Wheel Right Wheel Rectangle Mark Free Wheel Angle Camera Raspberry Pi, Arduino Position Control, Speed Control, Image Processing Leader Follower 4 1.6. System Scope There are a few scopes for this thesis to ensure the desired objectives to conduct intended boundary. Image processing, modelling and control are the main contribution of this research. This research addresses the problem of sensor used formation systems so, sensing the leader information is performed by camera vision using image processing technique. Black-box modelling is performed for the estimation of the transfer functions of the DC motors that are used in the mobile robot. This type of modelling does not need to measure physical parameters of the system. It needs only experimental data (Input and Output data) in appropriate sampling time. The control scheme of the thesis contains two levels of control structure, i.e. low level or speed control and high level or position control. A mobile robot may face systematic error and non-systematic error. The systematic error is formed because of the inaccuracies in the system of the robot such as modelling error. Non-systematic error is caused due to external disturbance such as environmental conditions. This system is designed to reduce the systematic error, neglecting nonlinear disturbance. For this reason, linear control scheme is sufficient for the system. So, PID control method is developed in control task of this research. 1.7. Organization In Chapter 1, it includes the introduction of the Multi Robot System, research motivation, aim and objectives, related works, an overview of this system, and scope of the thesis. Chapter 2 presents the theoretical background and required hardware and software components of this system. Chapter 3 is divided into two sections i.e. vision system and modelling and control of differential drive robots. In vision system, image processing techniques to detect the leader in which distance and angle estimation are described. In the next section, kinematic and dynamic models of vehicle systems, the system identification process, and the validation process of black-box modeling are described. In addition, design and simulation results of speed control, longitudinal and lateral controls are included in this section. Chapter 4 includes implementation results of the thesis. In Chapter 5, discussion and conclusion of this research are described. Moreover, system limitations and further research are presented in Chapter 5. CHAPTER 2 COMPONENTS AND THEORETICAL BACKGROUND This chapter describes the essential background theory of the components used in this research. The follower robot used in this study consists of three wheels, i.e. one is free, and others are DC gear motor drive. An optical encoder is fixed at each motor drive wheel of the robot to sense the speed of the wheel. Arduino is the main platform for system modelling and also implementing PID controllers for the speed and position control for the follower robot. System Identification Toolbox in Matlab is supported for the system modelling. Simulink Toolbox in Matlab is also the main tool in designing PID controllers for this research. In addition, Raspberry Pi using the Python language and OpenCV library has performed image processing task that is required to obtain desired information between leader and follower for this research. The detail explanations of software and hardware components and required background theories to design this research are presented in this chapter. 2.1. Hardware and Software Components This article describes hardware and software components that are required to design this research. Hardware components include optical encoders, Arduino, DC motor, L298N dual motor controller, and Raspberry pi and webcam. Vision system is performed using Raspberry pi and PID controllers and implemented in Arduino embedded controller board. Matlab is the main software platform for the modeling and controllers’ design in which Simulink library and System Identification Toolbox are the essential tools for system modelling, control design and verification for the controllers’ performance. 2.1.1. Optical Encoder A transducer that generates a coded reading of a measurement can be termed as an encoder. Digital transducers over their analog counterparts are high resolution depending on the word size of the encoder output and the number of pulses per 6 revolution of the encoder so they are very useful in digital control systems (because the transducer output is digital) with associated reduction in system cost and improvement of system reliability [15Che]. The signal generation of the encoder can be accomplished using any one of four techniques: Optical (photo sensor) method, Sliding contact (electrically conducting) method, Magnetic saturation (reluctance) method, and Proximity sensor method. Figure 2.1. Quadrature Optical Encoder The encoder used in this research shown in Fig. 2.1 is a quadrature optical encoder which includes rotating or opaque disk and light sensor (photo transistor). Opaque disk has one or more circular track, with some arrangement of identical transparent windows (slits) in each track. The quadrature light detector measures the amount of light passed through the slotted disc and generates two quadrature output pulse signals, denoted by A and B which are 90 degrees phase shift to each other [19Dyn]. These two channels indicate both position and direction. When the pulses in the channel-A lead channel-B, the disc is rotated in a counter-clockwise direction (CCW), otherwise, the disc is rotated in a clockwise direction (CW) as shown in Fig. 2.2. Figure 2.2. Quadrature Encoder Output Channels A and B 7 In this research, two quadrature optical encoders are implemented in the follower robot to measure the wheel speed. The encoder counts the digital pulses by the Arduino microcontroller to determine the wheel displacement, and then, this count value is directly converted to the rate of displacement in radian per second. The encoders used in this research have the specification of 5V supply voltage, 100 CPR (Count per Revolution), and 400 PPR (Pulse per Revolution). 2.1.2. Arduino Arduino is an open-source prototyping platform based on easy-to-use hardware and software. Arduino boards feature serial communication interfaces, including Universal Serial Bus (USB) on some models, for loading programs in personal computers. For programming the microcontrollers, the Arduino provides an Integrated Development Environment (IDE) based on a programming language named Processing, which also supports the languages C and C++ [18Get]. Figure 2.3. Arduino UNO In this research, Arduino UNO is shown in Fig. 2.3 is supported for input and output data logging for system modelling and implementation of PID controllers for speed and position control. UNO has fourteen digital pins and six analog pins. In digital pins, six pins can be used as PWM (Pulse-Width-Modulation) output pins. The Uno has a number of facilities for communicating with a computer, another Uno Grounds Serial Out Serial In Reset Button In-Circuit Serial Programmer Analog Pins Digital I/O Pins Analog Reference Pin Ground 5V supply 3.3V supply Reset Pin USB Plug External Power Supply Input Voltage Microcontroller 8 board, or other microcontrollers. The UNO has 60 MHz in clock speed, and 1KB in EEPROM (Electrically Erasable Programmable Read Only Memory). The ATmega328 provides serial communication, which is available on digital pins 0 (RX) and 1 (TX). An ATmega16U2 on the board channels this serial communication over USB and appears as a virtual com port to software on the computer. The 16U2 firmware uses the standard USB (Universal Serial Bus) drivers, and no external driver is needed. The Arduino Software (IDE) includes a serial monitor which allows simple textual data to be sent to and from the board [18Get]. 2.1.3. DC Motor DC Motors convert electrical energy (voltage or power source) to mechanical energy (produce rotational motion). They run on direct current. The DC motor works on the principle of Lorentz force which states that when a wire carrying current is placed in a region having magnetic field, then the wire experiences a force. This Lorentz force provides a torque to the coil to rotate. When the input DC current is passed to the coil through the brushes, it directly goes to the coil inside the motor body. This makes coil to work as an electromagnet. Magnetic fields of both magnets interact with each other that results in a force which in turn produces the necessary torque required to move the coil. This torque drives the coil to move round and a shaft attached with the coil moves too [11Eng]. The two differential wheels of the robot are implemented with DC gear motors. The speed of the DC motor is controlled by Pulse-Width-Modulation (PWM). The power applied to the motor can be controlled by varying the width of these applied pulses and thereby varying the average DC voltage applied to the motors terminals. By changing or modulating the timing of these pulses the speed of the motor can be controlled, i.e. the longer the pulse is “ON”, the faster the motor will rotate and likewise, the shorter the pulse is “ON” the slower the motor will rotate [16Ano]. Fig. 2.4 shows the PWM waveform that is required to know how to change the speed of the motors. Figure 2.4. Pulse-Width-Modulation Waveform Low Average Voltage High Average Voltage Narrow Pulse Wide Pulse T T 9 In this research, two 12V Tetrix DC motors shown in Fig. 2.5 are implemented and they have the specifications of the gear ratio of 2:1 and output speed of 152 RPM. The two motors of the robot are connected with L298 Dual H-Bridge motor driver which allows speed and direction of two DC motors at the same time described in the next article. Figure 2.5. Tetrix DC Gear Motor 2.1.4. L298N Dual Motor Controller Module The L298N motor driver can control the speed and direction of two DC motors, or control one bipolar stepper motor with ease. The L298N H-bridge module shown in Fig. 2.6 can be used with motors that have a voltage of between 5 and 35V DC with a peak current up to 2A [18Mah]. Figure 2.6. L298N H-bridge Motor Driver Module [18Mah] This module contains four inputs (IN1 to IN4) and four output pins (OUT1 to OUT4) to control the two DC motors, for example, digital inputs (1 and 0) from IN1 and IN2 are to drive or change direction of the two motors which are connected to the OUT1 and OUT2 respectively. 8-bit PWM signal (0 to 255) from Arduino must be applied to the module’s pins of Enable 1 and Enable 2 to change the speed of the two motors [18Mah]. 6-35V Ground 5V Inputs (IN1 to IN4) Outputs (OUT1,OUT2) Outputs (OUT3,OUT4) EnableA EnableB 12V Jumper 10 2.1.5. Raspberry Pi The Raspberry Pi operates in the open source ecosystem: it runs Linux (a variety of distributions), and its main supported operating system, Raspbian, is open source and runs a suite of open source software [14Don]. There have been three generations of Raspberry Pi: Pi 1, Pi 2, and Pi 3, and there has generally been a Model A and a Model B of most generations. Model A is a cheaper variant and tends to reduce RAM and the ports like USB and Ethernet. The Pi Zero is a spinoff of the original (Pi 1) generation, made even smaller and cheaper. A considerable number of programming languages have been adapted for the Raspberry Pi, either by the creator of the language or by users of the language who wanted to see their language of choice available on the Raspberry Pi. Python, C, C++, Java, Scratch, and Ruby all come installed by default on the Raspberry Pi. The Raspberry Pi Foundation recommends Python programming language the very idea of programming is reflected in the name of the device [14Don]. Figure 2.7. Raspberry Pi 3 Module B [14Don] In this research, camera vision system using image processing is supported by the Raspberry Pi 3, Module B is shown in Fig. 2.7. Raspberry Pi 3, Module B has 1GB in RAM, and 1.2 GHz clock speed. Image processing is performed by OpenCV (Open Source Computer Vision) library installed in Raspberry Pi using Python 3 programming language. OpenCV is a library of programming functions mainly aimed at real-time computer vision. 40GPIO Headers USB Ethernet Controller 2×2 USB ports Ethernet Port Camera Connector Audio and Composite Output Jack SD card slot Switching Regulator for less power Comsumption 5V Micro USB Power Display Connector Run Header Used to Reset the Pi 4 holes Boardcom BCM2835 HDMI In 11 2.1.6. Webcam A webcam is a compact digital camera that can hook up to the computer to broadcast video images in real time (as they happen). Just like a digital camera, it captures light through a small lens at the front using a tiny grid of microscopic lightdetectors built into an image-sensing microchip (either a charge-coupled device (CCD) or, more likely these days, a CMOS image sensor). Unlike a digital camera, a webcam has no built-in memory chip or flash memory card. The USB cable supplies power to the webcam from the computer and takes the digital information captured by the webcam image sensor back to the computer [19Mar]. There are two types of webcam: internal and external. Internal webcams are built into a monitor frame, and are usually located just above the screen. Internal webcams are more common on laptops, but they’re also built into some standalone monitors and all-in-one PCs (a PC where the computer is housed in the monitor case). Internal webcams have a small lens that looks similar to the tiny camera lenses on many mobile phones. Figure 2.8. Logitech C310 HD webcam [19Lif] Choosing the type of webcam depends on what’s most important. If computer came with a built-in webcam, that might be all need. The image sensor is the "electronic eye" of a webcam or a digital camera. It is a semiconductor chip made of millions of tiny, light-sensitive squares arranged in a grid pattern. These squares are called pixels. A basic webcam has about 0.3 megapixels (300,000, in other words). A better camera rated at 12 megapixels would have a 4000 x 3000 pixel sensor. [19Lif] The USB webcam installed in Raspberry pi and attached at the front center of the follower robot captures real time video of the leader. Then, image processing such as rectangle blob detection and position estimation is performed in Raspberry pi. The USB webcam used in this research is “Logitech C310” HD webcam shown in Fig. 2.8 and it has the frame rate of 30FPS/720p, viewing angle range of 60 degrees and fixed focus type. 12 2.1.7. Matlab Matlab is a high-performance language for technical computing. It integrates computation, visualization, and programming in an easy-to-use environment where problems and solutions are expressed in familiar mathematical notation. Typical uses include:  Math and computation  Algorithm development  Modeling, simulation, and prototyping  Data analysis, exploration, and visualization  Scientific and engineering graphics  Application development, including Graphical User Interface building Matlab is an interactive system whose basic data element is an array that does not require dimensioning. This allows you to solve many technical computing problems, especially those with matrix and vector formulations, in a fraction of the time it would take to write a program in a scalar non-interactive language such as C or Fortran. Matlab feature a family of application-specific solutions called toolboxes. Toolboxes are comprehensive collections of Matlab functions (M-files) that extend the Matlab environment to solve particular classes of problems. Areas in which toolboxes are available include signal processing, control systems, neural networks, fuzzy logic, wavelets, simulation, and many others. In this research, System Identification Toolbox in Matlab is supported for the black-box modelling to estimate dynamic transfer functions of the two DC motors. Working principle and process of black-box modelling using system identification are presented in detail in Chapter 3. An add-on product to Matlab, Simulink, provides an interactive, graphical environment for modelling, simulating, and analyzing of dynamic systems that need to be controlled, like cars, airplanes, and so on. It enables rapid construction of virtual prototypes to explore design concepts at any level of detail with minimal effort. In this research, Simulink is the main tool in designing PID controllers for speed and position control of the follower robot. 2.2. Theoretical Backgrounds This article describes essential background theories, including control theory related to Proportional-Integral-Differential (PID) controller, and useful image processing theories for this research. 19  Image restoration - Image restoration is an area that also deals with improving the appearance of an image. It is subjective, image restoration is objective, in the sense that restoration techniques tend to be based on mathematical or probabilistic models of image degradation.  Morphological processing - Morphological processing deals with tools for extracting image components that are useful in the representation and description of shape.  Segmentation - Segmentation procedures partition an image into its constituent parts or objects. In general, autonomous segmentation is one of the most difficult tasks in digital image processing.  Representation and description - Representation and description almost always follow the output of a segmentation stage, which usually is raw pixel data, constituting either the boundary of a region or all the points in the region itself. Description deals with extracting attributes that result in some quantitative information of interest or are basic for differentiating one class of objects from another.  Recognition - Recognition is the process that assigns a label, such as, “vehicle” to an object based on its description. 2.2.3.2. Colour based object detection Object detection and segmentation are the most important and challenging fundamental task of computer vision. However, it is still an open problem due to the variety and complexity of object classes and backgrounds. The easiest way to detect and segment an object from an image is the color based methods. The object and the background should have significant color difference in order to successfully segment objects using color based methods [13Fer]. 2.2.3.2.1. Colour spaces The representation of colours in an image is achieved using a combination of one or more colour channels that are combined to form the colour used in the image. The colour can be represented by specifying the number and nature of the colour channels, is generally known as the colour space. There are several different colour spaces and each has its own significance. Some of the popular spaces are RGB (Red, 20 Green, Blue), CMYK (Cyan, Magenta, Yellow, Black), HSV (Hue, Saturation, Value), and so on. OpenCV’s default color space is RGB. However, it actually stores colour in the BGR format. It is an additive colour model where the different intensities of Blue, Green, and Red give different shades of colour. A colour image in the computer vision system is described with colour pixels contains Red, Green and Blue values each measured in eight bits. The description of colour pixels in an image is shown in Fig. 2.13 in which each small box represents a pixel of the image. In real images, these pixels are so small that the human eye cannot differentiate. Figure 2.13. Description of a Colour Pixel in an Image Within the RGB representation, then RGB colour space is essentially a 3-D color space (cube) with axes, R, G, and B shown in Fig. 2.14. Each axis has the same range 0→1 (this is scaled to 0-255 for the common 1 byte per colour channel, 24-bit image representation). The colour black occupies the origin of the cube (position (0,0,0)), corresponding to the absence of all three colours; white occupies the opposite corner (position (1,1,1)), indicating the maximum amount of all three colours. All other colours in the spectrum lie within the cube. The primary problem with RGB is that it is perceptually nonlinear. This means that moving in the given direction in the RGB color cube does not necessarily produce a color that is perceptually consistent with the change in each of the channels. For example, starting at white and subtracting the blue component produces yellow; similarly, starting at red and subtracting at the blue component produces pink. For this reason, RGB color space is inherently difficult for humans to work and reason about because it is not related to the natural way of perceiving colors. As an alternative, perceptual colour representation such as HSV is usable [13Fer]. Perceptual colour space is an alternative way of representing true colour in a manner that is more natural to the human perception and understanding of more colour than the RGB presentation. Hue, Saturation, and Value (HSV) colour space is Red Matrix Green Matrix Blue Matrix Red Matrix Green Matrix Blue Matrix 21 very popular in image analysis applications. Changes within this colour space follow a perceptually acceptable colour gradient. For an image analysis perspective, it allows the separation of color from lighting to a greater degree. Figure 2.14. An Illustration of RGB Colour Space as a 3-D Cube An RGB image can be transformed into the HSV color space representation as shown in Fig. 2.15. Each of these parameters can be interpreted as follows:  H (hue) is the dominant wavelength of the colour, e.g. red, green, and blue  S (saturation) is the ‘purity’ of colour  V (value) is the brightness of the colour (also known as luminance). The HSV representation of a 2-D image is also as a 3-D array comprising three channels (h, s, v) and each pixel location within the image, for example, I (n, m), contains an (h, s, v) triplet that can be transformed back into RGB for true colour display. Figure 2.15. HSV Colour Space as a 3-D Cone [13Fer] By examining the individual colour channels of images in the HSV space, image objects are consistently contained in the resulting Hue field than in the 22 channels of the RGB representation, despite the presence of varying lighting conditions over the scene. HSV space is commonly used for colour-based image segmentation using a technique known as colour slicing. A portion of the Hue colour wheel (a slice of the cone, in Fig. 2.15) is isolated as the colour range of interest, allowing objects within this colour range to be identified within the image [13Fer]. In this research, HSV colour space is used for colour based image segmentation to be small error in leader detection because of the great performance in different lighting conditions at different environment. 2.2.3.3. Morphological transformation Morphological transformations are simple operation based on the image shape. It is normally performed on the binary images. It needs two inputs, one is the original image, second is structuring element or kernel which decides the nature of the operation. The structuring element can be manually created. It may be rectangular in shape and sometimes it may need elliptical or circular shaped kernel. In general, structuring element may consist of ones and zeros so as to define any neighborhood, but the practicalities of digital image processing mean that they must be padded with zeros in an appropriate fashion to make them rectangular in shape overall. Two basic morphological operators are Erosion and Dilation. Any morphological operations are represented as combinations of erosion, dilation, and simple set-theoretic operations such as the complement of a binary image. 2.2.3.3.1. Erosion Erosion erodes away the boundaries of foreground objects (always try to keep foreground in white). A pixel in the original image (either 1 or 0) will be considered 1 only if all the pixels under the kernel are 1, otherwise it is eroded (made to zero). All the pixels near boundary will be discarded depending on the size of the kernel. So the thickness or size of the foreground object decreases or simply white region decreases in the image. It is useful for removing small white noises [19Sol]. 2.2.3.3.2. Dilation Dilation is just opposite of erosion. With dilation, a pixel element is ‘1’ if at least one pixel under the kernel is ‘1’. So, it increases the white region in the image or size of foreground object increases. Normally, in case like noise removal, erosion is 23 followed by dilation. Erosion removes white noises, but it also shrinks our object. So, dilation is needed in increasing the object area. Dilation is very useful in joining broken parts of an object [19Sol]. 2.2.3.3.3. Opening The opening of an image is erosion followed by dilation. Opening is also called because of it can open up a gap objects connected by a thin bridge of pixels. Any regions that have survived the erosion are restored to their original size by the dilation. Before and after morphological opening of an image is shown in Fig. 2.16. (a) (b) Figure 2.16. Morphological Opening (a) Original image (b) Image after Opening The general effect of opening is to remove small, isolated objects from the foreground of an image, placing them in the background. It tends to smooth the contour of a binary object and breaks narrow joining regions in an object [19Sol]. 2.2.3.3.4. Closing The closing of an image is a dilation followed by erosion. In this case, the dilation and erosion should be performed with a rotated by 180 degree structuring elements. Typically, the latter is symmetrical, so that the rotated and initial versions of it do not differ. Closing is so called because it can fill holes in the regions while keeping the initial region sizes. Before and after morphological opening of an image is shown in Fig. 2.16. (a) (b) Figure 2.17. Morphological Closing (a) Original Image (b) Image after Closing Morphological Closing tends to remove small holes in the foreground, changing small regions of the background into the foreground. It tends to join narrow isthmuses between objects [19Sol]. CHAPTER 3 SYSTEM DESIGN CONSIDERATION AND SIMULATION This chapter can be divided into two main sections: vision system and robot control system. Vision system describes step by step process of image processing for the estimation of leader information (distance and angle between leader and follower). Robot control system includes modelling and control of mobile robot to travel in desired formation. 3.1. Vision System This section describes a computer vision system using image processing for estimation of distance and orientation angle between leader and follower robots. A single forward facing camera is implemented at the front of the follower robot. By using image processing, the follower robot determines leader information (distance and angle) according to the information from green rectangular blob implemented on the back of the leader robot. Image processing is performed by the OpenCV library with Python Language in Raspberry Pi 3, module B. Vision system is divided into target detection and position (distance and angle) estimation. Target detection involves capturing real time video from the camera and processing it to detect the rectangle mark behind the leader robot. Position estimation finds the distance from the leader robot due to the dimension of the rectangle blob implemented behind the leader robot. The procedure of visual system is shown in Fig. 3.1. Figure 3.1. Procedure of Visual System Target Detection •Captured Video •HSV Thresholding •Morphorlogical Operation •Rectangle Shape Detection Position Estimation •Distance Estimation •Angle Estimation 25 3.1.1. Target Detection This article describes the image processing, process for the target (Leader Robot) detection, including video capturing, HSV thresholding, morphological operation, and rectangle shape detection. 3.1.1.1. Captured video A forward facing Logitech C310 HD webcam captures video of leader robot. The webcam is implemented at the centre between the two wheels of follower robot and video is captured with the frame rate of 30 frames per second and 160×120 pixels resolution. Figure 3.2. Webcam Setup on the Follower Robot 3.1.1.2. HSV thresholding One of the main tasks of the vision system is to detect leader using green coloured rectangle blob mounted on the leader robot. Detection is performed through colour segmentation in Hue, Saturation, Value (HSV) colour space. Although Red, Green, Blue (RGB) colour space is used widely in most capturing tasks, it is not suitable for recognition tasks. In the HSV color space, Hue (H) value represents a color, Saturation (S) indicates the intensity of the color and Value (V) contains information about how bright the pixel is. Unlike RGB colour space where colours are mixed up from three different colour channels, using the HSV colour system to detect a feature of a particular colour is much more satisfactory. Leader robot Follower robot Webcam 36.6cm 18.3cm 18.3cm 26 This system is needed to specify the appropriate HSV values in the upper and lower levels for the detection of green rectangle blob. So, manual track bars for HSV levels are developed and the desired HSV levels are determined. The resulting HSV values are used in this system for the green rectangle detection. The Hue channel is individually threshold to detect a specific colour. The saturation and intensity values are threshold, depending on the lightening condition. At HSV threshold binary image (black and white (or) 0 and 1), the desired green area is white and the other area is black. 3.1.1.3. Morphological transformation After HSV threading is performed, the next step is morphological operation. In the HSV threshold binary image, some of the undesired spots appear on the image because of the colour mixing due to lighting conditions of the environment. Morphological transformation; Opening and Closing using Erosion and Dilation are performed to reduce this undesired noise in the binary image. Opening is a morphological transformation which is erosion followed by dilation. Closing is the reverse of the opening, dilation followed by erosion. It is useful in small holes in the foreground objects, or small black points on the object. 3.1.1.4. Rectangle shape detection After the morphological transformation to reduce undesired noise in the binary image, the next process is rectangle blob detection in this resulting image. This is the final step of target detection. At first, four-point contour in the image is found to detect rectangle blob region in the image, discarding any contour that does not have four points. But, some of the small four points, regions may appear in the image apart from the desired rectangle blob region. So, as the second, finding largest rectangle contour detection is supported to eliminate this fault detection. 3.1.2. Position Estimation Position estimation finds the distance and angle between the two robots (Leader and Follower robots) due to the information from the green rectangle mark behind the leader robot. Design considerations and calculations of distance and angle estimations are described in this article. 27 3.1.2.1. Distance estimation In this system, “Triangle Similarity” showed in Fig. 3.3 and Equation 3.1 is utilized to measure distance (‘d’). Figure 3.3. Triangle Similarity F d = H h (3.1) Where, F=Focal Length H=Object’s Height in perceived Image h=Actual Height of the object d=Distance between camera and object According to the similarity, the first step is to derive the focal length of camera in pixel with defined distance (‘d’) and object’s height (‘h’). ‘H’ is perceived from the Rectangle Area Extraction (height and width). After deriving focal length, the desired distance between leader and follower robot is calculated by using Equation 3.1, d=F (h/H). Note that actual height of the rectangle mark ‘h’ must be predefined. 3.1.2.2. Angle estimation The orientation angle between leader and follower is estimated from the distance between leader and follower (‘d’), and distance between center of rectangle and center of the image (‘C’) on the image plane. Representation of angle estimation between leader and follower is described in Fig. 3.4. In Fig. 3.4, to find the parameter (‘C’) in units of centimeters, the relation between the number of pixels on the image plane and real scene distance must be known. A testing to know the number of pixels (pixels) versus real measurement in centimeters (cm) is performed by collecting pixel data of 1cm height of the object at a different distance from the camera (‘C’). By using this logged data, an equation of d Image Lens H Object h F 28 pixels versus centimeters is estimated with the help of “Curve Fitting” application in Matlab. Figure 3.4. Representation of Angle Estimation Eq. 3.2 is the resulting equation which is the relation of the number of pixels ‘p’ for 1cm real measurement at a distance from the camera of “d (cm)”. In the estimation of this relation, the Root Mean Squared (RMS) error is 0.02. p = (2533×d)-(5.343×104) d2-(20.3×d)-16.67 (3.2) By using the relation in Eq. 3.2, the angle between leader and follower is estimated. According to Fig. 3.4, the orientation can be calculated by using Equation 3.3 in which ‘ө’ is the orientation angle between leader and follower, ‘d’ is the distance between leader and follower, and ‘C’ is the distance between the center of rectangle blob and center of the image. θ = sin-1(C d) (3.3) 3.2. Modelling, Control and Simulation of Differential Drive Robot This section firstly describes about system modelling for estimation of the DC motors in the follower robot. Proportional-Integral-Differential (PID) controller designs are performed for the speed control in the inner loop and position control in the outer loop. Kinematic equations are supported to calculate the actual position from the velocity of the two wheels in the follower robot and parameters of the robot frame. 3.2.1. Modelling When using a model to help in the design process, it is important that the right type of model is used. Using the wrong type of model can waste computing power and time, and either provide too little detail or far too much. There are three different 35 Equations 3.14 and 3.15 are the chosen transfer functions of left and right sided motors with the PWM input (V) and output speed (ω) of the unit of rad/s. ω V=7.732 s2+32.03s+158.1 (3.14) ω V=7.956 s2+40.23s+165.8 (3.15) 3.2.2. Open Loop Analysis The dynamic transfer functions of the two DC motors implemented in the left and right sides of the mobile robot are estimated with the input of PWM and output of rad/s. The open loop dynamic of the left or right motor G(s) is shown in block diagram at Fig. 3.11. Figure 3.11. Block Diagram of Open Loop System The transfer function’s input is PWM or duty-cycle, so that the system is converted to ω (rad/s) to 8-bit PWM value (0 to 255) by Equation 3.16, PWM value = Ke×ω×255 rated voltage (3.16) In Equation 3.15, voltage constant of the motor (Ke = 0.8) is the ratio of the supply voltage of motor (12V) to maximum no load speed in rad/s (15 rad/s). The step response plots of left and right DC motors are shown below in Fig. 3.12. And also, the desired specified stability criteria are specified as shown in following Table. 3.1. The regarding of stability criteria is to reach and remain close to the desired reference value in the minimum time possible and the minimum overshoot above the desired value. Time (seconds) Output (Rad/s) (a) 36 (b) Figure 3.12. Open Loop Step Response (a) Left Motor (b) Right Motor As can be seen in the step response plot for left motor in Fig. 3.12 (a), 0.3772 second in rise time, 0.6857 second in settling time, and 0% overshoot. Moreover, in the right motor’s response plot in Fig. 3.12 (b), 0.4796 second in rise time, 0.8717 second in settling time, and 0% overshot. So, both left and right motors do not satisfy desired specifications and so the controller must developed to compensate this problem. By root locus analysis, for both DC motor models, all of the poles are located on the left hand side and so the systems are originally stable and also this system are not studied about non-systematic disturbance, so PID control method is sufficient for the control structure of this research. Table 3.1. Desired Performances Specifications for Speed Control Parameter Symbol (Unit) Desired Performance Settling Time Ts (s) <0.5s Maximum Percent Overshoot Mp(%) <2% Peak Time Tp(s) <0.8s Rise Time Tr(s) <0.2s 3.2.3. Motion Planning of Differential Drive Mobile Robot Differential drive robot is a mobile robot with two driving wheels in which the overall velocity is split between left and right wheels. Mathematical model of differential drive mobile robot is described in the next article. 3.2.3.1. Mathematical model To maneuver any differential drive robot in a plane shown in Fig. 3.13. Time (seconds) Output (Rad/s) 37 The robot needs a linear velocity v and a heading θ. By controlling the velocity and orientation, the path of the robot can be planned. Figure 3.13. Differential Drive Mobile Robot Parameters In Fig. 3.13, vr is the velocity of the right wheel, vl is the velocity of the left wheel, L is the distance between the center of the wheels, θ is the directional angle of the robot with respect to the reference coordinate system and r is the radius of the wheel. 3.2.3.2. Kinematic equations Kinematic equations of a differential drive mobile vehicle in this article support to derive the position of the follower robot (distance and orientation) from the left and right wheel velocity with respect to the leader robot. The velocity for left wheel (vl) and right wheel (vr) are sent to the kinematic equations to calculate the linear (v) and angular (ω or θ′) velocities of the robot. By integrating these velocities the desired distance (d) and orientation angle (θ) are achieved and these values are feedback to the position controllers (lateral and longitudinal controllers) which are described in article 3.2.4.2. Kinematics is the branch of classical mechanics which describes the motion of a point (object’s center) without consideration of the mass of the objects or the forces that may have caused the motion. The kinematic equations are used to transform the motion from polar coordinates (r, θ) to rectangle coordinate (x, y) system. As mentioned earlier, the inputs required for the motion of a mobile robot are the linear 38 velocity (v) and the orientation θ. The rate of change of position of robot in xdirection is ẋ and that in y-direction is ẏ are given by ẋ=v cos (θ) (3.16) ẏ=v sin (θ) (3.17) And the angular velocity (ω) and linear velocity (v) of the robot are given by θ'=ω=(vl-vr) / L (3.18) V=(vl+vr) / 2 (3.19) By substituting linear velocity (v) in Equation 3.16 and Equation 3.17, ẋ=[(vl+vr) / 2]×cos (θ) (3.20) ẏ=[(vl+vr) / 2]×sin (θ) (3.21) The velocity (v) of the robot in fixed reference coordinates system is therefore given by v=√ẋ2+ẏ2 (3.22) Substituting Equation 3.20 and Equation 3.21 in Equation 3.22, v=√(v ×cos(θ))2+ (v× sin(θ))2 (3.23) The individual velocities, vr and vl, vl = ( v + (L/2) ω) (3.24) vr = ( v- (L/2) ω) (3.25) According to the kinematic equations, the value of the linear velocity (v) and angular velocity (ω) in Equations 3.18 and 3.19 are integrated to form the distance (d) and orientation angle (θ) which are used as the feedback for position controllers. 3.2.4. Design and Simulation of PID controllers This article includes controller design for DC motor speed control, and position control, including longitudinal and lateral controllers. 3.2.4.1. PID controllers for speed control of DC motors The dynamic transfer functions of the left and right sided DC motors are different as shown in Equations 3.11 and 3.12. So, a PID controller for each sided motor must be designed to have desired stability performance shown in Table 3.1. PID controllers are designed in Simulink in Matlab, and the performance of the 39 different controllers: P (Proportional), PI (Proportional-Integral), and PID (Proportional-Integral-Differential) for both manual and auto tuning are analyzed. The control block diagram of speed control for two sided motors is shown in Fig. 3.14. Figure 3.14. Control Block Diagram for Speed Control of Two DC Motors In PID controller design, firstly, manual Proportional (P) controller was tried by using the Table. 3.2. Increasing the proportional gain (Kp) has the effect of reducing rise time. The open loop model output is smaller than the desired speed as discussed previous article. So the system is needed to increase Kp. But the greater the Kp, the higher the maximum overshoot. It is not convenient the desired Percent Overshoot criterion of less than 2%. Then, Proportional-Integral Controller (PI) was tried and it is the best performance of this system. PID auto tuning result in the Matlab PID tuner application and best results of manual tuning for left and right motors are shown in Fig. 3.15 (a) and (b). (a) 40 (b) Figure 3.15. Closed Loop Step Response with Different Controllers (a) Left Motor (b) Right Motor The comparisons of the closed loop response’s characteristics of the left motor are shown in Table 3.2. Table 3.2. Comparison of Closed Loop Characteristics (Left Motor) Controller Manual (PI) Auto (PID) Kp 2.2 3.84 Ki 11.4 28.02 Kd 0 0.052 N 2035 Rise Time 0.12 0.08 Settling time 0.19 0.24 Percent Overshot 0 9.52 Peak Time 0.24 0.3 Steady State Error 0 0 The closed loop response’s characteristics of right motor are described in Table 3.3. Table 3.3. Comparison of Closed Loop Characteristics (Right Motor) Controller Manual (PI) Auto (PID) Kp 3 3.13 Ki 13 16.05 Kd 0 0.1 41 N 51.41 Rise Time 0.12 0.1 Settling time 0.1897 0.285 Percent Overshot 0 3.61 Peak Time 0.25 0.20 Steady State Error 0 0 As conclusion of the PID control, all of the results shown in Fig. 3.15, Table 3.2, and Table 3.3 are the best response for different PID controllers and among them, PID (manual tuning) is the best performance for both motors. 3.2.4.2. PID controllers for position control of mobile robot At the inner loop control scheme, the two DC motors have different dynamic equations according to the system identification, so a PID controller for each motor to achieve the desired performance at the speed of the motors. The outer loop, position control contains longitudinal and lateral PID controllers to perform desired criteria at distance travels (d) and orientation of the mobile robot (θ). Fig. 3.16 describes block diagram of Position Control. Generalized forward kinematic described previously is transformed from the speed of two differential-drive wheels and the parameters of robot frame. These equations are used to generate distance (d) and orientation angle (θ) of the robot travels that are required for feedback of the outer loop. Figure 3.16. Block Diagram of Position Control 3.2.4.2.1. Lateral PID controller Lateral PID controller compensates the mobile robot to travel to reach setpoint angle (θ) at desired performance. Reference of lateral PID controller is the orientation angle (θ) and the control effort of the lateral controller is only the speed of 42 right motor because the robot has changed its orientation by controlling the speed of right motor. To tune the lateral PID controller, the velocity of the mobile robot is set as constant. Control block of lateral PID controller is shown in Fig. 3.17. Figure 3.17. Control Block of Lateral PID controller The desired performance specifications of lateral PID controller are shown in Table 3.4. Table 3.4. Desired Performance Specifications for Lateral Controller Specification Symbol (Unit) Desired Performance Settling Time Ts (s) <0.2s Maximum Percent overshoot Mp(%) <5% Peak Time Tp(s) <0.3s Rise Time Tr(s) <=0.1s According to the block diagram in Fig. 3.17, the angle is derived from the speed of the left and right motor vr and v with respect to the kinematic equations of the differential drive mobile robot. By Equation 3.18, the angular velocity is (vl-vr)/L. In which equation, by the frame structure of the robot used in this research, distance between left and right wheels, L= 36.6 cm. Then, angle (θ) is derived by integrating the angular velocity in time in which the orientation angle is the unit of radian. Desired set-point orientation angle is calculated by the vision system and the unit is degree. So, the feedback of lateral controller is degree. The output angle is converted from radian to degree (1°= 2π 360 ≅0.01745 radian). Fig. 3.18 shows the step response of the Lateral PID controller. By PID tuning for lateral PID controller, PID controller at Kp=7, Ki=1, Kd=2, N=1 is the best response to the performance in settling time of 0.15 seconds, rise time of 0.084 second, and 0.106% overshoot. 43 Time (second) Figure 3.18. Step Response of Lateral PID Controller 3.2.4.2.2. Longitudinal PID controller The longitudinal PID controller is designed to compensate the linear velocity (v) to move the follower robot to the desired distance. The reference of the longitudinal controller is the desired distance (d) and the control effort is the velocity of the follower robot. Control block of longitudinal PID controller is shown in Fig. 3.19. To tune the longitudinal PID controller, the set-point orientation angle is fixed at ‘0’ degree. According to the block diagram in Fig. 3.19, output distance (d) is derived from the left and right wheel velocities (vl and vr) by using the kinematic equations. Figure 3.19. Control Block of Longitudinal PID Controller By Equation 3.19, the linear velocity is the average of the summation of the left and right wheel velocities (v rad/s). Desired set-point distance ‘d’ must be calculated by the vision system and the unit is centimeter (cm). The linear velocity in unit of cm/s is calculated by multiplying the radius of the wheel ‘r’ in cm and the linear velocity ‘v’ in rad/s (v (cm/s) = r (cm) × v (rad/s)). The radius (r) of the wheel used in the follower robot in this research is 4.8cm. The distance travel of the robot is the integrating of the linear velocity of the robot in time. Angle (degree) 44 The desired performance specifications of Longitudinal PID controller are shown in Table 3.5. Table 3.5. Desired Performance Specifications for Longitudinal Controller Parameter Symbol (Unit) Desired Performance Settling Time Ts (s) <0.3s Maximum Percent overshoot Mp(%) <10% Peak Time Tp(s) <0.4s Rise Time Tr(s) <=0.2s Fig. 3.20 describes the resulting step response of the longitudinal PID controller. Time (second) Figure 3.20. Step Response of Longitudinal PID Controller By tuning longitudinal PID controller, PD controller at Kp= 142, Ki= 4.5, N=138 is the best response to the sufficient performance in rise time of 0.06 second, settling time of 0.207 second and 5.5% overshoot. 3.2.4.2.3. Simulation results of position control After integrating lateral and longitudinal controllers in the previous article, performance simulation of position control in point to point motion of the mobile robot is described in this article. Fig. 3.21 shows point to point coordination from distance and angle. The desired distance and angle from the desired position (x, y) is calculated from the point to point equations from Equation 3.24 to 3.25 and inverse kinematic from Equation 3.26 into Equation 3.27. Distance (cm) 51 conditions and colour mixing in the environment. These spots become errors at the next step of vision system; distance and angle estimation. So, morphological transformation (Opening and Closing using erosion and dilation) is performed to reduce this undesired noise in the binary image. The concepts of morphological transformation are presented in Chapter 3. Fig. 4.4 shows reducing of error in the binary image using morphological opening and closing. (a) (b) Figure 4.4. Binary Image (a) Before Morphological Transformation (b) After Morphological Transformation After the morphological transformation to reduce noise in binary image, the next step is rectangle blob detection in this binary image. This step is detected of contour with four points in the image. In this case, target leader is considered as the largest rectangle contour because the system is fault when undesired small four points regions are appearing in the image. Fig. 4.5 shows target rectangle detection in which green rectangle area is described with red blob. Undesired White Spots 52 Figure 4.5. Rectangle Blob (Target Leader) Detection 4.2.2. Leader Position Estimation Leader position estimation includes distance and angle determination from the leader. Experimental results are presented in this article. Root mean Square (RMS) error and maximum error of the relative distance and orientation angle are calculated and described with plots. The distance from the leader robot is estimated from the height of the rectangle blob using the “Triangle Similarity” which is the relation of distance, focal length, real height of the object and the height of the object appears in the image. The concept of “Triangle Similarity” is described in Chapter 3. According to this relation, the first step is to estimate focal length ‘F’ of the camera in pixel at a defined distance between camera and object i.e. leader and follower. Fig. 4.6 shows estimation of focal length in pixel with a defined distance of 120 cm and resulting mean value of focal length is 207.8234 pixels and chosen focal length is 208 pixels. (a) 120cm 53 (b) Figure 4.6. Estimating Focal Length (a) Experiment (b) Resulting Focal Length By using the estimated focal length of 208 pixels, the desired parameter, distance between leader and follower is determined using the relation of “triangle similarity”. Fig. 4.7 shows distance estimation from the leader. The actual distance is measured practically and resulting estimated distance from the image processing is described with plots. In the experiment, the distance between the follower and the leader (distance between camera and rectangle blob) is 60 cm and estimated by image processing. In distance estimation, the observed RMS error is 1.4033% and the maximum error is 2.85% respectively. The orientation angle between leader and follower is estimated using the relation of distance between leader and follower, the distance between the center of the image and the center of the rectangular blob on the follower. (a) 60cm 54 (b) Figure 4.7. Distance Estimation (a) Actual Measurement (b) Using Image Processing This relation is described in Equation 3.2, in which distance between leader and follower is estimated previously and the actual distance between centers of the rectangle and image is calculated by using the relation between pixel and real distance (cm). (a) (b) Figure 4.8. Angle Estimation (a) Actual Measurement (b) Using Image Processing 10 degrees 55 Fig. 4.8 shows the angle estimation between leader and follower comparing practical measurement and estimated by image processing, in which maximum error of 6.2% and RMS error of 2.85%. 4.3. Implementation of Leader Follower System In this article, implementation results of control algorithm containing speed and position control of differential drive mobile robot system are described. Moreover, implementation results of leader follower system by integrating a vision system and control algorithm of a mobile robot system are verified in this article. Fig. 4.9 shows a circuit diagram of an implementation of leader follower system. For the implementation, choosing appropriate sampling interval is important. If the time, ‘T’ is the process time constant which is the time at 63% of process variable or the desired output, the required sampling time ‘Ts’ must be less than or equal to one half of the process time constant ‘T’ as described in Equation 4.1. Ts≤0.5T (4.1) This means that the sampling frequency must be at least twice of the maximum frequency of the system. Figure 4.9. Circuit Diagram of Leader-Follower Robot According to the dynamic transfer functions of two DC motors described in Chapter 3, 63% of process variable is reached at 0.207 second for the left motor and 0.2421 second for the right motor. So, maximum sampling times of left and right motor control for 0.1035 second and 0.12105 second respectively. So, sampling interval of the system must be less than or equal to 0.1 second. In this research, the 56 maximum processing time of image processing in Raspberry pi is 0.06 second, the maximum communication time between Raspberry pi and Arduino via USB is 26 nanoseconds, and maximum loop running time of controller code in Arduino is 0.01 second. The system can take maximum processing time of 0.08 seconds, which is compatible for the need of sampling interval less than or equal 0.1 seconds. In this system, the sampling time is 0.1 second, also described in data acquisition for modelling of two DC motors. The first step in implementation is speed control of two DC motors at inner loop. The simulation results of speed control are described in the article 3.2.4.1, and comparison of different controllers, P, PI, and PID controllers are analyzed in both auto and manual tuning. According to the output simulation responses, PID manual tuning is satisfied for both left and right sided DC motors. Figure 4.10. Block Diagram of Speed Control of a DC Motor In this article, implementation results for speed control of two DC motors will be presented. From the simulation result, the resulting gains for PID controllers are Kp=6, Ki=30, Kd=0.09 for left motor and Kp=9.5, Ki=38.5, Kd=0.15 for right motor respectively. PID controllers are implemented using an Arduino controller and feedback sensors are optical encoders to calculate the actual speed of the motors as shown in Fig. 4.10. (a) Rad/s PID controller Left/Right Motor Optical Encoder Rad/s _ + 57 (b) Figure 4.11. Implementation Results of Speed Control for DC Motors (a) Left Motor (b) Right Motor Implementation result of speed control for DC motors is logged by driving the robot on the smooth floor, and the resulting plots are demonstrated in Fig. 4.11 (a) for left motor and Fig. 4.11 (b) for right motor respectively at the differential input setpoint comparing with simulation results, in which, the implementation results of both motors track the desired setpoint speed. After the implementation of speed control for the two DC motors equipped at left and right sides of the robot is performed, the next step is the implementation of the outer loop, position control of mobile robot at lateral and longitudinal motion. The resulting speed of left and right sided motors (output of the inner loop) and the mobile robot physical parameters (distance between the left and right wheel and the radius of the wheels) are converted to the moving position (distance and orientation angle) by using the forward kinematic of differential drive mobile robot. Lateral and longitudinal PID controllers are developed for the robot to travel the desired position. The control effort of lateral controller is orientation angle and longitudinal controller is distance. In article 3.2.4.2, the simulation results of the position control of the mobile robot are described. In this article, the implementation results are presented. Fig. 4.12 shows the block diagram of the outer loop, position control. From the simulation result, lateral controller is suitable with PID controller, which has the gain Kp = 7, Ki=1, Kd=2, N=1. In the longitudinal controller, PD controller at Kp = 142, Kd = 4.5, N=138 is suitable with proper specifications. These PID controllers are implemented in Arduino controller and resulting outputs are verified by comparing with simulation results. 58 Figure 4.12. Control Block of Position Control Fig. 4.13 shows implementation results of position changes when the mobile robot moves forward direction to a distance = 100 cm and angle = 0 degree. The resulted moving positions of mobile robot are demonstrated in the X and Y coordinates as shown in Fig. 4.13 (a) by using the inverse kinematic equations which are described in Chapter 3. The follower robot reaches the desired distance 100 cm at the time of 1.7 seconds and the orientation angle is approximately 0 degree in the case of moving straight line. (a) (b) (c) 59 (d) (e) Figure 4.13. Implementation Results of Position Control for Straight Line Moving (a) Movement of Mobile Robot in XY Coordinate (b) Change in Distance (c) Change in Angle (d) Distance Error (%) (e) Angle Error (%) The changing in distance and orientation angle are described in Fig. 4.13 (b) and Fig. 4.13 (c). According to the implementation results in Fig. 4.13 (d), when the mobile robot moves forward horizontally, the maximum angle error is 2.5 % and RMS error is 0.6757 % with respect to the desired angle of 0 degree. The mobile robot moves horizontally from the starting point to a distance of 100 cm and reduces the distance error of 100% to 0%, shown in Fig. 4.13 (e). Fig. 4.14 shows implementation results when mobile robot turns to a distance=100 cm and angle=30 degrees. These results are plotted by comparing with simulation results. The follower robot moves from the starting point of (0,0) to the end point of (86.6,50) as shown in Fig. 4.14 (a). (a) (b) (c) 60 (d) (e) Figure 4.14. Implementation Results of Position Control for Turning to an Angle (a) Movement of Mobile Robot in XY Coordinate (b) Change in Distance (c) Change in Angle (d) Distance Error (%) (e) Angle Error (%) As shown in Fig. 4.14 (b) and Fig. 4.14 (c), the follower mobile robot is turned to an angle of 0 to 30 degrees and then moves in straight line with a distance of 100 cm. The position error (distance and angle) decreases as the robot is nearer to the desired position (100cm, 30 degrees) as shown in Fig. 4.14 (d) and Fig. 4.14 (e). After the implementation of control algorithm (speed and position control) in the leader follower system, the final step is system integration. The input set-points (distance and angle) of the control algorithm are given by the image processing and the control algorithm performs moving of follower robot to maintain the desired position. The control algorithm is implemented in Arduino controller and set-points are received from the Raspberry Pi via the USB serial communication as shown in Fig. 4.9. In the implementation of leader follower system, the performance is verified by comparing the moving position of leader and follower robots demonstrated in X-Y coordinate, and comparison of velocity and position changing. In Fig. 4.15, the leader robot moves with different speed values (20cm/s and 37cm/s) in straight forward. (a)