EMG-Controlled Robotic Prosthetic Arm With Neural Network Training

EMG-Controlled Robotic Prosthetic Arm With Neural Network Training Javier Civit-Masot, Alfonso P´ erez-Rodr´ ıguez, Francisco Luna-Perej´ on, Luis Mu˜ noz-Saavedra, Manuel Dom´ ınguez-Morales and Anton Civit Robotics and Computer Technology Lab Avda. Reina Mercedes s/n, E.T.S. Ingenier´ ıa Inform´ atica, Universidad de Sevilla, Sevilla, Spain Email: {jcivit, fralunper, luimunsaa, mdominguez, civit}@atc.us.es Abstract—The project consists of the creation of a robotic arm controlled remotely through a brace developed by Thalmic Labs, able to read the limb muscles’ biopotentials. This project aims to create an economic alternative to non-invasive active prostheses that exist today. Our prosthesis can perform the same functions but at a so much affordable price. In order to perform all the functions of a normal joint, the arm has several elements. Strings that simulate tendons and allow the movement of the fingers, gears that allow the rotation of the wrist and motors, which can generate movement based on the data extracted from the bracelet. The bracelet is responsible for transmitting information from the hand to the robotic arm through a wireless module that connects it with the computer, where the signal that extracts the bracelet goes through a filtering process to keep the information that interests us and Transmit it through the USB port to a microcontroller, which will be in charge of moving the engines according to the signals received. To avoid errors in the measurement of the sensors, the information received from the bracelet is trained in the computer using a Neural Network architecture before sending the information to the robotic arm. Keywords—Machine learning, neural network, exoskeleton, prosthetics, EMG. I. INTRODUCTION The evolution of prosthetics is long and full of stories, from its primitive beginnings, through the sophisticated present, to the incredible visions of the future. As in the development of any other field, some ideas and inventions have worked and have been explored in more detail, such as the fixed-position foot, while others have been left out or have become obsolete, such as the use of iron in prostheses [1]. The long and complex road to the computerized arm began around 1500 BC and has been in constant evolution ever since. There have been many refinements since the first wooden legs and hand hooks, and the result has been the highly customized fixation and molding found in today’s devices [2]. Today, bionics is the application of biological solutions to systems technologies in architecture, design, engineering and modern technology [3]. There is also bionic engineering that covers several disciplines with the aim of concatenating (making biological and electronic systems work together), for example, to create prostheses activated by robots controlled by a biological signal or also to create artificial models of things that only exist in nature, for example, artificial vision and artificial intelligence also called cybernetics [4]. One could say, bionics is that branch of cybernetics that tries to simulate the behavior of living beings by making them better in almost all branches by means of mechanical instruments. Bionics has had a great development in countries like Germany that has courses qualified in the same way in different schools, Japan that has a great development in Bio robots, United States and United Kingdom. In Latin America and Spain there are also developments of this type. In Mexico, the career of Bionic Engineering was founded at the UPIITA (Professional Interdisciplinary Unit in Engineering and Advanced Technologies) of the IPN (National Polytechnic Institute) in 1996 [5], which has yielded results in the creation of bionic devices [6]. In the United States, 82% of amputations are due to vascular disease, 22% are due to trauma, 4% are congenital, and 4% are tumorous. Approximately 1.6 million people in the United States live with an amputation. 1.5 amputees per 1,000 population in the US and Canada According to the Agency for Healthcare Research and Quality (AHRQ), about 113,000 lower extremity amputations are performed each year [7]. There is a 3:1 ratio between men and women (73.6% vs. 26.4%). Amputations occur predominantly in the lower limbs at 84%, compared to 16% in the upper limbs. •In the upper limbs the cause is predominantly traumatic with 70.4% followed by congenital cause with 18%. •In lower limbs the predominant cause is vascular with 69.5% followed by traumatic with 22.5%. Therefore we are going to try to put a cheap alternative to a big problem that affects millions of people in the world (extrapolating the figures previously seen only in USA and Canada). The main objective of this work is the creation of cheap noninvasive active prosthesis. Its user will be able to use it thanks to a bracelet, which has several integrated sensors (gyroscope, accelerometer and EMG (electromyographic signal sensors)). This bracelet will be placed on the user’s arm and will read the parameters of the resident muscular terminations of the user’s residual limb, transmitting them to a microprocessor that will move the specific motor in the prosthesis. To achieve this purpose, the ”Divide and Win” technique has been followed: the main aim of this work has been split into smaller objectives, which can be carried out individually, to finally bring them together and obtain the final goal. The rest of the manuscript is divided as follows: 11Copyright (c) IARIA, 2020. ISBN: 978-1-61208-763-4 eTELEMED 2020 : The Twelfth International Conference on eHealth, Telemedicine, and Social Medicine II. MATERIALS AND METHODS The system is divided into three parts: the Myo bracelet (green), the computer (blue) and the microcontroller and arm (red). The first communication (between the bracelet and the computer) is done via a wireless connection and the second communication (between the computer and the microcontroller) is done via a USB port communication. The full system diagram can be observed in Figure 1. Fig. 1. System block diagram. The information obtained from the sensors is sent from the bracelet to the computer. In the computer, a signal filtering application is executed to keep the data we are interested in for the arm movement. Once the signal has been filtered and we have the data we want, we transmit the control signal to the microcontroller, which takes care of the movement of the servomotors and therefore the movement of the robotic arm. The sensors in the wristband encompass the technologies used to measure the different values of the arm: •Electromyography sensors: These are the sensors placed in each module of the bracelet, they are in charge of measuring the biopotentials of the arm muscles. •Accelerometer: It is the sensor in charge of taking measurements of arm and wrist accelerations. •Gyroscope: This is the sensor in charge of taking measurements of changes in position and rotation of the wrist and arm. In the data transmission of the read values, a USB module is used which provides the bracelet. The device allows data to be sent through it wirelessly. In our case, it is connected to the computer and the bracelet sends the data via Bluetooth. The computer runs an application created in Visual Studio in C programming language, in which the data from the bracelet is received and training is carried out via a neural network in Fig. 2. System block diagram. order to be able to discretize the information to be given to the arm; in this way the system requires a previous training process, as well as a large amount of data to be trained before starting to give an output from the system. The result of the neural network is transformed into programmed movements that are sent to the microcontroller to control the reception of data. On the board, which controls the entire arm, are the necessary connections for the operation of the various devices. The robotic arm is in charge of imitating the movements that we exercise in the arm that we have placed the bracelet. Servo motors are used on the arm motors. This type of motor is the best alternative for the application that the arm will perform since we can control the movement of the fingers and the rotation of the hand. The servomotors are connected to the microcontroller through the PWM pins on the plate. In this case, the devices are not powered from the microcontroller. An external power supply is used to power the servomotors. III. RESULTS Below, we can see the block diagram of the operation of the electromyogram: The analogical signal is received from the electrodes placed in the bracelet, then the 8 bits analogical digital converter that the bracelet has, converts the analogical signal in a digital range from 0 to 255 bits, if the movements are not calibrated, 12Copyright (c) IARIA, 2020. ISBN: 978-1-61208-763-4 eTELEMED 2020 : The Twelfth International Conference on eHealth, Telemedicine, and Social Medicine all the movements are calibrated first, after having all the sensors perfectly calibrated, we make the calibration samples to be trained by the neural network system that we have put in the code so that the program is able to distinguish well the movements automatically. Finally, once the system is trained, the signals are sent to the arm so that it moves the way we send it through the electrical biopotentials of the muscles. On one hand, the system’s neural network structure is composed of an input layer, an output layer and a hidden layer. The entire structure can then be observed: On the other hand, the application developed, which is in charge of communication and training, can be seen below: Fig. 3. System block diagram. As for the tests, three different types were carried out: firstly, system integration tests were carried out to check the correct functioning of all the developed parts; then, for the system training, the success rates were obtained for several subjects and on multiple occasions (the training results did not fall below 85% in any case); and finally, usability tests were carried out with the patients who had served as test subjects for the neural network. IV. CONCLUSIONS Once the project is fully completed, the project objectives have been fully met. A system based on the movement of a robotic arm through the reading of EMG sensors from a bracelet and the training of the information received by the pc through neural networks has been designed, integrated and tested. The success rate after training and testing with 8 subjects did not decrease in any case from 85%; being the average above 90% of success in the classification of the neuronal system. The results are satisfactory and users have shown interest in the subject. ACKNOWLEDGMENT This work has been developed and funded by the research group TEP-108: Robotics and Computer Technology fron the University of Seville. REFERENCES [1] K. Norton, “A brief history of prosthetics,” InMotion, vol. 17, no. 7, pp. 11–3, 2007. [2] C. Lake and J. M. Miguelez, “Evolution of microprocessor based control systems in upper extremity prosthetics,” Technology and Disability, vol. 15, no. 2, pp. 63–71, 2003. [3] M. H. Dickinson, “Bionics: Biological insight into mechanical design,” Proceedings of the National Academy of Sciences, vol. 96, no. 25, pp. 14 208–14 209, 1999. [4] K. Krippendorff, “The cybernetics of design and the design of cybernetics,” in Design Cybernetics. Springer, 2019, pp. 119–136. [5] Z. Lovtchikova Khavrachenko, “M´ etodos de aprendizaje para los alumnos de ingenier´ ıa bi´ onica,” 2009. [6] C. D. Rico Mandujano, “An´ alisis estructural de una pr´ otesis bi´ onica de brazo para miembro superior,” Ph.D. dissertation, 2011. [7] D. J. Margolis and et al., “Prevalence of diabetes, diabetic foot ulcer, and lower extremity amputation among medicare beneficiaries, 2006 to 2008,” in Data Points Publication Series [Internet]. Agency for Healthcare Research and Quality (US), 2011. 13Copyright (c) IARIA, 2020. ISBN: 978-1-61208-763-4 eTELEMED 2020 : The Twelfth International Conference on eHealth, Telemedicine, and Social Medicine