Feasibility of discriminating UAV propellers noise from distress signals to locate people in enclosed environments using MEMS microphone arrays

sensors Article Feasibility of Discriminating UAV Propellers Noise from Distress Signals to Locate People in Enclosed Environments Using MEMS Microphone Arrays Alberto Izquierdo 1,* , Lara del Val 2, Juan J. Villacorta 1, Weikun Zhen 3, Sebastian Scherer 3 and Zheng Fang 4 1Signal Theory and Communications Department, University of Valladolid, 47011 Valladolid, Spain; [email protected] 2Mechanical Engineering Area, Industrial Engineering School, University of Valladolid, 47011 Valladolid, Spain; [email protected] 3Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15289, USA; [email protected] (W.Z.); [email protected] (S.S.) 4Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China; [email protected] *Correspondence: [email protected]; Tel.: +34-983-185801 Received: 10 October 2019; Accepted: 17 January 2020; Published: 21 January 2020   Abstract: Detecting and finding people are complex tasks when visibility is reduced. This happens, for example, if a fire occurs. In these situations, heat sources and large amounts of smoke are generated. Under these circumstances, locating survivors using thermal or conventional cameras is not possible and it is necessary to use alternative techniques. The challenge of this work was to analyze if it is feasible the integration of an acoustic camera, developed at the University of Valladolid, on an unmanned aerial vehicle (UAV) to locate, by sound, people who are calling for help, in enclosed environments with reduced visibility. The acoustic array, based on MEMS (micro-electro-mechanical system) microphones, locates acoustic sources in space, and the UAV navigates autonomously by closed enclosures. This paper presents the first experimental results locating the angles of arrival of multiple sound sources, including the cries for help of a person, in an enclosed environment. The results are promising, as the system proves able to discriminate the noise generated by the propellers of the UAV, at the same time it identifies the angles of arrival of the direct sound signal and its first echoes reflected on the reflective surfaces. Keywords: unmanned aerial vehicle (UAV); acoustic array; people localization; environments with reduced visibility; enclosed environments 1. Introduction There are a large number of systems on the market to locate and track people. The solutions can be divided into two categories: one based on sensors that explore a surveillance space, and the other one based on devices that people carry on and that are detected by a control network [1,2]. Sensor-based solutions detect people without their active collaboration. Among these solutions, the following systems can be highlighted, those based on: (i) cameras, either in the visible and in the infrared (thermal image) frequency bands [ 3 , 4 ], (ii) cameras with structured light patterns, which measure the range of each pixel (RGB (red, green and blue) depth) [ 5 ], (iii) LIDAR (Light Detection and Ranging) systems [ 6 , 7 ], (iv) ultra-wide band RADARs (Radio Detection and Ranging) [ 8 , 9 ] and (v) acoustic systems (microphones, arrays of microphones, etc.) [10,11]. Sensors 2020,20, 597; doi:10.3390/s20030597 www.mdpi.com/journal/sensors Sensors 2020,20, 597 2 of 16 On the other hand, device-based solutions require that each person carries a device on, which can be identified through a monitoring network. The most common devices are RFID (radio frequency identification) Tags [ 12 , 13 ], GPS (global positioning system) Tracker [ 14 , 15 ], Cellular Phones [ 16 – 18 ], BLE (Bluetooth low energy) Beacons [19–21] and Wi-Fi devices (tablet, portable, cellular, etc.) [22]. It is difficult to select one system, among all these possibilities, to detect people in an enclosed space, where the following circumstances can take place: 1. There is no sensor network installed, or is not operational. 2. There is an emergency caused by a fire or an explosion with dense smoke. 3. Anonymous people to be located, not knowing their identity a priori. If people who may be present are not known, it invalidates device-based solutions, since it is unknown who they are and whether they carry any generic or specific operational device. So, under these circumstances, sensor-based solutions would be the right choice to detect people using autonomous systems (robots, drones, etc.). These systems are usually based on: (i) image identification; (ii) analysis of signals disturbed by the movement of people; and (iii) detection of sounds generated by people (whistles, cries for help, etc.). In this way, these systems are not dependent on any previous sensor installation or the sensor operability. Image-based sensors will not work in environments with reduced visibility, such as environments with dense smoke and severe fire. In these cases, it is necessary to rely on ultra-wide band radars or smart acoustic systems, which are not based on optical image. Radar-based systems are very effective even at detecting people behind walls, but have a small operational range and high cost and weight/energy consumption. Smart acoustic systems are based on arrays, which are arranged sets of identical sensors, fed in a specific manner. The beampattern of an array can be controlled by modifying its geometry (linear, planar, etc.), the sensor spacing, and the amplitude and phase excitation of each sensor [ 23 ]. Microphone arrays are a particular case of these systems. By using beamforming techniques [ 24 ], the array beampattern, particularly its main lobe, can be electronically steered to different spatial positions, allowing spatial filtering, i.e., the discrimination of acoustic sources based on their position. These arrays are used in applications such as speech processing, echo cancellation, sound sources separation and localization [25]. To get 2D spatial information, working with sensors distributed on a surface (planar arrays) is necessary. Using planar arrays leads to a high system complexity and cost, where the acoustic sensors and the associated hardware require a large space. The acronym MEMS (micro-electro-mechanical system) refers to mechanical systems with a dimension smaller than 1 mm, which are manufactured with tools and technology arising from the integrated circuits (ICs) field [ 26 ]. The application of MEMS technology to acoustic sensors has allowed the development of high-quality microphones with high SNR (signal to noise ratio), low power consumption, and high sensitivity [ 27 ]. The combination of MEMS microphones with FPGA-based architectures are very common in applications based on sound sources localization [ 28 ]. An acquisition and processing system for an acoustic array, based on digital MEMS microphones, is reduced to two basic elements: MEMS microphones and a processing system. The integration of the microphone preamplifier and the ADC (analog to digital converter) in a single chip significantly reduces costs and the space occupied by the system. The most recent acoustic system developed by the authors from the University of Valladolid, and used in this work, is based on a planar array of digital MEMS microphones [29]. The Robotics Institute of the Carnegie Mellon University (CMU), managed by Professor Sebastian Scherer, has a working line that is based on flying unmanned aerial vehicles (UAV), or drones, inside visually degraded enclosed environments to locate people and fire focuses [ 30 ]. Inside these environments, navigating using GPS is not possible. This is the reason why the drone has to navigate autonomously, at the same time as it generates 3D maps, using SLAM (simultaneous localization and Sensors 2020,20, 597 3 of 16 mapping) optical techniques [ 7 , 31 – 34 ]. This navigation system uses a FLIR (Forward-Looking Infrared) thermal camera to detect fire and an RGB depth camera to navigate [30]. Within this framework, that is, with an UAV flying in enclosed environments, where visibility could be reduced, these RGB cameras do not show a good behavior, and the performance of the thermal ones is very limited. So, under these circumstances, an idea arose to analyze if it would be feasible to include an acoustic array on a drone, to detect people in danger from their cries for help. There are studies were the acoustic arrays are used to detect and track the position of drones [ 35 , 36 ]. The signals acquired by the microphones are processed to detect the noise produced by the drone propellers, which is strong enough to be eared form the distance. The noise level will increase, as the array is closer to the drone and may mask other acoustic sources. In this paper, based in this idea, the authors study the feasibility of discriminating UAV propellers noise from distress signals to locate people in danger, using an array of MEMS microphones. Although this paper is based on discriminating distress signals from only propellers noise, this kind of noise is not the only noise source that could be present in real scenarios. Ambient and fire noises could be also present in a real scenario. These kinds of noises have a low frequency spectrum, and therefore, in analogy to the noise generated by the drone propellers, they could be discriminated using a high pass filter. In addition, the intensity level of these noise sources is significantly lower than the noise generated by the propellers that are at a very small distance from the array of microphones. Other possible noise sources could be related with explosions. This kind of noises would have an intensity level much higher than the noise generated by the propellers and that it would saturate the MEMS microphones of the array during a short interval. During explosions, the system would not be able to detect people’s cries for help. However, as people would continue to cry for help, it would be possible to detect them between explosions. Another idea obtained from the joint work of the Array Processing Group and the Robotics Institute was the use of the reflections associated to the walls of the enclosed environments in order to obtain a 3D position estimation of people. Some tests have been carried out in order to locate the angles of arrival of different sound sources, including whistles and calls for help, in indoor environments. In the tests carried out in this paper, the acoustic array was not integrated on the drone, due to its high dimensions. Equivalent scenarios were built in order to evaluate the acoustic localization algorithms in the presence of the noise generated by the drone propellers. Section 2introduces the description of the system used in this study, showing the different hardware platforms that compose the system, and the system features. Section 3presents the results obtained on the tests that were carried out in the analysis, and the corresponding discussion of these results. Finally, Section 4contains the conclusions that authors have drawn on the basis of the obtained results. 2. Materials and Methods The work shown in this paper is focused on using an unmanned aerial vehicle (UAV), or drone, to locate people in enclosed environments with reduced visibility. The objective of this work is to analyze if an acoustic array, placed on an operative UAV, can localize people cries, discriminating between the desired acoustic signals and the noise of the drone propellers. 2.1. Drone As shown in Figure 1, the drone that has been employed in this work is the Autel X-Star model. This drone is a quadrotor micro aerial vehicle. It has a 4K camera and a 3-axis gimbal that can take smooth Ultra HD video or 12MP photos. An intelligent flight control system and autopilot functions power the quadcopter. Its Dual GPS and GLONASS navigation System, the Starpoint Positioning System, and exclusive SecureFly technology keep the drone safe and stable, even in challenging situations. Sensors 2020,20, 597 4 of 16 Sensors 2020, 20, x FOR PEER REVIEW 4 of 16 Figure 1. Autel X-star drone [37]. 2.2. Acoustic System Based on MEMS Array and a myRIO Platform The acoustic images acquisition system used in this paper is based on a Uniform Planar Array (UPA) of MEMS microphones [29]. This array, entirely developed by the authors from de University of Valladolid, is a square array of 64 (8 × 8) MEMS microphones that are uniformly spaced, every 2.125 cm, in a rectangular printed circuit board (PCB), as shown in Figure 2. Figure 2. Array module with myRIO and MEMS array board. This array was designed to work in an acoustic frequency range between 4 and 16 kHz. Therefore, the 2.125 cm spacing corresponds to λ/2 for the 8 kHz central frequency. This spacing allows a good resolution for low frequencies, and also avoids grating lobes for high frequencies in the angular exploration zone of interest. For the implementation of this array, MP34DT01 digital MEMS microphones of STMicroelectronics—with PDM interface—were chosen, with the following features: low-power, omnidirectional response, 63 dB SNR, high sensitivity (−26 dBFS) and a nearly flat frequency response (±6 dB in the range of 20 Hz to 20 kHz). A MyRIO platform [38] is the base unit for this system. This platform belongs to the reconfigurable input-output (RIO) family of devices from National Instruments, which is oriented to sensors with nonstandard acquisition procedures. The embedded processor included in myRIO can run all software algorithms to generate acoustic images, so it can be used as a standalone array module formed by a myRIO connected to a MEMS array board, as shown in Figure 1. In this work, the myRIO platform works as a standalone system, controlled from a PC connected using a Wi-Fi interface. Figure 3 shows the acoustic system hardware setup. Figure 1. Autel X-star drone [37]. 2.2. Acoustic System Based on MEMS Array and a myRIO Platform The acoustic images acquisition system used in this paper is based on a Uniform Planar Array (UPA) of MEMS microphones [ 29 ]. This array, entirely developed by the authors from de University of Valladolid, is a square array of 64 (8 × 8) MEMS microphones that are uniformly spaced, every 2.125 cm, in a rectangular printed circuit board (PCB), as shown in Figure 2. Sensors 2020, 20, x FOR PEER REVIEW 4 of 16 Figure 1. Autel X-star drone [37]. 2.2. Acoustic System Based on MEMS Array and a myRIO Platform The acoustic images acquisition system used in this paper is based on a Uniform Planar Array (UPA) of MEMS microphones [29]. This array, entirely developed by the authors from de University of Valladolid, is a square array of 64 (8 × 8) MEMS microphones that are uniformly spaced, every 2.125 cm, in a rectangular printed circuit board (PCB), as shown in Figure 2. Figure 2. Array module with myRIO and MEMS array board. This array was designed to work in an acoustic frequency range between 4 and 16 kHz. Therefore, the 2.125 cm spacing corresponds to λ/2 for the 8 kHz central frequency. This spacing allows a good resolution for low frequencies, and also avoids grating lobes for high frequencies in the angular exploration zone of interest. For the implementation of this array, MP34DT01 digital MEMS microphones of STMicroelectronics—with PDM interface—were chosen, with the following features: low-power, omnidirectional response, 63 dB SNR, high sensitivity (−26 dBFS) and a nearly flat frequency response (±6 dB in the range of 20 Hz to 20 kHz). A MyRIO platform [38] is the base unit for this system. This platform belongs to the reconfigurable input-output (RIO) family of devices from National Instruments, which is oriented to sensors with nonstandard acquisition procedures. The embedded processor included in myRIO can run all software algorithms to generate acoustic images, so it can be used as a standalone array module formed by a myRIO connected to a MEMS array board, as shown in Figure 1. In this work, the myRIO platform works as a standalone system, controlled from a PC connected using a Wi-Fi interface. Figure 3 shows the acoustic system hardware setup. Figure 2. Array module with myRIO and MEMS array board. This array was designed to work in an acoustic frequency range between 4 and 16 kHz. Therefore, the 2.125 cm spacing corresponds to λ /2 for the 8 kHz central frequency. This spacing allows a good resolution for low frequencies, and also avoids grating lobes for high frequencies in the angular exploration zone of interest. For the implementation of this array, MP34DT01 digital MEMS microphones of STMicroelectronics—with PDM interface—were chosen, with the following features: low-power, omnidirectional response, 63 dB SNR, high sensitivity ( − 26 dBFS) and a nearly flat frequency response (±6 dB in the range of 20 Hz to 20 kHz). A MyRIO platform [ 38 ] is the base unit for this system. This platform belongs to the reconfigurable input-output (RIO) family of devices from National Instruments, which is oriented to sensors with nonstandard acquisition procedures. The embedded processor included in myRIO can run all software algorithms to generate acoustic images, so it can be used as a standalone array module formed by a myRIO connected to a MEMS array board, as shown in Figure 1. In this work, the myRIO platform works as a standalone system, controlled from a PC connected using a Wi-Fi interface. Figure 3shows the acoustic system hardware setup. Sensors 2020,20, 597 5 of 16 Sensors 2020, 20, x FOR PEER REVIEW 5 of 16 Figure 3. Acoustic system hardware setup. The algorithms implemented in the system [29], shown in Figure 4, can be divided into three blocks: MEMS acquisition, signal processing, and direction of arrival (DOA) assessment (using wideband beamforming). The programming language used is LabVIEW, along with its Real Time and FPGA, modules, which allows developing applications on different hardware platforms such as those used in the system: FPGA, Embedded Processor (EP) and PC. Figure 4. Processing algorithms. • In the acquisition block, each MEMS microphone performs signal acquisition with a 1-bit sigmadelta AD converter working at high speed (sample frequency of 2 million Samples per second). • In the signal processing block, several routines are implemented: first the data from MEMS microphone are deinterlaced and then they are decimated obtaining 64 independent signals with a sampled rate of 50 kHz (one from each MEMS of the array). Finally, each signal is filtered with a high-pass filter in order to eliminate the low-frequency noise from the drone propellers or other sources like environmental noise or fire that have a low-frequency spectrum. • The DOA block estimates the angles of arrival of direct and reflected signals. First, an acoustic image is obtained representing the sound pressure level (SPL) received for a set of NxN arrival directions in azimuth and elevation. From this acoustic image the angular coordinates of the local maximums are extracted, which are the arrival angles of the direct and reflected signals. The acoustic image is obtained by a deterministic wideband beamforming algorithm [23–25]. This algorithm can be considered as a spatial filter to estimate the sound pressure level (SPL) for a given direction of arrival. Although, in this first stage, the signals captured by the microphones are transmitted through the Wi-Fi interface to a PC where the DOA is assessed. This processing could be done on the ARM processor included in the myRIO platform for autonomous operation. 2.3. Acoustic Localization Using the Direction of Arrival in an Enclosed Environment In an indoor space, typically reverberant, the acoustic distress signal issued by a person is propagated directly and through reflections. In a room/corridor, with a rectangular geometry, there will be typically 5 significant signals: the direct signal (red solid line) and 4 reflected signals/echoes (dashed lines) from each of the 4 reflection surfaces: floor (blue), ceiling (brown), left side wall (green) and right side wall (yellow); as it is shown in Figure 5. Figure 3. Acoustic system hardware setup. The algorithms implemented in the system [ 29 ], shown in Figure 4, can be divided into three blocks: MEMS acquisition, signal processing, and direction of arrival (DOA) assessment (using wideband beamforming). The programming language used is LabVIEW, along with its Real Time and FPGA, modules, which allows developing applications on different hardware platforms such as those used in the system: FPGA, Embedded Processor (EP) and PC. Sensors 2020, 20, x FOR PEER REVIEW 5 of 16 Figure 3. Acoustic system hardware setup. The algorithms implemented in the system [29], shown in Figure 4, can be divided into three blocks: MEMS acquisition, signal processing, and direction of arrival (DOA) assessment (using wideband beamforming). The programming language used is LabVIEW, along with its Real Time and FPGA, modules, which allows developing applications on different hardware platforms such as those used in the system: FPGA, Embedded Processor (EP) and PC. Figure 4. Processing algorithms. • In the acquisition block, each MEMS microphone performs signal acquisition with a 1-bit sigmadelta AD converter working at high speed (sample frequency of 2 million Samples per second). • In the signal processing block, several routines are implemented: first the data from MEMS microphone are deinterlaced and then they are decimated obtaining 64 independent signals with a sampled rate of 50 kHz (one from each MEMS of the array). Finally, each signal is filtered with a high-pass filter in order to eliminate the low-frequency noise from the drone propellers or other sources like environmental noise or fire that have a low-frequency spectrum. • The DOA block estimates the angles of arrival of direct and reflected signals. First, an acoustic image is obtained representing the sound pressure level (SPL) received for a set of NxN arrival directions in azimuth and elevation. From this acoustic image the angular coordinates of the local maximums are extracted, which are the arrival angles of the direct and reflected signals. The acoustic image is obtained by a deterministic wideband beamforming algorithm [23–25]. This algorithm can be considered as a spatial filter to estimate the sound pressure level (SPL) for a given direction of arrival. Although, in this first stage, the signals captured by the microphones are transmitted through the Wi-Fi interface to a PC where the DOA is assessed. This processing could be done on the ARM processor included in the myRIO platform for autonomous operation. 2.3. Acoustic Localization Using the Direction of Arrival in an Enclosed Environment In an indoor space, typically reverberant, the acoustic distress signal issued by a person is propagated directly and through reflections. In a room/corridor, with a rectangular geometry, there will be typically 5 significant signals: the direct signal (red solid line) and 4 reflected signals/echoes (dashed lines) from each of the 4 reflection surfaces: floor (blue), ceiling (brown), left side wall (green) and right side wall (yellow); as it is shown in Figure 5. Figure 4. Processing algorithms. • In the acquisition block, each MEMS microphone performs signal acquisition with a 1-bit sigma-delta AD converter working at high speed (sample frequency of 2 million Samples per second). • In the signal processing block, several routines are implemented: first the data from MEMS microphone are deinterlaced and then they are decimated obtaining 64 independent signals with a sampled rate of 50 kHz (one from each MEMS of the array). Finally, each signal is filtered with a high-pass filter in order to eliminate the low-frequency noise from the drone propellers or other sources like environmental noise or fire that have a low-frequency spectrum. • The DOA block estimates the angles of arrival of direct and reflected signals. First, an acoustic image is obtained representing the sound pressure level (SPL) received for a set of NxN arrival directions in azimuth and elevation. From this acoustic image the angular coordinates of the local maximums are extracted, which are the arrival angles of the direct and reflected signals. The acoustic image is obtained by a deterministic wideband beamforming algorithm [ 23 – 25 ]. This algorithm can be considered as a spatial filter to estimate the sound pressure level (SPL) for a given direction of arrival. Although, in this first stage, the signals captured by the microphones are transmitted through the Wi-Fi interface to a PC where the DOA is assessed. This processing could be done on the ARM processor included in the myRIO platform for autonomous operation. 2.3. Acoustic Localization Using the Direction of Arrival in an Enclosed Environment In an indoor space, typically reverberant, the acoustic distress signal issued by a person is propagated directly and through reflections. In a room/corridor, with a rectangular geometry, there will be typically 5 significant signals: the direct signal (red solid line) and 4 reflected signals/echoes (dashed lines) from each of the 4 reflection surfaces: floor (blue), ceiling (brown), left side wall (green) and right side wall (yellow); as it is shown in Figure 5. Sensors 2020,20, 597 6 of 16 Sensors 2020, 20, x FOR PEER REVIEW 6 of 16 Figure 5. Direct (black) and reflected (colored) acoustic signals in an indoor space. Taking into account that the drone knows its position inside the indoor space, as well as geometry of the space, the position of the person to be located can be estimated. With this objective, the arrival angles (azimuth and elevation) of the direct signal and the reflected signals received by the acoustic array boarded on the drone are used to estimate the position of the person. The hypothesis of specular reflection on surfaces is assumed. With the angle of arrival information (azimuth and elevation) of the direct signal, the drone can fly towards the person for a visual identification. In addition, if the angles of arrival of the four reflected signals are used, the 3D position of the person can be estimated by triangulation. By means of this technique, it is not necessary to estimate the range of the acoustic received signals, typical of an active SONAR system, based on the pulse-echo concept. As an example, Figure 6 shows the elevation angles of the direct (red solid line) and reflected (blue dashed line) signals in the YZ plane that contains the drone and the sound source, assuming that the floor of the room is the reflective surface Figure 6. Angles of arrival in elevation. The acoustic array estimates the angles of arrival, in elevation, of the direct signal ϕ D , and the reflected signal ϕ R , knowing the height Z d of the drone with respect to the reflection surface. An equivalent estimator could also be obtained by using the ceiling as the reflection surface. Analogously, the arrival angles in azimuth of the direct signal and the signals reflected in each of the sidewalls of the room could be analyzed in the XY plane. Merging the estimations of the arrival angles (azimuth and elevation) in the two orthogonal planes of analysis, the absolute position of the acoustic source [X s , Y s , Z s ] is obtained, as a function of the absolute position of the drone [X d , Y d , Z d ], by simple trigonometric relationships. As an example, based on Figure 6, Equations (1) and (2) show the expressions that can be used to obtained X s and Z s coordinates of the acoustic source position, knowing the drone position. XX 2Z  cos Φ∙ cos Φ sin󰇛ΦΦ 󰇜 (1) ZZ  sin󰇛ΦΦ 󰇜 sin󰇛ΦΦ 󰇜 (2) Figure 5. Direct (black) and reflected (colored) acoustic signals in an indoor space. Taking into account that the drone knows its position inside the indoor space, as well as geometry of the space, the position of the person to be located can be estimated. With this objective, the arrival angles (azimuth and elevation) of the direct signal and the reflected signals received by the acoustic array boarded on the drone are used to estimate the position of the person. The hypothesis of specular reflection on surfaces is assumed. With the angle of arrival information (azimuth and elevation) of the direct signal, the drone can fly towards the person for a visual identification. In addition, if the angles of arrival of the four reflected signals are used, the 3D position of the person can be estimated by triangulation. By means of this technique, it is not necessary to estimate the range of the acoustic received signals, typical of an active SONAR system, based on the pulse-echo concept. As an example, Figure 6shows the elevation angles of the direct (red solid line) and reflected (blue dashed line) signals in the YZ plane that contains the drone and the sound source, assuming that the floor of the room is the reflective surface. Sensors 2020, 20, x FOR PEER REVIEW 6 of 16 Figure 5. Direct (black) and reflected (colored) acoustic signals in an indoor space. Taking into account that the drone knows its position inside the indoor space, as well as geometry of the space, the position of the person to be located can be estimated. With this objective, the arrival angles (azimuth and elevation) of the direct signal and the reflected signals received by the acoustic array boarded on the drone are used to estimate the position of the person. The hypothesis of specular reflection on surfaces is assumed. With the angle of arrival information (azimuth and elevation) of the direct signal, the drone can fly towards the person for a visual identification. In addition, if the angles of arrival of the four reflected signals are used, the 3D position of the person can be estimated by triangulation. By means of this technique, it is not necessary to estimate the range of the acoustic received signals, typical of an active SONAR system, based on the pulse-echo concept. As an example, Figure 6 shows the elevation angles of the direct (red solid line) and reflected (blue dashed line) signals in the YZ plane that contains the drone and the sound source, assuming that the floor of the room is the reflective surface Figure 6. Angles of arrival in elevation. The acoustic array estimates the angles of arrival, in elevation, of the direct signal ϕ D , and the reflected signal ϕ R , knowing the height Z d of the drone with respect to the reflection surface. An equivalent estimator could also be obtained by using the ceiling as the reflection surface. Analogously, the arrival angles in azimuth of the direct signal and the signals reflected in each of the sidewalls of the room could be analyzed in the XY plane. Merging the estimations of the arrival angles (azimuth and elevation) in the two orthogonal planes of analysis, the absolute position of the acoustic source [X s , Y s , Z s ] is obtained, as a function of the absolute position of the drone [X d , Y d , Z d ], by simple trigonometric relationships. As an example, based on Figure 6, Equations (1) and (2) show the expressions that can be used to obtained X s and Z s coordinates of the acoustic source position, knowing the drone position. XX 2Z  cos Φ∙ cos Φ sin󰇛ΦΦ 󰇜 (1) ZZ  sin󰇛ΦΦ 󰇜 sin󰇛ΦΦ 󰇜 (2) Figure 6. Angles of arrival in elevation. The acoustic array estimates the angles of arrival, in elevation, of the direct signal φD , and the reflected signal φR , knowing the height Z d of the drone with respect to the reflection surface. An equivalent estimator could also be obtained by using the ceiling as the reflection surface. Analogously, the arrival angles in azimuth of the direct signal and the signals reflected in each of the sidewalls of the room could be analyzed in the XY plane. Merging the estimations of the arrival angles (azimuth and elevation) in the two orthogonal planes of analysis, the absolute position of the acoustic source [X s , Y s , Z s ] is obtained, as a function of the absolute position of the drone [X d , Y d , Z d ], by simple trigonometric relationships. As an example, based on Figure 6, Equations (1) and (2) show the expressions that can be used to obtained Xsand Zs coordinates of the acoustic source position, knowing the drone position. Xs=Xd+2Zd cos ΦR·cos ΦD sin(ΦR+ΦD)(1) Zs=Zd sin(ΦR−ΦD) sin(ΦR+ΦD)(2) Sensors 2020,20, 597 7 of 16 In each analysis plane, two independent estimators can be obtained, related to the two reflection surfaces: the floor and the ceiling in elevation, and the left sidewall and the right sidewall in azimuth. The equations proposed in this section are suitable for simple environments such as those proposed in the next section. In real scenarios, rooms contain more than 6 reflective surfaces, and these are not perpendicular to each other. As a result, the number of reflections increases considerably, making it difficult to estimate the direct signal direction. Other multi-source estimation algorithms such as those based on TDoAS [39,40] will be preferable in these cases. 3. Results The objective of this publication is to determine the feasibility of using an array of MEMS microphones together with a 2D beamforming algorithm to determine the position of a sound source in an enclosed space and in the presence of a drone in operation. The 3D position of the received signal could be estimated using the direct and reflected signals, while knowing the geometry of the workspace and the position of the drone in it (provided by the SLAM system). This estimation can be done without using an active system that transmits a pulsed signal and calculates the distance of the received echo. In this way, a passive location system can be used. In a real system, the 2D array should be shipped inside the drone, but in order to simplify and accelerate the experiment, the employed work procedure anchored the drone to a fixed position and placed the 2D array in its vicinity. A speaker, excited with a set of test signals, generates the acoustic source. This speaker is placed in a predefined spatial position, determining the arrival angles of the direct signals, as well as the signals reflected in the surfaces under analysis. In order to analyze the problem in different situations, three scenarios with different configurations have been arranged: • Setup 1: In this setup, the drone is anchored to a horizontal surface and the 2D array is placed at 100 cm. This distance is the same that have been used to measure the background noise levels with a sound level meter. This setup allows the characterization of the noise generated by the four propellers of the drone. • Setup 2: In this case, the 2D array is placed at 20 cm from the drone, in order to be as close as possible to it. In this setup, the emitting source is placed at 45 cm, which is the minimum distance that the drone could approach the person, because of its physical dimensions. This second setup allows the estimation of the DOA of a nearby emission source. • Setup 3: In this case, the emitting source has been placed at 160 cm from the 2D array. This setup also adds one horizontal surface and one vertical one, in order to reflect the emitted acoustic signal mention before. The distance between the array and the sound source has been defined on the basis of the physical dimensions of the test area and the size and position of the employed reflective surfaces. This last setup allows the estimation of the DOAs of the direct signal and the reflections, of this direct signal, on a horizontal plane and on a vertical plane. The challenge of the tests is the acquisition of the DOA of some acoustic signals, by discriminating these acoustic signals from the noise generated by the drone. Therefore, in these tests, there are two different kinds of sound sources: •The acoustic signals to be detected. Two different acoustic sources have been used in this work: i. A sinusoidal pulsed signal. ii. A continuous voice signal from a radio newscaster. • The background noise, which in this case is the noise generated by the drone, which is produced by its four propellers. For the tests carried out, three background noise levels have been identified taking into account the different situations of the drone: Sensors 2020,20, 597 8 of 16 i. Minimum: When the drone is on a stationary situation (hovering). ii. Medium: When the drone describes slow horizontal movements. iii. Maximum: When the drone describes fast movements or it is taking off, that is moving vertically. The associated noise levels for these situations have been measured with a sound level meter, placed 100 cm away from the UAV, in order to not interfere with the noise measurements. Table 1 shows the range of variation of the sound pressure level for each defined background noise situation along with its typical value. Table 1. Noise levels generated by the propellers of the UAV for the defined background noise situations. Background Noise Levels Range (dB) Typical Lp (dB) Minimum 70–80 75 Medium 80–100 90 Maximum 100–120 105 3.1. Frequency Analysis of the Noise Generated by the Drone Propellers For this experiment, setup 1 has been used, as shown in Figure 7. Sensors 2020, 20, x FOR PEER REVIEW 8 of 16 iii. Maximum: When the drone describes fast movements or it is taking off, that is moving vertically. The associated noise levels for these situations have been measured with a sound level meter, placed 100 cm away from the UAV, in order to not interfere with the noise measurements. Table 1 shows the range of variation of the sound pressure level for each defined background noise situation along with its typical value. Table 1. Noise levels generated by the propellers of the UAV for the defined background noise situations. Background Noise Levels Range (dB) Typical Lp (dB) Minimum 70–80 75 Medium 80–100 90 Maximum 100–120 105 3.1. Frequency Analysis of the Noise Generated by the Drone Propellers For this experiment, setup 1 has been used, as shown in Figure 7. Figure 7. Elements distribution on Setup 1. A 100-ms signal is acquired, obtaining an averaged periodogram in intervals of 5 ms for each of the 64 MEMS microphones. Finally, the averaged periodogram for the 64 channels is displayed. An example of this averaged periodogram is shown in Figure 8. Figure 8. Averaged periodogram of the drone noise. It is observed that the bandwidth of the noise generated by the drone is mostly comprised from 0 to 2 kHz. This information is essential to exclude these frequency bands in the beamforming algorithm. A cutoff frequency of 20 kHz assures a 20-dB noise reduction in the detection frequency band. Therefore, the frequency band from 2 kHz will be used to identify the angles of arrival of the distress signals of interest and frequencies bellow 2 kHz are filtered in the signal processing block of the processing algorithm. Figure 7. Elements distribution on Setup 1. A 100-ms signal is acquired, obtaining an averaged periodogram in intervals of 5 ms for each of the 64 MEMS microphones. Finally, the averaged periodogram for the 64 channels is displayed. An example of this averaged periodogram is shown in Figure 8. Sensors 2020, 20, x FOR PEER REVIEW 8 of 16 iii. Maximum: When the drone describes fast movements or it is taking off, that is moving vertically. The associated noise levels for these situations have been measured with a sound level meter, placed 100 cm away from the UAV, in order to not interfere with the noise measurements. Table 1 shows the range of variation of the sound pressure level for each defined background noise situation along with its typical value. Table 1. Noise levels generated by the propellers of the UAV for the defined background noise situations. Background Noise Levels Range (dB) Typical Lp (dB) Minimum 70–80 75 Medium 80–100 90 Maximum 100–120 105 3.1. Frequency Analysis of the Noise Generated by the Drone Propellers For this experiment, setup 1 has been used, as shown in Figure 7. Figure 7. Elements distribution on Setup 1. A 100-ms signal is acquired, obtaining an averaged periodogram in intervals of 5 ms for each of the 64 MEMS microphones. Finally, the averaged periodogram for the 64 channels is displayed. An example of this averaged periodogram is shown in Figure 8. Figure 8. Averaged periodogram of the drone noise. It is observed that the bandwidth of the noise generated by the drone is mostly comprised from 0 to 2 kHz. This information is essential to exclude these frequency bands in the beamforming algorithm. A cutoff frequency of 20 kHz assures a 20-dB noise reduction in the detection frequency band. Therefore, the frequency band from 2 kHz will be used to identify the angles of arrival of the distress signals of interest and frequencies bellow 2 kHz are filtered in the signal processing block of the processing algorithm. Figure 8. Averaged periodogram of the drone noise. It is observed that the bandwidth of the noise generated by the drone is mostly comprised from 0 to 2 kHz. This information is essential to exclude these frequency bands in the beamforming Sensors 2020,20, 597 9 of 16 algorithm. A cutofffrequency of 20 kHz assures a 20-dB noise reduction in the detection frequency band. Therefore, the frequency band from 2 kHz will be used to identify the angles of arrival of the distress signals of interest and frequencies bellow 2 kHz are filtered in the signal processing block of the processing algorithm. In addition, discarding these frequencies allows having a better spatial resolution to identify the position of the acoustic source. Therefore, the spectral band of 2 kHz to 10 kHz will be used to perform the spatial processing on the distress signals to detect them. 3.2. Estimation of the Angle of Arrival for a Tone Pulsed with the Drone’s Noise For this experiment, setup 2 has been used, as shown in Figure 9. In these tests, the acoustic signal arrived the array from the (−6◦, 0◦) direction, in azimuth and elevation coordinates. Sensors 2020, 20, x FOR PEER REVIEW 9 of 16 In addition, discarding these frequencies allows having a better spatial resolution to identify the position of the acoustic source. Therefore, the spectral band of 2 kHz to 10 kHz will be used to perform the spatial processing on the distress signals to detect them. 3.2. Estimation of the Angle of Arrival for a Tone Pulsed with the Drone’s Noise For this experiment, setup 2 has been used, as shown in Figure 9. In these tests, the acoustic signal arrived the array from the (−6°, 0°) direction, in azimuth and elevation coordinates. Figure 9. Elements distribution on Setup 2: (a) elevation view, (b) profile view. The objective of this experiment is twofold: (i) determining if the system can obtain the angle of arrival of the signal of interest, and (ii) evaluating if the noise of the four propellers of the drone degrade the results by MEMS saturation. For this experiment, a 5-kHz sinusoidal pulse of 5 ms width has been generated, and 200 captures have been done for the three background noise levels: minimum, medium and maximum. First, a high-pass filter with a cut-off frequency of 2 kHz processed the received signals and then the acoustic images were obtained using a deterministic beamforming algorithm based on a 64-point FFT. Figure 10 presents the acoustic images corresponding to this sinusoidal pulse. In this figure, a red cross represents the real position of the acoustic signal. (a) (b) (c) Figure 10. Acoustic image versus background noise level: (a) minimum, (b) medium and (c) maximum. The energy of the 5 ms sinusoidal pulse has been adjusted in order to have a typical SNR of 15 dB, 0 dB and −15 dB, for the three defined noise levels generated by the four propellers of the UAV, Figure 9. Elements distribution on Setup 2: (a) elevation view, (b) profile view. The objective of this experiment is twofold: (i) determining if the system can obtain the angle of arrival of the signal of interest, and (ii) evaluating if the noise of the four propellers of the drone degrade the results by MEMS saturation. For this experiment, a 5-kHz sinusoidal pulse of 5 ms width has been generated, and 200 captures have been done for the three background noise levels: minimum, medium and maximum. First, a high-pass filter with a cut-offfrequency of 2 kHz processed the received signals and then the acoustic images were obtained using a deterministic beamforming algorithm based on a 64-point FFT. Figure 10 presents the acoustic images corresponding to this sinusoidal pulse. In this figure, a red cross represents the real position of the acoustic signal. Sensors 2020, 20, x FOR PEER REVIEW 9 of 16 In addition, discarding these frequencies allows having a better spatial resolution to identify the position of the acoustic source. Therefore, the spectral band of 2 kHz to 10 kHz will be used to perform the spatial processing on the distress signals to detect them. 3.2. Estimation of the Angle of Arrival for a Tone Pulsed with the Drone’s Noise For this experiment, setup 2 has been used, as shown in Figure 9. In these tests, the acoustic signal arrived the array from the (−6°, 0°) direction, in azimuth and elevation coordinates. Figure 9. Elements distribution on Setup 2: (a) elevation view, (b) profile view. The objective of this experiment is twofold: (i) determining if the system can obtain the angle of arrival of the signal of interest, and (ii) evaluating if the noise of the four propellers of the drone degrade the results by MEMS saturation. For this experiment, a 5-kHz sinusoidal pulse of 5 ms width has been generated, and 200 captures have been done for the three background noise levels: minimum, medium and maximum. First, a high-pass filter with a cut-off frequency of 2 kHz processed the received signals and then the acoustic images were obtained using a deterministic beamforming algorithm based on a 64-point FFT. Figure 10 presents the acoustic images corresponding to this sinusoidal pulse. In this figure, a red cross represents the real position of the acoustic signal. (a) (b) (c) Figure 10. Acoustic image versus background noise level: (a) minimum, (b) medium and (c) maximum. The energy of the 5 ms sinusoidal pulse has been adjusted in order to have a typical SNR of 15 dB, 0 dB and −15 dB, for the three defined noise levels generated by the four propellers of the UAV, Figure 10. Acoustic image versus background noise level: ( a ) minimum, ( b ) medium and ( c ) maximum. Sensors 2020,20, 597 16 of 16 39. Dokmanic, I.; Parhizkar, R.; Ranieri, J.; Vetterli, M. Euclidean Distance Matrices: Essential Theory, Algorithms and Applications. IEEE Signal Proc. Mag. 2015,32, 12–30. [CrossRef] 40. Le, T.K.; Ho, K.C.; Le, T.H. Rank properties for matrices constructed from Time Differences of Arrival. IEEE Trans. Signal Process. 2018,66, 3491–3503. [CrossRef] © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).