Radar signal processing techniques for high-precision target detection in hybrid cognitive radar

 Corresponding author: Obiajulu C. Emmanuel. Copyright © 2025 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0. Radar signal processing techniques for high-precision target detection in hybrid cognitive radar Obiajulu C. Emmanuel *, Isa M. Danjuma and S.F Kolawole Department of Electrical/Electronic Engineering, Faculty of Engineering, Nigeria Defence Academy, Kaduna. World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 Publication history: Received on 19 July 2025; revised on 25 August 2025; accepted on 28 August 2025 Article DOI: https://doi.org/10.30574/wjarr.2025.27.3.3079 Abstract Radar signal processing is crucial for modern surveillance, defense, and autonomous navigation, requiring advanced techniques for accurate target detection and tracking. This paper reviews methods in hybrid cognitive radar, which integrates traditional techniques with deep learning models like YOLO, Mask R-CNN, and LSTM. Key components include Kalman filtering for predictive tracking, Doppler velocity estimation for differentiating moving objects, true track ID for consistent identification, and radar cross-section (RCS) analysis for target classification. By combining conventional radar methods with AI models, the study enhances detection accuracy and adaptability; YOLO enables rapid object detection, Mask R-CNN improves segmentation, and LSTM refines trajectory predictions. Simulation results show an increase in detection accuracy from 99.2% to 99.8%, fewer false positives, and improved trajectory predictions. The review highlights the potential of AI-driven radar technologies in defense, aerospace, and autonomous navigation, paving the way for future research in cognitive radar optimization and sensor fusion. Keywords: Radar Signal Processing; Hybrid Cognitive Radar; Target Detection and Tracking; Kalman Filtering; Doppler Velocity Estimation; Radar Cross Section (RCS) 1. Introduction Radar signal processing is essential in modern radar technology, as it enables accurate target detection, classification, and tracking in complex environments. Traditional radar systems often rely on fixed algorithms for signal interpretation, which can be insufficient in dynamic scenarios [1]. Factors such as environmental clutter, target maneuverability, and Doppler shifts can greatly impact detection accuracy, making advanced signal processing techniques necessary for improving radar performance [2]. Over the years, various techniques have been developed to enhance target detection and tracking capabilities, including Kalman filtering, Doppler velocity estimation, and radar cross-section (RCS) analysis. With the advent of artificial intelligence and deep learning, radar systems have evolved into cognitive models that are capable of real-time adaptability. Hybrid cognitive radar integrates traditional signal processing with AI-driven techniques to enhance detection accuracy and response time. This integration employs machine learning models such as YOLO (You Only Look Once) for object detection, Mask R-CNN for precise segmentation, and Long Short-Term Memory (LSTM) networks for trajectory prediction. When these models are combined with conventional radar techniques, they create an intelligent system that adapts to environmental changes and improves tracking performance. 1.1. Background on Radar Signal Processing Radar signal processing is a crucial component of modern radar systems. It focuses on extracting meaningful information from received signals to facilitate accurate target detection, classification, and tracking[2]. Radar works by World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 480 transmitting electromagnetic waves and analyzing their reflections from objects in the environment. These reflections provide valuable information about a target's location, velocity, size, and movement patterns [1]. Figure 1 Radar signal transmission process However, processing these signals effectively poses challenges due to environmental clutter, interference, noise, and the maneuverability of targets. Traditional radar signal processing involves several steps: signal detection, clutter suppression, Doppler processing, and tracking. Techniques like the Fast Fourier Transform (FFT) are employed to analyze frequency components, while filtering methods such as Kalman filters assist in estimating target motion. Adaptive signal processing methods, including Constant False Alarm Rate (CFAR) detection, enhance radar performance in dynamic environments by adjusting detection thresholds based on background noise levels. Over the years, radar systems have evolved from simple pulse and continuous-wave radars to more sophisticated cognitive radar systems that utilize machine learning and artificial intelligence (AI). These advancements allow for real-time adaptability, enabling radar systems to adjust their parameters according to changing environmental conditions. Techniques such as Doppler velocity estimation, radar cross-section (RCS) analysis, and true track identification significantly enhance the accuracy and reliability of radar operations, particularly in high-clutter environments [3]. The integration of AI-driven models has transformed radar systems by utilizing deep learning algorithms for enhanced detection and tracking capabilities [3]. Hybrid cognitive radar systems, which merge traditional signal processing techniques with advanced models like YOLO (You Only Look Once), Mask R-CNN, and Long Short-Term Memory (LSTM) networks, have significantly improved accuracy and efficiency. These systems can manage large amounts of radar data, minimize false alarms, and adapt to unpredictable target movements. As radar applications expand into areas such as defense, aerospace, autonomous navigation, and environmental monitoring, the demand for high-precision signal processing methods continues to increase. The combination of AI and advanced filtering techniques ensures that modern radar systems remain robust and capable of managing complex detection and tracking scenarios, setting new standards for accuracy and adaptability in target recognition. 1.2. Overview of Hybrid Cognitive Radar (YOLO, Mask R-CNN, LSTM) Hybrid cognitive radar represents a significant advancement in radar technology by integrating traditional signal processing with artificial intelligence (AI) and deep learning techniques. This integration enhances target detection, classification, and tracking capabilities. Unlike conventional radar systems, which rely on static algorithms and predefined parameters, cognitive radar systems are designed to adapt dynamically to environmental changes. They learn from past observations and optimize their signal processing strategies in real-time. The incorporation of machine learning models, such as YOLO (You Only Look Once), Mask R-CNN, and Long Short-Term Memory (LSTM) networks, significantly improves detection accuracy, object segmentation, and trajectory prediction. YOLO, a deep learning-based object detection model, plays a crucial role in the initial detection phase of hybrid cognitive radar systems [4]. Known for its high-speed processing and real-time detection capabilities, YOLO efficiently identifies targets in radar images by segmenting objects and classifying them into predefined categories. The advantage of YOLO is its ability to perform single-stage detection, which means it processes the entire image in a single forward pass. This makes YOLO highly suitable for real-time radar applications where speed is critical. While YOLO excels at fast object detection, it does have limitations in precisely segmenting object boundaries. This is where Mask R-CNN comes into play. Mask R-CNN is an advanced deep learning model that specializes in instance segmentation, providing pixel-level accuracy in object recognition. In hybrid cognitive radar systems, Mask R-CNN refines the detections made by YOLO by accurately World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 481 identifying object contours. This capability is particularly useful for detecting partially obscured targets in cluttered environments [5]. By integrating Mask R-CNN, the system achieves more detailed and precise target identification, thereby reducing false positives and improving overall accuracy [6]. Figure 2 Mask R-CNN model structure LSTM, or Long Short-Term Memory, is a type of recurrent neural network (RNN) used in hybrid cognitive radar to improve target tracking and trajectory prediction. Unlike traditional radar tracking methods that primarily depend on Kalman filters or other statistical models, LSTM networks can learn patterns from sequential data, making them particularly effective for predicting the future movements of detected targets, especially in situations where targets exhibit non-linear motion or rapid maneuvering. By analyzing historical radar data, LSTM enhances tracking accuracy, allowing the radar system to anticipate and respond to target movements in real-time. The integration of YOLO, Mask R-CNN, and LSTM with conventional radar signal processing techniques, such as Kalman filtering, Doppler velocity estimation, and radar cross-section (RCS) analysis, results in a highly adaptive and intelligent radar system that not only improves detection and tracking capabilities but also reduces false alarms, optimizes resource allocation, and enhances overall situational awareness. These advancements position hybrid cognitive radar as a promising solution for applications in defense, autonomous navigation, and remote sensing, where high-precision target detection is critical. 2. Methodology 2.1. Kalman Filter Kalman filters are techniques used to estimate the state of a dynamic system based on a series of noisy observations. They are widely applied in various fields for filtering and forecasting. In this project, the Kalman filter was crucial in refining and predicting the state of moving targets over time. It worked alongside the temporal data processed by the LSTM and the detection results from YOLO and Mask R-CNN, enhancing the overall accuracy and consistency of tracking. The prediction and the update parameters are the equations used in calculating the Kalman filter which can be seen in equations (2.1), (2.2), (2.3) and (2.4). Prediction [7] 𝑥𝑘|𝑘−1 =𝐴𝑥𝑘−1|𝑘−1 +𝐵𝑢𝑘 2.1 𝑃𝑘|𝑘−1 =𝐴𝑃𝑘−1|𝑘−1𝐴𝑇+ 𝑄 2.2 Where x_(k|k-1) is the predicted state estimate, P_(k|k-1) is the predicted error covariance, A is the state transition matrix, B is the control input matrix, u_k is the control input and Q is the process noise covariance. World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 482 Update 𝐾𝑘=𝑃𝑘|𝑘−1𝐻𝑇(𝐻𝑃𝑘|𝑘−1𝐻𝑇+𝑅)−1 (2.3) 𝑥𝑘|𝑘 =𝑥𝑘|𝑘−1 +𝐾𝑘(𝑧𝑘−𝐻𝑥𝑘|𝑘−1) (2.4) 𝑃𝑘|𝑘 =(𝐼−𝐾𝑘𝐻)𝑃𝑘|𝑘−1 (2.5) Where K_k is the Kalman gain, H is the measurement matrix, R is the measurement noise covariance, z_k is the measurement at time k and I is the identity matrix. Incorporating the Kalman filter into the hybrid model significantly improved the radar system's tracking capabilities by reducing observation noise and enhancing predictions made by the LSTM model. By defining the state transition matrix (A), the control input matrix (B), and fine-tuning the process and measurement noise covariances (Q and R), the Kalman filter increased detection accuracy from 89% to 93%, corresponding to a 4% increase in true positive rates for the YOLO model. Additionally, the segmentation accuracy of Mask R-CNN improved from 88% to 92%, resulting in a 3% decrease in false positives and a 5% increase in Intersection over Union (IoU) scores. For the LSTM component, applying the Kalman filter enhanced the hybrid model's detection accuracy from 99.2% to 99.8%, while also reducing tracking errors by 5%. Overall, these advancements contributed to improved performance metrics for the system, with the hybrid model achieving a detection accuracy of 99%, precision of 92%, recall of 95%, and an F1 score of 93.5%, underscoring the essential role of the Kalman filter in enhancing the efficiency of the radar system. 2.2. Doppler Velocity Doppler velocity was essential in this hybrid cognitive radar project as it offered information about the relative motion of a target, enabling the differentiation between stationary and moving objects, and allowing for speed estimation[8]. This parameter, obtained from the Doppler shift resulting from relative motion, is computed using the equation 𝜐𝒹 = 𝑓𝒹.𝒸 2.𝑓𝓉 2.6 where f_d is the Doppler shift, c is the speed of light, andf_t is the transmitted frequency. By integrating Doppler velocity into radar signal processing, the system can more effectively detect dynamic targets while disregarding stationary clutter, thus improving detection and tracking accuracy. In the hybrid model, Doppler velocity serves as a crucial feature that enhances the classification and tracking of mobile targets, as discussed in Section 3. For the YOLO model, Doppler velocity acts as an auxiliary input in conjunction with spatial radar data, allowing the model to focus on dynamic objects during detection. Meanwhile, Mask R-CNN incorporates Doppler velocity into its feature set to cluster moving targets with similar velocity patterns, which leads to improved segmentation accuracy. The LSTM model uses sequential Doppler velocity information to forecast trajectories and speed changes, making it essential for tracking temporal dynamics in active scenarios. Simulation experiments were conducted using the dataset to validate the hybrid system's incorporation of Doppler velocity. Scenarios with varying target speeds were generated from the dataset to analyze changes in Doppler shifts, which were then processed by the hybrid model. YOLO, Mask RCNN, and LSTM collaborated to extract and apply Doppler velocity, enhancing the radar system's ability to detect, classify, and monitor targets in complex environments. This integration emphasizes the significant role of Doppler velocity in optimizing radar performance. 2.3. True Track ID The true track ID is essential for cognitive radar systems, enabling reliable target identification during radar sweeps, particularly in environments with overlapping paths. It is created by combining a target's positional (RRR, θ) and kinematic (vr, σ) data through a specialized hashing function. Algorithms like Kalman filtering maintain track continuity, while YOLO assigns initial track IDs based on position and velocity data, and Mask R-CNN enhances precision by connecting IDs to segmentation outputs. Long Short-Term Memory (LSTM) networks use true track IDs to link World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 483 sequential data points, reducing data association errors in multi-target scenarios. Simulation experiments assess the algorithm's ability to maintain accurate track IDs amid intersecting trajectories. The hybrid model effectively associates observations with stable track IDs, improving detection, segmentation accuracy, and trajectory forecasting, highlighting the true track ID's vital role in adapting cognitive radar systems for real-world applications. 2.4. Radar Cross-Section (RCS) Radar Cross-Section (RCS) measures how effectively a target can reflect radar waves, which is influenced by its size, shape, and the materials used in its construction [9]. This characteristic is crucial for distinguishing between different types of targets, such as drones and vehicles. This advanced cognitive radar utilizes RCS information to enhance the accuracy of classification. RCS (σ) measures the target's ability to reflect radar signals. It is given by [10]: 𝜎 =4𝜋𝑅2(|𝐸𝑟| |𝐸𝑖|)2 2.7 Where σ is RCS, R is Target range (m), E_r is Electric field of the reflected wave (V/m), E_i is Electric field of the incident wave (V/m). In the experiments with the dataset, targets with various Radar Cross Section (RCS) values were assessed under different angles and clutter levels to evaluate the system's effectiveness. RCS enhances target recognition and improves radar performance in complex environments, with the hybrid model using YOLO to prioritize targets with higher reflectivity in cluttered settings. Meanwhile, Mask R-CNN employs RCS thresholds for accurate classification, effectively distinguishing overlapping targets. Additionally, the Long Short-Term Memory (LSTM) network utilizes temporal RCS information to track changes in target characteristics, which helps refine trajectory predictions. By integrating RCS with radar imagery and sequential data, the hybrid radar system significantly enhances detection, classification, and tracking capabilities. 2.5. Radar sweep optimization In this research Radar sweep was used to scan the environment, detect and localize targets, providing essential spatial and temporal data for tracking. It generates a point cloud with features such as sensor ID, range, azimuth, X and Y coordinates (x=r⋅co s⁡θ,y=r⋅si n⁡θ) and radial velocity. The equations used for the radar sweep are as follows[11]: (a) Range (R) The range of the target is calculated using the time delay (t) of the received signal: 𝑅 =𝑐⋅𝑡 2 2.8 Where 𝑅 is Range, 𝑡 Time delay (s), 𝑐 is Speed of light (3×108𝑚/𝑠) (𝑏) 𝐴𝑧𝑖𝑚𝑢𝑡ℎ (𝜃): The azimuth angle is determined using the phase difference between received signals from multiple antennas 𝜃 =arcsin(∆𝜙.𝜆 2𝜋𝑑) 2.9 Where 𝜃 is Azimuth angle (radians), ∆𝜙 is Phase difference (radians), 𝜆 is Wavelength (m), 𝑑 is Antenna separation (m) (c) Radial Velocity (𝑣𝑟): Radial velocity is related to Doppler frequency shift World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 484 𝑣𝑟=𝑓𝑑.𝜆 2 2.10 Where v_r is Radial velocity (m/s), f_d is Doppler frequency shift (Hz) and λ is Wavelength (m) (d) Compensated Radial Velocity (v_c ): Compensated radial velocity accounts for clutter suppression and platform motion 𝑣𝑐=𝑣𝑟−𝑣𝑝 2.11 Where v_c is Compensated radial velocity (m/s), v_r is Radial velocity (m/s) and v_p is Platform velocity (m/s) The hybrid model enhances target detection and tracking by integrating several methodologies. YOLO employs range (R) and azimuth (θ) for precise localization, while Mask R-CNN utilizes range (R) to improve target segmentation. LSTM analyzes radial velocity, compensated radial velocity, and azimuth (θ) to make accurate trajectory predictions, enabling dynamic tracking and anticipation of target movements. Simulated radar sweeps in diverse environments assess the model's effectiveness, showcasing its strengths in detecting and localizing targets. Collectively, these components enrich the hybrid cognitive radar framework, ensuring precise detection, effective segmentation, and adaptive tracking of targets in real time. 2.6. Workflow: How the Parameters were Utilized Together and Advantages of Integration The hybrid cognitive radar process begins with radar signal analysis, identifying key details like Doppler velocity, true track ID, radar cross-section (RCS), and radar sweep information to form a dataset used by various models. The YOLO model uses this dataset to find potential targets, creating bounding boxes based on range, RCS, and Doppler velocity, which sets the stage for further analysis. Mask R-CNN enhances YOLO's results by utilizing additional information such as RCS and azimuth, improving target segmentation and classification even in cluttered environments. The LSTM model then analyzes moving data to predict target movements and future locations, helping the system respond consistently to changes. A feedback loop enables LSTM predictions to adjust radar settings and improve real-time signal processing, so if YOLO detects a vehicle, Mask R-CNN refines its shape and classification, while LSTM continuously tracks its trajectory. This approach enhances the system's adaptability and effectiveness, significantly improving detection, classification, and tracking to create a highly accurate and flexible radar solution. Table 1 Summary of signal processing integration Signal Parameter Equation Used Used in YOLO Used in Mask RCNN Used in LSTM Doppler Velocity 𝜐𝒹 = 𝑓𝒹.𝒸 2.𝑓𝓉 Filters moving targets Classifies motion types Tracks speed variations True Track ID Track ID=Hash (R, θ, vr, σ) Generates target IDs Refines segmentation Maintains target identity Radar Sweep (Range) 𝑅 =𝑐⋅𝑡 2 Localizes targets Refines segmentation Tracks position changes Radar Sweep (Azimuth) 𝜃 =arcsin(∆𝜙.𝜆 2𝜋𝑑) Localizes targets Refines segmentation Tracks angular movement Radial Velocity 𝑣𝑟=𝑓𝑑.𝜆 2 Enhances detection Refines segmentation Tracks motion patterns Compensated Radial Velocity 𝑣𝑐=𝑣𝑟−𝑣𝑝 Filters clutter Improves classification Adapts to platform motion World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 485 Kalman Filter 𝑥𝑘|𝑘−1 =𝐴𝑥𝑘−1|𝑘−1 +𝐵𝑢𝑘 Predicts and corrects target state (position, velocity) for smooth tracking and noise reduction. Predicts and corrects target state (position, velocity) for smooth tracking and noise reduction. Uses filtered data to improve temporal trajectory predictions and minimize clutter interference. 3. Results 3.1. Doppler Velocity The radar output depicted in Figure 3 provides thorough quantitative information about the identified targets. The range measurement of 12.830 meters indicates the distance of the detected object from the radar, which is crucial for accurate localization in a 3D environment. The azimuth angle of -42.546° specifies the object's horizontal position relative to the radar, allowing for precise angular tracking. The Radar Cross Section (RCS) value of 4.891 dBsm indicates the target’s reflectivity; lower values suggest weaker signal returns, possibly due to the target’s small size or the characteristics of its material. The X (CC) and Y (CC) Cartesian coordinates, measured at 8.762 m and 12.557 m respectively, convert the polar radar data into a spatial grid, facilitating accurate placement of the target within the radar's coverage area. Figure 3 Doppler Velocity The radial velocity of 4.328 m/s indicates the object's speed toward the radar, and after adjustment for external influences, it becomes a compensated radial velocity of 4.325 m/s. These parameters, obtained from Doppler velocity integration, were crucial for the hybrid cognitive radar system, where range, azimuth, and velocity were integrated into the YOLO and Mask R-CNN frameworks to enhance the accuracy of moving target detection and segmentation. Additionally, the compensated radial velocity improved the precision of motion prediction and tracking using the LSTM model. 3.2. True Track ID Figure 4 provides radar data for a "Truck" (True Track ID 3). The Range of 58.891 meters indicates the distance from the radar sensor. An Azimuth of -32.128° shows the truck’s angular position relative to the radar's horizontal axis. The Radar Cross Section (RCS) value of 16.848 dBsm suggests moderate reflectivity, likely due to the truck's metallic structure. The X (62.283 m) and Y (-6.626 m) coordinates convert the radar data into Cartesian coordinates, pinpointing the truck's location relative to the radar. World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 486 Figure 4 True Track ID The Radial Velocity of -1.574 m/s indicates the truck's speed toward the radar, while the adjusted Compensated Radial Velocity of 17.331 m/s accounts for the radar's motion and environmental conditions. These parameters were integrated into a hybrid cognitive machine learning framework to improve target detection and tracking. The YOLO model utilized range and azimuth data for precise truck positioning, while Mask R-CNN effectively isolated the truck from nearby objects. Additionally, the LSTM model leveraged the compensated radial velocity to enhance trajectory prediction over time, providing more consistent tracking in environments with multiple objects. 3.3. Radar Cross Section Figure 5 Radar Cross Section The radar data in Figure 5 shows a detected object identified as a "Car" with a True Track ID of 2. The range is 91.649 meters, which is the straight-line distance from the radar to the car. This measurement is important for finding the car's location. The azimuth angle, measured at 15.772°, indicates the direction of the car relative to the radar. This helps in positioning the car accurately. The Radar Cross Section (RCS) is 13.403 dBsm, which measures how well the car reflects radar signals. This low RCS value suggests that the car is smaller or less reflective. The Cartesian coordinates for the car World Journal of Advanced Research and Reviews, 2025, 27(03), 479–490 487 are X (CC) = 93.811 meters and Y (CC) = -14.210 meters. These coordinates show the car's position in the radar's field of view. The Radial Velocity, measured at 8.920 m/s, indicates the car's speed relative to the radar, with a positive value showing it's moving away. After accounting for platform motion and environmental factors, the Compensated Radial Velocity is 21.114 m/s, reflecting the object's corrected dynamics. This data is essential in a hybrid cognitive machine learning framework for object detection, tracking, and classification. The YOLO model uses range and azimuth for positioning, while Mask R-CNN enhances segmentation with radar cross-section and positional data. Meanwhile, the LSTM model employs radial velocity to predict trajectories, facilitating reliable tracking even in dynamic environments with multiple moving objects. 3.4. Radar Sweep Visualization Figure 6 illustrates a radar sweep visualization that represents 1,567 loaded scenes with 256 radar points each, showcasing the radar's capacity to handle large data volumes. Each point signifies the reflection from an object, indicating its range and azimuth position. A notable concentration of targets is observed between 45° to 70° and 330° to 0°, likely due to road traffic patterns. The strength of these points reflects larger objects or vehicle clusters, emphasizing areas of heightened activity within the radar's surveillance zone. Figure 6 Radar sweep visualization This visualization is essential for the hybrid cognitive radar system, which supports YOLO, Mask R-CNN, and LSTM models for object detection, classification, and trajectory forecasting. Concentrated radar regions enable the models to focus on areas with higher object activity, enhancing detection accuracy and adaptability in real-time situations. By utilizing data from 1,567 scenes, the system ensures reliable performance in dynamic environments, effectively managing varying object densities and orientations. This targeted approach improves the radar's efficiency and accuracy when tracking multiple objects simultaneously. 3.5. Radar Map The radar sweep visualization map shown in Figure 7 provides a spatial representation of identified objects, with their radial velocities (measured in meters per second, or m/s) indicated through color coding. The chart highlights several stationary objects located at different coordinates (X and Y positions in meters) within the radar's detection range. These static objects maintain consistent radial velocities of approximately 0 m/s, as indicated by their color placement near the blue and red areas on the velocity scale. This consistent behavior suggests the presence of fixed objects, such as poles, barriers, or parked cars, that have been identified by the radar system.