Exploring Hybrid Models For Short-Term Local Weather Forecasting in IoT Environment

MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX ISSN: 1803-3814 (Printed), 2571-3701 (Online) https://doi.org/10.13164/mendel.2023.k.307 A Hybrid Extreme Gradient Boosting and Long Short-Term Memory Algorithm for Cyber Threats Detection Reham Amin  , Ghada El-Taweel, Ahmed Fouad Ali, Mohamed Tahoun Department of Computer Science, Faculty of Computers and Informatics, Suez Canal University, Ismailia, 41522, Egypt reham [email protected]  Abstract The vast amounts of data, lack of scalability, and low detection rates of traditional intrusion detection technologies make it impossible to keep up with evolving and increasingly sophisticated cyber threats. Therefore, there is an urgent need to detect and stop cyber threats early. Deep Learning has greatly improved intrusion detection due to its ability to self-learn and extract highly accurate features. In this paper, a Hybrid XG Boosted and Long Short-Term Memory algorithm (HXGBLSTM) is proposed. A comparative analysis is conducted between the computational performance of six established evolutionary computation algorithms and the recently developed bio-inspired metaheuristic algorithm called Zebra Optimisation Algorithm. These algorithms include the Particle Swarm Optimisation Algorithm, the Bio-inspired Algorithms, Bat Optimisation Algorithm, Firefly Optimisation Algorithm, and Monarch Butterfly Optimisation Algorithm, as well as the Genetic Algorithm as an Evolutionary Algorithm. The dimensionality curse has been mitigated by using these metaheuristic methods for feature selection, and the results are compared with the wrapper-based feature selection XGBoost algorithm. The proposed algorithm uses the CSE-CIC -IDS2018 dataset, which contains the latest network attacks. XGBoost outperformed the other FS algorithms and was used as the feature selection algorithm. In evaluating the effectiveness of the newly proposed HXGBLSTM, binary and multi-class classifications are considered. When comparing the performance of the proposed HXGBLSTM for cyber threat detection, it outperforms seven innovative deep learning algorithms for binary classification and four of them for multi-class classification. Other evaluation criteria such as recall, F1 score, and precision have been also used for comparison. The results showed that the best accuracy for binary classification is 99.8%, with F1-score of 99.83%, precision of 99.85%, and recall of 99.82%, in extensive and detailed experiments conducted on a real dataset. The best accuracy, F1-score, precision, and recall for multi-class classification were all around 100%, which does give the proposed algorithm an advantage over the compared ones. Keywords: Cyber Security, Intrusion Detection, Deep Learning, Feature Selection. Received: 18 November 2023 Accepted: 11 December 2023 Online: 17 December 2023 Published: 20 December 2023 1 Introduction Since 2007, Deep Learning (DL) has become an increasingly important subfield of machine learning in various industries, including speech and pattern recognition, healthcare, vehicle design, image processing, financial forecasting, transportation, agriculture and legal applications, to name a few [16]. In addition, online Skype translation, automatic image captioning, and voice search (using technologies such as Amazon Alexa, Google Assistant, and Apple Inc. Siri) are some examples of successful DL applications [29]. Researchers and technology companies prefer to use DL to process big data effectively and efficiently [14,36]. Large companies such as Facebook, Amazon, Google, and Microsoft are also using Deep Learning algorithms to analyze massive amounts of data on a daily basis. Artificial neural networks (ANN) are modeled after neurons in the human brain and are used in DL. The number of layers exhibited, which can be thousands, is referred to as deep [36]. There are many reasons why DL is well-liked. First of all, it can leverage large amounts of data for the best training and testing. For example, the ImageNet dataset of thousands of photos is easily accessible [31]. In addition to this the use of low-cost GPUs that can benefit from cloud services and are often used to train data [16]. With traditional methods of data analysis, it is difficult to properly understand the data and uncover important facts, even with machine learning algorithms [38]. Therefore, Deep Learning is used to dig deep into the network to obtain accurate and precise data. The DL-based algorithm has many advantages when it comes to cyber threat detection. Security risks that are constantly changing include denial-of-service (DoS) 307 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX attacks, data spying, spoofing, and network resource occupancy, to name a few [2]. The traditional Deep Learning-based system is trained on a large dataset and then computes a variety of unique security threats. The system is then implemented in the real-world communications environment, where it can quickly detect the presence of hostile entities on the network in question [46]. All of these factors suggest that critical security measures are urgently needed to address these potential security issues, which DL can [25]. In addition to applying DL algorithms to real-world situations, this type of intrusion detection system assessment and tuning (IDS) uses realistic and modern datasets that depict both typical and anomalous network activity. In recent decades, several initiatives have been undertaken to create realistic datasets. Researchers have been able to adopt and evaluate more modern and optimistic datasets, although some of them have come under criticism for their shortcomings, lack of heterogeneity, limited availability, and critical level. It is believed that the selection of an appropriate dataset for intrusion detection plays a key role in ensuring the highly successful performance of Deep Learning-based IDs. Moreover, datasets are outdated due to the extensive nature of network attacks that have evolved significantly in recent years. To create an effective DL-based IDS, it is necessary to consider the current developments in IDS datasets [30]. The inclusion of a real traffic dataset, new and recent attack scenarios in CSE-CIC-IDS -2018 as opposed to recent datasets was one of several aspects considered in the selection of this dataset [8,7,33,32,2,22,26]. The CSE-CIC -IDS2018 dataset, while new, has been used to evaluate and compare various DL algorithms. It is available to the public at [13] and is very well organized. The dataset has been used extensively by the scientific community to benchmark cybersecurity because it contains a variety of attacks performed with different tools, arranged in a timeline, and combines regular and abnormal packet flows from the network [8]. In addition, the traffic was dynamically generated to mimic an enterprise network. The dataset also contains the original packet capture or PCAP files with each collected packet, in addition to the features collected from the network flows. This increases flexibility when using different pre-treatment and processing algorithms [5]. Numerous metaheuristic algorithms are available for use in feature selection. Metaheuristics can be classified into two main categories: single-solution algorithms and population-based algorithms [44]. Single solution-based algorithms may lead to local optima and prevent finding the global optimum. However, population-based algorithms are naturally able to avoid local optima [4,39]. The population-based algorithms are classified into: evolutionary algorithms, swarm intelligence, bio-inspired algorithms, and physics based algorithms. Attempting each of these algorithms is challenging and time consuming. Therefore, we select the Zebra Optimization Algorithm (ZOA), a recently created bio-inspired metaheuristic algorithm, and six other long-standing algorithms from among them. In this paper, a Hybrid XG Boosted and Long Short-Term Memory algorithm (HXGBLSTM) is proposed. Additionally, a comparative analysis is conducted between the computational performance of six established evolutionary computation algorithms and the recently developed bio-inspired metaheuristic algorithm called Zebra Optimisation Algorithm (ZOA) [41]. These algorithms include the Particle Swarm Optimisation Algorithm (PSO) [34], the Bio-inspired Algorithms (Bat Optimisation Algorithm (BOA) [17], Firefly Optimisation Algorithm (FOA) [12], and Monarch Butterfly Optimisation Algorithm (MBOA) [40], as well as the Genetic Algorithm (GA) [37] as an Evolutionary Algorithm. The dimensionality curse has been mitigated by using these metaheuristic methods for feature selection, and the results are compared with the wrapper-based feature selection XGBoost algorithm. The proposed algorithm uses the CSE-CIC -IDS2018 dataset, which contains the latest network attacks. XGBoost outperformed the other FS algorithms and was used as the feature selection algorithm. Several authors compared different algorithms to evaluate the CSE-CIC -IDS2018 dataset. These deep impressions served as our motivation to thoroughly investigate the different deep learning algorithms used to analyze the CSE-CIC -IDS2018 dataset and tune the different crucial parameters that can be used to improve the DL algorithms and their learning processes on the dataset. Below is the main contribution of the paper: •We have proposed a hybrid defensive algorithm called HXGBLSTM that combines the LSTM algorithm and the XGBoost algorithm for feature selection. •We have compared six metahuristic algorithms against the proposed algorithm in order to select features for binary classification: Zebra Optimisation Algorithm (ZOA), Bat Optimisation Algorithm (BOA), Particle Swarm Optimisation Algorithm (PSO), Firefly Optimisation Algorithm (FOA), Monarch Butterfly Optimisation Algorithm (MBOA), and Genetic Algorithm (GA). •We improved the accuracy of the proposed algorithm using two scenarios with the standard LSTM (i.e., before using FS) and the proposed HXGBLSTM (i.e., after using FS). •We evaluated the efficiency of the proposed HXGBLSTM using the results of each classification mode (binary or multiclass). The proposed HXGBLSTM for cyber threat detection outperforms seven state-of-the-art deep learning algorithms for binary classification and four for multiclass classification when performance is compared. •Our proposed algorithm proved promising when 308 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX Amin-g2igHX,gA Hybrid Extreme Gradient Boosting and Long Short-Term Memory Algorithm for Cyber Threats Detection we examined AUC, precision, recall, and the confusion matrix generated by each classification mode (CM) compared to other related algorithms. The rest of this paper is organized as follows: the related work of the most popular and current DL algorithm applied to the CSE-CIC-IDS2018 dataset is provided in Section 2. Section 3provides a detailed explanation of our proposed algorithm. Section 4explains the experimental setup, parameter adjustment, the obtained results, and a comparison with other algorithms. Finally, the paper is concluded in Section 5, where the future perspectives are presented. 2 Literature Review Hnamte et al. [23] proposed a dependable intrusion detection system using deep convolutional neural network. The framework’s performance was assessed using significant factors like detection accuracy, false positive rate, and computational efficiency. ISCX-IDS 2012, DDoS (Kaggle), CICIDS2017, and CICIDS2018 are four publicly accessible IDS datasets that are used to assess the proposed system. The system achieved a detection accuracy range of 99.79% to 100%, Alzughaibi et al. [6] proposed two Deep Neural Network (DNN) models for the CSE-CIC -IDS2018 dataset: one based on a Multi-Layer Perceptron (MLP) with BackPropagation (BP), the other on an MLP with Particle Swarm Optimization (PSO). They obtained results of 98.41% for multi-class classification and 98.97% for binary classification. Wang et al. [45] analyzed network anomaly detection using the CSE-CIC -IDS2018 dataset in binary and multiclass classification. They tested several DL algorithms including DNN, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), LSTM, CNN RNN and CNN LSTM. The CNN RNN and CNN LSTM models achieved a maximum multiclass classification accuracy of 98.84% when compared to the individual DNN, RNN, CNN, and LSTM algorithms. To analyze cyber threat data, the authors in [1] presented the Principal Component ANALYSIS (PCA) with DNN Model, which is based on DNN and PCA. By using DNN-PCA and only 12 features from the dataset, they were able to reduce the training time by half while maintaining the accuracy rate. 20% in testing and 80% in training. From the confusion matrix in the results, we can easily see the highest accuracy of 97.93%, along with a precision of 99.97% and a recall of 97.42%. An innovative deep learning algorithm for behaviorbased network intrusion detection was proposed by Antunes et al. [8] There are several hybrid algorithms are used such as CNN-PCA, CNN-Autoencoder, LSTMPCA and LSTM-Autoencoder. The DDoS attack using the LSTM-PCA classifier provides the highest accuracy of 99.9%. Al-Razib et al. [2] proposed a hybrid DNN-LSTM framework that is SDN-enabled to detect cyberattacks in smart environments. DNNLSTM, DNNGRU, and BLSTM were the classifiers tested. The highest accuracy, precision, recall, and F1 score for DNN LSTM were 99.55%, 99.36%, 99.44% and 99.42%, respectively. In the Internet of Vehicles, Ullah et al. [42] suggested HDL-IDS: a hybrid deep learning architecture for intrusion detection. LSTM and GRU were combined with a DENSE ReLU layer in the suggested model. The outcomes revealed Precision of 0.9951, Recall of 0.996, F1-Score of 0.9952, and Accuracy of 0.9951. Based on the CSE-CIC-IDS2018 dataset, Farhan et al. [18] provided a Performance analysis of intrusion detection using a deep learning algorithm. The LSTM has three layers, each having 78, 64, and 8 neurons, respectively. ReLu and Softmax are two of the activation methods that are used. The model could have been detected with an accuracy 99%. Black Widow Optimized Convolutional Long ShortTerm Memory (BWO-CONV-LSTM) Neural Networks on a MapReduce based platform was utilised by Kanna et al. [27]. The accuracy of the system for the CSECIC-IDS2018 dataset was 98.25%. To improve learning from imbalanced samples, Zhang et al. [48] presented IoT intrusion detection based on data augmentation. The authors used ICVAE-BSM, which stands for Improved Conditional Variational Autoencoder and Borderline Synthetic Minority Oversampling Technique. The system obtained an accuracy of 98.67, an F1 score of 98.50, a recall of 98.95, and a precision of 98.04. Kilincer et al. [28] proposed a thorough intrusion detection framework using boosting algorithms. The machine learning algorithms with default parameters were used to classify the newly created sub-datasets, while the Extra Tree algorithm selected the most optimal features for all datasets. The boosting algorithms include K-Nearest Neighbour (KNN), Na¨ıve Bayes (NB), Decision Trees (DT), Multilayer Perceptron (MLP), Adaptive Boosting (AdaBoost), glioblastoma (GBM), Light Gradient Boosted Machine (LGBM), and eXtreme Gradient Boosting (XGBoost). The default settings were employed with five fold cross validation and achieved an accuracy of 99.94%. For the CSE-CIC-IDS2018 dataset, Azeroual et al. [11] provided a framework for implementing the DL algorithm to Improve Intrusion Detection Systems (IDS). Using CNN, the model accuracy on validation data decreased to 83.55% after 50 iterations, but it returned a respectable accuracy of 92% after 30 iterations. Assis et al. [10] presented a Gated Recurrent Unit (GRU) deep-learning system to protect against attacks in SDNs and achieved a 97% accuracy rate. Deepdetect: Distributed Denial of Service (DDoS) threat detection using Deep Learning was presented by Asad et al. [9]. DNN (Deep Neural Network) with feed-forward backpropagation was used in the proposed model to evaluate the performance. 98% accuracy, an 309 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX F1 score of 0.99, and an AUC close to 1. For effective detection of DDoS attacks in Software Defined Networks (SDNs), Haider et al. [21] presented a deep CNN ensemble system. Three 2-d convolutional layers with 128, 64, and 32 filters, respectively, form the ensemble CNN model, along with two maximum poolings, one flattening layer, and two dense FC layers. With Sigmoid as the output layer and ReLu as the activation function in the hidden layers, the system achieved 99.45% accuracy, 99.57% precision, 99.64% recognition, and 99.61% F1 score. Ferrag et al. [32] examined deep learning algorithms for cybersecurity intrusion detection. Using CNN, Deep Boltzmann Machines (DBM), RNN, Deep Belief Networks (DBN), DNN, and Restricted Boltzmann Machine (RBM), they were able to achieve an accuracy of 97.28 to 97.37 percent. Kim et al. [26] suggested a model of intrusion detection based on convolutional neural networks. The dataset was analyzed using vanilla RNN with 10 units and CNN with two layers. When applied to the same dataset, experimental results showed that the CNN model outperformed the RNN model. 3 The Proposed HXGBLSTM Algorithm In this section, the details of the proposed Hybrid XG Boosted and Long Short-Term Memory (HXGBLSTM) algorithm is introduced. The main structure, the preparation and preprocessing of the dataset, and the assessment metrics are presented. 3.1 XGBoost Algorithm Recursive feature elimination (RFE) is a sequential backward selection algorithm that is part of the Wrapper family. It uses a unique underlying method to pick features by recursively shrinking the feature set. Guyon I initially set up RFE based on the Support Vector Machine (SVM) model, which produced excellent results during the gene selection process [47]. All samples in the training set Tis first included by the algorithm. The cross-validation is then carried out in T. The feature set size is then attempted to be decreased by iteratively creating new feature subsets. The prediction accuracy in each iteration is assessed using the Mean Absolute Error (MAE) which is shown in Equation (1), and the set of characteristics with the lowest average score based on MAE is chosen for the set of characteristics chosen. The features associated with the lowest feature importance are then removed after each iteration of feature subset computation and sorting of feature significance. The main structure of the XGBoost is shortened in Algorithm 1. MAE = n X i=1 |yi−xi| n(1) where yiis the prediction, xiis the true value and n is number of samples. Algorithm 1 XGBoost Algorithm. Require: Training set T, which includes all samples Ensure: Fold T into five folds using the crossvalidation method 1: Build a new feature subset 2: while Feature subset is not empty do 3: Evaluate the prediction accuracy based on the Mean Absolute Error (MAE) as the objective function 4: Compute and order the IMportance IM(x) of each feature xin the feature set using the Gain 5: Remove the feature with the lowest feature importance in the sequence’s backward selection All of the training samples from the training set are initially loaded into the algorithm. The crossvalidation approach is then used to fold the training set into five folds. A new feature subset is created from these five folds in step 1. The loop begins iterating through the new feature subset in step 2, taking into account just the most significant features and disregarding others, until it becomes empty. The accuracy of the prediction is assessed in step 3using the Mean Absolute Error (MAE) as the objective function. The importance of each feature xin the feature set is calculated and arranged in step 4. The feature with the lowest feature relevance in the sequence’s backward selection is eliminated in step 5. 3.2 Long Short-Term Memory (LSTM) Among the various DL algorithms, some of them, like DNN and CNN, are better suited to non-sequential data. They are capable of processing multimedia content. While being effective, they should not be used to identify patterns in time series data. For this reason, recurrent neural networks (RNNs) were developed. They are applied to activities such as speech recognition, image-to-text conversion, and video event detection [36]. RNNs have short-term memories that are based on repeating processes in hidden layers that relate to contextual data. Moreover, RNNs cannot function as long-term memory due to the problem with gradient vanishing and explosion [49]. Hence, a Long Short-Term Memory network (LSTM) for time-series prediction is proposed by Hochreiter and Schmidhuber in 1997 [24]. When memory cells are added to the hidden layer, the LSTM can govern how time-series data is stored in memory. A set of programmable gates (input, output, and forget gate) is used to transmit data between the cells in the hidden layer [19]. The vanishing gradient and explosion problem are avoided because LSTM can preserve the cell state through its gate mechanism, which can resolve both short-term and long-term memory dependency issues [20]. The basic LSTM cell in a memory cell with three gates—input gate, output gate, and forget gate—is depicted in Figure 1, which was previously explained in 310 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX Amin-g2igHX,gA Hybrid Extreme Gradient Boosting and Long Short-Term Memory Algorithm for Cyber Threats Detection Figure 1: The block diagram of LSTM cell [20]. [20]. The input gate represented by the symbol IG is in charge of monitoring the most recent data within a memory cell. The input gate’s value at time instance t is expressed in Equation (2). IG(t) = sigmoid(WxiX(t) + WhiH(t−1) + Bi) (2) The output gate manages the distribution of the most recent data to other networks. It is denoted by the symbol OG and the output gate’s value at time instance tis expressed in Equation (3). OG(t) = sigmoid(WxoX(t) + WhoH(t−1) + Bo) (3) The forget gate determines whether or not the data should be erased based on the condition of the previous cell. It is denoted by the symbol F G and the forget gate’s value at time instance tis expressed in Equation (4). F G(t) = sigmoid(Wxf X(t) + Whf H(t−1) + Bf) (4) The hidden state is depending on the cell memory state. It is denoted by the symbol H(t) and the hidden state’s value at time instance tis expressed in Equation (5). H(t) = OG(t) tanh(M(t)) (5) The cell memory state is denoted as M(t) and its value at time instance tis expressed in Equation (6). M(t) =F G(t)∗M(t−1) + IG(t)∗ (sigmoid(WxcX(t) + WhcH(t−1) + Bc)) (6) The sigmoid function is expressed in Equation (7). sigmoid(x) = 1 1 + exp−x(7) where Xdenotes the input vector, the weight matrices Wxf ,Wxi,Wxc, and Wxo represent input weights, while Whf ,Whi,Whc, and Who represent recurrent weights. Why matrix of output weights; with the corresponding bias vectors Bf,Bi,Bc,Bo, and By. Algorithm 2 HXGBLSTM Algorithm. 1: Initialize parameters Batch size, Epoch, Input shape(I0,d) and Load data instances from dataset 2: Apply data preprocessing 3: Apply XGBoost for feature selection 4: for Run = 1 to the number of runs (N)do 5: Define sequential model 6: for Epoch = 1 to the number of epochs (M) do 7: Detect and classify attacks using a fully connected Dense layer with neurons= 64 and activation = ‘Tanh’ 8: Compile the model using optimizer = ‘Adam’, loss function = ‘Mean Squared Error’ 9: Fit the model on the training set, validation set = 0.2, and batch size 10: Evaluate and predict model for the testing set 11: Obtain the overall best solution 3.3 The Structure of the Proposed HXGBLSTM The block diagram of the proposed algorithm for improving the identification of cyber security threats is shown in Figure 2. One significant benefit is that it makes use of both LSTM as a DL method and XGBoost as a feature selection algorithm. Preprocessing the data, selecting the appropriate features, classifying the data, training and validating the algorithm, and lastly conducting testing and evaluation are the five main phases of the suggested approach. The proposed HXGBLSTM’s primary structure is shortened in Algorithm 2. The main steps of the proposed algorithm can be summarized as follows: •step 1, parameters are initialized according to the values listed in Table 2. •step 2, The files in the CSE-CIC-IDS-2018 dataset contain several data instances, which makes processing the data samples stored there and merging them to include each attack label extremely time-consuming and computationally challenging. Moreover, there is a high-class imbalance in the CIC-IDS-2017 and CSE-CIC-IDS-2018 datasets, which could indicate the system’s poor accuracy and high false positive rate. For this reason, the data preprocessing phase is quite important. It is in charge of eliminating null or missing values, duplicate and inaccurate records, and the issue of class imbalance. The process of preparing the data for use in our proposed algorithm was explained in the following lines: –Elimination of NAN and infinite values: Any null, missing, or duplicate values are found and eliminated during this phase. –Label encoding: In this phase, each of the label’s data is separated. The data associated with each label is then converted into a 311 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX Figure 2: HXGBLSTM Block Diagram. numerical matrix representation using label encoding. Target labels are additionally encoded with values ranging from zero to the number of classes minus one. –Data shuffling/re-sampling: The CIC-CSE IDS 2018 dataset has an issue with class imbalance. The distribution of classes is significantly impacted by imbalanced datasets. Because of its higher prior probability, classification models tend to overclassify the larger class. Smaller class instances hence tend to be misclassified more often than larger class instances. This phase aims to address the problem of class imbalance. This can only be achieved by eliminating bias from the data and creating equal distributions, which are equal amounts of data from different attacks. The learning process is improved by avoiding biassed data and unbalanced samples. Therefore, resampling is used to equalize the data distribution across all categories and prevent overfitting, which occurs when one class has more samples than the others. –Data normalization: This is the process of rescaling data from its original range to a new range ranging from zero to one where all values fall. Additionally, the values of all datasets have been scaled using the min-max normalisation approach to fall between zero and one. All values are divided by the greatest value that is encountered, or all values are divided by the range between the maximum and minimum values after the minimum value is eliminated. Assuming that xrepresents the data value that requires normalisation, yrepresents the normalised value of x, and that the lowest and highest values within the real range are min and max. Equation 8 normalises the xvalue. y=x−min max–min (8) •step 3, the proposed algorithm uses XGBoost for feature selection. With the 22 essential features indicated in Table 7, HXGBLSTM was retrained. Searching for the Uncorrelated List of Variables (SULOV) [43] serves as the foundation for XGBoost, and the features were selected. This algorithm determines which variable pair is more strongly related with the predefined correlation threshold. •From step 7to step 10, it focuses on the creation of the X (predictor) and Y (target) variables. In this paper, a deeper LSTM network is proposed and consists of two input and output layers and three hidden layers. First, it uses the feature input layer to transfer an input layer to their representations. Afterward, the one-dimensional convolutional layer receives the sequence feed. Once the Tanh is used as an activation function, the outputs of the convolution layer are fed to max pooling layers. A dropout rate of 0.1 is applied when transferring the output to the LSTM layer. At last, the fully connected layers acquire and aggregate the data that was extracted from the LSTM layer. The final output is passed through a linear output layer for binary classification (0 for real, 1 for fraudulent), as well as multiclass classification. Training and testing sets of data were created via data splicing. While testing data is used to assess the model’s performance on untested datasets, training data is used to train the DL algorithm. The data was split 80-20, which means that 80 percent was used for training and the remaining 20 percent was utilised for testing. Following that, performance is assessed using the evaluation metrics outlined in Section 4.3. 4 Experiments and Discussions In order to validate the experimental findings, further tests with different parameters were run. Additionally, a comparison is made between the results of the standard LSTM with all features and the proposed HXGBLSTM algorithm with the selected features. Additionally, a variety of binary and multiclass classification modes were exam312 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX Amin-g2igHX,gA Hybrid Extreme Gradient Boosting and Long Short-Term Memory Algorithm for Cyber Threats Detection ined. This Section presents the experimental setups and findings. The description of the hardware and software configuration is given in Section 4.1. The CSE-CIC-IDS2018 Dataset, which is detailed in Section 4.2, was utilised to test the proposed HXGBLSTM. Performance is assessed using the evaluation metrics outlined in Section 4.3. The dataset was preprocessed and the parameters of the proposed algorithm were adjusted as explained in Section 4.4. In the initial setting of the standard LSTM, all 79 features from the dataset are fed into the model. However, in the subsequent tests, we employed XGBoost as a feature selection technique to reduce the total number of features and enhance the performance that was obtained. The feature selection phase of HXGBLSTM is described in Section 4.5. Both binary and multiclass classifications were used to assess the proposed HXGBLSTM algorithm. Section 4.6 discusses the efficiency of the proposed HXGBLSTM for binary classification, whereas Section 4.7 discusses the efficiency of the proposed HXGBLSTM for multi-class classification. Both Sections cover comparisons with other similar studies, the confusion matrix (CM) generated from the classification, and a detailed description of the performance obtained from each classification mode. 4.1 System Configuration According to the resources for hardware and software acceleration for deep learning computing [50], a GPU working with NVIDIA CUDA (Compute Unified Device Architecture) is used to reduce computation times and possibly meet real-time data processing needs. Using a Kaggle Platform instance, we ran all of our studies on NVIDIA P100 and NVIDIA T4(x2) GPUs. We used Keras with a Tensorflow backend as our deep learning framework. The NVIDIA P100 has the Pascal architecture, 16GB of RAM, 9.5 TeraFLOP/s of Single Precision FLOPs, and 732 GB/s of memory bandwidth. The NVIDIA T4, on the other hand, has 16GB of memory, 8.1 TeraFLOP/s of Single Precision FLOP/s, and 320 GB/s of memory bandwidth. For CPU tasks, the 16GB RAM allocation is raised to 30GB per session there. 4.2 CSE-CIC-IDS2018 Dataset The CSE-CIC-IDS2018 intrusion detection dataset was created in 2018 by the Communications Security Establishment and the Canadian Institute for Cybersecurity, both of which have their headquarters in Fredericton, Canada [?]. The most recent intrusion dataset, acquired to conduct real attacks, is the CSE-CIC-IDS2018 [3]. It is available to the public. CSE-CIC-IDS2018 has a larger capacity than CICIDS2017, with over Table 1: Labels for CSE-CICIDS2018 Attacks. Label attack name 1 FTP-BruteForce 2 SSH-Bruteforce 3 DDOS attack-HOIC 4 Bot 5 DoS attacks-GoldenEye 6 DoS attacks-Slowloris 7 DDOS attack-LOIC-UDP 8 Brute Force-Web 9 Brute Force-XSS 10 SQL Injection 400GB. The dataset includes both the attack dataset’s required standards and a broad range of attacks. It primarily comprises seven distinct attack scenarios: distributed denial-of-service, botnet, denial-of-service, brute force, heartbleed, web attacks, and network infiltration [13]. A dataset was created using the HTTP, HTTPS, FTP, SSH, and email protocols to simulate the online activities of 25 fake users. The sub-datasets that make up CIC-2018 were collected over a period of 10 different days, following the injection of 16 distinct forms of attacks. There are several attacks, including SSHBruteForce, FTP-BruteForce, Brute Force -XSS attack, Brute Force -Web, SQL Injection, DoS attacks using Hulk, SlowHTTPPTest, Slowloris, DoS attacks using GoldenEye, DDOS attacks using HOIC, DDOS attacks using LOIC-UDP, DDOS attacks using LOIC-HTTP, and more. In Table 1, the attacks are listed. To more accurately represent the attacks, a network infrastructure with machine diversity similar to real-world networks was developed (five departments making up the victim organisation, with fifty attacker machines, four hundred victim machines, and thirty servers) [3,2]. The dataset includes 80 features that CICFlowMeter-V3 retrieved from the traffic, together with forward and backward collected system logs and network traffic for each machine [33]. Ten files totaling 15,450,706 rows, each with 80 characteristics, make up the CSE-CIC-IDS2018 dataset [26]. Table 2: Adjusted hyperparameters of the proposed HXGBLSTM. Paramter Value Optimizer Adam Learning rate 0.001 Hidden Nodes 64 Dropout rate 0.1 Batch size 32 Epochs 50 Activation function Tanh Loss function Mean Squared Error 313 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX 4.3 Evaluation Metrics Commonly used performance metrics like accuracy, precision, recall, AUC value, and F1 are used to evaluate classifiers based on our proposed algorithm [30]. Furthermore, trials were conducted extensively to distinguish between malicious and legitimate records, so the confusion matrix (CM) was used to compute the performance metrics in our work [15,8]. The proposed algorithm was assessed using the following metrics: –Accuracy (Acc) is the percentage of all expected occurrences, whether normal or abnormal, that were correctly predicted to all observations. This metric, which is commonly used to evaluate model performance, is especially useful when the classes are not balanced. Its value is calculated using the Equation 9. Accuracy =T P +T N T P +T N +F P +F N (9) –Precision is the number of accurately predicted positive observations relative to all expected positive observations. Low false positive rates are negatively correlated with accuracy. The greater precision levels correspond to better results. Its value is calculated using the Equation 10. P recision =T P T P +F P (10) –Recall is the ratio of correctly predicted positive observations to all observations, and it should be as high as possible. Its value is calculated using the Equation 11. Recall =T P T P +F N (11) –F1 score is produced by harmonically averaging Precision and Recall. Its value is calculated using the Equation 12. F1=2∗P recision ∗Recall P recision +Recall (12) –Area Under the Curve (AUC) indicates how effectively a machine learning model matches our expectations for identifying or classifying various scenario types [35]. It indicates the probability that a positive sample will outnumber a negative sample when rating is taken into consideration. Its value is calculated using the Equation 13. AUC =n P.N (13) The components of the confusion matrix are shown in Figure 3. The following acronyms can be used to refer to the CM: Figure 3: Confusion Matrix. •TP: The model accurately categorises benign events. •TN: The model accurately identifies malicious attempts. •FP: In all observations, anomalies are incorrectly expected to be normal occurrences. This number should ideally have a low value. •FN: Malicious attacks are incorrectly classified by the model as benign occurrences. This number should ideally have a low value. 4.4 Parameter Setting Setting the different parameters was harder than it seemed at first. At this point, we decide not to use any algorithms for hyperparameter optimization. However, when the outcomes weren’t satisfactory, we manually made a random adjustment at first. Afterthat, We proceed step-by-step to manual adjustment. Using the default hyperparameter settings and without making any parameter adjustments, the dataset was split 80-20 into 80% and 20% training and testing sections, respectively. For a fair evaluation, five cross-validation splits are performed, and the mean is computed. The obtained accuracy was 97.78 percent, the loss was 0.01; the precision was 97.88; the recall was 97.65; and the AUC was 98.72 . However, in contrast to other comparable research, that was low. In this step, the hyperparameters of the proposed algorithm were randomly adjusted in an attempt to improve performance. For five cross-validation splits, the suggested approach was evaluated using the same performance metrics. The suggested algorithm’s default settings were randomly altered for thirty epochs in each run to see if it affected the outcomes. The first random adjustment was done when the hidden nodes were 256 and the filters were 64. This led to an accuracy of 97.78%, F1 of 97.73, a precision of 97.88, recall of 97.65, and an AUC of 98.72. Next, several random adjustments are implemented for the proposed algorithm, including the following: •While hidden nodes were extended to 128 and filters to 512, accuracy decreased from 97.78 to 97. The results of HXGBLSTM are 97.5 percent AUC, 97.53% F1, 97.54% precision, and 96.48% recall. •After changing the number of hidden layers and reducing hidden nodes 128—a drop in accuracy from 97.78 to 96.82 occurred. It achieves 96.7 F1, 96.7 precision, 96.79 recall, and 98.23 AUC. •By halving the number of filters and hidden nodes, the accuracy drops from 97.78 to 95. The precision 314 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX Amin-g2igHX,gA Hybrid Extreme Gradient Boosting and Long Short-Term Memory Algorithm for Cyber Threats Detection Table 3: The results of the sequential adjustment of the dropout rate for the HXGBLSTM algorithm. Adjusted Parameters Performance metrics Dropout Hidden nodes Activation Function Accuracy Loss F1 Precision Recall AUC 0.0 64 relu 94 0.01 97.82 97.88 97.83 98.81 0.1 64 relu 95 0.01 97.38 97.74 97.07 98.42 0.2 64 relu 93 0.01 96.78 97.32 96.47 98.1 0.3 64 relu 89 0.02 87.93 97.66 83.68 91.75 0.4 64 relu 88 0.02 89.98 95.06 87.6 93.49 0.5 64 relu 90 0.02 96.53 97.85 95.44 97.61 0.6 64 relu 86 0.03 86.55 98.54 81.45 90.66 0.7 64 relu 85 0.04 92.07 96.88 89.62 94.65 0.8 64 relu 78 0.04 85.9 99.35 79.12 89.53 0.9 64 relu 60 0.06 27.33 36.36 27.3 63.65 is 97.15, the recall is 96.11, and the AUC is 97.91. The F1 is 96.53. After the random adjustment produced unsatisfactory results, this step switched to sequential manual adjustment to see if altering the hyperparameters had an impact on the HXGBLSTM algorithm’s performance. The accuracy of the system is influenced by various parameters, such as the activation function, number of hidden nodes, and dropout rate. We proceed with modifying each of these elements separately. The most appropriate parameters for evaluating the HXGBLSTM were identified after several runs, and the outcomes were contrasted with those of the earlier experiments. The results of several in-depth tests conducted with various hyperparameters for the IZOA algorithm are shown in Tables 3,4, and 5in where the overall best values are reported in bold text. 4.4.1 The Adjustment of the Dropout Rate As seen in Table 3, the HXGBLSTM algorithm’s dropout rate is updated progressively from 0.1 to 0.9. In each trial, the dropout rate is increasing by 0.1 and using fixed values for hidden nodes of 64 and the relu as an activation function. Bold text reports the overall best values. It has been observed that accuracy increases from 60% to 95% when the dropout rate is decreased from 0.9 to 0.1. The results clearly show that the ideal value for the dropout is 0.1, yielding the maximum accuracy of 95%. 4.4.2 The Adjustment of the Hidden Nodes The number of hidden nodes for the HXGBLSTM algorithm is adjusted consecutively from 8 to 128 while maintaining stable values for a dropout rate of 0.1, as it was the ideal value. To perform this update, each trial’s hidden node count is doubled. The relu served as an activation function throughout these upgrades. The performance changes while updating the hidden nodes are shown in Table 3. Reducing the number of hidden nodes from 8 to 64 has been found to increase accuracy from 61.69% to 97.74%. Thus, 64 is the best value for the hidden nodes, and when the other parameters are fixed, this gives the highest accuracy of 95%. 4.4.3 The Adjustment of the Activation Function Every trial, the activation function for the HXGBLSTM method is updated using a combination of tanh, sigmoid, softmax, linear, and relu. The dropout rate of 0.1 and the number of hidden nodes of 64 were fixed because these were the best values, as indicated in Tables 3and 4, respectively. Based on the best results obtained, Table 5displays the performance progress in order during updates. It is observed that increasing the accuracy from 93.92% to 98.23% occurs when the activation function is changed from relu to tanh. The data shown clearly shows that, while other parameters are fixed, tanh is the optimal choice for the activation function, yielding the highest accuracy of 98.23%. Table 2displays the optimised adjusted hyperparameters for the HXGBLSTM algorithm. Using the Adam optimizer, the trainable parameter was changed at a learning rate of 0.001. Furthermore, it’s clear that performance changes gradually as one advances from using the default hyperparameters, which produce an accuracy of 90%, to adjusting the hidden nodes to 64, which produces an accuracy of 96.74%, to enhancing the dropout rate to 0.1, which produces an accuracy of 95.23%, and lastly choosing tanh as an activation function, which produces an accuracy of 98.23%. 4.4.4 The Adjustment of the Epochs After selecting the best settings, the next trial was advanced by increasing the number of epochs in each iteration. The epoch count increased from 30 to 50, and the number of runs increased from 5 to 20. The top performing results overall are shown in Table 6. This table shows the improvement in the binary classification performance for HXGBLSTM based on Accuracy, Loss, F1, Precision, Recall, and AUC throughout several runs and epochs. Performance is enhanced by increasing the number of runs and epochs. When the 315 MENDEL — Soft Computing Journal, Volume 29, No.gk, .2+2K#2` 2023, Brno, Czech RepublicX neural network: A novel framework and performance evaluation approach. Telematics and Informatics Reports 11 (2023), 100077. [24] Hochreiter, S., and Schmidhuber, J. Long short-term memory. Neural computation 9, 8 (1997), 1735–1780. [25] Javeed, D., Gao, T., Khan, M. T., and Shoukat, D. A hybrid intelligent framework to combat sophisticated threats in secure industries. Sensors 22, 4 (2022), 1582. [26] Jiyeon, K., Yulim, S., and Eunjung, C. An intrusion detection model based on a convolutional neural network. Journal of Multimedia Information System 6, 4 (2019), 165–172. [27] Kanna, P. R., and Santhi, P. Hybrid intrusion detection using mapreduce based black widow optimized convolutional long short-term memory neural networks. Expert Systems with Applications 194 (2022), 116545. [28] Kilincer, I. F., Ertam, F., and Sengur, A. A comprehensive intrusion detection framework using boosting algorithms. Computers and Electrical Engineering 100 (2022), 107869. [29] Malik, J., Akhunzada, A., Bibi, I., Imran, M., Musaddiq, A., and Kim, S. W. Hybrid deep learning: An efficient reconnaissance and surveillance detection mechanism in sdn. IEEE Access 8 (2020), 134695–134706. [30] Mijwil, M., Salem, I. E., and Ismaeel, M. M. The significance of machine learning and deep learning techniques in cybersecurity: A comprehensive review. Iraqi Journal For Computer Science and Mathematics 4, 1 (2023), 87–101. [31] Mishra, S., Sagban, R., Yakoob, A., and Gandhi, N. Swarm intelligence in anomaly detection systems: an overview. International Journal of Computers and Applications 43, 2 (2021), 109–118. [32] Mohamed, A. F., Leandros, M., Sotiris, M., and Helge, J. Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study. Journal of Information Security and Applications 50 (2020), 102419. [33] Mossa, G., Ghaleb, G., Faisal, A., Reem, A., and Suad, O. A detailed analysis of benchmark datasets for network intrusion detection system. Asian Journal of Research in Computer Science 7, 4 (2021), 14–33. [34] Ragab, M. Hybrid firefly particle swarm optimisation algorithm for feature selection problems. Expert Systems (2023). [35] Ren, X., Yang, W., Jiang, X., Jin, G., and Yu, Y. A deep learning framework for multimodal course recommendation based on lstm+ attention. Sustainability 14, 5 (2022), 2907. [36] Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., and M¨ uller, K.-R. Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE 109, 3 (2021), 247–278. [37] Sohail, A. Genetic algorithms in the fields of artificial intelligence and data sciences. Annals of Data Science 10, 4 (2023), 1007–1018. [38] Thakkar, A., and Lohiya, R. Role of swarm and evolutionary algorithms for intrusion detection system: A survey. Swarm and evolutionary computation 53 (2020), 100631. [39] Thuy, T. T. T., Thuan, L. D., Duc, N. H., and Minh, H. T. A study on heuristic algorithms combined with lr on a dnn-based ids model to detect iot attacks. In MENDEL (2023), vol. 29, pp. 62–70. [40] Tiwari, A. A hybrid feature selection method using an improved binary butterfly optimization algorithm and adaptive β–hill climbing. IEEE Access (2023). [41] Trojovsk´ a, E., Dehghani, M., and Trojovsk` y, P. Zebra optimization algorithm: A new bio-inspired optimization algorithm for solving optimization algorithm. IEEE Access 10 (2022), 49445–49473. [42] Ullah, S., Khan, M. A., Ahmad, J., Jamal, S. S., e Huma, Z., Hassan, M. T., Pitropakis, N., and Buchanan, W. J. Hdlids: a hybrid deep learning architecture for intrusion detection in the internet of vehicles. Sensors 22, 4 (2022), 1340. [43] Vyshnia, G. Feature importance with featurewiz python. https: //www.kaggle.com/code/gvyshnya/ jan22-tpc-feature-importance-with-featurewiz# Introduction, Jan 2022. [44] Wang, F., Zhang, W., Yang, Q., Kang, Y., Fan, Y., Wei, J., Liu, Z., Dai, S., Li, H., Li, Z., et al. Generation of a hutchinson–gilford progeria syndrome monkey model by base editing. Protein & cell 11, 11 (2020), 809–824. [45] Wang, Y.-C., Houng, Y.-C., Chen, H.-X., and Tseng, S.-M. Network anomaly intrusion detection based on deep learning approach. Sensors 23, 4 (2023), 2171. [46] Yuan, S., and Wu, X. Deep learning for insider threat detection: Review, challenges and opportunities. Computers & Security 104 (2021), 102221. [47] Zhang, B., Zhang, Y., and Jiang, X. Feature selection for global tropospheric ozone prediction based on the bo-xgboost-rfe algorithm. Scientific Reports 12, 1 (2022), 1–10. [48] Zhang, Y., and Liu, Q. On iot intrusion detection based on data augmentation for enhancing learning on unbalanced samples. Future Generation Computer Systems 133 (2022), 213–227. [49] Zhou, C., and Chen, X. Predicting china’s energy consumption: Combining machine learning with three-layer decomposition approach. Energy Reports 7 (2021), 5086–5099. [50] Zhou, L., Zhang, C., Liu, F., Qiu, Z., and He, Y. Application of deep learning in food: a review. Comprehensive reviews in food science and food safety 18, 6 (2019), 1793–1811. 322