Extended Abstract of MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection [Póster]

Extended Abstract of MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection Aqib Rashid King’s College London Strand, London WC2R 2LS, United Kingdom aqib[email protected] Jose Such Departament de Sistemes Informatics i Computacio Universitat Politecnica de Valencia, Spain [email protected] Abstract—This extended abstract summarizes our publication of the same title in IEEE Transactions on Information Forensics and Security (TIFS) where we present MalProtect, a stateful defense against query attacks in the malware detection domain. MalProtect uses several threat indicators to detect attacks. Our results show that it reduces the evasion rate of adversarial query attacks by 80+% in Android and Windows malware, across a range of attacker scenarios. In the first evaluation of its kind, we show that MalProtect outperforms prior stateful defenses, especially under the peak adversarial threat. Index Terms—Jornadas, Ciberseguridad Tipo de contribuci´ on: Investigaci´ on ya publicada (l´ ımite 2 p´ aginas) I. INTRODUCTION ML has offered enormous capabilities, leading to the largescale use of Machine-Learning-as-a-Service (MLaaS) [1], which allows users to leverage remotely-hosted ML models by requesting predictions from them [2]. Recent work has shown that these systems are vulnerable to adversarial query attacks (e.g., [3], [4]), in which an attacker iteratively perturbs a malware sample based on feedback from the target model that includes its predicted output. Using the feedback, the perturbations to the input sample are tuned across the queries to observe how the target model responds until, eventually, the truly malicious sample is classified as benignware. Stateful defenses have been proposed to protect ML prediction models against query attacks by offering greater system awareness. This is achieved by monitoring queries received by the system in the look for queries abnormally similar to each other [5], [6] or out of distribution [6], [7]. However, they have not been considered in the malware detection domain, where more constraints exist regarding the discrete representation of feature vectors and the preservation of malicious functionality when generating adversarial examples [8], [9]. In our journal paper [10] that we summarize in this extended abstract, we present MalProtect, which is a modelagnostic stateful defense against query attacks in the ML- based malware detection domain. We make two key contributions: (1) We propose the first stateful defense against adversarial query attacks in ML-based malware detection, MalProtect, which uses several threat indicators to prevent query attacks and the generation of adversarial examples; (2) We provide the first evaluation of several prior stateful defenses applied to the ML-based malware detection domain, showing that MalProtect reduces the evasion rate of query attacks across Android and Windows under a range of threat models and attacker scenarios. II. MALPROTECT Figure 1 shows that MalProtect forms a layer of protection that monitors queries before they reach the prediction model F, which is the ML-based malware detection classifier to be protected. This allows MalProtect to detect adversarial queries that may evade the classifier (i.e., prediction model). The prediction model Fcan be a vanilla ML model or a defense (e.g., an adversarially-trained model, etc.). By having a defense as the prediction model used with MalProtect, it can offer defense in depth, as multiple layers of security can contribute to the overall security of the system. Time D: Attack detected ? 2. Analyze Query History (Q) using threat indicators Score vector, S (4.) Take defensive action 1. User submits qlatest No 3. Detect attack using decision model, D Yes q1 q2 q3 q4 ... qn qlatest Ind. 1 0.95... Ind. 2 0.81... ... ... MalProtect (4.) Return prediction from FClassifier, F (prediction model) Fig. 1: Overview of MalProtect. When a user submits a query (Step 1), MalProtect employs several threat indicators to analyze the sequence of queries for attack detection (Step 2). This is similar to a system that makes decisions using data from multiple sensors [11]. The threat indicators use different criteria, principally driven by an anomaly detection-based approach. This includes examining aspects such as the autoencoder loss of queries, the distance between queries, the features shared across queries, and whether other characteristics of queries fit within different distributions. Based on the state of the query history, each indicator produces a score that reflects the likelihood of an attack in progress according to that indicator, with adversarial queries expected to cause higher scores. A decision model then takes the indicator scores as its input and predicts whether there is an attack in progress (Step 3). If an attack is detected, MalProtect takes some defensive action. If no attack JNIC 2024 ISBN:978-84-09-62140-8 546 is detected, the prediction model Freturns a prediction for the user’s query (Step 4). Not only do the scores assist in interpreting how MalProtect produces a particular decision, but as we show later, each threat indicator can be analyzed to determine its influence on the final decision. Next, we provide details about each of MalProtect’s indicators. Indicator 1. Distance between queries: This indicator assesses the similarity of qlatest to other queries in Q, which is the primary method adopted by some stateful defenses (e.g., [5], [12]). A smaller distance between queries is an indication of iterative perturbations during a query attack, as an attacker makes modifications to their queries to assess how the target model responds [5], [3]. Indicator 2. Enabled features shared across queries: This indicator assesses whether qlatest shares a significant number of enabled features with another query in Qbeyond the norm. The rationale behind this indicator is that an attacker could enable unused features in qlatest to intentionally increase the L0distance between queries in order to evade basic similarity detection (e.g., while trying to evade Indicator 1). This is an attack strategy specific to the malware detection domain. That is, the query distance can be increased (e.g., by adding features in bulk) without disrupting the original malicious functionality (e.g., through dead-code, opaque predicates). Conversely, in domains such as image recognition, this attack strategy would not be as useful since adding features to deliberately increase the query distance in this manner would lead to the original image being distorted. Indicators 3A & 3B. Number of enabled features: An attacker could also rapidly traverse the decision boundary by enabling a substantial number of benign features in qlatest at once. That is, by adding many features to a malware sample that are present in benign executables, a misclassification can be achieved. Therefore, an anomalous number of enabled features could indicate an attack attempt. Hence, the Indicator 3A is the percentage change between the number of enabled features in qlatest (|qlatest|) and the average number of enabled features in training data samples (avgF eaturesD). However, the Indicator 3A only assesses whether |qlatest|is anomalous considering the training data and does not compare with other queries in Q. Therefore, we also use the Indicator 3B, which assesses to what extent the number of enabled features in qlatest is anomalous considering the queries in Q. The Indicator 3B uses the empirical rule (i.e., 3-σrule) [13], which is a standard technique in anomaly detection. Indicators 4A & 4B. Distribution similarity via autoencoder reconstruction loss: An autoencoder is a neural network where the input and output are the same but the hidden layers have fewer neurons [14]. This limits the amount of information that can travel through the model, requiring it to learn a compressed version of the input. Interestingly, input samples that belong to the distribution of the autoencoder’s training data will produce a smaller distance between the input and output representations (known as the reconstruction loss) [15]. Thus, we hypothesize that an autoencoder trained on legitimate queries will produce a significantly higher reconstruction loss for an adversarial query. As with Indicator 3B, we also consider the case where qlatest is only compared to other queries in Qregardless of the training data distribution. In this way, the Indicator 4B uses the empirical rule to assess whether the reconstruction loss of qlatest is anomalous considering the queries in Q. III. EVALUATION The details and results of our in-depth evaluation of Mal- Protect can be consulted in the journal paper [10]. As a summary, our results show that it reduces the evasion rate of adversarial query attacks by 80+% in Android and Windows malware, across a range of attacker scenarios (black-box, gray-box, white-box). Our results also show that MalProtect outperforms prior stateful defenses [12], [6], [5], especially under the peak adversarial threat. Finally, we also show the influence of the different indicators of MalProtect in detecting query attacks. REFERENCES [1] G. Apruzzese, H. S. Anderson, S. Dambra, D. Freeman, F. Pierazzi, and K. A. Roundy, “”real attackers don’t compute gradients”: Bridging the gap between adversarial ml research and practice,” in Proceedings of the 1st IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2023. [2] H. Yu, K. Yang, T. Zhang, Y.-Y. Tsai, T.-Y. Ho, and Y. Jin, “Cloudleak: Large-scale deep learning models stealing through adversarial examples.” in NDSS, 2020. [3] J. Chen, M. I. Jordan, and M. J. Wainwright, “Hopskipjumpattack: A query-efficient decision-based attack,” in 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 2020, pp. 1277–1294. [4] I. Rosenberg, A. Shabtai, Y. Elovici, and L. Rokach, “Query-efficient black-box attack against sequence-based malware classifiers,” in Annual Computer Security Applications Conference, 2020, pp. 611–626. [5] S. Chen, N. Carlini, and D. Wagner, “Stateful detection of blackbox adversarial attacks,” in Proceedings of the 1st ACM Workshop on Security and Privacy on Artificial Intelligence, 2020, pp. 30–39. [6] M. Juuti, S. Szyller, S. Marchal, and N. Asokan, “Prada: protecting against dnn model stealing attacks,” in 2019 IEEE European Symposium on Security and Privacy (EuroS&P). IEEE, 2019, pp. 512–527. [7] S. Kariyappa and M. K. Qureshi, “Defending against model stealing attacks with adaptive misinformation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. [8] I. Rosenberg, A. Shabtai, L. Rokach, and Y. Elovici, “Generic black-box end-to-end attack against state of the art api call based malware classifiers,” in International Symposium on Research in Attacks, Intrusions, and Defenses. Springer, 2018, pp. 490–510. [9] B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. ˇ Srndi´ c, P. Laskov, G. Giacinto, and F. Roli, “Evasion attacks against machine learning at test time,” in Joint European conference on machine learning and knowledge discovery in databases. Springer, 2013, pp. 387–402. [10] A. Rashid and J. Such, “Malprotect: Stateful defense against adversarial query attacks in ml-based malware detection,” IEEE Transactions on Information Forensics and Security (TIFS), vol. 18, pp. 4361–4376, 2023. [11] D. Hendrycks, N. Carlini, J. Schulman, and J. Steinhardt, “Unsolved problems in ml safety,” arXiv preprint arXiv:2109.13916, 2021. [12] H. Li, S. Shan, E. Wenger, J. Zhang, H. Zheng, and B. Y. Zhao, “Blacklight: Scalable defense for neural networks against Query-Based Black-Box attacks,” in 31st USENIX Security Symposium (USENIX Security 22). Boston, MA: USENIX Association, Aug. 2022. [Online]. Available: https: //www.usenix.org/conference/usenixsecurity22/presentation/li-huiying [13] F. Pukelsheim, “The three sigma rule,” The American Statistician, vol. 48, no. 2, pp. 88–91, 1994. [Online]. Available: http: //www.jstor.org/stable/2684253 [14] J. Schmidhuber, “Deep learning in neural networks: An overview,” Neural networks, vol. 61, pp. 85–117, 2015. [15] G. Pang, C. Shen, L. Cao, and A. V. D. Hengel, “Deep learning for anomaly detection: A review,” ACM Comput. Surv., vol. 54, no. 2, mar 2021. [Online]. Available: https://doi.org/10.1145/3439950 Extended Abstract of MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection 547