A Review of VAASI: Crafting Valid and Abnormal Adversarial Samples for Anomaly Detection Systems in Industrial Scenarios [Póster]

A Re iew o VAASI: C a ing Valid and Abno mal Ad e sa ial Samples o Anomaly De ec ion Sys ems in Indus ial Scena ios ´ Angel Luis Pe ales G´ omez*1, Lo enzo Fe n´ andez Maim´ o1, Albe o Hue as Celd ´ an2 and F´ elix J. Ga c´ ıa Clemen e1 1Facul y o Compu e Science, Uni e si y o Mu cia, 30100 Mu cia, Spain [email p o ec ed]; [email p o ec ed]; [email p o ec ed] 2Communica ion Sys ems G oup CSG, Depa men o In o ma ics I I, Uni e si y o Zu ich UZH, CH-8050, Swi ze land [email p o ec ed] Resumen—Exis ing ad e sa ial a acks a e no easible in indus ial scena ios since hey p ima ly deals wi h con inuous ea u es and no wi h ca ego ical ea u es. To enhance cybe - secu i y in indus ial se ings, his pape in oduces an inno- a i e ad e sa ial a ack app oach ailo ed speci ically o hese en i onmen s. This no el echnique allows o he c ea ion o a ge ed ad e sa ial samples alid wi hin supe ised cybe a ack de ec ion models in indus ial scena ios, main aining consis ency o disc e e alues and co ec ing cases whe e ad e sa ial samples appea no mal. Valida ion in ol ed assessing mean e o and o al ad e sa ial samples gene a ed, compa ing agains he P ojec ed G adien Descen me hod and Ca lini & Wagne a ack ac oss a ious pa ame e con igu a ions. Ou p oposal achie ed he bes balance be ween mean e o and gene a ed ad e sa ial samples, demons a ing i s supe io i y. Index Te ms—ad e sa ial a acks, anomaly de ec ion, deep lea ning, explainable a i icial in elligence, indus ial sys ems Tipo de con ibuci´ on: In es igaci´ on ya publicada (l´ ımi e 2 p´ aginas) I. INTRODUCTION Cu en ly, due o he inc ease o au oma ion in indus ial scena ios, he Anomaly De ec ion (AD) pa adigm is being explo ed o sa egua d de ices and echnologies agains indus- ial cybe a acks. In pa icula , AD sys ems implemen ed by means o Machine Lea ning (ML) and Deep Lea ning (DL) echniques ha e p o en e ec i e in his con ex . Howe e , his echniques a e ulne able o ad e sa ial a acks, which c ea es samples ha a e missclassi ied by he AD. In his con ex , we emphasize wo signi ican limi a ions o ad e sa ial a acks. Fi s ly, hese a acks a e ine ec i e wi h ca ego ical da a commonly gene a ed by indus ial de ices. Secondly, he exis ing echniques in oduce la ge e o s since hey modi y he whole se o ea u es. Mo e e o s a e equi ed o enhance he obus ness o AD and de elop sui able ad e sa ial a acks in indus ial scena ios. This pape e iews [1] and in oduces VAASI as a solu ion o he p e ious limi a ions. VAASI is a a ge ed ad e sa ial a ack speci ically designed o indus ial sys ems, gene a- ing alid abno mal samples wi h minimal e o in such en i onmen s. Valida ion was pe o med using he WADI da ase con aining bo h ca ego ical and con inuous ea u es, compa ing esul s wi h he P ojec ed G adien Descen (PGD) and Ca lini & Wagne (CW) ad e sa ial a acks. Ou a ack demons a ed he op imal balance be ween he numbe o gene a ed ad e sa ial samples and he c a ing e o , esul ing in samples challenging o expe s o de ec . II. IMPLEMENTATION OF VAASI ATTACK In his sec ion, we summa ized he s eps o implemen he VAASI a ack. In pa icula , he implemen a ion is di ided in o i e s eps: 1) selec ing ea u es o modi ica ion, 2) gene a ing con inuous ea u es, 3) gene a ing ca ego ical ea u es, 4) ensu ing he alidi y o samples, and 5) alida ion. Selec ing Fea u es o Modi ica ion. We sugges modi ying pa icula ea u es o in oduce he minimal necessa y e o and hus c ea e ad e sa ial samples closely esembling he o iginal ones. To iden i y he app op ia e ea u es, we u ilize SHapley Addi i e exPlana ions (SHAP) o assess ea u e impo ance. A e ob aining he impo ances o each ea u e, hey a e a anged in descending o de . This enables us o selec he op alues based on a pe cen ile s a is ic de e mined h ough expe imen a ion. Gene a ing Con inuous Fea u es. In his s ep, we use PGD o gene a e con inuous ea u es. This a ack calcula es in each i e a ion he g adien o he loss unc ion wi h espec o he inpu and modula es he g adien sign by a pe u ba ion pa ame e , o inally sub ac i om he o iginal sample. Gene a ing Ca ego ical Fea u es. This s udy in oduces a no el app oach o gene a ing ca ego ical ea u es by epla- cing selec ed ca ego ical ea u e alues in each sample wi h alues om he mos simila sample in he aining da ase . Ensu ing he Validi y o he Samples. In his wo k, we p opose ex ac ing ules om Decision T ees (DT) wi hin a Random Fo es (RF) o e i y i gene a ed samples, misclas- si ied as no mal by he AD sys em, s ill exhibi anomalous beha io . I he sample is classi ied as no mal by he RF, he ad e sa ial sample is conside ed o ha e been ans o med in o a no mal sample du ing he a ack. Subsequen ly, all pa hs o igina ing om an abno mal lea node o all DTs a e ex ac ed, and he necessa y co ec ions a e calcula ed o sa is y he condi ions o each pa h leading o an anomalous lea node in each DT. The pa h wi h he lowes e o is chosen o modi y he sample acco dingly. Valida ion. Mainly, he alida ion ocuses on wo key aspec s o ad e sa ial a acks. Fi s , i quan i ies he num- be o ad e sa ial samples ha he a ack has managed o gene a e, which helps us unde s and how easily ad e sa ial JNIC 2024 ISBN:978-84-09-62140-8 452 Tabla I PERFORMANCE COMPARISON BETWEEN STATE-OF-THE-ART ADVERSARIAL ATTACKS (PGD AND CW) AND OUR PROPOSAL PGD(0.1) PGD(0.3) PGD(0.5) PGD(0.7) PGD(1.0) CW Ou s L1Samples gene a ed 2.8 % 2.8 % 2.8 % 2.9 % 3.1 % - 5.4 % A e age e o 0.005 0.016 0.027 0.038 0.054 - 0.006 L2Samples gene a ed 3.35 % 9.25 % 22 % 33.4 % 49.5 % 86.25 % 62.25 % A e age e o 0.087 0.248 0.389 0.519 0.721 0.823 0.028 L∞Samples gene a ed 66.80 % 86.10 % 87.30 % 89.75 % 92 % 23.34 % 85.80 % A e age e o 1.844 5.435 9.016 12.540 17.574 0.027 0.205 samples a e gene a ed. Second, i e alua es he mean e o o hese samples in ela ion o he o iginal samples om which hey we e gene a ed. This e o de e mines which ad e sa ial a acks gene a e samples ha closely esemble he o iginals, making hem mo e challenging o expe s o iden i y. III. EXPERIMENTS This sec ion de ails he s eps o deploy ou a ack in a eal indus ial scena io o wa e dis ibu ion. In pa icula , we launched VAASI agains he samples con ained in he WADI da ase . To deploy he expe imen , a se ies o equisi es we e equi ed. In pa icula , a sligh p ep ocessing o he WADI da ase was ca ied ou , as well as he aining o he supe ised model ha o e s su icien ly high pe o mance o be deployed in eal en i onmen s. Selec ing Fea u es o Modi ica ion. In his s ep, we ex- ac ed he impo ance o each ea u e in es samples using he SHAP lib a y wi h a backg ound da ase composed o 100 no mal and 100 abno mal samples. Nex , we selec ed om he es da ase he ea u es whose SHAP alues we e he highes o he abno mal class, i.e., hose ea u es whose SHAP alues we e abo e he 90 h pe cen ile, and g ouped hem in o con inuous and ca ego ical ea u es. Gene a ing Con inuous Fea u es. In his s ep we launched he PGD a ack p o ided by he Ad e sa ial Robus ness Toolbox (ART). Fi s , we selec ed all a ack samples in he es da ase . A e launching he a ack, he majo i y o he samples (83.62 %) we e classi ied as no mal by he AD sys em, and he emaining samples (16.38%) we e s ill classi ied as anomalous. Gene a ing Ca ego ical Fea u es. We employed he nea es - neighbo algo i hm o selec simila samples. In pa icu- la , o acili a e he nea es -neighbo s a egy, we used he KBinsDisc e ize class o he sciki -lea n lib a y. This class disc e ized he aining da ase and he p e iously gene a ed ad e sa ial samples using 10 bins and a uni o m disc e iza ion s a egy. Nex , we selec ed he mos simila sample in he aining da ase and copied hei ca ego ical ea u es in o he co esponding ea u es o he ad e sa ial sample. Ensu ing he Validi y o he Samples. In his s ep, we ained an RF using he aining da ase o de e mine i he ad e sa ial a acks e ain i s abno mal beha io . The RF was ained using he sciki -lea n lib a y wi h he numbe o es ima o s se o 10 and he maximum dep h o he ees se o 10. The ained RF achie ed a 0.969 o F1-sco e. Subsequen ly, o all he samples classi ied as no mal, we u ilized he decision pa hs o he DTs ained wi hin he RF o iden i y he equi ed changes in each sample o e e hem o hei anomalous beha io . Valida ion. In his inal s ep, we compa ed ou esul s wi h hose ob ained using he aw PGD and CW me hods. We con- side ed he numbe o gene a ed ad e sa ial samples and hei a e age e o . Fo PGD, we es ed di e en con igu a ions o he maximum pe missible dis u bance, εand we se he i e a ions and dis u bance allowed in each i e a ion, εs ep, o 50 and 0,05, espec i ely. Rega ding CW, we se he i e a ions and he con idence le el o 10 and 0, espec i ely. Finally, we e alua ed ou p oposal and p e ious me hods using se e al l1, l2, and l∞no ms. The esul s a e shown in Tabla I, indica ing ha ou p oposal achie ed he bes ade-o be ween he numbe o ad e sa ial samples gene a ed and he esul ing e o . To be speci ic, o l1ou solu ion gene a ed 5.4 % ad e sa ial samples wi h an e o o 0.006. Rega ding l1, ou solu ion gene a ed 62.25 % ad e sa ial samples wi h an e o o 0.028. Finally, o lin , ou solu ion gene a ed 85.80% ad e sa ial samples wi h an e o o 0.205. The di e ence in he a e age e o be ween ou solu ion and he exis ing me hod is explained by he ac ha we only modi ied he mos impo an ea u es. IV. CONCLUSIONS In his s udy, we in oduce VAASI, a no el a ge ed ad e - sa ial a ack designed o indus ial se ings. The no el con- ibu ions o VAASI is ha he gene a ed ad e sa ial samples a e alid and e ain hei abno mal beha io . Valida ion using he WADI da ase om a wa e dis ibu ion indus ial plan compa es VAASI wi h PGD and CW me hods, highligh ing i s supe io balance be ween he numbe o gene a ed ad e sa ial samples and he incu ed e o . Speci ically, unde di e en no ms, VAASI p oduced smalle e o s compa ed o PGD and CW, demons a ing i s e ec i eness in gene a ing alid ad e sa ial samples o indus ial sys ems. ACKNOWLEDGEMENTS This wo k has been unded unde G an TED2021- 129300B-I00, by MCIN/AEI/10.13039/501100011033, Nex - Gene a ionEU/PRTR, UE, G an PID2021-122466OB-I00, by MCIN/AEI/10.13039/501100011033/FEDER, UE, by he s a egic p ojec CDL-TALENTUM/DEFENDER om he Spanish Na ional Ins i u e o Cybe secu i y (INCIBE), by he Reco e y, T ans o ma ion and Resilience Plan, Nex Gene a- ion EU, by he Swiss Fede al O ice o De ense P ocu emen (a masuisse) wi h he Cybe Fo ce (CYD-C-2020003), and by he Uni e si y o Zu ich (UZH). REFERENCIAS [1] Pe ales G´ omez, ´ A. L., Fe n´ andez Maim´ o, L., Hue as Celd ´ an, A., & Ga c´ ıa Clemen e, F. J. (2023): ”VAASI: C a ing alid and abno mal ad e sa ial samples o anomaly de ec ion sys ems in indus ial scena ios”, en Jou nal o In o ma ion Secu i y and Applica ions, ol. 79, 103647, 2023. A Re iew o VAASI: C a ing Valid and Abno mal Ad e sa ial Samples o Anomaly De ec ion Sys ems in Indus ial Scena ios 453