Academic Edi o : Pablo
Rod íguez-Gonzál ez
Recei ed: 25 Ma ch 2025
Re ised: 28 Ap il 2025
Accep ed: 29 Ap il 2025
Published: 6 May 2025
Ci a ion: De Cubbe , G.; Do o ei, D.;
Pe sio i, P.; Konia is, A.; B ewczy´nski,
K.; ˙
Zyczkowski, M.; Roman, R.; Sima,
S.; Mohamoud, A.; an de Pol, J.; e al.
S anda dized E alua ion o
Coun e -D one Sys ems: Me hods,
Technologies, and Pe o mance
Me ics. D ones 2025,9, 354.
h ps://doi.o g/10.3390/
d ones9050354
Copy igh : © 2025 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license
(h ps://c ea i ecommons.o g/
licenses/by/4.0/).
A icle
S anda dized E alua ion o Coun e -D one Sys ems: Me hods,
Technologies, and Pe o mance Me ics
Gee De Cubbe 1,* , Daniela Do o ei 1, Pa aske i Pe sio i 2, Alexios Konia is 2, Kon ad B ewczy´nski 3,
Ma ek ˙
Zyczkowski 3, Raz an Roman 4, Sil iu Sima 4, Ali Mohamoud 5, Johan an de Pol 5, I an Maza 6,
Anibal Olle o 6, Ch is ophe Chu ch 7and C is ina Popa 8
1
Robo ics and Au onomous Sys ems Uni , Depa men o Mechanics, Royal Mili a y Academy, 1000 B ussels,
Belgium; [email p o ec ed]
2Cen e o Secu i y S udies (KEMEA), GR-101 77 A hens, G eece; p.pe sio i@kemea- esea ch.g (P.P.);
a.konia is@kemea- esea ch.g (A.K.)
3Ins i u e o Op oelec onics, Mili a y Uni e si y o Technology, 00-908 Wa saw, Poland;
[email p o ec ed] (K.B.); ma [email p o ec ed] (M. ˙
Z.)
4Se iciul de P o ec ie si Paza, 060117 Bucha es , Romania; [email p o ec ed] (R.R.);
[email p o ec ed] (S.S.)
5Nede landse O ganisa ie Voo Toegepas Na uu we enschappelijk Onde zoek (TNO), 2595 DA Den Haag,
The Ne he lands; [email p o ec ed] (A.M.); [email p o ec ed] (J. .d.P.)
6GRVC Robo ics Lab, Uni e si y o Se ille, 41092 Se illa, Spain; [email p o ec ed] (I.M.); [email p o ec ed] (A.O.)
7Digi al Fo ensic Lab o he In e na ional C iminal Police O ganiza ion (INTERPOL), Global Complex o
Inno a ion, Singapo e 258510, Singapo e; c.chu [email p o ec ed]
8Asocia ia de S anda diza e din Romania (ASRO), 020085 Bucha es , Romania; [email p o ec ed]o
*Co espondence: [email p o ec ed]; Tel.: +32-244-14008
Abs ac : This pape aims o in oduce a s anda dized es me hodology o d one de ec ion,
acking, and iden i ica ion sys ems. I is he aim ha his s anda dized es me hodol-
ogy o assessing he pe o mance o coun e -d one sys ems will lead o a much be e
unde s anding o he capabili ies o hese solu ions. This is u gen ly needed, as he e is an
inc ease in d one h ea s and he e a e no cohesi e policies o e alua e he pe o mance o
hese sys ems and hence mi iga e and manage he h ea . The p esen ed me hodology has
been de eloped wi hin he amewo k o he p ojec COURAGEOUS unded by Eu opean
Union’s In e nal Secu i y Fund Police. This s anda dized es me hodology is based upon
a se ies o s anda d use -de ined scena ios ep esen ing a wide se o use cases. A his
momen , hese s anda d scena ios a e gea ed owa d ci il secu i y end use s. Howe e ,
he p oposed s anda d me hodology p o ides an open a chi ec u e whe e he s anda d
scena ios can be modula ly ex ended, p o iding s anda d use s he possibili y o easily
add new scena ios. Fo each o hese scena ios, ope a ional needs and unc ional pe o -
mance equi emen s a e p o ided. Using his in o ma ion, an in eg al es me hodology is
p esen ed ha allows o a ai quali a i e and quan i a i e compa ison be ween di e en
coun e -d one sys ems. The s anda d es me hodology concen a es on he quali a i e and
quan i a i e e alua ion o coun e -d one sys ems. This es me hodology was alida ed
du ing h ee use -sc ip ed alida ion ials.
Keywo ds: coun e -d one; s anda diza ion; quan i a i e e alua ion; s anda d es me hods;
coun e -UAS; pe o mance e alua ion
1. In oduc ion
The inc easing p oli e a ion o unmanned ae ial sys ems (UASs), anging om small
consume d ones o la ge ac ical pla o ms, has led o a su ge in bo h legi ima e and
D ones 2025,9, 354 h ps://doi.o g/10.3390/d ones9050354
D ones 2025,9, 354 2 o 27
malicious applica ions. While d ones o e subs an ial bene i s ac oss a ious sec o s,
including ag icul u e, logis ics, secu i y, and eme gency esponse, hei misuse by c iminal
and e o is ac o s p esen s signi ican secu i y conce ns. Repo s o d ones being used
o smuggling con aband, conduc ing unau ho ized su eillance, in e e ing wi h c i ical
in as uc u e, and execu ing a ge ed a acks ha e unde sco ed he u gency o de eloping
e ec i e coun e measu es [
1
–
3
]. Law en o cemen agencies and secu i y o ces ace he
challenge o de ec ing, acking, iden i ying, and mi iga ing hese h ea s in a apidly
e ol ing echnological landscape.
Despi e he a ailabili y o nume ous coun e -d one solu ions on he ma ke [
4
–
7
],
no widely accep ed s anda dized me hods exis o assessing hei pe o mance. This
lack o s anda diza ion complica es he e o s o acquisi ion agencies, law en o cemen
en i ies, and o he s akeholde s asked wi h selec ing and deploying app op ia e coun e -
d one echnologies. Wi hou objec i e, compa able da a, end use s s uggle o ma ch
speci ic ope a ional equi emen s wi h he capabili ies o a ailable coun e -d one sys ems.
As a esul , he selec ion and p ocu emen p ocess emains agmen ed, o en leading o
subop imal solu ions ha may ail o add ess eal-wo ld h ea s e ec i ely.
To b idge his gap, he COURAGEOUS p ojec [
8
] has de eloped a p e-s anda d docu-
men in he o m o a CENELEC Wo kshop Ag eemen (CWA). This documen , designa ed
CWA 18150 [
9
], ou lines a s anda dized es me hodology o assessing he pe o mance o
coun e -d one sys ems unde con olled and ele an ope a ional condi ions. The me hod-
ology de ines a se o e alua ion scena ios designed o eplica e eal-wo ld ope a ional
challenges, allowing o ai quali a i e and quan i a i e compa isons o di e en coun e -
d one solu ions. By adop ing his s anda dized amewo k, secu i y agencies, p ocu emen
bodies, and indus y s akeholde s can be e unde s and sys em capabili ies and limi a ions,
ul ima ely imp o ing he selec ion and deploymen o coun e -d one echnologies.
The pu pose o his pape is o p o ide a s uc u ed o e iew o he CWA 18150 docu-
men , se ing as bo h an abs ac and an in oduc ion o he s anda dized es me hodology
i de ines. This pape de ails he key aspec s o pe o mance assessmen , including isk anal-
ysis, es en i onmen s, ope a ional equi emen s, and e alua ion me ics. I aims o o e
a comp ehensi e unde s anding o he amewo k de eloped unde he COURAGEOUS
p ojec and o acili a e i s b oade adop ion wi hin he secu i y and de ense communi ies.
The ull CWA documen , which con ains de ailed me hodology and implemen a ion guide-
lines, can be accessed a h ps://www.cencenelec.eu/media/CEN-CENELEC/CWAs/
ICT/cwa-18150.pd (accessed on 22 Ap il 2025).
2. Rela ed Wo k
2.1. Coun e -D one
The echnical de elopmen o new coun e -d one solu ions is a common subjec o
esea ch in he coun e -d one a ea. The p ojec s H2020-ALFA [
10
], H2020-ALADDIN [
11
],
and H2020-Sa eSho e [
12
] a e no ewo hy examples, as hey all d one de ec ion sys ems
c ea ed o pa icula use cases. The c ea ion o an e ec i e d one de ec ion sys em is gene -
ally augh wi h wo majo challenges. Fi s , ega dless o he senso echnology employed,
d one pla o ms ypically ha e a e y small c oss-sec ion and de ec ion baseline [
13
]. In ac ,
d ones employ common adio ansmission equencies, ha e a small sound signa u e
( om a ele an dis ance), a small isual/in a ed signa u e, a small RADAR c oss sec ion,
e c. Fu he mo e, i is challenging o elimina e alse posi i es because he signa u es o
many d ones and bi ds a e ai ly simila [14].
RADAR [
15
], acous ics [
16
], isual [
17
], in a ed [
18
] ( he mal and sho -wa e), adio
spec um sensing [
19
], lida [
20
], and o he sensing modali ies can be employed o ackle he
d one de ec ion p oblem. Ne e heless, due o he di icul y o sol ing he p oblem unde
D ones 2025,9, 354 3 o 27
eal-wo ld ope a ing condi ions, he majo i y o cu en solu ions ely on a combina ion
o a ious sensing echniques o sol e he d one de ec ion p oblem [
13
] and combine
con en ional de ec ion and acking echniques [
21
] ha come om compu e ision o
accomplish mul i-senso acking.
The inco po a ion o coun e -d one solu ions and p ac ices in o s anda d ope a -
ing p ocedu es o law en o cemen agencies is also a c ucial ac o o ake in o accoun .
The p ojec s SkyFall [
22
] and D oneWise [
23
] a e pe inen in his con ex ; hey examine
a ious coun e -d one sys ems, in eg a e he bes a ailable sys ems in a law en o cemen o -
ganiza ion aining, and p o ide a numbe o use ul end-use - ocused measu es o enhance
he esponse o UAS e o is a acks.
2.2. S anda d Tes ing o Coun e -D one Solu ions
In e ms o s anda d es me hodology de elopmen o coun e -d one solu ions, he e
ha e been mul iple de elopmen s in he pas . To s a e he main p oblem: wo opposing
needs can be ound ega ding he c ea ion o s anda dized es echniques o assessing
d one de ec ion sys ems’ pe o mance [
13
]. In o de o iden i y he limi a ions o he sys em
being es ed, i is necessa y o ca e ully egula e he es se ings because d one de ec ion
sys ems ypically ely on in ica e da a usion and he p ocessing o senso da a. This i s
need calls o epea able es s in con olled en i onmen s. Howe e , since d one de ec ion
sys ems mus unc ion a ound he clock and in all wea he si ua ions, i is necessa y o
e alua e how well hey ope a e in a a ie y o en i onmen al se ings. This second need
calls o he e alua ion o he sys ems in a non-con olled ope a ional en i onmen . Since
hese cons ain s con lic wi h one ano he , a s anda dized es me hodology mus ca e ully
balance hese wo ca ego ies o needs. The e o e, he goal is o iden i y a alida ion
app oach ha mee s he needs o he pla o m de elope s o a quan i a i e s a is ically
meaning ul alida ion as well as he end use s’ equi emen o a quali a i e ope a ional
alida ion o he sys em [13].
In he ealm o obo ics, o example, he Na ional Ins i u e o S anda ds and Tech-
nology (NIST) in he Uni ed S a es has p e iously ecommended such quali a i e and
quan i a i e alida ion app oaches [
24
]. Based on he wo k pe o med a NIST, a i s qual-
i a i e and quan i a i e alida ion me hodology was p esen ed in [
25
] and alida ed in [
26
].
This me hodology was applied o he i s ime in he amewo k o s anda dized coun e -
d one es ing in he yea s 2017 and 2018 [
27
] in he amewo k o he Sa eSho e p ojec [
12
].
In ha pe iod, he Na ional Nuclea Secu i y Adminis a ion o he Uni ed S a es Depa -
men o Ene gy also conduc ed esea ch in he Uni ed S a es o build a coun e -d one
es ing and e alua ion me hodology [
28
]. Al hough i ne e ac ually became a s anda d,
he es app oach ou lined de ines es echniques, pe o mance measu emen s, UAS ypes
es ed, c i ical a iables, and he equi ed da a analysis o eliably assess he capabili ies o
coun e -d one echnology.
The absence o a s anda d es me hod s a ed becoming p oblema ic om a ound he
yea 2020, when mul iple coun e -d one solu ions became a ailable on he ma ke , and he
pe o mance capabili ies o hese sys ems became ha de and ha de o di e en ia e o
end use s. Law en o cemen agencies and o he go e nmen ac o s conduc ed a ious
coun e -d one equipmen es s in c i ical in as uc u e p o ec ion scena ios [
29
], bu he
p oblem is ha hese ials a e highly cos ly o all ac o s in ol ed, and esul s a e no easily
ans e able o o he en i onmen s.
This challenge has led o mul iple s anda diza ion e o s ac oss in e na ional o gani-
za ions. EUROCAE WG-115 [
30
] has been ac i ely de eloping s anda ds such as ED-286,
which p o ides an Ope a ional Se ices and En i onmen De ini ion (OSED) o coun e -
d one ope a ions in con olled ai space, and ED-322, which se s Sys em Pe o mance and
D ones 2025,9, 354 4 o 27
In e ope abili y Requi emen s o Non-Coope a i e UAS De ec ion Sys ems. These e o s
ocus speci ically on ai po en i onmen s and happen in close coo dina ion be ween Eu-
ope (EUROCAE) and he Uni ed S a es, whe e he Fede al A ia ion Adminis a ion (FAA)
has ini ia ed Special Commi ee SC-238 o es ablish comp ehensi e e alua ion guidelines
o coun e -d one echnologies. By wo king oge he , EUROCAE WG-115 and FAA SC-
238 aim o ensu e he sa e in eg a ion o coun e -d one sys ems wi hin exis ing a ia ion
s uc u es while main aining e ec i e de ec ion and mi iga ion capabili ies.
NATO is ano he key playe in es ablishing coun e -d one es p ocedu es. (i) Es-
ablishing a coun e -d one communi y; (ii) policy, concep s, doc ine, ac ics, echniques,
and p ocedu es; (iii) s anda diza ion; and (i ) esea ch, de elopmen , and ope a ion ac i i-
ies a e among he opics being wo ked on by he NATO coun e -d one Wo king G oup.
The annual TIE exe cises, which a e designed o p omo e in e ope abili y among a ious
coun e -d one asse s, a e a signi ican example [31].
In Ge many, DIN is wo king on DIN 5452-9 [
32
], which ocuses on d one de ec ion,
as well as an a achmen ha de ines con o mance es p ocedu es. These e o s aim
o c ea e a obus es ing amewo k ha allows o sys ema ic e alua ion o coun e -
d one e ec i eness. Simila ly, he UK’s Na ional P o ec i e Secu i y Au ho i y (NPSA) is
de eloping a “coun e -d one Tes ing and E alua ion S anda d”, which aims o p o ide a
s uc u ed me hodology o pe o mance assessmen .
The In e na ional S anda diza ion O ganiza ion ISO is also con ibu ing o he s an-
da diza ion landscape wi h ISO/CD 16746 [
33
], which p o ides guidance o end-use s on
he deploymen o coun e -d one equipmen , and ISO/CD 16747 [
34
], which suppo s man-
u ac u e s in p oducing e ec i e and complian coun e -d one solu ions. These s anda ds
seek o ha monize bes p ac ices and echnical speci ica ions, ensu ing consis ency and
in e ope abili y ac oss di e en ju isdic ions.
These ini ia i es demons a e he global ecogni ion o he need o s anda dized es
me hodologies. Howe e , dispa i ies in na ional egula ions and di e ing ope a ional
equi emen s p esen ongoing challenges. The CWA 18150 de eloped unde he COURA-
GEOUS p ojec aims o maximize he usage o wo k being ca ied ou in many coun ies
and complemen s hese e o s by p o iding a s uc u ed app oach o pe o mance e al-
ua ion while ensu ing compa ibili y wi h eme ging in e na ional s anda ds. By aligning
wi h hese ongoing s anda diza ion ini ia i es, CWA 18150 seeks o p o ide acquisi ion
agencies and secu i y s akeholde s wi h a eliable amewo k o assessing and compa ing
coun e -d one echnologies in a anspa en and epea able manne .
2.3. O ganiza ion o This Pape
The emainde o his pape is o ganized as ollows. In o de o p o ide a clea
g asp o he p oblem, Sec ion 3p o ides an analysis o inciden s ela ed o UAS and an
analysis o iden i ied gaps in coun e -d one capabili ies. To u he analyze he cu en
playing ield, Sec ion 4in es iga es he cu en echnologies and me hods used o coun e -
d one ope a ions. Based on he g oundwo k in he p e ious sec ions, Sec ion 5p oposes
10 s anda d coun e -d one scena ios. These s anda d scena ios will se e as a guideline
h oughou he whole s anda diza ion p ocess. Fo each o he s anda d scena ios, a isk
analysis is pe o med, as explained in Sec ion 6. In a pa allel line o wo k and h ough
discussions wi h end-use s o coun e -d one solu ions, a se o ope a ional needs o coun e -
d one co e age we e de ised in Sec ion 7, oge he wi h a se ies o coun e -d one sys em
pe o mance equi emen s and me ics. Based on all his inpu , a coun e -d one Sys em
E alua ion Me hod was de eloped in Sec ion 8. This me hodology was alida ed du ing
h ee la ge-scale ials in G eece, Belgium and Spain, as discussed in Sec ion 9. Sec ion 10
D ones 2025,9, 354 5 o 27
concludes his pape by in es iga ing and discussing he esul s ob ained and by iden i ying
emaining gaps and di ec ions o u u e esea ch.
3. Inciden s Analysis and Iden i ica ion o Gaps
3.1. Inciden and Th ea Analysis
The inc easing p oli e a ion o d ones has led o a ise in secu i y conce ns glob-
ally. Unau ho ized d one ac i i ies ha e been epo ed in a ious en i onmen s, anging
om ai po s o go e nmen acili ies, o en leading o ope a ional dis up ions o secu-
i y b eaches. In o de o ob ain a good o e iew o he p oblem, a sys ema ic e iew
o mo e han 200 d one- ela ed inciden s in he Eu opean e i o y was pe o med, an-
alyzing ecen d one inciden s, ca ego izing h ea ac o s, and highligh ing he gaps in
coun e -d one measu es:
•
Ta ge ed Loca ions: Ai po s, go e nmen buildings, mili a y ins alla ions, and c i ical
in as uc u e emain p ime a ge s o unau ho ized d one ac i i ies.
•
Inciden Types: The mos common inciden s include unau ho ized su eillance, smug-
gling, nea -miss collisions wi h manned ai c a , and weaponized d one deploymen s.
•
Ope a ional En i onmen s: Dense u ban a eas p esen unique challenges due o
adio equency (RF) conges ion and limi ed line-o -sigh de ec ion capabili ies.
•
Technological Capabili ies: Ad ances in comme cial d one echnology, including
au onomous na iga ion and payload adap abili y, ha e ou paced cu en coun e -
d one coun e measu es.
The analysis o d one- ela ed inciden s e eals ha he majo i y o cases in ol e
uninhibi ed ope a o s, ollowed by c iminal ac i i ies. Figu e 1ca ego izes hese h ea
ac o s in o six main g oups:
•
Uninhibi ed (55%): Ope a o s who equen ly iola e egula ions bu do no necessa -
ily ha e malicious in en .
•C iminal (25%): D ones used o illici ac i i ies such as smuggling con aband.
•Te o is (10%): D ones employed in a acks o econnaissance by hos ile en i ies.
•
Ca eless (7%): Ope a o s who occasionally b each ai space es ic ions unin en ionally.
•
Clueless (2%): Ope a o s unawa e o egula ions, o en leading o ai space iola ions.
•Complian (0.3%): Law-abiding d one use s who adhe e o egula ions.
The da a indica e ha while egula o y iola ions a e common, a signi ican numbe
o inciden s in ol e he in en ional misuse o d ones o c iminal o e o is pu poses.
D one- ela ed inciden s ha e been obse ed ac oss a ious sec o s. Figu e 2p o ides
an o e iew o he dis ibu ion o inciden s ac oss di e en en i onmen s:
•
P i a e/Non-Co po a e (24%): D ones lown o e esiden ial a eas, o en leading o
p i acy iola ions.
•Ai po s (24%): Inciden s causing ligh delays and sa e y conce ns.
•Go e nmen /Mili a y (16%): Unau ho ized su eillance o po en ial a acks.
•Law En o cemen /Fi s Responde s (13%): In e e ence wi h eme gency ope a ions.
•P isons (10%): D ones used o smuggling con aband.
•S adiums (6%): Uncon olled d one ope a ions du ing e en s.
The high occu ence o inciden s in ai po s and p i a e p ope ies sugges s an u gen
need o obus coun e -d one s a egies.
D ones 2025,9, 354 6 o 27
Figu e 1. Ca ego iza ion o h ea ac o s in d one inciden s.
Figu e 2. Dis ibu ion o d one inciden s ac oss di e en en i onmen s.
3.2. Gap Analysis o Coun e -D one Measu es
Despi e ad ancemen s in d one de ec ion and mi iga ion, se e al gaps emain unadd essed.
Legisla ion go e ning coun e -d one ope a ions a ies signi ican ly ac oss ju isdic-
ions, c ea ing ambigui y ega ding pe missible coun e measu es. Addi ionally, ee-access
D ones 2025,9, 354 7 o 27
a eas, such as public spaces and p i a e p ope ies, pose en o cemen di icul ies due o
p i acy and egula o y limi a ions. Fu he mo e, he e is no uni ied esponse amewo k
o handling d one incu sions.
Despi e he a ailabili y o a ious DTI echnologies, only 5% o he epo ed inciden s
indica e he p esence o an ope a ional de ec ion sys em. Fu he mo e, hese sys ems exhibi
educed e icacy in RF-dense en i onmen s, ad e se wea he condi ions, and scena ios
in ol ing mul iple ai bo ne objec s.
While o - he-shel d ones accoun o he majo i y o inciden s, he inc easing p e a-
lence o modi ied and cus om-buil UAVs p esen s a g owing conce n. Such d ones o en
u ilize non-s anda d communica ion p o ocols, ende ing con en ional RF-based de ec ion
me hods less e ec i e.
A signi ican p opo ion o inciden s in ol e d ones ca ying unde ined payloads,
aising unce ain ies ega ding hei in en and h ea le el. Dis inguishing be ween benign
and hos ile d ones emains a majo challenge, especially in a eas whe e mul iple UAS
ope a e simul aneously.
A uni ied inciden epo ing mechanism is lacking ac oss membe s a es, hinde ing
da a-d i en policy o mula ion and esponse coo dina ion. The absence o s uc u ed
in o ma ion-sha ing amewo ks limi s he de elopmen o p oac i e coun e measu es.
4. Re iew o Cu en Coun e -D one F amewo ks
The ield o coun e -d one has seen signi ican ad ancemen s, d i en by he inc easing
use o d ones o bo h comme cial and mili a y applica ions. This sec ion p o ides an
o e iew o he exis ing de ec ion, acking, and iden i ica ion (DTI) echnologies and hei
combina ions in coun e -d one solu ions.
4.1. Technologies o De ec , T ack, and Iden i y (DTI) in Coun e -D one Solu ions
Fo his s udy on coun e -d one echnologies, in o ma ion on 260 an i-d one sys ems
was ini ially collec ed. This ini ial da ase was downscaled o 144 sys ems based on
he le el o de ail and he app op ia eness o he da a ha could be ob ained o he
sys ems. These coun e -d one solu ions u ilize mul iple echnologies o de ec ing, acking,
and iden i ying d ones. Figu e 3illus a es he dis ibu ion o hese echnologies. The mos
widely used me hods include he ollowing [35].
•
Mic owa e ada s: Employed in 55% o he sys ems, ada s de ec d ones h ough
ac i e adio wa e emissions and e lec ions. Among hei p incipal ad an ages is
hei capabili y o ope a e independen ly o ligh ing condi ions, enabling con inuous
ope a ion bo h day and nigh . Rada s also main ain unc ionali y unde mode a e
ad e se wea he condi ions, al hough pe o mance may be deg aded in cases o
hea y ain, og, o snow. A signi ican s eng h lies in hei abili y o p o ide no
only de ec ion bu also p ecise measu emen s o ange, adial eloci y, and, in some
con igu a ions, al i ude. Fu he mo e, mic owa e ada s a e capable o wide-a ea
su eillance, especially when in eg a ed wi h mechanically o elec onically scanned
an enna a ays. Howe e , ada s exhibi ce ain limi a ions. The de ec abili y o small
d ones, o en cha ac e ized by minimal ada c oss-sec ions, emains challenging,
pa icula ly when using lowe - equency ada s. A mosphe ic a enua ion, especially
a highe equencies, can signi ican ly educe de ec ion anges in ad e se wea he .
Mo eo e , ada s a e suscep ible o elec omagne ic in e e ence, bo h acciden al and
delibe a e (e.g., jamming), which can comp omise hei eliabili y. Las ly, as ac i e
emi e s, ada sys ems can hemsel es be de ec ed by ad e sa ies, posing ope a ional
secu i y isks.
D ones 2025,9, 354 8 o 27
•
Visible ligh (VIS) came as: Used in 47% sys ems, VIS came as enable op ical de-
ec ion and isual acking, e y o en as a seconda y sensing modali y o suppo
he iden i ica ion o he d one h ea o o p o ide a isual cue o he ope a o s.
Visible ligh (VIS) came as a e widely employed in de ec ion and iden i ica ion asks
due o hei abili y o cap u e de ailed image y ha is easily in e p e able by human
ope a o s and compa ible wi h au oma ed image ecogni ion algo i hms. They o e
he ad an age o high spa ial esolu ion, enabling he iden i ica ion o small a ge s
a conside able dis ances, p o ided ha app op ia e op ics a e used. VIS came as
a e passi e senso s and hus do no e eal hei posi ion, making hem sui able o
co e ope a ions. Ne e heless, hei e ec i eness is highly dependen on ambi-
en illumina ion. In low-ligh condi ions, such as nigh ime o hea y cloud co e ,
hei de ec ion capabili y diminishes ma kedly unless supplemen ed wi h a i icial
illumina ion, which i sel comp omises ope a ional disc e ion. Addi ionally, en i on-
men al ac o s such as haze, og, and p ecipi a ion can signi ican ly impai VIS came a
pe o mance, educing bo h de ec ion p obabili y and iden i ica ion accu acy.
•
The mal imaging came as: In eg a ed in o 35% o sys ems, hese came as de ec
d ones based on in a ed signa u es. The mal imaging came as p o ide a comple-
men a y de ec ion capabili y by exploi ing di e ences in in a ed adia ion emi ed
by objec s o a ying empe a u es. They enable e ec i e su eillance in comple e
da kness and can de ec d ones based on he he mal signa u es o hei mo o s, ba -
e ies, and o he hea -gene a ing componen s. The mal came as a e less sensi i e o
isual obs uc ions like smoke o camou lage, enhancing hei u ili y in deg aded
isual en i onmen s. Howe e , hei pe o mance can be a ec ed by en i onmen al
condi ions such as hea y ain, dense og, o high ambien empe a u es, which educe
he mal con as . Fu he mo e, he mal came as gene ally o e lowe spa ial esolu-
ion compa ed o VIS came as, complica ing he ask o p ecise iden i ica ion. Thei
ield o iew is o en na owe when designed o long- ange de ec ion, necessi a ing
ca e ul in eg a ion in o mul i-senso sys ems o comp ehensi e co e age.
•
F equency moni o ing de ices: P esen in 64% sys ems, hese de ices de ec and
analyze adio equency emissions om d ones and hei con olle s. F equency moni-
o ing de ices cons i u e a passi e de ec ion echnology based on he in e cep ion and
analysis o adio equency (RF) communica ions be ween d ones and hei ope a o s.
Thei key ad an age lies in hei abili y o de ec d ones beyond isual o ada line
o sigh , as RF signals can p opaga e h ough ce ain obs acles. Addi ionally, some
sys ems can exploi RF inge p in s o decode ansmi ed eleme y o in e he ype
and s a us o a d one. F equency moni o ing is inhe en ly s eal hy, posing no isk
o e ealing he p esence o he de ec ion sys em. Ne e heless, he echnique is
limi ed o d ones ac i ely emi ing signals wi hin moni o ed equency bands. D ones
ope a ing au onomously, wi h p e-p og ammed ou es and no ac i e communica ion
links, emain unde ec able. Mo eo e , u ban en i onmen s sa u a ed wi h RF sig-
nals p esen signi ican challenges, including high alse ala m a es and di icul y in
accu a e localiza ion.
•
Acous ic senso s: Used in 10% sys ems, hese senso s cap u e and analyze he sound
signa u es o d one p opelle s. Acous ic senso s o e ano he passi e me hod o
d one de ec ion, elying on he iden i ica ion o cha ac e is ic sound signa u es gen-
e a ed by d one o o s and mo o s. Thei ad an ages include independence om
elec omagne ic spec um cons ain s and he abili y o de ec d ones a ela i ely sho
anges wi hou line o sigh , depending on en i onmen al condi ions. Acous ic a ays
can be compac and apidly deployable, making hem sui able o mobile ope a ions.
Howe e , acous ic de ec ion su e s om se e e limi a ions. En i onmen al noise,
D ones 2025,9, 354 9 o 27
such as wind, ehicula a ic, o indus ial machine y, can mask d one signa u es and
lead o missed de ec ions o alse ala ms. Addi ionally, he e ec i e de ec ion ange o
acous ic sys ems is limi ed, ypically o a ew hund ed me e s unde ideal condi ions,
which may be insu icien o ea ly h ea wa ning in many ope a ional scena ios.
•
In a ed (IR) senso s and lida sys ems: These a e less commonly used, wi h 2%
and 4% o implemen a ions, espec i ely. In a ed (IR) senso s, dis inc om he mal
came as in his con ex , ypically e e o simple , o en single-pixel o low- esolu ion
de ices de ec ing changes in in a ed adia ion ac oss a scene. They a e ad an ageous
o de ec ing wa m objec s agains coole backg ounds and can se e as e ec i e
igge s o cuing highe - esolu ion sys ems. Like he mal came as, hey ope a e
independen ly o isible ligh , p o iding nigh ime capabili y. Howe e , hei lack
o imaging capaci y cons ains hei u ili y p ima ily o de ec ion a he han classi-
ica ion o iden i ica ion asks. Fu he mo e, IR senso s sha e simila ulne abili ies
o en i onmen al ac o s such as og and p ecipi a ion, which a enua e in a ed a-
dia ion and deg ade pe o mance. Finally, lida sys ems, employing lase ligh o
measu e dis ances o objec s, o e high-p ecision h ee-dimensional mapping and
objec de ec ion capabili ies. Lida can p o ide exac spa ial posi ioning o de ec ed
objec s, an ad an age in clu e ed en i onmen s whe e o he senso s may s uggle
wi h alse posi i es. I is especially e ec i e o sho - o medium- ange applica ions
whe e de ailed scene unde s anding is c i ical. None heless, lida sys ems a e ac i e
emi e s and hus can po en ially be de ec ed by ad e sa ies. Thei pe o mance is also
sensi i e o a mosphe ic condi ions, wi h og, ain, and snow, causing signal sca e ing
and a enua ion. Mo eo e , eye-sa e y conside a ions cons ain he powe o lida
emissions, po en ially limi ing ope a ional ange in ce ain con ex s.
The p e alence o hese echnologies indica es a end owa d mul i-senso usion o
enhanced eliabili y and obus ness.
Recen T ends in Da a Fusion Algo i hms o De ec ion and T acking The ield
o senso usion o de ec ion and acking, especially in coun e -d one applica ions, is
apidly e ol ing h ough bo h AI-based me hods and enhancemen s o classical echniques.
The objec i e is o inc ease sys em obus ness, educe alse ala ms, and p o ide con in-
uous ope a ional capabili y unde a ying en i onmen al condi ions. Key algo i hmic
app oaches cu en ly in luencing he domain include he ollowing.
•Mul i-Senso Da a Fusion wi h AI Augmen a ion
Recen C-UAS sys ems inc easingly employ AI-enhanced senso usion, whe e ou pu s
om mul iple he e ogeneous senso s (e.g., ada , EO/IR came as, acous ic a ays, RF
de ec o s) a e me ged using machine lea ning models. Neu al-ne wo k-based usion
algo i hms, pa icula ly Con olu ional Neu al Ne wo ks o image-based inpu s and
Recu en Neu al Ne wo ks o T ans o me s o ime-se ies senso da a, a e used o
co ela e and e ine de ec ion and acking in o ma ion.
These me hods ain models on mul imodal da ase s o ecognize complex h ea
pa e ns, compensa e o senso -speci ic weaknesses, and p o ide p obabilis ic h ea
classi ica ion. AI helps handle missing o inconsis en da a, dynamically weigh ing
senso con ibu ions depending on con ex ual ac o s like wea he o e ain.
•Kalman Fil e and I s Mode n Ex ensions
The Kalman Fil e and i s a ian s, such as he Ex ended Kalman Fil e and he
Unscen ed Kalman Fil e , emain undamen al in acking mo ing objec s. In C-UAS
sys ems, Kalman-based me hods a e used o es ima e d one posi ions and eloci ies
by using inpu s om ada , EO/IR, and RF sys ems.
Recen ends include Adap i e Kalman Fil e s, whe e he p ocess and measu e-
men noise co a iances a e adjus ed online based on senso eliabili y indica o s.
D ones 2025,9, 354 16 o 27
neu alizing hos ile d ones. Addi ionally, en i onmen al adap abili y assesses sys em
pe o mance unde a ying wea he and geog aphical condi ions.
The isk analysis o each scena io (as p esen ed in he p e ious sec ion) is de i ed om
an agg ega ion o expe e alua ions and empi ical da a. Each scena io is assigned a o al
isk sco e based on he weigh ed sum o likelihood and impac assessmen s. The highes
isk scena io iden i ied was he P ison Scena io, as i conce ns a scena io in an u ban
en i onmen , whe e i is impossible o exe a con inuous o al con ol. The lowes isk
scena io was he In e na ional Summi , as o his ype o scena io, mul iple coun e -d one
ools can be employed and a s ic con ol can be en o ced.
The p ojec also in oduces Key Risk Indica o s (KRIs), which se e as benchma k
alues o assessing UAS h ea s. These indica o s include (among o he s):
• En i onmen al complexi y and ope a ional cons ain s.
• The echnical capabili ies o ad e sa ial UAS.
•
The e ec i eness o exis ing coun e -d one measu es agains a ying d one echnologies.
• The equency o d one incu sions in speci ic ope a ional en i onmen s.
• The abili y o coun e -d one ope a o s o espond e ec i ely o iden i ied h ea s.
The esul s o his analysis di ec ly con ibu e o e ining coun e -d one s a egies.
By mapping isk sco es o ope a ional esponse le els, secu i y agencies can op imize
esou ce alloca ion and esponse p o ocols. Fu he mo e, he adop ion o s anda dized
isk assessmen me hodologies os e s in e ope abili y among Eu opean secu i y o ces,
enhancing c oss-bo de collabo a ion in d one h ea mi iga ion.
7. Pe o mance Requi emen s
7.1. Ope a ional Needs and Pe o mance Requi emen s
Ope a ional needs de ine he essen ial capabili ies equi ed o coun e unau ho-
ized UAS incu sions. These equi emen s a e de i ed om s anda dized scena ios
encompassing di e se ope a ional en i onmen s, such as u ban cen e s, ai po s, p is-
ons, and ma i ime bo de s. Key ope a ional pa ame e s include ea ly wa ning sys ems,
eal- ime acking, mul i-senso in eg a ion, and seamless in e ope abili y wi h exis ing
secu i y in as uc u e.
A c i ical challenge in de ining ope a ional needs is he lack o uni e sally accep ed
echnical s anda ds. Consequen ly, ope a ional equi emen s mus be s uc u ed o ensu e
compa ibili y wi h e ol ing h ea s and eme ging echnologies. The abili y o accu a ely
de ec , ack, and iden i y h ea s emains he co ne s one o an e ec i e coun e -d one
s a egy, wi h p io i y conside a ions gi en o minimizing alse ala ms and maximizing
de ec ion ange.
7.2. Me hodology o Ope a ional Needs De ini ion
To sys ema ically cap u e he ope a ional needs o coun e -d one ope a ions, an ap-
p oach wi h s uc u ed equi emen s de ini ions was employed. The me hodology adhe es
o bes p ac ices in sys em enginee ing and equi emen s managemen , ensu ing consis-
ency, aceabili y, and usabili y o subsequen sys em de elopmen phases. The app oach
ollows a sequen ial p ocess:
1.
Ope a ional Needs De ini ion: Iden i ica ion o co e ope a ional equi emen s based
on s akeholde inpu and analysis o eal-wo ld h ea scena ios.
2.
Sys em P ocu emen Conside a ions: T ansla ion o ope a ional needs in o sys em-le el
equi emen s o guide he selec ion and de elopmen o coun e -d one echnologies.
3.
Valida ion and I e a ion: Con inuous e inemen o equi emen s based on es ing,
eedback, and e ol ing ope a ional challenges.
D ones 2025,9, 354 17 o 27
This i e a i e amewo k ensu es ha he ope a ional needs emain aligned wi h
eal-wo ld cons ain s while accommoda ing echnological ad ancemen s and policy
de elopmen s.
To s anda dize he equi emen s o mula ion, he IEEE 29148 p ocess e minology
was adop ed. This s anda d is widely ecognized ac oss Eu opean pa ne s and has
demons a ed e ec i eness in p io secu i y and de ense- ela ed p ojec s. The p io i iza ion
o equi emen s ollows a s uc u ed classi ica ion o acili a e clea decision-making:
•
Shall: High-p io i y equi emen s ha mus be me o he p ojec o achie e i s
co e objec i es.
•
Should: Medium-p io i y equi emen s ha a e desi able bu no essen ial, allowing
o ade-o s i necessa y.
•
May: Low-p io i y equi emen s ha p o ide addi ional alue bu do no impac he
undamen al ope a ional capabili y o he sys em.
This classi ica ion enables end use s o sys ema ically a icula e hei needs while
ensu ing echnical pa ne s can e ec i ely ansla e hese equi emen s in o implemen able
sys em speci ica ions.
To acili a e cla i y and aceabili y, ope a ional needs we e documen ed using a
s anda dized empla e. Each equi emen is uniquely iden i ied and ca ego ized based on
i s signi icance and in ended applica ion. The s uc u ed o ma includes
•Requi emen Numbe : A unique iden i ie o acking and e e ence.
•Requi emen Name: A concise desc ip o summa izing he need.
•
Desc ip ion: A de ailed explana ion, including he a ionale and ope a ional con ex .
•Impo ance: The assigned p io i iza ion le el (Shall, Should, May).
This app oach ensu es ha all s akeholde s— anging om sys em ope a o s o echnology
de elope s—ha e a common unde s anding o he ope a ional expec a ions and cons ain s.
The s uc u ed equi emen s amewo k encompasses a comp ehensi e ange o
ope a ional needs, pa icula ly ocusing on de ec ion, acking, and iden i ica ion (DTI)
capabili ies. Gi en he di e si y o deploymen en i onmen s, equi emen s a e ailo ed o
add ess speci ic scena ios, such as:
•
Gene al Ope a ional Needs: Baseline capabili ies applicable ac oss mul iple deploy-
men con ex s.
•Scena io-Speci ic Needs: Cus om equi emen s adap ed o dis inc secu i y en i on-
men s (e.g., ai po s, go e nmen buildings, public e en s).
•
In eg a ion Needs: In e ope abili y wi h exis ing secu i y in as uc u e and command-
and-con ol sys ems.
By linking ope a ional needs wi h speci ic deploymen con ex s, he app oach ensu es
ha he de ined equi emen s emain p ac ical and ac ionable.
7.3. Pe o mance Requi emen s and Me ics
Pe o mance equi emen s ansla e ope a ional needs in o quan i iable me ics. They
a e in luenced by mul iple ope a ional ac o s, including he size and speed o a ge d ones,
he complexi y o he ope a ional en i onmen , and he esponse ime needed o neu alize
h ea s. De ec ion capabili ies mus be comp ehensi e, co e ing a wide ange o al i udes
and dis ances while main aining high accu acy a es. The sys em should e ec i ely dis-
c imina e be ween d ones and o he ai bo ne objec s o minimize alse ala ms. T acking
pe o mance mus ensu e he con inuous and s able moni o ing o d one mo emen s, e en
in scena ios whe e mul iple UAS a e ope a ing simul aneously.
Iden i ica ion accu acy is ano he c i ical equi emen , as dis inguishing be ween
di e en d one models and payload ypes is essen ial o assessing h ea le els. This
D ones 2025,9, 354 18 o 27
in ol es in eg a ing mul ispec al imaging, adio equency (RF) analysis, and machine-
lea ning-based classi ica ion. Fu he mo e, mi iga ion e ec i eness is a key ac o in
assessing coun e -d one pe o mance. Whe he employing elec onic coun e measu es,
physical in e cep ion, o o he mi iga ion s a egies, he sys em mus neu alize h ea s
eliably while minimizing colla e al isks.
The COURAGEOUS amewo k ocuses on he DTI aspec and ca ego izes all DTI
equi emen s in o he ollowing.
•
De ec ion Pe o mance: The sys em mus iden i y d ones ac oss a ious size classi i-
ca ions and ope a ional al i udes.
•
T acking and Iden i ica ion: The con inuous acking o de ec ed UAS wi h high
posi ional accu acy, dis inguishing be ween coope a i e and non-coope a i e d ones.
•
Sys em In e ope abili y: Compa ibili y wi h o ensic analysis ools and in eg a-
ion wi h exis ing secu i y ne wo ks, including ai a ic managemen (ATM) and
U-space se ices.
•
Robus ness: Resis ance o en i onmen al ac o s such as ad e se wea he condi ions
and elec omagne ic in e e ence.
Pe o mance e alua ion necessi a es he de elopmen o objec i e me ics o assess
sys em e ec i eness ac oss di e en ope a ional con ex s. Typical me ics o assess pe o -
mance equi emen s include de ec ion ange, acking accu acy, esponse ime, and sys em
esilience agains en i onmen al pe u ba ions. De ec ion p obabili y measu es he likeli-
hood o success ully iden i ying a d one wi hin a gi en ange, while he alse ala m a e
assesses he equency o inco ec de ec ions. The sys em’s esponse ime is a c ucial me ic,
as apid h ea engagemen is necessa y o p e en unau ho ized d one ac i i y. T acking
accu acy is e alua ed based on he sys em’s abili y o main ain a d one’s posi ion da a o e
ime, ensu ing con inuous moni o ing and p edic i e analysis o i s ajec o y. Addi ionally,
en i onmen al adap abili y examines how well a sys em pe o ms unde a ying wea he
condi ions, elec omagne ic in e e ence, and u ban conges ion. Sys em esilience agains
coun e measu es, such as d one e asi e maneu e s and signal-jamming esis ance, is also
conside ed a i al pe o mance pa ame e . The in eg a ion o a i icial in elligence (AI) and
ad anced da a usion echniques u he enhances coun e -d one pe o mance, allowing
o imp o ed decision-making and au oma ion in eal- ime h ea scena ios.
7.4. Coun e -D one E alua ion F amewo k
The e alua ion o coun e -d one solu ions equi es a s uc u ed amewo k o assess
hei e ec i eness, eliabili y, and ope a ional sui abili y. This amewo k p o ides a s an-
da dized app oach o documen ing, e iewing, and compa ing es esul s ac oss a ious
coun e -d one solu ions, acili a ing in o med decision-making o he de elopmen , p o-
cu emen , and deploymen o such echnologies. The COURAGEOUS p ojec p oposes a
s uc u ed e alua ion me hodology o assessing coun e -d one e ec i eness.
An e ec i e e alua ion amewo k mus ensu e objec i i y h ough quan i a i e and
quali a i e me ics, main ain epea abili y by es ablishing es condi ions ha allow con-
sis en assessmen s, and be adap able o di e en ope a ional en i onmen s and e ol ing
h ea s. Addi ionally, anspa ency in documen a ion is c ucial o s akeholde s and egula-
o y bodies o ensu e compliance and us in he e alua ion p ocess.
The p oposed amewo k comp ises a s anda dized es bed o pe o mance alida-
ion, inco po a ing eal-wo ld scena ios o benchma k sys em capabili ies. E alua ion
me hodologies include
1.
Scena io-based es ing: Simula ing ealis ic h ea s in con olled en i onmen s o
analyze sys em esponses.
D ones 2025,9, 354 19 o 27
2.
Quan i a i e me ics assessmen : Measu ing sys em pe o mance agains p ede ined
benchma ks, such as de ec ion accu acy and la ency.
3.
Compa a i e analysis: Es ablishing a common baseline o e alua ing di e en
coun e -d one solu ions unde iden ical es ing condi ions.
4.
Con inuous imp o emen cycle: The i e a i e e inemen o equi emen s and me ics
based on empi ical es esul s.
The me hodology o e alua ing coun e -d one pe o mance consis s o mul iple
s ages. Ini ially, clea e alua ion goals mus be de ined, ocusing on aspec s such as de ec-
ion accu acy, esponse ime, and mi iga ion e ec i eness. Following his, es scena ios
should be ca e ully selec ed o ep esen a ange o ope a ional condi ions, including
u ban, u al, and ma i ime en i onmen s, ensu ing ha he sys em’s adap abili y is ho -
oughly es ed.
Once scena ios a e es ablished, pe o mance me ics a e employed o measu e key
capabili ies, such as de ec ion p obabili y, acking accu acy, and neu aliza ion success
a e. Da a collec ion and analysis play a c ucial ole in his p ocess, as in o ma ion is
ga he ed om senso logs, ope a o eedback, and o ensic assessmen s o ensu e a com-
p ehensi e e alua ion. Compa a i e benchma king is hen used o es ablish pe o mance
baselines, enabling an objec i e compa ison be ween di e en coun e -d one sys ems unde
iden ical condi ions.
This e alua ion amewo k p o ides a anspa en mechanism o s akeholde s o
e alua e and compa e coun e -d one echnologies and make in o med p ocu emen de-
cisions. The sys ema ic app oach ou lined in his pape unde sco es he necessi y o
a s uc u ed amewo k o coun e -d one de elopmen and assessmen . By aligning
ope a ional needs wi h pe o mance equi emen s and employing igo ous e alua ion
me hodologies, he p oposed me hodology aims o enhance he e icacy and eliabili y o
coun e -d one benchma king echnologies.
8. Pe o mance E alua ion and Valida ion
8.1. E alua ion Me ics
The p oposed e alua ion me hodology is designed o be objec i e-d i en, meaning
ha i ocuses on p ede ined ope a ional goals and pe o mance c i e ia. I is s uc u ed
a ound scena io-based es ing, in which DTI sys ems a e assessed unde condi ions ha
closely esemble eal-wo ld en i onmen s. These scena ios include sensi i e si es and
c i ical in as uc u e such as ai po s, powe plan s, and mili a y ins alla ions; public
spaces such as s adiums and mass ga he ings; and bo de secu i y en i onmen s whe e
d ones may be used o illici ac i i ies. Each scena io inco po a es en i onmen al a iables
such as wea he condi ions, signal in e e ence, and a ying d one beha io s, ensu ing a
comp ehensi e assessmen o sys em pe o mance.
The e alua ion amewo k employs clea ly de ined pe o mance me ics, as de ined
in Table 1, o enable s anda dized compa isons be ween di e en DTI solu ions. In his
e alua ion o de ec ion, classi ica ion, and acking algo i hms, he pe o mance is com-
monly assessed h ough ou undamen al ou comes: T ue Posi i es (TPs), False Posi i es
(FPs), False Nega i es (FNs), and T ue Nega i es (TNs).
•
T ue Posi i es (TPs) e e o he numbe o ins ances in which he sys em co ec ly
iden i ies a a ge o e en o in e es as p esen . In he con ex o d one de ec ion,
a ue posi i e occu s when a d one is co ec ly de ec ed and classi ied as such. The TPs
a e calcula ed as he du a ion o he g ound u h ajec o ies in which he e was a
leas a single iden i ied ack associa ed wi h he u h.
•
False Posi i es (FPs) deno e ins ances whe e he sys em inco ec ly iden i ies a non-
a ge as a a ge . In de ec ion sys ems, a alse posi i e co esponds o a alse ala m,
D ones 2025,9, 354 20 o 27
whe e an objec (e.g., a bi d, ehicle, o a mosphe ic phenomenon) is mis akenly
classi ied as a d one. The alue o he FP co esponds o he du a ion o he acked
ajec o ies in which he e was no associa ed u h, bu he ack was iden i ied as pos-
i i e.
•
False Nega i es (FNs) ep esen he numbe o cases whe e he sys em ails o de ec
o classi y a ue a ge . A alse nega i e occu s when an ac ual d one is p esen bu
he sys em does no de ec o ecognize i . The alue o he FN co esponds o he
du a ion o he u h ajec o ies in which he e was no an iden i ied ack associa ed
wi h he u h.
•
T ue Nega i es (TNs) a e he ins ances whe e he sys em co ec ly iden i ies he
absence o a a ge . In d one de ec ion, a ue nega i e occu s when no d one is
p esen , and he sys em app op ia ely epo s he absence o a h ea .
These ou ou comes o m he basis o calcula ing key pe o mance me ics such as
he P ecision and he F1 Sco e, which a e c i ical o a comp ehensi e assessmen o sys em
e ec i eness unde ope a ional condi ions.
Table 1. E alua ion me ics de ined in he COURAGEOUS s anda d es me hodology [37].
Me ic Name Desc ip ion Equa ion
Loca ion
accu acy
(2D/3D)
The loca ion accu acy o a de ec ion
ep esen ing a ue objec is de ined as he
dis ance be ween he de ec ion and he ue
objec , wi h dj he de ec ion j,Nd he o al
numbe o de ec ions,
g i
he g ound u h
i
,
and Ng he o al numbe o g ound u hs.
The me ic is unde ined o de ec ions ha
do no ep esen a ue objec .
Aij =(1, i g iand djassocia e
0, o he wise (1)
∆ij =dis ance be ween g iand dj(2)
Loca ion accu acy(g i) =
u
u
u
∑Nd
j=1Aij ·∆2
ij
∑Nd
j=1Aij
(3)
Range a io
The ela i e minimum and maximum
de ec ion ange o a ue objec is de ined as
he minimum and maximum dis ances o
he de ec ions ep esen ing he objec om
he DTI sys em no malized o he
minimum and maximum ange o he ue
objec wi hin he A ea o In e es (AoI)
om he DTI sys em.
∆(x, ) = Dis ance(x, DTI senso )(4)
Rnea (g i) = max (∆(g i, )) −minj(∆(g i, dj)·Aij)
max (∆(g i, )) −min (∆(g i, )) (5)
R a (g i) = maxj(∆(g i, dj)·Aij)−min (∆DTI (g i, ))
max (∆(g i, )) −min (∆(g i, )) (6)
De ec ion
immedia eness
The de ec ion immedia eness is de ined as
he di e ence be ween he ime Ta which
an objec en e s he AoI and he ime o i s
i s de ec ion.
Immedia eness(g i) = T(g s a
i)−
Nd
min
j=1T(dj)(7)
T ack
comple eness
The ack comple eness o a ue objec is
de ined as he ac ion o ime in which he
objec is ep esen ed by a leas one ack. T ack comple eness(g i) = DSN
j=1R(Aij · j)
D(g i)(8)
T ack
con inui y
The ack con inui y o a ue objec is
de ined as he o al numbe o acks
ep esen ing he objec . The me ic is
unde ined i he ue objec has no acks
ep esen ing he objec .
T ack con inui y(g i) = ∑N
j=1(Amin
ij )−1
DSN
j=1R(Aij · j)(9)
D ones 2025,9, 354 21 o 27
Table 1. Con .
Me ic Name Desc ip ion Equa ion
T ack
ambigui y
The ack ambigui y o a ue objec is
de ined as he ime-weigh ed a e age o he
numbe o acks ep esen ing he objec
du ing he ime he objec has a leas one
ack ep esen ing he objec . The me ic is
unde ined i he ue objec has no acks
ep esen ing he objec .
T ack ambigui y( k) = ∑N
j=1Aij( k)
Ng ( k)(10)
T ack ambigui y =∑K
k=1T ack ambigui y( k)·∆ k
∑K
k=1∆ k
(11)
T ack
spu iousness
The ack spu iousness is de ined as he
ime-weigh ed a e age o he numbe o
acks no ep esen ing a ue objec a
ha ime.
S( k) = NT( k)−NA( k)
NT( k)(12)
S=∑K
k=1S( k)·∆ k
∑K
k=1∆ k
(13)
T ack posi ional
accu acy
The ack posi ional accu acy o a ue
objec is de ined as he Roo Mean Squa e
(RMS) dis ance be ween he acks
ep esen ing he objec and he ue objec .
The me ic is unde ined when no ack
ep esen s he ue objec .
∆ij =dis ance be ween g iand j(14)
accij =
u
u
∑Nk,j
k=1Aij(k)·∆ij(k)2
Nk,j
(15)
TPA(g i) =
u
u
u
∑N
j=1D(Aij)·acc2
ij
∑N
j=1D(Aij)(16)
T ack eloci y
accu acy
The ack eloci y accu acy o a ue objec
is de ined as he RMS eloci y di e ence
be ween he acks ep esen ing he objec
and he ue objec . The me ic is unde ined
when no ack ep esen s he ue objec .
∆ij = eloci y di e ence be weeng iand j(17)
TVA (g i) =
u
u
u
∑N
j=1∑Nk,j
k=1Aij(k)·∆ij(k)2
∑N
j=1∑Nk,j
k=1Aij(k)(18)
Longes ack
segmen
The longes ack segmen o a ue objec is
de ined as he la ges ac ion o ime in
which he objec was ep esen ed by he
same ack while being in he AoI.
Longes T ack Segmen (g i) =N
max
j=1
D(Aij)
D(g i)(19)
T acking
immedia eness
The acking immedia eness is de ined as
he di e ence be ween he ime a which an
objec en e s he a ea o in e es and he
ime o i s i s associa ed ack.
TI(g i) = T(g s a
i)−
N
min
j=1T( s a
j)(20)
F1 sco e The F1-sco e is he ha monic mean o
p ecision and ecall.
F1 =2·
P ecision ·Recall
P ecision +Recall (21)
F1 =2TP
2TP +FP +FN (22)
False Ala m
Ra e (FAR)
The alse ala m a e is de ined as he
ac ion o alsely gi en ala ms ou o he
o al numbe o ala ms.
FAR =FP
FP +TN (23)
Missed Ala m
Ra e (MAR)
The missed ala m a e is de ined as he
ac ion o ala ms he DTI did no emi ou
o he o al numbe o ala ms i should
ha e emi ed.
MAR =FN
TP +FN (24)
D ones 2025,9, 354 22 o 27
Table 1. Con .
Me ic Name Desc ip ion Equa ion
P obabili y o
De ec ion (PoD)
The p obabili y o de ec ion is de ined as
he numbe o imes in which he DTI
sys em emi s he ala m igh ully ( ue
posi i es), di ided by he o al numbe o
ala ms i should ha e emi ed
PoD =TP
TP +FN (25)
P ecision
The p ecision is de ined as he ac ion o
all de ec ions ha ep esen a ue objec ,
i.e., associa ed wi h a g ound u h. P ecision =TP
TP +FP =∑Nd
j=1maxNg
i=1Aij
Nd
(26)
8.2. E alua ion Me hodology
To main ain ai ness and objec i i y, he me hodology ollows a compa a i e pe -
o mance e alua ion app oach. All es ed sys ems unde go iden ical scena ios unde
con olled condi ions, elimina ing ex e nal biases ha could in luence he esul s. The e-
o e, he ou pu s om sys em es s a e no malized in o a sco e anging om 0 (wo s )
o 1 (bes ), acili a ing consis en compa isons ac oss di e en DTI sys ems. The me ic
e alua ion p ocess is illus a ed in Figu e 6.
Figu e 6. Pe o mance e alua ion a he me ic le el. The p ocess in ol es execu ing sys em es s,
compu ing aw pe o mance alues, and no malizing hem in o sco es.
The componen le el in eg a es no malized me ic sco es in o unc ional ca ego ies
such as de ec ion, acking, and iden i ica ion. Each componen agg ega es he ele an
me ic sco es and applies use -de ined weigh s based on ope a ional p io i ies. This s ep
p o ides an o e all e alua ion sco e o each unc ional componen , o ming he basis o
sys em-le el assessmen . The p ocess is isualized in Figu e 7.
Figu e 7. Pe o mance e alua ion a he componen le el. This s age agg ega es me ic sco es in o
componen -le el e alua ions, allowing p io i iza ion based on ope a ional equi emen s.
The combina ion o me ic and componen e alua ions ensu es ha sys em pe o mance
is assessed holis ically, accoun ing o indi idual unc ion e ec i eness and o e all ope a ional
D ones 2025,9, 354 23 o 27
u ili y. This s uc u ed me hodology suppo s da a-d i en decision-making in selec ing
op imal DTI solu ions o speci ic use cases. The me hodology ensu es ha esul s a e no
only accu a e and epea able bu also ac ionable, enabling decision-make s o make in o med
choices when selec ing DTI solu ions. Fo a mo e de ailed desc ip ion o he es me hodology,
including he expe imen al se up and speci ic e alua ion p ocedu es, e e o [37].
9. Valida ion
Ensu ing he eliabili y and applicabili y o he es me hodology equi es a s uc u ed
alida ion p ocess. Valida ion is conduc ed in wo key s ages: simula ion-based e i ica ion
and ope a ional ield ials. Simula ion-based e i ica ion se es as he ini ial s ep, allowing
o con olled es ing o di e en scena ios in a i ual en i onmen . This s age is c ucial o
e ining pe o mance me ics, op imizing de ec ion algo i hms, and iden i ying po en ial
sys em limi a ions be o e eal-wo ld deploymen . By simula ing a wide ange o condi ions,
including ad e se wea he , elec onic in e e ence, and a ying d one speeds, his app oach
p o ides a p elimina y assessmen o sys em obus ness and adap abili y.
Following success ul simula ion-based e i ica ion, he me hodology unde wen al-
ida ion h ough eal-wo ld ope a ional ield ials. These ials a e conduc ed in di e se
en i onmen s o e alua e sys em adap abili y unde ac ual deploymen condi ions and in
di e en geog aphical con ex s. The e o e, ield ials we e o ganized in G eece, Belgium,
and Spain, as depic ed in Figu e 8.
(a) (b) (c)
Figu e 8. Valida ion o he s anda dized e alua ion me hodology ook place in di e en es si es wi h
a wide ange o d ones and a wide ange o coun e -d one solu ions in o de o assess he obus ness
o he pe o mance assessmen . The igu es show snapsho s o he es s being pe o med in: (a) G eece
(A hens Ai po ). (b) Belgium (Lomba dsijde Mili a y Base). (c) Spain (ATLAS es cen e).
Du ing he ials, di e en ypes o d ones we e used o es he pe o mance o he
DTI sys ems. The g ound u h used o measu e his pe o mance was he d one ajec o y
eco ded by GPS logge de ices ins alled on boa d he d ones. In addi ion, he eleme y
da a egis e ed by he d one au opilo was also logged o ha e an addi ional sou ce o
in o ma ion in case any da a inconsis encies we e de ec ed a e p ocessing he logs.
The coun e -d one companies p o ided hei da a in a s anda d da a o ma ha was
designed speci ically o log he in o ma ion om he DTI sys ems du ing hei ope a ion.
Fo his da a o ma , a s uc u ed o ma has been chosen as opposed o, e.g., a able,
as da a scopes can a y. Fo ins ance, he e sion o he o ma is global o he documen ,
whe eas he ele a ion o a poin is speci ic o a single da a poin . Using a s uc u ed o ma
also allows easily ex ending i wi hou b eaking backwa ds compa ibili y. JSON has been
chosen due o i s simplici y and he numbe o lib a ies a ailable o w i ing and pa sing
da a. This da a o ma is open sou ce and can be ound a h ps://g c.us.es/cou ageous/
(accessed on 22 Ap il 2025).
D ones 2025,9, 354 24 o 27
A ange o Cus om O -The-Shel and homebuil d ones we e used du ing he ials.
All he d ones we e equipped wi h GPS logge s and we e able o ly p e-p og ammed
ligh pa hs. A ange o ligh ope a ions we e planned based upon he scena ios desc ibed
abo e, cycling h ough he a ailable d ones and combining se e al changing pa ame e s
such as al i ude and speed. In addi ion, he e we e also blind ee- ligh missions ca ied
ou by police uni s in o de o simula e mo e e a ic ligh mo emen s.
Gi en he ou pu o he es ed DTI sys ems as well as he co esponding g ound u h,
a compa a i e analysis o he collec ed da a and g ound u h was pe o med. In a i s
s ep, da a p e-p ocessing was pe o med h ough ime window il e ing, ans o ma ion
o coo dina e sys ems and selec ion o a eas o in e es . In a second s ep, he de ec ion
acks a e associa ed wi h he g ound u h, and inally, he esul ing pe o mance me ics
a e calcula ed and no malized. These me ics a e used o gene a e an agg ega ed sco e
sepa a ely pe es ed DTI sys em. These da a we e hen compa ed o he da a p o ided by
he coun e -d one companies pa icipa ing in he ials.
These ield ials allowed o he assessmen o DTI pe o mance in uncon olled
se ings, whe e ac o s such as u ban densi y, en i onmen al noise, and unp edic able
d one beha io in luence esul s. Addi ionally, ope a ional ials p o ide empi ical pe -
o mance benchma ks, ensu ing ha he es me hodology emains ele an and p ac ical
o eal-wo ld applica ions. The i e a i e na u e o his alida ion p ocess allows o con-
inuous imp o emen s, enabling e inemen s based on ield obse a ions and eedback
om end-use s.
By combining simula ion-based e i ica ion wi h li e ope a ional ials, he alida-
ion me hodology ensu es ha he es ing amewo k is bo h scien i ically igo ous and
ope a ionally ele an . The in eg a ion o hese wo s ages p o ides a holis ic e alua ion ap-
p oach, mi iga ing isks associa ed wi h sys em deploymen and ensu ing ha pe o mance
assessmen s emain accu a e and ac ionable. Fo a de ailed discussion o he alida ion
p ocess, including speci ic ial designs and pe o mance analysis me hods, e e o [38].
10. Conclusions
This pape p esen ed a s anda dized e alua ion me hodology o coun e -d one so-
lu ions, de eloped wi hin he amewo k o he COURAGEOUS p ojec . The inc easing
h ea posed by unau ho ized d one ac i i y, coupled wi h he lack o widely accep ed
pe o mance assessmen me hods, necessi a es a igo ous and epea able es ing amewo k
o coun e -d one echnologies.
The p oposed me hodology de ines a se o s anda dized es scena ios designed o
e lec eal-wo ld ope a ional challenges ac oss di e se en i onmen s, including c i ical
in as uc u e p o ec ion, public e en secu i y, and bo de su eillance. A comp ehen-
si e e alua ion amewo k was de eloped, in eg a ing bo h quali a i e and quan i a i e
pe o mance me ics. These me ics, including de ec ion accu acy, acking s abili y, e-
sponse ime, and alse ala m a es, ensu e a s uc u ed and objec i e assessmen o coun e -
d one capabili ies.
To alida e he me hodology, ex ensi e ield ials we e conduc ed in mul iple loca-
ions, including G eece, Belgium, and Spain. These ials demons a ed he applicabili y
and obus ness o he e alua ion amewo k, con i ming i s abili y o di e en ia e sys-
em pe o mance ac oss a ious ope a ional condi ions. The in eg a ion o s uc u ed
da a logging and s anda dized epo ing o ma s u he enhances he anspa ency and
epea abili y o he assessmen p ocess.
The p esen ed esul is a p e-s anda d, which is open and ee o e e yone o con-
sul . Howe e , mo e wo k is equi ed o ans o m his p e-s anda d in o a ull s anda d,
which will be he subjec o u u e esea ch. Mo e esea ch is indeed equi ed o e-
D ones 2025,9, 354 25 o 27
ining s anda dized es ing p o ocols, pa icula ly in add essing e ol ing h ea s posed
by inc easingly au onomous and s eal hy d ones. Mo eo e , he limi a ion adop ed in
he p esen ed wo k o only conside ing he DTI aspec o he coun e -d one kill chain is
highly a i icial and canno be main ained in he u u e. Indeed, om a clien ’s pe spec-
i e, coun e -d one ools need o p o ide holis ic solu ions o he UAS incu sion p oblem,
and his does mean inco po a ing a clea link be ween de ec ion, acking, iden i ica ion
and neu aliza ion/in e cep ion and—no o be o go en— o ensics and aining. Fu u e
esea ch should hus ocus on linking DTI and neu aliza ion aspec s, expanding es sce-
na ios, and enhancing in e ope abili y wi h exis ing secu i y in as uc u es. By aligning
wi h ongoing in e na ional s anda diza ion ini ia i es, he COURAGEOUS amewo k
p o ides a c i ical s ep owa d he objec i e benchma king and ce i ica ion o coun e -
d one echnologies, ul ima ely suppo ing mo e in o med decision-making o secu i y
and de ense s akeholde s.
Au ho Con ibu ions: Concep ualiza ion, me hodology, w i ing— e iew and edi ing, alida ion,
o mal analysis, D.D., P.P., A.K., K.B., M. ˙
Z., R.R., S.S., A.M., J. .d.P., I.M., A.O., C.C. and C.P.; w i ing—
o iginal d a p epa a ion, p ojec adminis a ion, unding acquisi ion, G.D.C. All au ho s ha e ead
and ag eed o he published e sion o he manusc ip .
Funding: This p ojec has ecei ed unding om he Eu opean Union’s In e nal Secu i y Fund Police
unde G an Ag eemen 101034655 (COURAGEOUS).
Da a A ailabili y S a emen : The da ase s p esen ed in his a icle a e no eadily a ailable because
he pe o mance da a o coun e -d one ools ha p o ec c i ical in as uc u e ae secu i y-sensi i e
da a and canno be sha ed. Howe e , he CEN/CENELEC Wo kshop ag eemen , which is a p e-
s anda d documen ha ex ensi ely de ails he p esen ed wo k and includes a de ailed me hodology
and implemen a ion guidelines o assessing he pe o mance o d one-d one solu ions, can be
accessed eely a h ps://www.cencenelec.eu/media/CEN-CENELEC/CWAs/ICT/cwa-18150.pd
(accessed on 22 Ap il 2025).
Con lic s o In e es : The au ho s decla e no con lic s o in e es . The unde s had no ole in he design
o he s udy; in he collec ion, analyses, o in e p e a ion o da a; in he w i ing o he manusc ip ; o
in he decision o publish he esul s.
Abb e ia ions
The ollowing abb e ia ions a e used in his manusc ip :
C-UAS COun e Unmanned Ae ial Sys ems
CWA CENELEC Wo kshop Ag eemen
DIN Deu sches Ins i u ü No mung
DTI De ec ion, T acking, Iden i ica ion
EUROCAE Eu opean O ganisa ion o Ci il A ia ion Equipmen
FAA Fede al A ia ion Adminis a ion
KPI Key Pe o mance Indica o
KRI Key Risk Indica o s
LEA Law en o cemen agency
NATO No h A lan ic T ea y O ganiza ion
ISO In e na ional S anda diza ion O ganisa ion
NIST Na ional Ins i u e o S anda ds and Technology
NPSA Na ional P o ec i e Secu i y Au ho i y
OSED Ope a ional Se ices and En i onmen De ini ion
RF Radio F equency
UAS Unmanned Ae ial Sys ems
