senso s
A icle
Visual Diagnosis o he Va oa Des uc o Pa asi ic Mi e in
Honeybees Using Objec De ec o Techniques
Simon Bilik , Lukas K a och ila , Adam Ligocki , Ond ej Bos ik , Tomas Zemcik , Ma ous Hybl ,
Ka el Ho ak * and Ludek Zalud


Ci a ion: Bilik, S.; K a och ila, L.;
Ligocki, A.; Bos ik, O.; Zemcik, T.;
Hybl, M.; Ho ak, K.; Zalud, L. Visual
Diagnosis o he Va oa Des uc o
Pa asi ic Mi e in Honeybees Using
Objec De ec o Techniques. Senso s
2021,21, 2764. h ps://doi.o g/
10.3390/s21082764
Academic Edi o : C aig Michie
Recei ed: 24 Feb ua y 2021
Accep ed: 7 Ap il 2021
Published: 14 Ap il 2021
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2021 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
Depa men o Con ol and Ins umen a ion, B no Uni e si y o Technology, 61 600 B no, Czech Republic;
[email p o ec ed].cz (S.B.) [email p o ec ed].cz (L.K.); adam.ligocki@ u b .cz (A.L.);
[email p o ec ed].cz (O.B.); [email p o ec ed].cz (T.Z.); [email p o ec ed].cz (M.H.);
[email p o ec ed].cz (L.Z.)
*Co espondence: [email p o ec ed].cz
Abs ac :
The Va oa des uc o mi e is one o he mos dange ous Honey Bee (Apis melli e a) pa asi es
wo ldwide and he bee colonies ha e o be egula ly moni o ed in o de o con ol i s sp ead. In his
pape we p esen an objec de ec o based me hod o heal h s a e moni o ing o bee colonies. This
me hod has he po en ial o online measu emen and p ocessing. In ou expe imen , we compa e he
YOLO and SSD objec de ec o s along wi h he Deep SVDD anomaly de ec o . Based on he cus om
da ase wi h 600 g ound- u h images o heal hy and in ec ed bees in a ious scenes, he de ec o s
eached he highes F1 sco e up o 0.874 in he in ec ed bee de ec ion and up o 0.714 in he de ec ion
o he Va oa des uc o mi e i sel . The esul s demons a e he po en ial o his app oach, which will
be la e used in he eal- ime compu e ision based honey bee inspec ion sys em. To he bes o ou
knowledge, his s udy is he i s one using objec de ec o s o he Va oa des uc o mi e de ec ion on
a honey bee. We expec ha pe o mance o hose objec de ec o s will enable us o inspec he heal h
s a us o he honey bee colonies in eal ime.
Keywo ds:
Va oa des uc o ;Apis melli e a; wes e n honey bee; bee heal h moni o ing; objec de ec ion;
YOLO; SSD; deep lea ning
1. In oduc ion
Va oa des uc o mi e (V.-mi e) is a honey bee (Apis melli e a) ec opa asi e, which is
sp ead a ound he whole wo ld, excep Aus alia. This pa asi e can be seen a
Figu e 1
. I
o igina ed om Eas Asia and s a ed o sp ead a ound he wo ld a ound he hal o he
20 h cen u y. Because o i s Eas Asian descend and ela i ely ecen a i al, he wes e n
honey bee is no adap ed o his pa asi e. Due o his ac o , pe iodic V.-mi e moni o ing
and ea men has o be pe o med. The con en ional moni o ing me hods include he
op ical inspec ion o he bee colony de i us (when he bee keepe sea ches de i us o he
allen mi es), o he V.-mi e isola ion om he bee sample ( o example powde ing sample o
he bees wi h suga , which causes he mi es o all), bu hose me hods a e ime consuming
and labou in ensi e. The o he disad an age o hese me hods is, ha only a pa o he
mi es could be de ec ed. The V.-mi e also becomes esis an o he cu en medica ions, so
new ea men me hods and de ec ion app oaches o ea ly in ec ion wa ning ha e o be
de eloped [1].
The au ho s o he pape [
2
] also iden i y he mos common a eas, whe e he
V.-mi e
a ach o he bee. I a acks he bees in he la al s a e and eeds on i s a issue, mos ly in
he belly a ea. This causes de elopmen ailu es o he bee b ood and i s losses. Su i ing
bees a e o en de o med due o hese ailu es and hey could be ha med by seconda y i al
in ec ions p opaga ed by his mi e as well (V.-mi e is con i med, o suspec o ansmi
a leas 18 i uses) [
2
]. All hese ac o s can cause se ious, e en a al bee colony losses.
Senso s 2021,21, 2764. h ps://doi.o g/10.3390/s21082764 h ps://www.mdpi.com/jou nal/senso s
Senso s 2021,21, 2764 2 o 16
Al hough he mos c i ical a e losses in he win e pe iod, p ope ea men has o be pe -
o med al eady in he la e summe , so he bees spending win e a e al eady no signi ican ly
a ec ed by his mi e.
Bee colony win e losses a e also conce ned in he wo k [
3
], which p esen s he esul s
ob ained om he moni o ing o 1200 bee colonies. The au ho s p o e connec ion o
he V.-mi e in es a ion o bee colony win e mo ali y, whe e he in es a ion le el was
s a is ically lowe in he su i ing colonies. The s udy shows, ha he V.-mi e in es a ion
le el o 10% (V.-mi es o e 100 bees) inc eases he isk o he bee colony collapse up o 20%
and he in es a ion le el o 20% inc eases his isk up o 50%. The esul s also show he
co ela ion be ween he V.-mi e in es a ion le el, p esence o se e al bee i uses in such
in ec ed colonies and also wi h he s ill unclea Colony Collapse Diso de , which caused
huge losses in se e al s a es wo ldwide.
Liebig [
4
] conside s a sa e V.-mi e in es a ion a e in an un ea ed bee colony up o 7%.
The au ho shows he impo ance o a p ope and well imed ea men , when e en hea ily
in ec ed colonies wi h in es a ion le el up o 70% ea ed in la e Augus could be sa ed
wi h a single ea men , bu colonies wi h in es a ion le el o e 25% ha we e ea ed in
No embe a e s ill in a signi ican dange o collapse.
Figu e 1.
V.-mi e on an adul bee wo ke (in he cen e) [
1
] Rep in ed om Publica ion Jou nal o
In e eb a e Pa hology, 103, P. Rosenk anz; P. Aumeie ; B. Ziegelmann, Biology and con ol o Va oa
des uc o , 96–101, Copy igh (2010), wi h pe mission om Else ie .
In his pape we p esen a non-in asi e compu e ision (CV) based app oach o he
V.-mi e moni o ing, which can be la e used as a ounda ion o a po able and low-cos
online moni o ing sys em. We belie e, ha his me hod could ha e he po en ial o de ec
mo e mi es, compa ed o adi ional bee keeping me hods men ioned abo e. This could
esul in an ea ly and mo e accu a e V.-mi e in es a ion de ec ion and o he possibili y o a
as e and mo e p ope ea men .
Ou pape is s uc u ed as ollows: In he i s pa , we b ing a sho o e iew o
he exis ing bee moni o ing me hods wi h he emphasis on he exis ing V.-mi e au oma ic
de ec ion sys em and he ision based honey bee moni o ing sys ems. The second pa
desc ibes he echniques used in ou expe imen and he da ase we used. In he end, we
discuss he expe imen ’s esul s and we p o ide sugges ions o he u he esea ch.
2. Rela ed Wo k
Se e al s udies ocus on a ious honey bee moni o ing me hods, o example he
sound analysis o he swa m de ec ion in [
5
], bee ac i i y moni o ing in [
6
], o he RFID
based indi idual bee acking in [
7
]. A b ie o e iew o o he and abo e men ioned
honeybee moni o ing echniques can be also ound in [8].
Senso s 2021,21, 2764 3 o 16
O he image p ocessing based me hods, he pape [
9
] ocuses on bee au og ooming
phenomena. This s udy de elops a new me hodology o modeling he au og ooming
p ocess. The bees a e co e ed by baking lou and hei g ooming is obse ed by a came a
on a con olled backg ound. The speed o he g ooming p ocess is au oma ically analysed.
In [
10
] he au ho s y o classi y he honeybee b ood cells in o h ee ca ego ies:
occluded, isible and closed, and isible and uncapped o he bee colony sel -hygiene
moni o ing. Au ho s de elop a simple sys em o he honeycomb’s op ical inspec ion,
whe e hey compa e he pe o mance o se e al classi ie s. The e alua ed classi ying
algo i hms a e: suppo ec o machine (SVM), decision ee and boos ed classi ie s. The
bes esul s a e ob ained wi h he SVM.
The pape [
11
] p esen s a CNN based app oach o bee classi ica ion based on whe he
o no he bees a e ca ying pollen. Ha dwa e se up o image acquisi ion is desc ibed,
me hods o bee segmen a ion a e discussed and VGG16, VGG19 and ResNe 50 CNN
pe o mance is compa ed o classical classi ie s such as KNN, Nai e Bayes and SVM.
Rod iguez e al. conclude ha shallowe models such as VGG ha e be e pe o mance
compa ed o deepe models such as ResNe . Gene ally pe o mance o CNN based classi-
ie s was supe io o con en ional classi ie s. Wha is also impo an is he disco e y ha
he use o a human p ede ined colou model, needed o classical classi ie s, in luenced only
he aining ime bu no he inal pe o mance o he CNNs when compa ed o s aigh
RGB inpu . The au ho s also c ea ed a publicly a ailable da ase [
12
] wi h high- esolu ion
images o he pollen bea ing and non-pollen bea ing bees wi h 710 image samples in o al.
Simila ly o he pape abo e, he disse a ion [
13
] aims o he ecogni ion and acking
o pollen wea ing bees. The au ho uses a Fas e -RCNN objec de ec o o he pollen
de ec ion on a cus om da ase and compa es i s esul s wi h a s a is ic based me hods,
which a e clea ly ou pe o med by Fas e -RCNN. This wo k con ains mo e aluable esul s
in he bee acking using Kalman Fil e ing.
The pape [
14
] uses a non-in asi e hyb id app oach o ack bees in hei na u al
en i onmen . The p oposed algo i hm is qui e complex, bu in a simpli ied way he au ho s
use a usion o a backg ound sub ac ion based de ec ion and YOLO 2 objec de ec o .
The algo i hm is able o ack a single bee in a cap u ed scene and plo i s ajec o y in a
2D plane, which is an in e es ing con ibu ion o he s a e-o - he-a . This wo k migh be
use ul in he s udying o bee beha iou , o i s dances in u u e.
The au ho s o he a icle [
15
] ocus on he moni o ing o he bees en e ing and lea ing
he hi e, which could be used o he bee colony heal h diagnos ics as well. Fo he bee
de ec ion, hey use a pipeline o a mo ion de ec o o an objec segmen a ion and a usion
o he Random Fo es me hod oge he wi h a CNN based image classi ie . The au ho s
use p e iously de eloped BeePi in- ield moni o ing sys em o cap u e he inpu da a. Two
cus om da ase s c ea ed and used in his expe imen a e made publicly a ailable.
The Gi Hub eposi o y a ailable a [
16
] p esen s a YOLO ne wo k ained o ecognize
be ween honey bees, wasps and ho ne s. The esul s o his expe imen a e no discussed,
o published, bu a da ase should be a ailable on eques . This wo k shows he possibili y
o moni o ing bees en e ing he hi e wi h a common came a.
The in es iga ed V.-mi e de ec ion echniques a e desc ibed in he ollowing a icles:
The s udy p esen ed in [
17
] is ocused on de ec ion o V.-mi e in es a ion o honeybee
colonies. This s udy is based on beehi e ai measu emen s using an a ay o pa ially
selec i e gas senso s in ield condi ion. Collec ed da a was used o aining simple LDA
and k-NN classi ie s wi h he bes misclassi ica ion e o o k-NN we e 17%. The esul s
indica ed a possibili y o de ec honey bee diseases wi h his app oach, bu he me hod
mus be imp o ed be o e deploymen . In pa icula , he big and expensi e da a collec ion
ha dwa e alongside he classi ie need o be imp o ed.
The pape [
18
] shows ha he V.-mi e-in ec ed b ood cells a e sligh ly wa me han he
unin es ed ones. The expe imen was based on he he mal ideo sensing o b ood ames
wi h he a i icially in ec ed and unin ec ed cells in he con olled he mal condi ions. The
Senso s 2021,21, 2764 4 o 16
au ho s obse ed ha in ec ed cells a e sligh ly wa me (be ween 0.03 and 0.19
◦
C), which
shows he possibili y o he he mal measu emen s in o de o de ec mi e-in ec ed b oods.
The a icle p esen ed in [
19
] shows a me hod o moni o ing he V.-mi e’s mo ion on
he bee b ood samples. The au ho s use classical compu e ision me hods as backg ound
sub ac ion and geome ic pa e n analysis wi h double h esholding o de ec he mi es on
he ideo ames ollowed by hei acking on he bee b ood wi h he accu acy a e o 91%.
This me hod is no sui able o in- ield-analysis, because he b ood mus be emo ed om
he cell be o e he analysis.
The s udy [
20
] desc ibes he expe imen al se up o a non-des uc i e V.-mi e de ec ion
and b ings sugges ions o u he de elopmen . I also iden i ies he challenges o mi e
de ec ion, o example as mo ion blu , mi e colou , o e lec ions. This pape only b ie ly
desc ibes possible segmen a ion and de ec ion echniques.
The ollowing a icle [
21
] om he same au ho s desc ibes bee de ec ion and hei
classi ica ion, using classical compu e ision me hods. In he i s pa , he au ho s p esen
good esul s in segmen a ion o indi idual bees using a combina ion o he Gaussian
Mix u e Models and he colou h esholding based me hods. In he second pa , hey
p o ide he esul s o a wo class classi ica ion (heal hy bee/bee wi h mi e) using he Nai e
Bayes, SVM and Random o es classi ie s. The accu acy o hese me hods a ies acco ding
o he ame p ep ocessing echniques and in he bes cases mo es a ound 80%.
The las a icle [
22
] o he same au ho s ex ends he a icle [
21
] wi h he CNN based
classi ie s (ResNe and AlexNe ) and he DeepLabV3 seman ic segmen a ion. The p ocess-
ing pipeline and he classi ica ion classes emain he same, as hey we e in he p e ious
a icle. Classi ica ion o images wi h a single bee shows good esul s wi h he accu acy
o he “in ec ed” class a ound 86%. The expe imen s wi h he bees classi ica ion om he
whole image using he sliding window b ing weake esul s wi h he alse-nega i e a e
a ound 67%. The au ho s also made hei da ase publicly a ailable [23].
The s udy p esen ed in [
24
] builds on he [
11
,
20
,
21
]. The au ho s p esen an en i e
V.-mi e de ec ion pipeline. Fi s ly, hey desc ibe a ideo moni o ing uni wi h he mul i-
spec al illumina ion, which could be connec ed di ec ly o he beehi e h ough se e al
na ow pa hways. The cap u ed ames a e eco ded and p ocessed o line, as he bees
a e segmen ed wi h classical compu e ision me hods and he mi es a e de ec ed wi h a
cus om CNN model. The au ho s achie ed good esul s in he mi e in es a ion es ima ion
and hey p o ed he po en ial o he CV based on-si e es ing.
P ac ical applica ions s emming om e icien bee moni o ing sys ems and V.-mi e
de ec ion sys ems ha e also been s udied. The au ho s o [
25
] in es iga e he possibili y o
using a came a based bee moni o ing sys em on he hi e’s en ance o de ec V.-mi es and
hen des oy hem wi h a ocused lase beam. The au ho s ou lined ha dwa e and so wa e
equi emen s o such a sys em and ound i easible e en i such sys ems ha e no ye
been de eloped o a deployable s a e.
3. Ma e ials and Me hods
In his chap e , we i s ly desc ibe insigh s, ou da ase and i s s a is ics. Then ollows
a b ie desc ip ion o he used de ec o ’s a chi ec u es along wi h he hype pa ame e s and
he e alua ion me ics.
The goal o ou expe imen was o asce ain whe he he s a e-o - he-a objec de ec-
o s, such as YOLO 5 and SSD, can pe o m he V. mi e and bee de ec ion, al e na i ely
o de ec and dis inguish be ween he heal hy and he in ec ed bees. Cu en app oaches
as [
21
,
24
], o [
22
] use a compu a ionally expensi e me hods o a bee segmen a ion ( o
example Gaussian Mix u e Models in [
21
]), o a bee de ec ion (SIFT and SURF based
me hods in [
24
]). All hose pape s also sepa a e segmen a ion and classi ica ion, which
esul s in a slow p ocessing o he inpu image.
The SSD and YOLO objec de ec o s do no sepa a e he objec segmen a ion and
classi ica ion, which esul s in a possibili y o an online p ocessing e en on he embedded
Senso s 2021,21, 2764 5 o 16
pla o ms as NVIDIA Je son. We belie e, ha his ad an age o se s an expec ed lowe
de ec ion accu acy in compa ison wi h a ine uned segmen a ion algo i hms.
3.1. Da ase Desc ip ion
Du ing he ini ial phase o ou esea ch, we ound se e al publicly a ailable honey-bee
da ase s. The da ase om [
11
] is designed o he pollen-wea ing bee ecogni ion only and
despi e i s quali y and high esolu ion, i was unsui able o ou ask. The da ase p esen ed
in [
22
] con ains sho ideos wi h mi e-in ec ed and una ec ed bees in high esolu ion and
i was pa ially used in ou expe imen . The da ase [
26
] seemed p omising and b ough a
lo o addi ional in o ma ion, bu he images we e in a bad quali y and low esolu ion.
Fo he abo e easons, ou cus om da ase was compiled om publicly a ailable
images and pa ially om he da ase [
23
]. I con ains a o al o 803 unique samples,
whe e 500 samples cap u e bees in he gene al en i onmen and he es , aken om
he da ase [
23
], shows bees on an a i icial backg ound. Da ase samples can be seen a
Figu e 2.
Figu e 2.
B ie o e iew o da ase c ea ed o he pu pose o his wo k. Heal hy bees (g een), bees
wi h pollen (yellow), d ones (blue), queens (cyan), in ec ed bees (pu ple), V.-mi e ( ed).
In ou da ase and wi h u u e expe imen a ion in mind, we de ine six classes shown
in Figu e 3, he heal hy bee (1), he bee wi h pollen (2) (pollen may be miss-classi ied wi h a
V.-mi e), he d one (3), he queen (4), V.-mi e-in ec ed bee (5), and he V.-mi e (6). Howe e ,
only o his pape , we educed he da a anno a ion in o h ee di e en subse s. The i s
one is he bees (classes 1, 2, 3, 4, 5) and V.-mi e (class 6), he second one is he heal hy bees
(classes 1, 2, 3, 4) and in ec ed bees (class 5), inally in he las subse a e he da a anno a ed
only wi h V.-mi e (only class 6).
We c ea ed hese h ee de i a ions o he o iginal da ase anno a ion o es , which one
will gi e us he bes esul in de ec ing a oosis in ec ion in he beehi e. Ou da ase was
manually anno a ed wi h he LabelImg ool [
27
], and i s s a is ics a e p esen ed in Tables 1,
2,3and 4. All anno a ions we e consul ed wi h a p o essional beekeepe .
E en hough we used a pa o he images om he da ase [
23
], he numbe o
ga he ed images o ain he neu al ne wo k was no su icien . The e o e we ha e decided
o apply an augmen a ion o he en i e aining subse . Gene ally speaking, augmen a ion
helps inc ease he obus ness o he aining p ocess, helps o a oid o e i ing, and o e all
imp o es he neu al ne wo k’s pe o mance a he end o he aining phase [28].
To apply augmen a ion o ou da a, we ha e used he Py hon lib a y called Im-
gAug [
29
]. I p o ides a wide ange o a ious me hods o augmen he image by blu ing,
adding di e en noise signals, a ious e ec s (like mo ion, og, ain, colo space al e a ion,
e c.), andom e asing (Random E asing Da a Augmen a ion) bounding box modi ica ions,
o e en he colo space modi ica ions and shi ing o image cu ou s [28,30].
The main idea o he augmen a ion me hod is o c ea e sligh ly modi ied de i a i es
o he o iginal aining da a ha , e en a e modi ica ion, s ill ep esen s he o iginal

Senso s 2021,21, 2764 6 o 16
p oblem. In he case o objec de ec ion o , gene ally speaking, compu e ision, by
applying o a ion, geome ical dis o ion, addi ional noise, o he colou shi , we do no
modi y he in o ma ion ha he image con ains. La e , du ing he aining p ocess, he
neu al ne wo k is o ced o lea n how o sol e he gi en ask o pe ec -looking aining
da a as well as, o dis o ed, noise-added, o colou shi ed images. I makes models o
be e gene alize p oblems, and he en i e lea ning p ocess is way mo e obus agains
o e i ing.
In he case o V.-mi e de ec ion, we can illus a e he p oblem o he close simila i y
be ween he V.-mi e and he bee’s eye. Bo h objec s a e e y close o each o he in hei
geome ical shapes and dimensions. The di e ences he e a e he colou and he close
su oundings. The mi e is s ic ly b own whe eas he bee’s eyes a e black. I we le he
neu al ne wo k o ain on he unaugmen ed images, i could lea n o iden i y mi es only
by he p esence o he b own colou . I he images a e augmen ed, he neu al ne wo k has
o unde s and he en i e s uc u e o he mi e body.
Figu e 3.
Example o a single image (le op) augmen ed in o he en mo e aining samples. In he op le column, he e
is he o iginal image o a ed by 0, 90, 180, and 270 deg ees. In he es o he ow, he e a e he images augmen ed by he
ImgAug amewo k.
In ou case, we o a ed each image by 90, 180, and 270 deg ees, and o e e y o a ion,
we ha e c ea ed en de i a i es o images by applying a single, andomly selec ed aug-
men a ion s yle om he a ailable se . Using his me hod, we had augmen ed e e y single
image om he o iginal aining se and we c ea ed addi ional 43 samples. I is impo an
o no e ha he alida ion and he es se s ay wi hou any modi ica ions, so hey ep esen
he bees and V.-mi es eal-li e image da a.
In o al, we c ea ed he da ase con aining 803 images. The aining se con ains
561 images, la e augmen ed o he o al o 24,684 images, 127 images comp ise he
alida ion se , and he es se con ains u he 115 samples. All images in he o iginal
803-images se a e independen o each o he . Fo hose 803 images, we c ea ed h ee
anno a ions as men ioned be o e, and he numbe o anno a ed ins ances in each class o
aining, alida ion and es se is in Tables 1,2,3and 4.
Table 1. The able shows he numbe o anno a ed objec s o gi en classes.
All Classes
Da ase
No. o Anno a ed Objec s pe Class
Images
Bee Wo ke
(No Pollen)
Bee Wo ke
(Pollen)
Bee
D one
Bee
Queen
Bee wi h
V.mi e(s) V.-Mi e
1158 143 19 52 298 424 803
Senso s 2021,21, 2764 7 o 16
Table 2. The able shows he numbe o anno a ed objec s o gi en classes o he Bees and a oosis da ase .
Bees and
V.-mi es
Da ase
Classes Bees V.-Mi e Images
T ain Se 1148 250 561
T ain Aug Se 50,512 11,000 24,684
Val Se 274 92 127
Tes Se 248 59 115
Table 3. The able shows he numbe o anno a ed objec s o gi en classes o he Heal hy and In ec ed bees da ase .
Heal hy and Ill
Bees Da ase
Classes Heal hy Bees In ec ed Bees Images
T ain Se 956 192 561
T ain Aug Se 42,064 8448 24,684
Val Se 220 54 127
Tes Se 196 52 115
Table 4. The able shows he numbe o anno a ed objec s o gi en classes o he Va oosis only da ase .
V.-Mi es
Da ase -
Classes V.-Mi e Images
T ain Se 250 561
T ain Aug Se 11,000 24,684
Val Se 92 127
Tes Se 59 115
We a e awa e ha o image classi ica ion asks i is c ucial o ha e a balanced da ase
o all classes. In case a balanced da ase is no a ailable, he a chi ec u e will no be able o
ain classi ying he unde ep esen ed classes. In o de o iden i y his po en ial p oblem
in ou sligh ly unbalanced da ase , we use he mAP[0.5] sco e as one o ou me ics. This
sco e is a ec ed by all classes wi h he same weigh and i is desc ibed in mo e de ail below.
3.2. Ne wo k Desc ip ion
In his sec ion a b ie desc ip ion o he YOLO 5, SSD objec de ec o s and he Deep
SVDD anomaly de ec o is p o ided as used in ou expe imen .
3.2.1. YOLO 5
The o iginal YOLO [
31
] and all i s de i a i es (YOLO9000, YOLO 3, YOLO 4) a e
examples o end- o-end objec de ec ion models. I means he in e ence o he ne wo k wi h
he image is he only ope a ion pe o med du ing he objec de ec ion. The e is no hing like
egion p oposals, combina ion o de ec ed bounding boxes e c. [
32
]. The YOLO a chi ec u e
has only he image on he inpu and he ec o o de ec ions and p obabili ies o hese
de ec ions on he ou pu .
In his pape , we ha e used he open-sou ce implemen a ion o he YOLO objec
de ec o , called YOLO 5 om Ul aly ics, a ailable om [
33
]. The YOLO 5 is no a sel -
s anding e sion, ha would b ing signi ican imp o emen s in he YOLO-like neu al
ne wo ks a chi ec u es, mo e i is one example o implemen ing he YOLO 3 p inciples in
he PyTo ch amewo k [
34
]. Also, as o he day o w i ing his pape , he e is no o icial
pee - e iewed a icle abou YOLO 5.
The YOLO 5 consis s o h ee main pa s. The backend is a s anda d con olu ional
backend as we know i om o he neu al ne wo ks (VGG, ResNe , Da kNe , e c). The back-
end ex ac s he ea u e maps om he inpu image and pe o ms he geome ical pa e n
Senso s 2021,21, 2764 8 o 16
de ec ions. As we go deepe h ough he backend, he ex ac ed ea u e maps’ esolu ion is
dec easing, and he neu al ne wo k de ec s la ge and mo e complex geome ical shapes.
The second pa is he “neck” s age. I akes he ea u e maps om se e al backend le -
els (di e en backend le els de ec objec s o di e en sizes and geome ical complexi ies)
and combines hem in o h ee ou pu scales.
The las pa is he p edic ion pa k ha uses 1
×
1 con olu ion laye s o map di e en
scales o conca ena ed ea u e maps om he “neck” s age in o h ee ou pu enso s.
The e exis wo a ian s o he YOLO 5 a chi ec u e, he S and he X model. The di -
e ence be ween he YOLO 5 S and X e sions is in he dimensions o he neu al ne wo ks.
The X e sion mul iplies he numbe o ke nels pe con olu ional laye by 2.5 w. . . he
S a ian and he numbe o con olu ional laye s a e mul iplied by 4. Compa ing he S
and X models by numbe s, he X e sion has 476 laye s, 87.7 million pa ame e s, and he S
e sion has 224 laye s and 7.2 million pa ame e s. The de ailed inne s uc u e o e iew
can be ound on he p ojec ’s Gi hub webpage [33].
3.2.2. SSD
The Single Sho Mul ibox De ec o (SSD) is a as objec de ec o , o iginally p esen ed
in [
35
]. The a chi ec u e consis s o a ea u e ex ac ing base-ne (o iginally VGG16) and
classi ying laye s. The base-ne indes ea u e map using con olu ional laye s, wi h he
numbe o ea u es ising as we go deepe in o he base-ne . The classi ying laye s a e
connec ed o he base-ne on se e al le els o di e en le el ea u es. Fo his expe imen ,
we used he open sou ce implemen a ion a ailable om [36].
The SSD de ec o p ocesses images o ba ch o images h ough ne and compu e
ou pu . The ou pu is ep esen ed by he p edic ed objec ’s loca ion bounding box and
con idence. Fo his ou pu he a chi ec u e gene a es ens o housands o p io s, which a e
base bounding boxes simila o ancho boxes in Fas e RCNN [
32
] om which we compu e
eg ession loss.
This a chi ec u e c ea es he base ne and adds se e al p edic o s o inding di e en
scales o objec s. As we go deepe in o he base-ne , we assume inding a bigge objec as
he ea u e map co esponds o a bigge ecep i e ield. The i s p edic o inds boxes a e
14 con olu ion laye s, he e o e ails o ind e y small objec s ( ea u e map 38 ×38 px).
In ou expe imen , we use wo base-ne s. The i s one is he o iginal VGG16 and he
second in MobileNe 2. The main di e ence is in he ne wo k’s complexi y (VGG16 has
app ox 138 mil. pa ame e s and MobileNe 2 has app ox. 2.3 mil.).
3.2.3. Deep SVDD
The Deep Suppo Vec o Da a Desc ip ion (Deep SVDD) is a con olu ional neu al
ne wo k based anomaly de ec o (AD) p esen ed in he [
37
]. I is designed o he de ec ion
o anomalous samples (one class classi ica ion) and i al eady b ough us good esul s on
homogenous da ase s.
This echnique sea ches o a neu al ne wo k ans o m, which maps he majo i y o
he inpu da a o a hype sphe e wi h a cen e c and a adius R wi h a minimal olume.
The samples, which all in o his a ea a e conside ed as no mal and he samples ou o he
sphe e as anomalous. The cen e o he hype sphe e is compu ed om he mean ea u es
o he all inpu samples and he decision bounda y (pa ame e R) is se by a h eshold.
The inpu da a migh be p ocessed by an au oencode . Fo ou expe imen , we used he
implemen a ion a ailable om [38].
We used his model in ou expe imen e en hough i is no an objec de ec o , be-
cause we wan ed o p o e whe he his a chi ec u e yields good esul s in he in ec ed
bee de ec ion. As we men ioned in he in oduc ion, he V.-mi e in es ed bees a e o en
de o med and he pa asi e migh no be always isible, o p esen on he bee’s body. Wi h
he AD echnique based app oach, we could be able o de ec hose cases, o e en bees wi h
o he p oblems.
Senso s 2021,21, 2764 9 o 16
Fo his expe imen , we used 200 samples om he da ase [
23
], because his me hod
equi es simila looking samples, which could no be sou ced om ou pa o he da ase
desc ibed abo e.
3.3. Hype pa ame e s
We ained all ne wo ks on he N idia GTX 2080 GPU ca d and all ne wo ks we e
ained on a 640 by 640 px images. Fo he YOLO 5 he ba ch size was ou images. We used
he ADAM op imize . Fo he SSD we use ba ch size wi h en images and SGD op imize
wi h momen um o 0.9. Fo he YOLO 5 and SSD ne wo ks, we used he implemen a ion’s
de aul p e ained weigh s o boos he lea ning p ocess and sa e compu a ional esou ces.
In all cases, we le all ne wo k weigh s unlocked o aining.
We ained each YOLO model o 100 epochs and he pe o mance o he models
sa u a ed a e 30 epochs. The SSD models we e ained again o 100 epochs and hey
sa u a ed a e 40 epoch. The p obabili y h eshold was se o 0.4 o YOLO and 0.3 o SSD.
We le he de aul s ancho boxes o he YOLO 5 model: [10,13, 16,30, 33,23] (P3/8),
[30,61, 62,45, 59,119] (P4/16) and [116,90, 156,198, 373,326] (P5/32), as de ined in [
33
]. Fo
he SSD model, he ancho boxes equi ed mo e uning and we se he o bo h base ne s as
desc ibed in Table 5:
Table 5. Se ing o he SSD ancho boxes.
Base Ne Fea u e Map Size Sh ingkage Ancho Box Aspec Ra io
VGG 16
80 8 (15, 30) [2, 3]
40 16 (30, 60) [2, 3]
20 32 (60, 105) [2, 3]
10 64 (105, 150) [2, 3]
8 80 (150, 200) [2, 3]
6 107 (250, 340) [2, 3]
MobileNe 2
40 16 (15, 30) [2, 3]
20 32 (30, 60) [2, 3]
10 64 (60, 105) [2, 3]
5 128 (105, 150) [2, 3]
3 214 (150, 200) [2, 3]
2 320 (200, 340) [2, 3]
The size o he SSD ancho boxes was se acco ding o he size o he V-mi es in ou
da ase , which a ied in ange be ween 15–25 px.
3.4. Used Me ics
To es ima e he pe o mance o he a i icial in elligence models, we need o de ine
me ics ha will gi e us an idea o how well he model could sol e a gi en p oblem a e
he aining p ocess.
One o he mos common ways o exp ess objec de ec ion capabili y o pe o m well
is he mean a e age p ecision (mAP) me ics i s in oduced by he [
39
] as he mAP[0.5].
This me ic deno es he ela i e numbe o objec de ec ion p ecision using he minimal
in e sec ion o e union (IoU) wi h a alue equal o o bigge han 0.5.
Ano he a ian o he mAP me ics is he mAP[0.5:0.95], i s used by [
40
]. In addi ion
o mAP[0.5], mAP[0.5:0.95] calcula es he a e age o all mAP alues wi h he IoU le el
0.5 up o 0.95 wi h 0.05 s ep.
The meaning o ue posi i e, as well as alse posi i e and alse nega i e in con ex
o he IoU me ics can be seen in Figu e 4. As can be seen in he igu e, he esul is a ue
posi i e i he g ound u h (blue bounding box) and he de ec ion (g een bounding box)
has a le el o IoU a leas 0.5. We ob ain he alse posi i e esul i he neu al ne wo k
decla es he de ec ion, and he e is no g ound u h bounding box wi h a leas 0.5 IoU. The
alse nega i e esul means ha he g ound u h bounding box has no de ec ion o ma ch
wi h a leas 0.5 IoU. No e: he x alue de ines 0.5 IoU om mAP[x] me ic.
Senso s 2021,21, 2764 16 o 16
29. Jung, A.B. Imgaug. 2021. A ailable online: h ps://gi hub.com/aleju/imgaug (accessed on 1 Janua y 2021).
30.
Zoph, B.; Cubuk, E.D.; Ghiasi, G.; Lin, T.Y.; Shlens, J.; Le, Q.V. Lea ning da a augmen a ion s a egies o objec de ec ion.
In Eu opean Con e ence on Compu e Vision; Sp inge : Be lin/Heidelbe g, Ge many, 2020; pp. 566–583.
31.
Redmon, J.; Di ala, S.; Gi shick, R.; Fa hadi, A. You only look once: Uni ied, eal- ime objec de ec ion. In P oceedings o he
IEEE Con e ence on Compu e Vision and Pa e n Recogni ion, Las Vegas, NV, USA, 27–30 June 2016; pp. 779–788.
32.
Ren, S.; He, K.; Gi shick, R.; Sun, J. Fas e R-CNN: Towa ds eal- ime objec de ec ion wi h egion p oposal ne wo ks. IEEE T ans.
Pa e n Anal. Mach. In ell. 2016,39, 1137–1149. [C ossRe ] [PubMed]
33.
Joche , G.; Yolo 5. In Code Reposi o y; 2020. A ailable online: h ps://gi hub.com/ul aly ics/yolo 5 (accessed on
8 No embe 2020 ).
34.
Paszke, A.; G oss, S.; Massa, F.; Le e , A.; B adbu y, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; An iga, L.; e al. PyTo ch: An
Impe a i e S yle, High-Pe o mance Deep Lea ning Lib a y. In Ad ances in Neu al In o ma ion P ocessing Sys ems 32; Wallach, H.,
La ochelle, H., Beygelzime , A., d’Alch’e-Buc, F., Fox, E., Ga ne , R., Eds.; Cu an Associa es Inc.: Red Hook, NY, USA, 2019;
pp. 8024–8035.
35.
Liu, W.; Anguelo , D.; E han, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Be g, A.C. Ssd: Single sho mul ibox de ec o . In Eu opean
Con e ence on Compu e Vision; Sp inge : Be lin/Heidelbe g, Ge many, 2016; pp. 21–37.
36. Gao, H. Py o ch-Ssd. 2018. A ailable online: h ps://gi hub.com/q gaohao/py o ch-ssd (accessed on 14 Decembe 2020).
37.
Ru , L.; Vande meulen, R.; Goe ni z, N.; Deecke, L.; Siddiqui, S.A.; Binde , A.; Mülle , E.; Klo , M. Deep One-Class Classi ica ion.
In P oceedings o he Machine Lea ning Resea ch, P oceedings o he 35 h In e na ional Con e ence on Machine Lea ning,
S ockholm, Sweden, 10–15 July 2018; Dy, J., K ause, A., Eds.; PMLR: S ockholm, Sweden, 2018; Volume 80, pp. 4393–4402.
38.
Ru , L. PyTo ch Implemen a ion o Deep SVDD. 2019. A ailable online: h ps://gi hub.com/lukas u /Deep-SVDD-PyTo ch
(accessed on 21 Augus 2020).
39.
E e ingham, M.; Van Gool, L.; Williams, C.K.; Winn, J.; Zisse man, A. The pascal isual objec classes ( oc) challenge. In . J.
Compu . Vis. 2010,88, 303–338. [C ossRe ]
40.
Lin, T.Y.; Mai e, M.; Belongie, S.; Hays, J.; Pe ona, P.; Ramanan, D.; Dollá , P.; Zi nick, C.L. Mic oso coco: Common objec s in
con ex . In Eu opean Con e ence on Compu e Vision; Sp inge : Be lin/Heidelbe g, Ge many, 2014; pp. 740–755.
41.
K a och íla, L. Compu ing pla o ms o deep lea ning ask in compu e ision. In P oceedings o he 26 h Con e ence STUDENT
EEICT 2020; B no Uni e si y o Technology, FEEC: B no, Czech Republic, 2020; pp. 171–175.