Machine Vision and Applica ions (2023) 34:101
h ps://doi.o g/10.1007/s00138-023-01450-x
ORIGINAL PAPER
Towa d phy oplank on pa asi e de ec ion using au oencode s
Simon Bilik1,2 ·Daniel Ba akhano 1·Tuomas Ee ola1·Lumi Ha aguchi3·Kaisa K a 3·
Silke Van den Wyngae 4·Jonna Kangas4·Conny Sjöq is 5·Ka in Madsen5·Lasse Lensu1·
Heikki Käl iäinen1·Ka el Ho ak2
Recei ed: 15 Ma ch 2023 / Re ised: 16 June 2023 / Accep ed: 14 Augus 2023 / Published online: 13 Sep embe 2023
© The Au ho (s) 2023
Abs ac
Phy oplank on pa asi es a e la gely unde s udied mic obial componen s wi h a po en ially signi ican ecological in luence on
phy oplank on bloom dynamics. To be e unde s and he impac o phy oplank on pa asi es, imp o ed de ec ion me hods a e
needed o in eg a e phy oplank on pa asi e in e ac ions in o moni o ing o aqua ic ecosys ems. Au oma ed imaging de ices
commonly p oduce as amoun s o phy oplank on image da a, bu he occu ence o anomalous phy oplank on da a in such
da ase s is a e. Thus, we p opose an unsupe ised anomaly de ec ion sys em based on he simila i y be ween he o iginal and
au oencode - econs uc edsamples.Wi h hisapp oach,wewe eable o eachano e allF1sco eo 0.75inninephy oplank on
species, which could be u he imp o ed by species-speci ic ine- uning. The p oposed unsupe ised app oach was u he
compa ed wi h he supe ised Fas e R-CNN-based objec de ec o . Using his supe ised app oach and he model ained
on plank on species and anomalies, we we e able o each a highes F1 sco e o 0.86. Howe e , he unsupe ised app oach is
expec ed o be mo e uni e sal as i can also de ec unknown anomalies and i does no equi e any anno a ed anomalous da a
ha may no always be a ailable in su icien quan i ies. Al hough o he s udies ha e deal wi h plank on anomaly de ec ion
in e ms o non-plank on pa icles o ai bubble de ec ion, ou pape is, acco ding o ou bes knowledge, he i s ha ocuses
on au oma ed anomaly de ec ion conside ing pu a i e phy oplank on pa asi es o in ec ions.
Keywo ds Phy oplank on anomalies ·Phy oplank on pa asi es ·Anomaly de ec ion ·Au oencode s ·Objec de ec ion ·
Fas e R-CNN
S. Bilik and D. Ba akhano ha e con ibu ed equally o his wo k.
BSimon Bilik
[email p o ec ed]
Daniel Ba akhano
[email p o ec ed]
Tuomas Ee ola
[email p o ec ed]
Lumi Ha aguchi
[email p o ec ed]
Kaisa K a
[email p o ec ed]
Silke Van den Wyngae
silke. [email p o ec ed]
Jonna Kangas
[email p o ec ed]
Conny Sjöq is
conny[email p o ec ed]
Ka in Madsen
[email p o ec ed]
Lasse Lensu
[email p o ec ed]
Heikki Käl iäinen
[email p o ec ed]
Ka el Ho ak
[email p o ec ed]
1Compu e Vision and Pa e n Recogni ion Labo a o y,
Depa men o Compu a ional Enginee ing,
Lappeen an a-Lah i Uni e si y o Technology LUT,
Yliopis onka u 34, 53850 Lappeen an a, Finland
2Depa men o Con ol and Ins umen a ion, Facul y o
Elec ical Enginee ing and Communica ion, B no Uni e si y
o Technology, Technická 3058/10, 616 00 B no, Czech
Republic
3Ma ine Ecology Measu emen s, Finnish En i onmen
Ins i u e, Agnes Sjöbe gin Ka u 2, 00790 Helsinki, Finland
4En i onmen al and Ma ine Biology, Åbo Akademi
Uni e si y, Hen ikinka u 2, 20014, Tu ku, Finland
5En i onmen al and Ma ine Biology, Åbo Akademi
Uni e si y, Hen ikinka u 2, 20014 Tu ku, Finland
123
101 Page 2 o 18 S. Bilik e al.
1 In oduc ion
Phy oplank ona ekey playe s in aqua ic sys ems,whe e hey
media e biogeochemical cycles and o m he base o mul iple
ood webs [1]. The dynamics o phy oplank on popula ions
esul om he in e play be ween esou ce a ailabili y and
mo ali y losses [2]. While some loss mechanisms such as
g azing a e well known, he con ibu ion o loss mecha-
nisms like pa asi ism emains poo ly conside ed and la gely
unde s udied in many aqua ic sys ems. Phy oplank on a e
suscep ible o a wide a ie y o pa asi es, such as i uses,
bac e ia, p o is s, and ungi. Such pa asi es can cause mo -
ali y o ce ain phy oplank on species, he eby al e ing he
phy oplank on bloom dynamics and changing he cycling o
ma e and low o ene gy in aqua ic ecosys ems [3–5].
Zoospo ic o nano lagella e pa asi es ha in ec phy o-
plank on comp ise a highly di e se unc ional g oup o
euka yo ic p o is and ungal species [6]. They ha e in com-
mon he p oduc ion o ee-li ing mo ile s ages as hei
in ec i e p opagules, which a ach o a phy oplank on hos
cell and de elop ei he inside (endobio ic) o ou side (epibi-
o ic) he hos cell using hos esou ces o hei g ow h
and ep oduc ion. Due o hei inconspicuous na u e, phy-
oplank on pa asi es a e di icul o iden i y, and objec s ha
a e di icul o iden i y ypically end o be o e looked o
neglec ed.Consequen ly,al hough hep esence and po en ial
impo ance o hese phy oplank on pa asi es a e inc easingly
ecognized, quan i a i e da a o hei occu ence in na u e a e
ex emely sca ce.
An addi ional challenge o s udy o phy oplank on pa -
asi es is he need o cap u e apid in ec ion dynamics on a
ele an empo al and spa ial scale (e.g., days). Ob aining
quan i a i ein o ma ionabou pa asi ein ec ionsusing adi-
ionalme hods is labo -in ensi eand ime-consuming, which
limi s he spa ial and/o empo al co e age o many s udies
in es iga ing phy oplank on–pa asi e in e ac ions [7].
Recen echnological ad ances in imaging ins umen s
ha e made i possible o collec la ge olumes o plank on
image da a o s udy o plank on popula ions, hus opening
new esea ch possibili ies [8]. The possibili y o high-
equency sampling enabled by imaging ins umen s can
po en ially esul in be e unde s anding o phy oplank on
dynamics and hei po en ial in e ac ions wi h pa asi es [7].
Howe e , while me hods o au oma ic ecogni ion o phy-
oplank on classes ha e been widely de eloped, me hods
o au oma ic ecogni ion o phy oplank on pa asi e in ec-
ions emain unde de eloped. The absence o an e ec i e
app oach o pa asi ic in ec ion ecogni ion is likely associ-
a ed wi h challenges ela ed o ob aining su icien olumes
o image da a o plank on pa asi es, which equi es sc een-
ing o huge amoun s o aw image da a. Such asks a e bes
add essed wi h au oma ed solu ions.
Fig. 1 Anomaloussampleo heCen alesplank onspecies:aO iginal,
bencoded space, c econs uc ion, and ddi e ence image
The sca ci y o plank on pa asi e images is a majo chal-
lenge o he de elopmen o deep lea ning-based compu e
ision me hods o pa asi e de ec ion. While objec de ec ion
me hodssuch asFas e R-CNN[9]and YOLO[10]ha ebeen
shown o achie e high accu acy on a ious de ec ion asks,
including pa asi e de ec ion (see, e.g., [11]), hey s uggle
when he amoun o aining da a is limi ed. The e o e, a
mo e p omising app oach is o o mula e pa asi e de ec ion
as an anomaly de ec ion ask. He e, he idea is o ain he
model wi h images o heal hy plank on and de ec images
ha de ia e om he da a on which he models we e ained.
Due o he a ailabili y o la ge amoun s o plank on image
da a wi hou pa asi es o aining and ela i ely small in a-
class a ia ion among heal hy samples, images ha de ia e
no ably om he aining da a can be expec ed o con ain
po en ial pa asi es.
This wo k in es iga es au oma ed image-based phy o-
plank on pa asi e de ec ion. The p oblem is o mula ed as
an anomaly de ec ion p oblem and sol ed using an au oen-
code . The p oposed me hod consis s o a ec o -quan ized
a ia ionalau oencode (VQVAE)[12] ha encodes heinpu
image in o a comp essed la en ep esen a ion and uses he
comp essed ep esen a ion o econs uc he o iginal image.
The a ionale is ha when he au oencode is ained only
on images o heal hy phy oplank on, he au oencode ails o
econs uc he pa asi es, which allows hem o be de ec ed
om he di e ence image (see Fig. 1). The p oposed me hod
u he employs he Ha dNe [13] ea u e ex ac o and Local
Ou lie Fac o [14] o dis inguish be ween heal hy plank on
and plank on wi h pa asi es.
In he expe imen al pa o he wo k, an ex ensi e se o
di e en backbone con olu ional neu al ne wo ks (CNNs),
au oencode a chi ec u es, ea u e ex ac o s, and classi ie s
a e sys ema ically e alua ed on challenging phy oplank on
image da a o ind he bes combina ion and o demons a e
he pe o mance o he p oposed me hod. In addi ion, we
compa e heau oencode -basedanomalyde ec ionme hod o
a Fas e R-CNN-based objec de ec o . The esul s show ha
he p oposed me hod achie es compa able accu acy o he
s a e-o - he-a Fas e R-CNN objec de ec o while equi -
ing no images wi h pa asi es o aining. Consequen ly, he
au oencode -based me hod can be conside ed a p omising
app oach o u iliza ion in plank on image analysis whe e
123
Towa d phy oplank on pa asi e de ec ion using au oencode s Page 3 o 18 101
he collec ion o la ge aining da a o plank on wi h pa a-
si es is in easible.
The main con ibu ions o ou pape a e he de elopmen
o a no el anomaly de ec ion amewo k and i s applica ion
ophy oplank onpa asi ede ec ion.High- equencyimaging
da a coupled o au oma ic p eso ing o po en ially in ec ed
plank onallow o cap u eand quan i y in ec ion dynamics on
ele an empo al and spa ial scales. This is an essen ial s ep
owa d unde s anding he ole o pa asi es in shaping phy-
oplank on communi y dynamics and ecosys em p ocesses.
The p oposed amewo k is gene al and can be applied o
o he anomaly de ec ion ask such as indus ial aul con ol.
2 Rela ed wo k
Anomaly de ec ion is a da a classi ica ion echnique in which
ade ec o models he ep esen a iono sampleswi hinaspec-
i ica ion (OK) and classi ies all samples ha de ia e om
he speci ica ion as anomalous (NOK). This p oblem can be
challenging because o po en ially high di e si y wi hin he
NOK samples, imbalance be ween he numbe o samples
in he OK and NOK g oups, and i egula i y o he NOK
class. A comp ehensi e o e iew desc ibing anomaly de ec-
ion p oblems, echniques, and ca ego iza ion is p esen ed
in [15].
Fi s in oduced o image da a in [16], au oencode (AE)
models a e now widely used in compu e ision. The use o
insu icien gene aliza ion abili y on ou -o - aining da a o
an AE model wi h he aim o de ec ing anomalies in syn he ic
and eal-wo ld da a, in he case s udied, eleme y da a, was
i s demons a ed in [17]. The esul s showed ha such AEs
can be used o de ec p e iously unseen anomalous samples.
The concep was u he enhanced and used on image da a
in, o example, [18] and [19]. A comp ehensi e o e iew o
AE echniques can be ound in [20].
Plank on anomaly de ec ion has been p e iously s udied
in he con ex o open-se ecogni ion, i.e., image classi ica-
ion wi h he p esence o p e iously unseen classes (plank on
species). In [21], he au ho s p esen ed an unsupe ised
app oach o classi y a plank on sample and de ec po en ial
signi ican di e ences (i.e., anomalies) wi h espec o he
de ec ed class. Image ea u es we e ex ac ed using classical
compu e ision me hods u ilizing geome ical, momen -
based, and o he adi ional ea u es.
In [22], a CNN ained on OK samples and a i icial NOK
samples de i ed om he OK da a by common da a aug-
men a ion echniques such as blu ing and noise addi ion
was used as he ea u e ex ac o . An anomaly sco e was
hen compu ed om hese ea u es and used oge he wi h
he ained ea u e ex ac o o dis inguish be ween he OK
plank onsamples and anomalies. In he wo k, ai bubblesand
non-plank on wa e pa icles we e conside ed as anomalies.
In [23], he au ho s used a pa allel ne wo k o cus om
s a is icalclassi ie scalledTailDeTec (TDT) odisco e p e-
iously unseen plank on species. Each o he TDT classi ie s
was ained on one pa icula species, and a sample was con-
side ed as unknown i none o he classi ie s was able o
de ec i . Unknown samples we e collec ed and alida ed by
expe s. Fea u e ex ac ion and he concep i sel we e based
on wo k p esen ed in [21].
In [24], open-se ecogni ion plank on ecogni ion was
add essed using a simila i y lea ning app oach. Me ic lea n-
ing wi h angula ma gin loss was applied o ob ain image
embedding ec o s ha model he simila i y be ween images.
Theanomalies (images om p e iously unseen classes) we e
de ec ed by se ing h eshold alues o he simila i y.
Fas e R-CNN [9] is a popula deep lea ning (DL) algo-
i hm ha has been success ully applied o a ious domains
and asks, including objec de ec ion and anomaly de ec-
ion [25]. Anomaly de ec ion using Fas e R-CNN in ol es
aining he model on abno mal images o lea n he ea u es
o abno mal ins ances. Then, du ing classi ica ion, he model
is used o de ec abno mal samples ha de ia e om he
expec ed ou come. Fo ins ance, in indus ial manu ac u -
ing, abno mal beha io can include machine mal unc ions,
while in medical diagnosis, i can ake he o m o unusual
pa e ns in medical images.
An example o anomaly de ec ion is p esen ed in [25],
whe e an imp o ed Fas e R-CNN was used o de ec de ec s
in s eel pla es. The algo i hm was ained on a da ase o
abno mal egions on s eel pla e images and was able o
accu a ely de ec anomalies such as c acks and holes in es
images. Using a simila app oach, a sub le modi ica ion o
Fas e R-CNN o de ec ion o anomalies in CT images o
lungs was conside ed in [26].
Objec de ec ion me hods ha e also been success ully
used o pa asi e de ec ion. Fo example, in [11], whe e a
YOLO 5 objec de ec o is used o de ec a pa asi ic mi e on
he body o a honey bee. An o e iew o o he objec de ec-
ion echniques and commonly used da ase s can be ound,
o example, in [27].
In plank on esea ch, Fas e R-CNN has been widely
adop ed o segmen a ionandobjec de ec ion.Se e alobjec
de ec ionapp oaches,including Fas e R-CNN, we eu ilized
in [28] o e alua e a syn he ically augmen ed da ase . Simi-
la wo k is p esen ed in [29], whe e a plank on da ase om
a da k ield mic oscope was compiled and hen es ed wi h
a ious objec de ec ion me hods, including YOLO 3 [30],
R-CNN [31], and SSD [32].
123
101 Page 4 o 18 S. Bilik e al.
3 P oposed me hods o phy oplank on
anomaly de ec ion
In his wo k in es iga ing de ec ion o phy oplank on sam-
ples wi h anomalies, we p ima ily employ an unsupe ised
au oencode -based app oach, ollowed by he use o di e -
en ea u e ex ac o s and one-class classi ie s. Supe ised
objec de ec ion based on he Fas e R-CNN [9] is u ilized o
compa e he esul s o ou p oposed me hod wi h a s a e-o -
he-a app oach.
3.1 Au oencode -based app oach
The p oposed me hod o de ec anomalous plank on sam-
ples is cons uc ed on he amewo k a ailable in [33]. This
implemen a ion allows a ious combina ions o di e en AE
a chi ec u es wi hou conside a ion o he con olu ional lay-
e s (i.e., ully connec ed AE, a ia ional AE and o he s)
e med as AE co es, con olu ional laye s a chi ec u es, ea-
u e ex ac o s, and one-class classi ie s o be es ed. In he
app oach used in his pape , we combined i e AE co es, six
con olu ional encode s and decode s, six ea u e ex ac o s,
and ou classi ie s (720 combina ions in o al). The p ocess-
ing pipeline is shown in Fig. 2and desc ibed in mo e de ail
in he sec ions below.
The anomaly de ec ion is based on compa ison be ween
he o iginal da a and he au oencode - econs uc ed da a, ol-
lowed by ea u e ex ac ion and one-class classi ica ion.
3.1.1 Au oencode a chi ec u es and con olu ional laye s
As he i s s ep o anomaly de ec ion, we use AE mod-
els ained only on OK da a o econs uc unknown inpu
samples o bo h OK and NOK classes. On accoun o he
non-op imal gene aliza ion o he AE models and aining
only on he OK class o da a, we hypo hesize ha da a om
he NOK class will be econs uc ed wo se han da a om
he OK class, as desc ibed in [17].
To be e unde s and he e ec o he AE a chi ec u e’s
co e and he complexi y o he con olu ional encoding and
decoding laye s, we decided o build ou implemen a ion
such ha he co e o he model could be combined wi h he
selec edcon olu ionalpai so heencode sand hedecode s.
The p oposed s uc u e allows us o analyze he con ibu-
ions o he selec ed a chi ec u e and con olu ional laye s
sepa a ely.
We e alua ed i e di e en op ions o he AE co es. As
he i s al e na i e, we used implemen a ions o basic con o-
lu ional AE [34] as he BAE1 co e, con olu ional a ia ional
AE [35] as he VAE1 co e, and ec o -quan ized AE [12]as
he VQVAE1 co e. As well as using hese co es, we ied o
u he educe he ea u es ex ac ed by an encode by inse -
ing ully connec ed laye s o he basic con olu ional AE as
heBAE2co e[36] and o he a ia ional AE as he VAE2
co e.Thesemodi ica ions o he au oencode co es a e shown
in Fig. 3.
Fig. 2 P ocessing pipeline o he p oposed au oencode -based anomaly de ec ion me hod
123
Towa d phy oplank on pa asi e de ec ion using au oencode s Page 5 o 18 101
Fig. 3 Schemes o he modi ied
au oencode co es: aBAE1
co e, bVAE2 co e
We expec ha he basic con olu ional AE is going o
be su passed in pe o mance by bo h he a ia ional and
he ec o -quan ized co es because o hei non-p obabilis ic
encoding space, which allows leak o mo e inpu image’s
anomalous pa s o he encoded space and econs uc ed
image. The quali y o he econs uc ed images should be
be e in he case o he basic and ec o -quan ized co es
han wi h a ia ional co es, which ypically p oduce blu y
ou pu s [37]. When aining on di e en classes, he bes
esul s a e expec ed om he ec o -quan ized co e, which
should c ea e sepa able clus e s o each class in he encoded
space.
Besides he AE co es desc ibed abo e, we also conside
six pai s o con olu ional encoding and decoding laye a chi-
ec u es, whose s uc u e is desc ibed in he complemen a y
ables: Table 5 o encode s, and Table 6 o decode s. Each
con olu ional laye o block desc ibed in hese ables is com-
plemen ed wi h he ba ch no maliza ion laye . The ac i a ion
unc ion was se as Leaky ReLu by he Con M1 a chi ec u e
and as ReLu o he o he a chi ec u es.
The es ed con olu ional laye s go om he mo e com-
plex Con M2 a chi ec u e, sugges ed o anomaly de ec ion
in [19], and Con M1 a chi ec u e, whe e we expec he
abili y o econs uc ine ea u es and de ails, o he sim-
ple a chi ec u es Con M5, Con M4 and Con M3. By using
he simple a chi ec u es, we expec ha ine ea u es and
smalle image s uc u es will be supp essed and he a chi-
ec u e migh hus pe o m be e on shape o s uc u e
anomalies. The las a chi ec u e, Con M6, is unsymme -
ical, as sugges ed in [38], and uses he mo e complex
encode o he Con M5 a chi ec u e and he simple decode
o he Con M4 a chi ec u e. Using his a chi ec u e, we
expec ha anomalies ha a e p opaga ed o he encoded
space will be u he supp essed by he decode econs uc-
ion.
In he op imal case, anomalous a eas o he o iginal image
a e emo ed du ing he image econs uc ion as shown in
Fig. 11. A di e ence image be ween he o iginal sample and
he econs uc ed sample is hen compu ed and used in he
ea u e ex ac ion.
3.1.2 Fea u e ex ac ion
The second s ep o he amewo k applies ea u e ex ac-
o s o analyze he econs uc ions. The ea u es a e based
on compa ison be ween he o iginal and econs uc ed da a
(E o me ics, Ha dNe 3 and Ha dNe 4) o analysis o he
di e ence image (SIFT ea u e ex ac ion, Ha dNe 1 and
Ha dNe 2).
The i s ea u e ex ac ion app oach (E o me ics) c e-
a es a low-dimensional ea u e ec o o each image by
compu ing selec ed e o me ics be ween he o iginal and
econs uc ed images. The L2 and SSIM me ics applied
in [36] a e complemen ed wi h he A e age hash and mean-
squa ed e o me ics.
The second ea u e ex ac ion me hod (SIFT ea u e
ex ac ion) uses scale and me ics p ope ies o he image
keypoin s ound by he SIFT me hod. I is a di ec e-
implemen a ion he app oach p esen ed in [39]. The me hod
uses di e ence images be ween he o iginal and econ-
s uc ed da a.
Thelas ou ea u e ex ac ionme hods(Ha dNe 1, Ha d-
Ne 2, Ha dNe 3 and Ha dNe 4) a e all based on he ba ch
simila i y me ic p esen ed in [13]. Ha dNe 1 is he sim-
ples me hod whe e each sample is desc ibed by he Ha dNe
(HN) ea u e ec o o he o iginal image esized o he size
o 32 ×32 as equi ed by he o iginal HN implemen a ion.
Since such esizing migh no be op imal o small anoma-
lies, Ha dNe 2 spli s he image o he o iginal size o blocks
o 32×32 and compu es he HN ea u e ec o o each such
block. The esul ing ea u e ec o consis s o he no ms o e
hose ec o s. Ha dNe 3 spli s he o iginal and econs uc ed
images o 32×32 blocks as in he Ha dNe 2 me hod, bu he
esul ing ea u e ec o is compu ed as a cosine simila i y
be ween he HN ea u e ec o s o he co esponding blocks
o he o iginal and decoded images. Ha dNe 4 uses he same
echnique, bu he cosine simila i y is supplemen ed by he
loga i hm, which is supposed o emphasize smalle di e -
ences o he Ha dNe 3 ea u e ec o .
A 2D isualiza ion o he esul ing ea u e space ob ained
by he Con M5-BAE2 au oencode o e he Aphanizomenon
123

101 Page 6 o 18 S. Bilik e al.
Fig. 4 Example ea u e space o
he Aphanizomenon plank on
species
plank on species using he Ha dNe 2 ea u e ex ac o is
shown in Fig. 4. The OK samples o m an ellip ical clus e ,
and mos o he NOK samples a e sepa a e om ha clus e .
3.1.3 One-class classi ica ion
Fo he classi ica ion pa , we used he ollowing one-class
classi ie s:
•Robus co a iance (RC) [40]: The RC classi ie assumes
he same dis ibu ion o all OK samples and i s an ellip-
ic en elope o he cen al da a poin . The anomaly sco e
is compu ed using he dis ibu ion es ima ions and Maha-
lanobis dis ance.
•One-class SVM (OC-SVM) [41]: The OC-SVM clas-
si ie u ilizes he suppo ec o machine (SVM) and a
nonlinea ke nel o c ea e a sepa a ing hype plane o he
aining da a om he o igin o he ea u e space. Sam-
ples on he o he side o his hype plane a e conside ed
as anomalies.
•Isola ion Fo es (IF) [42]: The IF classi ie uses andom
ea u e selec ion and spli ing o isola e obse ed sam-
ples. The anomaly sco e is based on he o al numbe o
spli s. Anomalies a e supposed o ha e a smalle numbe
o spli s as i should be easie o sepa a e hem.
•Local Ou lie Fac o (LOF) [14]: The LOF classi ie is
based on he local densi y de ia ion o he obse ed poin
wi h espec o i s k-nea es neighbo s. The densi y o he
anomalies should be lowe in compa ison wi h he OK
samples, which a e conside ed o c ea e dense clus e s.
The ac ion o anomaly samples o he OC-SVM, IF and
LOF was se o 1% since his alue is he minimum alue o
common implemen a ions. Based on he no mal dis ibu ion,
we should also assume ha e en some OK samples migh
sligh ly di e om he majo i y. All classi ie s a e i on he
da ase con aining only OK samples.
Inpu ea u es o he one-class classi ica ion a e no -
malized using obus scaling, which no malizes he median
Fig. 5 Illus a ion o equal-e o - a e (EER) h eshold selec ion c i e-
ion on he ROC cu e
and he in e qua ile ange, as sugges ed in [43]. This no -
maliza ion should be mo e obus o ou lie s han simple
no maliza ion app oaches such as min–max no maliza ion
o s anda diza ion.
To selec he op imal decision h eshold o anomaly
de ec ion, we use he equal e o a e (EER) o e he ROC
cu e o he classi ie as shown in Fig. 5. All classi ie s a e
i only on he OK da a, and he ROC cu e is ob ained om
he es da ase .
3.2 Objec de ec ion-based app oach
The Fas e R-CNN [9] algo i hm is composed o h ee main
componen s: a base ea u e ex ac o ne wo k, a egion p o-
posal ne wo k (RPN) o ex ac ing he egions o in e es ,
and a de ec o ha uses he egion p oposals and espec i e
ea u e maps o classi y he de ec ed objec s as shown in
Fig. 6. The i s componen is he ea u e ex ac o esponsi-
ble o gene a ing ea u e maps om he inpu image. This
module is usually a CNN such as VGG-16 [44]o ResNe -
50 [45].
123
Towa d phy oplank on pa asi e de ec ion using au oencode s Page 7 o 18 101
Fig. 6 Fas e R-CNN a chi ec u e
Fig. 7 The Fas e R-CNN
app oach o anomaly de ec ion
Fig. 8 Objec de ec ion asks
using he Fas e R-CNN
app oach: aPlank on e sus
Anomalies; bPlank on e sus
Anomalous Plank on; c
Anomalies only. The NOK
samples a e shown in he op
ow and he OK samples in he
bo om ow
The RPN is a kind o ully con olu ional ne wo k ha
akes he ea u e maps om he p e ious s ep and e u ns a
se o egionp oposals ha guide hede ec o onwhe e o ind
he objec s in he image. The p oposals and co esponding
ea u e maps om he CNN a e hen u ilized o yield can-
dida e objec s wi h bounding boxes and ixed-leng h ea u e
ec o susing he ROIpoolinglaye .Finally, hese ou pu s a e
passed o he R-CNN ne wo k. The R-CNN ne wo k uses he
p oposed ea u e maps o classi y each bounding box as an
objec o backg ound and p edic inal class sco es wi h he
bounding boxes.
Fo ou objec de ec ion expe imen s, we used he Fas e
R-CNN implemen a ion a ailable om [46] based on he
ResNe -50 backbone p esen ed in [47]. To employ an
anomaly de ec ion ask in he Fas e R-CNN baseline, he
a chi ec u e is supplemen ed by a one-class classi ica ion
module based on he p edic ed objec labels, as shown in
Fig. 7.
Sinceanomaliessuchaspa asi esa e ela i elysmallcom-
pa ed o he image size, i is impo an o conside he ancho
gene a o which is a pa o he egion-p oposal ne wo k.
Ancho s de ine egions o an image, usually o di e en
aspec a ios and sizes, ha a e used as e e ences o de ec
objec s. The ancho gene a o c ea es a se o ancho s o
each loca ion in a ea u e map; hen, o each egion o in e -
es , he model p edic s which ancho box bes encloses he
objec . The choice o an ancho gene a o mos ly depends on
he ype o de ec ion ask. Fo example, i we wan o de ec
small objec s, hen a smalle ancho size should be used. On
he o he hand, i he ask is o de ec objec s o a ious sizes,
123
101 Page 8 o 18 S. Bilik e al.
a ange o ancho sizes should be de ined [9]. Addi ionally,
he aspec a ios o he ancho s should ma ch he aspec a ios
o he objec s in he image.
As sugges ed in [11], h ee sepa a e objec de ec o s a e
conside ed, each ained on di e en g ound u h: (1) plank-
on and anomalies, (2) plank on (clean) and anomalous
plank on, and (3) anomalies only (see Fig. 8). In he i s
column, we can see ha he model de ec s a plank on sample
in bo h cases and an anomaly in he op ow. The second col-
umn shows de ec ion o a plank on sample wi h anomaly in
he op owand de ec iono a cleansample in hebo om ow,
and inally, he hi d column shows de ec ion o an anomaly
in he op ow only.
4 Expe imen s
In his sec ion, we desc ibe he da ase s used, he e alua ion
me ics,and he esul so heau oencode -basedexpe imen s
and he objec de ec ion-based expe imen s.
Table 1 Species-speci ic s a is ics o he plank on anomaly da ase
Plank on class OK samples coun NOK samples coun
Aphanizomenon 830 140
Cen ales 400 57
Dolichospe mum 515 406
Chae oce os 606 371
Nodula ia 118 357
Pauliella 160 433
Pe idiniella Chain 183 31
Pe idiniella Single 459 63
Skele onema 769 419
4.1 Phy oplank on anomaly da ase
Na u al Bal ic Sea phy oplank on communi ies a e con in-
uously imaged wi h an Imaging FlowCy obo (IFCB) [48]
deployed a U ö A mosphe ic and Ma ine Resea ch S a ion,
Finland (59◦46.84’ N, 21◦22.13’ E). The IFCB is connec ed
o he s a ion low- h ough sys em, which ecei es wa e
pumped oman ∼5mdeepinle loca ed 250mo sho e, ep-
esen a i eo hesub-su acelaye .A U ö,IFCB akesa5-ml
sample nea ly e e y 20min and he sys em is se o igge
based on he de ec ion o chlo ophyll, i.e., a ge ing phy o-
plank on cells a he han non-li ing pa icles. The esea ch
s a ion and IFCB deploymen a U ö a e desc ibed in de ail
in [49] and [50].
The phy oplank on da a om U ö IFCB can be cu en ly
classi iednea eal- imein o50di e en classes,asdesc ibed
by [51]. Pu a i e pa asi e in ec ion images we e manually
anno a ed by expe s based on o he U ö da a, collec ed
be ween Feb ua y and Augus 2021, using phy oplank on
da a om nine classes. These classes we e selec ed based on
hei impo ance du ing he sp ing o summe blooms in he
Bal ic Sea.
In ou expe imen s, we used a phy oplank on anomaly
da ase de i ed om he anno a ed images used o ain he
classi ie desc ibed abo e wi h OK samples om he da ase
published in [51] and NOK samples om unpublished 2021
U ö da a. The anomaly da ase con ains o e 6200 manu-
ally anno a ed and expe - alida ed samples o 9 plank on
classes wi h known anomalies, as shown in Table 1. Non-
anomalous and anomalous samples o each class a e shown
in Fig. 9. As an anno a ion ool, we used he ee e sion o
he Label S udio a ailable a [52]. The anno a ed da ase is
a ailable online a [53] in bo h COCO and YOLO o ma s.
Fig. 9 Anomalous (le column,
o uppe ow) and
non-anomalous samples ( igh
column, o lowe ow) om all
da ase classes o he used
da ase
123
Towa d phy oplank on pa asi e de ec ion using au oencode s Page 9 o 18 101
Fig. 10 Example o he anno a ion bounding boxes
4.1.1 Da ase anno a ions
When anno a ing he da ase , we used h ee di e en labels
o de ine a sepa a e species se :
•The label Anomaly ma ks he pa asi e o o he anomalies
on he plank on sample.
•The label Plank onSpecies_Anomaly ma ks plank on
species wi h he a ached pa asi e.
•ThelabelPlank onSpecies_Clean ma ksplank onspecies
wi h o wi hou he pa asi e.
The las wo labels could o e lap, bu whene e i was
possible, he Plank onSpecies_Clean label does no co e he
sample pa wi h pa asi e. To dis inguish be ween he OK and
NOK samples, he Plank onSpecies_Clean label should be
emo ed i i o e laps wi h he Plank onSpecies_Anomaly
one.
An example o he anno a ion o e a Dolichospe mum
plank on species sample is shown in Fig. 10. The ed colo
ma ks a plank on anomaly and, in his case, he da ke g een
ma ks he clean sample and he ligh e g een ma ks he sam-
ple wi h an anomaly.
4.1.2 De i ed da ase o au oencode -based expe imen
Fo he pu poses o he au oencode -based expe imen , we
used he abo e-desc ibed da ase o de i e a one-class da ase
wi hnoNOK samples and 70%o he OK sample in he ain-
ingse .Tes and alida ionda ase salwayscon ainabalanced
numbe o OK and NOK samples. The expe imen wi h all
plank on species con ains all a ailable aining samples, 10
alida ion samples, and 10 es samples om each species.
In o de o help he AE model o lea n mo e obus ea-
u es, we added sal -and-peppe noise o he image samples
used du ing he aining wi h a clean sample used as a label as
sugges ed in [54]. Besides his noise augmen a ion, we also
use andom lipping, con as , sa u a ion, b igh ness, in e -
sion, and hue augmen a ion.
Because he Ha dNe -based ea u e ex ac o s wo k co -
ec ly only wi h image sizes o mul iples o 32, all samples
we e esized wi h espec o he majo aspec a io o each
class (1:4 o i e classes, 1:1 o h ee classes and 1:2 o
one class) as can be seen in Fig. 9. Fo he expe imen o e
all classes, we chose he aspec a io o 1:2 as a comp omise.
4.1.3 De i ed da ase o objec de ec ion-based
expe imen
Fo he objec de ec ion expe imen , he model was ained
in a supe ised manne . The spli a ios we e se as 70%,
10%, and 20% o he aining, alida ion, and es subse s,
espec i ely.T ainingand alida ionse s dono includeclean
samples, whe eas a es se con ains a balanced numbe o
anomalies and clean images.
Addi ionally,weapplied he ollowingaugmen a ion ech-
niques: ho izon al and e ical lip wi h a p obabili y o 30%,
and andom b igh ness, con as and sa u a ion adjus men
wi h a p obabili y o 10%.
4.2 Pe o mance me ics
To compa e he esul s o he au oencode and objec de ec-
ion expe imen s, we need o e alua e he p edic ions o he
models wi h espec o he g ound- u h labels. To do so,
we can de ine ue-posi i e (TP) and ue-nega i e (TN) p e-
dic ions, whe e he model co ec ly classi ies OK and NOK
samples, oge he wi h alse-posi i e (FP) and alse-nega i e
(FN) p edic ions, whe e he model misclassi ies NOK sam-
ples as OK in he FP case and OK samples as NOK in he
FN case.
P ecision, Recall and F1 sco e me ics a e used o com-
pa ison o he di e en a ia ions o au oencode s and objec
de ec ion me hods. The me ics a e de ined as ollows:
P ecision =TP
TP +FP (1)
Recall =TP
TP +FN (2)
F1=2∗P ecision ∗Recall
P ecision +Recall (3)
Simila ly o p ecision and ecall, we can also de ine speci-
ici y as:
Speci ici y =TN
TN +FP (4)
In he au oencode expe imen , we complemen ed he
me ics wi h he a ea unde he cu e (AUC) sco e. This
pa ame e is de ined as he a ea unde he ecei e ope a o
cha ac e is ics (ROC) cu e, an example o which is shown
in Fig. 5. This cu e is ob ained by changing he decision
h eshold o a bina y classi ie by a de ined s ep and plo ing
he esul ing speci ici y on he x-axis and ecall on he y-axis
o each h eshold s ep. Each poin o he ROC cu e hen
co esponds o one h eshold se ing.
123
101 Page 16 o 18 S. Bilik e al.
Appendix C: Complemen a y R-CNN esul s
See Tables 13,14 and 15.
Table 13 Fas e -RCNN de ec ion esul s o Plank on s Anomalies
expe imen
Plank on class F1 sco e P ec Rec
Aphanizomenon 0.98 1 0.96
Cen ales 0.85 0.73 1
Dolichospe mum 0.89 0.97 0.81
Chae oce os 0.80 0.89 0.73
Nodula ia 0.76 0.98 0.62
Pauliella 0.86 0.99 0.77
Pe idiniella Chain 0.60 0.75 0.5
Pe idiniella Single 0.76 1 0.62
Skele onema 0.98 0.97 1
Plank on & Anomalies 0.87 0.95 0.81
Table 14 Fas e -RCNN de ec ion esul s o Plank on s Anomalous
Plank on expe imen
Plank on class F1 sco e P ec Rec
Aphanizomenon 0.66 0.58 0.75
Cen ales 0.60 0.47 0.82
Dolichospe mum 0.85 0.82 0.88
Chae oce os 0.74 0.71 0.78
Nodula ia 0.78 0.96 0.66
Pauliella 0.79 0.76 0.83
Pe idiniella Chain 0.67 0.67 0.67
Pe idiniella Single 0.88 0.92 0.85
Skele onema 0.94 0.92 0.94
Plank on & Anomalous plank on 0.83 0.82 0.85
Table 15 Fas e -RCNN de ec ion esul s o Anomalies expe imen
Plank on class F1 sco e P ec Rec
Aphanizomenon 1 1 1
Cen ales 0.77 0.67 0.91
Dolichospe mum 0.38 0.87 0.25
Chae oce os 0.67 0.66 0.68
Nodula ia 0.85 0.88 0.82
Pauliella 0.85 0.78 0.94
Pe idiniella Chain 0.44 0.67 0.33
Pe idiniella Single 0.89 0.86 0.92
Skele onema 0.83 0.75 0.93
Anomalies 0.75 0.76 0.74
Re e ences
1. Falkowski, P.G., Ba be , R.T., Sme acek, V.: Biogeochemical
con ols and eedbacks on ocean p ima y p oduc ion. Science
281(5374), 200–206 (1998). h ps://doi.o g/10.1126/science.281.
5374.200
2. Reynolds, C.S.: The Ecology o Phy oplank on, Ecology, Biodi-
e si y and Conse a ion. Camb idge Uni e si y P ess, Camb idge
(2006). h ps://doi.o g/10.1017/CBO9780511542145
3. Su le, C.A., Chan, A.M., Co ell, M.T.: In ec ion o phy oplank-
on by i uses and educ ion o p ima y p oduc i i y. Na u e
347(6292), 467–469 (1990). h ps://doi.o g/10.1038/347467a0
4. Klawonn, I., Van den Wyngae , S., Pa ada, A.E., e al.: Cha ac e -
izing he “ ungal shun ”: Pa asi ic ungi on dia oms a ec ca bon
low and bac e ial communi ies in aqua ic mic obial ood webs.
P oc. Na l. Acad. Sci. 118(23), e2102225,118 (2021). h ps://doi.
o g/10.1073/pnas.2102225118
5. Klawonn, I., Van den Wyngae , S., I e sen, M.H., e al.: Fungal
pa asi ism on dia oms al e s o ma ion and bio-physical p ope ies
o sinking agg ega es. Commun. Biol. 6(1), 206 (2023). h ps://doi.
o g/10.1038/s42003-023-04453-6
6. Scholz, B., Guillou, L., Ma ano, A.V., e al.: Zoospo ic pa asi es
in ec ing ma ine dia oms-a black box ha needs o be opened. Fun-
gal Ecol. 19, 59–76 (2016). h ps://doi.o g/10.1016/j. uneco.2015.
09.002
7. Peacock, E.E., Olson, R.J., Sosik, H.M.: Pa asi ic in ec ion o he
dia om guina dia delica ula, a ecu en and ecologically impo an
phenomenon on he new england shel . Ma . Ecol. P og. Se . 503,
1–10 (2014). h ps://doi.o g/10.3354/meps10784
8. Lomba d,F.,Boss,E.,Wai e,A.M.,e al.:Globallyconsis en quan-
i a i e obse a ions o plank onic ecosys ems. F on . Ma . Sci. 6,
196 (2019). h ps://doi.o g/10.3389/ ma s.2019.00196
9. Ren, S., He, K., Gi shick, R., e al.: Fas e -cnn: Towa ds eal- ime
objec de ec ionwi h egionp oposal ne wo ks.Ad ances in neu al
in o ma ion p ocessing sys ems 28 (2015)
10. Joche , G.: YOLO 5 by Ul aly ics. h ps://doi.o g/10.5281/
zenodo.3908559,h ps://gi hub.com/ul aly ics/yolo 5 (2020)
11. Bilik, S., K a och ila, L., Ligocki, A., e al.: Visual diagnosis o he
a oa des uc o pa asi ic mi e in honeybees using objec de ec o
echniques. Senso s 21(8), 2764 (2021). h ps://doi.o g/10.3390/
s21082764
12. Van Den Oo d, A., Vinyals, O., e al.: Neu al disc e e ep esen a-
ion lea ning. In: Guyon, I., Luxbu g, U.V., Bengio, S., e al. (eds.)
Ad ances in Neu al In o ma ion P ocessing Sys ems, ol. 30. Cu -
an Associa es Inc, New Yo k (2017)
13. Mishchuk, A., Mishkin, D., Radeno ic, F., e al.: Wo king ha d
o know you neighbo ’s ma gins: Local desc ip o lea ning loss.
In: Guyon, I., Luxbu g, U.V., Bengio, S., e al. (eds.) Ad ances in
Neu alIn o ma ionP ocessingSys ems, ol.30.Cu anAssocia es
Inc, New Yo k (2017)
14. B eunig, M.M., K iegel, H.P., Ng, R.T., e al.: Lo : Iden i y-
ing densi y-based local ou lie s. In: P oceedings o he 2000
ACM SIGMOD In e na ional Con e ence on Managemen o Da a.
Associa ion o Compu ing Machine y, New Yo k, NY, USA,
SIGMOD’00, pp. 93–104, (2000) h ps://doi.o g/10.1145/342009.
335388
15. Pang, G., Shen, C., Cao, L., e al.: Deep lea ning o anomaly de ec-
ion: a e iew. ACM Compu . Su . 54(2), 1–38 (2021). h ps://doi.
o g/10.1145/3439950
16. Hin on, G.E., Salakhu dino , R.R.: Reducing he dimensionali y
o da a wi h neu al ne wo ks. Science 313(5786), 504–507 (2006).
h ps://doi.o g/10.1126/science.1127647
17. Saku ada, M., Yai i, T.: Anomaly de ec ion using au oencode s
wi h nonlinea dimensionali y educ ion. In: P oceedings o he
MLSDA 2014 2nd Wo kshop on Machine Lea ning o Senso y
123

Towa d phy oplank on pa asi e de ec ion using au oencode s Page 17 o 18 101
Da a Analysis. Associa ion o Compu ing Machine y, New Yo k,
NY, USA, MLSDA’14, pp. 4–11, (2014) h ps://doi.o g/10.1145/
2689746.2689747
18. An, J., Cho, S.: Va ia ional au oencode based anomaly de ec ion
using econs uc ion p obabili y. Spec. Lec . IE 2(1), 1–18 (2015)
19. Be gmann, P., Löwe, S., Fause , M., e al.: Imp o ing unsupe ised
de ec segmen a ion by applying s uc u al simila i y o au oen-
code s. In: P oceedings o he 14 h In e na ional Join Con e ence
on Compu e Vision, Imaging and Compu e G aphics Theo y and
Applica ions - Volume 5: VISAPP, INSTICC. SciTeP ess, pp. 372–
380 (2019). h ps://doi.o g/10.5220/0007364503720380
20. Cha e, D., Cha e, F., Ga cía, S., e al.: A p ac ical u o ial on
au oencode s o nonlinea ea u e usion: axonomy, models, so -
wa e and guidelines. In . Fusion 44, 78–96 (2018). h ps://doi.o g/
10.1016/j.in us.2017.12.007
21. Pas o e, V.P., Zimme man, T.G., Biswas, S.K., e al.: Anno a ion-
ee lea ning o plank on o classi ica ion and anomaly de ec ion.
Sci. Rep. 10(1), 12,142 (2020). h ps://doi.o g/10.1038/s41598-
020-68662-3
22. Pu, Y., Feng, Z., Wang, Z., e al.: Anomaly de ec ion o in si u
ma ine plank on images. In: P oceedings o he IEEE/CVF In e -
na ional Con e ence on Compu e Vision (ICCV) Wo kshops, pp.
3661–3671 (2021)
23. Pas o e, V.P., Megiddo, N., Bianco, S.: An anomaly de ec ion
app oach o plank on species disco e y. In: Scla o S, Dis an e C,
Leo M, e al (eds) Image Analysis and P ocessing – ICIAP 2022.
Sp inge In e na ional Publishing, Cham, pp. 599–609 (2022)
h ps://doi.o g/10.1007/978-3-031-06430-2_50
24. Bad eldeenBdawyMohamed,O.,Ee ola,T.,K a ,K.,e al.:Open-
se plank on ecogni ion using simila i y lea ning. In: Bebis G, Li
B, Yao A, e al (eds) Ad ances in Visual Compu ing. Sp inge
In e na ional Publishing, Cham, pp. 174–183 (2022). h ps://doi.
o g/10.1007/978-3-031-20713-6_13
25. Zhao, W., Chen, F., Huang, H., e al.: A new s eel de ec de ec ion
algo i hm based on deep lea ning. Compu . In ell. Neu osci. 2021,
1–13 (2021). h ps://doi.o g/10.1155/2021/5592878
26. Su, Y., Li, D., Chen, X.: Lung nodule de ec ion based on as e
-cnn amewo k. Compu . Me hods P og ams Biomed. 200(105),
866 (2021). h ps://doi.o g/10.1016/j.cmpb.2020.105866
27. Ho ak, K., Sabla nig, R.: Deep lea ning concep s and da ase s
o image ecogni ion: o e iew 2019. In: Hwang JN, Jiang X
(eds) Ele en h In e na ional Con e ence on Digi al Image P ocess-
ing (ICDIP 2019), In e na ional Socie y o Op ics and Pho onics,
ol 11179. SPIE, p. 111791S (2019). h ps://doi.o g/10.1117/12.
2539806
28. Li, Q., Sun, X., Dong, J., e al.: De eloping a mic oscopic image
da ase in suppo o in elligen phy oplank on de ec ion using deep
lea ning. ICES J. Ma . Sci. 77(4), 1427–1439 (2019). h ps://doi.
o g/10.1093/icesjms/ sz171
29. Chen, T., Li, J., Ju, W., e al.: Objec de ec ion and abundance anal-
ysis o oun ain- low imaging o ma ine plank on. In: OCEANS
2021: San Diego - Po o, pp. 1–9 (2021). h ps://doi.o g/10.23919/
OCEANS44145.2021.9705862
30. Redmon, J., Fa hadi, A.: Yolo 3: An inc emen al imp o emen .
(2018) a Xi p ep in a Xi :1804.02767
31. Gi shick, R.: Fas -cnn. In: P oceedings o he IEEE In e na ional
Con e ence on Compu e Vision (ICCV), pp. 1440–1448 (2015)
32. Liu, W., Anguelo , D., E han, D., e al.: Ssd: Single sho mul ibox
de ec o . In: Leibe B, Ma as J, Sebe N, e al (eds) Compu e Vision
– ECCV 2016. Sp inge In e na ional Publishing, Cham, pp. 21–37
(2016) h ps://doi.o g/10.1007/978-3-319-46448-0_2
33. Bilik, S.: Ae- econs uc ion-and- ea u e-based-ad. h ps://gi hub.
com/boo el/AE-Recons uc ion-And-Fea u e-Based-AD, open
sou ce so wa e a ailable om h ps://gi hub.com/boo el/AE-
Recons uc ion-And-Fea u e-Based-AD (2022)
34. Masci, J., Meie , U., Ci e¸san, D., e al.: S acked con olu ional
au o-encode s o hie a chical ea u e ex ac ion. In: Honkela T,
Duch W, Gi olami M, e al (eds) A i icial Neu al Ne wo ks and
Machine Lea ning – ICANN 2011. Sp inge Be lin Heidelbe g,
Be lin, Heidelbe g, pp 52–59, (2011) h ps://doi.o g/10.1007/978-
3-642-21735-7_7
35. Pu, Y., Gan, Z., Henao, R., e al.: Va ia ional au oencode o deep
lea ning o images, labels and cap ions. In: Lee, D., Sugiyama, M.,
Luxbu g, U., e al. (eds.) Ad ances in Neu al In o ma ion P ocess-
ing Sys ems, ol. 29. Cu an Associa es Inc, New Yo k (2016)
36. Bilik, S.: Fea u e space educ ion as da a p ep ocessing o he
anomaly de ec ion. In: P oceedings I o he 27 h Con e ence
STUDENT EEICT 2021, pp. 415–419. Facul y o Elec ical Engi-
nee ing and Communica ion, B no Uni e si y o Technology, B no
(2021)
37. Hou,X.,Shen,L.,Sun,K.,e al.:Deep ea u econsis en a ia ional
au oencode . In: 2017 IEEE Win e Con e ence on Applica ions o
Compu e Vision (WACV), pp 1133–1141, (2017) h ps://doi.o g/
10.1109/WACV.2017.131
38. Makhzani, A., F ey, B.J.: Winne - ake-all au oencode s. In: Co es,
C., Law ence, N., Lee, D., e al. (eds.) Ad ances in Neu al In o -
ma ion P ocessing Sys ems, ol. 28. Cu an Associa es Inc, New
Yo k (2015)
39. Bilik, S., Ho ak, K.: Si and su based ea u e ex ac ion o
he anomaly de ec ion. In: P oceedings I o he 28 h Con e ence
STUDENT EEICT 2022 Gene al Pape s, pp. 459–464. Facul y o
Elec ical Enginee ing and Communica ion, B no Uni e si y o
Technology, B no (2022)
40. Rousseeuw, P.J., D iessen, K.V.: A as algo i hm o he minimum
co a iance de e minan es ima o . Technome ics 41(3), 212–223
(1999). h ps://doi.o g/10.1080/00401706.1999.10485670
41. Schölkop , B., Pla , J.C., Shawe-Taylo , J., e al.: Es ima -
ing he Suppo o a High-Dimensional Dis ibu ion. Neu-
al Compu . 13(7), 1443–1471 (2001). h ps://doi.o g/10.1162/
089976601750264965
42. Liu, F.T., Ting, K.M., Zhou, Z.H.: Isola ion o es . In: 2008 Eigh h
IEEE In e na ional Con e ence on Da a Mining, pp. 413–422
(2008). h ps://doi.o g/10.1109/ICDM.2008.17
43. Iglewicz, B.: Robus scale es ima o s and con idence in e als o
loca ion.In:Unde s andingRobus and Explo a o y Da a Analysis,
1s edn., pp. 405–431. Wiley-In e science, New Yo k (2000)
44. Simonyan, K., Zisse man, A.: Ve y deep con olu ional ne -
wo ks o la ge-scale image ecogni ion. (2014) a Xi p ep in
a Xi :1409.1556 h ps://doi.o g/10.48550/a Xi .1409.1556
45. He, K., Zhang, X., Ren, S., e al.: Deep esidual lea ning o image
ecogni ion. In: P oceedings o he IEEE Con e ence on Compu e
Vision and Pa e n Recogni ion (CVPR), pp 770–778 (2016)
46. Li, Y., Xie, S., Chen, X., e al.: Fas e R-CNN (ResNe 50). (2021a)
h ps://py o ch.o g/ ision/main/models/gene a ed/ o ch ision.
models.de ec ion. as e cnn_ esne 50_ pn_ 2.h ml# o ch ision.
models.de ec ion. as e cnn_ esne 50_ pn_ 2
47. Li, Y., Xie, S., Chen, X., e al.: Benchma king de ec ion ans-
e lea ning wi h ision ans o me s (2021) a Xi p ep in
a Xi :2111.11429 h ps://doi.o g/10.48550/a Xi .2111.11429
48. Olson, R.J., Sosik, H.M.: A subme sible imaging-in- low ins u-
men o analyze nano-and mic oplank on: Imaging lowcy obo .
Limnol. Oceanog . Me hods 5(6), 195–203 (2007). h ps://doi.o g/
10.4319/lom.2007.5.195
49. Laakso, L., Mikkonen, S., D ebs, A., e al.: 100 yea s o a mo-
sphe ic and ma ine obse a ions a he innish u ö island in he
bal ic sea. Ocean Sci. 14(4), 617–632 (2018). h ps://doi.o g/10.
5194/os-14-617-2018
50. K a , K., Seppälä, J., Häll o s, H., e al.: Fi s applica ion o i cb
high- equency imaging-in- low cy ome y o in es iga e bloom-
o ming ilamen ous cyanobac e ia in he bal ic sea. F on ie s
123
101 Page 18 o 18 S. Bilik e al.
in Ma ine Science 8, (2021). h ps://doi.o g/10.3389/ ma s.2021.
594144
51. K a ,K.,Velhonoja,O.,Ee ola, T.,e al.: Towa ds ope a ionalphy-
oplank on ecogni ion wi h au oma ed high- h oughpu imaging,
nea - eal- ime da a p ocessing, and con olu ional neu al ne wo ks.
F on ie s in Ma ine Science 9,(2022). h ps://doi.o g/10.3389/
ma s.2022.867695
52. Tkachenko, M., Malyuk, M., Holmanyuk, A., e al.: Label
S udio: Da a labeling so wa e. (2020-2022) h ps://gi hub.com/
hea exlabs/label-s udio, open sou ce so wa e a ailable om
h ps://gi hub.com/hea exlabs/label-s udio
53. Bilik, S., Bak akhano , D., Ee ola, T., e al.: I cb phy oplank-
on anomaly da ase (i cb-pad). (2023) h ps://doi.o g/10.23729/
08b2ac4a-a80d-4e54-85e0-ab3ea46085ec
54. Vincen , P., La ochelle, H., Bengio, Y., e al.: Ex ac ing and
composing obus ea u es wi h denoising au oencode s. In: P o-
ceedings o he 25 h In e na ional Con e ence on Machine Lea n-
ing. Associa ion o Compu ing Machine y, New Yo k, NY,
USA, ICML’08, pp. 1096–1103, (2008) h ps://doi.o g/10.1145/
1390156.1390294
55. Abadi, M., Aga wal, A., Ba ham, P., e al.: Tenso Flow: La ge-
scale machine lea ning on he e ogeneous sys ems. (2015) h ps://
www. enso low.o g/, so wa e a ailable om enso low.o g
56. Ped egosa, F., Va oquaux, G., G am o , A., e al.: Sciki -lea n:
Machine lea ning in Py hon. J. Mach. Lea n. Res. 12, 2825–2830
(2011)
Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o ju is-
dic ional claims in published maps and ins i u ional a ilia ions.
Simon Bilik ecei ed he M.Sc. deg ee in echnical cybe ne ics om he
Depa men o Con ol and Ins umen a ion, B no Uni e si y o Tech-
nology, Czech Republic, in 2019. He is cu en ly a double deg ee PhD
s uden wi h he Machine Vision G oup, BUT Uni e si y and he Com-
pu e Vision and Pa e n Recogni ion Labo a o y, LUT Uni e si y. His
esea ch ield includes applied machine ision and machine lea ning.
Daniel Ba akhano a ained his M.Sc. deg ee in Compu a ional Engi-
nee ing and Technical Physics om he School o Enginee ing Science
a LUT Uni e si y, Lappeen an a, Finland in 2021. Cu en ly, he is a
Junio Resea che a he LUT Compu e Vision and Pa e n Recog-
ni ion Labo a o y, his ocus a eas include machine lea ning, image
p ocessing, and da a analysis.
Tuomas Ee ola ecei ed he M.Sc. and Ph.D. deg ees in in o ma ion
p ocessing om he Depa men o In o ma ion Technology, Lappeen-
an a Uni e si y o Technology, Finland, in 2006 and 2010, espec-
i ely. He is cu en ly an Associa e P o essso wi h he Compu e
Vision and Pa e n Recogni ion Labo a o y, LUT Uni e si y. His
esea ch in e es s include digi al image p ocessing, applied compu e
ision, and deep lea ning.
Lumi Ha aguchi is an oceanog aphe specialized in phy oplank on
and mic ozooplank on ecology. She ecei ed he PhD in Biosciences
om Aa hus Uni e si y (DK) in 2018 and is cu en ly wo king a he
Finnish En i onmen al Ins i u e.
Kaisa K a wo ks in Ma ine Ecology Measu emen s uni o he Finnish
En i onmen Ins i u e. He wo k includes high- equency measu e-
men s o phy oplank on communi y wi h main ocus on imaging. She
wo ks on implemen a ion o he imaging echnology o Bal ic Sea phy-
oplank on communi y and he consequen image analysis. He special
in e es is in phy oplank on bloom dynamics and he ela ed en i on-
men al o cing.
Silke Van den Wyngae is an aqua ic mic obial ecologis and ecei ed
he Ph.D. deg ee om he Swiss Fede al Ins i u e o Technology
(ETHZ), Swi ze land, in 2013. She is cu en ly an Academy Resea ch
ellow a he Depa men o Biology, Uni e si y o Tu ku, Finland. He
esea ch in e es s include plank on ecology, in pa icula di e si y o
aqua ic ungi and hei in e ac ions wi h phy oplank on.
Jonna Kangas g adua ed as a MSc in En i onmen al Sciences in 2022
om Uni e si y o Helsinki. He hesis p ojec ocused on he e ec s
o salini y change on he mic obial loop in he Bal ic Sea.
Conny Sjöq is g adua ed as a PhD in Ma ine Biology in 2015 om
Åbo Akademi Uni e si y (ÅAU), Finland. He is cu en ly wo king as
a esea che in plank on ecology a ÅAU. He was awa ded he Ti le o
Docen ship in Molecula and E olu iona y Ma ine Ecology in 2022.
Ka in Madsén g adua ed as a MSc. in En i onmen al and Ma ine Biol-
ogy in 2023 om Åbo Akademi Uni e si y. He hesis p ojec ocused
on de ec ion o phy oplank on pa asi es using imaging low cy ome y.
Lasse Lensu is a p o esso o machine ision and da a analysis a
Lappeen an a-Lah i Uni e si y o Technology LUT, Finland. He
ecei ed his D.Sc. (Tech.) deg ee in compu e science and enginee ing
in 2002 om he Depa men o In o ma ion Technology o LUT. His
esea ch in e es s include machine/compu e ision, pa e n ecogni-
ion wi h machine lea ning and da a analysis. P o . Lensu is a membe
o he Compu e Vision and Pa e n Recogni ion Labo a o y o he
Depa men o Compu a ional Enginee ing a LUT, and he has con-
ibu ed o he echnology ans e o h ee spin-o companies om
he uni e si y.
Heikki Käl iäinen is a ull p o esso o Compu e Science and Engi-
nee ing a he Lappeen an a-Lah i Uni e si y o Technology LUT,
School o Enginee ing Science, Finland. P o . Käl iäinen is a head
o he Compu e Vision and Pa e n Recogni ion Labo a o y a he
Depa men o Compu a ional Enginee ing. His esea ch in e es s
include compu e ision, pa e n ecogni ion, machine lea ning, and
especially applica ions o digi al image p ocessing and analysis.
Besides LUT, P o . Käl iäinen has been a P o esso o Compu ing a
Monash Uni e si y Malaysia, and a Visi ing P o esso a B no Uni e -
si y o Technology in Czech Republic, Czech Technical Uni e si y a
P ague, and Uni e si y o Su ey in UK.
Ka el Ho ak ecei ed he M.Sc. and Ph.D. deg ees in echnical cybe -
ne ics om he Depa men o Con ol and Ins umen a ion, B no Uni-
e si y o Technology, Czech Republic, in 2004 and 2008, espec-
i ely. He is cu en ly an Associa e P o essso wi h he Machine Vision
G oup, BUT Uni e si y. His esea ch in e es s include machine ision,
image p ocessing and machine lea ning.
123