Towards Phytoplankton Parasite Detection Using Autoencoders

Machine Vision and Applica ions (2023) 34:101 h ps://doi.o g/10.1007/s00138-023-01450-x ORIGINAL PAPER Towa d phy oplank on pa asi e de ec ion using au oencode s Simon Bilik1,2 ·Daniel Ba akhano 1·Tuomas Ee ola1·Lumi Ha aguchi3·Kaisa K a 3· Silke Van den Wyngae 4·Jonna Kangas4·Conny Sjöq is 5·Ka in Madsen5·Lasse Lensu1· Heikki Käl iäinen1·Ka el Ho ak2 Recei ed: 15 Ma ch 2023 / Re ised: 16 June 2023 / Accep ed: 14 Augus 2023 / Published online: 13 Sep embe 2023 © The Au ho (s) 2023 Abs ac Phy oplank on pa asi es a e la gely unde s udied mic obial componen s wi h a po en ially signi ican ecological in luence on phy oplank on bloom dynamics. To be e unde s and he impac o phy oplank on pa asi es, imp o ed de ec ion me hods a e needed o in eg a e phy oplank on pa asi e in e ac ions in o moni o ing o aqua ic ecosys ems. Au oma ed imaging de ices commonly p oduce as amoun s o phy oplank on image da a, bu he occu ence o anomalous phy oplank on da a in such da ase s is a e. Thus, we p opose an unsupe ised anomaly de ec ion sys em based on he simila i y be ween he o iginal and au oencode - econs uc edsamples.Wi h hisapp oach,wewe eable o eachano e allF1sco eo 0.75inninephy oplank on species, which could be u he imp o ed by species-speci ic ine- uning. The p oposed unsupe ised app oach was u he compa ed wi h he supe ised Fas e R-CNN-based objec de ec o . Using his supe ised app oach and he model ained on plank on species and anomalies, we we e able o each a highes F1 sco e o 0.86. Howe e , he unsupe ised app oach is expec ed o be mo e uni e sal as i can also de ec unknown anomalies and i does no equi e any anno a ed anomalous da a ha may no always be a ailable in su icien quan i ies. Al hough o he s udies ha e deal wi h plank on anomaly de ec ion in e ms o non-plank on pa icles o ai bubble de ec ion, ou pape is, acco ding o ou bes knowledge, he i s ha ocuses on au oma ed anomaly de ec ion conside ing pu a i e phy oplank on pa asi es o in ec ions. Keywo ds Phy oplank on anomalies ·Phy oplank on pa asi es ·Anomaly de ec ion ·Au oencode s ·Objec de ec ion · Fas e R-CNN S. Bilik and D. Ba akhano ha e con ibu ed equally o his wo k. BSimon Bilik [email p o ec ed] Daniel Ba akhano [email p o ec ed] Tuomas Ee ola [email p o ec ed] Lumi Ha aguchi [email p o ec ed] Kaisa K a [email p o ec ed] Silke Van den Wyngae silke. [email p o ec ed] Jonna Kangas [email p o ec ed] Conny Sjöq is conny[email p o ec ed] Ka in Madsen [email p o ec ed] Lasse Lensu [email p o ec ed] Heikki Käl iäinen [email p o ec ed] Ka el Ho ak [email p o ec ed] 1Compu e Vision and Pa e n Recogni ion Labo a o y, Depa men o Compu a ional Enginee ing, Lappeen an a-Lah i Uni e si y o Technology LUT, Yliopis onka u 34, 53850 Lappeen an a, Finland 2Depa men o Con ol and Ins umen a ion, Facul y o Elec ical Enginee ing and Communica ion, B no Uni e si y o Technology, Technická 3058/10, 616 00 B no, Czech Republic 3Ma ine Ecology Measu emen s, Finnish En i onmen Ins i u e, Agnes Sjöbe gin Ka u 2, 00790 Helsinki, Finland 4En i onmen al and Ma ine Biology, Åbo Akademi Uni e si y, Hen ikinka u 2, 20014, Tu ku, Finland 5En i onmen al and Ma ine Biology, Åbo Akademi Uni e si y, Hen ikinka u 2, 20014 Tu ku, Finland 123 101 Page 2 o 18 S. Bilik e al. 1 In oduc ion Phy oplank ona ekey playe s in aqua ic sys ems,whe e hey media e biogeochemical cycles and o m he base o mul iple ood webs [1]. The dynamics o phy oplank on popula ions esul om he in e play be ween esou ce a ailabili y and mo ali y losses [2]. While some loss mechanisms such as g azing a e well known, he con ibu ion o loss mecha- nisms like pa asi ism emains poo ly conside ed and la gely unde s udied in many aqua ic sys ems. Phy oplank on a e suscep ible o a wide a ie y o pa asi es, such as i uses, bac e ia, p o is s, and ungi. Such pa asi es can cause mo - ali y o ce ain phy oplank on species, he eby al e ing he phy oplank on bloom dynamics and changing he cycling o ma e and low o ene gy in aqua ic ecosys ems [3–5]. Zoospo ic o nano lagella e pa asi es ha in ec phy o- plank on comp ise a highly di e se unc ional g oup o euka yo ic p o is and ungal species [6]. They ha e in com- mon he p oduc ion o ee-li ing mo ile s ages as hei in ec i e p opagules, which a ach o a phy oplank on hos cell and de elop ei he inside (endobio ic) o ou side (epibi- o ic) he hos cell using hos esou ces o hei g ow h and ep oduc ion. Due o hei inconspicuous na u e, phy- oplank on pa asi es a e di icul o iden i y, and objec s ha a e di icul o iden i y ypically end o be o e looked o neglec ed.Consequen ly,al hough hep esence and po en ial impo ance o hese phy oplank on pa asi es a e inc easingly ecognized, quan i a i e da a o hei occu ence in na u e a e ex emely sca ce. An addi ional challenge o s udy o phy oplank on pa - asi es is he need o cap u e apid in ec ion dynamics on a ele an empo al and spa ial scale (e.g., days). Ob aining quan i a i ein o ma ionabou pa asi ein ec ionsusing adi- ionalme hods is labo -in ensi eand ime-consuming, which limi s he spa ial and/o empo al co e age o many s udies in es iga ing phy oplank on–pa asi e in e ac ions [7]. Recen echnological ad ances in imaging ins umen s ha e made i possible o collec la ge olumes o plank on image da a o s udy o plank on popula ions, hus opening new esea ch possibili ies [8]. The possibili y o high- equency sampling enabled by imaging ins umen s can po en ially esul in be e unde s anding o phy oplank on dynamics and hei po en ial in e ac ions wi h pa asi es [7]. Howe e , while me hods o au oma ic ecogni ion o phy- oplank on classes ha e been widely de eloped, me hods o au oma ic ecogni ion o phy oplank on pa asi e in ec- ions emain unde de eloped. The absence o an e ec i e app oach o pa asi ic in ec ion ecogni ion is likely associ- a ed wi h challenges ela ed o ob aining su icien olumes o image da a o plank on pa asi es, which equi es sc een- ing o huge amoun s o aw image da a. Such asks a e bes add essed wi h au oma ed solu ions. Fig. 1 Anomaloussampleo heCen alesplank onspecies:aO iginal, bencoded space, c econs uc ion, and ddi e ence image The sca ci y o plank on pa asi e images is a majo chal- lenge o he de elopmen o deep lea ning-based compu e ision me hods o pa asi e de ec ion. While objec de ec ion me hodssuch asFas e R-CNN[9]and YOLO[10]ha ebeen shown o achie e high accu acy on a ious de ec ion asks, including pa asi e de ec ion (see, e.g., [11]), hey s uggle when he amoun o aining da a is limi ed. The e o e, a mo e p omising app oach is o o mula e pa asi e de ec ion as an anomaly de ec ion ask. He e, he idea is o ain he model wi h images o heal hy plank on and de ec images ha de ia e om he da a on which he models we e ained. Due o he a ailabili y o la ge amoun s o plank on image da a wi hou pa asi es o aining and ela i ely small in a- class a ia ion among heal hy samples, images ha de ia e no ably om he aining da a can be expec ed o con ain po en ial pa asi es. This wo k in es iga es au oma ed image-based phy o- plank on pa asi e de ec ion. The p oblem is o mula ed as an anomaly de ec ion p oblem and sol ed using an au oen- code . The p oposed me hod consis s o a ec o -quan ized a ia ionalau oencode (VQVAE)[12] ha encodes heinpu image in o a comp essed la en ep esen a ion and uses he comp essed ep esen a ion o econs uc he o iginal image. The a ionale is ha when he au oencode is ained only on images o heal hy phy oplank on, he au oencode ails o econs uc he pa asi es, which allows hem o be de ec ed om he di e ence image (see Fig. 1). The p oposed me hod u he employs he Ha dNe [13] ea u e ex ac o and Local Ou lie Fac o [14] o dis inguish be ween heal hy plank on and plank on wi h pa asi es. In he expe imen al pa o he wo k, an ex ensi e se o di e en backbone con olu ional neu al ne wo ks (CNNs), au oencode a chi ec u es, ea u e ex ac o s, and classi ie s a e sys ema ically e alua ed on challenging phy oplank on image da a o ind he bes combina ion and o demons a e he pe o mance o he p oposed me hod. In addi ion, we compa e heau oencode -basedanomalyde ec ionme hod o a Fas e R-CNN-based objec de ec o . The esul s show ha he p oposed me hod achie es compa able accu acy o he s a e-o - he-a Fas e R-CNN objec de ec o while equi - ing no images wi h pa asi es o aining. Consequen ly, he au oencode -based me hod can be conside ed a p omising app oach o u iliza ion in plank on image analysis whe e 123 Towa d phy oplank on pa asi e de ec ion using au oencode s Page 3 o 18 101 he collec ion o la ge aining da a o plank on wi h pa a- si es is in easible. The main con ibu ions o ou pape a e he de elopmen o a no el anomaly de ec ion amewo k and i s applica ion ophy oplank onpa asi ede ec ion.High- equencyimaging da a coupled o au oma ic p eso ing o po en ially in ec ed plank onallow o cap u eand quan i y in ec ion dynamics on ele an empo al and spa ial scales. This is an essen ial s ep owa d unde s anding he ole o pa asi es in shaping phy- oplank on communi y dynamics and ecosys em p ocesses. The p oposed amewo k is gene al and can be applied o o he anomaly de ec ion ask such as indus ial aul con ol. 2 Rela ed wo k Anomaly de ec ion is a da a classi ica ion echnique in which ade ec o models he ep esen a iono sampleswi hinaspec- i ica ion (OK) and classi ies all samples ha de ia e om he speci ica ion as anomalous (NOK). This p oblem can be challenging because o po en ially high di e si y wi hin he NOK samples, imbalance be ween he numbe o samples in he OK and NOK g oups, and i egula i y o he NOK class. A comp ehensi e o e iew desc ibing anomaly de ec- ion p oblems, echniques, and ca ego iza ion is p esen ed in [15]. Fi s in oduced o image da a in [16], au oencode (AE) models a e now widely used in compu e ision. The use o insu icien gene aliza ion abili y on ou -o - aining da a o an AE model wi h he aim o de ec ing anomalies in syn he ic and eal-wo ld da a, in he case s udied, eleme y da a, was i s demons a ed in [17]. The esul s showed ha such AEs can be used o de ec p e iously unseen anomalous samples. The concep was u he enhanced and used on image da a in, o example, [18] and [19]. A comp ehensi e o e iew o AE echniques can be ound in [20]. Plank on anomaly de ec ion has been p e iously s udied in he con ex o open-se ecogni ion, i.e., image classi ica- ion wi h he p esence o p e iously unseen classes (plank on species). In [21], he au ho s p esen ed an unsupe ised app oach o classi y a plank on sample and de ec po en ial signi ican di e ences (i.e., anomalies) wi h espec o he de ec ed class. Image ea u es we e ex ac ed using classical compu e ision me hods u ilizing geome ical, momen - based, and o he adi ional ea u es. In [22], a CNN ained on OK samples and a i icial NOK samples de i ed om he OK da a by common da a aug- men a ion echniques such as blu ing and noise addi ion was used as he ea u e ex ac o . An anomaly sco e was hen compu ed om hese ea u es and used oge he wi h he ained ea u e ex ac o o dis inguish be ween he OK plank onsamples and anomalies. In he wo k, ai bubblesand non-plank on wa e pa icles we e conside ed as anomalies. In [23], he au ho s used a pa allel ne wo k o cus om s a is icalclassi ie scalledTailDeTec (TDT) odisco e p e- iously unseen plank on species. Each o he TDT classi ie s was ained on one pa icula species, and a sample was con- side ed as unknown i none o he classi ie s was able o de ec i . Unknown samples we e collec ed and alida ed by expe s. Fea u e ex ac ion and he concep i sel we e based on wo k p esen ed in [21]. In [24], open-se ecogni ion plank on ecogni ion was add essed using a simila i y lea ning app oach. Me ic lea n- ing wi h angula ma gin loss was applied o ob ain image embedding ec o s ha model he simila i y be ween images. Theanomalies (images om p e iously unseen classes) we e de ec ed by se ing h eshold alues o he simila i y. Fas e R-CNN [9] is a popula deep lea ning (DL) algo- i hm ha has been success ully applied o a ious domains and asks, including objec de ec ion and anomaly de ec- ion [25]. Anomaly de ec ion using Fas e R-CNN in ol es aining he model on abno mal images o lea n he ea u es o abno mal ins ances. Then, du ing classi ica ion, he model is used o de ec abno mal samples ha de ia e om he expec ed ou come. Fo ins ance, in indus ial manu ac u - ing, abno mal beha io can include machine mal unc ions, while in medical diagnosis, i can ake he o m o unusual pa e ns in medical images. An example o anomaly de ec ion is p esen ed in [25], whe e an imp o ed Fas e R-CNN was used o de ec de ec s in s eel pla es. The algo i hm was ained on a da ase o abno mal egions on s eel pla e images and was able o accu a ely de ec anomalies such as c acks and holes in es images. Using a simila app oach, a sub le modi ica ion o Fas e R-CNN o de ec ion o anomalies in CT images o lungs was conside ed in [26]. Objec de ec ion me hods ha e also been success ully used o pa asi e de ec ion. Fo example, in [11], whe e a YOLO 5 objec de ec o is used o de ec a pa asi ic mi e on he body o a honey bee. An o e iew o o he objec de ec- ion echniques and commonly used da ase s can be ound, o example, in [27]. In plank on esea ch, Fas e R-CNN has been widely adop ed o segmen a ionandobjec de ec ion.Se e alobjec de ec ionapp oaches,including Fas e R-CNN, we eu ilized in [28] o e alua e a syn he ically augmen ed da ase . Simi- la wo k is p esen ed in [29], whe e a plank on da ase om a da k ield mic oscope was compiled and hen es ed wi h a ious objec de ec ion me hods, including YOLO 3 [30], R-CNN [31], and SSD [32]. 123 101 Page 4 o 18 S. Bilik e al. 3 P oposed me hods o phy oplank on anomaly de ec ion In his wo k in es iga ing de ec ion o phy oplank on sam- ples wi h anomalies, we p ima ily employ an unsupe ised au oencode -based app oach, ollowed by he use o di e - en ea u e ex ac o s and one-class classi ie s. Supe ised objec de ec ion based on he Fas e R-CNN [9] is u ilized o compa e he esul s o ou p oposed me hod wi h a s a e-o - he-a app oach. 3.1 Au oencode -based app oach The p oposed me hod o de ec anomalous plank on sam- ples is cons uc ed on he amewo k a ailable in [33]. This implemen a ion allows a ious combina ions o di e en AE a chi ec u es wi hou conside a ion o he con olu ional lay- e s (i.e., ully connec ed AE, a ia ional AE and o he s) e med as AE co es, con olu ional laye s a chi ec u es, ea- u e ex ac o s, and one-class classi ie s o be es ed. In he app oach used in his pape , we combined i e AE co es, six con olu ional encode s and decode s, six ea u e ex ac o s, and ou classi ie s (720 combina ions in o al). The p ocess- ing pipeline is shown in Fig. 2and desc ibed in mo e de ail in he sec ions below. The anomaly de ec ion is based on compa ison be ween he o iginal da a and he au oencode - econs uc ed da a, ol- lowed by ea u e ex ac ion and one-class classi ica ion. 3.1.1 Au oencode a chi ec u es and con olu ional laye s As he i s s ep o anomaly de ec ion, we use AE mod- els ained only on OK da a o econs uc unknown inpu samples o bo h OK and NOK classes. On accoun o he non-op imal gene aliza ion o he AE models and aining only on he OK class o da a, we hypo hesize ha da a om he NOK class will be econs uc ed wo se han da a om he OK class, as desc ibed in [17]. To be e unde s and he e ec o he AE a chi ec u e’s co e and he complexi y o he con olu ional encoding and decoding laye s, we decided o build ou implemen a ion such ha he co e o he model could be combined wi h he selec edcon olu ionalpai so heencode sand hedecode s. The p oposed s uc u e allows us o analyze he con ibu- ions o he selec ed a chi ec u e and con olu ional laye s sepa a ely. We e alua ed i e di e en op ions o he AE co es. As he i s al e na i e, we used implemen a ions o basic con o- lu ional AE [34] as he BAE1 co e, con olu ional a ia ional AE [35] as he VAE1 co e, and ec o -quan ized AE [12]as he VQVAE1 co e. As well as using hese co es, we ied o u he educe he ea u es ex ac ed by an encode by inse - ing ully connec ed laye s o he basic con olu ional AE as heBAE2co e[36] and o he a ia ional AE as he VAE2 co e.Thesemodi ica ions o he au oencode co es a e shown in Fig. 3. Fig. 2 P ocessing pipeline o he p oposed au oencode -based anomaly de ec ion me hod 123 Towa d phy oplank on pa asi e de ec ion using au oencode s Page 5 o 18 101 Fig. 3 Schemes o he modi ied au oencode co es: aBAE1 co e, bVAE2 co e We expec ha he basic con olu ional AE is going o be su passed in pe o mance by bo h he a ia ional and he ec o -quan ized co es because o hei non-p obabilis ic encoding space, which allows leak o mo e inpu image’s anomalous pa s o he encoded space and econs uc ed image. The quali y o he econs uc ed images should be be e in he case o he basic and ec o -quan ized co es han wi h a ia ional co es, which ypically p oduce blu y ou pu s [37]. When aining on di e en classes, he bes esul s a e expec ed om he ec o -quan ized co e, which should c ea e sepa able clus e s o each class in he encoded space. Besides he AE co es desc ibed abo e, we also conside six pai s o con olu ional encoding and decoding laye a chi- ec u es, whose s uc u e is desc ibed in he complemen a y ables: Table 5 o encode s, and Table 6 o decode s. Each con olu ional laye o block desc ibed in hese ables is com- plemen ed wi h he ba ch no maliza ion laye . The ac i a ion unc ion was se as Leaky ReLu by he Con M1 a chi ec u e and as ReLu o he o he a chi ec u es. The es ed con olu ional laye s go om he mo e com- plex Con M2 a chi ec u e, sugges ed o anomaly de ec ion in [19], and Con M1 a chi ec u e, whe e we expec he abili y o econs uc ine ea u es and de ails, o he sim- ple a chi ec u es Con M5, Con M4 and Con M3. By using he simple a chi ec u es, we expec ha ine ea u es and smalle image s uc u es will be supp essed and he a chi- ec u e migh hus pe o m be e on shape o s uc u e anomalies. The las a chi ec u e, Con M6, is unsymme - ical, as sugges ed in [38], and uses he mo e complex encode o he Con M5 a chi ec u e and he simple decode o he Con M4 a chi ec u e. Using his a chi ec u e, we expec ha anomalies ha a e p opaga ed o he encoded space will be u he supp essed by he decode econs uc- ion. In he op imal case, anomalous a eas o he o iginal image a e emo ed du ing he image econs uc ion as shown in Fig. 11. A di e ence image be ween he o iginal sample and he econs uc ed sample is hen compu ed and used in he ea u e ex ac ion. 3.1.2 Fea u e ex ac ion The second s ep o he amewo k applies ea u e ex ac- o s o analyze he econs uc ions. The ea u es a e based on compa ison be ween he o iginal and econs uc ed da a (E o me ics, Ha dNe 3 and Ha dNe 4) o analysis o he di e ence image (SIFT ea u e ex ac ion, Ha dNe 1 and Ha dNe 2). The i s ea u e ex ac ion app oach (E o me ics) c e- a es a low-dimensional ea u e ec o o each image by compu ing selec ed e o me ics be ween he o iginal and econs uc ed images. The L2 and SSIM me ics applied in [36] a e complemen ed wi h he A e age hash and mean- squa ed e o me ics. The second ea u e ex ac ion me hod (SIFT ea u e ex ac ion) uses scale and me ics p ope ies o he image keypoin s ound by he SIFT me hod. I is a di ec e- implemen a ion he app oach p esen ed in [39]. The me hod uses di e ence images be ween he o iginal and econ- s uc ed da a. Thelas ou ea u e ex ac ionme hods(Ha dNe 1, Ha d- Ne 2, Ha dNe 3 and Ha dNe 4) a e all based on he ba ch simila i y me ic p esen ed in [13]. Ha dNe 1 is he sim- ples me hod whe e each sample is desc ibed by he Ha dNe (HN) ea u e ec o o he o iginal image esized o he size o 32 ×32 as equi ed by he o iginal HN implemen a ion. Since such esizing migh no be op imal o small anoma- lies, Ha dNe 2 spli s he image o he o iginal size o blocks o 32×32 and compu es he HN ea u e ec o o each such block. The esul ing ea u e ec o consis s o he no ms o e hose ec o s. Ha dNe 3 spli s he o iginal and econs uc ed images o 32×32 blocks as in he Ha dNe 2 me hod, bu he esul ing ea u e ec o is compu ed as a cosine simila i y be ween he HN ea u e ec o s o he co esponding blocks o he o iginal and decoded images. Ha dNe 4 uses he same echnique, bu he cosine simila i y is supplemen ed by he loga i hm, which is supposed o emphasize smalle di e - ences o he Ha dNe 3 ea u e ec o . A 2D isualiza ion o he esul ing ea u e space ob ained by he Con M5-BAE2 au oencode o e he Aphanizomenon 123 101 Page 6 o 18 S. Bilik e al. Fig. 4 Example ea u e space o he Aphanizomenon plank on species plank on species using he Ha dNe 2 ea u e ex ac o is shown in Fig. 4. The OK samples o m an ellip ical clus e , and mos o he NOK samples a e sepa a e om ha clus e . 3.1.3 One-class classi ica ion Fo he classi ica ion pa , we used he ollowing one-class classi ie s: •Robus co a iance (RC) [40]: The RC classi ie assumes he same dis ibu ion o all OK samples and i s an ellip- ic en elope o he cen al da a poin . The anomaly sco e is compu ed using he dis ibu ion es ima ions and Maha- lanobis dis ance. •One-class SVM (OC-SVM) [41]: The OC-SVM clas- si ie u ilizes he suppo ec o machine (SVM) and a nonlinea ke nel o c ea e a sepa a ing hype plane o he aining da a om he o igin o he ea u e space. Sam- ples on he o he side o his hype plane a e conside ed as anomalies. •Isola ion Fo es (IF) [42]: The IF classi ie uses andom ea u e selec ion and spli ing o isola e obse ed sam- ples. The anomaly sco e is based on he o al numbe o spli s. Anomalies a e supposed o ha e a smalle numbe o spli s as i should be easie o sepa a e hem. •Local Ou lie Fac o (LOF) [14]: The LOF classi ie is based on he local densi y de ia ion o he obse ed poin wi h espec o i s k-nea es neighbo s. The densi y o he anomalies should be lowe in compa ison wi h he OK samples, which a e conside ed o c ea e dense clus e s. The ac ion o anomaly samples o he OC-SVM, IF and LOF was se o 1% since his alue is he minimum alue o common implemen a ions. Based on he no mal dis ibu ion, we should also assume ha e en some OK samples migh sligh ly di e om he majo i y. All classi ie s a e i on he da ase con aining only OK samples. Inpu ea u es o he one-class classi ica ion a e no - malized using obus scaling, which no malizes he median Fig. 5 Illus a ion o equal-e o - a e (EER) h eshold selec ion c i e- ion on he ROC cu e and he in e qua ile ange, as sugges ed in [43]. This no - maliza ion should be mo e obus o ou lie s han simple no maliza ion app oaches such as min–max no maliza ion o s anda diza ion. To selec he op imal decision h eshold o anomaly de ec ion, we use he equal e o a e (EER) o e he ROC cu e o he classi ie as shown in Fig. 5. All classi ie s a e i only on he OK da a, and he ROC cu e is ob ained om he es da ase . 3.2 Objec de ec ion-based app oach The Fas e R-CNN [9] algo i hm is composed o h ee main componen s: a base ea u e ex ac o ne wo k, a egion p o- posal ne wo k (RPN) o ex ac ing he egions o in e es , and a de ec o ha uses he egion p oposals and espec i e ea u e maps o classi y he de ec ed objec s as shown in Fig. 6. The i s componen is he ea u e ex ac o esponsi- ble o gene a ing ea u e maps om he inpu image. This module is usually a CNN such as VGG-16 [44]o ResNe - 50 [45]. 123 Towa d phy oplank on pa asi e de ec ion using au oencode s Page 7 o 18 101 Fig. 6 Fas e R-CNN a chi ec u e Fig. 7 The Fas e R-CNN app oach o anomaly de ec ion Fig. 8 Objec de ec ion asks using he Fas e R-CNN app oach: aPlank on e sus Anomalies; bPlank on e sus Anomalous Plank on; c Anomalies only. The NOK samples a e shown in he op ow and he OK samples in he bo om ow The RPN is a kind o ully con olu ional ne wo k ha akes he ea u e maps om he p e ious s ep and e u ns a se o egionp oposals ha guide hede ec o onwhe e o ind he objec s in he image. The p oposals and co esponding ea u e maps om he CNN a e hen u ilized o yield can- dida e objec s wi h bounding boxes and ixed-leng h ea u e ec o susing he ROIpoolinglaye .Finally, hese ou pu s a e passed o he R-CNN ne wo k. The R-CNN ne wo k uses he p oposed ea u e maps o classi y each bounding box as an objec o backg ound and p edic inal class sco es wi h he bounding boxes. Fo ou objec de ec ion expe imen s, we used he Fas e R-CNN implemen a ion a ailable om [46] based on he ResNe -50 backbone p esen ed in [47]. To employ an anomaly de ec ion ask in he Fas e R-CNN baseline, he a chi ec u e is supplemen ed by a one-class classi ica ion module based on he p edic ed objec labels, as shown in Fig. 7. Sinceanomaliessuchaspa asi esa e ela i elysmallcom- pa ed o he image size, i is impo an o conside he ancho gene a o which is a pa o he egion-p oposal ne wo k. Ancho s de ine egions o an image, usually o di e en aspec a ios and sizes, ha a e used as e e ences o de ec objec s. The ancho gene a o c ea es a se o ancho s o each loca ion in a ea u e map; hen, o each egion o in e - es , he model p edic s which ancho box bes encloses he objec . The choice o an ancho gene a o mos ly depends on he ype o de ec ion ask. Fo example, i we wan o de ec small objec s, hen a smalle ancho size should be used. On he o he hand, i he ask is o de ec objec s o a ious sizes, 123 101 Page 8 o 18 S. Bilik e al. a ange o ancho sizes should be de ined [9]. Addi ionally, he aspec a ios o he ancho s should ma ch he aspec a ios o he objec s in he image. As sugges ed in [11], h ee sepa a e objec de ec o s a e conside ed, each ained on di e en g ound u h: (1) plank- on and anomalies, (2) plank on (clean) and anomalous plank on, and (3) anomalies only (see Fig. 8). In he i s column, we can see ha he model de ec s a plank on sample in bo h cases and an anomaly in he op ow. The second col- umn shows de ec ion o a plank on sample wi h anomaly in he op owand de ec iono a cleansample in hebo om ow, and inally, he hi d column shows de ec ion o an anomaly in he op ow only. 4 Expe imen s In his sec ion, we desc ibe he da ase s used, he e alua ion me ics,and he esul so heau oencode -basedexpe imen s and he objec de ec ion-based expe imen s. Table 1 Species-speci ic s a is ics o he plank on anomaly da ase Plank on class OK samples coun NOK samples coun Aphanizomenon 830 140 Cen ales 400 57 Dolichospe mum 515 406 Chae oce os 606 371 Nodula ia 118 357 Pauliella 160 433 Pe idiniella Chain 183 31 Pe idiniella Single 459 63 Skele onema 769 419 4.1 Phy oplank on anomaly da ase Na u al Bal ic Sea phy oplank on communi ies a e con in- uously imaged wi h an Imaging FlowCy obo (IFCB) [48] deployed a U ö A mosphe ic and Ma ine Resea ch S a ion, Finland (59◦46.84’ N, 21◦22.13’ E). The IFCB is connec ed o he s a ion low- h ough sys em, which ecei es wa e pumped oman ∼5mdeepinle loca ed 250mo sho e, ep- esen a i eo hesub-su acelaye .A U ö,IFCB akesa5-ml sample nea ly e e y 20min and he sys em is se o igge based on he de ec ion o chlo ophyll, i.e., a ge ing phy o- plank on cells a he han non-li ing pa icles. The esea ch s a ion and IFCB deploymen a U ö a e desc ibed in de ail in [49] and [50]. The phy oplank on da a om U ö IFCB can be cu en ly classi iednea eal- imein o50di e en classes,asdesc ibed by [51]. Pu a i e pa asi e in ec ion images we e manually anno a ed by expe s based on o he U ö da a, collec ed be ween Feb ua y and Augus 2021, using phy oplank on da a om nine classes. These classes we e selec ed based on hei impo ance du ing he sp ing o summe blooms in he Bal ic Sea. In ou expe imen s, we used a phy oplank on anomaly da ase de i ed om he anno a ed images used o ain he classi ie desc ibed abo e wi h OK samples om he da ase published in [51] and NOK samples om unpublished 2021 U ö da a. The anomaly da ase con ains o e 6200 manu- ally anno a ed and expe - alida ed samples o 9 plank on classes wi h known anomalies, as shown in Table 1. Non- anomalous and anomalous samples o each class a e shown in Fig. 9. As an anno a ion ool, we used he ee e sion o he Label S udio a ailable a [52]. The anno a ed da ase is a ailable online a [53] in bo h COCO and YOLO o ma s. Fig. 9 Anomalous (le column, o uppe ow) and non-anomalous samples ( igh column, o lowe ow) om all da ase classes o he used da ase 123 Towa d phy oplank on pa asi e de ec ion using au oencode s Page 9 o 18 101 Fig. 10 Example o he anno a ion bounding boxes 4.1.1 Da ase anno a ions When anno a ing he da ase , we used h ee di e en labels o de ine a sepa a e species se : •The label Anomaly ma ks he pa asi e o o he anomalies on he plank on sample. •The label Plank onSpecies_Anomaly ma ks plank on species wi h he a ached pa asi e. •ThelabelPlank onSpecies_Clean ma ksplank onspecies wi h o wi hou he pa asi e. The las wo labels could o e lap, bu whene e i was possible, he Plank onSpecies_Clean label does no co e he sample pa wi h pa asi e. To dis inguish be ween he OK and NOK samples, he Plank onSpecies_Clean label should be emo ed i i o e laps wi h he Plank onSpecies_Anomaly one. An example o he anno a ion o e a Dolichospe mum plank on species sample is shown in Fig. 10. The ed colo ma ks a plank on anomaly and, in his case, he da ke g een ma ks he clean sample and he ligh e g een ma ks he sam- ple wi h an anomaly. 4.1.2 De i ed da ase o au oencode -based expe imen Fo he pu poses o he au oencode -based expe imen , we used he abo e-desc ibed da ase o de i e a one-class da ase wi hnoNOK samples and 70%o he OK sample in he ain- ingse .Tes and alida ionda ase salwayscon ainabalanced numbe o OK and NOK samples. The expe imen wi h all plank on species con ains all a ailable aining samples, 10 alida ion samples, and 10 es samples om each species. In o de o help he AE model o lea n mo e obus ea- u es, we added sal -and-peppe noise o he image samples used du ing he aining wi h a clean sample used as a label as sugges ed in [54]. Besides his noise augmen a ion, we also use andom lipping, con as , sa u a ion, b igh ness, in e - sion, and hue augmen a ion. Because he Ha dNe -based ea u e ex ac o s wo k co - ec ly only wi h image sizes o mul iples o 32, all samples we e esized wi h espec o he majo aspec a io o each class (1:4 o i e classes, 1:1 o h ee classes and 1:2 o one class) as can be seen in Fig. 9. Fo he expe imen o e all classes, we chose he aspec a io o 1:2 as a comp omise. 4.1.3 De i ed da ase o objec de ec ion-based expe imen Fo he objec de ec ion expe imen , he model was ained in a supe ised manne . The spli a ios we e se as 70%, 10%, and 20% o he aining, alida ion, and es subse s, espec i ely.T ainingand alida ionse s dono includeclean samples, whe eas a es se con ains a balanced numbe o anomalies and clean images. Addi ionally,weapplied he ollowingaugmen a ion ech- niques: ho izon al and e ical lip wi h a p obabili y o 30%, and andom b igh ness, con as and sa u a ion adjus men wi h a p obabili y o 10%. 4.2 Pe o mance me ics To compa e he esul s o he au oencode and objec de ec- ion expe imen s, we need o e alua e he p edic ions o he models wi h espec o he g ound- u h labels. To do so, we can de ine ue-posi i e (TP) and ue-nega i e (TN) p e- dic ions, whe e he model co ec ly classi ies OK and NOK samples, oge he wi h alse-posi i e (FP) and alse-nega i e (FN) p edic ions, whe e he model misclassi ies NOK sam- ples as OK in he FP case and OK samples as NOK in he FN case. P ecision, Recall and F1 sco e me ics a e used o com- pa ison o he di e en a ia ions o au oencode s and objec de ec ion me hods. The me ics a e de ined as ollows: P ecision =TP TP +FP (1) Recall =TP TP +FN (2) F1=2∗P ecision ∗Recall P ecision +Recall (3) Simila ly o p ecision and ecall, we can also de ine speci- ici y as: Speci ici y =TN TN +FP (4) In he au oencode expe imen , we complemen ed he me ics wi h he a ea unde he cu e (AUC) sco e. This pa ame e is de ined as he a ea unde he ecei e ope a o cha ac e is ics (ROC) cu e, an example o which is shown in Fig. 5. This cu e is ob ained by changing he decision h eshold o a bina y classi ie by a de ined s ep and plo ing he esul ing speci ici y on he x-axis and ecall on he y-axis o each h eshold s ep. Each poin o he ROC cu e hen co esponds o one h eshold se ing. 123 101 Page 16 o 18 S. Bilik e al. Appendix C: Complemen a y R-CNN esul s See Tables 13,14 and 15. Table 13 Fas e -RCNN de ec ion esul s o Plank on s Anomalies expe imen Plank on class F1 sco e P ec Rec Aphanizomenon 0.98 1 0.96 Cen ales 0.85 0.73 1 Dolichospe mum 0.89 0.97 0.81 Chae oce os 0.80 0.89 0.73 Nodula ia 0.76 0.98 0.62 Pauliella 0.86 0.99 0.77 Pe idiniella Chain 0.60 0.75 0.5 Pe idiniella Single 0.76 1 0.62 Skele onema 0.98 0.97 1 Plank on & Anomalies 0.87 0.95 0.81 Table 14 Fas e -RCNN de ec ion esul s o Plank on s Anomalous Plank on expe imen Plank on class F1 sco e P ec Rec Aphanizomenon 0.66 0.58 0.75 Cen ales 0.60 0.47 0.82 Dolichospe mum 0.85 0.82 0.88 Chae oce os 0.74 0.71 0.78 Nodula ia 0.78 0.96 0.66 Pauliella 0.79 0.76 0.83 Pe idiniella Chain 0.67 0.67 0.67 Pe idiniella Single 0.88 0.92 0.85 Skele onema 0.94 0.92 0.94 Plank on & Anomalous plank on 0.83 0.82 0.85 Table 15 Fas e -RCNN de ec ion esul s o Anomalies expe imen Plank on class F1 sco e P ec Rec Aphanizomenon 1 1 1 Cen ales 0.77 0.67 0.91 Dolichospe mum 0.38 0.87 0.25 Chae oce os 0.67 0.66 0.68 Nodula ia 0.85 0.88 0.82 Pauliella 0.85 0.78 0.94 Pe idiniella Chain 0.44 0.67 0.33 Pe idiniella Single 0.89 0.86 0.92 Skele onema 0.83 0.75 0.93 Anomalies 0.75 0.76 0.74 Re e ences 1. Falkowski, P.G., Ba be , R.T., Sme acek, V.: Biogeochemical con ols and eedbacks on ocean p ima y p oduc ion. Science 281(5374), 200–206 (1998). h ps://doi.o g/10.1126/science.281. 5374.200 2. Reynolds, C.S.: The Ecology o Phy oplank on, Ecology, Biodi- e si y and Conse a ion. Camb idge Uni e si y P ess, Camb idge (2006). h ps://doi.o g/10.1017/CBO9780511542145 3. Su le, C.A., Chan, A.M., Co ell, M.T.: In ec ion o phy oplank- on by i uses and educ ion o p ima y p oduc i i y. Na u e 347(6292), 467–469 (1990). h ps://doi.o g/10.1038/347467a0 4. Klawonn, I., Van den Wyngae , S., Pa ada, A.E., e al.: Cha ac e - izing he “ ungal shun ”: Pa asi ic ungi on dia oms a ec ca bon low and bac e ial communi ies in aqua ic mic obial ood webs. P oc. Na l. Acad. Sci. 118(23), e2102225,118 (2021). h ps://doi. o g/10.1073/pnas.2102225118 5. Klawonn, I., Van den Wyngae , S., I e sen, M.H., e al.: Fungal pa asi ism on dia oms al e s o ma ion and bio-physical p ope ies o sinking agg ega es. Commun. Biol. 6(1), 206 (2023). h ps://doi. o g/10.1038/s42003-023-04453-6 6. Scholz, B., Guillou, L., Ma ano, A.V., e al.: Zoospo ic pa asi es in ec ing ma ine dia oms-a black box ha needs o be opened. Fun- gal Ecol. 19, 59–76 (2016). h ps://doi.o g/10.1016/j. uneco.2015. 09.002 7. Peacock, E.E., Olson, R.J., Sosik, H.M.: Pa asi ic in ec ion o he dia om guina dia delica ula, a ecu en and ecologically impo an phenomenon on he new england shel . Ma . Ecol. P og. Se . 503, 1–10 (2014). h ps://doi.o g/10.3354/meps10784 8. Lomba d,F.,Boss,E.,Wai e,A.M.,e al.:Globallyconsis en quan- i a i e obse a ions o plank onic ecosys ems. F on . Ma . Sci. 6, 196 (2019). h ps://doi.o g/10.3389/ ma s.2019.00196 9. Ren, S., He, K., Gi shick, R., e al.: Fas e -cnn: Towa ds eal- ime objec de ec ionwi h egionp oposal ne wo ks.Ad ances in neu al in o ma ion p ocessing sys ems 28 (2015) 10. Joche , G.: YOLO 5 by Ul aly ics. h ps://doi.o g/10.5281/ zenodo.3908559,h ps://gi hub.com/ul aly ics/yolo 5 (2020) 11. Bilik, S., K a och ila, L., Ligocki, A., e al.: Visual diagnosis o he a oa des uc o pa asi ic mi e in honeybees using objec de ec o echniques. Senso s 21(8), 2764 (2021). h ps://doi.o g/10.3390/ s21082764 12. Van Den Oo d, A., Vinyals, O., e al.: Neu al disc e e ep esen a- ion lea ning. In: Guyon, I., Luxbu g, U.V., Bengio, S., e al. (eds.) Ad ances in Neu al In o ma ion P ocessing Sys ems, ol. 30. Cu - an Associa es Inc, New Yo k (2017) 13. Mishchuk, A., Mishkin, D., Radeno ic, F., e al.: Wo king ha d o know you neighbo ’s ma gins: Local desc ip o lea ning loss. In: Guyon, I., Luxbu g, U.V., Bengio, S., e al. (eds.) Ad ances in Neu alIn o ma ionP ocessingSys ems, ol.30.Cu anAssocia es Inc, New Yo k (2017) 14. B eunig, M.M., K iegel, H.P., Ng, R.T., e al.: Lo : Iden i y- ing densi y-based local ou lie s. In: P oceedings o he 2000 ACM SIGMOD In e na ional Con e ence on Managemen o Da a. Associa ion o Compu ing Machine y, New Yo k, NY, USA, SIGMOD’00, pp. 93–104, (2000) h ps://doi.o g/10.1145/342009. 335388 15. Pang, G., Shen, C., Cao, L., e al.: Deep lea ning o anomaly de ec- ion: a e iew. ACM Compu . Su . 54(2), 1–38 (2021). h ps://doi. o g/10.1145/3439950 16. Hin on, G.E., Salakhu dino , R.R.: Reducing he dimensionali y o da a wi h neu al ne wo ks. Science 313(5786), 504–507 (2006). h ps://doi.o g/10.1126/science.1127647 17. Saku ada, M., Yai i, T.: Anomaly de ec ion using au oencode s wi h nonlinea dimensionali y educ ion. In: P oceedings o he MLSDA 2014 2nd Wo kshop on Machine Lea ning o Senso y 123 Towa d phy oplank on pa asi e de ec ion using au oencode s Page 17 o 18 101 Da a Analysis. Associa ion o Compu ing Machine y, New Yo k, NY, USA, MLSDA’14, pp. 4–11, (2014) h ps://doi.o g/10.1145/ 2689746.2689747 18. An, J., Cho, S.: Va ia ional au oencode based anomaly de ec ion using econs uc ion p obabili y. Spec. Lec . IE 2(1), 1–18 (2015) 19. Be gmann, P., Löwe, S., Fause , M., e al.: Imp o ing unsupe ised de ec segmen a ion by applying s uc u al simila i y o au oen- code s. In: P oceedings o he 14 h In e na ional Join Con e ence on Compu e Vision, Imaging and Compu e G aphics Theo y and Applica ions - Volume 5: VISAPP, INSTICC. SciTeP ess, pp. 372– 380 (2019). h ps://doi.o g/10.5220/0007364503720380 20. Cha e, D., Cha e, F., Ga cía, S., e al.: A p ac ical u o ial on au oencode s o nonlinea ea u e usion: axonomy, models, so - wa e and guidelines. In . Fusion 44, 78–96 (2018). h ps://doi.o g/ 10.1016/j.in us.2017.12.007 21. Pas o e, V.P., Zimme man, T.G., Biswas, S.K., e al.: Anno a ion- ee lea ning o plank on o classi ica ion and anomaly de ec ion. Sci. Rep. 10(1), 12,142 (2020). h ps://doi.o g/10.1038/s41598- 020-68662-3 22. Pu, Y., Feng, Z., Wang, Z., e al.: Anomaly de ec ion o in si u ma ine plank on images. In: P oceedings o he IEEE/CVF In e - na ional Con e ence on Compu e Vision (ICCV) Wo kshops, pp. 3661–3671 (2021) 23. Pas o e, V.P., Megiddo, N., Bianco, S.: An anomaly de ec ion app oach o plank on species disco e y. In: Scla o S, Dis an e C, Leo M, e al (eds) Image Analysis and P ocessing – ICIAP 2022. Sp inge In e na ional Publishing, Cham, pp. 599–609 (2022) h ps://doi.o g/10.1007/978-3-031-06430-2_50 24. Bad eldeenBdawyMohamed,O.,Ee ola,T.,K a ,K.,e al.:Open- se plank on ecogni ion using simila i y lea ning. In: Bebis G, Li B, Yao A, e al (eds) Ad ances in Visual Compu ing. Sp inge In e na ional Publishing, Cham, pp. 174–183 (2022). h ps://doi. o g/10.1007/978-3-031-20713-6_13 25. Zhao, W., Chen, F., Huang, H., e al.: A new s eel de ec de ec ion algo i hm based on deep lea ning. Compu . In ell. Neu osci. 2021, 1–13 (2021). h ps://doi.o g/10.1155/2021/5592878 26. Su, Y., Li, D., Chen, X.: Lung nodule de ec ion based on as e -cnn amewo k. Compu . Me hods P og ams Biomed. 200(105), 866 (2021). h ps://doi.o g/10.1016/j.cmpb.2020.105866 27. Ho ak, K., Sabla nig, R.: Deep lea ning concep s and da ase s o image ecogni ion: o e iew 2019. In: Hwang JN, Jiang X (eds) Ele en h In e na ional Con e ence on Digi al Image P ocess- ing (ICDIP 2019), In e na ional Socie y o Op ics and Pho onics, ol 11179. SPIE, p. 111791S (2019). h ps://doi.o g/10.1117/12. 2539806 28. Li, Q., Sun, X., Dong, J., e al.: De eloping a mic oscopic image da ase in suppo o in elligen phy oplank on de ec ion using deep lea ning. ICES J. Ma . Sci. 77(4), 1427–1439 (2019). h ps://doi. o g/10.1093/icesjms/ sz171 29. Chen, T., Li, J., Ju, W., e al.: Objec de ec ion and abundance anal- ysis o oun ain- low imaging o ma ine plank on. In: OCEANS 2021: San Diego - Po o, pp. 1–9 (2021). h ps://doi.o g/10.23919/ OCEANS44145.2021.9705862 30. Redmon, J., Fa hadi, A.: Yolo 3: An inc emen al imp o emen . (2018) a Xi p ep in a Xi :1804.02767 31. Gi shick, R.: Fas -cnn. In: P oceedings o he IEEE In e na ional Con e ence on Compu e Vision (ICCV), pp. 1440–1448 (2015) 32. Liu, W., Anguelo , D., E han, D., e al.: Ssd: Single sho mul ibox de ec o . In: Leibe B, Ma as J, Sebe N, e al (eds) Compu e Vision – ECCV 2016. Sp inge In e na ional Publishing, Cham, pp. 21–37 (2016) h ps://doi.o g/10.1007/978-3-319-46448-0_2 33. Bilik, S.: Ae- econs uc ion-and- ea u e-based-ad. h ps://gi hub. com/boo el/AE-Recons uc ion-And-Fea u e-Based-AD, open sou ce so wa e a ailable om h ps://gi hub.com/boo el/AE- Recons uc ion-And-Fea u e-Based-AD (2022) 34. Masci, J., Meie , U., Ci e¸san, D., e al.: S acked con olu ional au o-encode s o hie a chical ea u e ex ac ion. In: Honkela T, Duch W, Gi olami M, e al (eds) A i icial Neu al Ne wo ks and Machine Lea ning – ICANN 2011. Sp inge Be lin Heidelbe g, Be lin, Heidelbe g, pp 52–59, (2011) h ps://doi.o g/10.1007/978- 3-642-21735-7_7 35. Pu, Y., Gan, Z., Henao, R., e al.: Va ia ional au oencode o deep lea ning o images, labels and cap ions. In: Lee, D., Sugiyama, M., Luxbu g, U., e al. (eds.) Ad ances in Neu al In o ma ion P ocess- ing Sys ems, ol. 29. Cu an Associa es Inc, New Yo k (2016) 36. Bilik, S.: Fea u e space educ ion as da a p ep ocessing o he anomaly de ec ion. In: P oceedings I o he 27 h Con e ence STUDENT EEICT 2021, pp. 415–419. Facul y o Elec ical Engi- nee ing and Communica ion, B no Uni e si y o Technology, B no (2021) 37. Hou,X.,Shen,L.,Sun,K.,e al.:Deep ea u econsis en a ia ional au oencode . In: 2017 IEEE Win e Con e ence on Applica ions o Compu e Vision (WACV), pp 1133–1141, (2017) h ps://doi.o g/ 10.1109/WACV.2017.131 38. Makhzani, A., F ey, B.J.: Winne - ake-all au oencode s. In: Co es, C., Law ence, N., Lee, D., e al. (eds.) Ad ances in Neu al In o - ma ion P ocessing Sys ems, ol. 28. Cu an Associa es Inc, New Yo k (2015) 39. Bilik, S., Ho ak, K.: Si and su based ea u e ex ac ion o he anomaly de ec ion. In: P oceedings I o he 28 h Con e ence STUDENT EEICT 2022 Gene al Pape s, pp. 459–464. Facul y o Elec ical Enginee ing and Communica ion, B no Uni e si y o Technology, B no (2022) 40. Rousseeuw, P.J., D iessen, K.V.: A as algo i hm o he minimum co a iance de e minan es ima o . Technome ics 41(3), 212–223 (1999). h ps://doi.o g/10.1080/00401706.1999.10485670 41. Schölkop , B., Pla , J.C., Shawe-Taylo , J., e al.: Es ima - ing he Suppo o a High-Dimensional Dis ibu ion. Neu- al Compu . 13(7), 1443–1471 (2001). h ps://doi.o g/10.1162/ 089976601750264965 42. Liu, F.T., Ting, K.M., Zhou, Z.H.: Isola ion o es . In: 2008 Eigh h IEEE In e na ional Con e ence on Da a Mining, pp. 413–422 (2008). h ps://doi.o g/10.1109/ICDM.2008.17 43. Iglewicz, B.: Robus scale es ima o s and con idence in e als o loca ion.In:Unde s andingRobus and Explo a o y Da a Analysis, 1s edn., pp. 405–431. Wiley-In e science, New Yo k (2000) 44. Simonyan, K., Zisse man, A.: Ve y deep con olu ional ne - wo ks o la ge-scale image ecogni ion. (2014) a Xi p ep in a Xi :1409.1556 h ps://doi.o g/10.48550/a Xi .1409.1556 45. He, K., Zhang, X., Ren, S., e al.: Deep esidual lea ning o image ecogni ion. In: P oceedings o he IEEE Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR), pp 770–778 (2016) 46. Li, Y., Xie, S., Chen, X., e al.: Fas e R-CNN (ResNe 50). (2021a) h ps://py o ch.o g/ ision/main/models/gene a ed/ o ch ision. models.de ec ion. as e cnn_ esne 50_ pn_ 2.h ml# o ch ision. models.de ec ion. as e cnn_ esne 50_ pn_ 2 47. Li, Y., Xie, S., Chen, X., e al.: Benchma king de ec ion ans- e lea ning wi h ision ans o me s (2021) a Xi p ep in a Xi :2111.11429 h ps://doi.o g/10.48550/a Xi .2111.11429 48. Olson, R.J., Sosik, H.M.: A subme sible imaging-in- low ins u- men o analyze nano-and mic oplank on: Imaging lowcy obo . Limnol. Oceanog . Me hods 5(6), 195–203 (2007). h ps://doi.o g/ 10.4319/lom.2007.5.195 49. Laakso, L., Mikkonen, S., D ebs, A., e al.: 100 yea s o a mo- sphe ic and ma ine obse a ions a he innish u ö island in he bal ic sea. Ocean Sci. 14(4), 617–632 (2018). h ps://doi.o g/10. 5194/os-14-617-2018 50. K a , K., Seppälä, J., Häll o s, H., e al.: Fi s applica ion o i cb high- equency imaging-in- low cy ome y o in es iga e bloom- o ming ilamen ous cyanobac e ia in he bal ic sea. F on ie s 123 101 Page 18 o 18 S. Bilik e al. in Ma ine Science 8, (2021). h ps://doi.o g/10.3389/ ma s.2021. 594144 51. K a ,K.,Velhonoja,O.,Ee ola, T.,e al.: Towa ds ope a ionalphy- oplank on ecogni ion wi h au oma ed high- h oughpu imaging, nea - eal- ime da a p ocessing, and con olu ional neu al ne wo ks. F on ie s in Ma ine Science 9,(2022). h ps://doi.o g/10.3389/ ma s.2022.867695 52. Tkachenko, M., Malyuk, M., Holmanyuk, A., e al.: Label S udio: Da a labeling so wa e. (2020-2022) h ps://gi hub.com/ hea exlabs/label-s udio, open sou ce so wa e a ailable om h ps://gi hub.com/hea exlabs/label-s udio 53. Bilik, S., Bak akhano , D., Ee ola, T., e al.: I cb phy oplank- on anomaly da ase (i cb-pad). (2023) h ps://doi.o g/10.23729/ 08b2ac4a-a80d-4e54-85e0-ab3ea46085ec 54. Vincen , P., La ochelle, H., Bengio, Y., e al.: Ex ac ing and composing obus ea u es wi h denoising au oencode s. In: P o- ceedings o he 25 h In e na ional Con e ence on Machine Lea n- ing. Associa ion o Compu ing Machine y, New Yo k, NY, USA, ICML’08, pp. 1096–1103, (2008) h ps://doi.o g/10.1145/ 1390156.1390294 55. Abadi, M., Aga wal, A., Ba ham, P., e al.: Tenso Flow: La ge- scale machine lea ning on he e ogeneous sys ems. (2015) h ps:// www. enso low.o g/, so wa e a ailable om enso low.o g 56. Ped egosa, F., Va oquaux, G., G am o , A., e al.: Sciki -lea n: Machine lea ning in Py hon. J. Mach. Lea n. Res. 12, 2825–2830 (2011) Publishe ’s No e Sp inge Na u e emains neu al wi h ega d o ju is- dic ional claims in published maps and ins i u ional a ilia ions. Simon Bilik ecei ed he M.Sc. deg ee in echnical cybe ne ics om he Depa men o Con ol and Ins umen a ion, B no Uni e si y o Tech- nology, Czech Republic, in 2019. He is cu en ly a double deg ee PhD s uden wi h he Machine Vision G oup, BUT Uni e si y and he Com- pu e Vision and Pa e n Recogni ion Labo a o y, LUT Uni e si y. His esea ch ield includes applied machine ision and machine lea ning. Daniel Ba akhano a ained his M.Sc. deg ee in Compu a ional Engi- nee ing and Technical Physics om he School o Enginee ing Science a LUT Uni e si y, Lappeen an a, Finland in 2021. Cu en ly, he is a Junio Resea che a he LUT Compu e Vision and Pa e n Recog- ni ion Labo a o y, his ocus a eas include machine lea ning, image p ocessing, and da a analysis. Tuomas Ee ola ecei ed he M.Sc. and Ph.D. deg ees in in o ma ion p ocessing om he Depa men o In o ma ion Technology, Lappeen- an a Uni e si y o Technology, Finland, in 2006 and 2010, espec- i ely. He is cu en ly an Associa e P o essso wi h he Compu e Vision and Pa e n Recogni ion Labo a o y, LUT Uni e si y. His esea ch in e es s include digi al image p ocessing, applied compu e ision, and deep lea ning. Lumi Ha aguchi is an oceanog aphe specialized in phy oplank on and mic ozooplank on ecology. She ecei ed he PhD in Biosciences om Aa hus Uni e si y (DK) in 2018 and is cu en ly wo king a he Finnish En i onmen al Ins i u e. Kaisa K a wo ks in Ma ine Ecology Measu emen s uni o he Finnish En i onmen Ins i u e. He wo k includes high- equency measu e- men s o phy oplank on communi y wi h main ocus on imaging. She wo ks on implemen a ion o he imaging echnology o Bal ic Sea phy- oplank on communi y and he consequen image analysis. He special in e es is in phy oplank on bloom dynamics and he ela ed en i on- men al o cing. Silke Van den Wyngae is an aqua ic mic obial ecologis and ecei ed he Ph.D. deg ee om he Swiss Fede al Ins i u e o Technology (ETHZ), Swi ze land, in 2013. She is cu en ly an Academy Resea ch ellow a he Depa men o Biology, Uni e si y o Tu ku, Finland. He esea ch in e es s include plank on ecology, in pa icula di e si y o aqua ic ungi and hei in e ac ions wi h phy oplank on. Jonna Kangas g adua ed as a MSc in En i onmen al Sciences in 2022 om Uni e si y o Helsinki. He hesis p ojec ocused on he e ec s o salini y change on he mic obial loop in he Bal ic Sea. Conny Sjöq is g adua ed as a PhD in Ma ine Biology in 2015 om Åbo Akademi Uni e si y (ÅAU), Finland. He is cu en ly wo king as a esea che in plank on ecology a ÅAU. He was awa ded he Ti le o Docen ship in Molecula and E olu iona y Ma ine Ecology in 2022. Ka in Madsén g adua ed as a MSc. in En i onmen al and Ma ine Biol- ogy in 2023 om Åbo Akademi Uni e si y. He hesis p ojec ocused on de ec ion o phy oplank on pa asi es using imaging low cy ome y. Lasse Lensu is a p o esso o machine ision and da a analysis a Lappeen an a-Lah i Uni e si y o Technology LUT, Finland. He ecei ed his D.Sc. (Tech.) deg ee in compu e science and enginee ing in 2002 om he Depa men o In o ma ion Technology o LUT. His esea ch in e es s include machine/compu e ision, pa e n ecogni- ion wi h machine lea ning and da a analysis. P o . Lensu is a membe o he Compu e Vision and Pa e n Recogni ion Labo a o y o he Depa men o Compu a ional Enginee ing a LUT, and he has con- ibu ed o he echnology ans e o h ee spin-o companies om he uni e si y. Heikki Käl iäinen is a ull p o esso o Compu e Science and Engi- nee ing a he Lappeen an a-Lah i Uni e si y o Technology LUT, School o Enginee ing Science, Finland. P o . Käl iäinen is a head o he Compu e Vision and Pa e n Recogni ion Labo a o y a he Depa men o Compu a ional Enginee ing. His esea ch in e es s include compu e ision, pa e n ecogni ion, machine lea ning, and especially applica ions o digi al image p ocessing and analysis. Besides LUT, P o . Käl iäinen has been a P o esso o Compu ing a Monash Uni e si y Malaysia, and a Visi ing P o esso a B no Uni e - si y o Technology in Czech Republic, Czech Technical Uni e si y a P ague, and Uni e si y o Su ey in UK. Ka el Ho ak ecei ed he M.Sc. and Ph.D. deg ees in echnical cybe - ne ics om he Depa men o Con ol and Ins umen a ion, B no Uni- e si y o Technology, Czech Republic, in 2004 and 2008, espec- i ely. He is cu en ly an Associa e P o essso wi h he Machine Vision G oup, BUT Uni e si y. His esea ch in e es s include machine ision, image p ocessing and machine lea ning. 123