IFAC Pape sOnLine 54-16 (2021) 76–82
ScienceDi ec
A ailable online a www.sciencedi ec .com
2405-8963 Copy igh © 2021 The Au ho s. This is an open access a icle unde he CC BY-NC-ND license
.
Pee e iew unde esponsibili y o In e na ional Fede a ion o Au oma ic Con ol.
10.1016/j.i acol.2021.10.076
10.1016/j.i acol.2021.10.076 2405-8963
Copy igh ©
2021 The Au ho s. This is an open access a icle unde he CC BY-NC-ND license
(
h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/
)
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82 77
Copy igh ©
2021 The Au ho s. This is an open access a icle unde he CC BY-NC-ND license
(
h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/
)
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
Deep Lea ning based De ec ion, Segmen a ion and Coun ing o Ben hic
Mega auna in Uncons ained Unde wa e En i onmen s
Mona Lü jens* Ha ald S e nbe g*
*Ha enCi y Uni e si y Hambu g, Depa men o Hyd og aphy and Geodesy,
Henning-Vosche au-Pla z 1, 20457 Hambu g, Ge many
(e-mail: {mona.lue jens,ha ald.s e nbe g}@hcu-hambu g.de)
Abs ac : Assessing and moni o ing ben hic communi ies is inc easingly impo an in iew o global
al e a ion o ma ine en i onmen s. Deep lea ning has p o en o e ec i ely de ec ma ine specimen in
unde wa e image y bu s ill ace p oblems wi h small inpu da ase s, uncons ained en i onmen s and class
imbalance. This s udy e alua es a da a augmen a ion s a egy o alle ia e hese limi a ions. Th ough
syn he ically de i ed image composi ions, he en i e inpu da ase was g ea ly ex ended om 700 o 12700
images. Addi ionally, specimen numbe s o b i le s a s, so co als and glass sponges a e equalized
esul ing in a mean a e age p ecision inc ease o 24 %. The o e all mean a e age p ecision o box
de ec ions yields 76.7 and o ins ance segmen a ion 67.7 a an in e sec ion o e union h eshold o 0.5.
This s udy shows ha deep a chi ec u es such as he deployed Cen e Mask ia ResNeX -101 model can
success ully be ained wi h ew o iginal images om a ying unde wa e scenes.
Keywo ds: objec de ec ion, deep lea ning, da a augmen a ion, ma ine image y, ben hic mega auna
1. INTRODUCTION
Global al e a ion o ma ine en i onmen s due o o e ishing,
pollu ion, physical habi a des uc ion and clima e change ha e
led o an inc easing decline o animal di e si y and abundance
h oughou many ma ine ecosys ems (Jackson, 2008).
Especially ben hic mega auna in he Sou he n Ocean a e a isk
o en i onmen al change and a e o signi ican ecological
alue as hey al e small-scale opog aphy o seabed habi a s
a ec ing he en i e ben hic communi y (Gili e al., 2006).
Assessing he biodi e si y and cha ac e isa ion o ben hic
communi ies is inc easingly impo an o iden i ying
ulne able ma ine ecosys ems and de eloping conse a ion
s a egies.
Ma ine habi a s ha e been s udied based on mainly h ee
p ac ices: physical sampling me hods using sledges o
awling, acous ical echniques and op ical sys ems. While
physical me hods a e able o collec specimen on a lowe
axonomical scale hey des uc he en i onmen and hei
sampling a e is a he low. Op ical sys ems a e mo e cos
e ec i e, obus and p ecise han acous ic sys ems which led
o a la ge g owing lib a y o digi al unde wa e images
h oughou ecen yea s pa ing he way o new esea ch in
au oma ic analy ical me hods.
Objec de ec ion, classi ica ion and segmen a ion o image y
has become a subs an ial ask in he ield o compu e ision.
P io adi ional ea u e desc ip o s a e o en used o ex ac
colou , shape o ex u e in o ma ion and a e good a de ec ing
speci ic single objec s such as scallops (Dawkins e al., 2013)
o lobs e s using classi ie s such as suppo ec o machine
(Tan e al., 2018). Howe e , hey a e no obus o a ying
unde wa e scenes ha a e exposed o ma ine snow, wa e
u bidi y, lens dis o ion, spa se, uns able illumina ion and
colou shi due o he su ey pla o m’s a ia ion in speed,
angle o al i ude. Mo eo e , he e is a high a iabili y and
in a iabili y be ween ea u es belonging o he same class and
di e en classes, espec i ely (Pa oni e al., 2021).
Deep lea ning (DL) using con olu ional neu al ne wo ks ha e
p o en o ou pe o m adi ional based objec de ec ion
(Gonzalez-Cid e al., 2017) as hey a e mo e in a ian o he
de o ma ion o images. Addi ionally, images can di ec ly be
used as inpu wi hou he necessi y o p e-p ocessing. Be e
esul s a e o en achie ed using deepe laye s hus inc easing
he numbe o pa ame e s o se e al millions. To achie e high
pe o mances wi h hose models using images o
uncons ained unde wa e scenes o ac oss a ying pla o ms,
a la ge aining da ase size is he mos c ucial pa
(Langenkämpe e al., 2020), o en, howe e , e y ime-
consuming and cos ly o es ablish.
This pape in es iga es he e ec o inpu image da a
augmen a ion and composi ion s a egies in an a emp o
o e come limi a ions o la ge da a se size and class
imbalance. Taking accoun o he esul s, he s a e-o - he a
ancho - ee objec de ec ion and ins ance segmen a ion model
Cen e Mask (Lee and Pa k, 2019) ia ResNeX -101 (Xie e al.,
2017) will be ained based on a small, highly di e se 700
image da ase . In his wo k, he de ec ion, segmen a ion and
coun ing o glass sponges (hexac inellids), so co als
(p imnoids and ch ysogo giids) and b i le s a s (ophiu oids)
will be assessed p o iding i s s eps owa ds u u e abundance
and size es ima ions o hese specimen.
2. RELATED RESEARCH
Se e al p e ious wo ks add ess objec de ec ion and
classi ica ion o ma ine scenes using cu ing-edge DL
a chi ec u es such as Re inaNe wi h ResNe -50 (Boulais e al.,
2020) o FDCNe (Lu e al., 2018) showing good classi ica ion
esul s on single-labelled o iconic images. Au oma ic
segmen a ion o ben hic auna has been s udied o co als
using DeepLab 3+ (Pa oni e al., 2021) and scale wo ms
using U-Ne and VGG-16 CNN (Shashidha a e al., 2020).
Secu ing enough aining da a o DL is c ucial as desc ibed
abo e hence da a augmen a ion echniques a e widely applied.
F equen ly used echniques include he change o ligh
in ensi y, sha pness, noise and blu ing (Salman e al., 2016)
o change o pe spec i e (Huang e al., 2019). Also, o a ion
and c opping o unde wa e images ha e been used
(Langenkämpe e al., 2020). These me hods ha e p o en o
be use ul, howe e , only limi ed image manipula ions can be
pe o med. Mos no ably, he numbe o o iginal anno a ed
images om mos s a ed wo ks exceed he amoun o a ailable
aining da a o his esea ch. The e o e, ano he echnique
will be used which changes he en i e image composi ion and
adds addi ional al e a ion o syn he ically gene a ed images.
Also, no esea ch ega ding ins ance segmen a ion and
coun ing o he selec ed mo pho ypes is known o he au ho s.
3. MATERIALS AND METHODS
3.1 Unde wa e Image Da ase
The image da ase used o his s udy was collec ed du ing he
expedi ion PS118 o he esea ch essel RV Pola s e n in 2019
(Pu se e al., 2021). Images we e sampled using he owed
Ocean Floo Obse a ion and Ba hyme y Sys em (Pu se e
al., 2019) wi h a lying al i ude o app oxima ely 1.5 – 2.5 m
abo e he sea loo . Se en di e en sampling s a ions om he
wes e n Weddell Sea con inen al shel o he no he n Powell
Basin we e selec ed. Each s a ion a ea ea u es di e en
subs a e ypes anging om so and ine mud sedimen o
pebbles and complex ocky opog aphy. The o iginal 3840 x
5760 sized images we e iled o 1440 x 960 o main ain
esolu ion bu educe he need o compu ing powe . 1000
images we e selec ed and anno a ed using he web-based
image segmen a ion ool COCO Anno a o (B ooks, 2019). O
he 1000 images, 700 we e used as aining se , 100 as
alida ion se and 200 as es se . In o al, 3550 anno a ions o
he aining se we e made, o which 87 % belong o he class
b i le s a s, 8 % o he class glass sponges and 5 % o he class
so co als. Fo he es and alida ion se 85 % and 84 % o
he anno a ions belong o he class b i le s a s, 10 % and
12 % o he class glass sponges and 5 % and 4 % o he class
so co als, espec i ely. I is appa en ha he e is a high class
imbalance.
3.2 Da a Augmen a ion
To inc ease he numbe o images o aining, he image
gene a o COCO Syn h (Kelly, 2019) was u ilised which
composes cu ou o eg ound images o objec s o e andom
image backg ounds. Fo eg ounds a e andomly al e ed in
scale, amoun , o a ion and b igh ness o each composi ion
(Figu e 1). Fo his s udy, 30 o eg ounds pe class and 30
backg ounds om o iginal images we e used o aining.
Images o he composi ions we e selec ed om a ying
s a ions and di e om p e ious ones in 3.1. O e all, 12,000
syn he ic images we e deployed o aining o which 2000
images we e gene a ed o glass sponges and so co als each,
o educe he e ec o class imbalance. Now, 33 % o he o al
anno a ions belong o he class glass sponges, 33 % o he class
so co als and 34 % o he class b i le s a s. In o de o
emphasize he selec ed augmen a ion me hod, se e al
equen ly used image manipula ion echniques we e
addi ionally pe o med and compa ed o he selec ed me hod.
The ollowing image a ibu es we e al e ed: b igh ness, colou
one, con as , sa u a ion, sha pness and blu (Figu e 1). In
o al, en di e en al e a ions pe image we e conduc ed.
Fig. 1. Example images o syn he ically de i ed image composi ions (1s ow) and adi ional da a augmen a ion me hods (2nd
ow, om le o igh ): o iginal image, blu , low b igh ness, high b igh ness, blue colou , g een colou , high con as , low
con as , high sa u a ion, low sa u a ion, sha pness
78 Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82
3.3 Deep Lea ning A chi ec u e and T aining
The deep lea ning a chi ec u e chosen o his s udy is he
ancho - ee one s age ins ance segmen a ion and objec
de ec o Cen e Mask (Lee and Pa k, 2019) in combina ion
wi h he backbone ne wo k ResNeX -101 (Xie e al., 2017).
While objec de ec ion is he cen al ask o abundance and
assemblage s udies, p edic ing masks will be impo an o
u u e size es ima ions and biomass p edic ions o ben hic
species. An ins ance segmen a ion model was he e o e chosen
in o de o adap o di e se esea ch ques ions o u u e
analyses. Also, he model should ope a e on high in e ence
speed while main aining s ong pe o mances o di ec ly
e alua e da ase s on boa d o esea ch essels o s a egic da a
collec ion. Since bo h selec ed a chi ec u es mee he
men ioned equi emen s and u he p oduce excellen esul s
in ecen benchma k challenges such as COCO (Lin e al.,
2014), hey a e an app op ia e choice o he espec i e
compu e ision ask o ma ine imaging.
The backbone ResNeX -101 used o ea u e ex ac ion is an
ad ancemen o he deep esidual ne wo k ResNe (He e al.,
2016) ha has been ecen ly p oposed. Based on ResNe ,
ResNeX -101 ollows he s a egy o epea ing laye s bu
s acks hem pa allel a he han sequen ially. Thus, esul ing in
accu acy imp o emen s while educing he ne wo k
complexi y and numbe o pa ame e s. Fo his s udy he 101
laye ed ne wo k was used.
As a one s age de ec o , Cen e Mask does no ha e a p oposal
s ep and p io i izes in e ence speed. Addi ionally, as an
ancho - ee de ec o , i does no use p ede ined bounding
boxes o iden i y objec s and is hus insensi i e o di e en
da ase s and hype -pa ame e s (e.g. inpu size, scales, e c.).
Hence, ancho - ee de ec o s alle ia e limi a ions o objec s
ha ha e la ge shape a ia ions and a e a he small (Tian e
al., 2019) which is ideal o he selec ed classes used in his
esea ch. Cen e Mask adop s FCOS (Tian e al., 2019) as
de ec ion head ha di ec ly compu es a 4D ec o and a class
label a each p oposed loca ion o di e en le els o ea u e
maps. Then, he spa ial a en ion-guided mask (SAG-Mask)
compu es he segmen a ion masks on each p edic ed box
egion using he spa ial a en ion module (SAM) ha helps he
mask o ocus on signi ican pixels (Lee and Pa k, 2019).
T aining was execu ed on a 64-bi Linux machine equipped
wi h an In el® Xeon® Gold 6254 CPU @ 3.10 GHz and 5
NVIDIA® Tesla® V100 GPU. The base lea ning a e was se o
0.002 and educed by a ac o o 10 a e 25400 and again a e
38100 i e a ions. To educe ea ly o e i ing on highly
di e en ia ed da ase s, he lea ning a e was also educed o
he i s 5080 i e a ions. Addi ionally, a weigh decay was
implemen ed. The maximum numbe o i e a ions was 50800
which co esponds o 20 epochs. All backbone models a e
ini ialized by ImageNe p e- ained weigh s.
3.4 E alua ion P o ocol
To assess he pe o mance o he model, he e alua ion me ics
a e age p ecision, a e age ecall, F1 measu e and accu acy a e
u ilized. While he p ecision P e lec s he p opo ion o alse
posi i es FP, he ecall R de ines he p opo ion o alse
nega i es FN and can be ma hema ically exp essed as ollows:
P
= TP
(TP + FP) and
R
= TP
(TP + FN) , (1
)
wi h TP being he numbe o ue posi i e p edic ions. In o de
o classi y whe he a p edic ion is a TP o FP, he in e sec ion
o e union (IoU) h eshold is used as i measu es he o e lap
be ween he g ound u h and he p edic ed bounding box o
segmen a ion mask, espec i ely. Typical alues o IoU
h esholds a e 0.5 o 0.75. The e y common app oach o
summa ize p ecision and ecall in o one alue is he a e age
p ecision (AP). The AP o a single class is he a e aged
p ecision ac oss all ecall le els. Complemen a ily, he
a e age ecall sco e (AR) a e ages ecall alues o e all IoU
∈ [0.5, 1.0] o each class. The mean a e age p ecision (mAP)
and mean a e age ecall (mAR) ac oss all classes C a e de ined
as (Raphael e al., 2020):
𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 = 1
𝐶𝐶𝐶𝐶�𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑖𝑖𝑖𝑖
𝐶𝐶𝐶𝐶
𝑖𝑖𝑖𝑖=1
and
mAR
= 1
𝐶𝐶𝐶𝐶�𝑚𝑚𝑚𝑚𝐴𝐴𝐴𝐴𝑖𝑖𝑖𝑖
𝐶𝐶𝐶𝐶
𝑖𝑖𝑖𝑖=1
(2)
As he e alua ion me ics used o his esea ch is based on
COCO (Lin e al., 2014), i should be no ed ha mAP and mAR
sco es a e u he deno ed as AP and AR o simplici y
easons. They a e compu ed o e single (0.5) IoU o he
a e age o hen IoU le els s a ing om 0.5 o 0.95 in s eps o
0.05 ( he la e is u he deno ed as AP @.50:.95). AP and AR
a e also calcula ed o di e en objec scales (small: < 72²
pixels, medium: > 72² & < 214² pixels, la ge: > 214² pixels)
and o di e en maximum numbe o de ec ions pe image (1,
10, 100). Objec scales de ia e om COCO and a e adjus ed
o i scales in he p oposed images.
Addi ional adop ed pe o mance me ics a e he accu acy o
assess he o al numbe o p edic ions ha a e co ec and he
F1 measu e which e enly weighs be ween p ecision and ecall
(Manning e al., 2009):
accu acy
= TP + TN
(TP + TN + FP + FN) 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝐹𝐹𝐹𝐹1= 2𝑚𝑚𝑚𝑚𝐴𝐴𝐴𝐴
𝑚𝑚𝑚𝑚+𝐴𝐴𝐴𝐴 (3)
4. EXPERIMENTAL RESULTS
This sec ion p esen s he pe o mance e alua ion o di e en
in es iga ed me hods o de ec , segmen and coun ben hic
mega auna ac oss a ying models and da ase s as lis ed in
Table 1. All es uns a e pe o med on he 200 o iginal image
es se wi h no da a augmen a ion.
Table 1. Abb e ia ions o aining s a egies
CM-X-101
Cen e Mask and ResNeX -101
CM-V-99
Cen e Mask and VoVNe V2-99
CM-L-M
Cen e Mask-Li e and MobileNe V2
M-X-101
Mask R-CNN and ResNeX -101
R-X-101
Re inaNe and ResNeX -101
Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82 79
3.3 Deep Lea ning A chi ec u e and T aining
The deep lea ning a chi ec u e chosen o his s udy is he
ancho - ee one s age ins ance segmen a ion and objec
de ec o Cen e Mask (Lee and Pa k, 2019) in combina ion
wi h he backbone ne wo k ResNeX -101 (Xie e al., 2017).
While objec de ec ion is he cen al ask o abundance and
assemblage s udies, p edic ing masks will be impo an o
u u e size es ima ions and biomass p edic ions o ben hic
species. An ins ance segmen a ion model was he e o e chosen
in o de o adap o di e se esea ch ques ions o u u e
analyses. Also, he model should ope a e on high in e ence
speed while main aining s ong pe o mances o di ec ly
e alua e da ase s on boa d o esea ch essels o s a egic da a
collec ion. Since bo h selec ed a chi ec u es mee he
men ioned equi emen s and u he p oduce excellen esul s
in ecen benchma k challenges such as COCO (Lin e al.,
2014), hey a e an app op ia e choice o he espec i e
compu e ision ask o ma ine imaging.
The backbone ResNeX -101 used o ea u e ex ac ion is an
ad ancemen o he deep esidual ne wo k ResNe (He e al.,
2016) ha has been ecen ly p oposed. Based on ResNe ,
ResNeX -101 ollows he s a egy o epea ing laye s bu
s acks hem pa allel a he han sequen ially. Thus, esul ing in
accu acy imp o emen s while educing he ne wo k
complexi y and numbe o pa ame e s. Fo his s udy he 101
laye ed ne wo k was used.
As a one s age de ec o , Cen e Mask does no ha e a p oposal
s ep and p io i izes in e ence speed. Addi ionally, as an
ancho - ee de ec o , i does no use p ede ined bounding
boxes o iden i y objec s and is hus insensi i e o di e en
da ase s and hype -pa ame e s (e.g. inpu size, scales, e c.).
Hence, ancho - ee de ec o s alle ia e limi a ions o objec s
ha ha e la ge shape a ia ions and a e a he small (Tian e
al., 2019) which is ideal o he selec ed classes used in his
esea ch. Cen e Mask adop s FCOS (Tian e al., 2019) as
de ec ion head ha di ec ly compu es a 4D ec o and a class
label a each p oposed loca ion o di e en le els o ea u e
maps. Then, he spa ial a en ion-guided mask (SAG-Mask)
compu es he segmen a ion masks on each p edic ed box
egion using he spa ial a en ion module (SAM) ha helps he
mask o ocus on signi ican pixels (Lee and Pa k, 2019).
T aining was execu ed on a 64-bi Linux machine equipped
wi h an In el® Xeon® Gold 6254 CPU @ 3.10 GHz and 5
NVIDIA® Tesla® V100 GPU. The base lea ning a e was se o
0.002 and educed by a ac o o 10 a e 25400 and again a e
38100 i e a ions. To educe ea ly o e i ing on highly
di e en ia ed da ase s, he lea ning a e was also educed o
he i s 5080 i e a ions. Addi ionally, a weigh decay was
implemen ed. The maximum numbe o i e a ions was 50800
which co esponds o 20 epochs. All backbone models a e
ini ialized by ImageNe p e- ained weigh s.
3.4 E alua ion P o ocol
To assess he pe o mance o he model, he e alua ion me ics
a e age p ecision, a e age ecall, F1 measu e and accu acy a e
u ilized. While he p ecision P e lec s he p opo ion o alse
posi i es FP, he ecall R de ines he p opo ion o alse
nega i es FN and can be ma hema ically exp essed as ollows:
P
= TP
(TP + FP) and
R
= TP
(TP + FN) , (1
)
wi h TP being he numbe o ue posi i e p edic ions. In o de
o classi y whe he a p edic ion is a TP o FP, he in e sec ion
o e union (IoU) h eshold is used as i measu es he o e lap
be ween he g ound u h and he p edic ed bounding box o
segmen a ion mask, espec i ely. Typical alues o IoU
h esholds a e 0.5 o 0.75. The e y common app oach o
summa ize p ecision and ecall in o one alue is he a e age
p ecision (AP). The AP o a single class is he a e aged
p ecision ac oss all ecall le els. Complemen a ily, he
a e age ecall sco e (AR) a e ages ecall alues o e all IoU
∈ [0.5, 1.0] o each class. The mean a e age p ecision (mAP)
and mean a e age ecall (mAR) ac oss all classes C a e de ined
as (Raphael e al., 2020):
𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚 = 1
𝐶𝐶𝐶𝐶�𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑚𝑖𝑖𝑖𝑖
𝐶𝐶𝐶𝐶
𝑖𝑖𝑖𝑖=1
and
mAR
= 1
𝐶𝐶𝐶𝐶�𝑚𝑚𝑚𝑚𝐴𝐴𝐴𝐴𝑖𝑖𝑖𝑖
𝐶𝐶𝐶𝐶
𝑖𝑖𝑖𝑖=1
(2)
As he e alua ion me ics used o his esea ch is based on
COCO (Lin e al., 2014), i should be no ed ha mAP and mAR
sco es a e u he deno ed as AP and AR o simplici y
easons. They a e compu ed o e single (0.5) IoU o he
a e age o hen IoU le els s a ing om 0.5 o 0.95 in s eps o
0.05 ( he la e is u he deno ed as AP @.50:.95). AP and AR
a e also calcula ed o di e en objec scales (small: < 72²
pixels, medium: > 72² & < 214² pixels, la ge: > 214² pixels)
and o di e en maximum numbe o de ec ions pe image (1,
10, 100). Objec scales de ia e om COCO and a e adjus ed
o i scales in he p oposed images.
Addi ional adop ed pe o mance me ics a e he accu acy o
assess he o al numbe o p edic ions ha a e co ec and he
F1 measu e which e enly weighs be ween p ecision and ecall
(Manning e al., 2009):
accu acy
= TP + TN
(TP + TN + FP + FN) 𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎𝑎 𝐹𝐹𝐹𝐹1= 2𝑚𝑚𝑚𝑚𝐴𝐴𝐴𝐴
𝑚𝑚𝑚𝑚+𝐴𝐴𝐴𝐴 (3)
4. EXPERIMENTAL RESULTS
This sec ion p esen s he pe o mance e alua ion o di e en
in es iga ed me hods o de ec , segmen and coun ben hic
mega auna ac oss a ying models and da ase s as lis ed in
Table 1. All es uns a e pe o med on he 200 o iginal image
es se wi h no da a augmen a ion.
Table 1. Abb e ia ions o aining s a egies
CM-X-101
Cen e Mask and ResNeX -101
CM-V-99
Cen e Mask and VoVNe V2-99
CM-L-M
Cen e Mask-Li e and MobileNe V2
M-X-101
Mask R-CNN and ResNeX -101
R-X-101
Re inaNe and ResNeX -101
Baseline
O iginal da ase
Syn h
Syn he ic image composi ion wi h equal class
dis ibu ion & Baseline
Syn h-Blncd
Syn he ic image composi ion including ex a
glass sponges and so co als
& Baseline
(balanced aining da a se )
T ad. Augm.
T adi ional augmen a ion using b igh ness
(high
, low), colou one (g een, blue), con as
(
high, low), sa u a ion (high, low), sha pness,
blu & Baseline
Fusion
Syn h-Blncd wi h 6,000 images and T ad.
Augm.
wi h 7,000 images combined &
Baseline
4.1 E alua ion o de ec ion, segmen a ion and coun ing
Fo u u e ben hic assemblage and abundance analyses, he
p ecision sco e as well as ecall a e bo h equally impo an as
specimen should nei he be misclassi ied no missed. As can
be seen in Table 2, he ne wo k CM-X-101 ained on Syn h-
Blncd shows he highes bounding box esul s o 76.7 % AP
@.50 and 59 % AR @10 compa ed o all o he me hods.
De ec ions a e accu a e on a ious backg ounds, illumina ion,
came a dis ances and dis o ions (Figu e 2) con i ming ha
au oma ic de ec ion and classi ica ion me hods a e possible
e en on small highly di e se inpu da ase s. Also, high AP and
AR sco es on segmen a ion masks a e achie ed, yielding 67.7
% @.50 and 49.1 % @10, espec i ely (Table 2). Pe o mance
o bounding boxes a e sligh ly highe han ins ance
segmen a ion masks because e y coa se objec bounda ies a e
d awn on each objec including also many i ele an pixels.
Ins ance segmen a ion assigns only objec ele an pixels o a
label and is he e o e compu a ionally mo e ad anced.
I can be u he no ed, ha he ecall a e o single objec s pe
image is much lowe han he ecall o mul iple objec s pe
image. This accoun s o bo h segmen a ion masks and
bounding boxes. Addi ionally, smalle objec s <72² each
poo e esul s han la ge objec s which migh be caused by he
downsampling in he ResNeX backbone esul ing in ewe
ea u es being ex ac ed. Ano he ac o is he ela i ely la ge
a io be ween pixel size and objec size o small objec s which
migh o en lead o posi ioning e o s when compu ing IoU.
The pe o mance o he model on di e en classes can be
u he in es iga ed wi h a con usion ma ix (Table 3) which
is compu ed using a lowe IoU h eshold o a ou a high ecall
especially o small objec s. A i s , i can be seen ha no
specimen a e w ongly classi ied be ween classes. B i le s a s
ha e he highes p ecision o 99 % and so co als ha e he
highes ecall o 93 %. On he o he hand, glass sponges and
b i le s a s a e o en no de ec ed (high FN) and glass sponges
and so co als a e o en misclassi ied wi h backg ound clu e
(high FP). Using he alues o TP, FP and FN, he accu acy
amoun s o 87 %, 73 % and 58 % o b i le s a s, so co als
and glass sponges, espec i ely, esul ing in a mean accu acy
o 73 %.
In o al, 76 ou o 82 glass sponges we e coun ed, 49 ou o 41
so co als and 597 ou o 673 b i le s a s leading o a
pe cen age a ia ion o -7 %, 20 % and -11 % o each
espec i e class.
4.2 E alua ion o da a augmen a ion s a egies
The de ec ion pe o mance o ma ine o ganisms imp o es in
bo h cases using ei he he da a augmen a ion s a egy o
syn he ic image composi ions o he adi ional image
manipula ion echniques. In ac , wi h he Syn h-Blncd da ase ,
he bounding box AP @.50:.95 esul was he mos imp o ed
wi h an inc ease o 24 % o e he Baseline da ase .
Gene ally, AP and AR on bounding boxes a e sligh ly highe
on syn he ically gene a ed image composi ions han adi ional
augmen a ion me hods (Table 2). The disad an age o he
la e is ha no as many images can be c ea ed wi hou he isk
o o e i ing as objec composi ions a e no changing.
Syn he ic gene a ed images ha e p o en o be a success ul
al e na i e and can be gene a ed in a sho amoun o ime as
well. Howe e , wi h ega ds o ins ance segmen a ion
pe o mance, syn he ic image da ase s show in e io esul s on
AP sco es especially o small and medium sized objec s
(Table 2). The eason o his migh be he inco ec
downsizing o o eg ounds as hey could appea pixela ed in
he p ocess o image c ea ion.
Conside ing he p oblem o class imbalance, syn he ic de i ed
images ha e he po en ial o easily balance ou numbe s o
specimen be ween classes. Table 4 shows ha he de ec ion
pe o mance o glass sponges and so co als a e inc eased by
5-16 % o e he Syn h da ase . The pe o mance o b i le
s a s, howe e , is no imp o ed.
Fig. 2. Example images o de ec ion and segmen a ion esul s o es se wi h CM-X-101/Syn h-Blncd a a ying s a ions.
80 Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82
4.3 Compa ison wi h o he s a e-o - he-a algo i hms
The selec ed Cen e Mask ia ResNeX -101 a chi ec u e was
u he compa ed o o he s a e-o - he a de ec o s and
backbones such as Re inaNe (Lin e al., 2017), Mask R-CNN
(He e al., 2017), MobileNe V2 (Sandle e al., 2018) o
VoVNe V2-99 (Lee and Pa k, 2019). Resul s a e demons a ed
in Table 2 o bounding boxes and segmen a ion masks. Wi h
ega ds o compe ing de ec o s, bo h Cen e Mask and
Re inaNe a e one s age de ec o s whe eas Mask R-CNN is a
wo s age de ec o ha u ilizes a egion-o -in e es p oposal
s ep which ypically p io i izes de ec ion accu acy o e
in e ence speed. Mo eo e , Re inaNe and Mask R-CNN use
ancho boxes o ea u e de ec ion. F om he esul s in Table 2
i is e iden ha Re inaNe pe o ms nea ly as well as
Cen e Mask whe eas Mask R-CNN shows a much lowe
pe o mance conside ing bounding boxes and segmen a ion
masks. I can be no ed ha one s age ancho - ee de ec o s can
pe o m jus as well on unde wa e image y as o he de ec o
ypes.
Fu he mo e, e alua ions on h ee di e en ly deep backbones
we e conduc ed: ResNeX -101 wi h 114.3 million pa ame e s,
VoVNe -99 wi h 96 million pa ame e s and MobileNe V2
wi h 28.7 million pa ame e s (Lee and Pa k, 2019).
No iceably, MobileNe V2 wi h ewe laye s has conside ably
lowe AP and AR esul s han he o he wo. Meanwhile,
conside ing bounding boxes, VoVNe -99 pe o ms nea ly as
well as ResNeX -101 and wi h ega ds o small objec s, AP
and AR esul s a e e en sligh ly highe . The p oblem o
de ec ing small objec s is known o inc ease o e y deep
backbones as hey need mo e inpu in o ma ion o cope wi h
he massi e amoun o pa ame e s (Nguyen e al., 2020). Small
objec s a e desc ibed by ewe pixels and migh no be di e se
enough o eed he ne wo k wi h su icien in o ma ion
inc easing he changes o o e i ing.
Table 2. Summa y o de ec ion esul s wi h bounding boxes (1s ow) and segmen a ion masks (2nd ow/cu si e)
Table 3. Con usion ma ix o CM-X-101/Syn h-Blncd
Table 4. Summa y o pe o mance esul s pe class
Model/Da a
AP.50:.95
bbox
AP.50
bbox
APsmall
bbox
APmedium
bbox
APla ge
bbox
AR1
bbox
AR10
bbox
AR100
bbox
ARsmall
bbox
ARmedium
bbox
ARla ge
bbox
CM-X-101/
Baseline
41.7
35.2
68.2
63.0
25.3
5.70
29.3
21.2
54.7
50.7
21.6
19.4
51.6
44.4
55.2
45.2
25.4
8.90
45.1
32.7
70.8
57.9
CM-X-101/
Syn h
48.8
37.3
71.0
63.0
27.4
4.90
39.1
24.1
62.8
51.7
24.7
20.6
58.8
47.9
64.2
49.5
27.9
7.90
57.3
41.2
77.1
59.5
CM-X-101/
Syn h
-
Blncd
51.8
40.8
76.7
67.7
27.5
4.60
40.2
27.1
66.1
54.8
25.7
22.1
59.0
49.1
63.9
50.6
27.9
7.50
55.7
41.6
77.9
62.5
CM-X-101/
T ad. Augm
48.8
40.2
75.0
70.4
26.9
7.00
38.6
27.6
58.5
52.2
23.0
20.4
55.3
46.8
58.9
47.9
27.2
10.6
50.1
36.8
72.6
58.6
CM-X-101/
Fusion
51.7
41.5
74.1
69.7
27.1
6.60
42.1
29.0
65.1
53.9
24.9
21.7
57.6
47.0
61.6
48.4
27.5
10.6
52.2
37.1
77.0
59.3
CM-V-99/
Syn h
47.9
36.9
72.0
62.9
27.9
4.80
37.0
23.4
62.8
49.8
23.6
20.1
56.6
46.1
61.9
47.8
28.3
7.60
52.6
38.6
77.1
58.2
CM-L-M/
Syn h
27.3
18.4
48.6
32.6
19.1
0.60
19.0
6.30
40.0
34.6
18.3
15.2
39.1
31.8
43.7
33.4
20.0
2.00
34.4
20.3
59.5
50.5
M-X-101/
Syn h
33.3
25.7
53.2
41.6
13.2
1.70
22.7
11.8
53.0
37.9
20.7
15.2
39.2
31.8
40.0
31.9
13.2
3.80
30.6
20.5
60.7
43.6
R-X-101/
Syn h
47.8 70.7 27.9 37.1 62.2 24.2 56.6 61.9 28.4 53.8 76.7
P edic ed
Glass Sponges
So Co als
B i le S a s
Backg ound
Recall
G ound u h
Glass Sponges
58
0
0
24
70.1
So Co als
0
38
0
3
92.7
B i le S a s
0
0
591
82
87.8
Backg ound
18
11
6
P ecision
76.3
77.6
99.0
Accu acy: 73 %
Glass Sponges
So Co als
B i le S a s
Model/Da a
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
CM-X-101/
Baseline
67.7 67.7 41.4 46.3 70.3 69.5 36.8 45.5 80.2 67.9 46.9 13.9
CM-X-101/
Syn h
67.8 66.2 45.3 46.5 69.8 68.8 48.8 52.6 79.2 65.1 52.4 12.8
CM-X-101/
Syn h-Blncd
71.4 69.2 51.4 54.0 76.8 74.9 51.5 55.9 79.9 64.7 52.6 12.6

Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82 81
4.3 Compa ison wi h o he s a e-o - he-a algo i hms
The selec ed Cen e Mask ia ResNeX -101 a chi ec u e was
u he compa ed o o he s a e-o - he a de ec o s and
backbones such as Re inaNe (Lin e al., 2017), Mask R-CNN
(He e al., 2017), MobileNe V2 (Sandle e al., 2018) o
VoVNe V2-99 (Lee and Pa k, 2019). Resul s a e demons a ed
in Table 2 o bounding boxes and segmen a ion masks. Wi h
ega ds o compe ing de ec o s, bo h Cen e Mask and
Re inaNe a e one s age de ec o s whe eas Mask R-CNN is a
wo s age de ec o ha u ilizes a egion-o -in e es p oposal
s ep which ypically p io i izes de ec ion accu acy o e
in e ence speed. Mo eo e , Re inaNe and Mask R-CNN use
ancho boxes o ea u e de ec ion. F om he esul s in Table 2
i is e iden ha Re inaNe pe o ms nea ly as well as
Cen e Mask whe eas Mask R-CNN shows a much lowe
pe o mance conside ing bounding boxes and segmen a ion
masks. I can be no ed ha one s age ancho - ee de ec o s can
pe o m jus as well on unde wa e image y as o he de ec o
ypes.
Fu he mo e, e alua ions on h ee di e en ly deep backbones
we e conduc ed: ResNeX -101 wi h 114.3 million pa ame e s,
VoVNe -99 wi h 96 million pa ame e s and MobileNe V2
wi h 28.7 million pa ame e s (Lee and Pa k, 2019).
No iceably, MobileNe V2 wi h ewe laye s has conside ably
lowe AP and AR esul s han he o he wo. Meanwhile,
conside ing bounding boxes, VoVNe -99 pe o ms nea ly as
well as ResNeX -101 and wi h ega ds o small objec s, AP
and AR esul s a e e en sligh ly highe . The p oblem o
de ec ing small objec s is known o inc ease o e y deep
backbones as hey need mo e inpu in o ma ion o cope wi h
he massi e amoun o pa ame e s (Nguyen e al., 2020). Small
objec s a e desc ibed by ewe pixels and migh no be di e se
enough o eed he ne wo k wi h su icien in o ma ion
inc easing he changes o o e i ing.
Table 2. Summa y o de ec ion esul s wi h bounding boxes (1s ow) and segmen a ion masks (2nd ow/cu si e)
Table 3. Con usion ma ix o CM-X-101/Syn h-Blncd
Table 4. Summa y o pe o mance esul s pe class
Model/Da a
AP.50:.95
bbox
AP.50
bbox
APsmall
bbox
APmedium
bbox
APla ge
bbox
AR1
bbox
AR10
bbox
AR100
bbox
ARsmall
bbox
ARmedium
bbox
ARla ge
bbox
CM-X-101/
Baseline
41.7
35.2
68.2
63.0
25.3
5.70
29.3
21.2
54.7
50.7
21.6
19.4
51.6
44.4
55.2
45.2
25.4
8.90
45.1
32.7
70.8
57.9
CM-X-101/
Syn h
48.8
37.3
71.0
63.0
27.4
4.90
39.1
24.1
62.8
51.7
24.7
20.6
58.8
47.9
64.2
49.5
27.9
7.90
57.3
41.2
77.1
59.5
CM-X-101/
Syn h
-
Blncd
51.8
40.8
76.7
67.7
27.5
4.60
40.2
27.1
66.1
54.8
25.7
22.1
59.0
49.1
63.9
50.6
27.9
7.50
55.7
41.6
77.9
62.5
CM-X-101/
T ad. Augm
48.8
40.2
75.0
70.4
26.9
7.00
38.6
27.6
58.5
52.2
23.0
20.4
55.3
46.8
58.9
47.9
27.2
10.6
50.1
36.8
72.6
58.6
CM-X-101/
Fusion
51.7
41.5
74.1
69.7
27.1
6.60
42.1
29.0
65.1
53.9
24.9
21.7
57.6
47.0
61.6
48.4
27.5
10.6
52.2
37.1
77.0
59.3
CM-V-99/
Syn h
47.9
36.9
72.0
62.9
27.9
4.80
37.0
23.4
62.8
49.8
23.6
20.1
56.6
46.1
61.9
47.8
28.3
7.60
52.6
38.6
77.1
58.2
CM-L-M/
Syn h
27.3
18.4
48.6
32.6
19.1
0.60
19.0
6.30
40.0
34.6
18.3
15.2
39.1
31.8
43.7
33.4
20.0
2.00
34.4
20.3
59.5
50.5
M-X-101/
Syn h
33.3
25.7
53.2
41.6
13.2
1.70
22.7
11.8
53.0
37.9
20.7
15.2
39.2
31.8
40.0
31.9
13.2
3.80
30.6
20.5
60.7
43.6
R-X-101/
Syn h
47.8 70.7 27.9 37.1 62.2 24.2 56.6 61.9 28.4 53.8 76.7
P edic ed
Glass Sponges
So Co als
B i le S a s
Backg ound
Recall
G ound u h
Glass Sponges
58
0
0
24
70.1
So Co als
0
38
0
3
92.7
B i le S a s
0
0
591
82
87.8
Backg ound
18
11
6
P ecision
76.3
77.6
99.0
Accu acy: 73 %
Glass Sponges
So Co als
B i le S a s
Model/Da a
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
F1
bbox
F1
mask
AP.50:.95
bbox
AP.50:.95
mask
CM-X-101/
Baseline
67.7 67.7 41.4 46.3 70.3 69.5 36.8 45.5 80.2 67.9 46.9 13.9
CM-X-101/
Syn h
67.8 66.2 45.3 46.5 69.8 68.8 48.8 52.6 79.2 65.1 52.4 12.8
CM-X-101/
Syn h-Blncd
71.4 69.2 51.4 54.0 76.8 74.9 51.5 55.9 79.9 64.7 52.6 12.6
5. CONCLUSION AND FUTURE STEPS
In conclusion, he used da a augmen a ion s a egy o
syn he ically de i ed image composi ions p o ed o be a good
al e na i e o equen ly used augmen a ion echniques.
P oblems such as class imbalance can u he easily be
alle ia ed and boos he pe o mance o unde ep esen ed
classes. De ec ion, segmen a ion and coun ing o ben hic
mega auna is a ask ha can be sol ed wi h good pe o mance
using ew a ian o iginal inpu images using ancho - ee one
s age de ec o s. In compa ison wi h o he models, i is e iden
ha he de ec ion p oblem o small objec s is a challenge ye
o be sol ed. Addi ionally, u u e s eps in ol e he
in oduc ion o mo e ben hic mo pho ypes, imp o ed coun ing
o a e duplica ions o o e lapping images and he alloca ion
o posi ion and wa e dep h o each de ec ed specimen o
u u e assemblage s udies.
6. ACKNOWLEDGEMENTS
We hank he anno a o s o he images: Ga in DMello, Diana
Rubio and Seyed Liales ani. Fu he we hank he cap ain and
c ew o RV Pola s e n as well as he scien i ic pa y o he
c uise PS118 o hei suppo . Special hanks go o Au un
Pu se and Huw G i i hs o hei suppo , he da a collec ion
on boa d and he iden i ica ion o ben hic o ganisms.
REFERENCES
Boulais, O., Woodwa d, B., Schlining, B., Lunds en, L.,
Ba na d, K., Bell, K. C. and Ka ija, K. (2020). Fa homNe :
An unde wa e image aining da abase o ocean
explo a ion and disco e y. a Xi :2007.00114 3, Co nell
Uni e si y, Compu e Science, Compu e Vision and
Pa e n Recogni ion [cs.CV].
B ooks, J. (2019). COCO Anno a o . URL h ps://gi hub.com/
jsb oks/coco-anno a o /.
Dawkins, M., S ewa , C., Gallage , S. and Yo k, A. (2013).
Au oma ic scallop de ec ion in ben hic en i onmen s. 2013
IEEE Wo kshop on Applica ions o Compu e Vision
(WACV), 160-170. doi: 10.1109/WACV.2013.6475014.
Gili, J.-M., A n z, W. E., Palanques, A., O ejas, C., Cla ke, A.,
Day on, P. K., Isla, E., Teixidó, N., Rossi, S. and López-
González, P. J. (2006). A unique assemblage o epiben hic
sessile suspension eede s wi h a chaic ea u es in he high-
An a c ic. Deep Sea Resea ch Pa II: Topical S udies in
Oceanog aphy, olume (53), 1029-1052. doi:
10.1016/j.ds 2.2005.10.021.
Gonzalez-Cid, Y., Bu gue a, A., Bonin-Fon , F. and
Ma amo os, A. (2017). Machine lea ning and deep
lea ning s a egies o iden i y Posidonia meadows in
unde wa e images. OCEANS 2017 – Abe deen, 1-5. doi:
10.1109/OCEANSE.2017.8084991.
He, K., Zhang, X., Ren, S. and Sun, J. (2016). Deep Residual
Lea ning o Image Recogni ion. 2016 IEEE Con e ence
on Compu e Vision and Pa e n Recogni ion (CVPR),
770-778. doi: 10.1109/CVPR.2016.90.
He, K., Gkioxa i, G., Dollá , P. and Gi shick, R. (2017). Mask
R-CNN. a Xi :1703.06870 3, Co nell Uni e si y,
Compu e Science, Compu e Vision and Pa e n
Recogni ion [cs.CV].
Huang, H., Zhou, H., Yang, X., Zhang, L., Qi, L. and Zang,
A.-Y. (2019). Fas e R-CNN o ma ine o ganisms
de ec ion and ecogni ion using da a augmen a ion.
Neu ocompu ing, olume (337), 372-384. doi:
10.1016/j.neucom.2019.01.084.
Jackson, J. B. C. (2008). Ecological ex inc ion and e olu ion
in he b a e new ocean. P oceedings o he Na ional
Academy o Sciences, 11458-11465. doi:
10.1073/pnas.0802812105.
Kelly, A. (2019). COCO Syn h. URL h ps://gi hub.com/
akTwel e/cocosyn h.
Langenkämpe , D., an Ke elae , R., Pu se , A. and
Na kempe , T. W. (2020). Gea -Induced Concep D i in
Ma ine Images and I s E ec on Deep Lea ning
Classi ica ion. F on ie s in Ma ine Science, olume (7).
doi: 10.3389/ ma s.2020.00506.
Lee, Y. and Pa k, J. (2019). Cen e Mask : Real-Time Ancho -
F ee Ins ance Segmen a ion. a Xi : 1911.06667 6, Co nell
Uni e si y, Compu e Science, Compu e Vision and
Pa e n Recogni ion [cs.CV].
Lin, T.-Y., Mai e, M., Belongie, S., Bou de , L., Gi shick, R.,
Hays, J., Pe ona, P., Ramanan, D., Zi nick, C. L. and
Dollá , P. (2014). Mic oso COCO: Common Objec s in
Con ex . In: Flee D., Pajdla T., Schiele B., Tuy elaa s T.
(ed.), Compu e Vision – ECCV 2014, 740-755. Sp inge ,
Cham. doi:10.1007/978-3-319-10602-1_48.
Lin, T.-Y., Goyal, P., Gi shick, R., He, K. and Dollá , P.
(2017). Focal Loss o Dense Objec De ec ion. a Xi :
1708.02002 2, Co nell Uni e si y, Compu e Science,
Compu e Vision and Pa e n Recogni ion [cs.CV].
Lu, H., Li, Y., Uemu a, T., Ge, Z., Xu, X., He, L., Se ikawa,
S. and Kim, H. (2018). FDCNe : il e ing deep
con olu ional ne wo k o ma ine o ganism classi ica ion.
Mul imedia Tools and Applica ions, olume (77), 21847-
21860. doi: 10.1007/s11042-017-4585-1.
Manning, C. D., Ragha an, P. and Schü ze, H. (2008).
In oduc ion o in o ma ion e ie al. Camb idge
Uni e si y P ess, Camb idge. ISBN: 0521865719.
Nguyen, N.-D., Do, T., Ngo, T. D. and Le, D.-D. (2020). An
E alua ion o Deep Lea ning Me hods o Small Objec
De ec ion. Jou nal o Elec ical and Compu e
Enginee ing, olume (2020), 1-18. doi:
10.1155/2020/3189691.
Pa oni, G., Co sini, M., Pede sen, N., Pe o ic, V. and
Cignoni, P. (2021). Challenges in he deep lea ning-based
seman ic segmen a ion o ben hic communi ies om
O ho-images. Applied Geoma ics, olume (13), 131-146.
doi: 10.1007/s12518-020-00331-6.
Pu se , A., Ma con, Y., D eu e , S., Hoge, U., Sablo ny, B.
and Hehemann, L. (2019). Ocean Floo Obse a ion and
Ba hyme y Sys em (OFOBS): A New Towed
Came a/Sona Sys em o Deep-Sea Habi a Su eys.
IEEE Jou nal o Oceanic Enginee ing, olume (44), 87-
99. doi: 10.1109/JOE.2018.2794095.
Pu se , A., D eu e , S., G i i hs, H., Hehemann, L., Je osch,
K., No dhausen, A., Piepenbu g, D., Rich e , C., Sch öde ,
H. and Do schel, B. (2021). Seabed ideo and s ill images
om he no he n Weddell Sea and he wes e n lanks o
he Powell Basin. Ea h Sys em Science Da a, olume (13),
609–615. doi: 10.5194/essd-13-609-2021.
82 Mona Lü jens e al. / IFAC Pape sOnLine 54-16 (2021) 76–82
Raphael, A., Dubinsky, Z., Iluz, D. and Ne anyahu, N. S.
(2020). Neu al Ne wo k Recogni ion o Ma ine Ben hos
and Co als. Di e si y, olume (12). doi:
10.3390/d12010029.
Salman, A., Jalal, A., Sha ai , F., Mian, A., Sho is, M.,
Seage , J. and Ha ey, E. (2016). Fish species
classi ica ion in uncons ained unde wa e en i onmen s
based on deep lea ning. Limnology and Oceanog aphy:
Me hods, olume (14), 570-585. Doi:
10.1002/lom3.10113.
Sandle , M., Howa d, A., Zhu, M., Zhmogino , A. and Chen,
L.-C. (2018). MobileNe V2: In e ed Residuals and Linea
Bo lenecks. 2018 IEEE/CVF Con e ence on Compu e
Vision and Pa e n Recogni ion, 4510-4520. doi:
10.1109/CVPR.2018.00474.
Shashidha a, B. M., Sco , M. and Ma bu g, A. (2020).
Ins ance Segmen a ion o Ben hic Scale Wo ms a a
Hyd o he mal Si e. 2020 IEEE Win e Con e ence on
Applica ions o Compu e Vision (WACV), 1303-1312. doi:
10.1109/WACV45572.2020.9093574.
Tan, C. S., Lau, P. Y., Co eia, P. L. and Campos, A. (2018).
Au oma ic analysis o deep-wa e emo ely ope a ed
ehicle oo age o es ima ion o No way lobs e
abundance. F on ie s o In o ma ion Technology &
Elec onic Enginee ing, olume (19), 1042-1055. doi:
10.1631/FITEE.1700720.
Tian, Z., Shen, C., Chen, H. and He, T. (2019). FCOS: Fully
Con olu ional One-S age Objec De ec ion. IEEE/CVF
In e na ional Con e ence on Compu e Vision (ICCV),
9626-9635. doi: 10.1109/ICCV.2019.00972.
Xie, S., Gi shick, R., Dolla , P., Tu, Z. and He, K. (2017).
Agg ega ed Residual T ans o ma ions o Deep Neu al
Ne wo ks. 2017 IEEE Con e ence on Compu e Vision and
Pa e n Recogni ion (CVPR), 5987-5995. doi:
10.1109/CVPR.2017.634.