Ci a ion: T oupio is-Kapelia is, A.;
Zissis, D.; Be e a, K.; Vodas, M.;
Spiliopoulos, G.; Ka an aidis, G. The
Big Pic u e: An Imp o ed Me hod o
Mapping Shipping Ac i i ies. Remo e
Sens. 2023,15, 5080. h ps://doi.o g/
10.3390/ s15215080
Academic Edi o s: Emanuele Sale no,
Claudio Di Paola and Angelica Lo
Duca
Recei ed: 07 Sep embe 2023
Re ised: 19 Oc obe 2023
Accep ed: 20 Oc obe 2023
Published: 24 Oc obe 2023
Copy igh : © 2023 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
emo e sensing
A icle
The Big Pic u e: An Imp o ed Me hod o Mapping
Shipping Ac i i ies
Alexand os T oupio is-Kapelia is 1,2,* , Dimi is Zissis 1, Kons an ina Be e a 2, Ma ios Vodas 2,
Giannis Spiliopoulos 2and Giannis Ka an aidis 2
1Sma Mo e Lab, Depa men o P oduc and Sys ems Design Enginee ing, Uni e si y o he Aegean,
84100 Sy os, G eece; [email p o ec ed]
2Resea ch Labs, Ma ineT a ic, 11525 A hens, G eece; kons an ina.be e a@ma ine a ic.com (K.B.);
[email p o ec ed](M.V.); giannis.spiliopoulos@ma ine a ic.com (G.S.);
[email p o ec ed] (G.K.)
*Co espondence: alex [email p o ec ed]
Abs ac :
Densi y maps suppo a bi d’s eye iew o essel a ic, h ough p o iding an o e iew o
essel beha io , ei he a a egional o global scale in a gi en ime ame. Howe e , any inaccu acies in
he unde lying da a, due o senso noise o o he ac o s, e iden ly lead o e oneous in e p e a ions
and misleading isualiza ions. In his wo k, we p opose a no el algo i hmic amewo k o gene a ing
highly accu a e densi y maps o shipping ac i i ies, om incomple e da a collec ed by he Au oma ic
Iden i ica ion Sys em (AIS). The comple e amewo k in ol es a numbe o compu a ional s eps o
(1) cleaning and il e ing AIS da a, (2) imp o ing he quali y o he inpu da ase ( h ough ajec o y
econs uc ion and sa elli e image analysis) and (3) compu ing and isualizing he subsequen essel
a ic as densi y maps. The amewo k desc ibes an end- o-end implemen a ion pipeline o a eal
wo ld sys em, capable o add essing se e al o he unde lying issues o AIS da ase s. Real-wo ld
da a a e used o demons a e he e ec i eness o ou amewo k. These expe imen s show ha ou
ajec o y econs uc ion me hod esul s in signi ican imp o emen s up o 15% and 26% o empo al
gaps o 3–6 and 6–24 h, espec i ely, in compa ison o he baseline me hodology. Addi ionally, a use
case in Eu opean wa e s highligh s ou capabili y o de ec ing “da k essels”, i.e., essel posi ions
no p esen in he AIS da a.
Keywo ds:
ma i ime a ic moni o ing; essel ajec o y mining; ajec o y econs uc ion; map
isualiza ion; au oma ic iden i ica ion sys em; sa elli e image y; ea h obse a ion da a
1. In oduc ion
The Eu opean Commission has se a goal o achie ing ze o emissions o g eenhouse
gases by 2050 [
1
], in line wi h he Pa is Ag eemen [
2
], whe e coun ies globally ha e
ag eed o pu sue e o s o limi global wa ming o below 2 deg ees Celsius, compa ed
o p e-indus ial le els. Scena ios explo ed in his con ex sugges ed ha achie ing his
will equi e doubling elec ici y p oduc ion wi h abou a qua e o i being p oduced
o sho e [
1
], esul ing in eno mous changes o Eu opean wa e s. Up o a qua e o ce ain
coun ies’ wa e s could be de o ed o wind a ms, ine i ably impac ing o he di e en
ma ine ac i i ies. Cu en sea- ela ed uses and ac i i ies in ol e ma i ime shipping, ishing,
ex ended aquacul u e, oil and gas explo a ion and d illing, leisu e and boa ing ac i i ies,
cul u al he i age conse a ion ini ia i es and many mo e. Imp o ing ou unde s anding o
ac i i ies ha ake place a sea, including hei spa ial and empo al aspec s, is i al in he
ligh o an in ensi ied use o ma i ime space [3].
Fo he pu pose o de ailed analyses on he ac i i ies occu ing a sea, he cons an
moni o ing o essel beha io is necessa y. Sys ema ic moni o ing echniques can be spli
in o wo sepa a e ca ego ies, depending on whe he he subjec in ques ion, i.e., he essels,
ake pa in he p ocess. The i s g oup ely on he de ec ions om senso s such as RADAR,
Remo e Sens. 2023,15, 5080. h ps://doi.o g/10.3390/ s15215080 h ps://www.mdpi.com/jou nal/ emo esensing
Remo e Sens. 2023,15, 5080 2 o 29
Sona and sa elli e image y, and a e able o ex ac he loca ion and mo emen o essels
h ough p ocessing hei inpu [
4
]. Recen ly, he ad ancemen s o emo ely ope a ed and
au onomous ehicles ha e lead o some wo ks on he cons an moni o ing a eas nea he
coas [
5
]. Such sys ems do no equi e he aid o e en he pe mission o he essels o
ope a e and hus a e mo e sui able o s udying illegal o malicious ac i i y [
6
]; howe e ,
hei accu acy la gely depends on ha dwa e speci ica ions, he co e age o e an a ea o
in e es and pos -p ocessing me hods. On he o he hand, he second g oup is comp ised
om sys ems whose unc ion depends on he coope a ion o he essels hemsel es. In
o he wo ds, hese sys ems wo k h ough he ansmission o posi ional messages om he
essels, allowing o he s o eco d hei mo emen . A ew sys ems ha e been in oduced
and a e ope a ing oday, wi h he mos p ominen ones being he Long- ange Iden i ica ion
and T acking (LRIT) and he Au oma ic Iden i ica ion Sys em (AIS) [
7
,
8
]. The main eason
ha essels use such sys ems o ansmi hei posi ions, o o he ele an in o ma ion, is
mainly o ensu e sa e y while a elling. Rega dless o he e ec i eness o any sys em,
cap u ing he accu a e and comple e ajec o y o a mo ing objec is almos impossible
in eal condi ions. This is mainly due o he inhe en limi a ions o da a acquisi ion and
s o age mechanisms used. As a esul , he con inuous mo emen o an objec is usually
ob ained as an app oxima e o m o disc e e samples o spa io- empo al loca ions [
9
].
Some imes he e o is accep able o a gi en use case while in o he occasions i may lead
o e oneous in e p e a ions.
Since being decla ed as manda o y o la ge essels in 2004, he mos commonly
used da ase s o essel acking ha e been based on he AIS. Besides hei loca ion, he
essels b oadcas hei iden i ica ion in o ma ion, cha ac e is ics and des ina ion, along
wi h o he in o ma ion o igina ing om on-boa d de ices and senso s, such as hei speed
and heading [
9
–
11
]. Due o he la ge numbe o essels ansmi ing hei posi ions globally,
ad anced analysis is equi ed o in e p e hese da a. Towa ds an imp o ed unde s anding
o such da ase s, isualiza ions h ough densi y maps is commonly used. The added alue
o densi y maps is ha hey suppo a bi d’s eye iew o ma i ime a ic, h ough p o iding
an o e iew o essel beha io ei he a a egional o a global scale. F om densi y maps,
one can de e mine he pa e ns o li e in a gi en s udy a ea. Pa e ns o li e a e unde s ood
as obse able human ac i i ies ha can be desc ibed as pa e ns in he ma i ime domain
ela ed o a speci ic ac i i y (e.g., ishing) [
9
]. Essen ially, essel-based ma i ime ac i i y can
be desc ibed in space and ime while being classi ied o a numbe o known ac i i ies a sea
( ishing e c.). The spa ial elemen desc ibes ecognized a eas whe e ma i ime ac i i y akes
place, including po s, ishing g ounds, o sho e ene gy in as uc u e, d edging a eas, e c.
The ansi pa hs o and om hese a eas also desc ibe he spa ial elemen (e.g., comme cial
shipping, e y ou es), while he empo al elemen o en holds addi ional in o ma ion o
ca ego izing hese ac i i ies (e.g., ishing pe iod, ime o yea , e c.) [12].
In his se ing, we iden i y wo main asks ha we add ess in his pape . The i s ask
is o design and de elop a pipeline capable o p oducing sound and clea in e p e a ions
o shipping ac i i y in a gi en a ea o in e es om noisy and o en inaccu a e AIS da a,
ex ending he high-le el desc ip ion in [
13
]. We design and implemen a numbe o
algo i hmic me hods ha suppo he handling o hese da a and hei ans o ma ion in o
accu a e densi y maps. We p o ide modules o he da a ans o ma ion p ocess, om da a
cleaning and smoo hing, o he inal esul isualiza ion. In ligh o an in ensi ied use o
ma i ime space, he e will be a g owing need o such applica ions in he nea u u e.
The second ask is o c ea e mechanisms o imp o ing he quali y o he AIS da ase s,
by including addi ional essel posi ions. Such mechanisms would esul in mo e comple e
da ase s and in u n would assis in gene a ing mo e accu a e densi y map depic ions
o essel ac i i y. Two echniques a e desc ibed o his ask. Fi s , a no el me hod o
ajec o y gap illing is p oposed. This me hod le e ages his o ical da a o es o e missing
posi ional messages in any gi en ship ajec o y, imp o ing he comple eness and quali y
o he unde lying da a. In o de o e alua e ou app oach, we alida ed i s pe o mance
on a eal-wo ld da ase o AIS messages, co e ing Eu opean wa e s om Oc obe o 2021.
Remo e Sens. 2023,15, 5080 3 o 29
The esul s indica e ha ou sugges ed me hod can e ec i ely econs uc a ship ajec o y
wi h highe accu acy (up o 26%), compa ed o a s aigh -line in e pola ion. Fu he mo e,
we p esen a dedica ed mechanism ha is able o ex ac essel posi ions based on sa elli e
image y. This mechanism is sui able o bo h ada and op ical da a, allowing o he
moni o ing o essels in cases whe e he AIS does no . This app oach is based on a Con o-
lu ional Neu al Ne wo k (CNN) a chi ec u e and allows o ex ac ing essel coo dina es
in cases whe e AIS da a a e no a ailable, making an excellen complemen ing da a sou ce.
P elimina y esul s o his la e amewo k can also be obse ed in [14].
The o e all amewo k desc ibes an end- o-end implemen a ion pipeline o a eal
wo ld sys em, capable o add essing se e al o he unde lying issues o AIS da ase s
and p oducing highly accu a e densi y maps. Speci ic componen s o he amewo k,
demons a e se e al ad an ages o e exis ing algo i hms, while he o e all comple e
amewo k is unique o p oducing his ype o isualiza ion o AIS da a. The con ibu ions
o his wo k can be desc ibed as ollows:
•
We p o ide an o e iew o he AIS sys em and how i can be used o cap u e essel
mo emen , while also highligh ing some o he issues wi h he i ness o AIS da a.
•
We desc ibe wo mechanisms o imp o ing he quali y o an inpu AIS da ase , in
cases o signi ican empo al gaps in he ajec o ies. The i s mechanism sugges s he
mos p obable pa h he essel ollowed du ing i s AIS-messages gap, using his o ical
mobili y in o ma ion o e he a ea o in e es . The o he le e ages sa elli e image y
o en ich ou o iginal da ase wi h addi ional essel posi ions h ough an accu a e
de ec ion echnique based on CNN a chi ec u e.
•
We p esen a comple e con igu able amewo k ha is capable o c ea ing e ec i e
densi y isualiza ions based on aw da a.
•
We conduc ex ensi e expe imen s o demons a e he e ec i eness o ou app oach,
using eal AIS da a and sa elli e images om he Eu opean wa e s om a one-
mon h pe iod. The esul s indica e signi ican imp o emen o e he s aigh -line-
in e pola ion baseline echnique, o he ajec o y econs uc ion, and highligh he
amewo ks abili y o de ec essels ha do no ansmi AIS messages.
The es o he pape is o ganized as ollows: i s we men ion ela ed wo k ega ding
AIS manipula ion, econs uc ion and isualiza ion, as well as wo ks ha deal wi h essel
iden i ica ion h ough sa elli e image y (Sec ion 2). Then, we desc ibe ou p ocess o
c ea ing densi y maps om aw AIS messages (Sec ion 3). Following his, we desc ibe he
wo mechanisms o imp o ing he quali y o he AIS da ase (Sec ion 4) and p esen he
subsequen esul s o he expe imen al e alua ion (Sec ion 5). A discussion on he esul s o
ou expe imen s and he e ec i eness o ou p oposed amewo k is desc ibed in Sec ion 6.
Finally, we conclude ou wo k wi h a discussion on he end esul s and possible u u e
s eps o u he imp o emen s on ou me hod’s accu acy (Sec ion 7).
2. Backg ound
In his sec ion, we p o ide impo an de ini ions ega ding essel mo emen and
densi y map isualiza ion. Mo eo e , we p o ide a b ie e iew o pas wo k conce ning
essel ajec o y econs uc ion and essel iden i ica ion h ough sa elli e image y.
2.1. Spa io-Tempo al Da a and he Au oma ic Iden i ica ion Sys em (AIS)
A ajec o y can be cap u ed as a ime-s amped se ies o loca ion poin s deno ed as
p0(x0
,
y0
,
0)
,
p1(x1
,
y1
,
1)
, ...,
pn(xn
,
yn
,
n)
, whe e
xi
,
yi
ep esen s geog aphic coo dina es
o he mo ing objec a ime
i
and
n
is he o al numbe o elemen s in he se ies (e.g.,
Figu e 1demons a es a ajec o y o 16 poin s). To gene a e he ajec o y, a senso needs
o acqui e i s coo dina es x,ya ime .
Remo e Sens. 2023,15, 5080 4 o 29
Figu e 1.
In g ey is he ue ajec o y, while in pu ple he app oxima ed ajec o y is cap u ed by a
gi en senso .
No ice ha he app oxima ed ajec o y can also be ep esen ed as a se ies o line
segmen s be ween he s amped posi ions (gi en ha he e is a unique iden i ie g ouping
hese posi ions in o he same ajec o y, usually he mo ing objec id):
∑ aji=p0p1,p1p2,p2p3,p3p4,p4p5,p5p6, ..., p10 p11,p11 p12,p12 p13,p13 p14,p14 p15
AIS messages a e b oadcas ed pe iodically and can be ecei ed by o he essels
equipped wi h he app op ia e anscei e s, as well as by on- he-g ound o sa elli e-based
senso s [
15
]. This in o ma ion is ansmi ed a egula in e als anging anywhe e om 2 s
o 3 min, depending on he essel’s beha io . Since 31 Decembe 2004, AIS mus be i ed
aboa d all essels o 300 g oss onnage and upwa ds engaged on in e na ional oyages,
ca go essels o 500 g oss onnage and upwa ds no engaged on in e na ional oyages and
all passenge essels, i espec i e o size [16].
E en hough he AIS p o ocol is mainly a ge ed o la ge essels, a la ge numbe o
smalle boa s a e also equipped wi h anscei e s o ensu e hei sa e y while a elling.
The essels equi ed o ca y AIS a e equipped wi h Class A AIS ansponde s, whe eas
o he essels can ca y ei he Class A o Class B AIS ansponde s. The Class A ansponde
has a mo e powe ul signal and ansmi s messages mo e equen ly han he Class B
ansponde s; he e o e, Class A ansponde s ypically ha e a ine spa ial and empo al
esolu ion. Vessels which may ca y Class B ansponde s include ec ea ional essels,
ishing essels o small passenge essels [17].
G ound-based (Te es ial AIS o TER-AIS) and space-based AIS (Sa elli e AIS o
SAT-AIS) da ase s ha e some conside able di e ences:
•
Te es ial ecei e s a e land-based s a ions which ecei e messages om essels
wi hin hei line o sigh . Once he message is ecei ed, i is elayed ia ne wo k
connec ion o a compu e o s o age, p ocessing and isualiza ion. Typically, wi h an
op imal e es ial ecei e se up, messages om up o 40–60 nau ical miles away can
be ecei ed.
•
Sa elli e ecei e s unc ion simila ly o e es ial ecei e s by ansmi ing he ecei ed
AIS message o a compu e o da a s o age, p ocessing and isualiza ion. Ha ing
a la ge ield o iew (up o 5000 km), sa elli e ecei e s a e always in iew o he
ansponde s [18].
2.1.1. Da a Fi ness
The concep o da a quali y is somewha ague, bu an e ec i e and commonly used
de ini ion o da a quali y is “ i ness o use”, which is he abili y o he da a collec ed o
mee use equi emen s [
12
]. Many wo ks ha e ocused on add essing such unce ain y in
loca ion da a ela ed o modelling and ep esen a ion. Speci ic applica ions may allow some
imp ecision based on hei equi emen s. AIS da ase s ha e long been used o ma i ime
Remo e Sens. 2023,15, 5080 5 o 29
densi y maps and esea che s ha e iden i ied some o he unde lying di icul ies a ec ing
he da a i ness [19–24].
Al hough AIS da ase s a e some o he mos impo an sou ces o in o ma ion o
ma i ime a ic, he esul ing spa ial ajec o ies may ha e se e al missing da a poin s due
o se e al ac o s, including design ea u es o malicious ac i i y:
1.
Fi s ly, da ase s ha ha e been collec ed by Sa elli es and hose by Te es ial s a ions
will ha e di e en g anula i ies and esolu ions. Ea h o bi ing sa elli e collec ing
AIS messages a e easily conges ed when he e is a la ge numbe o essels wi hin
hei gi en ield o iew. AIS is based on he Time Di ision Mul iple Access (TDMA)
adio access scheme which ensu es ha no wo ships wi hin adio ange o each o he
a e ansmi ing a he same ime. The TDMA de ined in he AIS s anda d c ea es
4500 a ailable ime-slo s in each minu e bu his can be easily o e whelmed by he
la ge sa elli e ecep ion oo p in s and he inc easing numbe s o AIS anscei e s,
esul ing in message collisions, which he sa elli e ecei e canno p ocess. Schemes
such as he TDMA we e designed o success ul ship- o-ship o ship- o-sho e com-
munica ion, no o ship- o-sa elli e communica ion, which hea ily deg ades hei
e iciency [
25
]. Howe e , in he case o he sa elli e segmen o he AIS, he e iciency
o he implemen ed access schemes is hea ily deg aded due o he high a io o he
AIS packe s’ collisions.
2.
Addi ionally, acco ding o he AIS speci ica ions, Class A anscei e s ese e hei
ime slo s o ansmission ia Sel O ganized Time Di ision Mul iple Access (SOT-
DMA). A e pe o ming a scan o asce ain which slo s ha e al eady been ese ed
by o he essels, hey ese e an emp y slo . The de ice le s nea by AIS de ices
know ha i in ends o use his slo o u u e b oadcas s. On he o he hand, Class
B anscei e s a e pe mi ed o ansmi ia Ca ie Sense Time Di ision Mul iple
Access (CSTDMA), whe e, unlike SOTDMA, slo s a e no ese ed. They ins ead
simply scan o a ailable space and ansmi when a ee one is de e mined o be
a ailable. T ansmission p io i y is gi en o Class A anscei e s, which use SOTDMA
since hey ese e ime slo s. The iming o Class B ansmissions ia CSTDMA mus
wo k a ound he ime slo s ese ed by Class A anscei e s. I a Class B anscei e is
unable o ind an emp y space, hei ansmissions a e delayed.
3.
Recen ly, a di e en ype o Class B ansmi e ha uses SOTDMA, namely Class
B “SO” (Sel -O ganizing), was p oduced. Class B “SO” and Class A ansmi e s
i ed aboa d essels ha e a c i ical di e ence which also a ec s Sa elli e ecep ion.
Acco ding o he In e na ional Telecommunica ions Union speci ica ions, p o ision
should be made o wo le els o nominal powe (high powe and low powe ), as
equi ed by some applica ions. The de aul ope a ion o he AIS s a ion should be on
he high nominal powe le el. The wo powe se ings should be 1 W and 12.5 W o
1 W and 5 W o Class B “SO”. E iden ly, he weake signal o Class B de ices means
i is mo e di icul o ecei e hese signals om space.
Addi ionally, SAT-AIS canno cap u e he signal o all ansmi ing essels a once;
se e al o bi s a e equi ed in o de o cap u e a ep esen a i e densi y sample. In p e ious
s udies [
26
], he da a compila ion app oach o acqui ing in o ma ion ele an o he essel
popula ion was based on he gene a ion o essel posi ion ‘snapsho s’ in a speci ied ime
window. As he du a ion o his window inc eases, he amoun o in o ma ion gained in
e ms o dis inc essels, newly de ec ed by he a ailable senso s, ends o dec ease. On
he o he hand, coas al ecep ion om he e es ial ecei e s is only possible i a essel
is wi hin he line o sigh o app oxima ely 40 nau ical miles in ideal condi ions, which is
a ec ed by bad wea he and o he condi ions [27].
Besides i s limi a ions, AIS emains he go- o sou ce o da a o unde s anding ma -
i ime ac i i ies and o his se e al wo ks ha e ocused on iden i ying he issues [
27
–
29
] and
de eloping echniques ha imp o e he exis ing AIS by o e ing be e acking accu acies
and gua an ees.
Remo e Sens. 2023,15, 5080 6 o 29
De ini ion 1
(Incomple e T ajec o y)
.
Gi en he spa se spa ial da a
p0(x0
,
y0
,
0)
,
p1(x1
,
y1
,
1)
,
p3(x3
,
y3
,
3)
,
p5(x5
,
y5
,
5)
,
p7(x7
,
y7
,
7)
,
p8(x8
,
y8
,
8)
,
p15(x15
,
y15
,
15)
o a mo ing ship, con-
sis ing o i s ime s amped loca ions, he esul ing ajec o y can be de ined as:
∑ aj2=p0p1,p1p3, p3p5,p5p7,p7p8,p8p15
I should be no ed ha aj16= aj2.
Such inaccu acies o spa se ajec o ies, can easily lead o aul y analysis and conclu-
sions ega ding essel mo emen , i no deal wi h app op ia ely. In he ollowing sec ions,
wo echniques o imp o ing he quali y o spa se ajec o ies a e desc ibed.
2.1.2. Incomple e T ajec o ies
Despi e he high a e o ansmission o AIS messages, spa se ajec o ies o en occu
in da ase s. Recons uc ing he ajec o ies du ing la ge gaps is no simple ask. Mos
cu en app oaches o densi y map gene a ion will simply emo e in e pola e ajec o ies
wi h la ge missing pa s [
30
,
31
]. Se e al wo ks ha e p oposed s a egies o dealing
wi h he impe ec ions o AIS da a and speci ically o comple ing o illing he gaps in
ajec o ies. The majo i y o hese wo ks assume ha a leas in heo y, he ajec o y o
a ship can be app oxima ed as a s aigh line o a sho ime in e al. The e o e, linea
in e pola ion is he mos widely used gap- illing me hod [
32
–
35
]. Acco ding o his me hod,
he posi ions o he ship be ween wo poin s a e calcula ed by in e pola ing hei coo dina es
(la i ude and longi ude) acco ding o he desi ed imes amp. Howe e , his me hod is
only sui able o small o ecas ing windows (e.g., se e al minu es), high equency da a
and in si ua ions when he essel is expec ed o ollow uni o m linea mo ion. Addi ional
in e pola ion me hods ha e been p oposed, such as polynomial, cubic spline in e pola ion,
Lag ange and He mi e in e pola ion me hods, which ake in o conside a ion addi ional
ea u es such as di ec ion, heading and speed o econs uc ajec o ies wi h cu es [
34
,
36
].
Un o una ely, he abo e me hods do no ake in o accoun he en i onmen o p io
in o ma ion ega ding a a el a ea. They also show deg aded accu acy in scena ios whe e
ships conduc maneu e s o en (e.g., i e s).
Se e al wo ks ha e a emp ed o combine he me hods abo e wi h he ships’ na iga-
ional s a us o imp o e esul s. Zhang e al. p oposed a ajec o y econs uc ion app oach
conside ing he na iga ion s a es, namely ho elling, maneu e ing, and no mal-speed sail-
ing [
37
]. In his wo k, bo h linea in e pola ion and cubic spline in e pola ion o s aigh
and cu e sub- ajec o ies, espec i ely, a e applied o econs uc a new smoo h ajec o y.
The majo i y o hese app oaches emain geome y-based app oaches, un o una ely no
making use o he kinema ic in o ma ion o he ship o his o ical in o ma ion o he a ea;
hus, he accu acy o he esul s is limi ed [38].
Recen ly, ad ances in machine lea ning ha e led o nume ous wo ks wi h applica ions
in anspo a ion [
39
], wi h a la ge po ion being abou p ocessing essel ajec o ies.
Fo example, in [
20
], CNNs a e u ilized o econs uc AIS ajec o ies. Expe imen al
esul s show ha he p oposed me hod is capable o highe accu acy han he cubic spline
in e pola ion baseline me hod, especially when he ajec o ies a e cu ed and ha e a high
loss a e. Un o una ely, he es esul s co e only a small geog aphical a ea. In [
30
], a
Long Sho -Te m Memo y (LSTM)-based supe ised lea ning me hod is used o econs uc
he essel ajec o ies, achie ing good esul s in sho - e m o ecas s, wi hou aking in o
accoun he en i onmen al ac o s o ship sailing. Simila ly, [
40
] uses an LSTM-based
a chi ec u e ha add esses some o he AIS- ela ed issues, gi ing p edic ions o ho izons
up o 1 h in he u u e. Au ho s in [
41
] apply a deep lea ning me hod based on Bi-
di ec ional Long Sho -Te m Memo y Recu en Neu al Ne wo ks (BLSTM-RNNs) o
ajec o y es o a ion. The me hod demons a es highe accu acy han linea in e pola ion
me hods in complex wa e ways such as he Yang ze Ri e , bu he deep lea ning app oach
comes wi h a highe compu a ional cos .
Remo e Sens. 2023,15, 5080 7 o 29
In [
42
] a no el sequence- o-sequence essel ajec o y p edic ion model based on
encode –decode ecu en neu al ne wo ks (RNNs) ha a e ained on his o ical ajec o y
da a o p edic u u e ajec o y samples gi en p e ious obse a ions is p oposed. The sug-
ges ed me hod showcases supe io accu acy han baseline me hods bu elies hea ily upon
he his o ical da a o pa hs belonging o he same mo ion pa e n o he es ajec o y, i.e., a
la ge and ep esen a i e sample o da a om he domain. The au ho s men ion ha his
po en ially limi s he me hod’s applicabili y since i may be sensi i e o he aining da ase ,
in pa icula he numbe o ship ajec o ies a ailable and domain ep esen a i eness.
2.2. Sa elli e Images
As s a ed abo e, he e a e si ua ions whe e he AIS p o ocols a e no enough o cap u e
he p esence o a essel in an a ea. These may be caused by he na u e o he AIS sys em,
such as packe collisions o dense a eas o co e age issues, as well as he in en ional swi ch-
o o he ansmi e . To o e come such issues and o ha e a mo e comple e si ua ional
awa eness, addi ional da a sou ces should be included o complemen AIS da a.
One o he mos p ominen da a sou ces ha can be used o moni o ing an a ea a e
sa elli e images, bo h op ical and Syn he ic Ape u e Rada (SAR). Being a non-coope a i e
sou ce, i does no ely on he essels hemsel es o ecei ing da a. Addi ionally, since
such da a a e collec ed by sa elli es, global co e age can be conside ed a possibili y [
43
,
44
].
O cou se, as in all cases, elying solely on sa elli e image y o moni o ing an a ea comes
wi h i s own disad an ages. Mo e speci ically, op ical image y is o en a ec ed by wea he
condi ions and en i onmen al e ec s, while SAR da a ha e limi ed spa ial esolu ion.
Mo eo e , a common issue wi h sa elli e images a e long pe iods be ween consecu i e
isi s o he sa elli e o he a ge a ea, esul ing in spa se da a [
45
]. Ne e heless, bo h
op ical and SAR images can be g ea ly use ul—complemen ing AIS da a—while ying o
de ec essels in an a ea o in e es , especially when dealing wi h gaps in he espec i e
ship ajec o ies.
In he ela ed li e a u e, he p oblem o essel de ec ion in sa elli e image y has been
he ocus o many academic and indus ial esea ch ac i i ies [
43
,
44
]. These ac i i ies can
be classi ied in o wo b oad ca ego ies:
•
The app oaches ha a e based on he employmen o h eshold-based algo i hms,
such as he Cons an False Ala m Ra e (CFAR) algo i hms [
46
]. The CFAR is a g oup
o adap i e algo i hms ha a y he de ec ion h eshold as a unc ion o he sensed
en i onmen , a he han a single alue, in o de o y o ix he p obabili y o alse
ala ms due o noise o jamming a a p ede ined alue. Di e en ools we e p esen ed
o SAR image y h ough CFAR algo i hms o e he yea s, as men ioned in [
47
].
Amongs hem, he Eu opean Commission’s Join Resea ch Cen e has eleased he
Sea ch o Uniden i ied Ma i ime Objec s (SUMO), a ool speci ically designed o
de ec ing essels in such images [48].
•
AI-based app oaches ha employ Neu al Ne wo ks (NNs) in o de o de ec essels
based on ained models [49].
Re . [
43
] p esen s a de ailed, up- o-da e o e iew o app oaches o essel de ec ion
in op ical image y co e ing bo h ca ego ies. The p ocessing chain ha we used o pe o m
essel de ec ion in sa elli e image y o he needs o his p ojec is based on ou p e ious
wo k ha belongs o he AI-based ca ego y and is pa ially co e ed in [
50
]. Howe e , in
o de o add ess he known scalabili y p oblems ha come wi h he use o NNs in la ge
olumes o sa elli e da a [
51
], in his wo k, we u he ex end he app oach desc ibed in [
50
]
and de eloped a hyb id app oach ha employs bo h h eshold-based mechanisms and
CNNs, in o de o ob ain he bes o bo h wo lds: high accu acy esul s p o ided by CNNs
while add essing scalabili y issues by using h esholding o il e ou edundan image iles.
2.3. Densi y Maps
Toge he wi h being a widely used isualiza ion ool o mobili y da a, densi y maps
allow o a u he unde s anding o he a ic and subsequen ac i i y in a speci ic a ea
Remo e Sens. 2023,15, 5080 8 o 29
o in e es [
52
,
53
]. Analyzing densi y maps as hey e ol e o e ime also suppo s a ic
changes and pa e n dis ibu ion in e p e a ion. Mo eo e ,
The e m “ essel densi y” has se e al co-no a ions and is hus used wi h se e al
meanings in he ma i ime domain. The e o e, essel densi y can e e o:
1.
The a e age numbe o essels de ec ed wi hin a de ined geog aphical a ea (spa ial
g id) in a gi en ime ame;
2.
The a e age numbe o c ossings wi hin a de ined geog aphical a ea (spa ial g id) in a
gi en ime ame (o en also e e ed o as “ essel a ic densi y”);
3.
The o al ime o he p esence o a essel wi hin a de ined geog aphical a ea (spa ial
g id) in a gi en ime ame;
The e is a conside able di e ence in he me hods used o he c ea ion o densi y maps
acco ding o he de ini ion used, including calcula ions based on he numbe o essel
posi ions de ec ed, he numbe o essel acks, hei leng h c ossing a gi en a ea and many
mo e a ia ions.
3. Gene a ing Densi y Maps om Raw AIS Da a
In his sec ion, we illus a e he s eps ollowed o p oduce highly accu a e densi y
maps o shipping ac i i y om aw AIS da a. The o e all iew o he p ocess, as depic ed in
Figu e 2, begins wi h he decoding o he da a and he emo al o e oneous and i ele an
messages. This s ep is ollowed by he calcula ion o he a ic densi y wi h he subsequen
maps gene a ed acco ding o he g id esolu ion selec ed. Fo mo e accu a e esul s, he
op ion o imp o ing he quali y o he AIS da ase h ough addi ional essel posi ions
(o ange componen in Figu e 2) may be conside ed. The wo echniques p oposed in his
wo k o his s ep a e desc ibed in de ail in he ollowing sec ions.
Fo he implemen a ion o ou pipeline, we elied mainly on open-sou ce lib a ies in
Py hon o all s eps. Mo eo e , o boos he pe o mance o ou sys em we used Py hon’s
p ocess pa alleliza ion op ion o he s eps o cleaning, il e ing and densi y calcula ion. Fo
he handling o he geome ies, du ing he g id gene a ion and land-masking, he Shapely
lib a y was u ilized, along wi h he GDAL lib a y o he as e iza ion p ocess.
Figu e 2.
Flowcha o ex ac ing densi y maps om aw AIS, by decoding and cleaning he da a and
calcula ing he co esponding densi y. Imp o ing he AIS da ase h ough he gap illing mechanism
o using he da a wi h sa elli e image de ec ions (o ange) may be inco po a ed o mo e accu a e
esul s.
3.1. Da a Cleaning
As aw AIS messages ha e di e en ypes, s a ic and dynamic in o ma ion should
be coupled manually. In o ma ion ega ding he ype and class o he essel is ex ac ed
by ‘S a ic and Voyage Rela ed Da a’ messages and me ged wi h he dynamic (posi ional)
messages o he co esponding essel, esul ing in a mo e comple e AIS da ase .
Fu he mo e, emo ing e oneous and unnecessa y messages om AIS da ase s is a
c ucial componen o any u he analysis o ma i ime mo emen . In his s ep, we clean
he da a h ough a se ies o il e s on AIS messages, in o de o smoo h he noise and
po en ially dec ease he e o in hei measu emen s. Mo e p ecisely, he il e s applied o
each inpu message a e he ollowing:
Remo e Sens. 2023,15, 5080 9 o 29
1.
Emp y ields: messages ha moni o mo emen , like he AIS messages, may include a
ple ho a o ea u es. Besides p ima y ea u es (posi ional and empo al) ha deno e
he exac posi ion o he mo ing essel, o he ields ega ding i s cha ac e is ics o
i s cu en s a e a e usually p o ided. Fo he pu pose o e ec i ely analyzing he
inpu da a, we equi e ha each posi ional message includes non-emp y alues in he
ollowing ields:
•
Posi ional ea u es: Vessel Longi ude, Vessel La i ude, Times amp o AIS occu -
ence (exp essed in UNIX ime in milliseconds).
•
Mo emen measu emen s: Vessel Speed-o e -G ound (SoG-measu ed in kno s)
and Vessel Cou se-o e -G ound (CoG-measu ed in deg ees).
2.
In alid mo emen ields: While mos messages include he ields ega ding a essel’s
mo emen (SoG, CoG), in some ins ances, hese ields ca y in alid alues. In such
cases, he messages a e cha ac e ized as e oneous and a e disca ded. The h esholds
indica ing alid mo emen a e as ollows:
• Speed-o e -G ound: a eal numbe be ween 0 and 80 kno s.
• Cou se-o e -G ound: a eal numbe be ween 0 and 360 deg ees.
3.
In alid essel iden i ica ion numbe : Wi h each AIS message e e ing o a single essel, a
ield dedica ed o i s iden i ica ion is needed. The Ma i ime Mobile Se ice Iden i y
(MMSI) con en ion is widely used while e e ing o AIS ansmi e s (i.e., essels) [
11
],
wi h each single en y being a se ies o nine cha ac e s. Messages wi h a sho e o
longe MMSI leng h, as well as messages whose MMSI alls in o some excep ion
alues (123456789, 0.12345, 000000000, 111111111, e al.), a e disca ded.
4.
A eas o in e es /Land-masking: While ou app oach may be applied ega dless o he
a ea in ques ion, de ining he space o e e ence is a c ucial pa o mo ing o wa d
o wo easons:
•
Remo ing da a ha e e o a eas ou side o he pu pose o he execu ion scena io.
•
Remo ing da a ha include e oneous coo dina es, i.e., no alid longi ude/la i ude
o poin s on land.
5.
Down-sampling: Al hough he equency whe e each essel is ansmi ing a posi ional
message is usually desi ed o be as high as possible, ha ing oo many messages
may esul in conside able delays while p ocessing. In o de o o e come his issue,
a down-sampling is pe o med upon he inpu da a. The ques ion a his poin is
whe he we a e able o dis ega d some sample poin s wi hou sac i icing he quali y o
he ajec o y da a equi ed o he a ge applica ion. Fo his pu pose, he ajec o ies
a e il e ed so ha consecu i e messages om he same essel ha e a leas
k
minu es
be ween hem, which also emo e all duplica e messages as a esul .
6.
Time- ame: Res ic ing he ime- ame whe e AIS messages a e o be included in he
end esul may be use ul o c ea ing a cus om da ase o analysis and emo ing
messages wi h e oneous imes amps, due o noise. This il e can also be used o
excluding messages e e ing o a ime be o e he da ase ’s speci ica ions, caused by
delays du ing hei ansmission.
7.
Noise- il e : In some cases, consecu i e AIS messages o a single essel indica e an
in alid ansi ion be ween he wo poin s [
54
,
55
]. Mo e p ecisely, i he dis ance
be ween consecu i e messages is so la ge ha i would no be possible o a essel o
co e in he co esponding ime ame, his ansi ion is conside ed noise in ou da a
and he second AIS message is emo ed. A ansi ion is conside ed o be imp obable
i he calcula ed speed o he essel o co e he dis ance in ques ion exceeds he
h eshold o 92 km/h (app oxima ely 50 kno s).
8.
Insigni ican acks: Fo he pu pose o p ocessing only meaning ul ajec o ies, all da a
ega ding essels ha ha e less han 10 AIS messages a e he me ging s ep o ou
p ocessing a e disca ded. This h eshold can be adjus ed depending on he use case.
Al hough hese il e s should be applied in mos scena ios, he app op ia e h esholds
o some hea ily ely on he inpu da ase quali y, as well as he na u e o he desi ed
Remo e Sens. 2023,15, 5080 16 o 29
whe e
g(nc)
is he cumula i e o al cos om he s a node o he cu en node
(nc)
,
w(nc
,
ni)
is he g aph weigh be ween he cu en node and he neighbou in ques ion
(ni)
and
h(ni
,
n )
is he esul o he heu is ic unc ion ha es ima es he cos om he neighbo
o he a ge node (n ).
In ou app oach, we al e he A* algo i hm by inco po a ing he knowledge ex ac ed
om he his o ical da a wi hin he cos unc ions. Mo e p ecisely, a penal y unc ion is
added o he calcula ion o he o al cos :
(ni) = [g(nc) + w(nc,ni) + p(nc,ni)] + h(ni,n )
whe e:
p(nc,ni) = w(nc,ni)∗Wh∗hp(nc,ni)
wi h
w(nc
,
ni)
being he weigh (dis ance) be ween he wo nodes as abo e and
hp(nc
,
ni)
being he weigh -penal y based on his o ical da a,
Wh
is a ac o ha de e mines he
impo ance o he his o ical in o ma ion du ing he gap- illing, no mally anging om 0 o
1. Addi ionally, since bo h a e bounded by 0 and 1 (wi h he excep ion o he
Hn
penal y
de ined as la ge han one), he
p(nc
,
ni)
unc ion is in u n bounded by 0 and he eal
dis ance be ween he cells, hus p o iding a penal y no malized o he o iginal g aph. A
high-le el desc ip ion o his sho es pa h me hod can also be ound in Algo i hm 2.
Algo i hm 2 Sho es Pa h
Requi e:
T ansi ion g id (
g
), ‘s a ’ cell (
cs
), ‘end’ cell (
ce
), ac o o conside ing his o ical
in o ma ion (p ac )
Ensu e: Re u n he sho es pa h acco ding o weigh s om cs o ce
.
ge T ansi ionWeigh is a unc ion ha calcula es he ansi ion cos using his o ical in o ma ion
om he g aph and he dis ance be ween he cells, aking in o accoun he ac o
p ac
.
h
is he
heu is ic unc ion, e u ning an es ima ed cos om a cell o he a ge
open ← {cs}
while open is no emp y do
cu ←cell in open wi h smalles o al es ima ed cos (pop)
i cu == ce hen
Re u n ull pa h . he algo i hm ound he sho es pa h
end i
o each neighbo (nb ) o cu do
ncos ←(cos un il cu ) + ge T ansi ionWeigh ( g,cu ,nb ,p ac )
i ncos < cu en min. cos o nb hen
upda e min. cos o nb o ncos
i nb no in open hen
o al es ima ed cos o nb ←ncos +h(nb ,ce)
add nb o open
end i
end i
end o
end while
4.2. Vessel De ec ion Based on Sa elli e Images
Se e al s eps a e equi ed o ex ac ing de ec ions o essels om sa elli e image y.
In his wo k, we p esen a pipeline sui able o bo h op ical and SAR images, as p o ided
by he Eu opean ini ia i e Cope nicus (h ps://www.cope nicus.eu/en, (accessed on 6
Sep embe 2023)); wi h da a coming om Sen inel-2 and Sen inel-1, espec i ely [58,59].
Since op ical images a e e y di e en om SAR images, di e en p ep ocessing s eps
o each kind o image y a e equi ed. Fo SAR image y (i.e., Sen inel-1 da a), we pe o m
co ec ions, such as con e ing SAR geome ies o geo- e e enced geome ies using GDAL
ools in Py hon (e.g., gdalwa p). SAR images come as single band images in h ee di e en
Remo e Sens. 2023,15, 5080 17 o 29
esolu ions: 10 m, 20 m and 60 m. The op ical images (i.e., Sen inel-2 da a) come in RGB
bands oge he wi h an in a ed (IR) band. We s ack he RGB and he SWIR bands oge he
and we compose a panch oma ic image in he GeoTIFF o ma , using 16-bi encoding, and
we pe o m pan-sha pening o inc ease i s esolu ion. The panch oma ic image is c ea ed
by compu ing he a e age o all he 10 m esolu ion bands, ha is, RGB and IR in o de o
ob ain a single high- esolu ion image. The aim o he pan-sha pening echnique is o use
he highe spa ial in o ma ion om he panch oma ic image and he spec al in o ma ion
om a lowe spa ial in o ma ion mul i-spec al image. A ue colo image (TCI) in 10 m
is eadily a ailable in a Sen inel-2 image di ec o y; howe e , we p e e o compose he
panch oma ic pan-sha pened image using he indi idual bands since he TCI image comes
in 8-bi encoding, which is lossy. Main aining a 16-bi encoding will be c ucial o he il e
s age which ollows.
A e we ha e p ep ocessed he images, we di ide hem in o smalle iles, wi h
dimensions o 256
×
256 pixels, in o de o be u he p ocessed mo e easily. Each pixel is
o 10m leng h in each side, meaning ha i co e s an a ea o 100 m
2
. Since we in end o
use an AI-based app oach o de ec ing essels in sa elli e image y, eeding housands o
sa elli e images which co espond o se e al Te aby es o da a in o he ne wo k can c ea e a
se ious bo leneck; i will comp omise he pe o mance and applicabili y o ou app oach
in a densi y maps use case, in which housands o sa elli e images need o be p ocessed.
Thus, we il e ou he image iles which a e edundan , meaning he ones o which we
ha e indica ions ha hey do no con ain essels be o e we eed hem o he neu al ne wo k.
Fo example, a Sen inel-1 ile ha is o ally black depic s he sea and no objec appea s in i .
Fo Sen inel-1 image iles, we use s a is ics and h esholding (i.e., amoun o black/whi e
pixels), while o Sen inel-2 image iles, we use h esholds ha a e based on he di e ence
o pixel alues be ween he ed band and he in a ed band (R-SWIR). We also use he ACL
as a mask o il e ou clouds. E en ually, he il e ing s ep esul s in image iles sizing up
o a leas one o de o magni ude less han he size o he o iginal image.
Fo he essel iden i ica ion in he p ep ocessed sa elli e images, he YOLO 4 neu al
ne wo k a chi ec u e [
60
] was selec ed. The YOLO ne wo k is a CNN-based s a e-o - he-a
solu ion ha has p o en o be e y e icien in simila asks o de e mining mo ing objec s
in images ( anging om mo ing ca s o ai c a s) [
61
–
65
]. P o iding high accu acy wi h
low esponse ime, he YOLO 4 is highly sui able o handling la ge olumes o da a in
an e ec i e manne . In ecen yea s, he e ec i eness o he YOLO 4 e sion has been
e alua ed o de ec ing essels in SAR and op ical images [
66
,
67
]. We ain he NN wi h
essels de ec ed in sa elli e images, using AIS da a as g ound u h. Since we also wan
o de ec he p ecise loca ion o essels, we used an objec de ec ion amewo k ins ead
o a simple CNN. The objec de ec ion amewo k deploys a CNN pe image pixel and
de e mines he bounding box o he de ec ed objec . Ou ained model is able o de ec
essels wi h a p ecision o 92%, as shown in he expe imen esul s p o ided in Table 1.
Table 1.
Vessel de ec ion expe imen esul s using 400 semi-manually labelled Sen inel-1 and Sen inel-
2 sa elli e image y. A o al o 80% o images in he benchma k da ase we e used as aining se and
20% o he images we e used as es se .
Me ic Value
P ecision 92%
Recall 93%
F1-sco e 92%
T ue Posi i e (TP) 80%
False Posi i e (FP) 7%
False Nega i e (FN) 6%
Fo each image, he ou pu o he essel de ec ion includes (a) he bounding box o
he de ec ed essel (wi h espec o he cen e o he image), (b) he class o he de ec ed
objec (e.g., essel, anke , e c.), and (c) a con idence alue ha indica es he possibili y o
Remo e Sens. 2023,15, 5080 18 o 29
he de ec ed objec belonging o he de ec ed class. In a pos -p ocessing ask, we e ain
only he de ec ions ha we e classi ied wi h high-con idence (>0.6), and we geo- e e ence
he coo dina es o he de ec ed objec s by ans o ming hem om opological coo dina es,
so ha hey can be co ela ed wi h he espec i e AIS coo dina es. Hence, ou esul s
includes a lis o de ec ions ha include he image ile ha he de ec ed objec belongs o,
i s geo- e e enced loca ion ( he geo- e e enced cen e o i s bounding box), he acquisi ion
ime o he image. Di e en esul s we e p oduced pe essel ype de ec ed (e.g., Tanke ,
Ca go and Tug o Sen inel-1 image y).
A e we de ec essels in sa elli e images, we co ela e his da ase wi h he espec i e
AIS posi ions. The co ela ion ask in ol es he ollowing s eps:
1.
Spa io- empo al il e ing: The empo al esolu ion o sa elli e images is signi ican ly
lowe han he empo al esolu ion o AIS messages (i.e., he e isi ime o Sen inel
sa elli es is 2–3 days in high co e age a eas, whe eas essels wi h AIS ansponde s
ansmi AIS messages e e y ew seconds o minu es, depending on hei na iga ional
s a us and speed). In o de o be able o co ela e hese wo da a sou ces, o e e y
image, we ex ac all AIS posi ions ha a e loca ed in o he a ea co e ed by he image
du ing a 1 hou ime window, spanning 30 min be o e and a e he image acquisi ion
ime. Fo he il e , we c ea e a empo al index on he geo-da a ame whe e we load
all AIS posi ions, il e ing ou all posi ions ha all ou o he ime window; hen,
we pe o m spa ial joins ha e ain only he posi ions ha a e co e ed by he spa ial
ex en o he image.
2.
In e pola ion: Then, we c ea e ajec o ies o each essel con ained in he da ase .
Fo each ajec o y, we e ie e he posi ion o he essel a he ime he image was
acqui ed by in e pola ing he essel’s loca ions be o e and a e he acquisi ion ime
o he image. The ou pu o his s ep is a snapsho o all AIS essel posi ions which
spa io- empo ally in e sec wi h he image. Mo e speci ically, we p oduce a ile o
each image, s o ing he posi ion o e e y essel loca ed in he spa ial ex en o he
image oo p in a he ime he image was acqui ed.
3.
Fusion: The usion ask ma ches he in e pola ed AIS-posi ions o he p e ious s ep
wi h he essels de ec ed in he image ha a e he ou pu o he essel de ec ion
s ep and he pos -p ocessing s ep. Since he AIS da ase con ains an iden i ie o
each essel, he usion ask assigns each de ec ion o a essel posi ion. We pe o m a
kNN-join be ween he wo da ase s in he ollowing way: Fo each essel de ec ed
in a sa elli e image, we sea ch o he nea es AIS neighbo (i.e., nea es in e pola ed
posi ion o a essel). To achie e his, we s o e he in e pola ed posi ions ha a e he
ou pu o he p e ious s ep in a KD-T ee [68] in o de o speed up he dis ance joins.
A he end o his p ocess, he image essels a e anno a ed acco ding o hei ma ched
AIS ajec o y, hence being an addi ional sou ce o essel acking o he o iginal AIS
da ase s. Vessels de ec ed in sa elli e images ha do no ha e ma ching AIS messages,
gi en a dis ance h eshold, a e conside ed as “da k essels”, meaning ha hey a e ei he
loca ed in a low co e age a ea o hey ha e in en ionally swi ched o hei ansponde s.
In gene al, a essel can only be isible ia AIS and no in a sa elli e image due o one o he
ollowing easons:
•
The esolu ion o Sen inel-1 and Sen inel-2 image y does no allow o he de ec ion
o small essels wi h high con idence. As p e iously men ioned, wi h he highes
esolu ion p o ided by he sa elli e images, each pixel co esponds o an a ea o 100 m
2
(since each pixel is o 10 m leng h). This means ha essels o small sizes can in some
cases be igno ed by he CNN.
•
The AIS posi ion o he essel migh be w ong. Since AIS is a collabo a i e ma i ime
epo ing sys em, a essel’s c ew migh al e he GPS posi ion o he essel when
ansmi ing he AIS messages. The phenomenon is called spoo ing and has been he
subjec o se e al s udies o e he yea s [24,55,69,70].
•
In e pola ion e o : when he AIS messages ha we ha e a ound he acquisi ion ime
o an image ansmi ed by a essel a e no enough, and he essel has changed i s
Remo e Sens. 2023,15, 5080 19 o 29
na iga ional s a us in he mean ime (e.g., a essel suddenly s ops o i accele a es and
changes heading), he es ima ed posi ion o he essel in he in e pola ion s ep migh
di e conside ably om he ac ual posi ion o he essel.
5. Expe imen al Resul s and E alua ion
In his sec ion, he expe imen al esul s o he wo a o emen ioned echniques a e
p esen ed. Fi s , he AIS da ase s used, along wi h he espec i e me ics, a e desc ibed.
Then, o bo h me hods, we p esen and analyze he esul s o ou expe imen s.
5.1. AIS T ajec o y Recons uc ion
5.1.1. Da ase
Fo he pu pose o an expe imen al e alua ion o he p oposed app oach, a da ase
o syn he ic AIS gaps was c ea ed. Mo e p ecisely, ocusing on he eas e n pa o he
Medi e anean Sea, i.e., he Aegean and Le an ine Sea, a sample o 3000 ajec o ies was
p o ided by Ma ineT a ic. A cleaned da ase om he mon h o Oc obe (5–31 Oc obe
2021) was used as a s a ing poin . The ex ac ed ajec o ies we e con inuous essel pa hs,
wi h no empo al gaps la ge han 15 min be ween consecu i e messages. Addi ionally, in
o de o a oid including essel s ops a po a eas, segmen s sho e han 30 min whe e
he essel emained s opped o idle we e emo ed om he ajec o ies. These segmen s
we e subs i u ed by a single ansi ion, om he s a o he s op un il he end, wi h he
co esponding ime in e al sub ac ed om he o al ip du a ion. Fo each ex ac ed
ajec o y, a syn he ic “gap” was o med using hei s a ing and ending poin s. O e all, in
ou e alua ion we conside la ge empo al gaps, meaning ha he syn he ic gaps ange
om a du a ion o 1 h o a ull day (24 h).
Addi ionally, we demons a e he impac o his gap- illing echnique by compa ing
he densi y maps o he o iginal cleaned AIS and he co esponding imp o ed da ase .
Fo his ask, we employ a da ase om he mon h o Oc obe o igina ing om SAT-AIS,
ocusing again on he Aegean and Le an ine Sea (Figu e 11). Du ing he cleaning p ocess,
a 3 min downsampling is applied o he da a.
Figu e 11.
Imp o ed ajec o y o a single essel; in g een— he segmen s om he o iginal AIS
messages, in o ange— he econs uc ed pa s o he ajec o y using he p oposed me hod.
The AIS da ase s we e p o ided by Ma ineT a ic (h ps://www.ma ine a ic.com,
(accessed on 6 Sep embe 2023)). Due o he na u e o he esea ch and comme cial es ic-
ions, suppo ing da a a e no a ailable.
Remo e Sens. 2023,15, 5080 20 o 29
5.1.2. E alua ion Me ics
Fo each syn he ic gap, he econs uc ed ajec o y gi en by he p oposed algo i hm is
e alua ed, as compa ed wi h he essel’s ue pa h. Since we only conside la ge ( empo al)
gaps in his e alua ion, he s aigh -line-in e pola ion is used as a baseline app oach o
compa e i wi h he p oposed gap- illing echnique. E en hough he e ha e been a numbe
o me hods p oposed o sho - e m o ecas ing (usually up o a 1 h ho izon), o he bes o
ou knowledge, he e a e no o he wo ks published ha co e such la ge ime in e als
ha can be used as baselines.
Since he esul ing ajec o y does no necessa ily ha e he same numbe o poin s o
he o iginal, he Fas DTW (Fas Dynamic Time Wa ping) [
71
] me ic was used o de e mine
he app oach’s accu acy. The Fas DTW is an e icien mechanism ha c ea es a ma ching
o any gi en wo ajec o ies be ween hei poin s, esul ing o a w apped pa h along wi h
he o al dis ance be ween each ma ched pai . In his wo k, he mean dis ance be ween
he essel’s ue pa h and he econs uc ed ajec o y is conside ed. The end esul s a e
ca ego ized acco ding o he espec i e gap du a ion in o h ee g oups (1–3, 3–6 and
6–24 h
),
wi h he mean dis ance calcula ed o each ca ego y.
5.1.3. Resul s
Fo he pu poses o a mo e comple e s udy, we expe imen ed wi h ou app oach using
di e en his o ical penal y ac o s (
Wp
) and compa ed hei accu acy. Table 2p o ides
he esul s o he p oposed app oach, as well as he o al a e o imp o emen o e he
s aigh -line in e pola ion, calcula ed as ollows:
Imp .=100 ∗Sli −Ash
Sli
whe e
Sli
is he mean Fas DTW o he s aigh -line in e pola ion and
Ash
is he espec i e
accu acy o he p oposed me hod.
Table 2.
Table Accu acy o he s aigh -line in e pola ion, he p oposed gap illing me hod (A* his .)
and he subsequen imp o emen (%). The mean dis ance be ween he econs uc ed ajec o ies and
he essel’s ue pa h (in km) is calcula ed using he Fas DTW me ic.
Mean Fas DTW Dis ance (km)
T ajec o y Gap (Hou s) 1–3 Imp . (%) 3–6 Imp . (%) 6–24 Imp . (%)
Numbe o ajec o ies 1655 - 664 - 681 -
S aigh -line in e pola ion 1.94 - 5.38 - 16.45 -
A* his . (0.1) 1.91 1.67 4.5 16.34 12.16 26.08
A* his . (0.3) 1.91 1.44 4.47 16.91 12.11 26.37
A* his . (0.5) 1.92 1.19 4.52 15.83 12.12 26.29
A* his . (0.7) 1.94 0.3 4.61 14.21 12.55 23.67
5.2. Vessel De ec ions Based on EO Images
In he second pa o ou expe imen s, we gene a ed h ee ypes o densi y maps
based on EO de ec ions. Fi s , we isualize he densi y o all essels de ec ed in sa elli e
images, o bo h Sen inel-1 and Sen inel-2 images. Then, we isualize he densi y o all da k
essels de ec ed in sa elli e images, i.e., essels ha we e only de ec ed in sa elli e images
wi hou ma ching AIS posi ions. This se o densi y maps will help us iden i y “da k” a eas
(e.g., g id cells) wi h inc eased a ic o “da k” essels, which can ei he be a eas o low
AIS co e age o a eas associa ed wi h illegal ac i i ies (e.g., sanc ioned a eas). Finally, an
imp o ed map based on bo h AIS and EO de ec ion is p o ided.
5.2.1. Da ase
Fo his e alua ion, we ocus again on essel ac i i y in Eu opean wa e s o he mon h
o Oc obe 2021. EO da a include images om bo h Sen inel-1 and Sen inel-2 o he pe iod.
Remo e Sens. 2023,15, 5080 21 o 29
Rega ding AIS da a, we use da ase s om bo h Te es ial s a ions o he maps showcasing
he de ec ion o da k essels. Fu he mo e, SAT-AIS a e used, combined wi h EO de ec ions,
o demons a e he e ec i eness o he p oposed app oach. Ou CNN model was ained
wi h 2019 da a and is able o dis inguish essels om clouds (in he case o Sen inel-2),
wa es, and land pa cels. One o he ad an ages o he deep lea ning-based me hods is ha
he model can be ained o de ec essels ega dless o he backg ound in con as o he
h eshold-based me hods. This is mos ly highligh ed in op ical (Sen inel-2) images. A ew
examples o de ec ions wi h high con idence can be obse ed in Figu e 12.
Figu e 12.
Posi i e de ec ions om he p oposed CNN-based amewo k, o bo h SAR images om
Sen inel-1 ( op h ee) and op ical images om Sen inel-2 (bo om h ee), om Oc obe 2021.
We acqui e Sen inel-1 and Sen inel-2 da a om he Cope nicus open access hub
using he Sen inelSa Py hon API (h ps://scihub.cope nicus.eu, (accessed on 6 Sep embe
2023)). We also use he Alaska Sa elli e Facili y eposi o ies o Sen inel-1 image y (h ps:
//as .alaska.edu, (accessed on 6 Sep embe 2023)) as a back-up eposi o y. Fo Sen inel-1
da a, we download IW GRD p oduc s coming om bo h S1A and S1B sa elli es, and o
Sen inel-2 da a, we download all bands a ailable o S2A and S2B p oduc s. The AIS
da ase s we e p o ided by Ma ineT a ic (h ps://www.ma ine a ic.com, (accessed on
6 Sep embe 2023)); due o he na u e o he esea ch and due o comme cial es ic ions,
suppo ing da a a e no a ailable.
5.2.2. Resul s
Fi s , ollowing he pipeline desc ibed in he me hodology chap e , we ex ac ed
de ec ions o essels o he Eu opean wa e s o he ull mon h o Oc obe . This p ocess
was conduc ed o bo h he Sen inel-1 and Sen inel-2 da ase s. Mo eo e , a densi y map
was gene a ed o each da a sou ce, based on he o al numbe o de ec ions loca ed in each
g id cell. Fo he pu poses o his wo k, we used a g id wi h cells o leng hs o 10 km (see
Figu e 13).
Fu he mo e, we use he espec i e AIS da a o ha pe iod and a ea o de e mine
which o he de ec ions canno be ma ched o he exis ing ajec o ies and hus co espond o
da k essels. In an ini ial e iew o ou indings, he co ela ion be ween highly conges ed
a eas and da k essels was no ed [
14
]. This phenomenon can be a ibu ed o a possible
high numbe o packe -collisions in he a ea [
72
,
73
], whe e, due o high a ic, some AIS
messages a e no being ecei ed. In o de o coun e ha e ec and o di e en ia e be ween
low AIS co e age due o packe collision and in en ional swi ch-o o AIS ansponde s,
we pe o med an analysis desc ibed in his sec ion, which is based on iden i ying a eas
Remo e Sens. 2023,15, 5080 22 o 29
(e.g., g id cells) whe e essels a e in close dis ance, hus inc easing he isk o losing AIS
packe s due o conges ion.
Figu e 13.
Densi y maps based on essel de ec ions om EO image y, o bo h Sen inel-1 (
le
) and
Sen inel-2 (
igh
) da a, o essels in EU wa e s o he mon h o Oc obe 2021. The g id is comp ised
o cells wi h edges o 10 km leng h, wi h he densi y compu ed as he o al numbe o de ec ions o
each cell (acco ding o he colo cap ion).
To achie e his, we applied he ollowing o mula o di e en snapsho s o AIS da a
(we used he same snapsho s o he espec i e Sen inel-1 images used o iden i y da k
“cells”). Fo each g id cell, we calcula ed he mean nea es neighbou dis ance, i.e., he
mean dis ance be ween a essel and i s neighbou s:
NND =∑nNND
n
wi h
NND
being he Dis ance wi h he Nea es Neighbou essel and
n
he o al numbe
o essels. This me ic indica es ha he lowe he mean dis ance be ween neighbo ing
essels is, he highe he isk o conges ion. Thus, we de ine he “de ec abili y” maps as he
maps ha showcase conges ed a eas ha migh lead o de ec abili y issues. We cons uc ed
de ec abili y maps o he whole a ea o in e es ; o isualiza ion pu poses, we include he
espec i e map o he Medi e anean Sea in Figu e 14.
Figu e 14. De ec abili y map o he Medi e anean Sea a ea o he mon h o Oc obe 2021.
Using such in o ma ion, a co ec ed densi y is calcula ed o each g id cell wi h he
esul ing densi y map depic ed in Figu e 15.
Remo e Sens. 2023,15, 5080 23 o 29
Figu e 15.
Densi y map o da k essels de ec ed using image y om Sen inel-1 co ela ed wi h
SAT-AIS da a, o he mon h o Oc obe 2021 (acco ding o he colo cap ion).
Finally, a e p oducing he da k essel densi y maps, we use his in o ma ion o
complemen he Sa -AIS based densi y maps. He e, i s we gene a ed densi y maps o
Oc obe 2021 using only SAT-AIS da a (numbe o posi ions/cell). Then, he espec i e
densi y map o da k essels o he same pe iod was c ea ed. We conside only he da k
essels, and no all essels de ec ed by sa elli es o duplica e elimina ion (so ha we do
no coun he same essel posi ion wice); using his combina ion o da a sou ces (SAT-AIS
and Sen inel-1), we p oduce he combined densi y map, as shown in Figu e 16.
Figu e 16.
Densi y maps o essels in EU wa e s o he mon h o Oc obe 2021, coun ing numbe o
essels pe cell o 10 km edge leng h, om SAT-AIS only (
op le
), de ec ions om Sen inel-1 da a
(bo om le ) and o he combined da ase ( igh ).
Remo e Sens. 2023,15, 5080 24 o 29
6. Discussion
6.1. AIS T ajec o y Recons uc ion
As indica ed by he esul s, he p oposed me hod p o ides a signi ican imp o emen
o e he s aigh -line in e pola ion echnique, especially o la ge empo al gaps. Mo e
p ecisely, ou expe imen s show ha i he AIS gap las s be ween 3 and 6 h, he p oposed
me hod p o ides an imp o emen up o 16.91% o e he s aigh -line in e pola ion esul s,
while his numbe goes up o 26.37% o e en la ge gaps (6–24 h). The inclusion o he
his o ical in o ma ion du ing he ajec o y econs uc ion is qui e bene icial; a ac o o
0.3 upon he espec i e weigh s esul s in a be e accu acy. Las ly, since he inco po a ed
g aph is based solely on g id cells ha include sea a eas, ansi ions o e land is a oided in
mos cases. This ac deems he p oposed app oach as mo e sui able o all AIS gaps o e
1 h han he s aigh -line in e pola ion.
Using he p oposed mechanism o imp o ing a da ase wi h AIS gaps, we expe i-
men ed on SAT-AIS messages om Oc obe 2021 and ex ac ed a new se o ajec o ies
(Figu e 11). Densi y maps o bo h he o iginal as well as he imp o ed da ase s (Figu e 17)
we e gene a ed using a 10 km g id cell esolu ion. As seen in he esul s, ou app oach
c ea es meaning ul ajec o ies in place o he AIS gaps. Majo pa hways be ween po s a e
ei he unco e ed o enhanced in he imp o ed da ase , while a clea e image abou a ic
in less a e sed wa e s is also p o ided.
Figu e 17.
Densi y maps o he cleaned ansmi ed AIS da a (
le
) and o co esponding he
imp o ed ajec o ies gene a ed using he p oposed me hod (
igh
). The da ase e e s o SAT-AIS
messages om he mon h o Oc obe 2021 o he Aegean and Le an ine Sea, while he maps depic
he o al ime essels spen in each g id cell in hou s), acco ding o he colo cap ion.
6.2. Da k Vessel Iden i ica ion
We demons a ed he capabili ies o de ec ing da k essels, i.e., ships ha do no
ansmi AIS posi ional messages, h ough he use o EO image y o Eu ope o a ull
mon h pe iod. E alua ing he gi en EO images and compa ing he da a om he wo
da a sou ces, one can no e ha he cloud co e age p oblem is ob ious in he case o
Sen inel-2. Mo eo e , we can obse e ha e en in Sen inel-1 he e a e signi ican gaps in
co e age, especially in a eas a away om he coas line. This is a known issue o bo h
Eu opean Space Agency (ESA) and also some comme cial sa elli es. Fo example, he
Sen inel-1 Oc obe o bi s can be obse ed in Figu e 18, demons a ing limi ed co e age in
he high seas. These obse a ions highligh he need o ha ing mul iple sou ces o da a
complemen ing each o he o achie e be e si ua ional awa eness o e an a ea o in e es .
Reasonably, he combined densi y map o AIS- and EO-based de ec ions esembles he
SAT-AIS densi y map, as he limi ed e isi ime o Sen inel sa elli es compa ed o he AIS
message equency do no allow o signi ican di e ences o be highligh ed (Figu e 16).
Howe e , looking a he combined densi y map mo e ca e ully, we can obse e ha some
a eas ha appea in o ange/ ed appea sligh ly da ke (e.g., Gib al a , No way, Iceland,
English Channel, e c.). This means ha a eas ha a e known o be dense using SAT-AIS a e
Remo e Sens. 2023,15, 5080 25 o 29
e en dense in eali y. A eas ha a e bo h o high SAT-AIS densi y and da k essel densi y
a e p one o packe collisions; al hough ESA sa elli es a e limi ed by hei low e isi ime
and co e age, hey can s ill be used o highligh hese a eas, al hough he di e ences o he
combined densi y map and he SAT-AIS densi y map a e no signi ican .
Figu e 18.
Sa elli e co e age based on Sen inel-1 o bi o he mon h o Oc obe 2021, indica ing
limi ed co e age in he high seas. The yellow ec angle is he a ea o in e es o his s udy.
7. Conclusions and Fu u e Wo k
In his wo k, a p o o ype o a AIS da a handling pipeline capable o p oducing im-
p o ed densi y maps was p esen ed. Fi s , a se ies o il e s we e p esen ed in o de o clean
he aw AIS messages and es ic he da a o he app op ia e a ea and ime ame o in e es .
Fu he mo e, wo echniques o imp o ing he AIS da a, comple ing i wi h addi ional
essel posi ions, we e desc ibed. The i s is a g aph-based echnique o econs uc ing
ajec o ies when la ge empo al gaps occu in he da a, while he seconds ex ac s essel
de ec ions om sa elli e image y. Finally, he pipeline allows o c ea ing a spa ial g id
acco ding o he use ’s equi emen s and calcula ing he cumula i e ime he essels spen
wi hin i s cells, as well as he gene a ion o he densi y map isualiza ion o he esul s.
In o de o econs uc ajec o ies wi h la ge empo al gaps be ween hei messages, a
gap- illing mechanism was also in oduced. Based on his o ical da a, his p ocess epu -
poses he A* algo i hm so ha pa e ns o mo emen o pas ajec o ies a e conside ed
when disco e ing he pa h he essel ollowed du ing he AIS gap. Expe imen s on eal
AIS da a ha e shown ha ou me hod esul s in a signi ican imp o emen o e he linea
in e pola ion app oach, up o 15% and 26% o empo al gaps o 3–6 and 6–24 h, espec-
i ely. This allows o he imp o emen o AIS da ase s and consequen ly he c ea ion o
mo e accu a e densi y isualiza ions. Fu he mo e, he mul i-s ep p ocess o ex ac ing
geo- e e enced essel de ec ions om op ical and SAR images elies on he combina ions
o de ailed p ep ocessing, a CNN a chi ec u e and a usion mechanism. The p oposed
app oach allows o no only comple ing AIS ajec o ies wi h addi ional essel posi ions,
bu also disco e “da k essels”, whose posi ions canno be accoun ed o by he AIS. Real
wo ld da a om Cope nicus and AIS s a ions we e used o demons a e he e ec i eness
o his mechanism.
In he u u e, we in end o s udy he e ec o limi ing he his o ical da a used du ing
he ajec o y econs uc ion o he essel ype in ques ion, o a esul mo e sui ed o
he occasion. Addi ionally, li e-long lea ning mechanisms will be u ilized so ha ecen
