Ci a ion: Pi onak, R.; Mucha, J.;
Dobis, L.; Ja o ka, M.; Ma usin, M.
CloudSa Ne -1: FPGA-Based
Ha dwa e-Accele a ed Quan ized
CNN o Sa elli e On-Boa d Cloud
Co e age Classi ica ion. Remo e Sens.
2022,14, 3180. h ps://doi.o g/
10.3390/ s14133180
Academic Edi o s: Massimiliano
Pas ena and Luca Fanucci
Recei ed: 3 June 2022
Accep ed: 27 June 2022
Published: 2 July 2022
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2022 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
emo e sensing
A icle
CloudSa Ne -1: FPGA-Based Ha dwa e-Accele a ed Quan ized
CNN o Sa elli e On-Boa d Cloud Co e age Classi ica ion
Radosla Pi onak 1, Jan Mucha 2,* , Lukas Dobis 1, Ma in Ja o ka 1and Ma ek Ma usin 1
1Zai a s. .o., Plyna enska 499/1, 60200 B no, Czech Republic; [email p o ec ed] (R.P.);
[email p o ec ed] (L.D.); [email p o ec ed] (M.J.); [email p o ec ed] (M.M.)
2Depa men o Telecommunica ions, B no Uni e si y o Technology, Technicka 12,
61600 B no, Czech Republic
*Co espondence: [email p o ec ed]
Abs ac :
CubeSa s, he nanosa elli es and mic osa elli es wi h a we mass up o 60 kg, accompanied
by he cos dec ease o accessing he space, ampli ied he apid de elopmen o he Ea h Obse a ion
indus y. Acqui ed image da a se e as an essen ial sou ce o in o ma ion in a ious disciplines
like en i onmen al p o ec ion, geosciences, o he mili a y. As he quan i y o emo e sensing da a
g ows, he bandwid h esou ces o he da a ansmission (downlink) a e exhaus ed. The e o e, new
echniques ha educe he downlink u iliza ion o he sa elli es mus be in es iga ed and de eloped.
Fo ha eason, we a e p esen ing CloudSa Ne -1: an FPGA-based ha dwa e-accele a ed quan ized
con olu ional neu al ne wo k (CNN) o sa elli e on-boa d cloud co e age classi ica ion. We aim o
explo e he e ec s o he quan iza ion p ocess on he p oposed CNN a chi ec u e. Addi ionally, he
pe o mance o cloud co e age classi ica ion by biomes di e si y is in es iga ed, and he ha dwa e
a chi ec u e design space is explo ed o iden i y he op imal FPGA esou ce u iliza ion. Resul s o
his s udy showed ha he weigh s and ac i a ions quan iza ion adds a mino e ec on he model
pe o mance. Ne e heless, he memo y oo p in educ ion allows he model deploymen on low-
cos FPGA Xilinx Zynq-7020. Using he RGB bands only, up o 90% o accu acy was achie ed, and
when omi ing he iles wi h snow and ice, he pe o mance inc eased up o 94.4% o accu acy wi h a
low alse-posi i e a e o 2.23% o he 4-bi wid h model. Wi h he maximum pa alleliza ion se ings,
he ha dwa e accele a o achie ed 15 FPS wi h 2.5 W o a e age powe consump ion (0.2 W inc ease
o e he idle s a e).
Keywo ds:
CNN; FPGA; ha dwa e accele a o s; image p ocessing; on-boa d p ocessing; quan iza ion
1. In oduc ion
O e he las decade, he Ea h Obse a ion (EO) indus y has expe ienced a d ama ic
dec ease in he cos o accessing space [
1
]. Wi h he in oduc ion o CubeSa s, nanosa el-
li es and mic osa elli es wi h we mass up o 60 kg [
2
], he apid de elopmen o emo e
sensing echnologies was ampli ied [
3
]. As o 2021, mo e han 1500 CubeSa s ha e been
launched [
4
], and acco ding o [
5
], i will inc ease up o a housand sa elli es pe yea
ill 2028. Na u ally, as he numbe o sa elli es g ows, sa elli e image y becomes eadily
a ailable. Ha es ed da a plays a signi ican ole in a ious disciplines like en i onmen al
p o ec ion, ag icul u e enginee ing, land o mine al esou ce explo a ion, geosciences, o
mili a y econnaissance [
6
,
7
]. In line wi h he amoun o emo e sensing da a acqui ed,
he bandwid h esou ces o he da a ansmission inclines o be o e loaded. The e-
o e, new echniques o e icien bandwid h esou ces managemen mus be in es iga ed
and de eloped.
Se e al s udies es ima e ha app oxima ely 67% o he Ea h’s su ace is co e ed
wi h clouds [
6
,
8
,
9
]. Consequen ly, mos o he emo e sensing image ies (RSI) will be
con amina ed by hem, which de alues he quali y o RSI and nega i ely a ec s he pos -
p ocessing [
6
]. Cloudy condi ions impai sa elli e senso capabili ies o ob ain clea iews
Remo e Sens. 2022,14, 3180. h ps://doi.o g/10.3390/ s14133180 h ps://www.mdpi.com/jou nal/ emo esensing
Remo e Sens. 2022,14, 3180 2 o 21
o he Ea h’s su ace, and hence he quick and accu a e de ec ion o he cloudy images is
necessa y [
6
,
10
,
11
]. In gene al, he cu en me hods o cloud co e age es ima ion o classi i-
ca ion a e mainly ca ego ized in o adi ional and machine-lea ning-based app oaches [
12
].
T adi ional ones consis o h eshold-based ( ixed o adap i e), ime di e en ia ion, and
s a is ical me hods. The h eshold-based app oaches ely on a isible e lec ion and in-
a ed empe a u e o he clouds, he e o e i s pe o mance weakens on low-con as ed
(cloud s. su ace) images [
13
–
15
]. Time di e en ia ion me hods e ec i ely iden i y he
changing pixel alues as clouds in mul i- empo al images, howe e , hey do no con-
side changes in he op o a mosphe e e lec ance a ec ed by loods [
12
,
16
]. S a is ical
me hods combine spec al and spa ial ea u es ex ac ed om RSIs wi h classical ma-
chine lea ning algo i hms (suppo ec o machine, decision ee), bu hey lack o ob ain
he desi ed esul s [
17
,
18
]. To sum up, adi ional me hods p o ide some capabili ies
o cloud de ec ion, hough, hey a e suscep ible o he backg ounds, a e non-uni e sal
and subjec i e [12].
A mo e e icien app oach o cloudy image de ec ion comp ises con olu ional neu-
al ne wo ks (CNNs), simple linea i e a i e clus e ing, o seman ic segmen a ion algo-
i hms [
12
]. Especially a ac i e a e CNNs, which p o ide s a e-o - he-a esul s o many
di e en asks, including image classi ica ion, segmen a ion, and objec de ec ion. This
success is o en achie ed hanks o models wi h a huge numbe o pa ame e s which means
he la ge size and limi ed abili y o he deploymen on esou ce-cons ained ha dwa e.
In ecen yea s, he e has been a endency o deploy hese models in line wi h he edge
compu ing pa adigm on esou ce-cons ained ha dwa e [
12
,
19
–
21
]. Va ious ha dwa e
accele a o s a e a ailable on he ma ke anging om mic ocon olle s o smalle models
o boa ds equipped wi h GPU, isual p ocessing uni (VPU), o ield-p og ammable ga e a -
ay (FPGA). FPGA in pa icula p o ides in e es ing capabili ies in e ms o cos , lexibili y,
pe o mance, and powe consump ion. A possible disad an age is he long ime o ma ke
in compa ison o GPU o VPU solu ions. Ne e heless, his gap is being closed by ecen
ad ancemen s in he ha dwa e deploymen o machine lea ning models [
22
,
23
] c ea ed in
well-known machine lea ning amewo ks like Py o ch o Tenso low. Conside ing he
payload limi a ions o he CubeSa s, he op imal solu ion o he CubeSa ’s cloud de ec ion
sys em is a sys em es ima ing RSI cloud co e age unning di ec ly on boa d. To educe he
cos s and de elopmen ime o such eal- ime de ec ion sys ems, Comme cial-O -The-Shel
(COTS) componen s p o ide a a o able deploymen op ion [
24
]. The c ucial c i e ion
o an onboa d de ec ion sys em is i s powe consump ion, whe eas he usual limi is
below
5 W
and he a io o he alsely disca ded images below
2%
[
12
,
19
,
20
]. Gene ally,
emo e-sensing sa elli es can be equipped wi h a pale e o senso s p o iding in o ma ion
in a ious bands. F om he simples one (RGB image ies) ollowed by mul ispec al im-
age ies (usually a combina ion o RGB and nea -in a ed band (NIR)) o he hype spec al
image ies p o iding a complex spec um o he sensed a ea [25,26].
In line wi h he abo e men ioned, Zhang e al. [
27
] in oduced a ligh weigh CNN
o cloud de ec ion based on U-Ne using ed, g een, blue, and in a ed wa eband images
om he Landsa -8 da ase . Applying he
LeGall-5/3
wa ele ans o m (4 le els) o
da ase comp ession and p ocessing ime accele a ion, he au ho s epo ed
94.3%
o o e all
accu acy unning on an ARM-based pla o m. Simila ly, in [
28
], he au ho s applied
dep hwise sepa able con olu ions o comp ess he model o U-Ne and accele a e he
in e ence speed. The S udy epo ed he bes accu acy o
90.54%
e i ied on Landsa
8 emo e sensing images. Ano he u iliza ion o a ligh weigh MobU-Ne ained on
Landsa 8 da ase and using JPEG comp ession s a egy was pe o med by [
29
]. The
achie ed o e all accu acy was a ound
93.1%
o a model deployed on ARM9 p ocesso
on Zynq-7020 boa d.
Maskey e al. [3]
p oposed an ul aligh CNN designed o on-o bi
bina y image classi ica ion called CubeSa Ne . The model was ained on BIRDS3 sa elli e
images and deployed on ARM Co ex M7 MCU. An accu acy o 90% was achie ed when
classi ying images as “bad” o cloudy, sunbu n , acing space, o sa u a ed images and
“good” in all o he cases. A p omising me hod o cloud de ec ion using RS-Ne and RGB
Remo e Sens. 2022,14, 3180 3 o 21
bands exclusi ely was published by [
30
]. Fo model aining, he Sen inel-2 da ase was
used, and
76%
o accu acy was epo ed by he model deployed on an ARM-based pla o m.
Ano he possibili y is o use he Fo wa ds Looking Image ins umen , which p o ides
analysis o he upcoming en i onmen o he sa elli e. This app oach was examined
in [
31
], es ing a ious ligh weigh CNNs deployed on he Zynq-7020 boa d using FPGA.
The au ho s epo ed high accu acy o
98%
, howe e , 100 images only we e used o
es ing.
Vieille ille e al. [32]
in es iga ed he possibili ies o he deep neu al ne wo k
(DNN) dis illa ion p ocess in o de o educe he size o DNN while accommoda ing
e iciency in e ms o bo h accu acy and in e ence cos . The au ho s we e able o educe he
numbe o DNN pa ame e s om se e al million o less han one wi h a minimal d op in
pe o mance in he image segmen a ion p ocess.
To sum up, ligh weigh CNNs p o ide a compe i i e on-boa d cloud de ec ion pe -
o mance in compa ison o he s a e-o - he-a deep con olu ional neu al ne wo ks, like
CDNe V1 [
6
], CDNe V2 [
10
] o CD-FM3SFs [
33
]. CDNe V1 is a neu al ne wo k o cloud
mask ex ac ion om ZY-3 sa elli e humbnails wi h he accu acy o
96.47%
[
6
]. I s ex-
ended e sion, CDNe V2, ocuses on adap i ely using mul i-scale ea u e maps and
emedying high-le el seman ic in o ma ion dilu ed a decode laye s o imp o e cloud
de ec ion accu acy wi h cloud-snow coexis ence. The au ho s con i med he obus ness
o he p oposed me hod using alida ion on se e al o he da ase s like Landsa -8 o GF-1.
La ely,
Li e al. [33]
in oduced a ligh weigh ne wo k o cloud de ec ion, using mul iscale
spec al and spa ial ea u es (CD-FM3SFs) using Sen inel-2A mul ispec al images. The
bes accu acy o 98.32% was achie ed using he CPU as a compu a ional uni .
To he bes o ou knowledge, he CloudScou cloud de ec ion me hod p oposed by
Giu ida e al. [
34
] and la e ex ended by Rapuano e al. [
20
] is he mos ela ed wo k o his
s udy. The me hod was de eloped in he ame o he
Phisa -1
ESA mission, which exploi s
a hype spec al came a o dis inguish be ween he clea and cloud-co e ed images. To
educe he bandwid h, he mission has se a c i e ion ha only images ha p esen less han
70%
o he cloudiness a e ansmi ed o he g ound. CloudScou was ained using Sen inel-
2 hype spec al da a and achie ed he
92%
o accu acy,
1%
o alse posi i es wi h he powe
consump ion o 1.8 W deployed on e-con igu able My iad-2 VPU by Mo idius In el [
34
].
Ne e heless, he au ho s iden i ied mul iple d awbacks due o he My iad-2 design, which
is no speci ically sui able o he space en i onmen (no based on a adia ion- ole an
echnology) [
20
]. The e o e, he au ho s ex ended hei wo k and p oposed an FPGA-based
ha dwa e accele a o o CloudScou CNN. The au ho s compa ed he My iad-2 VPU
wi h wo FPGA boa ds: Zynq Ul ascale+ ZCU106 de elopmen boa d and Xilinx Kin ex
Ul ascale XQRKU060 adia ion-ha dened boa d. Resul s ob ained by Zynq Ul ascale+
ZCU106 show ha he FPGA-based solu ion educed he in e ence ime by 2.4 imes
(141.68 ms) bu a he cos o 1.8 imes g ea e powe consump ion (3.4 W) [
20
]. In e ence
ime es ima ed o he Xilinx Kin ex Ul ascale XQRKU060 boa d was 1.3 imes as e
(264.7 ms) in compa ison wi h he My iad-2 de ice, howe e , he powe consump ion was
no epo ed.
Rega ding he p esen ed achie emen s o he ela ed wo ks and ends in he Cube-
Sa s de elopmen , we may expec a new e a o sma nanosa elli es equipped wi h e-
con igu able, p og ammable ha dwa e accele a o s wi h an on-demand edge compu ing
pa adigm a payload le el [
3
,
12
,
19
,
20
,
27
–
29
,
31
,
34
]. A usual aspec o he p esen ed s udies
is he employmen o mul ispec al o hype spec al RSI o he cloud de ec ion sys em.
Gene ally, he bands’ composi ion o mul i/hype spec al RSI di e s o indi idual mis-
sions, ye all a e equipped wi h an RGB came a. The e o e, a cloud de ec ion sys em buil
on RGB bands only may p o ide be e po abili y o a ious missions independen o
i s mul i/hype spec al bands. In addi ion, he RGB came as a e se e al imes cheape
and mo e con enien o sho - e m CubeSa s missions. To he bes o ou knowledge, we
iden i ied only h ee s udies [
20
,
21
,
31
] ha pe o med deploymen and e alua ion o he
CNN-based cloud de ec ion me hod on an FPGA-based pla o m. Hence, in he scope o
his s udy, we would like o p esen CloudSa Ne -1: an FPGA-based ha dwa e-accele a ed
Remo e Sens. 2022,14, 3180 4 o 21
quan ized CNN o sa elli e on-boa d cloud co e age classi ica ion. Mo e speci ically, we
aim o:
•
explo e e ec s o quan iza ion in oduced o he p oposed CNN a chi ec u e o cloud
co e age classi ica ion,
•
in es iga e and op imize he pe o mance o cloud co e age classi ica ion by biomes
di e si y and i s alse-posi i e iden i ica ions,
•
explo e ha dwa e a chi ec u e design space o iden i y op imal FPGA esou ce u iliza ion.
The es o he pape is o ganized as ollows. Sec ion 2.1 desc ibes he used da ase
and i s p ep ocessing. Me hodology is desc ibed in Sec ion 2.3. In Sec ion 3, he esul s a e
summa ized. The discussion can be ound in Sec ion 4and he conclusions a e d awn in
Sec ion 5.
2. Ma e ials and Me hods
2.1. Da ase
Fo he pu pose o his s udy, he Landsa 8 Cloud Co e Assessmen Valida ion
da a (
L8
biome da ase ) [
35
] was used. The
L8
biome da ase o e s a balanced cloud
dis ibu ion and di e se se s o land and wa e co e , which makes i a sui able sou ce
o da a o he p oposed CNN-based classi ica ion model. The
L8
biome da ase was
acqui ed by he Landsa 8 Ope a ional Land Image (OLI) and The mal In a ed Senso
(TIRS) [
36
]. Fu he mo e, da a a e o ho ec i ied and co ec ed o e ain elie using
Le el-1T p ocessing [37].
The
L8
biome da ase consis s o 96 scenes di ided in o 8 biomes. The scene size is
185 km by 180 km, and each scene con ains 11 mul ispec al bands wi h a esolu ion o 30 m
pe pixel (excep bands 8, 10, and 11, which a e no used in his wo k). Manually anno a ed
cloud co e age is s o ed as a cloud alida ion mask. The cloud alida ion mask is an image
whose pixel alues con ain in o ma ion abou he le el (o class) o cloudiness, in e p e ed
using he ollowing Table 1. The example o he scene image (na u al colo composi ion)
om he
L8
biome da ase can be ound in Figu e 1a, wi h i s espec i e cloud mask in
Figu e 1b.
(a) (b)
Figu e 1. L8
Biome da ase image pa ch example (
a
) econs uc ed om bands B4, B3, and B2 wi h i s
associa ed mul i-class cloud mask (b) [35]. (a) Image pa ch. (b) Cloud alida ion mask.
Remo e Sens. 2022,14, 3180 5 o 21
Table 1. In e p e a ion o he L8 biome da ase cloud mask pixel alues [35].
Value In e p e a ion
0 Fill
64 Cloud Shadow
128 Clea
192 Thin Cloud
255 Cloud
Two cloud mask classes ( hin cloud and cloud) a e ca ego ized as cloud pixels. F om
hese pixels, he Cloud Co e Assessmen (CCA) is compu ed as a a io o cloud pixels
o all pixels wi h alues exp essed in pe cen age [
35
]. The a e age CCA alue o one
scene is 48.35%. The dis ibu ion o he CCA alues o he
L8
biome da ase scenes is
shown in Figu e 2. Scenes a e ca ego ized by hei a ea o cap u e in o biome classes by he
In e na ional Geosphe e-Biosphe e P og amme [
38
] in o 8 ollowing biomes: Ba en (BA),
Fo es (FO), G ass/C ops (GC), Sh ubland (SH), Snow/Ice (SI), U ban (UR), Wa e (WA),
We lands (WE). They a e dis inguishable om each o he by hei isual p ope ies, and
hey ha e a ious in ensi ies o cloud o e ain con as , which leads o di e en challenges
o he cloud de ec ion sys em wo king wi h RGB da a. Fo example, he biomes wi h
sha p cloud o e ain con as , like G ass/C ops, ha e a la ge alue o he de i a i e a he
ansi ion be ween e ain and cloud. The e o e G ass/C ops biome is easy o classi y as
cloud bo de s isibly s and ou om he biome’s e ain. On he con a y, he o he biomes
like Snow/Ice ha e a e ain wi h cloud-like ea u es, which may lead o a la ge numbe
o alse posi i es in classi ie p edic ions as hei e ain blends wi h clouds. Examples o
image pa ches o each biome o he L8 biome da ase a e shown in Figu e 3.
29
554 4
12
5 5
3
24
0–10%
10–20%
20–30%
30–40%
40–50%
50–60%
60–70%
70–80%
80–90%
90–100%
0
5
10
15
20
25
30
CCA ange
Numbe o scenes
Figu e 2. Dis ibu ion o he L8 biome da ase scenes CCA alues [35].

Remo e Sens. 2022,14, 3180 6 o 21
(a) (b) (c)
(d) (e) ( )
(g) (h)
Figu e 3.
Image pa ches examples o each biome in he
L8
biome da ase [
35
]. (
a
) Ba en. (
b
) Fo es .
(c) G ass/C ops. (d) Sh ubland. (e) Snow/Ice. ( ) U ban. (g) Wa e . (h) We lands.
2.2. Da a P ep ocessing
The image pa ch o each scene is a na u al colo composi e om he combina ion o
bands B4 ( ed), B3 (g een), and B2 (blue). Values in pa ch images a e e-scaled om he
ange 0–65,535 o 0–255 using a MinMax no maliza ion. Pa ch images in he
L8
biome
da ase a e geo e e enced. The o bi pa h o he Landsa -8 does no go s aigh om sou h
o no h. The scene acquisi ion ollows he o bi pa h o he sa elli e. The e o e he image
appea s o be o a ed o il ed, like in Figu e 4a. Redundan geo e e encing in o ma ion
can be neglec ed when de ec ing clouds om sa elli e images. Nex , he black (no-da a)
pa s o he image need o be emo ed. The emo al o he black pa s consis s o wo s eps.
Fi s , he image is o a ed, so he ac ual image da a a e pa allel o he whole scene image,
as shown in Figu e 4b. The o a ion is using a nea es -neighbo in e pola ion me hod.
Then, he image is c opped o lowe esolu ion ( om app ox. 8000
×
8000 o app ox.
6400 ×6400), so only image da a a e p ese ed, as illus a ed in Figu e 4c.
Remo e Sens. 2022,14, 3180 7 o 21
0 2000 4000 6000
7000
6000
5000
4000
3000
2000
1000
0
(a)
0 2000 4000 6000
7000
6000
5000
4000
3000
2000
1000
0
(b)
0
1000
2000
3000
4000
5000
6000
6000
5000
4000
3000
2000
1000
0
(c)
Figu e 4.
L8 Biome da ase scene du ing di e en p ep ocessing s eps [
35
]. (
a
) Image pa ch. (
b
) Ro-
a ed pa ch. (c) C opped pa ch wi h iling.
Image pa ch (wi h dimensions app ox. 6400
×
6400
×
3) is c opped o
512 ×512 ×3
iles, acco ding he whi e lines in Figu e 4c, omi ing iles a he edge ha do no ha e ull
esolu ion. Each pa ch has a sligh ly di e en esolu ion a e c opping, which causes a
di e en numbe o gene a ed iles pe pa ch (app ox. 140). F om 8 biomes each con aining
12 scenes, he e a e in o al 13,525 iles. The o iginal CCA alues o he scene om
Figu e 2
do no apply o indi idual iles. Gene a ed iles usually co e cloudy o cloud- ee a eas.
This gene a es ewe iles wi h balanced cloud co e age (o CCA alue) in he inal da ase
( ade-o o c ea ing many iles om ewe image pa ches). Tiles wi h
CCA ≥
70% a e
ca ego ized as cloud and he es a e ca ego ized as non-cloud iles. Each o he 13,525 iles
has been assigned a co esponding bina y cloud co e age label. To p ese e he e enly
dis ibu ed cloud co e age in he ain, alida ion and es da ase , he iles om a single
image pa ch a e di ided in o 5 CCA quin iles: 0–20%, 20–40%, 40–60%, 60–80%, 80–100%.
The dis ibu ion o he iles and hei CCA alues pe biome o he ull
L8
biome da ase
is isualized in Figu e 5. The iles om each pa ch CCA quin ile a e di ided o ain,
alida ion and es da ase s in he a io 2:1:7, wi h he cohe en a ia ion o he biomes and
hei CCA alues, as isualized in Figu e 6. In his s udy, he eliabili y o he esul s and he
model po abili y a e p ominen . The e o e, he es ing da ase is dominan in compa ison
o he aining o alida ion da ase . Mo eo e , mo e han 2700 iles a e conside ed a
sa is ac o y quan i y o he model aining. Since he a ia ion o he ain, alida ion,
and he es da ase is cohe en , supp ession o ad an age o any o he biomes o he CCA
quin ile du ing he model aining is no expec ed.
BA FO GC SH SI UR WA WE
0
200
400
600
800
1000
1200
1400
1600
CCA Bucke
80–100%
60–80%
40–60%
20–40%
0–20%
Label
cloudy
no cloudy
Biome
Numbe o iles
Figu e 5.
Dis ibu ion o he iles and i s CCA alues pe biome o he ull L8 biome da ase . Tiles
wi h CCA ≥70% a e ca ego ized as cloudy and he es a e ca ego ized as no cloudy iles.
Remo e Sens. 2022,14, 3180 8 o 21
BA FO GC SH SI UR WA WE
0
50
100
150
200
250
300
BA FO GC SH SI UR WA WE
0
20
40
60
80
100
120
140
160
BA FO GC SH SI UR WA WE
0
200
400
600
800
1000
1200
CCA Range 80–100% 60–80% 40–60% 20–40% 0–20%
Biome Biome Biome
Numbe o iles
T ain Valida ion Tes
Figu e 6.
Dis ibu ion o he iles and i s CCA alues pe biome o he aining, alida ion and
es ing da ase .
2.3. Me hodology
The p ocedu e is di ided in o h ee s ages. Fi s , he baseline model o CNN wi h
loa ing poin pa ame e s is ained. Then he weigh s and ac i a ions o he model a e
quan ized and he model is e- ained. The las s ep is he deploymen o he model on
FPGA o achie e high h oughpu and low powe consump ion sui able o on-boa d
da a p ocessing on sa elli e. To be able o deploy a CNN on he edge he e a e many
echniques how o educe he model memo y oo p in such as p uning o quan iza ion. In
his wo k, he ocus o in e es is on quan iza ion which eplaces loa ing poin ope a ions
and weigh enso s wi h lowe bi wid hs which a e especially use ul o FPGA whe e
a bi a y p ecision da a ypes can be implemen ed.
2.3.1. Quan ized CNN
The main idea o his sec ion is o in oduce he quan iza ion o CNN and i s imple-
men a ion o he pu pose o his s udy.
Quan iza ion in neu al ne wo ks is a echnique used o op imiza ion which p o ed
o p oduce g ea success in he ecen yea s [
39
]. I s main ocus is on educing memo y
oo p in and compu a ion ime by eplacing loa ing poin compu e ope a ions and s o ing
enso weigh s wi h lowe bi wid hs. This is especially use ul o esou ce-cons ained
applica ions. The e a e wo ways how o in oduce quan iza ion o a neu al ne wo k. The
i s one is o ain he neu al ne wo k wi h quan ized pa ame e s and he second one is
a quan iza ion o pa ame e s a e he model is ained wi h loa ing poin p ecision. In
he o me case, he p ocess is called quan iza ion-awa e aining (QAT); in he la e , i is
e e ed o as pos - aining quan iza ion (PTQ). PTQ may dis u b he model pa ame e s and
change he poin o which i con e ged du ing he aining wi h loa ing poin p ecision.
Fo his eason, QAT is used o he expe imen s conduc ed in his s udy and aining wi h
quan ized model pa ame e s is pe o med. Fo a mo e comp ehensi e e iew o he cu en
s a e o quan iza ion in neu al ne wo ks e e o he ecen su ey [39].
The ne wo k was implemen ed using he B e i as amewo k. B e i as is a PyTo ch
lib a y used o QAT o neu al ne wo ks [
40
]. A he ime o w i ing he PyTo ch lib a y
suppo s he quan iza ion as well bu allows jus educ ion om 32-bi loa ing poin o
8-bi in ege [
41
]. B e i as in compa ison allows educing he weigh and ac i a ion bi
wid hs o as low as 1-bi which enables he c ea ion o bina y neu al ne wo ks (BNN) [
42
].
Ano he eason why he B e i as lib a y is used is ha a model ained using B e i as
can be expo ed and used by he FINN amewo k o da a low a chi ec u e accele a ion
(DFA) on Xilinx FPGAs [
23
]. The FINN amewo k is a compile o eed o wa d DFA
o deep neu al ne wo ks (DNN) in e ence. When DFA is used, e e y laye o DNN is
mapped o i s own se o dedica ed compu e and memo y esou ces [
43
] which mimics he
opology o DNN. In FINN he pe o mance and esou ce usage can be con olled wi h a
concep called Folding. FINN uses wha is called ma ix- ec o h eshold uni s (MVTU)
Remo e Sens. 2022,14, 3180 9 o 21
o con olu ional and ully connec ed laye s. The e a e h ee pa ame e s ha can be se :
ma ix- ec o ma ix-mul iple ec o (MMV) leng h, p ocessing elemen s (PE), and single
ins uc ion mul iple da a (SIMD) lanes. Using hese pa ame e s, i is possible o con ol he
h oughpu o he ne wo k wi h espec o esou ce u iliza ion o he FPGA.
2.3.2. CloudSa Ne -1 A chi ec u e
In he ollowing pa ag aph, he p oposed CNN a chi ec u e and loss unc ion used
du ing he aining pe iod a e desc ibed.
The p oposed ne wo k a chi ec u e consis s o 10 con olu ional laye s and 2 ully
connec ed laye s, hei speci ic pa ame e s a e isualized in Figu e 7. Each laye excep he
las laye uses he ReLU ac i a ion unc ion and has no bias. The ne wo k s a s wi h an
ini ial con olu ional laye which p ocesses 512
×
512
×
3 uin 8 inpu and con inues wi h
3 sequences o 3 laye s each. The inpu size was chosen o allow di ec compa ison wi h
CloudScou a chi ec u e [
34
]. Each sequence middle laye has a lowe numbe o il e s
o implemen bo leneck o be e gene aliza ion p ope ies. A e each sequence and
ini ial laye , he e is ba ch no maliza ion and max pooling wi h ke nel size o 4, his leads
o he e ec i e educ ion o ea u e dimensions. The las ully connec ed laye ou pu s
unno malized p obabili y o each class whe e he i s class ep esen s cloud p esence
below 70% CCA in he image and he second class signals he p esence o clouds abo e
his h eshold.
Inpu C0
3
512
FC1
1024
FC2
512
C2
32
6
86C3
8
6
86
C1
128
10 6 6
2
Ou pu
Figu e 7. CloudSa Ne -1 a chi ec u e.
The loss unc ion used o aining he model was a modi ied bina y c oss en opy loss
wi h an inc eased penal y o alse posi i es (FP) e o s shown in Equa ion (1). Penal y o
FP e o s is mul iplied by a pa ame e
α
which is inspi ed by he app oach epo ed in [
34
]
whe e he au ho s showed a dec ease in he numbe o FP e o s while keeping accu acy
on accep able alue when pa ame e αwas se o 2.
F(y,ˆ
y) = −1
N
N
∑
i=1
yi·log(ˆ
yi) + α·(1−yi)·log(1−ˆ
yi), (1)
whe e
y
is he g ound- u h label,
ˆ
y
is he p edic ed ou pu o he ne wo k and
α
is a
hype -pa ame e o inc ease penal y o FP e o s.
2.3.3. Quan iza ion P ocess
Fi s , he model wi h loa ing poin p ecision is ained as a baseline. A e su icien
accu acy has been achie ed weigh and ac i a ion bi wid hs a e p og essi ely educed and
he change in accu acy is obse ed. To i he model on FPGA and achie e high h oughpu
wi h accep able accu acy and low powe consump ion, in his pape he ocus o in e es is
bi wid hs o hidden laye s lowe o equal o 4. In all expe imen s, he same bi wid hs a e
used o weigh s and ac i a ions. The i s and las laye o he neu al ne wo k can be mo e
Remo e Sens. 2022,14, 3180 16 o 21
Table 9. Resul s o cloud co e age classi ica ion o bes -pe o med quan ized models on FPGA.
BW ACC (SI) [%] ACC (EXSI) [%] FPS RU [%] APC [W]
2 83.41 92.08 15.462 31.63 2.520
388.24 93.37 15.467 40.20 2.556
4 87.42 94.84 15.468 48.45 2.592
BW—bi wid h; ACC—accu acy; SI—Snow/Ice biome included; EXSI—Snow/Ice biome excluded; FPS— ames
pe second; RU—FPGA esou ce u iliza ion; APC—a e age powe consump ion.
4. Discussion
4.1. Quan ized Model o Cloud De ec ion
Based on he esul s o he bes -pe o ming models epo ed in he uppe pa o
Table 4
, he inc ease o he quan iza ion le el esul ed in sligh o e all pe o mance de e-
io a ion. E en hough, he quan ized models achie ed compa able esul s o he 32-bi
baseline model (excep he 2-bi model). The dec ease o he o e all accu acy o he 4-bi
and 3-bi model is jus a ound 2%, while o he 2-bi model i is mo e han 6%. Howe e ,
a he han he highes o e all accu acy, his s udy emphasizes on he low FPR (i is mo e
con enien o p ocess a edundan image han o disca d he ele an one). The e o e, a
balance be ween he ACC and FPR is in demand. Fo he baseline model and 3-bi model,
he FPR is iden ically equal o 9.93%. In he case o he 4-bi model, almos 3% o FPR
dec ease can be no iced, howe e , he ecall is lowe by 10% in compa ison o he 32-bi
model. The 2-bi model su e s he mos om he quan iza ion e ec esul ing in e y in-
su icien
FPR =
17.59%. Mo e balanced (ACC s. FPR) esul s a e p o ided in he bo om
pa o Table 4, whe e he bes models by FPR om he op 10 models so ed by ACC a e
epo ed. Un o una ely, he pe o mance o he quan ized models keeps almos on he
same le els, ye he baseline model signi ican ly educed i s FPR o 2.25%, while dec easing
i s accu acy by a ound 3%. A mo e eadable compa ison o he model’s pe o mance can
be seen in Figu e 9. A end o he ade-o be ween ACC and FPR ac oss all quan ized
models oge he wi h he baseline is highligh ed by Pa e o on s. I can be obse ed, ha
he baseline model ou pe o ms he quan ized ones, howe e , he e can be ound adequa e
al e na i es o he 32-bi model.
To collec mo e insigh s and o imp o e he o e all pe o mance o he p oposed cloud
de ec ion sys em, each biome o he
L8
biome da ase was in es iga ed sepa a ely. We
hypo hesize ha some biomes p oduce signi ican noise du ing he aining p ocess due o
he alse cloud-like ea u es (snow, ice, o og). The 4-bi models we e selec ed o in es iga e
biomes in quan ized cases, and i s esul s a e epo ed in Table 5. Bes pe o mance
was ob ained by a model ained on G ass/C ops biome wi h
ACC =
95.91% and low
FPR =
0.83%. Ye , he bes
FPR =
0.49% and p ecision o mo e han 99% was achie ed by
Fo es biome. Howe e , his model lags on accu acy due o low
ecall =
68.36%, which
will esul in a high numbe o unde ec ed cloudy images. This may be caused by he cloud
ca ego ies me ge ( hin, hick) o by he og, which is a usual alse cloud-like ea u e in
he Fo es biome [
34
]. Simila ly, he We lands biome (also o en a ec ed by og) esul ed
in low
FPR =
0.94% and high
p ecision =
98.51%, bu wi h low
ecall =
68.33%. The
Sh ubland, U ban, and Wa e biomes achie ed compa able pe o mance wi h ACC om
91.73% o 93.89% and FPR om 1.89% o 3.92%. The Ba en biome ob ained he second
wo se pe o mance in e ms o
FPR =
10.14%. The eason o high FPR may lie in he na u e
o he Ba en biome, which exagge a es he hin clouds ea u es o hick clouds. The wo s
pe o mance epo s he Snow/Ice biome. Low p ecision o 50.47% and high
FPR =
31.11%
make i s decision almos andom. Since only he RGB channels we e conside ed, he eason
o misclassi ica ion is he inabili y o ecognize be ween cloud, ice, and snow. To be able
o classi y he clouds abo e he snow and ice, an addi ional spec um capable o al i ude
esolu ion will be necessa y [6,10,34].
Rega ding he p e iously men ioned esul s, all biomes, o a ce ain deg ee, su e om
he cloud-like ea u es p oblem. An example is gi en in Figu e 10, whe e six misclassi ied
cases a e p esen ed. The i s example o he Snow/Ice biome (A) has
CCA =
0%, ye he

Remo e Sens. 2022,14, 3180 17 o 21
snow in he image was misclassi ied o a cloud. The second example o Snow/Ice biome (B)
wi h
CCA =
42% me ged cloud wi h u bid snow cu en s. Nex , he smoo h hilly e ain
o he Ba en biome (C) s e ches he ea u es o hinly dispe sed clouds. This esul ed in
he alsely posi i e image, howe e , he CCA is 10% in eali y. Simila ly, he Wa e biome
example (D) wi h
CCA =
1% was misclassi ied due o he wa y, se pen ine ea u es o
he shallow wa e . The las wo examples (E, F) in Figu e 10 ep esen he case nea he
h eshold (
CCA =
70%). He eabou s, a small numbe o cloud pixels may lip he CCA
o e he h eshold bounda y. In addi ion, he p ecise alue o he CCA o each ile may be
so ly di e en om he CCA label [35,37].
Following he epo ed esul s, he Snow/Ice biome is no sui able o he cloud
de ec ion using he p oposed me hod. Mo eo e , p oblema ic coexis ence o he snow, ice
and cloud in cloud de ec ion sys ems has been also iden i ied by [
6
,
10
,
34
]. The e o e, we
decided o wi hd aw he Snow/Ice biome om he ain, alida ion and es da ase s, and
o pe o m he expe imen wi hou his noisy da a. In he eal use case, he cloud de ec ion
sys em can omi known a eas pe manen ly co e ed by snow o ice om he analysis. Based
on he esul s epo ed in Table 6, assumed imp o emen s o all me ics can be obse ed.
The bes pe o ming baseline model achie ed
ACC =
94.92% wi h
FPR =
2.81%. Top
quan ized models ob ained compa able accu acy om 94.84% o 92.02%, and FPR om
2.23% o 5.72%. We would like o s ess ou , ha 4-bi quan ized model pe o med sligh ly
be e in e ms o p ecision (96.82%) and FPR (2.23%) in compa ison o he 32-bi model.
This makes i a p ope quan ized subs i u ion o deploymen on FPGA. Resul s o his
analysis con i m ou hypo hesis ha Snow/Ice biome is na u ally p one o being alse
posi i e when using RGB channels only.
In Figu e 11, he accu acy s. FPR is isualized o models ained wi h excluded
Snow/Ice biome. F om ele a ed posi ion o all models wi hin his Figu e i is e iden
ha accu acy inc eased all-a ound in compa ison o Figu e 9. Cu es o Pa e o on s lie
close oge he and o he baseline on , as he quan iza ion akes a lowe oll on models
pe o mance when wi hou isually ambiguous da a.
Based on hese esul s, ollowing obse a ions will be emphasised o make a deduc ion.
Inc eased quan iza ion did no cause subs an ial d op in alues o e alua ion me ics sco es
o esul s wi h excluded Snow/ice biome. The 4-bi model ma ched o o e ook baseline’s
me ic sco es in accu acy and FPR. This implies equali y be ween ep esen a ional capaci y
o 32-bi baseline and quan ized models in classi ie p oblems ha do no equi e high
esolu ion o disce ning disc imina i e ea u es. This s a emen is in line wi h esul s
achie ed in o he wo ks [46,55,56] dealing wi h he quan iza ion.
The mos ele an s udy, CloudScou [
20
,
34
], used hype spec al bands o model
aining, esul ing o 16-bi model wi h
ACC =
92% and
FPR =
1%. Ou p oposed me hod
ou pe o med his esul by a 4-bi model wi h highe accu acy up o 3%, howe e wi h
lowe FPR by 1.23%. Conside ing, ha ou model used RGB bands only (wi hou Snow/Ice
biome), he p esen ed CloudSa Ne -1 me hod b ings p omising imp o emen s in he on-
boa d cloud de ec ion sys ems. Ano he ele an s udy [
21
] used a la ge aining da ase
and achie ed ACC o 91%. Ne e heless, when au ho s deployed he model on FPGA, a
signi ican d op o accu acy o 64% occu ed. The me hod in oduced in his pape does
no encoun e a simila issue. The compa ison o hese me hods is summa ized in Table 10.
Table 10. Compa ison o he bes -pe o med models wi h di e en s udies.
Me hod BW ACC [%] FPR [%] APC [W] Da a
CloudSa Ne -1 * 4 94.84 2.23 2.5 RGB
CloudScou [20] 16 92 11.8
Hype spec al
CNV-W1A1 [21] 1 64 - 2.4 RGB
*—p oposed me hod; BW—bi wid h; ACC—accu acy; FPR— alse posi i e a e; APC—a e age powe consump ion.
Remo e Sens. 2022,14, 3180 18 o 21
4.2. FPGA-Based Ha dwa e Accele a o
The quan ized models we e deployed in h ee olding con igu a ions o each bi
wid h se ing. Th oughpu , powe consump ion, and FPGA esou ce u iliza ion we e
measu ed. Models wi h maximum olding achie ed 15 FPS wi h inpu ba ch size o 1 and
almos 20 FPS wi h ba ch size 120 which is he maximal ba ch size allowed o be loaded
in o RAM. Inc ease o he FPS wi h highe ba ch size was expec ed, and also con i med
by [
3
]. Powe consump ion measu ed wi h a USB powe me e epo ed an inc ease o
jus o e
≈
0.2
W
du ing he in e ence, compa ed o he es ing s a e. In compa ison wi h
ela ed s udies, he au ho s o CloudScou [
20
,
34
] epo ed a h oughpu o 2.89 FPS and
1.8 W o powe consump ion using My iad VPU wi h 512
×
512
×
3 inpu size, 7 FPS
and 3.4 W o powe consump ion using Zynq Ul ascale+ ZCU106, and 3.77 FPS using
XQRKU060 solu ion (es ima ion only). Nex , in he s udy by Rei e e al. [
21
], he au ho s
epo ed 358.1 FPS wi h a much smalle inpu size 32
×
32
×
3, and maximum powe
consump ion o 2.4 W. Rega ding hese esul s, he h oughpu and powe consump ion o
he ha dwa e accele a o achie ed in his s udy is compa able wi h he cu en s a e-o - he-
a solu ions.
Based on he es ima ed numbe o cycles pe laye epo ed in Table 3, i is isible ha
a bo le-neck in he i s laye limi ed he op imal h oughpu , and i would equi e a change
in he ne wo k a chi ec u e o allow a highe h oughpu a ge . I was demons a ed ha
he ne wo k h oughpu can be con olled o a ge a speci ic FPS desi ed by he needs o
he mission. A se o expe imen s we e conduc ed o a ge speci ic h oughpu o 10 FPS.
Used pa alleliza ion se ings a e epo ed in Table 8. This app oach may be use ul when he
ins umen on he CubeSa does no ha e a high h oughpu , e.g., he came a is gene a ing
da a a lowe FPS. I showed lexibili y in h oughpu con ol o he FPGA-based ha dwa e
accele a o c ea ed by he FINN amewo k. The di e ences o each bi wid h a e in FPGA
esou ce u iliza ion, whe e he 2-bi model in base olding con igu a ion u ilized he lowes
numbe o he esou ces (
LUT = 46.27%
,
FF = 31.41%
,
BRAM = 29.29%
,
DSP = 0.45%
). This
is achie ed due o no pa alleliza ion and a low memo y oo p in o 2-bi weigh s and
ac i a ions. I shows he po en ial o educe bi wid h o weigh s and ac i a ions e en
u he o 1-bi and expe imen wi h BNN in he u u e o enable highe h oughpu and
deepe ne wo k on he same FPGA. As p esen ed in Table 7, DSP slices o he i s laye
we e selec ed o be u ilized by Vi ado jus o SPEC and max olding in all bi wid h
con igu a ions. Memo y oo p in (BRAM u iliza ion) a ies om 1.43 Mb o 3.06 Mb in
he ascending o de ela i e o bi wid h.
5. Conclusions
Mos o he RSI is con amina ed by he clouds, hence he quick and accu a e me hod
o cloud emo al unning on-boa d o he sa elli e has po en ial o signi ican ly sa e
he downlink. In his s udy, we in oduced CloudSa Ne -1, an FPGA-based ha dwa e-
accele a ed quan ized CNN o sa elli e on-boa d cloud co e age classi ica ion. We can
conclude ha he weigh s and ac i a ions quan iza ion has a minimal o no e ec on he
model accu acy. Howe e , he memo y oo p in educ ion allows he model deploymen
and es ing on low-cos FPGA Xilinx Zynq-7020. Using he
L8
biome da ase and i s
RGB bands only, up o 90% o accu acy was achie ed. Nex , we omi ed he Snow/Ice
biome iles om he da ase due o high noise p oduc ion. The accu acy inc eased up o
94.4% o accu acy wi h low
FPR =
2.23% o he 4-bi wid h model. Wi h he maximum
pa alleliza ion se ings, he ha dwa e accele a o achie ed 15 FPS wi h 2.5 W o a e age
powe consump ion (0.2 W inc ease o e he idle s a e). Addi ionally, we p o ed ha we
can con ol h oughpu o a ge speci ic FPS o he p oposed classi ie . Conside ing he
epo ed esul s, he p esen ed no el app oach achie ed ou come compa able wi h he
s a e o he a .
The p esen ed solu ion has se e al limi a ions ha we would like o s ess ou . Fi s ly,
he high numbe o alse posi i e iles wi h a e ain con aining cloud-like ea u es may be
in he u u e compensa ed wi h he analysis in ol ing he mul i-spec al bands. Nex , he
Remo e Sens. 2022,14, 3180 19 o 21
cloud ca ego ies om he o iginal
L8
biome da ase we e me ged o o m a bina y p oblem.
The e o e, his s udy did no e alua e he esul on he o iginal cloud ca ego ies o he
L8
biome da ase , which migh p o ide mo e insigh s on miss-classi ica ions. Fu he mo e, we
did no co e he e ec s o he adia ion on he cloud de ec ion sys em and he edundancy
will be subjec o he u u e wo ks. This wo k is he beginning o he g ea e e o o
p o ide solu ions based on AI o he space missions ha can bene i om i , hus his wo k
is a pilo one in na u e. In he u u e, we aim o imp o e his solu ion o p o ide seman ic
segmen a ion o clouds wi h clouds ca ego iza ion o espec i e classes compensa ing o
he bina y decision p o ided in his s udy.
Au ho Con ibu ions: Concep ualiza ion
, J.M. and R.P.; me hodology, R.P., J.M. and L.D.; alida ion,
L.D., R.P. and M.M.; o mal analysis, J.M., M.M.; in es iga ion, R.P., L.D. and J.M.; esou ces, M.J.;
da a cu a ion, M.J. and R.P.; w i ing—o iginal d a p epa a ion, J.M., R.P., L.D., M.J.; w i ing— e iew
and edi ing, J.M., R.P., L.D., M.J. and M.M.; isualiza ion, M.J. and L.D.; supe ision, J.M.; p ojec
adminis a ion, M.M. All au ho s ha e ead and ag eed o he published e sion o he manusc ip .
Funding: This esea ch ecei ed no ex e nal unding.
Da a A ailabili y S a emen : No applicable
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Lo q is , M.; Cano, J. Op imizing Da a P ocessing in Space o Objec De ec ion in Sa elli e Image y. a Xi
2021
, a Xi :2107.03774.
2. Caldwell, S. S a e-o - he-A o Small Spacec a Technology; NASA: Washing on, DC, USA, 2021.
3.
Maskey, A.; Cho, M. CubeSa Ne : Ul aligh Con olu ional Neu al Ne wo k designed o on-o bi bina y image classi ica ion on
a 1U CubeSa . Eng. Appl. A i . In ell. 2020,96, 103952. [C ossRe ]
4. Kulu, E. Nanosa s Da abase. 2022. A ailable online: h ps://www.nanosa s.eu/ (accessed on 15 Sep embe 2021).
5. EUSPA. EUSPA EO and GNSS Ma ke Repo ; EUSPA: P ague, Czech Republic, 2022.
6.
Yang, J.; Guo, J.; Yue, H.; Liu, Z.; Hu, H.; Li, K. CDne : CNN-based cloud de ec ion o emo e sensing image y. IEEE T ans.
Geosci. Remo e Sens. 2019,57, 6195–6211. [C ossRe ]
7.
Fen a, A.A.; Yasuda, H.; Ha egeweyn, N.; Belay, A.S.; Hadush, Z.; Geb emedhin, M.A.; Mekonnen, G. The dynamics o u ban
expansion and land use/land co e changes using emo e sensing and spa ial me ics: The case o Mekelle Ci y o no he n
E hiopia. In . J. Remo e Sens. 2017,38, 4107–4129. [C ossRe ]
8.
King, M.D.; Pla nick, S.; Menzel, W.P.; Acke man, S.A.; Hubanks, P.A. Spa ial and empo al dis ibu ion o clouds obse ed by
MODIS onboa d he Te a and Aqua sa elli es. IEEE T ans. Geosci. Remo e Sens. 2013,51, 3826–3852. [C ossRe ]
9.
Zhang, Y.; Rossow, W.B.; Lacis, A.A.; Oinas, V.; Mishchenko, M.I. Calcula ion o adia i e luxes om he su ace o op o
a mosphe e based on ISCCP and o he global da a se s: Re inemen s o he adia i e ans e model and he inpu da a. J. Geophys.
Res. A mos. 2004,109, D19105. [C ossRe ]
10.
Guo, J.; Yang, J.; Yue, H.; Tan, H.; Hou, C.; Li, K. CDne V2: CNN-based cloud de ec ion o emo e sensing image y wi h
cloud-snow coexis ence. IEEE T ans. Geosci. Remo e Sens. 2020,59, 700–713. [C ossRe ]
11.
Wu, Z.; Li, J.; Wang, Y.; Hu, Z.; Molinie , M. Sel -a en i e gene a i e ad e sa ial ne wo k o cloud de ec ion in high esolu ion
emo e sensing images. IEEE Geosci. Remo e Sens. Le . 2019,17, 1792–1796. [C ossRe ]
12.
Li, L.; Li, X.; Jiang, L.; Su, X.; Chen, F. A e iew on deep lea ning echniques o cloud de ec ion me hodologies and challenges.
Signal Image Video P ocess. 2021,15, 1527–1535. [C ossRe ]
13.
K eu e , A.; Zange l, M.; Schwa zmann, M.; Blum hale , M. All-sky imaging: A simple, e sa ile sys em o a mosphe ic esea ch.
Appl. Op . 2009,48, 1091–1097. [C ossRe ]
14.
Long, C.N.; Sabbu g, J.M.; Calbó, J.; Pagès, D. Re ie ing cloud cha ac e is ics om g ound-based day ime colo all-sky images. J.
A mos. Ocean. Technol. 2006,23, 633–652. [C ossRe ]
15.
F an z, D.; Haß, E.; Uhl, A.; S o els, J.; Hill, J. Imp o emen o he Fmask algo i hm o Sen inel-2 images: Sepa a ing clouds
om b igh su aces based on pa allax e ec s. Remo e Sens. En i on. 2018,215, 471–481. [C ossRe ]
16.
Zhu, Z.; Woodcock, C.E. Au oma ed cloud, cloud shadow, and snow de ec ion in mul i empo al Landsa da a: An algo i hm
designed speci ically o moni o ing land co e change. Remo e Sens. En i on. 2014,152, 217–234. [C ossRe ]
17.
Sui, Y.; He, B.; Fu, T. Ene gy-based cloud de ec ion in mul ispec al images based on he SVM echnique. In . J. Remo e Sens.
2019
,
40, 5530–5543. [C ossRe ]
18.
Zi, Y.; Xie, F.; Jiang, Z. A cloud de ec ion me hod o Landsa 8 images based on PCANe . Remo e Sens.
2018
,10, 877. [C ossRe ]
19.
Mi alles, P.; Scannapieco, A.F.; Jagadam, N.; Ba anwal, P.; Faldu, B.; Abhang, R.; Bha ia, S.; Bonna , S.; Bha naga , I.; Ba ul, B.;
e al. Machine Lea ning in Ea h Obse a ion Ope a ions: A e iew. In P oceedings o he 72nd In e na ional As onau ical
Cong ess (IAC), Dubai, Uni ed A ab Emi a es, 25–29 Oc obe 2021; In e na ional As onau ical Fede a ion: Pa is, F ance, 2021.
Remo e Sens. 2022,14, 3180 20 o 21
20.
Rapuano, E.; Meoni, G.; Pacini, T.; Dinelli, G.; Fu ano, G.; Giu ida, G.; Fanucci, L. An FPGA-Based Ha dwa e Accele a o o
CNNs In e ence on Boa d Sa elli es: Benchma king wi h My iad 2-Based Solu ion o he CloudScou Case S udy. Remo e Sens.
2021,13, 1518. [C ossRe ]
21.
Rei e , P.; Ka agiannakis, P.; I eland, M.; G eenland, S.; C ocke , L. FPGA accele a ion o a quan ized neu al ne wo k o
emo e-sensed cloud de ec ion. In P oceedings o he 7 h In e na ional Wo kshop on On-Boa d Payload Da a Comp ession,
Online, 21–23 Sep embe 2020.
22. Vlonca . as machinelea ning/hls4ml: Co is; Zenodo: Gene a, Swi ze land, 2021. [C ossRe ]
23.
Blo , M.; P euße , T.B.; F ase , N.J.; Gamba della, G.; O’b ien, K.; Umu oglu, Y.; Leese , M.; Visse s, K. FINN-R: An end- o-end
deep-lea ning amewo k o as explo a ion o quan ized neu al ne wo ks. ACM T ans. Recon igu able Technol. Sys . TRETS
2018
,
11, 1–23. [C ossRe ]
24.
Yao, Y.; Jiang, Z.; Zhang, H.; Zhou, Y. On-boa d ship de ec ion in mic o-nano sa elli e based on deep lea ning and COTS
componen . Remo e Sens. 2019,11, 762. [C ossRe ]
25.
Schwa z, C.; Sande , I.; Jo dão, R.; B uhn, F.; Pe sson, M.; Ekblad, J.; Fuglesang, C. On-boa d sa elli e da a p ocessing o achie e
sma in o ma ion collec ion. In Op ics, Pho onics and Digi al Technologies o Imaging Applica ions VII; Schelkens, P., Kozacki, T.,
Eds.; In e na ional Socie y o Op ics and Pho onics, SPIE: Bellingham, WA, USA, 2022; Volume 12138, pp. 121–131. [C ossRe ]
26.
ElMas y, G.; Sun, D.W. P inciples o Hype spec al Imaging Technology. In Hype spec al Imaging o Food Quali y Analysis and
Con ol; Else ie : Ams e dam, The Ne he lands, 2010; pp. 3–43.
27.
Zhang, Z.; Iwasaki, A.; Xu, G.; Song, J. Cloud de ec ion on small sa elli es based on ligh weigh U-ne and image comp ession. J.
Appl. Remo e Sens. 2019,13, 026502. [C ossRe ]
28.
Zhang, J.; Li, X.; Li, L.; Sun, P.; Su, X.; Hu, T.; Chen, F. Ligh weigh U-Ne o cloud de ec ion o isible and he mal in a ed
emo e sensing images. Op . Quan um Elec on. 2020,52, 1–14. [C ossRe ]
29.
Zhang, Z.; Xu, G.; Song, J. CubeSa cloud de ec ion based on JPEG2000 comp ession and deep lea ning. Ad . Mech. Eng.
2018
,10,
1687814018808178. [C ossRe ]
30.
Pa k, J.H.; Inamo i, T.; Hamaguchi, R.; O suki, K.; Kim, J.E.; Yamaoka, K. RGB Image P io i iza ion Using Con olu ional Neu al
Ne wo k on a Mic op ocesso o Nanosa elli es. Remo e Sens. 2020,12, 3941. [C ossRe ]
31.
G eenland, S.; I eland, M.; Kobayashi, C.; Mendham, P.; Pos , M.; Whi e, D. De elopmen o a mina u ised o wa ds looking
image using deep lea ning o esponsi e ope a ions. In P oceedings o he 4S Symposium, So en o, I aly, 28 May–1 June 2018;
ESA: Pa is, F ance, 2018.
32.
de Vieille ille, F.; Lag ange, A.; Ruiloba, R.; May, S. Towa ds Dis illa ion o Deep Neu al Ne wo ks o Sa elli e On-Boa d Image
Segmen a ion. In . A ch. Pho og amm. Remo e Sens. Spa . In . Sci. 2020,43, 1553–1559. [C ossRe ]
33.
Li, J.; Wu, Z.; Hu, Z.; Jian, C.; Luo, S.; Mou, L.; Zhu, X.X.; Molinie , M. A Ligh weigh Deep Lea ning-Based Cloud De ec ion
Me hod o Sen inel-2A Image y Fusing Mul iscale Spec al and Spa ial Fea u es. IEEE T ans. Geosci. Remo e Sens.
2021
,60, 1–19.
[C ossRe ]
34.
Giu ida, G.; Diana, L.; de Gioia, F.; Benelli, G.; Meoni, G.; Dona i, M.; Fanucci, L. Cloudscou : A deep neu al ne wo k o
on-boa d cloud de ec ion on hype spec al images. Remo e Sens. 2020,12, 2205. [C ossRe ]
35.
Foga, S.; Sca amuzza, P.L.; Guo, S.; Zhu, Z.; Dilley J , R.D.; Beckmann, T.; Schmid , G.L.; Dwye , J.L.; Hughes, M.J.; Laue, B. Cloud
de ec ion algo i hm compa ison and alida ion o ope a ional Landsa da a p oduc s. Remo e Sens. En i on.
2017
,194, 379–390.
[C ossRe ]
36.
Ma kham, B.; S o ey, J.; Mo i , R. Landsa -8 senso cha ac e iza ion and calib a ion. Remo e Sens.
2015
,7, 2279–2282. [C ossRe ]
37.
USGS. Landsa Da a Con inui y Mission Le el 1 (L1) Da a Fo ma Con ol Book (DFCB), LDCM-DFCB-004, Ve sion 6.0; USGS: Res on,
VA, USA, 2012.
38.
Lo eland, T.R.; Reed, B.C.; B own, J.F.; Ohlen, D.O.; Zhu, Z.; Yang, L.; Me chan , J.W. De elopmen o a global land co e
cha ac e is ics da abase and IGBP DISCo e om 1 km AVHRR da a. In . J. Remo e Sens. 2000,21, 1303–1330. [C ossRe ]
39.
Gholami, A.; Kim, S.; Dong, Z.; Yao, Z.; Mahoney, M.W.; Keu ze , K. A Su ey o Quan iza ion Me hods o E icien Neu al
Ne wo k In e ence. a Xi 2021, a Xi :2103.13630.
40. Pappala do, A. Xilinx/B e i as; Zenodo: Gene a, Swi ze land, 2021. [C ossRe ]
41. Quan iza ion. A ailable online: h ps://py o ch.o g/docs/s able/quan iza ion.h ml (accessed on 28 No embe 2021).
42.
Cou ba iaux, M.; Huba a, I.; Soud y, D.; El-Yani , R.; Bengio, Y. Bina ized neu al ne wo ks: T aining deep neu al ne wo ks wi h
weigh s and ac i a ions cons ained o+ 1 o -1. a Xi 2016, a Xi :1602.02830.
43.
Alonso, T.; Pe ica, L.; Ruiz, M.; Pe i-Koenig, J.; Umu oglu, Y.; S amelos, I.; Ko omilas, E.; Blo , M.; Visse s, K. Elas ic-DF: Scaling
Pe o mance o DNN In e ence in FPGA Clouds h ough Au oma ic Pa i ioning. ACM T ans. Recon igu able Technol. Sys .
2021
,
15, 1–34. [C ossRe ]
44.
P zewlocka, D.; Wasala, M.; Szolc, H.; Blachu , K.; K yjak, T. Op imisa ion o a Siamese Neu al Ne wo k o Real-Time Ene gy
E icien Objec T acking. a Xi 2020, a Xi :2007.00491
45.
Mellempudi, N.; Kundu, A.; Mudige e, D.; Das, D.; Kaul, B.; Dubey, P. Te na y Neu al Ne wo ks wi h Fine-G ained Quan iza ion.
a Xi 2017, a Xi :1705.01462.
46.
Zhou, S.; Wu, Y.; Ni, Z.; Zhou, X.; Wen, H.; Zou, Y. DoReFa-Ne : T aining Low Bi wid h Con olu ional Neu al Ne wo ks wi h
Low Bi wid h G adien s. a Xi 2016, a Xi :1606.06160.
Remo e Sens. 2022,14, 3180 21 o 21
47.
Io e, S.; Szegedy, C. Ba ch No maliza ion: Accele a ing Deep Ne wo k T aining by Reducing In e nal Co a ia e Shi . P oc. Mach.
Lea n. Res. PMLR 2015,37, 448–456.
48. Umu oglu, Y.; Jah e, M. S eamlined Deploymen o Quan ized Neu al Ne wo ks. a Xi 2017, a Xi :1709.04060.
49.
C ocke , L.; No hco e, D.; Ramsay, C. Explo ing Zynq MPSoC: Wi h PYNQ and Machine Lea ning Applica ions; S a hclyde
Academic Media: Glasgow, Sco land, 2019.
50.
Xilinx. Zynq-7000 SoC Da a Shee : O e iew. 2018. A ailable online: h ps://www.xilinx.com/con en /dam/xilinx/suppo /
documen s/da a_shee s/ds190-Zynq-7000-O e iew.pd (accessed on 8 Sep embe 2021).
51. PYNQ: PYTHON PRODUCTIVITY. A ailable online: h p://www.pynq.io/ (accessed on 8 Feb ua y 2022).
52.
De elope s, O.R. ONNX Run ime. Ve sion: X.y.z. 2021. A ailable online: h ps://onnx un ime.ai/ (accessed on 1 Sep embe
2021).
53.
Shah ia i, B.; Swe sky, K.; Wang, Z.; Adams, R.P.; de F ei as, N. Taking he Human Ou o he Loop: A Re iew o Bayesian
Op imiza ion. P oc. IEEE 2016,104, 148–175. [C ossRe ]
54.
Fa aone, J.; Gamba della, G.; Boland, D.; F ase , N.; Blo , M.; Leong, P.H. Cus omizing Low-P ecision Deep Neu al Ne wo ks o
FPGAs. In P oceedings o he 2018 28 h In e na ional Con e ence on Field P og ammable Logic and Applica ions (FPL), Dublin,
I eland, 26–30 Augus 2018; pp. 97–973. [C ossRe ]
55.
Han, S.; Mao, H.; Dally, W.J. Deep comp ession: Comp essing deep neu al ne wo ks wi h p uning, ained quan iza ion and
hu man coding. a Xi 2015, a Xi :1510.00149.
56. Zhu, C.; Han, S.; Mao, H.; Dally, W.J. T ained e na y quan iza ion. a Xi 2016, a Xi :1612.01064.