CloudSatNet-1: FPGA-Based Hardware-Accelerated Quantized CNN for Satellite On-Board Cloud Coverage Classification

Ci a ion: Pi onak, R.; Mucha, J.; Dobis, L.; Ja o ka, M.; Ma usin, M. CloudSa Ne -1: FPGA-Based Ha dwa e-Accele a ed Quan ized CNN o Sa elli e On-Boa d Cloud Co e age Classi ica ion. Remo e Sens. 2022,14, 3180. h ps://doi.o g/ 10.3390/ s14133180 Academic Edi o s: Massimiliano Pas ena and Luca Fanucci Recei ed: 3 June 2022 Accep ed: 27 June 2022 Published: 2 July 2022 Publishe ’s No e: MDPI s ays neu al wi h ega d o ju isdic ional claims in published maps and ins i u ional a il- ia ions. Copy igh : © 2022 by he au ho s. Licensee MDPI, Basel, Swi ze land. This a icle is an open access a icle dis ibu ed unde he e ms and condi ions o he C ea i e Commons A ibu ion (CC BY) license (h ps:// c ea i ecommons.o g/licenses/by/ 4.0/). emo e sensing A icle CloudSa Ne -1: FPGA-Based Ha dwa e-Accele a ed Quan ized CNN o Sa elli e On-Boa d Cloud Co e age Classi ica ion Radosla Pi onak 1, Jan Mucha 2,* , Lukas Dobis 1, Ma in Ja o ka 1and Ma ek Ma usin 1 1Zai a s. .o., Plyna enska 499/1, 60200 B no, Czech Republic; [email p o ec ed] (R.P.); [email p o ec ed] (L.D.); [email p o ec ed] (M.J.); [email p o ec ed] (M.M.) 2Depa men o Telecommunica ions, B no Uni e si y o Technology, Technicka 12, 61600 B no, Czech Republic *Co espondence: [email p o ec ed] Abs ac : CubeSa s, he nanosa elli es and mic osa elli es wi h a we mass up o 60 kg, accompanied by he cos dec ease o accessing he space, ampli ied he apid de elopmen o he Ea h Obse a ion indus y. Acqui ed image da a se e as an essen ial sou ce o in o ma ion in a ious disciplines like en i onmen al p o ec ion, geosciences, o he mili a y. As he quan i y o emo e sensing da a g ows, he bandwid h esou ces o he da a ansmission (downlink) a e exhaus ed. The e o e, new echniques ha educe he downlink u iliza ion o he sa elli es mus be in es iga ed and de eloped. Fo ha eason, we a e p esen ing CloudSa Ne -1: an FPGA-based ha dwa e-accele a ed quan ized con olu ional neu al ne wo k (CNN) o sa elli e on-boa d cloud co e age classi ica ion. We aim o explo e he e ec s o he quan iza ion p ocess on he p oposed CNN a chi ec u e. Addi ionally, he pe o mance o cloud co e age classi ica ion by biomes di e si y is in es iga ed, and he ha dwa e a chi ec u e design space is explo ed o iden i y he op imal FPGA esou ce u iliza ion. Resul s o his s udy showed ha he weigh s and ac i a ions quan iza ion adds a mino e ec on he model pe o mance. Ne e heless, he memo y oo p in educ ion allows he model deploymen on low- cos FPGA Xilinx Zynq-7020. Using he RGB bands only, up o 90% o accu acy was achie ed, and when omi ing he iles wi h snow and ice, he pe o mance inc eased up o 94.4% o accu acy wi h a low alse-posi i e a e o 2.23% o he 4-bi wid h model. Wi h he maximum pa alleliza ion se ings, he ha dwa e accele a o achie ed 15 FPS wi h 2.5 W o a e age powe consump ion (0.2 W inc ease o e he idle s a e). Keywo ds: CNN; FPGA; ha dwa e accele a o s; image p ocessing; on-boa d p ocessing; quan iza ion 1. In oduc ion O e he las decade, he Ea h Obse a ion (EO) indus y has expe ienced a d ama ic dec ease in he cos o accessing space [ 1 ]. Wi h he in oduc ion o CubeSa s, nanosa el- li es and mic osa elli es wi h we mass up o 60 kg [ 2 ], he apid de elopmen o emo e sensing echnologies was ampli ied [ 3 ]. As o 2021, mo e han 1500 CubeSa s ha e been launched [ 4 ], and acco ding o [ 5 ], i will inc ease up o a housand sa elli es pe yea ill 2028. Na u ally, as he numbe o sa elli es g ows, sa elli e image y becomes eadily a ailable. Ha es ed da a plays a signi ican ole in a ious disciplines like en i onmen al p o ec ion, ag icul u e enginee ing, land o mine al esou ce explo a ion, geosciences, o mili a y econnaissance [ 6 , 7 ]. In line wi h he amoun o emo e sensing da a acqui ed, he bandwid h esou ces o he da a ansmission inclines o be o e loaded. The e- o e, new echniques o e icien bandwid h esou ces managemen mus be in es iga ed and de eloped. Se e al s udies es ima e ha app oxima ely 67% o he Ea h’s su ace is co e ed wi h clouds [ 6 , 8 , 9 ]. Consequen ly, mos o he emo e sensing image ies (RSI) will be con amina ed by hem, which de alues he quali y o RSI and nega i ely a ec s he pos - p ocessing [ 6 ]. Cloudy condi ions impai sa elli e senso capabili ies o ob ain clea iews Remo e Sens. 2022,14, 3180. h ps://doi.o g/10.3390/ s14133180 h ps://www.mdpi.com/jou nal/ emo esensing Remo e Sens. 2022,14, 3180 2 o 21 o he Ea h’s su ace, and hence he quick and accu a e de ec ion o he cloudy images is necessa y [ 6 , 10 , 11 ]. In gene al, he cu en me hods o cloud co e age es ima ion o classi i- ca ion a e mainly ca ego ized in o adi ional and machine-lea ning-based app oaches [ 12 ]. T adi ional ones consis o h eshold-based ( ixed o adap i e), ime di e en ia ion, and s a is ical me hods. The h eshold-based app oaches ely on a isible e lec ion and in- a ed empe a u e o he clouds, he e o e i s pe o mance weakens on low-con as ed (cloud s. su ace) images [ 13 – 15 ]. Time di e en ia ion me hods e ec i ely iden i y he changing pixel alues as clouds in mul i- empo al images, howe e , hey do no con- side changes in he op o a mosphe e e lec ance a ec ed by loods [ 12 , 16 ]. S a is ical me hods combine spec al and spa ial ea u es ex ac ed om RSIs wi h classical ma- chine lea ning algo i hms (suppo ec o machine, decision ee), bu hey lack o ob ain he desi ed esul s [ 17 , 18 ]. To sum up, adi ional me hods p o ide some capabili ies o cloud de ec ion, hough, hey a e suscep ible o he backg ounds, a e non-uni e sal and subjec i e [12]. A mo e e icien app oach o cloudy image de ec ion comp ises con olu ional neu- al ne wo ks (CNNs), simple linea i e a i e clus e ing, o seman ic segmen a ion algo- i hms [ 12 ]. Especially a ac i e a e CNNs, which p o ide s a e-o - he-a esul s o many di e en asks, including image classi ica ion, segmen a ion, and objec de ec ion. This success is o en achie ed hanks o models wi h a huge numbe o pa ame e s which means he la ge size and limi ed abili y o he deploymen on esou ce-cons ained ha dwa e. In ecen yea s, he e has been a endency o deploy hese models in line wi h he edge compu ing pa adigm on esou ce-cons ained ha dwa e [ 12 , 19 – 21 ]. Va ious ha dwa e accele a o s a e a ailable on he ma ke anging om mic ocon olle s o smalle models o boa ds equipped wi h GPU, isual p ocessing uni (VPU), o ield-p og ammable ga e a - ay (FPGA). FPGA in pa icula p o ides in e es ing capabili ies in e ms o cos , lexibili y, pe o mance, and powe consump ion. A possible disad an age is he long ime o ma ke in compa ison o GPU o VPU solu ions. Ne e heless, his gap is being closed by ecen ad ancemen s in he ha dwa e deploymen o machine lea ning models [ 22 , 23 ] c ea ed in well-known machine lea ning amewo ks like Py o ch o Tenso low. Conside ing he payload limi a ions o he CubeSa s, he op imal solu ion o he CubeSa ’s cloud de ec ion sys em is a sys em es ima ing RSI cloud co e age unning di ec ly on boa d. To educe he cos s and de elopmen ime o such eal- ime de ec ion sys ems, Comme cial-O -The-Shel (COTS) componen s p o ide a a o able deploymen op ion [ 24 ]. The c ucial c i e ion o an onboa d de ec ion sys em is i s powe consump ion, whe eas he usual limi is below 5 W and he a io o he alsely disca ded images below 2% [ 12 , 19 , 20 ]. Gene ally, emo e-sensing sa elli es can be equipped wi h a pale e o senso s p o iding in o ma ion in a ious bands. F om he simples one (RGB image ies) ollowed by mul ispec al im- age ies (usually a combina ion o RGB and nea -in a ed band (NIR)) o he hype spec al image ies p o iding a complex spec um o he sensed a ea [25,26]. In line wi h he abo e men ioned, Zhang e al. [ 27 ] in oduced a ligh weigh CNN o cloud de ec ion based on U-Ne using ed, g een, blue, and in a ed wa eband images om he Landsa -8 da ase . Applying he LeGall-5/3 wa ele ans o m (4 le els) o da ase comp ession and p ocessing ime accele a ion, he au ho s epo ed 94.3% o o e all accu acy unning on an ARM-based pla o m. Simila ly, in [ 28 ], he au ho s applied dep hwise sepa able con olu ions o comp ess he model o U-Ne and accele a e he in e ence speed. The S udy epo ed he bes accu acy o 90.54% e i ied on Landsa 8 emo e sensing images. Ano he u iliza ion o a ligh weigh MobU-Ne ained on Landsa 8 da ase and using JPEG comp ession s a egy was pe o med by [ 29 ]. The achie ed o e all accu acy was a ound 93.1% o a model deployed on ARM9 p ocesso on Zynq-7020 boa d. Maskey e al. [3] p oposed an ul aligh CNN designed o on-o bi bina y image classi ica ion called CubeSa Ne . The model was ained on BIRDS3 sa elli e images and deployed on ARM Co ex M7 MCU. An accu acy o 90% was achie ed when classi ying images as “bad” o cloudy, sunbu n , acing space, o sa u a ed images and “good” in all o he cases. A p omising me hod o cloud de ec ion using RS-Ne and RGB Remo e Sens. 2022,14, 3180 3 o 21 bands exclusi ely was published by [ 30 ]. Fo model aining, he Sen inel-2 da ase was used, and 76% o accu acy was epo ed by he model deployed on an ARM-based pla o m. Ano he possibili y is o use he Fo wa ds Looking Image ins umen , which p o ides analysis o he upcoming en i onmen o he sa elli e. This app oach was examined in [ 31 ], es ing a ious ligh weigh CNNs deployed on he Zynq-7020 boa d using FPGA. The au ho s epo ed high accu acy o 98% , howe e , 100 images only we e used o es ing. Vieille ille e al. [32] in es iga ed he possibili ies o he deep neu al ne wo k (DNN) dis illa ion p ocess in o de o educe he size o DNN while accommoda ing e iciency in e ms o bo h accu acy and in e ence cos . The au ho s we e able o educe he numbe o DNN pa ame e s om se e al million o less han one wi h a minimal d op in pe o mance in he image segmen a ion p ocess. To sum up, ligh weigh CNNs p o ide a compe i i e on-boa d cloud de ec ion pe - o mance in compa ison o he s a e-o - he-a deep con olu ional neu al ne wo ks, like CDNe V1 [ 6 ], CDNe V2 [ 10 ] o CD-FM3SFs [ 33 ]. CDNe V1 is a neu al ne wo k o cloud mask ex ac ion om ZY-3 sa elli e humbnails wi h he accu acy o 96.47% [ 6 ]. I s ex- ended e sion, CDNe V2, ocuses on adap i ely using mul i-scale ea u e maps and emedying high-le el seman ic in o ma ion dilu ed a decode laye s o imp o e cloud de ec ion accu acy wi h cloud-snow coexis ence. The au ho s con i med he obus ness o he p oposed me hod using alida ion on se e al o he da ase s like Landsa -8 o GF-1. La ely, Li e al. [33] in oduced a ligh weigh ne wo k o cloud de ec ion, using mul iscale spec al and spa ial ea u es (CD-FM3SFs) using Sen inel-2A mul ispec al images. The bes accu acy o 98.32% was achie ed using he CPU as a compu a ional uni . To he bes o ou knowledge, he CloudScou cloud de ec ion me hod p oposed by Giu ida e al. [ 34 ] and la e ex ended by Rapuano e al. [ 20 ] is he mos ela ed wo k o his s udy. The me hod was de eloped in he ame o he Phisa -1 ESA mission, which exploi s a hype spec al came a o dis inguish be ween he clea and cloud-co e ed images. To educe he bandwid h, he mission has se a c i e ion ha only images ha p esen less han 70% o he cloudiness a e ansmi ed o he g ound. CloudScou was ained using Sen inel- 2 hype spec al da a and achie ed he 92% o accu acy, 1% o alse posi i es wi h he powe consump ion o 1.8 W deployed on e-con igu able My iad-2 VPU by Mo idius In el [ 34 ]. Ne e heless, he au ho s iden i ied mul iple d awbacks due o he My iad-2 design, which is no speci ically sui able o he space en i onmen (no based on a adia ion- ole an echnology) [ 20 ]. The e o e, he au ho s ex ended hei wo k and p oposed an FPGA-based ha dwa e accele a o o CloudScou CNN. The au ho s compa ed he My iad-2 VPU wi h wo FPGA boa ds: Zynq Ul ascale+ ZCU106 de elopmen boa d and Xilinx Kin ex Ul ascale XQRKU060 adia ion-ha dened boa d. Resul s ob ained by Zynq Ul ascale+ ZCU106 show ha he FPGA-based solu ion educed he in e ence ime by 2.4 imes (141.68 ms) bu a he cos o 1.8 imes g ea e powe consump ion (3.4 W) [ 20 ]. In e ence ime es ima ed o he Xilinx Kin ex Ul ascale XQRKU060 boa d was 1.3 imes as e (264.7 ms) in compa ison wi h he My iad-2 de ice, howe e , he powe consump ion was no epo ed. Rega ding he p esen ed achie emen s o he ela ed wo ks and ends in he Cube- Sa s de elopmen , we may expec a new e a o sma nanosa elli es equipped wi h e- con igu able, p og ammable ha dwa e accele a o s wi h an on-demand edge compu ing pa adigm a payload le el [ 3 , 12 , 19 , 20 , 27 – 29 , 31 , 34 ]. A usual aspec o he p esen ed s udies is he employmen o mul ispec al o hype spec al RSI o he cloud de ec ion sys em. Gene ally, he bands’ composi ion o mul i/hype spec al RSI di e s o indi idual mis- sions, ye all a e equipped wi h an RGB came a. The e o e, a cloud de ec ion sys em buil on RGB bands only may p o ide be e po abili y o a ious missions independen o i s mul i/hype spec al bands. In addi ion, he RGB came as a e se e al imes cheape and mo e con enien o sho - e m CubeSa s missions. To he bes o ou knowledge, we iden i ied only h ee s udies [ 20 , 21 , 31 ] ha pe o med deploymen and e alua ion o he CNN-based cloud de ec ion me hod on an FPGA-based pla o m. Hence, in he scope o his s udy, we would like o p esen CloudSa Ne -1: an FPGA-based ha dwa e-accele a ed Remo e Sens. 2022,14, 3180 4 o 21 quan ized CNN o sa elli e on-boa d cloud co e age classi ica ion. Mo e speci ically, we aim o: • explo e e ec s o quan iza ion in oduced o he p oposed CNN a chi ec u e o cloud co e age classi ica ion, • in es iga e and op imize he pe o mance o cloud co e age classi ica ion by biomes di e si y and i s alse-posi i e iden i ica ions, • explo e ha dwa e a chi ec u e design space o iden i y op imal FPGA esou ce u iliza ion. The es o he pape is o ganized as ollows. Sec ion 2.1 desc ibes he used da ase and i s p ep ocessing. Me hodology is desc ibed in Sec ion 2.3. In Sec ion 3, he esul s a e summa ized. The discussion can be ound in Sec ion 4and he conclusions a e d awn in Sec ion 5. 2. Ma e ials and Me hods 2.1. Da ase Fo he pu pose o his s udy, he Landsa 8 Cloud Co e Assessmen Valida ion da a ( L8 biome da ase ) [ 35 ] was used. The L8 biome da ase o e s a balanced cloud dis ibu ion and di e se se s o land and wa e co e , which makes i a sui able sou ce o da a o he p oposed CNN-based classi ica ion model. The L8 biome da ase was acqui ed by he Landsa 8 Ope a ional Land Image (OLI) and The mal In a ed Senso (TIRS) [ 36 ]. Fu he mo e, da a a e o ho ec i ied and co ec ed o e ain elie using Le el-1T p ocessing [37]. The L8 biome da ase consis s o 96 scenes di ided in o 8 biomes. The scene size is 185 km by 180 km, and each scene con ains 11 mul ispec al bands wi h a esolu ion o 30 m pe pixel (excep bands 8, 10, and 11, which a e no used in his wo k). Manually anno a ed cloud co e age is s o ed as a cloud alida ion mask. The cloud alida ion mask is an image whose pixel alues con ain in o ma ion abou he le el (o class) o cloudiness, in e p e ed using he ollowing Table 1. The example o he scene image (na u al colo composi ion) om he L8 biome da ase can be ound in Figu e 1a, wi h i s espec i e cloud mask in Figu e 1b. (a) (b) Figu e 1. L8 Biome da ase image pa ch example ( a ) econs uc ed om bands B4, B3, and B2 wi h i s associa ed mul i-class cloud mask (b) [35]. (a) Image pa ch. (b) Cloud alida ion mask. Remo e Sens. 2022,14, 3180 5 o 21 Table 1. In e p e a ion o he L8 biome da ase cloud mask pixel alues [35]. Value In e p e a ion 0 Fill 64 Cloud Shadow 128 Clea 192 Thin Cloud 255 Cloud Two cloud mask classes ( hin cloud and cloud) a e ca ego ized as cloud pixels. F om hese pixels, he Cloud Co e Assessmen (CCA) is compu ed as a a io o cloud pixels o all pixels wi h alues exp essed in pe cen age [ 35 ]. The a e age CCA alue o one scene is 48.35%. The dis ibu ion o he CCA alues o he L8 biome da ase scenes is shown in Figu e 2. Scenes a e ca ego ized by hei a ea o cap u e in o biome classes by he In e na ional Geosphe e-Biosphe e P og amme [ 38 ] in o 8 ollowing biomes: Ba en (BA), Fo es (FO), G ass/C ops (GC), Sh ubland (SH), Snow/Ice (SI), U ban (UR), Wa e (WA), We lands (WE). They a e dis inguishable om each o he by hei isual p ope ies, and hey ha e a ious in ensi ies o cloud o e ain con as , which leads o di e en challenges o he cloud de ec ion sys em wo king wi h RGB da a. Fo example, he biomes wi h sha p cloud o e ain con as , like G ass/C ops, ha e a la ge alue o he de i a i e a he ansi ion be ween e ain and cloud. The e o e G ass/C ops biome is easy o classi y as cloud bo de s isibly s and ou om he biome’s e ain. On he con a y, he o he biomes like Snow/Ice ha e a e ain wi h cloud-like ea u es, which may lead o a la ge numbe o alse posi i es in classi ie p edic ions as hei e ain blends wi h clouds. Examples o image pa ches o each biome o he L8 biome da ase a e shown in Figu e 3. 29 554 4 12 5 5 3 24 0–10% 10–20% 20–30% 30–40% 40–50% 50–60% 60–70% 70–80% 80–90% 90–100% 0 5 10 15 20 25 30 CCA ange Numbe o scenes Figu e 2. Dis ibu ion o he L8 biome da ase scenes CCA alues [35]. Remo e Sens. 2022,14, 3180 6 o 21 (a) (b) (c) (d) (e) ( ) (g) (h) Figu e 3. Image pa ches examples o each biome in he L8 biome da ase [ 35 ]. ( a ) Ba en. ( b ) Fo es . (c) G ass/C ops. (d) Sh ubland. (e) Snow/Ice. ( ) U ban. (g) Wa e . (h) We lands. 2.2. Da a P ep ocessing The image pa ch o each scene is a na u al colo composi e om he combina ion o bands B4 ( ed), B3 (g een), and B2 (blue). Values in pa ch images a e e-scaled om he ange 0–65,535 o 0–255 using a MinMax no maliza ion. Pa ch images in he L8 biome da ase a e geo e e enced. The o bi pa h o he Landsa -8 does no go s aigh om sou h o no h. The scene acquisi ion ollows he o bi pa h o he sa elli e. The e o e he image appea s o be o a ed o il ed, like in Figu e 4a. Redundan geo e e encing in o ma ion can be neglec ed when de ec ing clouds om sa elli e images. Nex , he black (no-da a) pa s o he image need o be emo ed. The emo al o he black pa s consis s o wo s eps. Fi s , he image is o a ed, so he ac ual image da a a e pa allel o he whole scene image, as shown in Figu e 4b. The o a ion is using a nea es -neighbo in e pola ion me hod. Then, he image is c opped o lowe esolu ion ( om app ox. 8000 × 8000 o app ox. 6400 ×6400), so only image da a a e p ese ed, as illus a ed in Figu e 4c. Remo e Sens. 2022,14, 3180 7 o 21 0 2000 4000 6000 7000 6000 5000 4000 3000 2000 1000 0 (a) 0 2000 4000 6000 7000 6000 5000 4000 3000 2000 1000 0 (b) 0 1000 2000 3000 4000 5000 6000 6000 5000 4000 3000 2000 1000 0 (c) Figu e 4. L8 Biome da ase scene du ing di e en p ep ocessing s eps [ 35 ]. ( a ) Image pa ch. ( b ) Ro- a ed pa ch. (c) C opped pa ch wi h iling. Image pa ch (wi h dimensions app ox. 6400 × 6400 × 3) is c opped o 512 ×512 ×3 iles, acco ding he whi e lines in Figu e 4c, omi ing iles a he edge ha do no ha e ull esolu ion. Each pa ch has a sligh ly di e en esolu ion a e c opping, which causes a di e en numbe o gene a ed iles pe pa ch (app ox. 140). F om 8 biomes each con aining 12 scenes, he e a e in o al 13,525 iles. The o iginal CCA alues o he scene om Figu e 2 do no apply o indi idual iles. Gene a ed iles usually co e cloudy o cloud- ee a eas. This gene a es ewe iles wi h balanced cloud co e age (o CCA alue) in he inal da ase ( ade-o o c ea ing many iles om ewe image pa ches). Tiles wi h CCA ≥ 70% a e ca ego ized as cloud and he es a e ca ego ized as non-cloud iles. Each o he 13,525 iles has been assigned a co esponding bina y cloud co e age label. To p ese e he e enly dis ibu ed cloud co e age in he ain, alida ion and es da ase , he iles om a single image pa ch a e di ided in o 5 CCA quin iles: 0–20%, 20–40%, 40–60%, 60–80%, 80–100%. The dis ibu ion o he iles and hei CCA alues pe biome o he ull L8 biome da ase is isualized in Figu e 5. The iles om each pa ch CCA quin ile a e di ided o ain, alida ion and es da ase s in he a io 2:1:7, wi h he cohe en a ia ion o he biomes and hei CCA alues, as isualized in Figu e 6. In his s udy, he eliabili y o he esul s and he model po abili y a e p ominen . The e o e, he es ing da ase is dominan in compa ison o he aining o alida ion da ase . Mo eo e , mo e han 2700 iles a e conside ed a sa is ac o y quan i y o he model aining. Since he a ia ion o he ain, alida ion, and he es da ase is cohe en , supp ession o ad an age o any o he biomes o he CCA quin ile du ing he model aining is no expec ed. BA FO GC SH SI UR WA WE 0 200 400 600 800 1000 1200 1400 1600 CCA Bucke 80–100% 60–80% 40–60% 20–40% 0–20% Label cloudy no cloudy Biome Numbe o iles Figu e 5. Dis ibu ion o he iles and i s CCA alues pe biome o he ull L8 biome da ase . Tiles wi h CCA ≥70% a e ca ego ized as cloudy and he es a e ca ego ized as no cloudy iles. Remo e Sens. 2022,14, 3180 8 o 21 BA FO GC SH SI UR WA WE 0 50 100 150 200 250 300 BA FO GC SH SI UR WA WE 0 20 40 60 80 100 120 140 160 BA FO GC SH SI UR WA WE 0 200 400 600 800 1000 1200 CCA Range 80–100% 60–80% 40–60% 20–40% 0–20% Biome Biome Biome Numbe o iles T ain Valida ion Tes Figu e 6. Dis ibu ion o he iles and i s CCA alues pe biome o he aining, alida ion and es ing da ase . 2.3. Me hodology The p ocedu e is di ided in o h ee s ages. Fi s , he baseline model o CNN wi h loa ing poin pa ame e s is ained. Then he weigh s and ac i a ions o he model a e quan ized and he model is e- ained. The las s ep is he deploymen o he model on FPGA o achie e high h oughpu and low powe consump ion sui able o on-boa d da a p ocessing on sa elli e. To be able o deploy a CNN on he edge he e a e many echniques how o educe he model memo y oo p in such as p uning o quan iza ion. In his wo k, he ocus o in e es is on quan iza ion which eplaces loa ing poin ope a ions and weigh enso s wi h lowe bi wid hs which a e especially use ul o FPGA whe e a bi a y p ecision da a ypes can be implemen ed. 2.3.1. Quan ized CNN The main idea o his sec ion is o in oduce he quan iza ion o CNN and i s imple- men a ion o he pu pose o his s udy. Quan iza ion in neu al ne wo ks is a echnique used o op imiza ion which p o ed o p oduce g ea success in he ecen yea s [ 39 ]. I s main ocus is on educing memo y oo p in and compu a ion ime by eplacing loa ing poin compu e ope a ions and s o ing enso weigh s wi h lowe bi wid hs. This is especially use ul o esou ce-cons ained applica ions. The e a e wo ways how o in oduce quan iza ion o a neu al ne wo k. The i s one is o ain he neu al ne wo k wi h quan ized pa ame e s and he second one is a quan iza ion o pa ame e s a e he model is ained wi h loa ing poin p ecision. In he o me case, he p ocess is called quan iza ion-awa e aining (QAT); in he la e , i is e e ed o as pos - aining quan iza ion (PTQ). PTQ may dis u b he model pa ame e s and change he poin o which i con e ged du ing he aining wi h loa ing poin p ecision. Fo his eason, QAT is used o he expe imen s conduc ed in his s udy and aining wi h quan ized model pa ame e s is pe o med. Fo a mo e comp ehensi e e iew o he cu en s a e o quan iza ion in neu al ne wo ks e e o he ecen su ey [39]. The ne wo k was implemen ed using he B e i as amewo k. B e i as is a PyTo ch lib a y used o QAT o neu al ne wo ks [ 40 ]. A he ime o w i ing he PyTo ch lib a y suppo s he quan iza ion as well bu allows jus educ ion om 32-bi loa ing poin o 8-bi in ege [ 41 ]. B e i as in compa ison allows educing he weigh and ac i a ion bi wid hs o as low as 1-bi which enables he c ea ion o bina y neu al ne wo ks (BNN) [ 42 ]. Ano he eason why he B e i as lib a y is used is ha a model ained using B e i as can be expo ed and used by he FINN amewo k o da a low a chi ec u e accele a ion (DFA) on Xilinx FPGAs [ 23 ]. The FINN amewo k is a compile o eed o wa d DFA o deep neu al ne wo ks (DNN) in e ence. When DFA is used, e e y laye o DNN is mapped o i s own se o dedica ed compu e and memo y esou ces [ 43 ] which mimics he opology o DNN. In FINN he pe o mance and esou ce usage can be con olled wi h a concep called Folding. FINN uses wha is called ma ix- ec o h eshold uni s (MVTU) Remo e Sens. 2022,14, 3180 9 o 21 o con olu ional and ully connec ed laye s. The e a e h ee pa ame e s ha can be se : ma ix- ec o ma ix-mul iple ec o (MMV) leng h, p ocessing elemen s (PE), and single ins uc ion mul iple da a (SIMD) lanes. Using hese pa ame e s, i is possible o con ol he h oughpu o he ne wo k wi h espec o esou ce u iliza ion o he FPGA. 2.3.2. CloudSa Ne -1 A chi ec u e In he ollowing pa ag aph, he p oposed CNN a chi ec u e and loss unc ion used du ing he aining pe iod a e desc ibed. The p oposed ne wo k a chi ec u e consis s o 10 con olu ional laye s and 2 ully connec ed laye s, hei speci ic pa ame e s a e isualized in Figu e 7. Each laye excep he las laye uses he ReLU ac i a ion unc ion and has no bias. The ne wo k s a s wi h an ini ial con olu ional laye which p ocesses 512 × 512 × 3 uin 8 inpu and con inues wi h 3 sequences o 3 laye s each. The inpu size was chosen o allow di ec compa ison wi h CloudScou a chi ec u e [ 34 ]. Each sequence middle laye has a lowe numbe o il e s o implemen bo leneck o be e gene aliza ion p ope ies. A e each sequence and ini ial laye , he e is ba ch no maliza ion and max pooling wi h ke nel size o 4, his leads o he e ec i e educ ion o ea u e dimensions. The las ully connec ed laye ou pu s unno malized p obabili y o each class whe e he i s class ep esen s cloud p esence below 70% CCA in he image and he second class signals he p esence o clouds abo e his h eshold. Inpu C0 3 512 FC1 1024 FC2 512 C2 32 6 86C3 8 6 86 C1 128 10 6 6 2 Ou pu Figu e 7. CloudSa Ne -1 a chi ec u e. The loss unc ion used o aining he model was a modi ied bina y c oss en opy loss wi h an inc eased penal y o alse posi i es (FP) e o s shown in Equa ion (1). Penal y o FP e o s is mul iplied by a pa ame e α which is inspi ed by he app oach epo ed in [ 34 ] whe e he au ho s showed a dec ease in he numbe o FP e o s while keeping accu acy on accep able alue when pa ame e αwas se o 2. F(y,ˆ y) = −1 N N ∑ i=1 yi·log(ˆ yi) + α·(1−yi)·log(1−ˆ yi), (1) whe e y is he g ound- u h label, ˆ y is he p edic ed ou pu o he ne wo k and α is a hype -pa ame e o inc ease penal y o FP e o s. 2.3.3. Quan iza ion P ocess Fi s , he model wi h loa ing poin p ecision is ained as a baseline. A e su icien accu acy has been achie ed weigh and ac i a ion bi wid hs a e p og essi ely educed and he change in accu acy is obse ed. To i he model on FPGA and achie e high h oughpu wi h accep able accu acy and low powe consump ion, in his pape he ocus o in e es is bi wid hs o hidden laye s lowe o equal o 4. In all expe imen s, he same bi wid hs a e used o weigh s and ac i a ions. The i s and las laye o he neu al ne wo k can be mo e Remo e Sens. 2022,14, 3180 16 o 21 Table 9. Resul s o cloud co e age classi ica ion o bes -pe o med quan ized models on FPGA. BW ACC (SI) [%] ACC (EXSI) [%] FPS RU [%] APC [W] 2 83.41 92.08 15.462 31.63 2.520 388.24 93.37 15.467 40.20 2.556 4 87.42 94.84 15.468 48.45 2.592 BW—bi wid h; ACC—accu acy; SI—Snow/Ice biome included; EXSI—Snow/Ice biome excluded; FPS— ames pe second; RU—FPGA esou ce u iliza ion; APC—a e age powe consump ion. 4. Discussion 4.1. Quan ized Model o Cloud De ec ion Based on he esul s o he bes -pe o ming models epo ed in he uppe pa o Table 4 , he inc ease o he quan iza ion le el esul ed in sligh o e all pe o mance de e- io a ion. E en hough, he quan ized models achie ed compa able esul s o he 32-bi baseline model (excep he 2-bi model). The dec ease o he o e all accu acy o he 4-bi and 3-bi model is jus a ound 2%, while o he 2-bi model i is mo e han 6%. Howe e , a he han he highes o e all accu acy, his s udy emphasizes on he low FPR (i is mo e con enien o p ocess a edundan image han o disca d he ele an one). The e o e, a balance be ween he ACC and FPR is in demand. Fo he baseline model and 3-bi model, he FPR is iden ically equal o 9.93%. In he case o he 4-bi model, almos 3% o FPR dec ease can be no iced, howe e , he ecall is lowe by 10% in compa ison o he 32-bi model. The 2-bi model su e s he mos om he quan iza ion e ec esul ing in e y in- su icien FPR = 17.59%. Mo e balanced (ACC s. FPR) esul s a e p o ided in he bo om pa o Table 4, whe e he bes models by FPR om he op 10 models so ed by ACC a e epo ed. Un o una ely, he pe o mance o he quan ized models keeps almos on he same le els, ye he baseline model signi ican ly educed i s FPR o 2.25%, while dec easing i s accu acy by a ound 3%. A mo e eadable compa ison o he model’s pe o mance can be seen in Figu e 9. A end o he ade-o be ween ACC and FPR ac oss all quan ized models oge he wi h he baseline is highligh ed by Pa e o on s. I can be obse ed, ha he baseline model ou pe o ms he quan ized ones, howe e , he e can be ound adequa e al e na i es o he 32-bi model. To collec mo e insigh s and o imp o e he o e all pe o mance o he p oposed cloud de ec ion sys em, each biome o he L8 biome da ase was in es iga ed sepa a ely. We hypo hesize ha some biomes p oduce signi ican noise du ing he aining p ocess due o he alse cloud-like ea u es (snow, ice, o og). The 4-bi models we e selec ed o in es iga e biomes in quan ized cases, and i s esul s a e epo ed in Table 5. Bes pe o mance was ob ained by a model ained on G ass/C ops biome wi h ACC = 95.91% and low FPR = 0.83%. Ye , he bes FPR = 0.49% and p ecision o mo e han 99% was achie ed by Fo es biome. Howe e , his model lags on accu acy due o low ecall = 68.36%, which will esul in a high numbe o unde ec ed cloudy images. This may be caused by he cloud ca ego ies me ge ( hin, hick) o by he og, which is a usual alse cloud-like ea u e in he Fo es biome [ 34 ]. Simila ly, he We lands biome (also o en a ec ed by og) esul ed in low FPR = 0.94% and high p ecision = 98.51%, bu wi h low ecall = 68.33%. The Sh ubland, U ban, and Wa e biomes achie ed compa able pe o mance wi h ACC om 91.73% o 93.89% and FPR om 1.89% o 3.92%. The Ba en biome ob ained he second wo se pe o mance in e ms o FPR = 10.14%. The eason o high FPR may lie in he na u e o he Ba en biome, which exagge a es he hin clouds ea u es o hick clouds. The wo s pe o mance epo s he Snow/Ice biome. Low p ecision o 50.47% and high FPR = 31.11% make i s decision almos andom. Since only he RGB channels we e conside ed, he eason o misclassi ica ion is he inabili y o ecognize be ween cloud, ice, and snow. To be able o classi y he clouds abo e he snow and ice, an addi ional spec um capable o al i ude esolu ion will be necessa y [6,10,34]. Rega ding he p e iously men ioned esul s, all biomes, o a ce ain deg ee, su e om he cloud-like ea u es p oblem. An example is gi en in Figu e 10, whe e six misclassi ied cases a e p esen ed. The i s example o he Snow/Ice biome (A) has CCA = 0%, ye he Remo e Sens. 2022,14, 3180 17 o 21 snow in he image was misclassi ied o a cloud. The second example o Snow/Ice biome (B) wi h CCA = 42% me ged cloud wi h u bid snow cu en s. Nex , he smoo h hilly e ain o he Ba en biome (C) s e ches he ea u es o hinly dispe sed clouds. This esul ed in he alsely posi i e image, howe e , he CCA is 10% in eali y. Simila ly, he Wa e biome example (D) wi h CCA = 1% was misclassi ied due o he wa y, se pen ine ea u es o he shallow wa e . The las wo examples (E, F) in Figu e 10 ep esen he case nea he h eshold ( CCA = 70%). He eabou s, a small numbe o cloud pixels may lip he CCA o e he h eshold bounda y. In addi ion, he p ecise alue o he CCA o each ile may be so ly di e en om he CCA label [35,37]. Following he epo ed esul s, he Snow/Ice biome is no sui able o he cloud de ec ion using he p oposed me hod. Mo eo e , p oblema ic coexis ence o he snow, ice and cloud in cloud de ec ion sys ems has been also iden i ied by [ 6 , 10 , 34 ]. The e o e, we decided o wi hd aw he Snow/Ice biome om he ain, alida ion and es da ase s, and o pe o m he expe imen wi hou his noisy da a. In he eal use case, he cloud de ec ion sys em can omi known a eas pe manen ly co e ed by snow o ice om he analysis. Based on he esul s epo ed in Table 6, assumed imp o emen s o all me ics can be obse ed. The bes pe o ming baseline model achie ed ACC = 94.92% wi h FPR = 2.81%. Top quan ized models ob ained compa able accu acy om 94.84% o 92.02%, and FPR om 2.23% o 5.72%. We would like o s ess ou , ha 4-bi quan ized model pe o med sligh ly be e in e ms o p ecision (96.82%) and FPR (2.23%) in compa ison o he 32-bi model. This makes i a p ope quan ized subs i u ion o deploymen on FPGA. Resul s o his analysis con i m ou hypo hesis ha Snow/Ice biome is na u ally p one o being alse posi i e when using RGB channels only. In Figu e 11, he accu acy s. FPR is isualized o models ained wi h excluded Snow/Ice biome. F om ele a ed posi ion o all models wi hin his Figu e i is e iden ha accu acy inc eased all-a ound in compa ison o Figu e 9. Cu es o Pa e o on s lie close oge he and o he baseline on , as he quan iza ion akes a lowe oll on models pe o mance when wi hou isually ambiguous da a. Based on hese esul s, ollowing obse a ions will be emphasised o make a deduc ion. Inc eased quan iza ion did no cause subs an ial d op in alues o e alua ion me ics sco es o esul s wi h excluded Snow/ice biome. The 4-bi model ma ched o o e ook baseline’s me ic sco es in accu acy and FPR. This implies equali y be ween ep esen a ional capaci y o 32-bi baseline and quan ized models in classi ie p oblems ha do no equi e high esolu ion o disce ning disc imina i e ea u es. This s a emen is in line wi h esul s achie ed in o he wo ks [46,55,56] dealing wi h he quan iza ion. The mos ele an s udy, CloudScou [ 20 , 34 ], used hype spec al bands o model aining, esul ing o 16-bi model wi h ACC = 92% and FPR = 1%. Ou p oposed me hod ou pe o med his esul by a 4-bi model wi h highe accu acy up o 3%, howe e wi h lowe FPR by 1.23%. Conside ing, ha ou model used RGB bands only (wi hou Snow/Ice biome), he p esen ed CloudSa Ne -1 me hod b ings p omising imp o emen s in he on- boa d cloud de ec ion sys ems. Ano he ele an s udy [ 21 ] used a la ge aining da ase and achie ed ACC o 91%. Ne e heless, when au ho s deployed he model on FPGA, a signi ican d op o accu acy o 64% occu ed. The me hod in oduced in his pape does no encoun e a simila issue. The compa ison o hese me hods is summa ized in Table 10. Table 10. Compa ison o he bes -pe o med models wi h di e en s udies. Me hod BW ACC [%] FPR [%] APC [W] Da a CloudSa Ne -1 * 4 94.84 2.23 2.5 RGB CloudScou [20] 16 92 11.8 Hype spec al CNV-W1A1 [21] 1 64 - 2.4 RGB *—p oposed me hod; BW—bi wid h; ACC—accu acy; FPR— alse posi i e a e; APC—a e age powe consump ion. Remo e Sens. 2022,14, 3180 18 o 21 4.2. FPGA-Based Ha dwa e Accele a o The quan ized models we e deployed in h ee olding con igu a ions o each bi wid h se ing. Th oughpu , powe consump ion, and FPGA esou ce u iliza ion we e measu ed. Models wi h maximum olding achie ed 15 FPS wi h inpu ba ch size o 1 and almos 20 FPS wi h ba ch size 120 which is he maximal ba ch size allowed o be loaded in o RAM. Inc ease o he FPS wi h highe ba ch size was expec ed, and also con i med by [ 3 ]. Powe consump ion measu ed wi h a USB powe me e epo ed an inc ease o jus o e ≈ 0.2 W du ing he in e ence, compa ed o he es ing s a e. In compa ison wi h ela ed s udies, he au ho s o CloudScou [ 20 , 34 ] epo ed a h oughpu o 2.89 FPS and 1.8 W o powe consump ion using My iad VPU wi h 512 × 512 × 3 inpu size, 7 FPS and 3.4 W o powe consump ion using Zynq Ul ascale+ ZCU106, and 3.77 FPS using XQRKU060 solu ion (es ima ion only). Nex , in he s udy by Rei e e al. [ 21 ], he au ho s epo ed 358.1 FPS wi h a much smalle inpu size 32 × 32 × 3, and maximum powe consump ion o 2.4 W. Rega ding hese esul s, he h oughpu and powe consump ion o he ha dwa e accele a o achie ed in his s udy is compa able wi h he cu en s a e-o - he- a solu ions. Based on he es ima ed numbe o cycles pe laye epo ed in Table 3, i is isible ha a bo le-neck in he i s laye limi ed he op imal h oughpu , and i would equi e a change in he ne wo k a chi ec u e o allow a highe h oughpu a ge . I was demons a ed ha he ne wo k h oughpu can be con olled o a ge a speci ic FPS desi ed by he needs o he mission. A se o expe imen s we e conduc ed o a ge speci ic h oughpu o 10 FPS. Used pa alleliza ion se ings a e epo ed in Table 8. This app oach may be use ul when he ins umen on he CubeSa does no ha e a high h oughpu , e.g., he came a is gene a ing da a a lowe FPS. I showed lexibili y in h oughpu con ol o he FPGA-based ha dwa e accele a o c ea ed by he FINN amewo k. The di e ences o each bi wid h a e in FPGA esou ce u iliza ion, whe e he 2-bi model in base olding con igu a ion u ilized he lowes numbe o he esou ces ( LUT = 46.27% , FF = 31.41% , BRAM = 29.29% , DSP = 0.45% ). This is achie ed due o no pa alleliza ion and a low memo y oo p in o 2-bi weigh s and ac i a ions. I shows he po en ial o educe bi wid h o weigh s and ac i a ions e en u he o 1-bi and expe imen wi h BNN in he u u e o enable highe h oughpu and deepe ne wo k on he same FPGA. As p esen ed in Table 7, DSP slices o he i s laye we e selec ed o be u ilized by Vi ado jus o SPEC and max olding in all bi wid h con igu a ions. Memo y oo p in (BRAM u iliza ion) a ies om 1.43 Mb o 3.06 Mb in he ascending o de ela i e o bi wid h. 5. Conclusions Mos o he RSI is con amina ed by he clouds, hence he quick and accu a e me hod o cloud emo al unning on-boa d o he sa elli e has po en ial o signi ican ly sa e he downlink. In his s udy, we in oduced CloudSa Ne -1, an FPGA-based ha dwa e- accele a ed quan ized CNN o sa elli e on-boa d cloud co e age classi ica ion. We can conclude ha he weigh s and ac i a ions quan iza ion has a minimal o no e ec on he model accu acy. Howe e , he memo y oo p in educ ion allows he model deploymen and es ing on low-cos FPGA Xilinx Zynq-7020. Using he L8 biome da ase and i s RGB bands only, up o 90% o accu acy was achie ed. Nex , we omi ed he Snow/Ice biome iles om he da ase due o high noise p oduc ion. The accu acy inc eased up o 94.4% o accu acy wi h low FPR = 2.23% o he 4-bi wid h model. Wi h he maximum pa alleliza ion se ings, he ha dwa e accele a o achie ed 15 FPS wi h 2.5 W o a e age powe consump ion (0.2 W inc ease o e he idle s a e). Addi ionally, we p o ed ha we can con ol h oughpu o a ge speci ic FPS o he p oposed classi ie . Conside ing he epo ed esul s, he p esen ed no el app oach achie ed ou come compa able wi h he s a e o he a . The p esen ed solu ion has se e al limi a ions ha we would like o s ess ou . Fi s ly, he high numbe o alse posi i e iles wi h a e ain con aining cloud-like ea u es may be in he u u e compensa ed wi h he analysis in ol ing he mul i-spec al bands. Nex , he Remo e Sens. 2022,14, 3180 19 o 21 cloud ca ego ies om he o iginal L8 biome da ase we e me ged o o m a bina y p oblem. The e o e, his s udy did no e alua e he esul on he o iginal cloud ca ego ies o he L8 biome da ase , which migh p o ide mo e insigh s on miss-classi ica ions. Fu he mo e, we did no co e he e ec s o he adia ion on he cloud de ec ion sys em and he edundancy will be subjec o he u u e wo ks. This wo k is he beginning o he g ea e e o o p o ide solu ions based on AI o he space missions ha can bene i om i , hus his wo k is a pilo one in na u e. In he u u e, we aim o imp o e his solu ion o p o ide seman ic segmen a ion o clouds wi h clouds ca ego iza ion o espec i e classes compensa ing o he bina y decision p o ided in his s udy. Au ho Con ibu ions: Concep ualiza ion , J.M. and R.P.; me hodology, R.P., J.M. and L.D.; alida ion, L.D., R.P. and M.M.; o mal analysis, J.M., M.M.; in es iga ion, R.P., L.D. and J.M.; esou ces, M.J.; da a cu a ion, M.J. and R.P.; w i ing—o iginal d a p epa a ion, J.M., R.P., L.D., M.J.; w i ing— e iew and edi ing, J.M., R.P., L.D., M.J. and M.M.; isualiza ion, M.J. and L.D.; supe ision, J.M.; p ojec adminis a ion, M.M. All au ho s ha e ead and ag eed o he published e sion o he manusc ip . Funding: This esea ch ecei ed no ex e nal unding. Da a A ailabili y S a emen : No applicable Con lic s o In e es : The au ho s decla e no con lic o in e es . Re e ences 1. Lo q is , M.; Cano, J. Op imizing Da a P ocessing in Space o Objec De ec ion in Sa elli e Image y. a Xi 2021 , a Xi :2107.03774. 2. Caldwell, S. S a e-o - he-A o Small Spacec a Technology; NASA: Washing on, DC, USA, 2021. 3. Maskey, A.; Cho, M. CubeSa Ne : Ul aligh Con olu ional Neu al Ne wo k designed o on-o bi bina y image classi ica ion on a 1U CubeSa . Eng. Appl. A i . In ell. 2020,96, 103952. [C ossRe ] 4. Kulu, E. Nanosa s Da abase. 2022. A ailable online: h ps://www.nanosa s.eu/ (accessed on 15 Sep embe 2021). 5. EUSPA. EUSPA EO and GNSS Ma ke Repo ; EUSPA: P ague, Czech Republic, 2022. 6. Yang, J.; Guo, J.; Yue, H.; Liu, Z.; Hu, H.; Li, K. CDne : CNN-based cloud de ec ion o emo e sensing image y. IEEE T ans. Geosci. Remo e Sens. 2019,57, 6195–6211. [C ossRe ] 7. Fen a, A.A.; Yasuda, H.; Ha egeweyn, N.; Belay, A.S.; Hadush, Z.; Geb emedhin, M.A.; Mekonnen, G. The dynamics o u ban expansion and land use/land co e changes using emo e sensing and spa ial me ics: The case o Mekelle Ci y o no he n E hiopia. In . J. Remo e Sens. 2017,38, 4107–4129. [C ossRe ] 8. King, M.D.; Pla nick, S.; Menzel, W.P.; Acke man, S.A.; Hubanks, P.A. Spa ial and empo al dis ibu ion o clouds obse ed by MODIS onboa d he Te a and Aqua sa elli es. IEEE T ans. Geosci. Remo e Sens. 2013,51, 3826–3852. [C ossRe ] 9. Zhang, Y.; Rossow, W.B.; Lacis, A.A.; Oinas, V.; Mishchenko, M.I. Calcula ion o adia i e luxes om he su ace o op o a mosphe e based on ISCCP and o he global da a se s: Re inemen s o he adia i e ans e model and he inpu da a. J. Geophys. Res. A mos. 2004,109, D19105. [C ossRe ] 10. Guo, J.; Yang, J.; Yue, H.; Tan, H.; Hou, C.; Li, K. CDne V2: CNN-based cloud de ec ion o emo e sensing image y wi h cloud-snow coexis ence. IEEE T ans. Geosci. Remo e Sens. 2020,59, 700–713. [C ossRe ] 11. Wu, Z.; Li, J.; Wang, Y.; Hu, Z.; Molinie , M. Sel -a en i e gene a i e ad e sa ial ne wo k o cloud de ec ion in high esolu ion emo e sensing images. IEEE Geosci. Remo e Sens. Le . 2019,17, 1792–1796. [C ossRe ] 12. Li, L.; Li, X.; Jiang, L.; Su, X.; Chen, F. A e iew on deep lea ning echniques o cloud de ec ion me hodologies and challenges. Signal Image Video P ocess. 2021,15, 1527–1535. [C ossRe ] 13. K eu e , A.; Zange l, M.; Schwa zmann, M.; Blum hale , M. All-sky imaging: A simple, e sa ile sys em o a mosphe ic esea ch. Appl. Op . 2009,48, 1091–1097. [C ossRe ] 14. Long, C.N.; Sabbu g, J.M.; Calbó, J.; Pagès, D. Re ie ing cloud cha ac e is ics om g ound-based day ime colo all-sky images. J. A mos. Ocean. Technol. 2006,23, 633–652. [C ossRe ] 15. F an z, D.; Haß, E.; Uhl, A.; S o els, J.; Hill, J. Imp o emen o he Fmask algo i hm o Sen inel-2 images: Sepa a ing clouds om b igh su aces based on pa allax e ec s. Remo e Sens. En i on. 2018,215, 471–481. [C ossRe ] 16. Zhu, Z.; Woodcock, C.E. Au oma ed cloud, cloud shadow, and snow de ec ion in mul i empo al Landsa da a: An algo i hm designed speci ically o moni o ing land co e change. Remo e Sens. En i on. 2014,152, 217–234. [C ossRe ] 17. Sui, Y.; He, B.; Fu, T. Ene gy-based cloud de ec ion in mul ispec al images based on he SVM echnique. In . J. Remo e Sens. 2019 , 40, 5530–5543. [C ossRe ] 18. Zi, Y.; Xie, F.; Jiang, Z. A cloud de ec ion me hod o Landsa 8 images based on PCANe . Remo e Sens. 2018 ,10, 877. [C ossRe ] 19. Mi alles, P.; Scannapieco, A.F.; Jagadam, N.; Ba anwal, P.; Faldu, B.; Abhang, R.; Bha ia, S.; Bonna , S.; Bha naga , I.; Ba ul, B.; e al. Machine Lea ning in Ea h Obse a ion Ope a ions: A e iew. In P oceedings o he 72nd In e na ional As onau ical Cong ess (IAC), Dubai, Uni ed A ab Emi a es, 25–29 Oc obe 2021; In e na ional As onau ical Fede a ion: Pa is, F ance, 2021. Remo e Sens. 2022,14, 3180 20 o 21 20. Rapuano, E.; Meoni, G.; Pacini, T.; Dinelli, G.; Fu ano, G.; Giu ida, G.; Fanucci, L. An FPGA-Based Ha dwa e Accele a o o CNNs In e ence on Boa d Sa elli es: Benchma king wi h My iad 2-Based Solu ion o he CloudScou Case S udy. Remo e Sens. 2021,13, 1518. [C ossRe ] 21. Rei e , P.; Ka agiannakis, P.; I eland, M.; G eenland, S.; C ocke , L. FPGA accele a ion o a quan ized neu al ne wo k o emo e-sensed cloud de ec ion. In P oceedings o he 7 h In e na ional Wo kshop on On-Boa d Payload Da a Comp ession, Online, 21–23 Sep embe 2020. 22. Vlonca . as machinelea ning/hls4ml: Co is; Zenodo: Gene a, Swi ze land, 2021. [C ossRe ] 23. Blo , M.; P euße , T.B.; F ase , N.J.; Gamba della, G.; O’b ien, K.; Umu oglu, Y.; Leese , M.; Visse s, K. FINN-R: An end- o-end deep-lea ning amewo k o as explo a ion o quan ized neu al ne wo ks. ACM T ans. Recon igu able Technol. Sys . TRETS 2018 , 11, 1–23. [C ossRe ] 24. Yao, Y.; Jiang, Z.; Zhang, H.; Zhou, Y. On-boa d ship de ec ion in mic o-nano sa elli e based on deep lea ning and COTS componen . Remo e Sens. 2019,11, 762. [C ossRe ] 25. Schwa z, C.; Sande , I.; Jo dão, R.; B uhn, F.; Pe sson, M.; Ekblad, J.; Fuglesang, C. On-boa d sa elli e da a p ocessing o achie e sma in o ma ion collec ion. In Op ics, Pho onics and Digi al Technologies o Imaging Applica ions VII; Schelkens, P., Kozacki, T., Eds.; In e na ional Socie y o Op ics and Pho onics, SPIE: Bellingham, WA, USA, 2022; Volume 12138, pp. 121–131. [C ossRe ] 26. ElMas y, G.; Sun, D.W. P inciples o Hype spec al Imaging Technology. In Hype spec al Imaging o Food Quali y Analysis and Con ol; Else ie : Ams e dam, The Ne he lands, 2010; pp. 3–43. 27. Zhang, Z.; Iwasaki, A.; Xu, G.; Song, J. Cloud de ec ion on small sa elli es based on ligh weigh U-ne and image comp ession. J. Appl. Remo e Sens. 2019,13, 026502. [C ossRe ] 28. Zhang, J.; Li, X.; Li, L.; Sun, P.; Su, X.; Hu, T.; Chen, F. Ligh weigh U-Ne o cloud de ec ion o isible and he mal in a ed emo e sensing images. Op . Quan um Elec on. 2020,52, 1–14. [C ossRe ] 29. Zhang, Z.; Xu, G.; Song, J. CubeSa cloud de ec ion based on JPEG2000 comp ession and deep lea ning. Ad . Mech. Eng. 2018 ,10, 1687814018808178. [C ossRe ] 30. Pa k, J.H.; Inamo i, T.; Hamaguchi, R.; O suki, K.; Kim, J.E.; Yamaoka, K. RGB Image P io i iza ion Using Con olu ional Neu al Ne wo k on a Mic op ocesso o Nanosa elli es. Remo e Sens. 2020,12, 3941. [C ossRe ] 31. G eenland, S.; I eland, M.; Kobayashi, C.; Mendham, P.; Pos , M.; Whi e, D. De elopmen o a mina u ised o wa ds looking image using deep lea ning o esponsi e ope a ions. In P oceedings o he 4S Symposium, So en o, I aly, 28 May–1 June 2018; ESA: Pa is, F ance, 2018. 32. de Vieille ille, F.; Lag ange, A.; Ruiloba, R.; May, S. Towa ds Dis illa ion o Deep Neu al Ne wo ks o Sa elli e On-Boa d Image Segmen a ion. In . A ch. Pho og amm. Remo e Sens. Spa . In . Sci. 2020,43, 1553–1559. [C ossRe ] 33. Li, J.; Wu, Z.; Hu, Z.; Jian, C.; Luo, S.; Mou, L.; Zhu, X.X.; Molinie , M. A Ligh weigh Deep Lea ning-Based Cloud De ec ion Me hod o Sen inel-2A Image y Fusing Mul iscale Spec al and Spa ial Fea u es. IEEE T ans. Geosci. Remo e Sens. 2021 ,60, 1–19. [C ossRe ] 34. Giu ida, G.; Diana, L.; de Gioia, F.; Benelli, G.; Meoni, G.; Dona i, M.; Fanucci, L. Cloudscou : A deep neu al ne wo k o on-boa d cloud de ec ion on hype spec al images. Remo e Sens. 2020,12, 2205. [C ossRe ] 35. Foga, S.; Sca amuzza, P.L.; Guo, S.; Zhu, Z.; Dilley J , R.D.; Beckmann, T.; Schmid , G.L.; Dwye , J.L.; Hughes, M.J.; Laue, B. Cloud de ec ion algo i hm compa ison and alida ion o ope a ional Landsa da a p oduc s. Remo e Sens. En i on. 2017 ,194, 379–390. [C ossRe ] 36. Ma kham, B.; S o ey, J.; Mo i , R. Landsa -8 senso cha ac e iza ion and calib a ion. Remo e Sens. 2015 ,7, 2279–2282. [C ossRe ] 37. USGS. Landsa Da a Con inui y Mission Le el 1 (L1) Da a Fo ma Con ol Book (DFCB), LDCM-DFCB-004, Ve sion 6.0; USGS: Res on, VA, USA, 2012. 38. Lo eland, T.R.; Reed, B.C.; B own, J.F.; Ohlen, D.O.; Zhu, Z.; Yang, L.; Me chan , J.W. De elopmen o a global land co e cha ac e is ics da abase and IGBP DISCo e om 1 km AVHRR da a. In . J. Remo e Sens. 2000,21, 1303–1330. [C ossRe ] 39. Gholami, A.; Kim, S.; Dong, Z.; Yao, Z.; Mahoney, M.W.; Keu ze , K. A Su ey o Quan iza ion Me hods o E icien Neu al Ne wo k In e ence. a Xi 2021, a Xi :2103.13630. 40. Pappala do, A. Xilinx/B e i as; Zenodo: Gene a, Swi ze land, 2021. [C ossRe ] 41. Quan iza ion. A ailable online: h ps://py o ch.o g/docs/s able/quan iza ion.h ml (accessed on 28 No embe 2021). 42. Cou ba iaux, M.; Huba a, I.; Soud y, D.; El-Yani , R.; Bengio, Y. Bina ized neu al ne wo ks: T aining deep neu al ne wo ks wi h weigh s and ac i a ions cons ained o+ 1 o -1. a Xi 2016, a Xi :1602.02830. 43. Alonso, T.; Pe ica, L.; Ruiz, M.; Pe i-Koenig, J.; Umu oglu, Y.; S amelos, I.; Ko omilas, E.; Blo , M.; Visse s, K. Elas ic-DF: Scaling Pe o mance o DNN In e ence in FPGA Clouds h ough Au oma ic Pa i ioning. ACM T ans. Recon igu able Technol. Sys . 2021 , 15, 1–34. [C ossRe ] 44. P zewlocka, D.; Wasala, M.; Szolc, H.; Blachu , K.; K yjak, T. Op imisa ion o a Siamese Neu al Ne wo k o Real-Time Ene gy E icien Objec T acking. a Xi 2020, a Xi :2007.00491 45. Mellempudi, N.; Kundu, A.; Mudige e, D.; Das, D.; Kaul, B.; Dubey, P. Te na y Neu al Ne wo ks wi h Fine-G ained Quan iza ion. a Xi 2017, a Xi :1705.01462. 46. Zhou, S.; Wu, Y.; Ni, Z.; Zhou, X.; Wen, H.; Zou, Y. DoReFa-Ne : T aining Low Bi wid h Con olu ional Neu al Ne wo ks wi h Low Bi wid h G adien s. a Xi 2016, a Xi :1606.06160. Remo e Sens. 2022,14, 3180 21 o 21 47. Io e, S.; Szegedy, C. Ba ch No maliza ion: Accele a ing Deep Ne wo k T aining by Reducing In e nal Co a ia e Shi . P oc. Mach. Lea n. Res. PMLR 2015,37, 448–456. 48. Umu oglu, Y.; Jah e, M. S eamlined Deploymen o Quan ized Neu al Ne wo ks. a Xi 2017, a Xi :1709.04060. 49. C ocke , L.; No hco e, D.; Ramsay, C. Explo ing Zynq MPSoC: Wi h PYNQ and Machine Lea ning Applica ions; S a hclyde Academic Media: Glasgow, Sco land, 2019. 50. Xilinx. Zynq-7000 SoC Da a Shee : O e iew. 2018. A ailable online: h ps://www.xilinx.com/con en /dam/xilinx/suppo / documen s/da a_shee s/ds190-Zynq-7000-O e iew.pd (accessed on 8 Sep embe 2021). 51. PYNQ: PYTHON PRODUCTIVITY. A ailable online: h p://www.pynq.io/ (accessed on 8 Feb ua y 2022). 52. De elope s, O.R. ONNX Run ime. Ve sion: X.y.z. 2021. A ailable online: h ps://onnx un ime.ai/ (accessed on 1 Sep embe 2021). 53. Shah ia i, B.; Swe sky, K.; Wang, Z.; Adams, R.P.; de F ei as, N. Taking he Human Ou o he Loop: A Re iew o Bayesian Op imiza ion. P oc. IEEE 2016,104, 148–175. [C ossRe ] 54. Fa aone, J.; Gamba della, G.; Boland, D.; F ase , N.; Blo , M.; Leong, P.H. Cus omizing Low-P ecision Deep Neu al Ne wo ks o FPGAs. In P oceedings o he 2018 28 h In e na ional Con e ence on Field P og ammable Logic and Applica ions (FPL), Dublin, I eland, 26–30 Augus 2018; pp. 97–973. [C ossRe ] 55. Han, S.; Mao, H.; Dally, W.J. Deep comp ession: Comp essing deep neu al ne wo ks wi h p uning, ained quan iza ion and hu man coding. a Xi 2015, a Xi :1510.00149. 56. Zhu, C.; Han, S.; Mao, H.; Dally, W.J. T ained e na y quan iza ion. a Xi 2016, a Xi :1612.01064.