O e coming segmen a ion con usions in PRISMA
hype spec al images
Se ban Op isescu1, And ei Raco i eanu1,2, Mihai I ano ici1
1 Elec onics and Compu e s Depa men , T ansil ania Uni e si y o B așo , România
{se ban.op isescu, mihai.i ano ici}@uni b . o
2 Image P ocessing and Analysis Labo a o y, Na ional Uni e si y o Sci. & Tech. Poli ehnica, Bucu eș i, România
and ei. aco i eanu@upb. o
Abs ac —Hype spec al sa elli e images (HSI) which con ain
hund eds o spec al bands, o e he possibili y o a much ine
cha ac e iza ion o g ound su aces. Ag icul u al c op
iden i ica ion and segmen a ion emains one o he mos used
echniques o cha ac e iza ion o land use. E en hough HSI
allows be e segmen a ion esul s han in he case o
mul ispec al da a, he e a e s ill challenges when di e en c ops
look he same a ce ain de elopmen s ages. This pape p oposes
wo me hods o segmen a ion con usion elimina ion. The i s
me hod elies in analyzing he his og am o SAM alues ob ained
by compa ing di e en pa cels. The second me hod elies on he -
dis ibu ed S ochas ic Neighbo Embedding ( -SNE). We illus a e
he e iciency o ou p oposed me hods wi h PRISMA
hype spec al images om he egion o B aso , Romania.
I. INTRODUCTION
Sma ag icul u e is one o he main esea ch a eas oday,
conside ing he challenges in oduced by clima e change.
A i icial in elligence applied on he da a p o ided by sa elli es
and g ound senso s aims o ob ain be e yield, o de ec
diseases, d ough e c. Op ical sa elli e imaging has e ol ed om
mul ispec al (which is s ill used oday wi hin sa elli es such as
Sen inel 2) o hype spec al. Hype spec al sa elli e imaging
(HSI) o e s g ound images in hund eds o na ow spec al
bands. Fo ins ance, he PRISMA (Hype spec al P ecu so o
he Applica ion Mission) sa elli e buil by ASI (Agenzia
Spaziale I aliana) [1] o e s 239 spec al bands wi h a spec al
esolu ion lowe han 12nm, in he ange 400 – 2500 nm.
Ag icul u al c op iden i ica ion and classi ica ion based on
hype spec al images is an ongoing esea ch ield. Mainly he e
a e wo app oaches: c op iden i ica ion based on a single
sa elli e image o based on a ime se ies (se e al sa elli e images
aken a some ime in e als). The ime se ies c op iden i ica ion
ies o classi y di e en c ops based on he knowledge o hei
g ow h cha ac e is ics in ime. In his pape we ocus on he
ag icul u al c op iden i ica ion based on a single hype spec al
sa elli e image.
As shown in [2], he e a e many echniques which can be
used o hype spec al image segmen a ion, such as: Suppo
Vec o Machine (SVM), Ex ended Mo phological P o ile
(EMP), Join Spa se Rep esen a ion (JSR), 3D Con olu ional
Neu al Ne wo k (3D-CNN), CNN wi h Poin Pai Fea u es
(CNN-PFF), Gabo -based Con olu ional Neu al Ne wo k
(Gabo -CNN), 3D Gene a i e Ad e sa ial Ne wo k (3D-GAN),
Deep Fea u e Fusion Ne wo k (DFFN) e c. The accu acy o he
segmen a ion ob ained using any o hese me hods a ies [2]
unc ion o he numbe o classes, he cha ac e is ic o he
image i sel , he amoun o aining samples e c. The e is no
uni e sal me hod, and he esul s depend on each classi ica ion
scena io.
In ou segmen a ion expe imen s on PRISMA hype spec al
images, we no iced ha he accu acy s ongly depends on he
selec ed classes (o he selec ed c op ypes) and hei
de elopmen s age. The e a e c ops which can be easily
disce ned wi hin he i s weeks o de elopmen , such as whea
and apeseed. And he e a e c ops ha can be misclassi ied such
as he con usion be ween whea and al al a in he i s s age o
de elopmen .
In his pape we add ess he p oblem o c op segmen a ion
con usion by p oposing a me hod based on his og ams o
spec al angle mappe (SAM).
The es o he pape is s uc u ed as ollows. Sec ion II
desc ibes he segmen a ion con usions and challenges and
depic s he p oposed me hodology. Sec ion III includes he
esul s ob ained on selec ed hype spec al c ops. Sec ion IV
concludes he pape .
II. THE PROPOSED METHOD
Fi s o all, we would like o illus a e a case when pixel-wise
segmen a ion ails because o he high simila i y be ween he
spec al signa u es belonging o wo di e en c ops. Fig. 1
shows a c op (RGB band selec ion) om a PRISMA image –
he a ea o B aso , Romania, 23 Ma s 2024. Two pa cels a e
highligh ed: g assland (P1) and apeseed (P2).
Figu e 1. A g assland pa cel (P1) and a apeseed pa cel (P2)
Fig. 2. shows he spec al signa u es ( he spec al e lec ance
cu es – SRC) o all he pixels wi hin he wo highligh ed
pa cels om Fig. 1.
Figu e 2. The SRCs o all pixels om he pa cels o Fig. 1
One no ices ha he SRCs o he pixels om he wo pa cels
a e e y simila , which leads o segmen a ion con usions.
To show ha con usions appea be ween he wo pa cels, we
pe o med a Random Fo es (RF) segmen a ion on he PRISMA
c op, choosing i e classes: apeseed, g assland, whea ,
buildings and ba e soil. Fig. 3.a) shows he aining a eas.
Figu e 3. a) T aining a eas o he i e classes ( apeseed, g assland, whea ,
buildings and ba e soil); b) RF segmen a ion esul . The con usion a ea is
ma ked wi h a ci cle
The esul o he RF segmen a ion is shown in Fig. 3. b). One
no ices, as expec ed, ha a apeseed pa cel (ma ked wi h a
ci cle) is segmen ed as g assland. Tha a ea should be ed.
These con usions appea i we y o he segmen a ion me hods
such as K-means o Fuzzy C means (FCM). We canno y deep
lea ning me hods on he selec ed image because o he e y
limi ed aining se . Bu he cause o he con usion is clea ly he
simila i y be ween he SRCs in his de elopmen s age o he
wo c ops.
To o e come he con usion, we p opose in his pape he use
o his og ams o Spec al Angle Mappe (SAM) [3] alues.
SAM compa es wo spec a; in ou case, as shown in (1),
deno es a e e ence SRC and a es SRC. The alue o SAM
should be ze o i he wo SRCs a e iden ical.
The p oposed me hod is explained in Algo i hm 1. The i s
s ep in ou algo i hm is he image ans o ma ion using he
Minimum Noise F ac ion ans o m (MNF), educing he
numbe o bands o 30. We hen choose wo pa cels, a e e ence
pa cel P1 and a es pa cel P2, and build a ma ix deno ed ISAM
which con ains he SAM alues compu ed on all pai s o pixels
be ween he wo pa cels. And hen we compu e he his og am o
he ISAM ma ix.
Algo i hm 1 The p oposed me hod
Inpu : he hype spec al image I
Ou pu : H ( he his og am o SAM alues)
I = MNF(I)
manually choose wo pa cels P1 and P2 om I
o each pixel i o P1
o each pixel j o P2
deno e he SRC o he i pixel
deno e he SRC o he j pixel
compu e ISAM(i,j) = he SAM alue be ween and
compu e H = his og am(ISAM)
(1)
III. EXPERIMENTAL RESULTS
We gi e he e an example whe e we selec ed ou pa cels as
shown in Fig. 4.a) P1 is he e e ence pa cel ( apeseed). The
es pa cels a e: P2 ( apeseed), P3 (g assland) and P4 (whea ).
The c op is om he same PRISMA hype spec al image,
B aso egion, 23 Ma s 2024. Fig. 5 shows he ob ained
his og am o SAM alues. Algo i hm 1 was applied on he h ee
pai s o pa cels, P1-P2, P1-P3 and P1-P4 and all ob ained SAM
alues we e conca ena ed in a single ec o . The pu pose o
such expe imen s is o p opose a me hod o c op ype
iden i ica ion. Thus, one akes a e e ence known pa cel P1 and
hen selec many o he pa cels (P2, P3, …, Pn). When we ob ain
he his og am o SAM alues om all P1-Pi pai s o pa cels, he
pa cels wi h he same c op will be he i s . Because SAM
compu es he simila i ies be ween all pa cels pixels, and smalle
SAM alues indica e iden ical c ops in ou scena io. This is
easily no iced in Fig. 5 whe e he P1-P2 ( apeseed- apeseed)
his og am is he i s . The p oposed me hod esembles he
image indexing me hods, whe e he sys em ecei es a que y
image and e u ns all simila images in dec easing o de o
simila i y.
Figu e 4. a) The e e ence pa cel P1 ( apeseed) and he es pa cels P2
( apeseed), P3 (g assland) and P4 (whea ); b) Fou pa cels: P1 (g assland –
lowe igh ), P2 (g assland – igh ), P3 ( apeseed) and P4 ( apeseed)
Figu e 5. The SAM his og ams be ween P1-P2, P1-P3 and P1-P4
As we men ioned a he beginning o his s udy, we add essed
he con usions which may appea be ween c ops a ce ain
de elopmen s ages. We also applied o hese cases he -
dis ibu ed S ochas ic Neighbo Embedding ( -SNE) [5] which
is a powe ul dimensionali y educ ion and da a isualiza ion
echnique. -SNE is an unsupe ised non-linea me hod ha
ocuses on p ese ing he simila i ies and ela ionships be ween
da a poin s in a lowe dimensional space.
We applied -SNE on he ou pa cels om Fig. 4.b) The
esul is shown in Fig. 7. The wo g assland pa cels (P1 and P2)
g oup oge he , while he wo apeseed pa cels (P3 and P4) a e
sepa a ed because o he di e ence in de elopmen s age.
Anyway, one can see in Fig. 7 ha he con usion be ween
g assland and apeseed ha was ob ained a e RF segmen a ion
(Fig. 3 b)) is no longe isible a e -SNE. Hence, -SNE could
also be used o disc imina ing be ween di e en c op pa cels,
bu as we saw, i is sensi i e also o in a-class a ia ions ha
a e caused by de elopmen s ages. Fig. 6 shows he p oposed
me hod applied on he pai s o pa cels om Fig. 4 b) when we
ake P1 (g assland) as e e ence. The sepa a ion be ween
g assland and apeseed is isible in Fig. 6.
Figu e 6. His og am o he pai s o pa cels om Fig. 4.b)
Figu e 7. -SNE o he pai s o pa cels om Fig. 4.b)
Finally, in Fig. 8 one shows a selec ion o i e pa cels ha
ha e much lowe con usion be ween hem. These pa cels a e:
P1 (whea ), P2 ( apeseed), P3 ( apeseed), P4 (whea ) and P5
(whea ). We applied ou algo i hm on hese pa cels, aking as
e e ence he whea pa cel (P1). The ob ained his og am is
shown on Fig. 9. One clea ly sees he much bigge sepa a ion
be ween he pai s o whea pa cels (P1-P4 and P1-P5) and he
pai s o di e en c op pa cels.
Figu e 8. The e e ence pa cel P1 (whea ) and he es pa cels: P2 ( apeseed),
P3 ( apeseed), P4 (whea ) and P5 (whea )
Figu e 9. His og am o pai s o pa cels om Fig. 8
Fig. 10 shows he empi ical cumula i e dis ibu ion unc ions
(ECDF) o he SAM alues co esponding o he pa cels om
Fig. 8. One no ices ha he whea plo s (P1-P4 and P1-P5)
g oup oge he i.e. he dis ance on he X-axis be ween hem is
small and hey a e nea he o igin o he X-axis. The
combina ions be ween di e en c ops (whea - apeseed) P1-P2
and P1-P3 also g oup oge he bu a a g ea e dis ance han he
cu es plo ed o he same c op (whea -whea ).
To ha e a nume ical measu e o he simila i y be ween hese
plo ed dis ibu ions, we compu ed he wo-sample
Kolmogo o -Smi no es (ks es 2) and he Jensen–Shannon
di e gence (JSD). K es 2 is a g ea s a is ical measu e ha ells
i wo samples came om he same dis ibu ion (in ou case i
he c op ype is he same). The alue o he es is called k1. A
alue o k1 close o ze o a i ms he null hypo hesis ( ha he
samples a e om he same dis ibu ion); a alue o k1 close o
one ejec s he null hypo hesis ( he pa cels con ain di e en
c op ypes). Table I shows he ob ained alues o Fig. 10. The
JSD is an en opic measu e o di e gence be ween wo
p obabili y dis ibu ions based on he Kullback-Leible
di e gence. The ob ained alues a e shown in Table I and ha e
he same meaning as he k1 alues. Hence, by a simple h eshold
on k1 o he JSD alue we a e able o decide i he compa ed
pa cels ha e he same c op (small alues) o no .
Figu e 10. ECDF o pa cels om Fig. 8
TABLE I. SAM DISTRIBUTIONS SIMILARITY METRICS
Pai s o pa cels
K
1
alue
JSD alue
P1-P2 s P1-P3 0.36 0.16
P1-P3 s P1-P4 1.00 0.99
P1-P4 s P1-P5 0.26 0.07
P1-P2 s P1-P4 1.00 0.99
IV. CONCLUSIONS
The iden i ica ion and segmen a ion o ag icul u al c ops
applied on sa elli e images s ill has i s challenges mainly due o
he simila i y o ce ain c ops wi hin some de elopmen s ages.
In his pape we p oposed wo me hods o c op iden i ica ion
and disc imina ion. The i s me hod elies on analyzing he
his og am o SAM alues compu ed pixelwise on pai s o SRCs
om selec ed pa cels. The p oposed me hod esembles image
indexing and sea ching echniques, by aking a known e e ence
pa cel and compa e i wi h a numbe o di e en pa cels. The
pa cels ha ing he same c ops as he e e ence pa cel will be
plo ed i s in he ob ained his og am. The second me hod
consis s o isualizing he 2D -SNE o he selec ed pa cels.
This la e me hod, e en hough i clea ly sepa a es he di e en
c ops, also sepa a es he same ype c ops ha a e in sligh ly
di e en de elopmen s ages. Ks es 2 and JSE ha e been used
o p o e ha ou me hod disc imina es well be ween c ops.
ACKNOWLEDGMENT
This wo k was unded by he Eu opean Union, om he
AI4AGRI p ojec . The AI4AGRI p ojec en i led “Romanian
Excellence Cen e on A i icial In elligence on Ea h
Obse a ion Da a o Ag icul u e” ecei ed unding om he
Eu opean Union’s Ho izon Eu ope esea ch and inno a ion
p og am unde g an ag eemen no. 101079136.
REFERENCES
[1] ASI websi e: www.asi.i /en/ea h-science/p isma
[2] S. Li, W. Song, L. Fang, Y. Chen, P. Ghamisi and J. A. Benedik sson,
"Deep Lea ning o Hype spec al Image Classi ica ion: An O e iew,"
in IEEE T ansac ions on Geoscience and Remo e Sensing, ol. 57, no. 9,
pp. 6690-6709, Sep . 2019, doi: 10.1109/TGRS.2019.2907932.
[3] F.A. K use e al, "The Spec al Image P ocessing Sys em (SIPS) -
In e ac i e Visualiza ion and Analysis o Imaging spec ome e Da a."
Remo e Sensing o En i onmen 44 (1993): 145-163.
[4] A. A. G een, M. Be man, P. Swi ze , and M. D. C aig, 1988, A
ans o ma ion o o de ing mul ispec al da a in e ms o image quali y
wi h implica ions o noise emo al: IEEE T ans. on Geos. and Remo e
Sensing, . 26, no. 1, p. 65-74.
[5] L. Van de Maa en and G. Hin on, Visualizing Da a Using -SNE. Jou nal
o Machine Lea ning Resea ch, 1, 1-48, 2008
