A icle
Alignmen o Hype spec al Images Using
KAZE Fea u es
Ál a o O dóñez 1,*ID , F ancisco A güello 2ID and Do a B. He as 1ID
1Cen o Singula de In es igación en Tecnoloxías da In o mación (CiTIUS), Uni e sidade de San iago de
Compos ela, 15782 San iago de Compos ela, Spain; [email p o ec ed]
2Depa amen o de Elec ónica e Compu ación, Uni e sidade de San iago de Compos ela,
15782 San iago de Compos ela, Spain; [email p o ec ed]
*Co espondence: al a [email p o ec ed]; Tel.: +34-8818-16474
Recei ed: 2 Ap il 2018; Accep ed: 14 May 2018; Published: 15 May 2018
Abs ac :
Image egis a ion is a common ope a ion in any ype o image p ocessing, specially in
emo e sensing images. Since he publica ion o he scale–in a ian ea u e ans o m (SIFT) me hod,
se e al algo i hms based on ea u e de ec ion ha e been p oposed. In pa icula , KAZE builds
he scale space using a nonlinea di usion il e ins ead o Gaussian il e s. Nonlinea di usion
il e ing allows applying a con olled blu while he impo an s uc u es o he image a e p ese ed.
Hype spec al images con ain a la ge amoun o spa ial and spec al in o ma ion ha can be used
o pe o m a mo e accu a e egis a ion. This a icle p esen s HSI–KAZE, a me hod o egis e
hype spec al emo e sensing images based on KAZE bu conside ing he spec al in o ma ion.
The p oposed me hod combines he in o ma ion o a se o p eselec ed bands, and i adap s he
keypoin desc ip o and he ma ching s age o ake in o accoun he spec al in o ma ion. The me hod
is adequa e o egis e images in ex eme si ua ions in which he scale be ween hem is e y di e en .
The e ec i eness o he p oposed algo i hm has been es ed on eal images aken on di e en da es,
and p esen ing di e en ypes o changes. The expe imen al esul s show ha he me hod is obus
achie ing image egis a ions wi h scales o up o 24.0×.
Keywo ds: hype spec al da a; image egis a ion; KAZE ea u es; emo e sensing
1. In oduc ion
The ad ances in senso de elopmen in he las decades allow us o ob ain hype spec al images
(HSI) a a lowe cos han be o e. Each pixel o hese images con ains a con inuous spec um ha is
o med by hund eds o na ow bands. As a esul o his high spec al esolu ion, objec s, plan species,
and land–co e classes among o he s, can be dis inguished wi h highe accu acy. Thanks o his la ge
amoun o a ailable in o ma ion, he use o hype spec al emo e sensing images has been ex ended
o a mul i ude o applica ions such as ege a ion science [
1
], land use classi ica ion [
2
], geology [
3
],
quali y con ol [4], and change de ec ion [5] among o he s.
A p e ious undamen al ask in many o hese applica ions is he egis a ion o images o he
same scene which ha e been aken a di e en imes om di e en iewpoin s and which, u he mo e,
p esen changes in objec s, in illumina ion, e c. The goal o he egis a ion is o de e mine he geome ic
ans o ma ion ha aligns he images. The e a e di e en ypes o egis a ion algo i hms due o
he a ie y o images and he condi ions ha he me hod has o handle. They can be classi ied
in o wo ca ego ies acco ding o hei na u e [
6
]: a ea–based and ea u e–based me hods. In he
i s g oup, he me hods based on co ela ion be ween images [
7
], mu ual in o ma ion (MI) [
8
],
and Fou ie ans o m [
9
] s and ou . These me hods wo k di ec ly wi h image in ensi y unlike
ea u e–based me hods which seek o de ec dis inc i e ea u es in objec s o in e es poin s a a
Remo e Sens. 2018,10, 756; doi:10.3390/ s10050756 www.mdpi.com/jou nal/ emo esensing
Remo e Sens. 2018,10, 756 2 o 26
highe le el. Fea u e–based me hods ex ac signi ican egions, lines o poin s ha mus ha e ce ain
cha ac e is ics: in a ian o geome ic ans o ma ions, good localiza ion accu acy, and be insensi i e o
image deg ada ion e ec s. This high le el ep esen a ion makes ea u e–based me hods mo e esilien
o illumina ion changes, in ensi y changes in oduced by noise, and changes in oduced by he use
o di e en senso s. On he o he hand, a ea–based me hods a e compu a ionally mo e e icien and
wo k be e on images ha a e no so ich in de ails, e.g., medical images.
Due o he ac ha emo e sensing images a e ich in de ails such as lines, edges, co ne s,
egions, e c., and hey no mally p esen illumina ion changes, ea u e–based me hods a e he p e e ed
app oach when dealing wi h his ype o images. The scale–in a ian ea u e
ans o m (SIFT) [10]
is
he mos popula ea u e–based algo i hm. I consis s o ou s ages: scale–space ex ema de ec ion,
keypoin localiza ion, o ien a ion assignmen , and keypoin desc ip ion. In he i s s ep, a Gaussian
scale space is c ea ed pe o ming Gaussian con olu ions and in e pola ions. The o iginal images
a e smoo hed a di e en le els using Gaussian il e s and a di e en scales. The esul is a
mul i– esolu ion py amid o he images. This way, he cha ac e is ic o in a iance o image scaling
is achie ed. A e wa ds, a di e ence o Gaussian (DoG) is ob ained o de ec he keypoin s. A poin
is conside ed keypoin i i is he local minimum o maximum compa ed o i s 26 neighbou s
(8 neighbou s a he same scale and 9 a he adjacen scales) o he DoG scale space. Once candida e
keypoin s a e ob ained, he second s ep ca ies ou a selec ion o disca d low–con as keypoin s,
and hei loca ion and scale a e accu a ely de e mined. In he hi d s ep, one o mo e consis en
o ien a ions a e assigned o each keypoin o achie e he o a ion in a iance. I is calcula ed om he
o ien a ion his og am o med om he g adien o ien a ions o poin s wi hin he egion o he keypoin .
The size o his egion depends on he scale and he g adien magni ude. Finally, in he las s ep,
a 128 alue desc ip o o each keypoin is gene a ed om a se o weigh ed his og ams o he g adien
o ien a ion compu ed on di e en egions o he no malized neighbou hood. Then, he keypoin s
o each image a e ma ched acco ding o a a io o he Euclidean dis ance be ween desc ip o s.
The ma ched keypoin s a e used as con ol poin s o eco e he image ans o ma ion in o de
o align he images.
Some app oaches we e p oposed ollowing he SIFT scheme. The Speeded Up Robus Fea u es
(SURF) [
11
] was p oposed o be compu ed much as e han SIFT. This me hod exploi s in eg al
images o build he scale space. This is an app oxima ion o he Gaussian scale space ha educes he
compu a ion ime. Rega ding he desc ip o , i is o med only by 64 dimensions and he o ien a ion
is es ima ed using he Haa wa ele . The p oblem o he echniques o build he scale space in SIFT
and SURF is ha hey do no espec he edges o he images, deg ading impo an de ails a he same
le el as noise. The e o e, we a e losing p ecision o de ec keypoin s and dis inc i eness. KAZE [
12
]
p oposes he use o a nonlinea di usion il e o build he scale space applying a selec i e blu ing.
This way, he noise is blu ed bu he de ails and edges a e p ese ed. Mo eo e , i uses he M–SURF
desc ip o [13] which is as e and handles he bounda ies be e han he o iginal SURF desc ip o .
These ea u e–based me hods a e commonly well–known due o hei e ec i eness bu hey
a e no sui able o some applica ions. The bina y desc ip o s a e an al e na i e o he SIFT and
SURF desc ip o s in cases whe e he speed–up is mo e impo an han he e ec i eness. This ype o
desc ip o s p o ide a mo e e icien ma ching and a e mo e compac o s o e hanks o hei bina y
desc ip o . The compu a ion o he Hamming dis ance is used o ma ch he keypoin s. I can be done
in a single ins uc ion, i.e., i s compu a ion is cheape han he Euclidean dis ance sa ing compu a ion
ime in la ge se s o ea u es. In he las decade, many ea u e–based me hods including a bina y
desc ip o ha e been p oposed in he li e a u e: BRIEF [
14
], ORB [
15
], LATCH [
16
], and A–KAZE [
17
],
among o he s. Many o hese also accele a e he scale space building p ocess, e.g., A–KAZE uses a
py amidal Fas Explici Di usion (FED) scheme o building i much as e .
Howe e , all hese algo i hms we e de eloped o wo k wi h g eyscale o RGB images, and no
o deal wi h he spec al in o ma ion a ailable in mul i and hype spec al images. In o de o
egis e mul ispec al images, some au ho s p oposed modi ica ions o SIFT. In his line, Re . [
18
]
Remo e Sens. 2018,10, 756 3 o 26
sugges ed adding a scale es ic ion c i e ia o emo e inco ec ma ching poin s in mul ispec al emo e
image egis a ion. In addi ion, Re . [
19
] also p oposed applying an o ien a ion es ic ion. A simila
app oach was ollowed by [
20
], who p oposed a desc ip o ec o wi h 4 o ien a ion bins ins ead o
8. These es ic ions we e also adap ed o SURF in [
21
]. Mo eo e , Re . [
22
] p esen ed an app oach
ha uses he spec al in o ma ion o neighbou ing keypoin s in addi ion o he desc ip o o disca d
alse ma ches. Howe e , hese me hods only use one band o each image o pe o m he egis a ion,
i.e., hey do no use all he a ailable spec al in o ma ion. Re . [
23
] in oduced an in e es poin
de ec o o hype spec al images ha uses p incipal componen analysis (PCA) in o de o educe
he dimensionali y. Fo each PCA componen , a scale space is buil . Re . [
24
] p esen ed a me hod
whe e he scale space is c ea ed using a nonlinea di usion equa ion aking in o accoun he spec al
in o ma ion. The keypoin localiza ion consis s in compa ing each pixel ec o o i s neighbou hood
acco ding o hei spec al signa u e. None o hese me hods is based on KAZE.
In his a icle, we p esen an au oma ic algo i hm, called HSI–KAZE, o egis e hype spec al
emo e sensing images aking in o accoun he spec al in o ma ion. I consis s o a ade–o solu ion
be ween KAZE and A–KAZE o hype spec al images (HSI), and i is based on a py amidal nonlinea
scale space and on M–SURF desc ip o s. The algo i hm is o ien ed owa ds ex eme si ua ions in
which he images a e e y di e en in e ms o scale, o a ion, and o he a ia ions. An example o he
egis a ion p oblem conside ed in his wo k can be seen in Figu e 1. Sec ion 2p esen s a e iew on
he o iginal KAZE. The p oposed algo i hm is desc ibed in Sec ion 3. In Sec ion 4, he esul s ob ained
unde di e en images and unde di e en condi ions a e discussed. Finally, some conclusions a e
gi en in Sec ion 5.
(a) (b) (c)
Figu e 1.
Example o egis a ion conside ed in his wo k: (
a
) Re e ence image (size
1096 ×715),
(b) Ta ge
image, and (
c
) Resul o he egis a ion p ocess showing he co ec ly egis e ed
supe posi ion o he e e ence and a ge egis e ed image (scale 23.0×and o a ion angle 60◦).
2. Desc ip ion o he KAZE Algo i hm
The KAZE algo i hm was p esen ed by [
12
]. I is cha ac e ized by building he scale space using a
nonlinea di usion il e , along wi h he u iliza ion o he M–SURF desc ip o s.
The me hod ollows a simila app oach o ha o SIFT, building a scale space o ganized in o
oc a es and, in u n, in o scales. The di e ence lies in he app oach o build his scale space.
KAZE eplaces he Gaussian app oach p oposed by Lowe [
10
] wi h a nonlinea di usion il e ing
ope a ion. The Gaussian app oach blu s bo h noise and de ails, losing he na u al bounda ies o
objec s, buildings, e c. The nonlinea di usion il e ing me hod allows us o ealize an adap i e blu
keeping he de ails unal e ed, as shown in Figu e 2a. Speci ically, KAZE builds he scale space using a
semi–implici scheme, one o he possible disc e iza ions o he di usion equa ions. This scheme is
based on an Addi i e Ope a o Spli ing (AOS) and is o ally s able o any s ep size.
Remo e Sens. 2018,10, 756 4 o 26
(a)
.
(b)
Figu e 2.
(
a
) Scale space used by KAZE (
b
) Py amidal scale space used by A–KAZE. Each image
ep esen s he i s suble el o he nex oc a e.
The classical nonlinea di usion o an image
X
wi h spa ial coo dina es
(x
,
y)
and ime
is
exp essed as
∂X
∂ =di (c(x,y, )· ∇X), (1)
whe e
di
and
∇
a e he di e gence and g adien ope a o s, espec i ely, and
c
is he conduc i i y
unc ion ha allows applying he di usion o he image s uc u e. Re . [
25
] p oposes ha his unc ion
be dependen on he g adien in o de o blu he inside o egions while p ese ing sha p edges, i.e.,
c(x,y, ) = g(|∇Xσ(x,y, )|), (2)
whe e unc ion
∇Xσ
is he g adien o a Gaussian smoo hed e sion o he o iginal image
X
wi h a
s anda d de ia ion σ. The conduc i i y unc ion gis chosen o p omo e wide egions,
g=1
1+|∇Xσ|2
k2
, (3)
whe e cons an
k
is he con as ac o ha con ols he le el o di usion. Following [
12
,
26
],
he disc e ized e sion o Equa ion (1) can be exp essed in ma ix– ec o no a ion as
Xi+1−Xi
τ=
m
∑
l=1
Al(Xi)Xi+1, (4)
whe e
Xi
and
Xi+1
a e he smoo hed images a he cu en and nex le el, espec i ely,
m
is he numbe
o dimensions o image
X
( wo in ou case),
τ
is he ime s ep size, and
Al
is he ma ix ha encodes he
di usi i ies o each image dimension (de i a i es along he
l
- h coo dina e axis). This way,
NL
le els
o il e ed images a e compu ed, i=0, . . . , NL−1.
As we men ioned and ollowing he SIFT app oach, his se o le els is mapped o e oc a es and
suble els (scales in he SIFT no a ion) using he oc a e index
o
and he suble el index
s
. The e o e,
in a scale space o
NL=Noc ·Nsub
il e ed images he oc a es and suble els a e mapped o hei
co esponding scales σi, as ollows
σi(o,s) = σ02o+s
S,i∈[0, . . . , NL−1],o∈[0, . . . , Noc −1],s∈[0, . . . , Nsub −1], (5)
Remo e Sens. 2018,10, 756 5 o 26
whe e
σ0
is he base scale le el. I is necessa y o no e ha KAZE does no pe o m a subsampling in
each oc a e as SIFT does. Nonlinea di usion il e ing is de ined in e ms o ime, so he scale uni s
σi
a e mapped o ime uni s i,
i=1
2σ2
i,i∈[0, . . . , NL−1]. (6)
Summa izing, o build he scale space, KAZE smoo hs he o iginal image using a Gaussian ke nel
o a s anda d de ia ion
σ0
o educe noise. Nex , he
k
pa ame e is ob ained as he 70 h pe cen ile
o he g adien his og am o his smoo hed e sion o he image. In o de o compu e
Xi+1
om
Xi
in Equa ion (4) a idiagonal linea sys em mus be sol ed which can be e icien ly done using
Thomas algo i hm.
Once he scale space has been buil , i is necessa y o de ec he keypoin s, which a e poin s wi h
ce ain cha ac e is ics (independence o posi ion, obus ness agains image ans o ma ions, and scale
independence). A pixel will be conside ed as keypoin i i is he maximum o i s neighbou hood when
he Hessian ma ix is calcula ed. The de e minan o he Hessian ma ix a he di e en
σi
scale le els
is compu ed as ollows
Xi
Hessian =σ2(XxxXyy −X2
xy), (7)
whe e
Xxx
and
Xyy
a e he second o de ho izon al and e ical de i a i es, espec i ely, and
Xxy
is he
second o de c oss de i a i e. This se o i s and second o de de i a i es a e app oxima ed by 3
×
3
Scha il e s which p o ide be e o a ion in a iance han o he popula il e s [
27
]. The maximum
is compu ed a all he scale le els excluding he i s and he las scales as well as he image bo de s.
The neighbou hood is conside ed in a 3D space conside ing he 8 neighbou s on he same scale and
he 9 neighbou s on he uppe and lowe scales. The e o e, we ha e 26 neighbou s pe pixel. To end
wi h he ea u e de ec ion s ep, he loca ion o he ex emas, keypoin s om now on, is e ined in
posi ion and scale using he local in e pola ion model p oposed o SIFT in [10].
The nex s ep o heKAZE algo i hm is heo ien a ion assignmen o each keypoin . KAZE calcula es
he possible o ien a ion h ough he de i a i es a ound each poin wi hin a adius o 6
σi
, being
σi
he
scale o he keypoin . Simila o SURF, each de i a i e in he ci cula a ea is weigh ed wi h a Gaussian
cen ed a he keypoin . Fo each poin , he dominan o ien a ion is calcula ed by summing he
de i a i es wi hin a sliding ci cle window co e ing an angle o
π/
3. The sums a e ep esen ed as
ec o s. The dominan o ien a ion is selec ed om he longes ec o . To sa e compu a ion ime,
hese de i a i es a e eused om he loca ion e inemen in he p e ious s ep.
Wi h he main o ien a ion es ima ed, he nex s ep is he desc ip o cons uc ion. KAZE uses
he M–SURF desc ip o [
13
] adap ed o i s nonlinea scale space. Fi s ly, o each keypoin , he i s
de i a i es,
Xx
and
Xy
, a e compu ed o e a egion o size 24
σi×
24
σi
. This egion is spli in o 4
×
4
sub egions o size 9
σi×
9
σi
wi h an o e lap o 2
σi
among hem in o de o a oid he bounda y e ec s.
Each sub egion is weigh ed using a Gaussian (
σ=
2.5
σi
) cen ed in he sub egion cen e and he
de i a i e esponses a e summed up in o a desc ip o ec o . Nex , each sub egion ec o is weigh ed
using ano he Gaussian (
σ=
1.5
σi
) de ined o e a mask o size 4
×
4 and cen ed on he keypoin .
Bo h, he samples o he 24
σi×
24
σi
egion and he compu a ion o de i a i es, a e aken in o accoun o
de e mine he main o ien a ion o he keypoin . Finally, o achie e in a iance o con as , he desc ip o
ec o is con e ed in o an uni ec o .
An accele a ed me hod called Accele a ed–KAZE (A–KAZE) was p esen ed in [
17
]. A–KAZE
is based on a py amidal nonlinea scale space and on bina y desc ip o s ha makes i mo e e icien
in e ms o compu a ion a he cos o accu acy loss. Figu e 2b illus a es his scale space. Mo eo e ,
he scale space is buil using a Fas Explici Di usion (FED) embedded in he py amidal app oach
which is as e han any o he disc e iza ion scheme [
28
], e.g., he AOS scheme. A–KAZE p oposes an
imp o emen o he Local Di e ence Bina y (LDB) [
29
] called he Modi ied–LDB (M–LDB) ha uses
bina y es s be ween he a e age o a eas ins ead o single pixels, and adds he mean o he de i a i es
o he bi s o he compa ison.
Remo e Sens. 2018,10, 756 6 o 26
3. HSI–KAZE: KAZE o Hype spec al Remo e Sensing Images
Hype spec al emo e sensing images a e usually ich in edges, co ne s, bounda ies, i.e., a la ge
se o keypoin s is ob ained. In addi ion, hey no mally ha e a lo o epea ed s uc u es e.g., buildings,
oads, ields, e c. These wo condi ions make ha de egis a ion, as i will need mo e compu a ion
ime o he keypoin ma ching p ocess, and se e al keypoin s will p obably be e y simila , making i
di icul o ob ain co ec ma ches. In his sec ion, we p esen an au oma ic me hod o egis e wo
hype spec al emo e sensing images called HSI–KAZE ha o e comes hese limi a ions. I is based on
KAZE and e icien ly uses he spec al in o ma ion a ailable in he hype spec al images. The spec al
in o ma ion is conside ed by sea ching o ea u es in pai s o selec ed bands and by adding a spec al
pa o he desc ip o . The main con ibu ions o he me hod a e he ollowing:
•
Band selec ion. A ea u e ex ac ion me hod is p oposed o selec a se o mos ep esen a i e
bands acco ding o hei en opy and hei spec al dis ance.
•
Keypoin de ec ion. An in e pola ion is applied o he o iginal images in o de o highligh de ails
and ex ac a la ge numbe o keypoin s. The scale space is buil using a nonlinea di usion il e
o blu noise and p ese e de ails such as edges. In addi ion, i ollows a py amid scheme ha
imp o es he obus ness o scale in a iance. The disc e iza ion is ca ied ou using a FED scheme.
•
Keypoin desc ip ion. HSI–KAZE uses a desc ip o o med by a spa ial and a spec al pa :
he M–SURF desc ip o and he spec al signa u e.
•
Keypoin ma ching using spec al in o ma ion. The ma ching is based on he dis ance be ween
he M–SURF desc ip o s and he cosine simila i y be ween he spec al signa u es. The spec al
in o ma ion allows e ining he ma ching p ocess disca ding ou lie s.
•
Band combina ion. Each selec ed band has unique cha ac e is ics ha a e no p esen in he o he
bands. The ma ched keypoin s o he di e en bands a e conside ed oge he in o de o achie e
a mo e accu a e egis a ion and use all he image in o ma ion.
•
Exhaus i e sea ch o egis a ion. All he possible pai s o ma ched keypoin s a e conside ed and
he ou lie s a e disca ded using a his og am–based app oach.
Figu es 3and 4show he ou line o he p oposed algo i hm. The i s s age pe o ms a band
selec ion in o de o keep only he ele an spec al in o ma ion and educe he dimensionali y
o he hype spec al images. In he second and hi d s ages, keypoin de ec ion and keypoin
desc ip ion, he ea u es o each band a e ex ac ed and desc ibed. The ou h s age, keypoin ma ching,
pe o ms he ma ching o he ex ac ed keypoin s o bo h hype spec al images. Then, in he i h
s age, band combina ion, all he ma ched keypoin s a e joined. Finally, in he las s age, egis a ion,
an exhaus i e sea ch based on his og ams is pe o med o egis e he images. The pseudocode o he
algo i hm is p esen ed in Figu e 5. The main s ages o he me hod a e explained in he nex subsec ions.
Re e ence image
Ta ge image
Band selec ion
S age I
Keypoin de ec ion
S age II
Keypoin desc ip ion
S age III
Band
combina ion
S age V
Regis a ion
S age VI
Spa ial Spec al
Spa ial Spec al
Keypoin
ma ching
S age IV
Figu e 3. HSI–KAZE algo i hm o egis e wo hype spec al images.
Remo e Sens. 2018,10, 756 7 o 26
Figu e 4. Flow cha o he p oposed HSI–KAZE algo i hm o egis e wo hype spec al images.
HSI–KAZE
Inpu : Hype spec al e e ence image I1and hype spec al a ge image I2wi h NTbands.
Ou pu : Scale ac o ρ, o a ion angle θ, and ansla ion (x,y).
1: Pe o m ea u e selec ion o e bo h images →B1and B2.Band selec ion
2: o each band bin images B1and B2do
3: Ex ac keypoin s o Bb
1→Pb
1.Keypoin de ec ion
4: Ex ac keypoin s o Bb
2→Pb
2.Keypoin de ec ion
5: Calcula e he M–SURF desc ip o o each keypoin in Pb
1.Keypoin desc ip ion
and append he spec al signa u e →Kb
1
6: Calcula e he M–SURF desc ip o o each keypoin in Pb
2.Keypoin desc ip ion
and append he spec al signa u e →Kb
2
7: Ma ch keypoin s in Kb
1and Kb
2→Mb.Keypoin ma ching
8: end o
9: Combine all he ma ched keypoin s Mb→M.Band combina ion
10: Pe o m an exhaus i e sea ch o eco e he egis a ion .Regis a ion
pa ame e s →ρ,θ,(x,y)
Figu e 5. HSI–KAZE pseudocode.
Remo e Sens. 2018,10, 756 8 o 26
3.1. Band Selec ion
The edundancy o in o ma ion among bands in hype spec al images is a well known p oblem.
The di e en me hods p oposed in he li e a u e can be classi ied in o wo g oups: ea u e ex ac ion
and ea u e selec ion [30].
The i s g oup, ea u e ex ac ion me hods, consis s in combining he di e en bands o he
image in o a small se o new ea u es. Some examples a e P incipal Componen Analysis (PCA),
Independen Componen Analysis (ICA), o wa ele ans o ms, among o he s. These me hods
p esen he d awbacks o pe o ming di e en ans o ma ions o each image and no p ese ing he
o iginal spec al in o ma ion, making he ea u e–based egis a ion me hods a e unable o iden i y he
same spa ial s uc u es p esen in bo h images. This e ec has a highe in luence when bo h images
p esen a la ge di e ence in scale, small o e lapping a eas, o changes in spa ial s uc u es due o he
ime di e ence.
On he o he hand, he second g oup, ea u e selec ion me hods, does no modi y he o iginal
da a and a e essen ially limi ed o choosing a band o a se o dis inc i e bands. The e o e, i is be e
o pe o m ea u e selec ion ins ead o ea u e ex ac ion. Many me hods choose he bands ollowing a
band selec ion c i e ion such as en opy, mu ual in o ma ion and he spec al angle mappe (SAM)
among o he s. Some o he s apply a clus e ing me hod o g oup simila bands acco ding di e en
co ela ion measu es.
In his wo k, we p opose a ea u e selec ion me hod based on en opy and in e –band dis ance
which is called En opy–based Band Selec ion (EBS). A s udy o di e en ea u e educ ion me hods
acco ding o hei e ec i eness in o de o p oduce a success ully egis a ion was ca ied ou .
In he ea u e ex ac ion ca ego y, PCA [
31
] and BandClus [
32
] a e e alua ed, and in he ea u e
selec ion one, Wa d’s Linkage s a egy using Mu ual In o ma ion (WaluMI) [
33
] and ou p oposal
(EBS) a e conside ed.
PCA is a well known s a is ical me hod o educe he dimensionali y in di e en ypes o p oblems.
The main idea is o elimina e da a edundancy p esen in he bands by means o a high co ela ion
analysis. PCA gene a es a new se o linea ly unco ela ed a iables whe e he i s ew e ain mos o
he a ia ion p esen in all o iginal a iables.
BandClus ollows a di e en app oach since i is based on unsupe ised clus e ing. I consis s
in spli ing he ini ial se o bands in o disjoin clus e s acco ding o a mu ual in o ma ion c i e ion.
Each i e a ion spli s he o iginal se o bands in o wo clus e s. The me hod s ops when he minimum
o he c i e ion is ound. A he end, he a e age o each clus e is compu ed o ob ain he inal
educed image.
On he o he hand, WaluMI pe o ms an agglome a i e clus e ing s a egy based on Wa d’s
linkage me hod [
34
]. A he beginning, each band o ms a single clus e . Nex , he algo i hm sea ches
o he wo clus e s wi h he minimum dissimila i y di e ence. In o de o do ha , he dissimila i y
ma ix is buil based on he mu ual in o ma ion be ween each pai o bands. In he nex s eps,
his ma ix is upda ed using Wa d’s linkage s a egy. Mo eo e , being ea u e selec ion me hod,
WaluMI chooses he mos ep esen a i e band o each clus e a he end. These ep esen a i e bands
de ine he inal comp essed ep esen a ion o he image.
The p oposed me hod in his pape , EBS, pe o ms a band selec ion based on en opy and
in e –band dis ance. In con as o he o he me hods, EBS uses bo h hype spec al images. The me hod
is de ailed in he pseudocode shown in Figu e 6. Fi s , he en opy o each band o each image is
compu ed (Figu e 6, lines 2–3). Nex , he minimum en opy o each band be ween he wo images is
selec ed (Figu e 6, line 4). Finally, he
Nb
bands o highes en opy wi h an in e –band dis ance g ea e
o equal han
DB
be ween consecu i e pai s a e selec ed (Figu e 6, lines 6–19). Speci ically, his check
is pe o med in line 11, whe e he candida e o be he nex selec ed band (wi h index
indexE[i]
) mus
ha e, a leas , an in e –band dis ance o
DB
wi h espec o he p e ious selec ed band (wi h index
indexB[b0]
). I i is no possible o selec a se o bands wi h an in e –band dis ance
DB
,
D
is dec eased
Remo e Sens. 2018,10, 756 9 o 26
un il i becomes possible (Figu e 6, line 17).
Nb
is ixed o 8 and he band sepa a ion
DB
o 20 a e
expe imen al analysis.
En opy–based Band Selec ion
Inpu : Hype spec al e e ence image I1and hype spec al a ge image I2wi h NTbands.
Ou pu : Se o selec ed bands B1and B2
Pa ame e s: Numbe o selec ed bands NB, minimum in e –band dis ance DB.
1: o each band bin images I1and I2do
2: e1←En opy o band bo I1
3: e2←En opy o band bo I2
4: E[b]←min(e1,e2)
5: end o
6:
So he elemen s o
E
in descending o de , le
indexE[i]
be he o iginal posi ion o he band be o e
he so and indexB[b0] he posi ion o he inal selec ed bands in he hype spec al image.
7: b0←0, D←DB
8: while b0<NBdo
9: indexB[0]←indexE[0]
10: o i←1, NT−1do
11: i abs(indexE[i]−indexB[b0]) ≥D hen
12: b0←b0+1
13: indexB[b0]←indexE[i]
14: end i
15: i b0=NB−1 hen b eak
16: end o
17: D←D−1, b0←0
18: end while
19: Ex ac he selec ed bands o I1and I2 om he indexes indexB[b0]→B1and B2
Figu e 6. Pseudocode o he band selec ion s age o he HSI–KAZE.
3.2. Keypoin De ec ion and Desc ip ion
In o de o de ec a highe numbe o ea u es and use he spec al in o ma ion, he p oposed
HSI–KAZE p esen s some modi ica ions in he ea u e ex ac o and in he desc ip o as compa ed
wi h he o iginal KAZE.
The pseudocode o hese wo s ages is shown in Figu e 7. The pa ame e
Nsub
is ixed o 4 as in
he o iginal KAZE [12]. The op imal numbe o oc a es Noc is calcula ed as ollows [35]
Noc =min 8, $log2min(w,h)
Dsub +1%!+1, (8)
whe e
Dsub
is he dis ance o ini ial upsampling applied o he image, in ou case 2 which co esponds
o a 2
×
in e pola ion, and
w
and
h
a e he wid h and heigh o he in e pola ed image, espec i ely.
Fi s , he upsampling o he o iginal image (Figu e 7, line 4) is ca ied ou using bilinea in e pola ion.
I minimizes aliasing a e ac s and allows ex ac ing a highe numbe o keypoin s. Second, he image
da a is used wi hou no maliza ion o a oid loss o p ecision. Thi d, ega ding he scale space, i is
buil ollowing a py amidal scheme as in SIFT (Figu e 7, lines 5–18), i.e., he image is subsampled o
each oc a e using a bilinea in e pola ion. Mo eo e , HSI–KAZE uses FED o he disc e iza ion o he
di usion equa ions such as in A–KAZE (Figu e 7, lines 8–17). FED schemes a e cha ac e ized by hei
as e compu a ion, ease o implemen a ion, and highe accu acy han AOS app oaches.
Finally, HSI–KAZE uses a desc ip o consis ing in wo pa s: he M–SURF desc ip o as in he
o iginal KAZE and he spec al signa u e o he keypoin (Figu e 7, lines 24–29). Thus, hanks o he use
o spec al in o ma ion, a mo e obus ma ching is pe o med. This spec al in o ma ion co esponds
o he componen s o he selec ed bands which p o ide enough in o ma ion o disca d alse ma ches.
Remo e Sens. 2018,10, 756 16 o 26
PCA p o ides wo se esul s because a di e en ans o ma ion is applied o he e e ence and he
a ge images, leading o a smalle numbe o common keypoin s p esen in bo h images, o e en none,
being eco e ed.
Table 3.
Success ully egis e ed cases o each scene using HSI–KAZE wi h di e en ea u e educ ion
me hods. The numbe in pa en heses summa izes he numbe o scales ha we e co ec ly egis e ed
o all angles. I an angle is inco ec ly egis e ed, he whole scale ac o is conside ed inco ec ,
i.e., his case
is no included in he able. The pe cen age alues a e calcula ed o e 65, he numbe o
scales conside ed.
Scene PCA BandClus WaLuMI EBS
Pa ia Uni e si y 1/4× o 4.5 ×(26)1/9× o 12.5 ×(32)1/12× o 13.0 ×(36)1/11× o 13.0 ×(35)
Pa ia Cen e 1/15× o 18.5 ×(50)1.0× o 22.5 ×(44)1/14× o 25.5 ×(63)1/16× o 24.0 ×(62)
Indian Pines 1.5× o 3.0 ×(4)1/4× o 5.5 ×(13)1/3× o 5.0 ×(11)1/4× o 5.5 ×(13)
Salinas 1/4× o 4.5 ×(11)1/7× o 6.5 ×(18)1/7× o 6.5 ×(18)1/7× o 6.0 ×(17)
Jaspe Ridge 1/4× o 4.0 ×(10)1/9× o 11.0 ×(29)1/9× o 8.5 ×(24)1/12× o 12.5 ×(35)
San a Ba ba a F on 1/4× o 3.0 ×(8)1/8× o 9.0 ×(24)1/8× o 8.5 ×(23)1/9× o 9.5 ×(26)
San a Ba ba a Box 1/2× o 4.0 ×(8)1/5× o 8.5 ×(20)1/9× o 8.5 ×(24)1/12× o 8.5 ×(27)
Numbe o scalings (a e age) (16.71) (25.71) (28.43) (30.71)
Numbe o scalings (pe cen age) 25.71% 39.56% 43.74% 47.25%
4.3. Regis a ion E ec i eness
In his sec ion, he e alua ion o he HSI–KAZE algo i hm is p esen ed. Ou p oposal is compa ed
wi h o he me hods in he li e a u e. In pa icula , i is compa ed wi h wo me hods based on he Fou ie
ans o m, he Fou ie –Mellin in a ian symme ic phase–only ma ched il e ing (FMI–SPOMF) [
39
]
and he HYpe spec al Fou ie –Mellin algo i hm (HYFM) [
9
], and on he o he hand, wi h he o iginal
KAZE [12] and A–KAZE me hods [17].
FMI–SPOMF uses he phase co ela ion and he log–pola g id o pe o m a ansla ion, o a ion,
and scale-in a ian g ey–le el image egis a ion. Howe e , HYFM was pa icula ly designed o
egis e ing hype spec al images. I exploi s he in o ma ion con ained in di e en bands and is based
on p incipal componen analysis, mul ilaye ac ional Fou ie ans o m (MLFFT), combina ion o
log-pola maps, and peak p ocessing.
As explained in Sec ion 2, KAZE is a ea u e–based me hod which uses he M–SURF desc ip o
and a nonlinea di usion il e ins ead o Gaussian il e s, as in SIFT, in o de o build he scale space
p ese ing sha p edges and smoo hing noise. In con as , A–KAZE uses a bina y desc ip o and builds
a py amidal scale space using a FED scheme which allows as e compu a ion and highe accu acy.
The bina y desc ip o also allows a as e compu a ion because he ma ching s age is compu ed using
he Hamming dis ance ins ead o he Euclidean dis ance.
The e alua ion p ocedu e is he same as he one in oduced in he p e ious sec ion. An exhaus i e
scaling ange om 1
/
16
×
o 25.5
×
, wi h 72 angles pe scale, is applied o he a ge image. Fo KAZE
and A–KAZE, he band o highes en opy is selec ed o pe o m he egis a ion. On he o he hand,
FMI–SPOMF uses he i s PCA componen o pe o m he egis a ion, while HYFM uses he i s
8 PCA componen s [9]. The andom sample consensus (RANSAC) algo i hm was used in KAZE and
A–KAZE in o de o compu e he egis a ion pa ame e s om he keypoin s and is a ailable in he
OpenCV Lib a y.
Table 4summa izes he cases ha we e co ec ly egis e ed o each scene and algo i hm.
The p oposed me hod HSI–KAZE p o ides he bes esul s on a e age, speci ically, 30.71 cases as
compa ed wi h 11.43 cases achie ed by HYFM. FMI–SPOMF, KAZE and A–KAZE do no eco e a high
numbe o cases because hey we e de eloped o wo k wi h g ey–le el images and no o deal wi h
spec al in o ma ion. The bes s esul s a e achie ed o he Pa ia Cen e and Jaspe Ridge scenes in
which a ange o 1/16× o 24.0×and 1/12× o 12.5×a e success ully egis e ed, espec i ely.
Remo e Sens. 2018,10, 756 17 o 26
Table 4.
Success ully egis e ed cases o each scene. The numbe in pa en heses summa izes he
numbe o scales ha we e co ec ly egis e ed o all angles. I an angle is inco ec ly egis e ed,
he whole scale ac o is conside ed inco ec , i.e., his case is no included in he able. KAZE and
A–KAZE a e applied o he band wi h he highes en opy. KAZE and A–KAZE use RANSAC.
The pe cen age alues a e calcula ed o e 65, he numbe o scales conside ed.
Scene FMI–SPOMF HYFM KAZE (RANSAC) A–KAZE (RANSAC) HSI–KAZE
Pa ia Uni e si y 1/5× o 4.5 ×(12)1/4× o 5.5 ×(13)1/3× o 2.5 ×(6)1/3× o 2.0 ×(5)1/11× o 13.0 ×(35)
Pa ia Cen e 1/6× o 6.0 ×(16)1/5× o 7.5 ×(18)1/4× o 5.0 ×(12)1/4× o 4.0 ×(10)1/16× o 24.0 ×(62)
Indian Pines 1/2× o 3.0 ×(6)1/2× o 4.0 ×(8)1.0× o 2.0 ×(3)1.0×(1)1/4× o 5.5 ×(13)
Salinas 1/2× o 4.0 ×(8)1/2× o 4.5 ×(9)1/2× o 2.0 ×(4)1/2× o 2.0 ×(4)1/7× o 6.0 ×(17)
Jaspe Ridge 1/3× o 2.5 ×(6)1/5× o 3.0 ×(9) ( 0) ( 0)1/12× o 12.5 ×(35)
San a Ba ba a F on 1/4× o 2.5 ×(7)1/4× o 3.5 ×(9)1.0× o 2.0 ×(3)1/2× o 2.0 ×(4)1/9× o 9.5 ×(26)
San a Ba ba a Box 1/2× o 2.0 ×(4)1/4× o 6.0 ×(14)1.0× o 1.5 ×(2)1/2× o 1.5 ×(3)1/12× o 8.5 ×(27)
Numbe o scalings (a e age) (8.43) (11.43) (4.29) (3.86) (30.71)
Numbe o scalings (pe cen age) 12.97% 17.58% 6.59% 5.93% 47.25%
To make a deepe s udy, we ha e analysed he pe o mance o KAZE and A–KAZE eplacing
he RANSAC me hod based on he andom sampling heo y by he egis a ion me hod p oposed in
HSI–KAZE (see Sec ion 3.4). Table 5shows he co ec egis e ed cases o each scene and me hod.
The a e age numbe o success ul egis e ed cases has been doubled o KAZE and A–KAZE o all
scenes, speci ically, 9.00 and 8.29 cases ha e co ec ly been egis e ed e sus 4.29 and 3.86 using he
RANSAC e sion, espec i ely (see Table 4). In pa icula , o Pa ia Cen e, anges o 1
/
7
×
o 9.0
×
and 1
/
7
×
o 8.5
×
ha e been eco e ed o KAZE and A–KAZE, espec i ely. Mo eo e , KAZE and
A–KAZE ou pe o m HYFM on Pa ia Cen e and Pa ia Uni e si y scenes bu no in he o he h ee
scenes aken a di e en da es. Using ou exhaus i e egis a ion me hod, KAZE and A–KAZE achie e
egis e ing 1 and 2 cases o Jaspe Ridge images (Table 5), espec i ely, ins ead o 0 whe e RANSAC
is used (Table 4). E en eplacing RANSAC by he p oposed egis a ion me hod in hese me hods,
HSI–KAZE achie es he bes esul s hanks o exploi a ion o he spec al in o ma ion.
Table 5.
Success ully egis e ed cases o each scene. The numbe in pa en heses summa izes he
numbe o scales ha we e co ec ly egis e ed o all angles. I an angle is inco ec ly egis e ed,
he whole scale ac o is conside ed inco ec , i.e., his case is no included in he able. KAZE and
A–KAZE a e applied o he band wi h he highes en opy. In his case, KAZE and A–KAZE use he
egis a ion me hod p oposed in HSI–KAZE. The pe cen age alues a e calcula ed o e 65, he numbe
o scales conside ed.
Scene FMI–SPOMF HYFM KAZE A–KAZE HSI–KAZE
Pa ia Uni e si y 1/5× o 4.5 ×(12)1/4× o 5.5 ×(13)1/5× o 6.0 ×(15)1/4× o 4.5 ×(11)1/11× o 13.0 ×(35)
Pa ia Cen e 1/6× o 6.0 ×(16)1/5× o 7.5 ×(18)1/7× o 9.0 ×(23)1/7× o 8.5 ×(22)1/16× o 24.0 ×(62)
Indian Pines 1/2× o 3.0 ×(6)1/2× o 4.0 ×(8)1/2× o 2.5 ×(5)1.0× o 1.5 ×(2)1/4× o 5.5 ×(13)
Salinas 1/2× o 4.0 ×(8)1/2× o 4.5 ×(9)1/4× o 3.5 ×(9)1/3× o 3.0 ×(7)1/7× o 6.0 ×(17)
Jaspe Ridge 1/3× o 2.5 ×(6)1/5× o 3.0 ×(9)1.0 ×(1)1.0× o 1.5 ×(2)1/12× o 12.5 ×(35)
San a Ba ba a F on 1/4× o 2.5 ×(7)1/4× o 3.5 ×(9)1/2× o 2.5 ×(5)1/4× o 3.0 ×(8)1/9× o 9.5 ×(26)
San a Ba ba a Box 1/2× o 2.0 ×(4)1/4× o 6.0 ×(14)1/2× o 2.5 ×(5)1/4× o 2.0 ×(6)1/12× o 8.5 ×(27)
Numbe o scalings (a e age) (8.43) (11.43) (9.00) (8.29) (30.71)
Numbe o scalings (pe cen age) 12.97% 17.58% 13.85% 12.75% 47.25%
The o al numbe o success ully egis e ed cases conside ing bo h he scale ac o s and angles
o o a ion ( om he all 65
×
72 es cases) is shown in Table 6. HSI–KAZE p esen s he bes esul s,
i co ec ly egis e s 17,153 cases, i.e., 52.36% o he es cases. These esul s a e e y simila o hose
al eady p esen ed in Table 5in which only he scales in which he 72 angles ha e been success ully
egis e ed a e included.
Remo e Sens. 2018,10, 756 18 o 26
Table 6.
Numbe and pe cen age o success ully egis e ed cases o each scene and me hod o he
en i e es ange ( om 1/16× o 25.5×, 65 scale ac o s and 72 angles pe scale).
Scene FMI–SPOMF HYFM KAZE A–KAZE HSI–KAZE
Pa ia Uni e si y 999 (21.35%) 1139 (24.34%) 1446 (30.90%) 856 (18.29%) 2781 (59.42%)
Pa ia Cen e 1268 (27.12%) 1472 (31.45%) 1859 (39.72%) 1829 (39.08%) 4647 (99.29%)
Indian Pines 586 (12.52%) 739 (15.79%) 393 ( 8.40%) 152 ( 3.25%) 954 (20.38%)
Salinas 876 (18.72%) 856 (18.29%) 907 (19.38%) 625 (13.35%) 1312 (28.03%)
Jaspe Ridge 663 (14.17%) 697 (14.89%) 148 ( 3.16%) 352 ( 7.52%) 2920 (62.39%)
San a Ba ba a F on 733 (15.66%) 859 (18.35%) 873 (18.65%) 799 (17.07%) 2404 (51.37%)
San a Ba ba a Box 508 (10.58%) 948 (20.26%) 715 (15.28%) 720 (15.38%) 2135 (45.62%)
To al success ul cases 5634 (17.20%) 6710 (20.48%) 6341 (19.36%) 5333 (16.28%) 17,153 (52.36%)
4.4. Measu es o Regis a ion Accu acy
In gene al, a small scale ac o ange is analysed in he li e a u e, e en in many cases, only one
scale. Table 7summa izes he scale ac o ange conside ed in he ecen li e a u e. I includes
me hods o egis e ing bo h panch oma ic images as well as mul ispec al ( om up o 31 bands)
and hype spec al ( om up o 242 bands) images. In ou s udy, he conside ed scale ac o ange
goes om 1
/
16
×
o 25.5
×
(65 scale ac o s) whe e o each scale 72 angles a e e alua ed, as shown in
Tables 3–5. Table 7also summa izes he mos common measu es used o e alua e egis a ion me hods
in he li e a u e: cheque boa d, numbe o ma ches, numbe o co ec ma ches, co ec ma ch a e,
oo -mean-squa e e o (RMSE) and egis a ion e o . Fo ha eason, in his sec ion, a e alua ion o
he ea u e–based me hods KAZE, A–KAZE and HSI–KAZE is p esen ed acco ding o hese measu es.
Table 7.
The mos common measu es used o e alua e he e ec i eness, accu acy and obus ness o
egis a ion me hods in he li e a u e as well as he image ype and he scale ac o ange conside ed.
Re . Image Type Scale Fac o Range Measu es
[18] Mul ispec al 0.7× o 1.0×Cheque boa d, co ec ma ch a e, egis a ion e o
[19] Mul ispec al 0.5× o 2.0×Cheque boa d, co ec ma ch a e, RMSE
[20] Mul ispec al 1.0× o 2.0×Co ec ma ch a e, numbe o ma ches
[21] Mul ispec al 1.0× o 2.0×Co ec ma ch a e, numbe o co ec ma ches, numbe o ma ches
[22] Mul ispec al 1.0×Numbe o co ec ma ches
[40] Mul ispec al and hype spec al 0.7×RMSE
[41] Hype spec al 1.0×Regis a ion e o
[42] Mul ispec al and hype spec al 1.0× o 2.0×Cheque boa d, numbe o co ec ma ches, RMSE
[24] Hype spec al 8.0×Numbe o keypoin s, co ec ma ch a e
[43] Mul ispec al 1.0×Cheque boa d, co ec ma ch a e, numbe o co ec ma ches, RMSE
[44] Mul ispec al and hype spec al 2.0×Co ec ma ch a e, numbe o co ec ma ches, RMSE
[45] Mul ispec al and panch oma ic 2.8× o 4.0×RMSE
[46] Mul ispec al 1.0×Numbe o ma ches, RMSE
[47] Mul ispec al 6.0×Cheque boa d, numbe o co ec ma ches, RMSE
[48] Mul ispec al 0.5× o 1.5×Regis a ion e o , cheque boa d
The expe imen al esul s shown in his sec ion co espond o he pai s o images o he second
da ase desc ibed in Sec ion 4.1, bu in his case we only conside he o iginal scale ac o , he angle
o o a ion and he ansla ion, i.e., no addi ional ans o ma ions will be apply o he images.
The e e ence egis a ion pa ame e s o each scene a e shown in Table 8.
Table 8. Re e ence egis a ion pa ame e s o he second g oup o he es hype spec al images.
Scene Scale Fac o Ro a ion Angle (Deg ees) T ansla ion (x,y) (Pixels)
Jaspe 0.97 −6.05 (−12,24)
San a Ba ba a F on 1.45 3.52 (2,6)
San a Ba ba a Box 1.00 0.00 (33,40)
Remo e Sens. 2018,10, 756 19 o 26
Figu e 13 shows he cheque boa d egis e ed images o he Jaspe Ridge and San a Ba ba a
scenes using HSI–KAZE. I can be seen ha he bo de s and objec s o he wo images a e co ec ly
o e lapped e en when hey we e ob ained in di e en da es.
(a) (b) (c)
Figu e 13.
Cheque boa d egis e ed images o he scenes aken by he AVIRIS senso a di e en da es:
(a) Jaspe Ridge, (b) San a Ba ba a Box, and (c) San a Ba ba a F on .
Al hough he numbe o keypoin ma ches ound by a me hod does no in luence he quali y
o he egis a ion, i is in e es ing o compa e he alues ob ained by he di e en me hods in o de
o be e explain he egis a ion de ails. Table 9compa es he numbe o keypoin ma ches o each
scene using he me hods KAZE, A–KAZE and HSI–KAZE. I is impo an o no e ha HSI–KAZE
disca ds alse ma ches hanks o he use o he spec al signa u e as pa o he keypoin desc ip o
(see Sec ion 3.3), as can be seen in he second ow o each scene. In he hi d ow, i can be seen ha
he elimina ion o epea ed ma ches pe o med by HSI–KAZE leads o e en a highe educ ion o
he inal numbe o used ma ches. To calcula e he numbe o co ec ma ches shown in his sec ion,
he Euclidean dis ance be ween he keypoin o he e e ence image and he keypoin o he a ge
image a e applying he o iginal ans o ma ion is used as e o measu e. A ma ch is conside ed
inco ec i he e o is highe han 2 pixels.
Mo e de ailed esul s o he egis a ion p ocess a e shown in Table 10 in which he numbe
o ma ches ac ually used o egis e he images is p esen ed. Fo HSI–KAZE, hese ma ches ha e
been ob ained using he exhaus i e esea ch me hod explained in Sec ion 3.4 which calcula es
he egis a ion pa ame e s aken in o accoun all he possible combina ions be ween he ma ched
keypoin s. Then, he his og am o he ob ained angles o o a ion o all hese ma ches is compu ed
and he egis a ion pa ame e s o he bin wi h he maximum alue a e selec ed. The inal egis a ion
pa ame e s a e ob ained om he median o he scales o he pa ame e s o his bin. I is impo an
o no e ha he ma ches used o egis e ing he images a e en i ely co ec in he case o HSI–KAZE
as can be seen in he hi d ow o he able, i.e., he co ec ma ch a io is 100%. This co ec ma ch
a io s ands a a ound 83% on a e age in he case o KAZE and A–KAZE wi h RANSAC used o il e
inco ec ma ches.
The RMSE and he egis a ion e o a e measu es equen ly used in he li e a u e o e alua e
egis a ion algo i hms as seen in Table 7. The egis a ion e o is measu ed in pixels and is compu ed
as he a e age Euclidean dis ance be ween he keypoin s used in he egis a ion o he e e ence
image and he keypoin s used in he egis a ion o he a ge image a e applying he e e ence
ans o ma ion (see Table 8). The esul s ob ained o KAZE, A–KAZE and HSI–KAZE a e e y simila
Remo e Sens. 2018,10, 756 20 o 26
as shown in Table 11. Howe e , i is impo an o poin ou ha HSI–KAZE allows egis e ing images
o a ange o scales much highe han ha he ob ained by KAZE and A–KAZE, as shown in Sec ion 4.3
in Tables 4and 5.
Table 9.
Compa isons o KAZE, A–KAZE and he p oposed me hod HSI–KAZE ega ding he o iginal
numbe o ma ches ob ained o each scene.
KAZE A–KAZE HSI–KAZE
Jaspe
Numbe o ma ches 17 23 21,437
Numbe o ma ches a e spec al disca ding - - 21,105
Numbe o ma ches a e emo ing epea ed ma ches - - 20,860
Numbe o co ec ma ches 13 18 20,535
San a Ba ba a F on
Numbe o ma ches 207 328 44,550
Numbe o ma ches a e spec al disca ding - - 44,490
Numbe o ma ches a e emo ing epea ed ma ches - - 43,537
Numbe o co ec ma ches 176 282 20,741
San a Ba ba a Box
Numbe o ma ches 307 230 37,185
Numbe o ma ches a e spec al disca ding - - 37,001
Numbe o ma ches a e emo ing epea ed ma ches - - 36,314
Numbe o co ec ma ches 234 171 18,502
Table 10.
Compa isons o KAZE, A–KAZE and he p oposed me hod HSI–KAZE ega ding he ma ches
used o egis e he images.
KAZE A–KAZE HSI–KAZE
Jaspe
Numbe o ma ches used in egis a ion 17 22 4
Numbe o co ec ma ches used in egis a ion 13 18 4
Co ec ma ch a io 0.76 0.82 1.00
San a Ba ba a F on
Numbe o ma ches used in egis a ion 185 262 4
Numbe o co ec ma ches used in egis a ion 165 234 4
Co ec ma ch a io 0.89 0.89 1.00
San a Ba ba a Box
Numbe o ma ches used in egis a ion 241 221 4
Numbe o co ec ma ches used in egis a ion 204 170 4
Co ec ma ch a io 0.85 0.77 1.00
Table 11. Resul s in e ms o RMSE and egis a ion e o o KAZE, A–KAZE and HSI–KAZE.
KAZE A–KAZE HSI–KAZE
Jaspe RMSE 0.98 1.54 1.35
Regis a ion e o (pixels) 0.92 1.38 1.23
San a Ba ba a F on RMSE 1.33 1.37 1.44
Regis a ion e o (pixels) 1.16 1.18 1.42
San a Ba ba a Box RMSE 1.48 1.68 0.72
Regis a ion e o (pixels) 1.28 1.44 0.70
Figu es 14 and 15 show all he ma ches ob ained o he eigh selec ed bands by HSI–KAZE o
San a Ba ba a F on and Jaspe scenes. The ma ches which we e disca ded by he spec al in o ma ion
a e in b own colou , he co ec ma ches used in egis a ion a e g een, he emainde o co ec ma ches
a e yellow, and he inco ec ma ches a e blue. In hese igu es i is easy o no e ha he use o di e en
pai o bands allows de ec ing di e en keypoin s due o he di e en in o ma ion con ained in each
one. The inal ma ches used o egis e hese scenes a e ob ained om di e en bands, bands 126 and
146 o San a Ba ba a F on , and bands 21, 135 and 207 o Jaspe Ridge. Mo eo e , a high numbe o
alse ma ches a e disca ded hanks o he use o he spec al signa u e, as can be seen, o example,
in Figu e 14 .
Remo e Sens. 2018,10, 756 21 o 26
(a) (b)
(c) (d)
(e) ( )
(g) (h)
Figu e 14.
Ma ched keypoin s de ec ed in he eigh selec ed bands o de ed by dec easing en opy
om San a Ba ba a F on scene wi h scale 9.5
×
: (
a
) Band 46, (
b
) Band 26, (
c
) Band 71, (
d
) Band 126,
(
e
) Band 93, (
) Band 6, (
g
) Band 185, and (
h
) Band 146. Ma ches disca ded a e conside ing spec al
in o ma ion (b own), inco ec ma ches (blue), co ec ma ches (yellow), and co ec ma ches used in
egis a ion (g een).
Remo e Sens. 2018,10, 756 22 o 26
(a) (b)
(c) (d)
(e) ( )
(g) (h)
Figu e 15.
Ma ched keypoin s de ec ed in he eigh selec ed bands o de ed by dec easing en opy
om Jaspe Ridge scene wi h scale 9.0
×
: (
a
) Band 46, (
b
) Band 71, (
c
) Band 21, (
d
) Band 94,
(
e
) Band 135, (
) Band 187, (
g
) Band 207, and (
h
) Band 1. Ma ches disca ded a e conside ing spec al
in o ma ion (b own), inco ec ma ches (blue), co ec ma ches (yellow), and co ec ma ches used in
egis a ion (g een).
Remo e Sens. 2018,10, 756 23 o 26
4.5. Limi a ions o he Me hod
HSI–KAZE has an uppe limi in he scales ha can egis e success ully. Ne e heless, his limi
is highe han he p e iously p oposed me hods (see Table 7) hanks o he selec ion o di e en bands,
he use o spec al in o ma ion in he desc ip o , and he p oposed exhaus i e egis a ion me hod.
An example o he ex eme ange o scales s udied is shown in Figu e 1. The highe scale ac o ,
he ha de egis a ion due o he lack o o e lapping be ween he images.
Table 12 shows he compu a ional cos (execu ion imes and memo y equi emen s expe imen ally
ob ained) in CPU o each scene and me hod conside ing he las scale success ully egis e ed in Table 5.
Rega ding compu a ional ime, HSI–KAZE is mo e cos ly han he o he me hods. The eason is he
exploi a ion o he spec al in o ma ion and he exhaus i e sea ch o eco e he egis a ion pa ame e s
conside ing all he possible pai s o ma ches ha in ol es highe compu a ional cos s. This cos is
educed by disca ding alse and epea ed keypoin ma ches, as i was explained in Sec ion 3.3. I is
impo an o no e ha FMI–SPOMF, KAZE and A–KAZE we e de eloped o egis e g eyscale o
RGB images and no hype spec al ones, so hey pe o m he egis a ion by using only a pai o
bands o he images. HYFM uses di e en pai s o bands as HSI-KAZE, so hei execu ion imes a e
simila . These compu a ion imes can be addi ionally dec eased by p ojec ing HSI–KAZE on o HPC
in as uc u es o on o commodi y de ices such as gene al pu pose GPU (GPGPU).
Rega ding he memo y equi emen s, HYFM needs an a e age o 233.49 MiB. The highes peak o
memo y is achie ed du ing he keypoin de ec ion and desc ip ion s ages due o he building o scale
space, and du ing he keypoin ma ching s age whe e all he possible pai s o ma ches a e conside ed.
KAZE is he me hod ha equi es he highes amoun o memo y. On he o he hand, FMI–SPOMF
and HYFM p esen simila memo y equi emen s due o he highes memo y equi emen is in he
compu a ion o he p incipal componen s (PCA). I is a common s age in bo h me hods o which he
whole hype spec al image mus be in memo y.
Table 12.
Execu ion imes (in seconds) and memo y equi emen s (in MiB) conside ing he las scale
success ully egis e ed in Table 5 o each scene and me hod in CPU.
Scene FMI–SPOMF HYFM KAZE A–KAZE HSI–KAZE A e age
Pa ia Uni e si y 30.64 s 123.27 s 1.53 s 1.11 s 67.38 s 44.79 s
30.70 MiB 37.95 MiB 238.26 MiB 169.49 MiB 121.40 MiB 119.56 MiB
Pa ia Cen e 136.99 s 507.59 s 6.56 s 6.72 s 454.98 s 222.57 s
114.83 MiB 151.78 MiB 547.93 MiB 276.16 MiB 482.22 MiB 314.58 MiB
Indian Pines 1.65 s 6.59 s 0.19 s 0.10 s 4.72 s 2.65 s
6.83 MiB 6.83 MiB 140.78 MiB 130.05 MiB 19.93 MiB 60.88 MiB
Salinas 6.89 s 28.79 s 0.94 s 0.57 s 22.71 s 11.98 s
32.62 MiB 32.62 MiB 188.17 MiB 149.40 MiB 40.57 MiB 88.67 MiB
Jaspe Ridge 140.17 s 508.71 s 11.04 s 12.80 s 628.25 s 260.19 s
242.83 MiB 242.83 MiB 541.02 MiB 272.57 MiB 482.24 MiB 356.30 MiB
San a Ba ba a F on 33.66 s 120.87 s 4.61 s 4.50 s 226.62 s 78.05 s
170.50 MiB 184.50 MiB 147.02 MiB 209.09 MiB 163.27 MiB 174.88 MiB
San a Ba ba a Box 32.47 s 121.93 s 10.29 s 11.00 s 616.88 s 158.51 s
252.87 MiB 252.87 MiB 556.77 MiB 278.06 MiB 324.79 MiB 333.07 MiB
A e age 54.64 s 202.54 s 5.02 s 5.26 s 288.79 s 111.25 s
121.60 MiB 129.91 MiB 337.14 MiB 212.12 MiB 233.49 MiB 206.85 MiB
I is impo an o no e ha HSI–KAZE is simila o he o he me hods in e ms o RMSE, as can be
seen in Table 11 and discussed in Sec ion 4.4, in spi e o he wide ange o scales ha HSI–KAZE can
eco e ed. The RMSE is only a measu e o p ecision in he cases co ec ly egis e ed.
Remo e Sens. 2018,10, 756 24 o 26
Finally, HSI–KAZE could ail in he de ailed alignmen o he images. I sea ches o a consensus
global a ine ans o ma ion o he whole image. Due o images usually p esen di e en dis o ions
inconsis en ly, i will be necessa y o apply a ine g ained egis a ion in a second s age in o de o
co ec hese dis o ions wi h a highe le el o de ail and dec ease he RMSE and he egis a ion e o .
5. Conclusions
In his wo k, a ea u e–based me hod o egis e ing hype spec al emo e sensing images called
HSI–KAZE is p oposed. I has been designed o egis e images wi h la ge scale ac o s and any angle o
o a ion and ansla ion. HSI–KAZE is based on KAZE bu imp o ing he egis a ion e ec i eness by
exploi ing he spec al in o ma ion a ailable in he images. The spec al in o ma ion is conside ed when
a se o ep esen a i e bands o he image a e selec ed based on hei en opy and spec al dis ance.
Each keypoin desc ip o also inco po a es spec al in o ma ion because i consis s o a M–SURF
desc ip o and a spec al signa u e. A combina ion s age conside s oge he all he keypoin s o he
selec ed bands. HSI–KAZE ca ies ou an exhaus i e sea ch o egis a ion using a his og am-based
app oach whe e all pai s o ma ched keypoin a e aking in o accoun .
We ha e pe o med se e al expe imen s using ou well–known hype spec al images in he
emo e sensing a ea, and h ee pai s o hype spec al images aken by he AVIRIS senso on di e en
da es. In addi ion o he changes ha he images al eady p esen , we ha e applied addi ional simila i y
ans o ma ions o ex end ou s udy. Mo eo e , we ha e e alua ed di e en band selec ion me hods
p esen in he li e a u e ega ding hei e ec i eness in e ms o egis a ion obus ness.
The p oposed me hod has been compa ed wi h he KAZE, A–KAZE, FMI–SPOMF, and HYFM
egis a ion me hods in e ms o success ul egis a ions, numbe o ma ches, numbe o co ec
ma ches, co ec ma ch a io, RMSE and egis a ion e o . The esul s show ha he highes numbe
o co ec ly eco e ed scales o all angles a e ob ained by HSI–KAZE. Fo example, a scale ac o o
24.0
×
has been eco e ed in Pa ia Cen e by he HSI–KAZE me hod, while o he KAZE me hod he
maximum achie ed scale is 9.0×.
As u u e wo k, HSI–KAZE can be used as a i s s age o a ine g ained mul ile el egis a ion
me hod. HSI–KAZE would pe o m a high–le el egis a ion ha would be e ined in a second
egis a ion s age o co ec geome ic de o ma ions a a lowe le el.
Au ho Con ibu ions:
F.A. p o ided he esea ch idea; Á.O. and F.A. de eloped he me hod; D.B.H. and F.A.
p o ided guidance h oughou he esea ch p ocess; Á.O. designed and pe o med he expe imen s; all he au ho s
join ly w o e he manusc ip .
Funding:
This esea ch was suppo ed in pa by he Conselle ía de Cul u a, Educación e O denación Uni e si a ia,
Xun a de Galicia [g an numbe s GRC2014/008 and ED431G/08] and Minis e io de Educación, Cul u a y Depo e
[g an numbe TIN2016-76373-P] bo h a e co– unded by he Eu opean Regional De elopmen Fund. The wo k o
Ál a o O dóñez was suppo ed by he Minis e io de Educación, Cul u a y Depo e unde an FPU G an [g an
numbe FPU16/03537]. This wo k was also pa ially suppo ed by Conseje ía de Educación, Jun a de Cas illa y
León (PROPHET P ojec ) [g an numbe VA082P17].
Acknowledgmen s:
The au ho s would like o hank Claude Ca iou a Uni e si é de Rennes 1, F ance,
o p o iding he band limi s o BandClus used in his wo k.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Thenkabail, P.S.; Lyon, J.G. Hype spec al Remo e Sensing o Vege a ion; CRC P ess: Boca Ra on, FL, USA, 2016.
2.
Ga ea, A.S.; He as, D.B.; A güello, F. GPU classi ica ion o emo e-sensing images using ke nel ELM and
ex ended mo phological p o iles. In . J. Remo e Sens. 2016,37, 5918–5935. [C ossRe ] [C ossRe ]
3.
K use, F.A.; Boa dman, J.W.; Hun ing on, J.F. Compa ison o ai bo ne hype spec al da a and EO-1 Hype ion
o mine al mapping. IEEE T ans. Geosci. Remo e Sens. 2003,41, 1388–1400. [C ossRe ]
4. Sun, D.W. Hype spec al Imaging o Food Quali y Analysis and Con ol; Else ie : London, UK, 2010.
Remo e Sens. 2018,10, 756 25 o 26
5.
López-Fandiño, J.; He as, D.B.; A güello, F.; Mu a, M.D. GPU F amewo k o Change De ec ion in
Mul i empo al Hype spec al Images. In P oceedings o he 10 h In e na ional Symposium on High-Le el
Pa allel P og amming and Applica ions, Valladolid, Spain, 10–11 July 2017; pp. 115–132.
6.
Zi o a, B.; Flusse , J. Image egis a ion me hods: A su ey. Image Vis. Compu .
2003
,21, 977–1000. [C ossRe ]
7.
P a , W.K. Co ela ion echniques o image egis a ion. IEEE T ans. Ae osp. Elec on. Sys .
1974
,AES-10,
353–358. [C ossRe ]
8.
Wells, W.M.; Viola, P.; A sumi, H.; Nakajima, S.; Kikinis, R. Mul i-modal olume egis a ion by maximiza ion
o mu ual in o ma ion. Med. Image Anal. 1996,1, 35–51. [C ossRe ]
9.
O dóñez, Á.; A güello, F.; He as, D.B. GPU Accele a ed FFT-Based Regis a ion o Hype spec al Scenes.
IEEE J. Sel. Top. Appl. Ea h Obs. Remo e Sens. 2017,10, 4869–4878. [C ossRe ]
10.
Lowe, D.G. Dis inc i e image ea u es om scale-in a ian keypoin s. In . J. Compu . Vis.
2004
,60, 91–110.
[C ossRe ]
11.
Bay, H.; Ess, A.; Tuy elaa s, T.; Van Gool, L. Speeded-up obus ea u es (SURF). Compu . Vis. Image Unde s .
2008,110, 346–359. [C ossRe ]
12.
Alcan a illa, P.F.; Ba oli, A.; Da ison, A.J. KAZE ea u es. In P oceedings o he Eu opean Con e ence on
Compu e Vision, Flo ence, I aly, 7–13 Oc obe 2012; Sp inge : Be lin, Ge many, 2012; pp. 214–227.
13.
Ag awal, M.; Konolige, K.; Blas, M.R. Censu e: Cen e su ound ex emas o eal ime ea u e de ec ion
and ma ching. In P oceedings o he Eu opean Con e ence on Compu e Vision, Ma seille, F ance,
12–18 Oc obe 2008; Sp inge : Be lin, Ge many, 2008; pp. 102–115.
14.
Calonde , M.; Lepe i , V.; S echa, C.; Fua, P. B ie : Bina y obus independen elemen a y ea u es.
In P oceedings o he Compu e Vision–ECCV 2010, C e e, G eece, 5–11 Sep embe 2010; pp. 778–792.
15.
Rublee, E.; Rabaud, V.; Konolige, K.; B adski, G. ORB: An e icien al e na i e o SIFT o SURF.
In P oceedings o he 2011 In e na ional Con e ence on Compu e Vision, Ba celona, Spain, 6–13 No embe
2011; pp. 2564–2571.
16.
Le i, G.; Hassne , T. LATCH: Lea ned a angemen s o h ee pa ch codes. In P oceedings o he WACV 2016:
IEEE Win e Con e ence on Applica ion o Compu e Vision, Lake Placid, NY, USA, 7–9 Ma ch 2016; pp. 1–9.
17.
Alcan a illa, P.F.; Nue o, J.; Ba oli, A. Fas explici di usion o accele a ed ea u es in nonlinea scale
spaces. IEEE T ans. Pa e n Anal. Mach. In ell. 2011,34, 1281–1298.
18.
Yi, Z.; Zhiguo, C.; Yang, X. Mul i-spec al emo e image egis a ion based on SIFT. Elec on. Le .
2008
,44,
107–108. [C ossRe ]
19.
Li, Q.; Wang, G.; Liu, J.; Chen, S. Robus scale-in a ian ea u e ma ching o emo e sensing image
egis a ion. IEEE Geosci. Remo e Sens. Le . 2009,6, 287–291.
20.
Mehme , F.; Ya dimci, Y.; Temlzel, A. Regis a ion o mul ispec al sa elli e images wi h o ien a ion- es ic ed
SIFT. In P oceedings o he 2009 IEEE In e na ional Geoscience and Remo e Sensing Symposium, Cape Town,
Sou h A ica, 12–17 July 2009; Volume 3, p. III-243.
21.
Teke, M.; Temizel, A. Mul i-spec al sa elli e image egis a ion using scale- es ic ed SURF. In P oceedings
o he 20 h In e na ional Con e ence on Pa e n Recogni ion, Is anbul, Tu key, 23–26 Augus 2010;
pp. 2310–2313.
22.
Hasan, M.; Jia, X.; Robles-Kelly, A.; Zhou, J.; Picke ing, M.R. Mul i-spec al emo e sensing image egis a ion
ia spa ial ela ionship analysis on si keypoin s. In P oceedings o he 2010 IEEE In e na ional Geoscience
and Remo e Sensing Symposium, Honolulu, HI, USA, 25–30 July 2010; pp. 1011–1014.
23.
Mukhe jee, A.; Velez-Reyes, M.; Roysam, B. In e es poin s o hype spec al image da a. IEEE T ans. Geosci.
Remo e Sens. 2009,47, 748–760. [C ossRe ]
24.
Do ado-Muñoz, L.P.; Velez-Reyes, M.; Mukhe jee, A.; Roysam, B. A ec o SIFT de ec o o in e es poin
de ec ion in hype spec al image y. IEEE T ans. Geosci. Remo e Sens. 2012,50, 4521–4533. [C ossRe ]
25.
Pe ona, P.; Malik, J. Scale-space and edge de ec ion using aniso opic di usion. IEEE T ans. Pa e n Anal.
Mach. In ell. 1990,12, 629–639. [C ossRe ]
26.
Weicke , J.; Romeny, B.T.H.; Vie ge e , M.A. E icien and eliable schemes o nonlinea di usion il e ing.
IEEE T ans. Image P ocess. 1998,7, 398–410. [C ossRe ] [PubMed]
27.
Weicke , J.; Scha , H. A scheme o cohe ence-enhancing di usion il e ing wi h op imized o a ion
in a iance. J. Vis. Commun. Image Rep esen . 2002,13, 103–118. [C ossRe ]
28.
G ewenig, S.; Weicke , J.; B uhn, A. F om Box Fil e ing o Fas Explici Di usion. In DAGM-Symposium;
Sp inge : Be lin, Ge many, 2010; pp. 533–542.
