Con en -based Image Re ie al o Map Geo e e encing
Jonas Lu a∗
, Jochen Schiewe a
aLab o Geoin o ma ics and Geo isualiza ion, Ha enci y Uni e si y Hambu g, jonas.lu @hcu-hambu g.de
* Co esponding Au ho
Abs ac : In ecen yea s, lib a ies ha e made g ea p og ess in digi ising o es o his o ical maps wi h high- esolu ion
scanne s. P o iding use - iendly in o ma ion access o cul u al he i age h ough spa ial sea ch and webGIS equi es
geo e e encing o he hund eds o housands o digi ised maps.
Geo e e encing is usually done manually by inding “g ound con ol poin s”, loca ions in he digi al map image, whose
iden i y is unambiguous and can easily be ound in mode n-day e e ence geoda a/mapping da a. To decide whe he
wo symbols om di e en maps desc ibe he same objec , hei seman ic and spa ial ela ions need o be ma ched.
Au oma ing his p ocess is he only easible way o geo e e ence he immense quan i ies o maps in concei able ime.
Howe e , au oma ed solu ions o spa ial ma ching quickly ail when aced wi h incomple e da a – which is he g ea es
challenge when compa ing maps o di e en ages o scales.
These p oblems can be o e come by compu ing map simila i y in he image domain. T ea ing maps as a special case o
image p ocessing allows e icien and obus ma ching and hus iden i ica ion o geog aphical egions wi hou he need o
explici ly model seman ics. We p opose a me hod o encode wo ldwide e e ence VGI mapping da a as image ea u es,
allowing he cons uc ion o an e icien lookup index. Wi h his index, con en -based image e ie al can be used o bo h
geoloca ing a gi en map o geo e e encing wi h high accu acy. We demons a e ou app oach on hund eds o map shee s
o di e en his o ical opog aphical su ey map se ies, success ully geo e e encing mos o hem wi hin me e seconds.
Keywo ds: image analysis, au oma ic geo e e encing, his o ical maps, VGI
1. In oduc ion
Resea ch in digi al map p ocessing has picked up he pace
in ecen yea s (Chiang e al., 2014, 2020, Jiao e al., 2021),
mos ly ocusing on digi ising con en wi h ec o isa ion
and a ibu ion. P e ious esea ch o en assumes al eady
geo e e enced maps (Iosi escu e al., 2016, Hei zle and
Hu ni, 2019) o lea es he edious manual geo e e encing
ask o c owdsou cing (Flee e al., 2012, Ta akkol e al.,
2019).
Au oma ic geo e e encing can close he gap o a ully au-
oma ic map- o-geoda a pipeline. Aligning maps makes
compa ison o map con en ac oss di e en se ies and de-
signs easie (Schlegel, 2019). Compa abili y o old and
p esen -day maps, in u n, leads o many new esea ch pos-
sibili ies and unleashes he ull po en ial o au oma ed spa-
ial analyses o his o ical da a, e. g. o long- e m moni o -
ing o coas lines (Fab is, 2021), inding po en ial a chaeo-
logical si es (Whi e, 2013) and many mo e. Fu he mo e,
geo e e encing can signi ican ly inc ease he accessibili y
o he impo an cul u al he i age ha a e his o ical maps
(C om, 2016, Buckley, 2019).
Some wo k has been done on he cons ained ask o align-
ing wo maps which a e oughly co e ing he same a ea
(e. g. Howe e al., 2019, Duan e al., 2020). Howe e , o
sol e he gene al ask o geoloca ing a map wi h li le o
no p io in o ma ion on loca ion, maps ha e o be com-
pa ed agains la ge olumes o compa able e e ence geo-
da a, possibly ac oss he globe. Luckily, la ge amoun s o
e e ence da a a e a ailable in he o m o olun ee ed geo-
g aphical in o ma ion (VGI), o example OpenS ee Map
(OSM)1. The challenge is o ind a way o e icien ly sea ch
h ough i o ind he mos ele an pa . He e image e-
ie al, pa icula ly con en -based image e ie al (CBIR)
when me ada a o maps is no su icien ly a ailable, comes
in o play.
CBIR is used o ind images in a e e ence da abase ha
a e mos simila o a gi en que y image. In he con ex
o maps, his equi es a spa ial simila i y unc ion, which
a e o en sol ed wi h g aph ep esen a ions Li and Fon-
seca (2006). Howe e , g aph ep esen a ions equi e com-
plex de ini ions (Janowicz e al., 2011). Sol ing simi-
la i y g aphs algo i hmically is compu a ionally expensi e
and has no ye been p o en o p o ide unambiguous e-
sul s. Fu he mo e, ge ing om a map o seman ic ep e-
sen a ion equi es ull ec o isa ion (Szend ei e al., 2011,
Iosi escu e al., 2016) and a ibu ion (Sun e al., 2020)
i s , which a e in ense ields o s udy and s ill no com-
ple ely sol ed. De eloping a gene al me hod ha wo ks
o any and all maps is pa icula ly challenging. A mo e
obus and easily scalable app oach o map simila i y is
needed.
1.1 Con ibu ions
This wo k expands on ou own me hod p esen ed in Lu
and Schiewe (2021) by de eloping a mo e gene al de ini-
1h ps://www.opens ee map.o g/
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
2 o 8
ion o spa ial simila i y using obus image ea u es (Sec-
ion 2.3). This o e comes he need o ine uning o pa-
ame e s o each high accu acy and consequen ly allows
o ans e ou me hod mo e easily o new map se ies. We
a gue ha exploi ing he image domain ins ead o complex
seman ic ep esen a ions enables he de elopmen o e i-
cien and obus algo i hms o calcula ing he simila i y
o map con en . This implemen a ion o map simila i y al-
lows exploi a ion o a e e se index o as que ying in
la ge e e ence da abases (Sec ion 2.5).
Assuming a mos ly success ul segmen a ion, e ie al o
simila maps p o ides a gene al solu ion o au oma ed
geo e e encing. In se e al expe imen s in Sec ion 3, we
will add ess he ollowing challenges o geo e e encing
wi h ou p oposed CBIR app oach:
•E oneous symbols and noise in oduced by segmen-
a ion.
•Occlusion: Missing o ex a symbols due o his o i-
cal change o ill- i ing e e ence da a esul ing om
a di e en le el o gene alisa ion be ween que y and
e e ence maps.
•Ill-aligned e e ence da a, because o da um shi o
imp ecise e e ence quad angles.
•Maps ha e di e en scales.
1.2 Rela ed Wo k
Howe e al. (2019) and Ta akkol e al. (2019) ha e p o en
ha oponyms allow geoloca ing a di e se ange o maps.
Howe e , i is challenging o eliably ecognise oponyms
on old p in s wi h unusual ype. Fu he mo e, clu e ed o-
pog aphic maps show many o e lapping ea u es, which
make op ical cha ac e ecogni ion un eliable (Lu , 2020).
Bu e al. (2019) and Hei zle e al. (2018) ins ead ex-
ploi he g a icule isible on mo e mode n opog aphic
maps, which allows e y p ecise ec i ica ion and align-
men . Howe e , he g a icule is no isible on all maps
and assigning g a icule co ne s o geog aphic coo dina es
elies on ma ginal in o ma ion and p io knowledge abou
he map.
Con en -based geo e e encing can o e come he chal-
lenges o bo h o hese app oaches (Lu and Schiewe,
2021). Using con en equi es a obus amewo k o spa-
ial simila i y, especially when using only he geome y
o map symbols wi hou u he a ibu ion (which would
need o be in e ed om symbol classi ica ion o ex un-
de s anding).
A p o en me hod o ma ching image con en is he use o
image ea u es. Image ea u es a e equen ly used o au-
oma ically ex ac ie-poin s o alignmen o pho og am-
me ic images. Mo e ecen ly, he e ha e been p oposals
o c oss-modal ma ching be ween di e en imaging sys-
ems (Ye and Shen, 2016, Zhuo e al., 2017, Li e al., 2020)
o ac oss ime-se ies wi h changes in illumina ion and land
co e (Liu e al., 2008).
In con as o pho og amme y, no much esea ch has gone
in o ma ching opog aphic maps by image ea u es. The
eason may be ha image ea u es a e designed o wo k
on pho os bu can no immedia ely be applied o p in ed o
manusc ip maps. Wi hou p ep ocessing, he gene alised
con en and he e ogeneous s yle o maps usually leads o
low epea abili y o ea u e de ec ion.
2. CBIR on maps
Gene ally speaking, CBIR is he ask o inding images in
a huge da abase ha a e mos simila o a gi en que y im-
age, speci ically simila in con en . The main challenge
lies in inding a sui able de ini ion o simila i y ha closes
he “seman ic gap”, i. e. e u ns images ha a e no jus
simila in abs ac ma hema ical p ope ies bu also aligned
wi h he expec a ion o simila i y o a human use .
The concep o CBIR can be applied o geo e e encing o
a map as ollows: To ind he geog aphic a ea shown in a
map ( he que y map), we sea ch h ough a comp ehensi e
se o al eady geo e e enced maps o ind hose wi h he
mos simila con en ( e e ence maps). The disco e y o a
e e ence map wi h iden ical con en allows o use i s ge-
og aphical in o ma ion o geoloca e he que y map. Sol -
ing geo e e encing wi h his app oach equi es us o sol e
some asks, ou solu ion o which we will elabo a e s ep-
by-s ep in he ollowing pa ag aphs ( e e o Figu e 1 o
an o e iew):
•Ob aining geo e e enced e e ence maps o possible
map loca ions (Sec ion 2.2).
•De ini ion o a simila i y unc ion o map con en
(Sec ion 2.4).
•Since he e can be a lo o e e ence maps, hey need
o be que ied e icien ly (Sec ion 2.5).
2.1 P e equisi es
In o de o calcula e simila i y be ween he que y map and
e e ence maps, we need o emo e di e ences in ca o-
g aphic design and in e e ing in o ma ion. We ensu e his
by segmen ing he que y map o ex ac a single class o
symbols. Segmen a ion elimina es clu e and makes maps
compa able ega dless o hei design, bu in oduces noise
and o he e o s. The key is o segmen objec classes
which we can ma ch easonably well o he e e ence VGI
da a. Objec s wi h complex geome y a e mos sui able,
since hey ca y a lo o spa ial in o ma ion.
Segmen a ion and map symbol ex ac ion is a challenging
ask ha dese es a dedica ed s udy. Fo his pape we as-
sume a mo e o less eliable segmen a ion as a p e equisi e.
2.2 Re e ence da a
OSM is a aluable sou ce o spa ial in o ma ion, because
i s inc easing global a ailabili y in consis en o ma and i
con ains many di e en ea u e ypes in g ea de ail. The
selec ion o objec classes happens by il e ing o so called
“ ags”. Tags need o be selec ed o p oduce e e ence da a
ha is as close as possible o he symbols ha ha e been
segmen ed om he que y map.
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
3 o 8
Figu e 1. O e iew o he CBIR p ocess o geo e e encing. To geoloca e a gi en que y map, i s image ea u es a e
ex ac ed. Wi h hose image ea u es, he index is que ied, which was p e iously cons uc ed om a la ge numbe o
e e ence maps, c ea ed om OSM da a a known quad loca ions. The index e u ns a se o e e ence maps, so ed by
simila i y. The ea u es a e ma ched be ween he que y map and he e e ence maps and subsequen ly e ined by spa ial
e i ica ion. The e e ence map wi h he mos emaining ma ches is e u ned. Toge he wi h he known loca ion o he
e e ence map, we hus ob ain a loca ion p edic ion o he que y map.
OSM e e ence da a a e ec o based, bu que y maps a e
as e images o di e se s yle. The il e ed OSM ec-
o da a a e con e ed o bina y as e images and subse-
quen ly p ocessed analogously o he que y maps. Re e -
ence images a e c ea ed om quad angles o all possible
map loca ions which need o be known in ad ance. Al-
e na i ely, quad angles can be au oma ically gene a ed i
hey a e a anged in a egula g id wi h known spacing.
2.3 Image ea u es
In mode n CBIR, image con en is desc ibed by he image
ea u es ha can be ex ac ed om he image (Smeulde s e
al., 2000). Image ea u es ha e been in ensely in es iga ed
by he compu e ision communi y. Image ea u es p o ide
a solid me hodological basis o desc ip ion and ma ching
o image con en wi h obus and e icien algo i hms.
Image ea u es a e ex ac ed a in e es poin s in he image.
This leads o a educ ion o in o ma ion, ocusing on he
mos desc ip i e egions in he image. Ma ching a se o
a ew hund ed image desc ip o s is signi ican ly mo e e -
icien and obus han compa ing ull high esolu ion im-
ages.
A majo conside a ion o he choice o ea u e desc ip o s
is hei abili y o cap u e s uc u e ins ead o ex u e. S uc-
u e is e y impo an o iden i y geog aphical ea u es.
Con e sely, he e is no ex u e a all in he segmen ed maps
and e e ence maps. Fea u es should be ex ac ed p edom-
inan ly a in o ma i e poin s, such as bends o a i e as in
Figu e 2. Fu he mo e, ea u es should be scale-in a ian ,
because inpu images come in di e en esolu ions. Scale-
in a ian ea u es a e mo e likely o cap u e la ge-scale in-
e es poin s and a e less suscep ible o small i egula i ies
due o noise, a e ac s o as e isa ion o minuscule sym-
bols wi h li le in o ma ion.
Speci ically, we ex ac KAZE desc ip o s (Alcan a illa e
al., 2012), since hey showed he bes indexing esul s in
Figu e 2. KAZE in e es poin s (colou ed ci cles) de ec ed
on a KDR100 e e ence map. I is appa en , ha he KAZE
ea u e de ec o is able o de ec s uc u e- ich a eas o he
depic ed i e s. Da a © OpenS ee Map con ibu o s.
compa a i e expe imen s. Appa en ly, hey a e mo e e-
pea able in ace o he high con as and low ex u e in
segmen ed maps han e. g. he well-es ablished SIFT o
SURF desc ip o s. Fu he mo e, KAZE desc ip o s ha e
a o a ion- ee a ian , which educes ambigui y on maps
whe e he di ec ion o ea u es is ele an and we can ex-
pec oughly no h-up o ien ed maps. The downside o
KAZE desc ip o s is ha hey ake abou ou imes longe
o compu e han SURF.
2.4 Simila i y
I an image ea u e in he que y image is e y simila o an
image ea u e in a e e ence image (called a ma ch), i is
likely o be a sensible ie-poin . Thus, a e e ence image is
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
4 o 8
simila o he que y image, i a high numbe o ie poin s
could be ound.
When we compa e a se o ea u es, unce ain y is in o-
duced by inco ec ma ches. Pe haps mo e limi ing, he
spa ial ela ionship be ween image ea u es is no aken
in o accoun . Bu o geog aphical objec s, i is e y el-
e an in wha o ien a ion hei pa s a e (e. g. whe he a
i e bends o he le o o he igh a e a di ide).
The e o e, we apply a second s ep o spa ial e i ica-
ion: he desc ip o ma ches o each image pai a e il-
e ed by andom sample consensus (RANSAC, Fischle
and Bolles, 1981), signi ican ly inc easing he con idence
o he ma ches. Wi h a sui able choice o a ans o m
model, RANSAC is able o compensa e o an icipa ed de-
o ma ions (in oduced e. g. h ough di e en da um, p o-
jec ion o map ex en ). Fo he opog aphic maps in his
s udy he simila i y ans o m is su icien o compensa e
o di e en ly c opped map ma gins and sligh o a ion.
Basing simila i y on a disc e e se o ie poin s (in con as
o image-le el me ics, such as he s uc u al simila i y-
index o his og ams) makes i obus o missing map sym-
bols in ei he map (e. g. due o ca og aphic gene alisa ion).
When he e a e ew symbols on one map he e is, o cou se,
a lowe possible simila i y sco e. I has o be no ed ha
simila i y sco es do no adhe e o an absolu e scale and
canno be compa ed be ween image pai s. Howe e , he
numbe o ie-poin s can be used o o de he e e ence
maps by simila i y o a single que y image and ind he
mos simila .
2.5 E icien lookup
An index o all e e ence maps has o be cons uc ed
once o enable he e icien sea ch o simila maps in wo
s ages: index que y and spa ial e i ica ion.
2.5.1 Index cons uc ion
1. Que y VGI da abase o each o he expec ed map lo-
ca ions while il e ing he ec o da a o ele an ob-
jec classes, using O e pass2.
2. Pain bina y as e maps om ec o da a o each map
loca ion.
3. De ec in e es poin s and ex ac image desc ip o s in
he as e maps.
4. Popula e an app oxima e nea es neighbou s ee wi h
all desc ip o s, s o ing an iden i ie o he map hey
we e ex ac ed om. We implemen ed he ee wi h
Annoy3.
2.5.2 Index que y
1. F om a gi en segmen ed que y map, ex ac image
ea u es in he same way as be o e.
2. Que y he index o ge he k nea es neighbou s o
each desc ip o .
2h ps://o e pass-api.de/
3h ps://gi hub.com/spo i y/annoy
3. Each o he k desc ip o belongs o a e e ence map.
To penalise bad ma ches, o e o each o he k e e -
ence maps wi h dec easing weigh : nea es neighbou
wi h a single o e, second-nea es neighbou wi h a
hal o e, e c. The o es o each e e ence map a e
added o e e y desc ip o in he que y map.
4. So all e e ence maps by hei numbe o o es.
5. Re u n he N mos simila e e ence maps as loca ion
hypo heses.
2.5.3 Spa ial e i ica ion
1. The desc ip o s o each e e ence map ha had been
used o building he index ha e been s o ed alongside
he index o sa e compu a ion ime. Ge he desc ip-
o s o he p e iously de e mined loca ion hypo heses.
2. B u e- o ce ma ch he desc ip o s o in e es poin s in
he segmen ed que y map o ind all co espondences
in he desc ip o s o each e e ence map.
3. Use RANSAC o i a ans o m model om he que y
desc ip o s o he e e ence desc ip o s.
4. The e e ence map wi h he ans o m model ha bes
explains he ma ches and has he highes numbe o
inlie s a e RANSAC is e u ned as he loca ion p e-
dic ion o he que y map.
3. E alua ion
Fo all expe imen s, we use hyd ology ea u es ( i e s,
lakes, coas lines) because hei dis inc colou makes hem
easy o segmen o mos maps and hey ha e e y dis inc
geome y (Wol e e al., 2017). Fu he mo e, hyd ology
is expec ed o ha e li le his o ical change and can be il-
e ed easonably well wi h O e pass. O he ea u es, such
as oads o ail oad acks, a e hinkable as well and can be
used jus as easily wi h ou me hod, bu hey a e ha de o
ell apa om o he symbols on he map and hey a e ex-
pec ed o ha e changed signi ican ly since p in ing o he
in es iga ed maps.
3.1 Baseline/p oo -o -me hod
Fi s , we demons a e ha he image domain is in ac sui -
able o calcula e map con en simila i y by ma ching im-
age ea u es. We can demons a e he applicabili y o ou
me hod and p o ide a baseline o he maximal eachable
accu acy by using he a i icial black-and-whi e e e ence
map images used o build he index (wi h some padding
o simula e he map ma gins). This baseline expe imen
shows, ha he simila i y unc ion is able o ind he co -
ec maps i i is no impai ed by e oneous segmen a ion
o his o ical changes in opog aphy.
3.1.1 Da a
The que y maps o his expe imen a e he same as he
e e ence maps o building he index (see Sec ion 2.5.1
abo e) and can hus be used o de e mine baseline accu acy
(see Sec ion 3.1 below). An example o a que y map can
be seen in Figu e 3 on he le . The index is cons uc ed
om 911 shee s and hus con ains mo e shee s han he
657 maps o be que ied.
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
5 o 8
map se ies #maps mean (median) index ank maps w/ ank=0 (%) maps w/ ank<N (%) #co ec (%)
KDR100 baseline 657 0.9 (0) 656 (99.8) 656 (99.8) 656 (99.8)
KDR100 deg aded 657 23 (0) 377 (57.4) 677 (87.8) 529 (80.5)
KDR100 ac ual 657 37.4 (0) 399 (60.7) 569 (86.5) 539 (82.0)
KDR500 ac ual 27 4.7 (3) 9 (33.3) 27 (100) 15 (55.6)
USGS100 baseline 207 511.4 (0) 122 (59.5) 125 (61.0) 124 (60.5)
USGS100 ac ual 207 430.7 (52) 51 (24.6) 107 (51.7) 92 (43.6)
Table 1. Compa ison o esul s o all expe imen s. The i h column shows he maximum eachable accu acy o spa ial
e i ica ion: o he KDR100 expe imen s only he N=30 mos simila e e ence maps we e e ied. Fo USGS100, N is
100. Fo KDR500 all shee s we e e i ied.
Figu e 3. OSM baseline image (le ) and an a i icially de-
g aded image ( igh ) o he same map (KDR100 shee 29).
The occluded map s ill p o ided enough in o ma ion o be
success ully geo e e enced. Da a © OpenS ee Map con-
ibu o s.
3.1.2 Resul
When que ying he index wi h he OSM maps hemsel es,
he co ec e e ence map is e u ned as he mos simila
map o almos all maps, bo h by he index as well as by
he subsequen spa ial e i ica ion (compa e Table 1). This
is expec ed and p o es ha he s uc u e o he map sym-
bols and hei ep esen a ion as image ea u es is no am-
biguous. I a map canno be ma ched o i sel , i p obably
does no con ain any disce nible ea u es and hus can no
be localised.
3.2 Robus ness o occlusion
3.2.1 Da a
This da ase aims o simula e his o ical changes and bad
segmen a ion quali y wi h deg aded OSM e e ence maps.
500 emp y ci cles o a ying diame e ha e been pain ed
andomly o e baseline images o KDR100 quad angles
esul ing in 73% occlusion on a e age (compa e Figu e 3).
The expe imen uses he same index as o he baseline.
3.2.2 Resul
The esul ing pe o mance is s ill qui e good: 80% o maps
could be loca ed co ec ly (as can be seen in Table 1). Mos
maps wi h educed image con en could be ma ched un-
ambiguously. This indica es, ha ou me hod is e y o-
bus o occlusion, which migh occu because o his o ical
changes o opog aphy o because o incomple e segmen-
a ion.
3.3 Real-wo ld applica ion
3.3.1 Da a
This expe imen uses he ac ual map shee s o he su -
ey map se ies Ka e des Deu schen Reiches in scale
1 : 100 000 (KDR100). The maps we e made a he u n
o he 20 h cen u y and use a apezoid p ojec ion wi h he
Rauenbe g da um. 657 shee s a e aligned in egula g id.
Again, we use he same index as o he baseline. The map
a e ei he one o h ee colou p in s wi h hyd ology in blue.
Because he colou s a e almos iden ical ac oss he se ies,
we could use colou h esholding o segmen a ion. Using
a single colou ange ac oss he whole se ies, howe e , in-
oduces a lo o speckle noise and some imes imp ecise
ou lines (compa e Figu e 4).
3.3.2 Resul
When wo king wi h eal maps, he main issue a e e o s
and noise in oduced by segmen a ion which leads o un-
ce ain y du ing ma ching. Fo he KDR100 expe imen s,
we decided o only conside he 30 mos simila e e ence
maps o spa ial e i ica ion (dashed ed line in Figu e 5),
allowing a maximal accu acy o 86.45%. A e spa ial e -
i ica ion 82% o he 657 shee s we e co ec ly p edic ed.
Respec ing he da um shi , he p ojec ion in oduces no
appa en de o ma ions wi h espec o WGS84. The co e -
age o hyd ology ea u es in OSM is simila o he symbols
isible on he maps. The e o e, he maps can be ma ched o
he e e ence da a wi h high ce ain y in mos cases. The
esul s a e qui e simila o he expe imen wi h deg aded
OSM da a, which sugges s, ha he me hod is mos ly o-
bus o his o ical changes and segmen a ion noise. Inco -
ec ma ches a e p obably owed o segmen a ion e o s.
3.4 Di e en scales and p ojec ions
3.4.1 Da a
Ca l Vogel’s Ka e des Deu schen Reiches consis s o 27
shee s in scale 1 : 500 000 (KDR500) om he u n o he
20 h cen u y. I uses he Bonne p ojec ion wi h unknown
cen al me idian and s anda d pa allel. The maps a e mul i-
colou p in s, which we segmen ed by colou h esholding.
The index is cons uc ed om 36 quad angles in a egula
g id ha ha e been ep ojec ed o WGS84.
3.4.2 Resul
The map p ojec ion is signi ican ly di e en om he e e -
ence da a. In WGS84, he maps look il ed and de o med.
In his small scale, he coas line becomes he mos desc ip-
i e symbol in a map. On he o he hand, some inland
shee s show e y ew i e s because o s ong gene ali-
sa ion. This gene alisa ion makes i ha d o ma ch some
inland shee s o he e e ence da a, which con ains many
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
6 o 8
Figu e 4. Segmen a ion e o s o an ac ual KDR100 map (shee 258) compa ed o OSM e e ence. Le : he o iginal
que y map. Middle: segmen a ion o hyd ology by colou h esholding. Righ : he co esponding e e ence map om
OSM. The dominan symbols in his map p o ided he in o ma ion o geoloca e his map despi e he noise and missing
symbols. Map © S aa sbiblio hek zu Be lin, e e ence da a © OpenS ee Map con ibu o s.
mo e i e s, as we could no ind OSM ags o il e hem
mo e p ecisely. Subsequen ly hose inland shee s we e
no success ully loca ed, esul ing in he low accu acy o
55.6%, as seen in Table 1.
3.5 Di e en designs ac oss he wo ld
3.5.1 Da a
A se o 197 quad angles o his o ical opog aphic maps
ac oss he Uni ed S a es in 1 : 100 000 scale om he yea s
1972–1980 in NAD23-p ojec ion (USGS100). Segmen-
a ion is a bi mo e di icul han o he o he maps be-
cause he USGS maps use mo e colou s. The e o e, we
use a con olu ional neu al ne wo k o segmen a ion, as
has been p oposed by Jiao e al. (2020). The index co e s
he Uni ed S a es comple ely wi h 1772 e e ence quad an-
gles.
3.5.2 Resul
The maps show many hyd ology symbols which a e no
mapped in OSM a all. Pa icula ly maps washes in dese
a eas and small c eeks in moun ain a eas could no be
ma ched o he co esponding e e ence maps, which a e
o en almos emp y. A eas wi h big i e s and coas lines,
on he o he hand, a e ma ched o OSM be e . The e-
sul is an o e all subpa pe o mance o only 43.6% co -
ec p edic ions. The compa ison wi h he OSM baseline
o he USGS100 quad angles (see Table 1) sugges s ha a
be e sou ce o e e ence da a4o di e en symbols al o-
ge he (e. g. oads) a e necessa y o inc ease he accu acy
o USGS maps.
3.6 Run ime pe o mance
On a single CPU co e wi h 2.9 GHz and 24 GB o RAM,
cons uc ing he index akes abou 0.7 s pe e e ence quad-
angle excluding he download o e e ence da a. Que y-
ing he index akes app oxima ely 6.7 s pe uncomp essed
que y map, plus he a iable ime needed o spa ial e -
i ica ion. Each i e a ion o spa ial e i ica ion akes 0.3 s,
4such as he Na ional Hyd og aphy Da abase:
h ps://www.usgs.go /co e-science-sys ems/ngp/na ional-
hyd og aphy/na ional-hyd og aphy-da ase
Figu e 5. Index ank dis ibu ion and e ec on he maxi-
mal eachable accu acy in ela ion o he cu o alue o
KDR100 maps. Inc easing he numbe o loca ion hy-
po heses o e i y inc eases compu a ion ime linea ly, bu
has diminishing e u ns o accu acy.
leading o app oxima ely 9 s when e i ying he 30 mos
simila e e ence maps.
This clea ly illus a es he impo ance o a sensible choice
o he numbe o loca ion hypo heses o e u n om he
index. We wan o sa e as many i e a ions o spa ial e i i-
ca ion as possible, o sa e compu a ion ime. Howe e , his
comes as a ade-o , since when he co ec ma ch is no
among he loca ion hypo heses e u ned om he index, i
canno be loca ed co ec ly. None heless, a e y high num-
be o hypo heses does no always u n ou o be help ul,
since maps which ank low in he index o en do no ca y
enough in o ma ion o be success ully p edic ed by spa ial
e i ica ion ei he . Figu e 5 illus a es, how he numbe o
loca ion hypo heses o e i y in luences he maximum ac-
cu acy o KDR100 p edic ions.
We ha e expe imen ed wi h an ea ly e mina ion heu is ic,
which aims o sa e some spa ial e i ica ion s eps o he
maps ha ha e al eady been co ec ly p edic ed by he in-
dex alone. I wo ks by in es iga ing he index o es o he
index esponse. When he numbe o o es o he n mos
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
7 o 8
simila e e ence image is wo imes la ge han o he n+1
mos simila image, spa ial e i ica ion s ops a e n s eps.
Fo he KDR100, his heu is ic enables 270 co ec skips,
usually a e he i s e e ence image, hus sa ing almos
a hi d o he o al compu a ion ime when p ocessing he
comple e se ies, a he cos o ailed p edic ions o only
18 addi ional shee s.
4. Conclusion
4.1 Summa y
We ha e shown ha CBIR wi h spa se image ea u es-
based simila i y can success ully geoloca e mos shee s o
di e en opog aphic map se ies. Wi h ou me hod, la ge
olumes o maps can be geo e e enced in a sho ime wi h
signi ican ly educed manual labou . The algo i hm scales
e y well and can be pa allelised o be inco po a ed in o
an in e ac i e web applica ion such as he one p oposed by
Ta akkol e al. (2019).
Many maps a e loca ed wi h high con idence, bu o he s
migh no ha e been success ully loca ed wi h inc eased
unce ain y (e. g. h ough noise). We can p o ide a mea-
su e o con idence wi h he numbe and dis ibu ion o
RANSAC inlie s. This allows o educe manual quali y
con ol and e o co ec ion by only in es iga ing he low
con idence p edic ions.
RANSAC no only ou pu s a sco e, bu also a ans o m
model, which can be used o alignmen o he geolocalised
maps. Howe e , hese ans o ma ions did no consis en ly
p oduce accep able egis a ion accu acy on hei own and
should be supplemen ed wi h one o he p oposed me hods
o alignmen in o de o exploi mo e a ailable spa ial in-
o ma ion (Howe e al., 2019, Duan e al., 2020, Lu and
Schiewe, 2021).
Compa ing wi h p e ious s udies, ou numbe s sugges a
wo se pe o mance han he 96% accu acy o Lu and
Schiewe (2021). Howe e , we ha e subs an ially inc eased
he size o he es da a se . Fu he mo e, he imp o ed
me hod p esen ed he e signi ican ly imp o es que y ime
and is be e sui ed o apply o a ious inpu maps wi hou
much pa ame e uning.
Calcula ing he o al accu acy o all h ee map se ies,
ou me hod sligh ly ou pe o ms he me hod p oposed by
Ta akkol e al. (2019). They ha e ound loca ion p edic-
ions o 68% o 500 e y di e se maps o di e en ype
and scale. Howe e , he quali y o hei solu ions, gi en
by e o dis ance, is no pa icula ly meaning ul wi hou
aking he scales o he espec i e maps in o conside a-
ion. Thei wo k indica es ha oponym-based geoloca ion
ans e s easily o a a ie y o maps bu is e o -p one and
challenging o implemen obus ly.
4.2 Fu u e wo k
The KDR500 e e ence maps we e shi ed and skewed
sligh ly because o ep ojec ion. The expe imen shows
ha he me hod is o some ex en obus o inexac e -
e ence loca ions in he index. Pushing he bounda ies o
he index obus ness o displacemen , in he hopes o be-
ing able o do wi hou he p io knowledge o quad angle
loca ions al oge he , is subjec o u he esea ch.
Complex o unusual map p ojec ions make di ec compa i-
son o he e e ence OSM da a di icul . Rep ojec ion o ei-
he he e e ence da a o he inpu maps (compa e ou ex-
pe imen s on KDR500) a oids any issues. Howe e , ep o-
jec ion equi es p io knowledge o he used map p ojec-
ions. Some imes he exac spa ial e e ence sys em migh
be unknown and in o de o be able o deal wi h de o -
ma ions in he inpu maps, mo e complex ans o m mod-
els o RANSAC ha e o be e alua ed. This also aises
he ques ion o how obus image ea u e desc ip o s a e
o p ojec ion de o ma ions. They a e usually designed o
be in a ian o geome ic de o ma ions, like o a ion and
scaling, bu hei pe o mance unde e. g. polynomial de-
o ma ions has o be e alua ed.
Segmen a ion is he c ux o gene alisabili y. The me hod
he e wo ks on an abs ac ed image and is uni e sal, bu
ge ing he e by ex ac ing he “ igh ” symbols is di e -
en o each map se ies and does no easily ans e wi h-
ou manual pa ame e uning. Deep lea ning segmen a ion
models which a e able o ans e be ween di e en map
s yles could be he solu ion. High pe o mance wi h li le
aining da a is pa amoun , because collec ing and anno a -
ing la ge amoun s o aining da a quickly ge s ou o p o-
po ion o he expec ed ime sa ings o an au oma ed geo-
e e encing sys em. We a e cu en ly wo king on a deep
lea ning segmen a ion model using syn he ic aining da a,
which is he p e e ed solu ion when he e is limi ed a ail-
abili y o aining da a, as is he case wi h his o ical maps.
Ou esea ch con inues wi h he goal o imp o ing accu-
acy, making ou me hod mo e gene al by allowing he use
o mul iple di e en symbols a once and inc easing ease
o use by educing me ada a equi emen s.
Re e ences
Alcan a illa, P. F., Ba oli, A. and Da ison, A. J., 2012.
KAZE ea u es. In: A. Fi zgibbon, S. Lazebnik, P. Pe -
ona, Y. Sa o and C. Schmid (eds), P oceedings o
he 12 h Eu opean Con e ence on Compu e Vision,
Lec u e No es in Compu e Science, Vol. 4, Sp inge ,
p. 214–227.
Buckley, A. R., 2019. Sha ing collec ions o his o ical
maps online. Abs ac s o he ICA.
Bu , J., Whi e, J., Allo d, G., Then, K. and Zhu, A.-X.,
2019. Au oma ed and semi-au oma ed map geo e e enc-
ing. Ca og aphy and Geog aphic In o ma ion Science
47, pp. 40–66.
Chiang, Y.-Y., Duan, W., Leyk, S., Uhl, J. H. and
Knoblock, C. A., 2020. Using His o ical Maps in Scien-
i ic S udies: Applica ions, Challenges, and Bes P ac-
ices. Sp inge B ie s in Geog aphy, Sp inge In e na-
ional Publishing, Cham, Swi ze land.
Chiang, Y.-Y., Leyk, S. and Knoblock, C. A., 2014. A su -
ey o digi al map p ocessing echniques. ACM Com-
pu ing Su eys 47(1), pp. 1–44.
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
8 o 8
C om, W., 2016. Ka endigi alisie ung – bun es Bild
ode Meh we . Ka og aphische Nach ich en 66(5),
pp. 243–248.
Duan, W., Chiang, Y.-Y., Leyk, S., Uhl, J. H. and
Knoblock, C. A., 2020. Au oma ic alignmen o con em-
po a y ec o da a and geo e e enced his o ical maps us-
ing ein o cemen lea ning. In e na ional Jou nal o Ge-
og aphical In o ma ion Science 34(4), pp. 824–849.
Fab is, M., 2021. Moni o ing he coas al changes o he
Po i e del a (no he n i aly) since 1911 using a chi al
ca og aphy, mul i- empo al ae ial pho og amme y and
lida da a: Implica ions o coas line changes in 2100
a.d. Remo e Sensing 13(3), pp. 1–23.
Fischle , M. A. and Bolles, R. C., 1981. Random sam-
ple consensus: a pa adigm o model i ing wi h ap-
plica ions o image analysis and au oma ed ca og aphy.
Communica ions o he ACM 24(6), pp. 381–395.
Flee , C., Kowal, K. C. and P idal, P., 2012. Geo e -
e ence : C owdsou ced geo e e encing o map lib a y
collec ions. D-Lib Magazine 18(11/12), pp. 1–11.
Hei zle , M. and Hu ni, L., 2019. Unlocking he geospa ial
pas wi h deep lea ning – es ablishing a hub o his o -
ical map da a in swi ze land. Abs ac s o he ICA 1,
pp. 1–2.
Hei zle , M., Gkonos, C., Tso lini, A. and Hu ni, L., 2018.
A modula p ocess o imp o e he geo e e encing o he
sieg ied map. e-Pe ime on 13(2), pp. 85–100.
Howe, N. R., Weinman, J., Gouwa , J. and Shamji, A.,
2019. De o mable pa models o au oma ically geo-
e e encing his o ical map images. In: P oceedings
o he 27 h ACM SIGSPATIAL In e na ional Con e -
ence on Ad ances in Geog aphic In o ma ion Sys ems,
SIGSPATIAL ’19, Associa ion o Compu ing Machin-
e y, Chicago, IL, USA, p. 540–543.
Iosi escu, I., Tso lini, A. and Hu ni, L., 2016. Towa ds
a comp ehensi e me hodology o au oma ic ec o -
iza ion o as e his o ical maps. e-Pe ime on 11(2),
pp. 57–76.
Janowicz, K., Raubal, M. and Kuhn, W., 2011. The
seman ics o simila i y in geog aphic in o ma ion e-
ie al. Jou nal o Spa ial In o ma ion Science 2011(2),
pp. 29–57.
Jiao, C., Hei zle , M. and Hu ni, L., 2020. Ex ac ing
we lands om swiss his o ical maps wi h con olu ional
neu al ne wo ks. In: Au oma ic Vec o isa ion o His o i-
cal Maps: In e na ional wo kshop o ganized by he ICA
Commission on Ca og aphic He i age in o he Digi al,
Depa men o Ca og aphy and Geoin o ma ics ELTE,
Budapes , Hunga y, p. 33–38.
Jiao, C., Hei zle , M. and Hu ni, L., 2021. A su ey o oad
ea u e ex ac ion me hods om as e maps. o appea
in T ansac ions in GIS.
Li, B. and Fonseca, F., 2006. TDD: A comp ehensi e
model o quali a i e spa ial simila i y assessmen . Spa-
ial Cogni ion & Compu a ion 6(1), pp. 31–62.
Li, J., Hu, Q. and Ai, M., 2020. RIFT: Mul i-modal image
ma ching based on adia ion- a ia ion insensi i e ea-
u e ans o m. IEEE T ansac ions on Image P ocessing
29, pp. 3296–3310.
Liu, L., Wang, Y. and Wang, Y., 2008. SIFT based au-
oma ic ie-poin ex ac ion o mul i empo al SAR im-
ages. In: 2008 In e na ional Wo kshop on Educa ion
Technology and T aining 2008 In e na ional Wo kshop
on Geoscience and Remo e Sensing, Vol. 1, Shanghai,
China, p. 499–503.
Lu , J., 2020. Au oma ic geo e e encing o his o ical
maps by geocoding. In: Au oma ic Vec o isa ion o
His o ical Maps: In e na ional wo kshop o ganized by
he ICA Commission on Ca og aphic He i age in o he
Digi al, Depa men o Ca og aphy and Geoin o ma ics
ELTE, Budapes , Hunga y, p. 77–89.
Lu , J. and Schiewe, J., 2021. Au oma ic con en -based
geo e e encing o his o ical opog aphic maps. o ap-
pea in T ansac ions in GIS.
Schlegel, I., 2019. Empi ical s udy o a deploymen o a
me hodology o imp o ing he compa abili y be ween
his o ical and cu en maps. KN - Jou nal o Ca og a-
phy and Geog aphic In o ma ion 69(2), pp. 121–130.
Smeulde s, A., Wo ing, M., San ini, S., Gup a, A. and
Jain, R., 2000. Con en -based image e ie al a he end
o he ea ly yea s. IEEE T ansac ions on Pa e n Analy-
sis and Machine In elligence 22(12), pp. 1349–1380.
Sun, K., Hu, Y., Song, J. and Zhu, Y., 2020. Aligning geo-
g aphic en i ies om his o ical maps o building knowl-
edge g aphs. o appea in In e na ional Jou nal o Geo-
g aphical In o ma ion Science.
Szend ei, R., Elek, I. and Má on, M., 2011. A knowledge-
based app oach o as e - ec o con e sion o la ge scale
opog aphic maps. Ac a Cybe ne ica 20, pp. 145–165.
Ta akkol, S., Chiang, Y.-Y., Wa e s, T., Han, F., P asad, K.
and Ki e is, R., 2019. Ka a labs: Un ende ing his o i-
cal maps. In: P oceedings o he 3 d ACM SIGSPATIAL
In e na ional Wo kshop on AI o Geog aphic Knowl-
edge Disco e y, GeoAI 2019, Associa ion o Compu -
ing Machine y, New Yo k, NY, USA, p. 48–51.
Whi e, A. P., 2013. X ma ks he spo : Ex ac ing da a om
his o ical maps o loca e a chaeological si es. Jou nal o
Map & Geog aphy Lib a ies 9(1–2), pp. 140–156.
Wol e , D., Blank, D. and Hen ich, A., 2017. Geo e e enc-
ing i e ne wo ks using spa ial easoning. In: P oceed-
ings o he 11 h Wo kshop on Geog aphic In o ma ion
Re ie al - GIR’17, ACM P ess, Heidelbe g, Ge many.
Ye, Y. and Shen, L., 2016. HOPC: a no el simila i y me ic
based on geome ic s uc u al p ope ies o mul i-modal
emo e sensing image ma ching. III–1, pp. 9–16.
Zhuo, X., Koch, T., Ku z, F., F aundo e , F. and Reina z,
P., 2017. Au oma ic UAV image geo- egis a ion by
ma ching UAV images o geo e e enced image da a. Re-
mo e Sensing 9(44), pp. 376.
P oceedings o he In e na ional Ca og aphic Associa ion, 4, 2021.
30 h In e na ional Ca og aphic Con e ence (ICC 2021), 14–18 Decembe 2021, Flo ence, I aly. This con ibu ion unde wen
single-blind pee e iew based on submi ed abs ac s. h ps://doi.o g/10.5194/ica-p oc-4-69-2021 | © Au ho (s) 2021. CC BY 4.0 License.
