IOP Con e ence Se ies: Ea h and En i onmen al Science
PAPER • OPEN ACCESS
Accu acy E alua ion and Compa ison o Mobile Lase Scanning and
Mobile Pho og amme y Da a
To ci e his a icle: Pe Kal oda e al 2020 IOP Con . Se .: Ea h En i on. Sci. 609 012091
View he a icle online o upda es and enhancemen s.
This con en was downloaded om IP add ess 147.229.117.44 on 20/04/2021 a 06:59
Con en om his wo k may be used unde he e ms o heC ea i eCommonsA ibu ion 3.0 licence. Any u he dis ibu ion
o his wo k mus main ain a ibu ion o he au ho (s) and he i le o he wo k, jou nal ci a ion and DOI.
Published unde licence by IOP Publishing L d
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
1
Accu acy E alua ion and Compa ison o Mobile Lase
Scanning and Mobile Pho og amme y Da a
Pe Kal oda 1, Jakub Nosek 1, Michal Ku uc 1, Tomas Vola ik 1, Pe a
Kal odo a 1
1 B no Uni e si y o Technology, Ve eří 331/95, B no, Czech Republic
ku uc.m@ ce. u b .cz
Abs ac . Mobile mapping sys ems (MMS) a e becoming used in s anda d geode ic asks mo e
common in he las yea s. This pape deals wi h he accu acy e alua ion o wo ypes o da a
acqui ed by MMS RIEGL VMX-450, and hei compa ison. The i s ype is da a om mobile
lase scanning (MLS). The second ype is mobile pho og amme y da a. The new high accu a e
es poin ield was buil in a ea o Ad anced Ma e ials, S uc u es and Technologies (AdMaS)
esea ch cen e ha is pa o B no Uni e si y o Technology. Geode ic ne wo k and es poin
ield we e measu ed by T imble R8s GNSS sys em and T imble S8 HP o al s a ion. The es ima e
o he 3D s anda d de ia ion de e mined by an adjus men is 2 mm. The accu acy o MLS and
mobile pho og amme y da a was es ed based on he di e ences be ween he coo dina es o he
poin s de e mined om he MMS da a and de e mined by be o e men ioned high p ecise
measu emen . The esul ing coo dina es om pho og amme ic da a we e de e mined by manual
de ec ion o a ge s in he images. The es ima e o he 3D s anda d de ia ion is 0.017 m om he
MLS da a, and 0.061 m om he mobile pho og amme y da a. As we supposed, he mobile lase
scanning da a a e signi ican ly mo e accu a e han mobile pho og amme y da a. Achie ed
accu acy o MLS exceeds he o iginal expec a ions wi h espec o he GNSS/IMU posi ioning
accu acy, which is acco ding o he manu ac u e RIEGL be ween 0.02–0.05 m. The same scene
is o en scanned wi h mul iple scanning passes o ensu e high quali y o he scanned poin cloud,
he e o e we es ed he ela i e accu acy o mobile lase scanning da a om wo MMS ehicle
passes in he same locali y o in e es . Two di e en da a se s we e e alua ed, i s da a se
con ains poin s on oads, second da a se on buildings. The s anda d de ia ion es ima e does no
exceed 0.008 m and he maximum absolu e de ia ion does no exceed 0.030 m o bo h da a se s.
The di e ence be ween he wo passes is no signi ican in compa ison wi h he accu acy c i e ia
equi ed o s anda d mapping pu poses. We also compa ed au oma ic poin cloud p oduc ion
om pho og amme y da a p ocessed in Ben ley Con ex Cap u e o he poin cloud om lase
scanning. The MLS da a has been used as a e e ence because i is signi ican ly mo e accu a e
as men ioned be o e. This compa ison was done only on he second da a se (buildings). The
s anda d de ia ion es ima e is 0.16 m and he maximum absolu e de ia ion is 0.25 m. Ou
e alua ion con ains also s a is ical es ing o ou lie s and s aggle s. In con as o many au ho s,
we don' use he simpli ied app oach 3σ ule, and in 1D. We use mo e exac app oach using
c i ical alues o he s a is ics o signi icance le els α = 5 % and α = 1 % o s aggle s and
ou lie s es in 3D.
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
2
1. In oduc ion
Mobile me hods become widely used o cap u e spa ial da a o many applica ions, o example, ci il
enginee ing, oad-su eying, and 3D ci y modeling [1, 2]. The geome ic accu acy o inal p oduc s is
one o he key p ope ies, depends on a la ge numbe o ac o s. Accu acy can be assessed acco ding o
se e al c i e ia. Simply pu , he accu acy o he inal p oduc (3D model) depends on he inpu da a
accu acy, densi y ( esolu ion), and on he algo i hms used.
In he case o mobile lase scanning (MLS), he inpu da a o modeling consis s o egis e ed and
il e ed poin clouds. The basic p e equisi es o he accu acy o he 3D model a e he e o e he accu acy
and densi y o MLS poin clouds. This accu acy can be di ided in o absolu e and ela i e componen s,
which co espond o he posi ioning subsys em and mapping subsys em o MMS. The posi ioning
subsys em uses a Global Na iga ion Sa elli e Sys em (GNSS), Ine ial Measu ing Uni (IMU), and
Dis ance Measu emen Indica o s (DMIs) [3, 4]. The esul o GNSS, IMU, and DMI da a combina ion
in a Kalman il e is Smoo hed Bes Es ima ed T ajec o y (SBET). The accu acy o SBET can be
imp o ed by using con ol poin s [5]. The mapping subsys em pe o ms he spa ial da a acquisi ion and
ypically consis s o one o mo e LIDAR senso s and came as. Accu a e calib a ion o bo h subsys ems
is a p e equisi e o accu a e geo e e enced images and LIDAR da a.
In he case o mobile pho og amme y, he esul ing 3D model can be c ea ed in wo ways: manual
and au oma ic model gene a ion. In he case o manual model gene a ion, he model elemen s a e c ea ed
di ec ly abo e he images. The undamen al elemen s a e poin s and he inal accu acy depends on hei
accu acy. The au oma ic me hod consis s o he poin cloud gene a ion using a ma ching algo i hm [6]
and 3D model c ea ion om his poin cloud (meshing). The accu acy and densi y o he poin cloud is
he basic p e equisi e o he accu acy o he 3D model.
2. MMS desc ip ion
The used MMS RIEGL VMX-450 ( able 1, igu e 1) in eg a es wo RIEGL VQ-450 lase scanne s,
modula VMX-450-CS6 came a sys em wi h ou indus ial came as, POINT GREY ladybug5 sphe ical
came a imaging sys em, GNSS/IMU na iga ion ha dwa e, dis ance measu emen uni VMX-450-DMI
and a po able con ol uni VMX-450-CU.
Table 1. RIEGL VMX-450 echnical cha ac e is ics [7].
Senso
P ope y name
P ope y alue
VQ-450
Measu ing p inciple
Time o Fligh
Max. measu emen a e
1.1 MHz (2×0.55 MHz), ρ ≥ 10 % up o 140 m
Scan a e (selec able)
up o 400 lines/sec
Accu acy
8 mm, 1σ @ 50 m ange
P ecision
5 mm, 1σ @ 50 m ange
IMU/GNSS
Absolu e posi ion accu acy
0.02–0.05 m, 1σ
Roll and pi ch accu acy
0.005°, 1σ
Yaw (heading) accu acy
0.015°, 1σ
VMX-450-CS6
Resolu ion
5 Mpx
Pixel size
3.45 μm
Senso size
2452 × 2056 px
Nominal ocal leng h
5 mm
Ladybug5
Resolu ion
30 Mpx ( 5 Mpx × 6 senso s)
Pixel size
3.45 μm
Senso size
2048 × 2448 px
Nominal ocal leng h
4.4 mm
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
3
RIEGL de ines he accu acy in da ashee [7] as a deg ee o con o mi y o measu ed quan i y o i s
ue alue in he VQ-450 sec ion o able 1. The p ecision RIEGL de ines in da ashee [7] as epea abili y
( he deg ee o which u he measu emen s show he same esul ). Values in he IMU/GNSS sec ion o
able 1 a e alid i he ollowing condi ions a e ul illed: no GNSS ou ages, DMI op ion, and pos -
p ocessed using base s a ion da a. The came a calib a ion pa ame e s a e p o ided by he manu ac u e .
Ex e io o ien a ion pa ame e s (posi ions and o a ions o he came as) can be compu ed by RIEGL
so wa e in pos -p ocessing. The came a da a (digi al came a images) can be expo ed in JPEG o ma .
Op ionally, index iles con aining a imes amp, posi ion, and o ien a ion can also be c ea ed o each
image. The images can be expo ed undis o ed ( adial and angen ial dis o ions a e emo ed) [8].
Figu e 1. RIEGL VMX-450 con igu a ion
3. Tes poin ield
Ou es ing is based on he analysis o he di e ences be ween he coo dina es o he poin s de e mined
om he MMS da a and de e mined by a signi ican ly mo e accu a e me hod. Iden i iabili y o hese
poin s is an impo an p ope y o esul ing accu acy. We can dis inguish wo ypes o poin
iden i iabili y o ou pu poses: poin iden i iabili y in eal-wo ld and poin iden i iabili y in cap u ed
da a (poin clouds, images). Resul ing poin coo dina es e o consis s o he iden i ica ion e o and he
e o o he used coo dina e de e mina ion me hod. Fo example, he e o o a poin de e mined by a
o al s a ion is a ec ed by a poin iden i ica ion e o in he eal wo ld. The e o o a poin measu ed
om a poin cloud is a ec ed by a poin iden i ica ion e o in a poin cloud.
In his pape , we ocus on he e o s o he used me hods. The i s s ep is minimizing he impac o
iden i ica ion e o s, he e o e i was decided o signal he es poin s by a ge s. The signaliza ion by
a ge s elimina es he e o o ambiguous iden i iabili y o some na u al poin s. I was necessa y o c ea e
a es poin s ield ha will ha e sa is ac o y accu acy cha ac e is ics o he needs o MMS accu acy
es ing. The coun o poin s mus be su icien o ensu e he eliabili y o he esul s. The AdMaS
Resea ch Cen e complex was chosen due o he easy a ailabili y and long- e m sus ainabili y o he
a ge s. 214 poin s we e es ablished in he AdMaS cen e, which we e signalized and s abilized using
checke boa d a ge s. Ho izon al a ge s (119) we e ma ked by whi e colo on asphal oadways ( igu e
2). The e ical a ge s (95) we e made om a black ma aluminum shee , which was supplemen ed
wi h a e lec i e oil ( igu e 2). Ve ical a ge s we e placed on buildings, e ical a ic signs, conc e e
pilla s and o he sui able e ical s uc u es. The a ge s on he buildings we e placed in wo heigh le els
abo e he g ound: 2 m, 10 m.
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
4
Figu e 2. Checke boa d a ge s
The geode ic ne wo k and he es poin ield we e measu ed wi h high accu acy. T imble R8s GNSS
sys em and T imble S8 HP o al s a ion we e used. Fi s ly, a pu pose-buil geode ic ne wo k was c ea ed.
Secondly, es ield poin s we e de e mined. The coo dina es o he es ield poin s we e calcula ed by
he geode ic ne wo k leas squa es adjus men wi h a combina ion o GNSS and e es ial
measu emen s. The Eu opean Te es ial Re e ence Sys em 89 (ETRS89) and Eu opean Te es ial
Re e ence F ame 2000 (ETRF2000) we e used [9]. The minimum-cons ained ne wo k adjus men was
used o compu ing o geode ic ne wo k. Inpu da a in adjus men was pola coo dina es measu ed by
he o al s a ion and coo dina es o ou poin s de e mined by he s a ic GNSS me hod. The cons ained
ne wo k adjus men was used o compu ing o es ield a ached o ixed poin s ( he geode ic ne wo k).
The o e all accu acy o he es ield de e mined by adjus men can be exp essed as he es ima e o he
3D s anda d de ia ion sX,Y,Z = 2 mm.
4. MMS da a acquisi ion and p ocessing
MMS da a we e acqui ed by wo ehicle passes (in bo h di ec ions) 750 m long a a speed o 20 km/h.
Lase da a we e acqui ed a a equency o 1.1 MHz. Came a da a we e egis e ed e e y 1.5 m. The
MMS ajec o y was calcula ed using Applanix POSPac. The esul s o he GNSS Pos P ocessing
Kinema ic me hod we e e ined and smoo hed by a o wa d-backwa d Kalman il e using IMU and
DMI da a.
The p ocessing o MLS da a was pe o med in RIEGL RiPROCESS. RiPROCESS p ocessing
consis s o da a con e sion, poin clouds gene a ion, and ajec o y adjus men . In he i s s age, a poin
clouds we e c ea ed based on he POSPac ajec o y. Fu he mo e, con ol poin s and check poin s in
he poin cloud we e manually iden i ied. Mo e p ecise ajec o y was p ocessed using he
RiPRECISION module based on he co espondences be ween he poin clouds and he con ol poin s.
The esul ing poin cloud consis s o wo pa ial poin clouds co esponding o wo ehicle passes. The
esul ing poin cloud con ains mo e han 247,000,000 poin s wi h a densi y o a ound 4 mm. The wo
pa ial poin clouds con ain a ound 123,500,000 poin s wi h a densi y o a ound 8 mm.
The came a da a (ex e io o ien a ion pa ame e s and undis o ed images) we e expo ed by
RiPROCESS. These da a we e p ocessed in he Ben ley Con ex Cap u e so wa e. A o al o 3,872
images was p ocessed. Fi s ly, au oma ic keypoin ex ac ion, au oma ic ie poin ma ching, and bundle
adjus men we e pe o med. Secondly, he con ol poin s we e ma ked manually. Thi dly, he check
poin s we e ma ked manually [10]. The basic pa ame e s o pho og amme ic p ocessing a e in able 2.

6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
5
Table 2. O e iew o Con ex Cap u e p ocessing pa ame e s.
Pa ame e name
Pa ame e alue
Coun o pho os
3872
Coun o ie poin s
490794
Median o ie poin s pe pho o
412
Rep ojec ion e o (RMS)
0.70 px
Fil e ing and cloud di e ences we e pe o med in CloudCompa e so wa e 2.10. The 2.5D olume
unc ion was used o calcula e cloud di e ences a he 5 × 5 cm g id nodes. Es ima ions o di e ences
be ween wo clouds, ha we e p ojec ed o he ho izon al plane in case o he oad sub-clouds, and o
he acade plane in case o he acade sub-clouds, we e pe o med.
5. Accu acy e alua ion me hodology
Tes ield poin s and poin clouds we e ans o med in o a local opocen ic sys em Eas , No h, Up
(E, N, U) o easie in e p e a ion. Con ol poin s we e no used as check poin s o ensu e independen
accu acy e alua ion. The coo dina es o poin s de e mined by geode ic me hod a e signi ican ly mo e
accu a e hen coo dina es de e mined om MMS da a. The e o e, we can deal wi h coo dina es Ei, Ni, Ui
de e mined by a geode ic me hod as ue alues and deal wi h di e ences (δE, δN, δU) on check poin s
as ue e o s:
𝛿𝐸𝑖=𝐸𝑖−𝐸
𝑖, 𝛿𝑁𝑖=𝑁𝑖−𝑁
𝑖, 𝛿𝑈𝑖=𝑈𝑖−𝑈
𝑖 , (1)
whe e Ẽi, Ñi, Ũi deno e coo dina es de e mined om MMS da a. I we compa e pai s o he same
accu acy, we ha e o wo k wi h he hal alues o he di e ences. The 3D di e ences a e compu ed as:
𝛿3𝐷𝑖=√𝛿𝐸𝑖
2+𝛿𝑁𝑖
2+𝛿𝑈𝑖
2 . (2)
The s anda d de ia ions a e es ima ed as:
𝑠𝐸=√∑𝛿𝐸𝑖
𝑛
𝑖=1
𝑛, 𝑠𝑁=√∑𝛿𝑁𝑖
𝑛
𝑖=1
𝑛, 𝑠𝑈=√∑𝛿𝑈𝑖
𝑛
𝑖=1
𝑛,𝑠3𝐷 =√∑𝛿3𝐷𝑖
𝑛
𝑖=1𝑛 , (3)
whe e n is coun o di e ences (coun o check poin s). Su aces o cons an p obabili y densi y
co espond o he e o s (δE, δN, δU) o which:
[𝛿𝐸 𝛿𝑁 𝛿𝑈] 𝐶𝑥
−1 [𝛿𝐸 𝛿𝑁 𝛿𝑈]𝑇=𝑡2, (4)
whe e Cx is 3 × 3 a iance-co a iance ma ix, and is a size pa ame e [11]. The con idence ellipsoid
can be exp essed as:
𝛿𝐸
2
𝑠𝐸
2+𝛿𝑁
2
𝑠𝑁
2+𝛿𝑈
2
𝑠𝑈
2=𝑡2, (5)
The semi-p incipal axes a, b, and c o he con idence ellipsoid can be exp essed as a = ∙ sE, b = ∙ sN,
c = ∙ sU. The e o dis ibu ion p obabili y depends on he size pa ame e . The olume o he ellipsoid
equals con idence le el 1 - α. By changing in equa ions (4), and (5), he olume and wi h i he
con idence le el is changed. Fo = 1 he ellipsoid is called he s anda d con idence ellipsoid. The
con idence le el o his ellipsoid is 1 - α = 19.1 %. The p obabili y ha he e o will lie inside his
ellipsoid is 19.1 % [11].
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
6
An impo an indica o o accu acy is he occu ence o g oss e o s (ou lie s). The s a is ical me hods
o esul s analysis desc ibed in ISO 5725-2 [12] s anda d can be used. I he es s a is ic in “nume ical
ou lie es ” a e less han o equal o i s 5% c i ical alue (α = 5 %), he i em es ed is accep ed as co ec .
I he es s a is ic is g ea e han i s 5% c i ical alue (α = 5 %) and less han o equal o i s 1% c i ical
alue (α = 1 %), he i em es ed is called a s aggle . I he es s a is ic is g ea e han i s 1% c i ical
alue (α = 1 %), he i em is called a s a is ical ou lie [12].
The size pa ame e o con idence ellipsoid o 1 – α = 95 % p obabili y ha he e o lie inside he
ellipsoid is = 2.8, and o 1 – α = 99 % p obabili y is = 3.4. The poin s wi h e o s (δE, δN, δU) ha lie
inside 95% con idence ellipsoid (Z95%) a e accep ed. The poin s wi h e o s (δE, δN, δU) ha lie ou side
95% con idence ellipsoid (Z95%) and inside o on 99% con idence ellipsoid (Z99%) a e called s aggle s.
The poin s wi h e o s (δE, δN, δU) ha lie ou side 99% con idence ellipsoid (Z99%) a e called ou lie s.
This es can be simpli ied using he con idence sphe e ins ead o he ellipsoid. The 3D e o s ha
mee he condi ion δ3D ≤ 2.8 ∙ s3D a e accep ed. The 3D e o s ha mee he condi ion
2.8 ∙ s3D < δ3D ≤ 3.4 ∙ s3D a e called s aggle s. The 3D e o s ha mee he condi ion δ3D ≥ 3.4 ∙ s3D a e
called ou lie s.
6. Resul s
The accu acy o MLS and mobile pho og amme y da a was es ed based on he di e ences be ween
he coo dina es o he poin s de e mined om he MMS da a and de e mined by be o e men ioned high
p ecise measu emen . The esul ing coo dina es om pho og amme ic da a we e de e mined by manual
de ec ion o a ge s in he images. The o e iew o con ol poin s and check poin s loca ion is shown in
igu e 3. The esul s o he accu acy e alua ion a check poin s a e desc ibed in able 3.
The esul s o he ela i e accu acy e alua ion om wo MMS ehicle passes a e desc ibed in able 4.
Two di e en MLS da a se s we e e alua ed, he i s da a se con ains poin s on oads (see column 2 in
able 4 and igu e 4), he second da a se on he acade (see column 3 in able 4 and igu e 5 le ). We also
compa ed au oma ic poin cloud p oduc ion om pho og amme y da a p ocessed in Ben ley
Con ex Cap u e o he poin cloud om lase scanning. The MLS da a has been used as a e e ence
because i is signi ican ly mo e accu a e as men ioned be o e. This compa ison was done only on he
second da a se on he acade (see column 4 in able 4 and igu e 5 igh ). These analyses a e based on
he di e ences in he poin clouds compu ed in CloudCompa e 2.10.
Figu e 3. O e iew con ol poin s and check poin s loca ion
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
7
Table 3. Accu acy e alua ion a check poin s
MOBILE LASER SCANNING
MOBILE PHOTOGRAMMETRY
E
N
U
3D
E
N
U
3D
Coun o con ol poin s
48
48
48
48
48
48
48
48
Coun o check poin s
158
158
158
158
96
96
96
96
Maximum absolu e de ia ion
0.037
0.025
0.044
0.050
0.095
0.216
0.124
0.230
S anda d de ia ion
0.011
0.009
0.009
0.017
0.031
0.045
0.027
0.061
Table 4. Rela i e accu acy e alua ion o poin clouds
Two passes MLS
oads
Two passes MLS
acade
MLS × pho og amme y
acade
Coun o g id poin s
83039
14510
10120
Maximum absolu e de ia ion
0.030
0.028
0.249
S anda d de ia ion
0.006
0.008
0.164
Figu e 4. Visualiza ion o he di e ences be ween he wo passes o MLS – oads
Figu e 5. Visualiza ion o he di e ences be ween he wo passes o MLS on he acade (le ),
be ween MLS and mobile pho og amme y on he acade ( igh )
6 h Wo ld Mul idisciplina y Ea h Sciences Symposium
IOP Con . Se ies: Ea h and En i onmen al Science 609 (2020) 012091
IOP Publishing
doi:10.1088/1755-1315/609/1/012091
8
The e o dis ibu ion on he check poin s is displayed in he his og ams in igu es 6, 7. The esul s
o he 3D e o es o ou lie s and s aggle s on he check poin s using Z95% and Z99% con idence sphe es
a e shown in able 5, and con idence ellipsoids in able 6 and igu e 8.
Table 5. The esul s o he ou lie s and s aggle s es on check poin s using con idence sphe e
MOBILE LASER
SCANNING
MOBILE
PHOTOGRAMMETRY
δ3D ≤ Z95%
157
92
Z95% < δ3D ≤ Z99%
1
2
δ3D > Z99%
0
2
Table 6. The esul s o he ou lie s and s aggle s es on check poin s using con idence ellipsoids
MOBILE LASER
SCANNING
MOBILE
PHOTOGRAMMETRY
(δE, δN, δU) ≤ Z95%
142
88
Z95% < (δE, δN, δU) ≤ Z99%
13
4
(δE, δN, δU) > Z99%
3
4
Figu e 6. His og ams o mobile lase scanning e o s on check poin s
Figu e 7. His og ams o mobile pho og amme y e o s on check poin s