GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
UTILIZATION OF AUGMENTED AND VIRTUAL REALITY
IN GEOSCIENCE
Vě osla HOLUŠA , Filip BENEŠ , Michal VANĚK
VSB – Technical Uni e si y o Os a a, Facul y o Mining and Geology,
Depa men o Economics and Con ol Sys ems
E-mail: e osla .ho[email p o ec ed]
ABSTRACT
In oday's age o mode n in o ma ion echnologies (IT), i is no wonde ha mode n isualiza ion ools and
echniques a e inding hei way in o a eas such as geological and en i onmen al sciences. The ma ke o e s a
wide ange o so wa e esou ces o use and especially o he c ea ion o scenes in i ual eali y. The aim o ou
a icle is o e alua e a numbe o ools o c ea ing i ual eali y (VR) con en and e alua e hei p ac ical
applicabili y. In he expe imen s, se e al ools o c ea ing i ual eali y scenes we e es ed and hese ools we e
compa ed in e ms o hei pa ame e s. The conclusion o he a icle shows he esul s, while Pho omodele and
Reali yCap u e we e he mos sui able so wa e o he academic sphe e. On he con a y, Ben ley Con ex Cap u e
so wa e seems he leas sui able.
Keywo ds: Augmen ed eali y; Benchma king; Geoscience; Vi ual eali y.
1 INTRODUCTION
Vi ual eali y is inc easingly in e wined wi h he wo ld o science and academic sphe e. I is used o he pu poses
o eaching and c ea ing eaching aids. Ve ga a e al. [1] desc ibed in hei s udy he sp ead o i ual eali y
lea ning en i onmen s in schools and desc ibed he p ocesses by which hese didac ic ools can imp o e he
e ec i eness o he educa ional p ocess. In his wo k, Fe nandez [2] p esen s and discusses he ole o i ual and
augmen ed eali y in educa ion. Lee e al. [3] explo e he possibili ies o using he Google Ca dboa d VR in
class ooms as a pla o m o p esen lea ning ma e ial. Speci ically, Lee's eam examined how VR pe cep ions
di e ed om pe cep ions o he adi ional la sc een o ma . Wang [4] examined he impac o in eg a ing
augmen ed eali y in o he cu iculum o a pa icula uni e si y cou se. Salah e al. [5] ocus on he eaching and
aining o indus y needs, especially in iew o he eme ging ends o Indus y 4.0. Visualiza ion echnologies,
especially i ual eali y, a e emphasized he e in o de o in oduce a sus ainable way o aining and educa ing
young s uden s.
Buyuksalih and his eam [6] c ea ed a h ee-dimensional space (3D) i ual model o he Incegiz ca es, whe e hey
used he me hod o scanning using a 3D scanne . They edi ed he images om he 3D scanne in he RiScan P o
so wa e, whe e hey ob ained a pho o- ealis ic isualiza ion. Toge he wi h he 3D scanne , hey used a Canon
EOS 7D Ma k II digi al came a wi h a esolu ion o 20.2 Mpx. Thanks o his con igu a ion, hey p o ided a
pano ama esolu ion o 422.5 Mpx a he en ance o he ca es. Au odesk ReCap P o so wa e was used o c ea e
he 3D model. The gene a ed models we e hen impo ed in o he Uni y3D de elopmen en i onmen and es ed
on he HTC Vi e P o VR glasses.
Sang and Xue [7] de eloped an applica ion o i ual eali y educa ion, whe e hey ocused on he na u al
en i onmen . They used a lying d one wi h a digi al came a and a GPS senso o cap u e he a ea. The collec ed
da a we e hen s o ed in a spa ial da abase, which hey used o c ea e a 3D model using pho og amme y. They
p esen ed he esul o hei wo k o s uden s who can acqui e men al pe cep ion o he ield en i onmen ,
knowledge o he ield, mas e y o skills in ield geoscience and expand he scope o knowledge.
23
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
The e a e a la ge numbe o scien i ic eams ha inco po a e i ual eali y in he p epa a ion o eaching aids in
academia. The e o e, we see a s ong need o explo e ools sui able o c ea ing i ual scenes sui able o academic
en i onmen s. The ma ke o e s a wide ange o so wa e esou ces o use and especially o he c ea ion o
scenes in i ual eali y. The main goal o ou a icle is o e alua e a numbe o ools o c ea ing VR con en and
e alua e hei p ac ical applicabili y.
The means o c ea ing he con en o i ual scenes a e pa ame ically simila on many le els, such as ha dwa e
equi emen s, use complexi y, use base, p ice, numbe o expo /impo ypes, use in e ace, manu ac u e
suppo , a ailabili y in Czech language, e c.
A he beginning o he wo k, he au ho s asked hemsel es he ollowing esea ch ques ion: “Is i possible o selec
he op imal ool o he academic en i onmen based on he compa ison o pa ame e s? Which one is he mos
sui able o academic en i onmen ?”
2 POTENTIAL OF VIRTUAL AND AUGMENTED REALITY
In o de o selec he app op ia e pa ame e s o assessing so wa e ools, we i s decided o conduc a sea ch o
meaning ul applica ions o augmen ed eali y (AR) and i ual eali y in he ield geoscience- ela ed indus ies.
Mos o hese ields ha e a simila one in he composi ion o indi idual depa men s a he Facul y o Mining and
Geology o VSB – Technical Uni e si y o Os a a.
2.1 Mining escue s aining in VR
The use o i ual eali y in he aining o mine escue s is beginning o be used in Aus alian mining [8].
Inno a ions a e e lec ed in he aining o mine escue s. The e is a s ong e o in his sec o o apply p e en i e
measu es o educe mine acciden s. The use o in o ma ion echnology, especially i ual eali y o he aining
o mine escue s and o he wo ke s in he mining indus y, is gi en a lo o space he e.
Figu e 1. VR simula o CAVE sys em [8]
The main ad an age o using i ual eali y is he simula ion o ex ao dina y e en s. These can be, o example,
mine i es, gas explosions, coal dus explosions in a mine, mine ea hquakes, gas bu s s and he escue o wo ke s
[8].
24
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
2.2 Augmen ed eali y in he ield o main enance and o mining machines
AR can also be used o machine main enance and epai . An essen ial pa o he mining si e is he main enance
o a ious ools. Wi h AR, a wo ke can ha e he da a o any machine o de ice in digi al o m. This da a can be
used o analyze he cu en si ua ion o a pa icula machine and decide on i s main enance. This will inc ease he
li e o he equipmen . Li e ideo cha s wi h emo e expe s can be a anged o consul a ion on wo king on a
speci ic ask. An expe would use a came a o see wha an AR use is seeing. Using i ual 3D holog ams, which
he expe can con ol, s/he would discuss wi h he use speci ic a eas and componen s o he machine being
epai ed [9]. The use o AR emo e assis ance ensu es he main enance and epai o machines in he mos e icien
way possible. In he e en o a b eakdown o any machine, all he in o ma ion necessa y o i s epai is se ed
di ec ly o he si e echnicians. Th ough de ailed in o ma ion and en iched wi h pic u es o pd iles, si e
echnicians can ollow s ep by s ep he p ocess o how o ix a ce ain pa . All his is done in a simple and as
way since he use o sma glasses enables hand- ee pe o mance.
Figu e 2. Augmen ed Reali y in he Mining Indus y main enance [9]
2.3 3D scanning o ob ain a i ual model o he mine
Today d ones a e used no only o en e ainmen , bu also in many a eas o he comme cial sphe e. D ones can
scan he condi ions o buildings, e ilize ields o c ea e maps in mines. I we use hese de ices, we ge a 3D
model o he mapped a ea.
Figu e 3. Ob aining a 3D model o a mine using a d one [10]
Combina ions o dep h senso s, lida and pho og amme y echniques a e used o ob ain he shape o he e ain.
D ones o e ain scanning ha e hei applica ion especially in su ace mines. Recen ly, he e ha e been se e al
cases o using a mapping d one in he co ido s o a deep mine. The use o his echnology can be expec ed mainly
in non-gassing mines. In gaseous mines wi h me hane emissions, he d one would ha e o mee s ic e
equi emen s o ATEX p o ec ion and i s p oduc ion and ope a ion would be se e al imes mo e expensi e.
25
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
2.4 Visualiza ion o a su ace mine
Fo example, i we al eady ha e a d one-scanned e ain down. We can isualize i using augmen ed o i ual
eali y and be able o se spa ial connec ions o scale much be e han on a wo-dimensional space (2D) moni o
o map d awing. I is li e ally possible o isualize hese plans e en in he middle o he li ing oom wi h he help
o glasses o augmen ed eali y o wa ching he space ia a mobile phone.
Figu e 4. Visualiza ion o a su ace mine using Hololens [11]
I we wan o ele a e he managemen mee ing o e he ope a ion and pe o mance o he mine, we can also use
ools o modeling and isualizing he ope a ion o he mine in augmen ed eali y. The en i e managemen can hen
be in oduced o pe o mance ables, plans o expanding mining, machine pe o mance, employee mo emen s
and a lo o o he in o ma ion ha is impo an o decision-making. Thanks o he ac ha in hololens glasses, all
pa icipan s can sha e one model and a he same ime see he e wha an indi idual pa icipan is looking a , i is
clea which pa o he isualized model he is alking abou . Such a model allows zooming and na iga ion using
ly- h ough and oice o ges u e con ol.
2.5 Visual suppo o mining
Map ek mining company [12] is among he i s companies using isualiza ion ia Augmen ed Reali y du ing
mining ope a ions, such as suppo in eal- ime decision-making p ocess in he opencas mine. Visual guide
assis ance h ough AR makes i possible o signi ican ly inc ease he accu acy and e iciency o d illing. When
applying AR o aid d illing p ac ices, i allows he ansmission o necessa y in o ma ion by emo e expe s.
D illing ope a o s also ha e a wide ield o in o ma ion by imp o ing he accu acy o hei wo k du ing his
di icul p ocess. Inc easing accu acy du ing his p ocess enables he educ ion o po en ial damages and he
a oidance o a ious isks.
Figu e 5. Map ek solu ion wi h augmen ed eali y [12]
26
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
2.6 Educa ional i ual ou o a coal mine
I a coal mine ends i s ac i i ies and is closed, i can s ill se e as an excellen eaching aid o u u e gene a ions.
Thanks o pho og amme y and o he ools o scanning spaces, we a e able o cap u e he en i e map o he a ea,
including ex u es, and ans e hem o he en i onmen o scenes o i ual eali y. Use s can hen en e his
scene a any ime om he com o o hei homes and explo e unde g ound spaces and iew how mining has aken
place. S uden s may be accompanied by a i ual educa o gi ing hem a p o essional explana ion, o his
explana ion may be p e- eco ded in ce ain places on he i ual scene and played when he s uden isi s i .
Figu e 6. Vi ual scene o a closed mine
Use s will begin hei sub e anean ad en u e a he su ace o he coal p epa a ion plan be o e making hei way
pas he coal mixing dome and lamp s o age, and en e ing he unde g ound ail oad sys em.
2.7 Visualiza ion o measu ed alues in isible by human eye
In he mine, which is equipped wi h a numbe o senso s o physical quan i ies and concen a ions o gases ha
canno be seen by he human eye, i is possible o display hese alues g aphically using augmen ed eali y. Some
alues o hese quan i ies can be modeled using a compu a ional luid dynamics (CFD) p og am and hei esul
hen blended wi h he eal wo ld using a sma phone o mobile phone [13].
Figu e 7. Demons a ion o AR model in he case o degassing chimney;
con ou s o mole ac ion o CH4 [13]
E icien isualiza ion ools should espec cogni i e and pe cep ual p ope ies o he human b ain. Visualiza ion
aims o imp o e he cla i y and aes he ic appeal o he displayed in o ma ion and allows a pe son o unde s and
la ge amoun s o da a and in e ac wi h i in mo e na u al way [13].

27
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
3 METHODS AND MATERIAL
The mu ual compa ison o specialized so wa e o he c ea ion o 3D models o objec s, which a e ob ained in he
eal wo ld by he me hod o pho og amme y, and he de e mina ion o he mos sui able so wa e o he academic
sphe e was pe o med using mul i-c i e ia e alua ion.
Speci ically, an app oach based on de e mining he bene i ( alue) o indi idual a ian s was applied in he
e alua ion. The bene i o he assessed a ian s is de e mined as a weigh ed sum o pa ial e alua ions o a ian s
wi h espec o indi idual c i e ia, which exp esses he ollowing ela ionship (1).
(1)
whe e:
Hj – o e all e alua ion o he j- h a ian ,
i – weigh o he i- h c i e ion,
hij – pa ial e alua ion o he j- h a ian wi h espec o he i- h c i e ion,
n – numbe o e alua ion c i e ia,
m – numbe o a ian s. [16]
The basis o e alua ing a ian s is c i e ia ma ix, see Figu e 8, which cha ac e izes he indi idual a ian s using
selec ed c i e ia. In ou case, hese 12 c i e ia we e chosen: K1 – HW equi emen s, K2 – Use complexi y, K3 –
P ice pe license, K4 – Mul ilicense a ailabili y, K5 – A ailabili y in Czechia, K6 – Numbe o expo ypes, K7
– Numbe o impo ypes, K8 – Use in e ace, K9 – Manu ac o e suppo , K10 – Use base, K11 – T ial/Demo
a ailabili y, K12 – Opensou ce.
The au ho s decided o build weigh s using he pai wise compa ison me hod, he so-called Fulle 's me hod. The
weigh o indi idual c i e ia is de e mined on he basis o he so-called Fulle 's iangle, wi hin which all c i e ia
a e compa ed. The weigh s o he c i e ia a e hen c ea ed by di iding he numbe o p e e ences o he indi idual
c i e ia and he o al sum o p e e ences.
In o de o he p e e ences in pai wise compa ison o be less a ec ed by he subjec i e pe cep ion o indi idual
c i e ia, he au ho s app oached 10 expe s who deal wi h he gi en issue. Indi idual p e e ences in pai wise
compa ison a e he esul o b ains o ming.
Fo he c i e ion o HW equi emen s, a ypical PC se o he academic sphe e was selec ed, cu en as o 09/2021
acco ding o in e nal DNS egula ions. Speci ically, hese a e he ollowing echnical speci ica ions: CPU Ma k a
leas 13000 poin s, o example AMD Ryzen5 2600, 16GB RAM, 512GB SSD + 1TB HDD, ex e nal g aphics
ca d 4GB.
The e a e many so wa e ools o c ea ing 3D models sui able o use in VR. The se o es ed so wa e was c ea ed
a e consul a ion wi h expe s in VR and 3D modeling, and he ac ual es ing o ools on a eal p ojec . So he
ollowing so wa e ools came ou o he whole ange.
3.1 3DF Zephy
3DF Zephy is a comme cial pho og amme ic and 3D modeling so wa e. I is de eloped by he I alian 3DFLOW
eam, which i s launched i in 2014 and has since been con inuously upda ed un il he cu en e sion 6.009,
which is cu en as o Oc obe 2021. I is a comple e pho og amme ic SW package, which con ains a numbe o
ools o he mos accu a e p ocessing o 3D models. I enables 3D econs uc ion no only om pho os, bu also
om ideo by au oma ically so ing ou indi idual usable images. 3DF Zephy only wo ks unde Windows Vis a
and la e .
28
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
3.2 Reali y Cap u e
I is a so wa e ool ha c ea es 3D models om uno de ed pho os. I s mos common a eas o cu en use a e
he i age si es, ull body scanning, he gaming indus y and VR c ea ion. The main ad an age o his SW is he
au oma ic calib a ion o pho og aphs, assignmen o unambiguous coo dina es in space, e y iendly and in ui i e
en i onmen . You can also use he command line and o he so wa e de elopmen ools. I is also possible o mix
pho o images wi h lase scans. C ea ed 3D models can be expo ed o majo o ma s such as OBJ, XYZ, DXF,
DAE, e c. [14]
3.3 Au odesk ReCap P o
I is a ool o in elligen con e sion om pho os and scans o 3D da a using a poin cloud. I is possible o
pho og aph o scan a i ac s, buildings, elie s, e en he whole ci y. Da a is s o ed in a poin cloud (millions o
poin s) and Au odesk ReCap P o p ocesses ha amoun o da a, including isualiza ion measu emen s and
p epa a ion o o he Au odesk p og ams such as Au oCAD, Re i , o Au odesk In en o . E e y hing akes place
in he ollowing ou phases. [15]
Scanning, whe e da a om a lase 3D scanne is s o ed in a poin cloud. Impo , whe e Au odesk ReCap P o
con e s scanned iles o Reali y Cap u e Scan (RCS) o ma o o he Au odesk p og ams. Du ing his indexing,
he aw da a is con e ed o Au odesk ReCap P o ( he numbe o poin s can be se a he beginning). The hi d s ep
is o ganiza ion, cleanup, and analysis, whe e a ious me hods a e a ailable o o ganize he indexed da a and dele e
o hide i in he poin cloud. The scene can hen be iewed in a ious colo modes and ligh ing. Finally, he e is
he connec ion phase, whe e indexed da a om scans and p ojec s can be opened o connec ed o o he p og ams
o c ea e a ealis ic 3D model. ASC, CL3, CLR, E57, FLS, FWS, ISPROJ, LAS, PCG, PTG, PTS, PTX, RDS,
TXT, XYB, XYZ, ZFS, and ZFPRJ iles can be indexed. Au odesk ReCap P o hen c ea es RCS (one uni o
measu emen ile), RCP (p ojec ile), and PCG, PTS, and E57, OBJ expo o ma s.
3.4 Mesh oom
Mesh oom is ee open sou ce so wa e ha has an in ui i e en i onmen and is easy o use. I includes a la ge
numbe o se ings, hanks o which a e y good esul o he econs uc ed building can be achie ed. Pho og aphs
a e p ocessed using he pho og amme ic amewo k o compu e ision AliceVision. This amewo k is he esul
o collabo a ion be ween se e al uni e si ies, labo a o ies and Mik os Images. The Czech Technical Uni e si y in
P ague is one o he uni e si ies in ol ed in he de elopmen .
Gene a ing pho og amme ic models is easy and in ui i e he e. Li e ally, i is jus a ma e o agging he sou ce
pho os, d agging hem in o he so wa e window and hen clicking o s a he p ocess. O cou se, he e a e g ea
op ions o se ing up he model c ea ion p ocess, while i is possible o achie e a highe quali y o he esul ing
model compa ed o au oma ic se ing. Mesh oom is a ailable o Windows, OSX and Linux ope a ing sys ems. I
equi es CUDA lib a y suppo – so i is sui able o machines wi h nVidia g aphics ca ds.
The so wa e allows you o choose be ween h ee modes o model c ea ion. S anda d econs uc ion is sui able in
cases whe e we ha e a se o pho os, wi hou he possibili y o aking and adding mo e pic u es. Ex ended
econs uc ion is app op ia e in a si ua ion whe e we ha e comple ely aken all pho os, bu i necessa y, i is
possible o add mo e pho os. This op ion is especially use ul in cases whe e we wan o imp o e an al eady inished
model. The inse ed pho os a e au oma ically synch onized wi h he al eady used pho os, so ha , o example, a
blu ed o missing space o he model can be illed.
Li e Recons uc ion mode is con olled by pho os inse ed in he sou ce olde a he ime o shoo ing. Thus, a li e
econs uc ion g adually c ea es a 3D model o a scanned objec o space. The use o his op ion is ideal i we do
no know how many pho os a e needed o c ea e he desi ed quali y o he c ea ed model, o i we equi e he bes
possible esul and we can adjus he se ings o he scanning de ice.
29
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
3.5 Ben leyCon ex Cap u e
I is a p op ie a y so wa e o c ea ing 3D models using he me hod o pho og amme y. I also p o ides he abili y
o use he so wa e as a se ice, so he use does no need o ha e hei own powe ul modeling de ice. I is mo e
use -in ensi e so wa e o mo e ad anced use s. I con ains a la ge numbe o adjus able op ions o achie e he
bes possible inal model. The so wa e is sui able o la ge eams and p ojec s ha can con ain esolu ions up o
300GPx. Howe e , he pu chase p ice co esponds o his, which makes i a ool only o p o essional use, as he e
is no ee e sion.
3.6 Agiso Me ashape
The so wa e is a solu ion o pho og amme ic p ocessing o digi al images. Ou pu 3D da a can be used o wo k
in geog aphic in o ma ion sys em (GIS) applica ions, o documen a ion o cul u al he i age, o o c ea ing isual
e ec s. I is an au oma ed da a p ocessing sys em ha is sui able o bo h no ices in he ield o pho og amme y
and o p o essionals, which allows hem o se pa ame e s o speci ic asks and di e en ypes o da a.
3.7 Pho omodele
I is a so wa e applica ion ha pe o ms modeling based on pho og aphs and nea - ange s e eo pho og amme y
– i c ea es 3D models and measu emen s om pho og aphs. This so wa e is used o nea and ae ial
pho og amme y. Close- ange pho og amme y (CRP) can mean pho og aphs a e aken om he g ound wi h a
hand-held came a o aken om a UAV / d one a a ela i ely low al i ude. Pho o modele and CRP a e used o
pe o m measu emen s and modeling in ag icul u e, a cheology, a chi ec u e, biology, enginee ing,
manu ac u ing, ilm p oduc ion, o ensics, mining, in en o y olumes.
3.8 Rega d3D
I is an open sou ce applica ion ha is eely a ailable unde he MIT license. I was de eloped as a hobby p ojec
by a Swiss p og amme who decided o make i a ailable o he public ee o cha ge. The p og am d aws on open
sou ce lib a ies, which i combines in a ious ways o achie e he bes possible esul . Applica ion de elopmen
was suspended in 2019 and has no been upda ed since.
4 RESULTS
The s a ing poin o pe o ming a mul i-c i e ia e alua ion is he c i e ia ma ix, see Figu e 8.
Figu e 8. C i e ia Ma ix
30
GeoScience Enginee ing Vol. 68 (2022), No. 1
geoscience.cz pp. 22–32, ISSN 1802-5420
DOI 10.35180/gse-2022-0066
As we can see in Figu e 8, he c ea ed se o c i e ia consis s o quan i a i e and quali a i e c i e ia. In addi ion,
he e a e c i e ia o di e en uni s. In he case o quan i a i e e alua ion, he e alua ion was pe o med using a
scale in which 5 poin s indica e ha he le el o ul illmen o he gi en c i e ion is ully ensu ed and, con e sely,
1 poin indica es ha he le el o ul illmen does no mee he gi en c i e ion.
In o de o be able o exp ess he u ili y o indi idual a ian s on he basis o inpu da a, i is necessa y o ans o m
he c i e ion ma ix in o he so-called simple u ili y ma ix. (Figu e 9), which eplaces absolu e alues wi h ela i e
alues. We pe o m his ans o ma ion by de e mining he mos ad an ageous a ian o each c i e ion and
assigning he alue 100. Subsequen ly, he o he alues o he indi idual c i e ia will be ecalcula ed.
The esul ing sum is hen a pa ial e alua ion o he assessed a ian s, because he indi idual c i e ia ha e he same
weigh .
Figu e 9. Simple U ili y Ma ix
The weigh s o he e alua ion c i e ia we e de e mined in acco dance wi h he me hodology using he Fulle
iangle.
Figu e 10. De e mina ion o Weigh s