5 h Wo kshop on Replica ion
in Ex ended Reali y (WoRXR)
P og am
•14:00 - 14:15 Welcome
•14:15 - 15:00 Ensu ing Replicabili y: Da a S o age & Digi al Appendices.
Tu o ial by Flo ian Weidne
•15:00 - 15:30: Co ee B eak
•15:30 - 16:00: Keyno e - Beyond Ini ial Findings: Replica ion, Reliabili y,
and S a is ical P ac ices in Empi ical S udies by Daniel Zielasko
•16:15 - 16:30: Pe cep ion Th esholds o Unob usi e Mo emen in VR
Ve sus AR by Ve ena Biene
•16:30 - 17:00: B ains o ming and agenda se ing, w ap-up
Why?
•Failu e o eplica e many published indings, e en ex book
indings
•Resea ch biases
•Publica ion bias: only signi ican esul s ge published
•Selec ion bias: only signi ican esul s analysed
•Repo ing bias: only signi ican esul s epo ed in pape
•Replica ions?
•Ra ely unded
•Ra ely published
→No incen i e; mos esea ch explo a o y
… a IEEE ISMAR and IEEE VR?
Mohammed Sa aye A e in, Ve ena Biene , Nijhum Hasan S M Rashidul, Flo ian Weidne , Jens G ube , and J. Edwa d Swan. 2025. The S a e o Replica ion a IEEE
ISMAR and IEEE VR: A Scoping Li e a u e Re iew (2010 –2024) and Online Su ey. In 2025 IEEE In e na ional Symposium on Mixed and Augmen ed Reali y, 2025.
•14:00 - 14:15 Welcome
•14:15 - 15:00 Ensu ing Replicabili y: Da a S o age & Digi al
Appendices. Tu o ial by Flo ian Weidne
•15:00 - 15:30: Co ee B eak
•15:30 - 16:00: Keyno e - Beyond Ini ial Findings: Replica ion,
Reliabili y, and S a is ical P ac ices in Empi ical S udies by Daniel
Zielasko
•16:15 - 16:30: Pe cep ion Th esholds o Unob usi e Mo emen in
VR Ve sus AR by Ve ena Biene
•16:30 - 17:00: B ains o ming and agenda se ing, w ap-up
Replicabili y and da a/algo i hm
s o age and a ailabili y
Flo ian Weidne , Uni e si y o Glasgow
Ou line o he nex 30-45 minu es
•Why publish da a and algo i hms?
•Wha could (should) we s o e?
•Whe e o s o e?
•Zenodo
•OSF
•ACM and IEEE
•How do we documen i ?
•Summa y
Why publish da a?
Rep oducibili y and eplicabili y. Fo example:
•P o o ype sha ed: es exac ly he same.
•Da a sha ed: calcula e exac ly he same
•E e y hing: do exac ly he same.
Bu also:
•Compa ison wi h new indings easie (s a is ics)
•Making imp o emen s is easie (i e a ions)
Rep oducibili y: eanalysing exis ing da a in he same manne . Replicabili y: econduc ing he analysis, including da a collec ion.
… do s udy …
OSF Wo k low
Add da a.
OSF commen s
•F ee
•Founda ion well- unded (long- e m pe spec i e)
•P e egis a ion
•Anonymous links
•Ve sioning possible
•50GB
•Links o gi hub elease
→no ile mi o .
•Co e s esea ch li ecycle
h ps://gi hub.com/eblondel/zen4R
Tl;d : OSF
1. C ea e and ill ou p e- egis a ion.
2. Do s udy.
3. Upload da a, analy ics code, ma e ials, pape s, and
supplemen al.
4. C ea e iew-only links (p i a e o no )
5. Add hem o manusc ip .
6. Be happy ☺
Example: IEEE VR 24 submission:
h ps://os .io/u 46b/
Zenodo
All abou s o age.
Zenodo UI
•Signup / login
•C ea e new upload
•Ge DOI
•Fill in me ada a
•Upload iles
•Re iew
•Publish
➔s aigh owa d
Example
P i a e?
1. C ea e anon
eposi o y
2. Publish es ic ed
3. C ea e link
4. Sha e link
(change au ho s la e )
Commen s on Zenodo
•F ee
•Funded by CERN (long- e m pe spec i e)
•Ve sioning possible
•50GB
•Links o gi hub elease
•au o-upda e on new elease
•Nice gi hub Zenodo badge
h ps://gi hub.com/eblondel/zen4R
ACM / PCS / TAPS and IEEE
Summa y
All oge he ?
•OSF: all ounde , ee, p e- egis a ion, 50GB = mo e wo k?
•Zenodo: Gi hub, ee, 50GB, no p e- egis a ion
•ACM / PCS / TAPS: F ee, limi ed ile size
unde ined
unde ined
Wha now?
•Make ma e ials a ailable. My (so ) equi emen s
•S ipped execu able + se up ins uc ions
•S udy p o ocol and sel -de ined ques ions
•Raw da a (i possible), agg ega e da a + sc ip s o calcula e esul s
•Wha we should do: OSF wi h p e- eg ( ime, hough)
•Wha I ac ually do:
•Submission:
•Upload mos impo an s u o PCS
•Rese e and link Zenodo/DOI in pape in acknowledgemen s (anon)
•A e accep ance
•Da a, sc ip s, p o o ype, pape , and ideo on Zenodo
•Link/DOI in pape
Fo aspi ing academics
… i makes a nice pa o you academic c o applica ions
Thank you! Ques ions?
