Investigating the performance of the order-picking process by using smart glasses: A laboratory experimental approach

Chond oma idis, Nikolaos; Gialos, Anas asios; Zeimpekis, Vasileios A icle In es iga ing he pe o mance o he o de -picking p ocess by using sma glasses: A labo a o y expe imen al app oach Logis ics P o ided in Coope a ion wi h: MDPI – Mul idisciplina y Digi al Publishing Ins i u e, Basel Sugges ed Ci a ion: Chond oma idis, Nikolaos; Gialos, Anas asios; Zeimpekis, Vasileios (2022) : In es iga ing he pe o mance o he o de -picking p ocess by using sma glasses: A labo a o y expe imen al app oach, Logis ics, ISSN 2305-6290, MDPI, Basel, Vol. 6, Iss. 4, pp. 1-26, h ps://doi.o g/10.3390/logis ics6040084 This Ve sion is a ailable a : h ps://hdl.handle.ne /10419/310293 S anda d-Nu zungsbedingungen: Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen Zwecken und zum P i a geb auch gespeiche und kopie we den. Sie dü en die Dokumen e nich ü ö en liche ode komme zielle Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich machen, e eiben ode ande wei ig nu zen. So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen (insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en, gel en abweichend on diesen Nu zungsbedingungen die in de do genann en Lizenz gewäh en Nu zungs ech e. Te ms o use: Documen s in EconS o may be sa ed and copied o you pe sonal and schola ly pu poses. You a e no o copy documen s o public o comme cial pu poses, o exhibi he documen s publicly, o make hem publicly a ailable on he in e ne , o o dis ibu e o o he wise use he documen s in public. I he documen s ha e been made a ailable unde an Open Con en Licence (especially C ea i e Commons Licences), you may exe cise u he usage igh s as speci ied in he indica ed licence. h ps://c ea i ecommons.o g/licenses/by/4.0/ Ci a ion: Chond oma idis, N.; Gialos, A.; Zeimpekis, V. In es iga ing he Pe o mance o he O de -Picking P ocess by Using Sma Glasses: A Labo a o y Expe imen al App oach. Logis ics 2022,6, 84. h ps://doi.o g/ 10.3390/logis ics6040084 Academic Edi o : Robe Hand ield Recei ed: 24 Oc obe 2022 Accep ed: 1 Decembe 2022 Published: 8 Decembe 2022 Publishe ’s No e: MDPI s ays neu al wi h ega d o ju isdic ional claims in published maps and ins i u ional a il- ia ions. Copy igh : © 2022 by he au ho s. Licensee MDPI, Basel, Swi ze land. This a icle is an open access a icle dis ibu ed unde he e ms and condi ions o he C ea i e Commons A ibu ion (CC BY) license (h ps:// c ea i ecommons.o g/licenses/by/ 4.0/). logis ics A icle In es iga ing he Pe o mance o he O de -Picking P ocess by Using Sma Glasses: A Labo a o y Expe imen al App oach Nikolaos Chond oma idis, Anas asios Gialos and Vasileios Zeimpekis * Depa men o Financial and Managemen Enginee ing, School o Enginee ing, Uni e si y o he Aegean, 82100 Chios, G eece *Co espondence: [email p o ec ed] Abs ac : Backg ound: O de picking p ocess is c i ical o accu a e and e icien o de ul ilmen . Pick- by- ision is a p omising echnology ha may suppo o de picking p ocess, howe e he e is s ill a limi ed amoun o esea ch conce ning he impac o his echnology on he pe o mance o o de - picking. The pu pose o his pape is o in es iga e ce ain ope a ional and echnical pa ame e s ha a ec he pe o mance o pick-by- ision echnology in i em-le el o de picking ia a se ies o labo a o y es s. Me hods: A sys ema ic li e a u e e iew is conduc ed o he iden i ica ion o pa ame e s ha a ec pick-by- ision pe o mance. Subsequen ly, he analy ical hie a chy p ocess is adop ed o ank hese pa ame e s, conce ning hei impac on o de picking. Then, he design o expe imen and NASA ask load index me hodology a e implemen ed o assessing pick-by- ision e iciency and pe cei ed wo kload. Resul s: The esul s e eal he pa ame e s ha signi ican ly a ec he pe o mance o he pick-by- ision sys em, as well as he bes con igu a ion o pa ame e s o he implemen a ion o he p oposed sys em in eal en i onmen s. Conclusions: The esul s ob ained a e encou aging, showing how pick-by- ision echnology can suppo o de picking e iciency. Fu he mo e, p ac ical implica ions a e p esen ed ha deal wi h he o ganiza ional cul u e, p ocess e-enginee ing, s a esis ance o change, and mo i a ion o main aining he new way o execu ing o de -picking p ocesses. Keywo ds: pick by ision; sys ema ic li e a u e e iew; analy ical hie a chy p ocess; design o expe imen s; labo a o y es ing 1. In oduc ion The inc ease o e-comme ce sales, he globaliza ion o ade, he cus ome s’ demand o equen and low- olume o de s, and he need o as e esponse imes a e he main ac o s ha inc ease he complexi y o logis ics p ocesses [ 1 , 2 ]. The managemen o he a o emen ioned challenges and he op imiza ion o wa ehouse ope a ions, coupled wi h logis ics cos educ ion, a e complica ed asks o wa ehouse manage s o cope wi h, because mos wa ehouses a e manually o semimanually ope a ed, esul ing in deli e ing labo -in ensi e se ices o hei cus ome s [ 3 ]. Once he p ocesses o a s anda d wo k low in a wa ehouse has been aken in o accoun , i can be a gued ha o de picking con ibu es signi ican ly o logis ics cos s and cus ome se ice [ 1 , 3 , 4 ]. Indeed, in manual wa ehouses, he o de -picking p ocess emb aces he mos labo -in ensi e ope a ion, esul ing om 55% o 65% o he o al ope a ional wa ehouse cos [ 5 ], while in au oma ed wa ehouses, he o de -picking cos becomes capi al in ensi e because o he high in es men cos [ 6 ]. This is he main a gumen o logis ics p o essionals who p io i ize wa ehouse imp o emen s by ocusing mainly on he o de -picking p ocess. Focusing on he de elopmen o in o ma ion and communica ion echnology (ICT) and a numbe o o he echnologies (e.g., pick by ligh , pick by oice, ing scanne s, augmen ed eali y, RFID, e c.) ha e al eady been adop ed o o de picking. These ech- nologies digi alize he adi ional pape -based picking lis , acili a ing in ha way he Logis ics 2022,6, 84. h ps://doi.o g/10.3390/logis ics6040084 h ps://www.mdpi.com/jou nal/logis ics Logis ics 2022,6, 84 2 o 26 ul illmen o mode n cus ome needs o inc eased e iciency and accu acy [ 7 ]. Pick by ision h ough sma glasses is an inno a i e solu ion ha may imp o e bo h ime e i- ciency and o de -picking accu acy [ 8 ]. Acco ding o S ol z e al. [ 9 ], pick by ision uses wea able echnology and p o ides a as and hands- ee solu ion o he execu ion o he o de -picking p ocess. This inno a i e echnology combines he bes o ision-guided pick- ing so as o p oduce a mo e e icien and mo e accu a e ope a ion beyond he con en ional o de -picking echnologies [7]. Despi e he gene al imp ession ha pick by ision is a p omising o de -picking ech- nology, he e is s ill a limi ed amoun o esea ch conce ning he impac o his ech- nology on he pe o mance o he o de -picking p ocess, which makes i di icul o de i e solid esul s and make p ac ical ecommenda ions. Indeed, on he basis o he a ailable li e a u e [ 10 – 13 ], a signi ican numbe o s udies ha e deal wi h he op imiza ion o he o de -picking p ocess, by conside ing a se ies o s a egic, ac ical, and ope a ional pa ame e s, bu only a small numbe o s udies ha e conside ed he pa ame e s ha a ec o de -picking accu acy and e iciency du ing he design and op imiza ion phase o he o de picking p ocess [ 14 ]. Indeed, in [ 14 ], he au ho s ake in o accoun ou pa ame e s: (a) display holde , (b) ield o iew, (c) ba code ype, and (d) exis ence o con i ma ion, wi h- ou , howe e , conside ing o he , equally impo an pa ame e s ha deal wi h e gonomic pa ame e s, such as ba e y posi ion, and o de -p o ile pa ame e s, such as o de lines pe o de , i ems pe o de line, e c. To his end, we a gue ha he e is s ill a need o u he in es iga ion in pa ame e s ha a ec o de picking, in o de o ho oughly analyze hei impac on o de -picking p oduc i i y, e iciency, and ope a ional cos . To his end, he aim o his pape is o in es iga e ce ain pa ame e s ha a ec he ope a ional pe o mance o a pick-by- ision sys em ia a se ies o labo a o y es s. Ini ially, 27 pa ame e s and h ee pe o mance measu emen indices a e iden i ied o he pick-by- ision sys em design, de elopmen , and es ing by adop ing a sys ema ic li e a u e e iew (SLR) me hodology. Six pa ame e s a e selec ed ia an analy ical hie a chy p ocess (AHP) in o de o in es iga e hei impac on he p oposed pick-by- ision sys em. Subsequen ly, a se ies o labo a o y es s a e conduc ed by adop ing he design o expe imen s (DoE) me hodology. The p oposed pick-by- ision sys em is in es iga ed o assess i s o de -picking ime and wo kload. The pe cei ed wo kload o he pick-by- ision sys em is e alua ed ia NASA TLX su ey. The emainde o his pape is s uc u ed as ollows. Sec ion 2p esen s he indings om he li e a u e e iew and desc ibes he selec ed pa ame e s o he e alua ion o he p oposed pick-by- ision sys em. Sec ion 3p esen s he necessa y s eps ha ha e been adop ed o he design o he labo a o y expe imen s. In Sec ions 4and 5, he esul s o he labo a o y es s and he discussion o he indings, coupled wi h heo e ical con ibu ion and p ac ical implica ions, a e p esen ed. Finally, Sec ion 6concludes he pape . 2. Iden i ica ion and Selec ion o Pa ame e s 2.1. Subsec ion The sys ema ic li e a u e e iew (SLR) me hodology, coupled wi h a se ies o esea ch ques ions (RQs), was adop ed wi h he aim o iden i ying, de ec ing, and ca ego izing he pa ame e s ha a ec he design and ope a ion o he p oposed pick-by- ision sys em. Acco ding o Table 1, a o al o ou esea ch ques ions we e de eloped. In o de o answe he abo e RQs, we use he sys ema ic li e a u e e iew (SLR) me hod. Mo e speci ically, we ollowed a h ee-s ep p o ocol based on p e ious p ominen a icles [ 15 – 21 ], in o de o come up wi h eliable and p o en wo k. The s eps o selec ing p o ocol a e desc ibed as ollows. Logis ics 2022,6, 84 3 o 26 Table 1. Resea ch ques ions o he iden i ica ion, de ec ion, and ca ego iza ion o pa ame e s ha a ec he design and ope a ion o he p oposed pick-by- ision sys em. No Desc ip ion o Resea ch Ques ion RQ1 Which a e he main pa ame e s ha should be aken in o conside a ion o a pick-by- ision de ice pa ame e iza ion (sys em design)? RQ2 Which a e he main pa ame e s ha should be aken in o conside a ion o he e alua ion and op imiza ion o pick-by- ision sys ems in e ms o ope a ional pe o mance? RQ3 Which a e he main pa ame e s ha should be aken in o conside a ion o he e alua ion and compa a i e assessmen o pick-by- ision echnology in e ms o o de p o ile? RQ4 Which a e he main pe o mance measu emen indices and side e ec s ha should be aken in o conside a ion o he e alua ion pick-by- ision echnology in e ms o pe o mance and e gonomics? In he i s s ep, a se ies o sea ch e ms/keywo ds and induc ion c i e ia we e de- e mined, in o de o conduc comp ehensi e esea ch (Table 2). The esea ch ocused on pape s published in pee - e iewed jou nals, a in e na ional con e ences, in disse a ions and in echnical epo s on he ield o logis ics. The main eason o including a icles om in e na ional con e ences, disse a ions, and epo s in his wo k is ha he num- be o pape s in pee - e iewed jou nals ha deal wi h he e alua ion o pick-by- ision sys ems is limi ed. Table 2. Inclusion c i e ia o implemen ing SLR me hodology. Inclusion C i e ia Desc ip ion Sea ch e ms/Keywo ds Vision picking, pick-by- ision, wea able echnology, wea able compu e s, o de picking, augmen ed eali y, head-moun ed displays, sma glasses, use in e ace, logis ics Sou ce ypes (a) pee - e iewed jou nals, (b) in e na ional con e ences, (c) epo s, (d) disse a ions Language English In he second s ep, a e iew o selec ed a icles ( om S ep 1) ook place on he basis o he i les and abs ac s o he a icles. Du ing his e iew, a se ies o a icles ou o he esea ch scope was excluded om ou lis . Mo e speci ically, 46 s udies ocused on di e en aspec s o pick-by- ision echnology and ields. A e he comple ion o his s ep, he emaining numbe o a icles was 31. In he las s ep, he eading o ull e sions o a ailable s udies led o he inal selec ion o he lis o s udies o be conside ed. Eigh s udies we e excluded, and by implemen ing he snowballing me hod, we we e able o add o ou ini ial lis some addi ional pape s, epo s, and disse a ions ha me he inclusion c i e ia. The inal co pus in ol ed 66 s udies. The la e we e e iewed in o de o iden i y he key pa ame e s ha a ec he design and implemen a ion o a pick-by- ision sys em. 2.2. Desc ip i e Analysis o he Co pus A e he esul s o SLR we e aken in o accoun , i was e ealed ha 44.6% o he e iewed s udies we e coming om jou nals, 40% om con e ences, 4.6% om epo s, and 10.8% om disse a ions. The small numbe o published s udies and he e o e he limi ed numbe o jou nal a icles we e ep esen a i e signs ha he ield is qui e new, om a esea ch poin o iew. The la e seems o be con i med i we ake in o accoun he ime dis ibu ion o he e iewed s udies. Gi en he esul s o he ime dis ibu ion o he e iewed s udies, i is e iden ha he yea s o publica ion among he iden i ied publica ions a y om 2001 o 2021. The numbe o s udies ac o ing in he design, de elopmen , and es ing o pick-by- ision echnology has apidly g own du ing he pas se e al yea s. Almos , hal o he Logis ics 2022,6, 84 4 o 26 conside ed s udies we e published wi hin he pas ou yea s, indica ing ha he a ea has been signi ican ly expanding o e he pas ew yea s. The peak in he numbe o s udies is obse ed du ing he wo-yea pe iod om 2018 o 2020, when 27 s udies we e published. Acco ding o a geog aphical analysis o he s udy a eas, 70.5% o he s udies we e conduc ed in Eu ope and 25.6% on he Ame ican con inen s. The majo i y o he Eu opean s udies we e conduc ed in Ge many (60% o he s udies). Fu he mo e, 7.3% o he s udies we e conduc ed in G eece, same as Sweden, and Slo enia is nex , a 5.4%. The emain- ing 20% o he s udies we e conduc ed in a ious o he Eu opean coun ies, as shown in Figu e 1. Logis ics 2022, 6, x FOR PEER REVIEW 4 o 26 is obse ed du ing he wo-yea pe iod om 2018 o 2020, when 27 s udies we e published. Acco ding o a geog aphical analysis o he s udy a eas, 70.5% o he s udies we e conduc ed in Eu ope and 25.6% on he Ame ican con inen s. The majo i y o he Eu opean s udies we e conduc ed in Ge many (60% o he s udies). Fu he mo e, 7.3% o he s udies we e conduc ed in G eece, same as Sweden, and Slo enia is nex , a 5.4%. The emaining 20% o he s udies we e conduc ed in a ious o he Eu opean coun ies, as shown in Figu e 1. Figu e 1. Geog aphical dis ibu ion o he e iewed s udies. 2.3. Iden i ica ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems A e he s uc u e o he basic esea ch ques ions had been inalized, he iden i ied pa ame e s we e classi ied in ou ca ego ies: (a) pa ame e s o sys em design, (b) pa ame e s o sys em e alua ion in e ms o ope a ion, (c) pa ame e s o sys em e alua ion in e ms o o de p o ile, and (d) pe o mance measu emen indices ha can be used o he e alua ion o pick-by- ision echnology. All he e iewed pa ame e s pe ca ego y a e p esen ed in Table 3. Table 3. O e iew o he e iewed pa ame e s o he design and in es iga ion o he p oposed sys em. Ca ego y Subca ego y Pa ame e Re e ences A. Pa ame e s o sys em design E gonomic aspec s Display Posi ion [22–24] Display Type [24–27] In e ac ion De ice [24,25,28–31] Ba e y Posi ion [27] Display Holde [22,30,32] Scanne Posi ion [9,32–35] Weigh o Equipmen [27,34,36–38] Visualiza ion aspec s Field o View [26,30,39–41] Focal Dis ance [26,39,42] Visualiza ion Op ics [29,38,41,43,44] In o ma ion Mode [8,22,25,26,45,46] In o ma ion A ailabili y [25,34] Display View [9,33,37,47,48] Exis ence o AR [45,46,49] Di ec ion In e ace [28,32,50–52] 1 1 1 1 1 33 4 1111131 4213 18 Figu e 1. Geog aphical dis ibu ion o he e iewed s udies. 2.3. Iden i ica ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems A e he s uc u e o he basic esea ch ques ions had been inalized, he iden i ied pa ame e s we e classi ied in ou ca ego ies: (a) pa ame e s o sys em design, (b) pa am- e e s o sys em e alua ion in e ms o ope a ion, (c) pa ame e s o sys em e alua ion in e ms o o de p o ile, and (d) pe o mance measu emen indices ha can be used o he e alua ion o pick-by- ision echnology. All he e iewed pa ame e s pe ca ego y a e p esen ed in Table 3. Table 3. O e iew o he e iewed pa ame e s o he design and in es iga ion o he p oposed sys em. Ca ego y Subca ego y Pa ame e Re e ences A. Pa ame e s o sys em design E gonomic aspec s Display Posi ion [22–24] Display Type [24–27] In e ac ion De ice [24,25,28–31] Ba e y Posi ion [27] Display Holde [22,30,32] Scanne Posi ion [9,32–35] Weigh o Equipmen [27,34,36–38] Logis ics 2022,6, 84 5 o 26 Table 3. Con . Ca ego y Subca ego y Pa ame e Re e ences A. Pa ame e s o sys em design Visualiza ion aspec s Field o View [26,30,39–41] Focal Dis ance [26,39,42] Visualiza ion Op ics [29,38,41,43,44] In o ma ion Mode [8,22,25,26,45,46] In o ma ion A ailabili y [25,34] Display View [9,33,37,47,48] Exis ence o AR [45,46,49] Di ec ion In e ace [28,32,50–52] Display Se ings [25,26,29] Technical aspec s Ba code Type [9,35,53–55] Scanning Dis ance [8,44,56–58] Ba e y Li e [27,38,59–61] Exis ence o T acking Sys em [32,40] Con i ma ion Equipmen [28,34,36,41,62] B. Pa ame e s o sys em e alua ion in e ms o ope a ion Picking S a egy [63,64] Handling Uni [32,57] Exis ence o Con i ma ion [14,28,29,37,65] C. Pa ame e s o sys em e alua ion in e ms o o de p o ile Numbe o O de s [33,36,37,47,49,56,59,66] Lines pe O de [14,24,33,37,47,56,64,66–68] I ems pe Line [14,32,33,37,47,49,56,64,66] D. Pe o mance measu emen indices E iciency (Time) [9,25,34,47,62,65] Accu acy [23,25,34,39,47,65] Wo kload [30,33,34,37,41,42,68] The i s ca ego y comp ises he de ice design and de elopmen o he p oposed sys em and includes 21 e iewed pa ame e s. Because o he high numbe o pa ame e s in his ca ego y, hey we e u he classi ied in o h ee dis inc i e subca ego ies. The i s subca ego y deals wi h he e gonomic aspec s and in ol es se en pa ame e s, he second one ocuses on isualiza ion aspec s and includes nine pa ame e s, and he hi d subca ego y is associa ed wi h echnical aspec s and encompasses i e pa ame e s. The e gonomic aspec s o de ice pa ame e iza ion play a c i ical ole du ing he design and de elopmen o he sys em in ha hey deal wi h pa ame e s ha de ine how com o able a wo ke would eel while using he sys em. A c ucial issue is ha he wo ke has o wea he equipmen needed o a pick-by- ision sys em du ing a shi . To his end, he pick-by- ision equipmen mus be ligh , e gonomically designed, and sa e and mus ha e an eigh -hou ba e y li e [ 32 ]. The isualiza ion aspec s du ing he design and de elopmen o pick-by- ision echnology deal wi h he g aphical use in e ace (GUI) o he de ice. Indeed, one o he mos impo an ea u es o a pick-by- ision sys em is he GUI, because he i ual in o ma ion mus be displayed in he lens o he glasses a he igh ime and a he igh posi ion [ 49 , 59 ]. Ne e heless, he display o necessa y in o ma ion (i.e., s ock loca ion, a icle numbe , goods desc ip ion, equi ed quan i ies, e c.) on picke s’ glasses o headbands does no always e icien ly appea , because o a ious p oblems, such as eye s ain, di icul ies seeing he display image, eye pain, eye concen a ion p oblems, and headaches, which ha e been obse ed du ing he es ing o pick-by- ision echnology [ 23 ]. The echnical aspec s ocus mainly on he ha dwa e used Logis ics 2022,6, 84 6 o 26 du ing pick by ision. This ca ego y is i al du ing he design and de elopmen o pick-by- ision sys ems because i includes pa ame e s ha a ec he abili y o a sys em o ead and ecognize he p oduc s, o moni o he p ocess and di ec he o de picke s, and o ensu e o de -picking-p ocess accu acy. The second ca ego y deals wi h h ee pa ame e s ha enable he ope a ional pe o mance o he sys em o be e alua ed. Acco ding o he a ailable li e a u e [ 4 , 22 , 28 , 29 , 37 , 57 , 63 – 65 ], his ca ego y includes a se ies o pa ame e s (e.g., picking s a egy, handling uni , e c.) ha can be used by in es iga o s when hey assess he ope a ion pe o mance o he sys em in he ield. The hi d ca ego y includes h ee pa ame e s ha a e used o he compa ison as- sessmen o he p oposed sys em wi h o he picking echnologies. This ca ego y de- sc ibes a se ies o ac o s, such as he numbe o o de s, lines pe o de , e c., ha can be used in o de o compa e he pick-by- ision sys em wi h o he con en ional o de - picking echnologies and sys ems (e.g., ligh picking, oice picking, RF-scanning picking, e c.) [14,24,33,37,47,56,64,66–68]. Las ly, he ou h ca ego y includes h ee pe o mance measu emen indices and deals wi h he inal ou pu o he e alua ion p ocess. These indices a e used by p o essionals and academics o e alua e pick-by- ision echnology and o he o de -picking echnologies in o de o compa e hem [23,25,34,39,47,65]. 2.4. Selec ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems A e aking in o accoun he esul s o he SLR, we implemen ed he p ede ined s eps o an analy ical hie a chy p ocess (AHP) [ 69 ] in o de o iden i y he mos impo an pa ame e s ha a ec a pick-by- ision sys em. All he e iewed pa ame e s we e alida ed om expe s’ opinions on i e aspec s: e gonomic aspec s (EAs), isualiza ion aspec s (VAs), echnical aspec s (TAs), ope a ional aspec s (OAs), and o de -p o ile aspec s (OPAs). In addi ion o he 27 pa ame e s ha we e iden i ied in he SLR p ocess, one mo e pa ame e , ha is s o age le el, was men ioned by he expe s and was aken in o accoun du ing he AHP p ocess, esul ing in 28 pa ame e s in o al o be assessed. Following he me hodology ecommended by Saa y [ 69 ], du ing he i s s ep, he cons uc ion o hie a chy s uc u e ook place. Mo e speci ically, he AHP amewo k o e alua ing pick-by- ision pa ame e s was s uc u ed in h ee le els (Figu e 2). Logis ics 2022, 6, x FOR PEER REVIEW 6 o 26 echnologies and sys ems (e.g., ligh picking, oice picking, RF-scanning picking, e c.) [14,24,33,37,47,56,64,66–68]. Las ly, he ou h ca ego y includes h ee pe o mance measu emen indices and deals wi h he inal ou pu o he e alua ion p ocess. These indices a e used by p o essionals and academics o e alua e pick-by- ision echnology and o he o de - picking echnologies in o de o compa e hem [23,25,34,39,47,65]. 2.4. Selec ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems A e aking in o accoun he esul s o he SLR, we implemen ed he p ede ined s eps o an analy ical hie a chy p ocess (AHP) [69] in o de o iden i y he mos impo an pa ame e s ha a ec a pick-by- ision sys em. All he e iewed pa ame e s we e alida ed om expe s’ opinions on i e aspec s: e gonomic aspec s (EAs), isualiza ion aspec s (VAs), echnical aspec s (TAs), ope a ional aspec s (OAs), and o de -p o ile aspec s (OPAs). In addi ion o he 27 pa ame e s ha we e iden i ied in he SLR p ocess, one mo e pa ame e , ha is s o age le el, was men ioned by he expe s and was aken in o accoun du ing he AHP p ocess, esul ing in 28 pa ame e s in o al o be assessed. Following he me hodology ecommended by Saa y [69], du ing he i s s ep, he cons uc ion o hie a chy s uc u e ook place. Mo e speci ically, he AHP amewo k o e alua ing pick-by- ision pa ame e s was s uc u ed in h ee le els (Figu e 2). Figu e 2. AHP-based hie a chical model o e alua e pick-by- ision design and de elopmen pa ame e s. The i s le el includes he goal ( o p io i ize c i ical pick-by- ision design and de elopmen pa ame e s), he second le el ocuses on he dimensions o pa ame e s ( i e dimensions), and he hi d le el deals wi h he cons uc s o dimensions (28 pa ame e s). Du ing he second s ep o he ecommended me hodology, he necessa y ques ionnai e wi h he pai -wise compa ison ma ices (PWCM) was cons uc ed. Fo he cons uc ion o he ques ionnai e, all dimensions and cons uc s o he AHP-based hie a chical model we e aken in o accoun , and he scale o numbe s sugges ed by Saa y [69] was used. A e he cons uc ion o he necessa y ques ionnai e, he anking o he selec ed pa ame e s was comple ed by expe s. In his phase, a se ies o in e iews wi h logis ics/wa ehouse manage s, specialis s, and execu i es ook place. The ques ionnai e was comple ed by 15 expe s who wo k in logis ics se ice p o ide s and in comme cial and manu ac u ing companies (wi h in-house logis ics) in G eece. The s eps o comple ing he ques ionnai e we e speci ic and he same o all pa icipan s, and hey a e p esen ed below. • S ep 1: P esen a ion o he main aim and objec i e o his esea ch. • S ep 2: De ailed desc ip ion o e iewed pa ame e s o he pa icipan s (expe s). • S ep 3: Speci ic ins uc ions gi en o pa icipan s on how o comple e he ques ionnai e. Figu e 2. AHP-based hie a chical model o e alua e pick-by- ision design and de elopmen pa ame e s. The i s le el includes he goal ( o p io i ize c i ical pick-by- ision design and de- elopmen pa ame e s), he second le el ocuses on he dimensions o pa ame e s ( i e dimensions), and he hi d le el deals wi h he cons uc s o dimensions (28 pa ame e s). Du ing he second s ep o he ecommended me hodology, he necessa y ques ionnai e wi h he pai -wise compa ison ma ices (PWCM) was cons uc ed. Fo he cons uc ion o he ques ionnai e, all dimensions and cons uc s o he AHP-based hie a chical model we e aken in o accoun , and he scale o numbe s sugges ed by Saa y [ 69 ] was used. A e he cons uc ion o he necessa y ques ionnai e, he anking o he selec ed pa ame e s was comple ed by expe s. In his phase, a se ies o in e iews wi h logis ics/wa ehouse Logis ics 2022,6, 84 7 o 26 manage s, specialis s, and execu i es ook place. The ques ionnai e was comple ed by 15 expe s who wo k in logis ics se ice p o ide s and in comme cial and manu ac u ing companies (wi h in-house logis ics) in G eece. The s eps o comple ing he ques ionnai e we e speci ic and he same o all pa icipan s, and hey a e p esen ed below. •S ep 1: P esen a ion o he main aim and objec i e o his esea ch. •S ep 2: De ailed desc ip ion o e iewed pa ame e s o he pa icipan s (expe s). • S ep 3: Speci ic ins uc ions gi en o pa icipan s on how o comple e he ques ionnai e. • S ep 4: Ra ing o pick-by- ision sys em design pa ame e s by expe s (comple ed ia ques ionnai e). A e he anking was comple ed, a sho discussion on he pa icipan s ook place in o de o gi e us hei eedback abou he pick-by- ision echnology, he p e e able ope a ional/ unc ional se ices, he challenges and ine iciencies, and he po en ial bene i s o i s implemen a ion in eal-li e scena ios. A e he comple ion o in e iews and da a collec ion (expe s’ inpu s), he da a analysis and he calcula ion o consis ency we e accomplished acco ding o he me hodology ecommended by Saa y [69]. In he las s ep, he p io i ies we e calcula ed on he basis o he AHP me hodology by aking in o accoun he hie a chical model and he a ings achie ed h ough he ques ion- nai e. Fu he mo e, o each pai -wise compa ison ma ix (PWCM), he maximum eigen alues ( λ max), CI, and CR we e calcula ed. Mo eo e , alues o he consis ency a io (CR) we e wi hin an accep able ange o all he pai -wise compa ison ma ices, ensu ing he eliabili y o decision make s. Taking in o accoun he esul s o he anking, we concluded ha he mos impo an dimension o pick-by- ision echnology design and de elopmen was he e gonomic as- pec s (EAs), ollowed by o de -p o ile aspec s (OPAs), isualiza ion aspec s (VAs), ope a ion aspec s (OAs), and echnical aspec s (TAs). In e ms o e gonomic aspec s (EAs), he weigh o equipmen (WE) and scanne posi ion (SP) we e ound o be as he mos impo an cons uc s, while o he o de - p o ile aspec s (OPAs) dimension, he lines pe o de (LO) and numbe o o de s (NOs) we e anked as he mos impo an cons uc s. Finally, acco ding o he expe s, he echnical aspec s (TAs) dimension was less impo an han he o he ou dimensions. The comple e anking o c i ical cons uc s/pa ame e s o pick-by- ision echnology design and de elopmen is p esen ed in Table 4. Table 4. O e all weigh ing and anking o pick-by- ision design and de elopmen pa ame e s. Dimension Desc ip ion Weigh o Dimensions Rank Pa ame e s Desc ip ion Local Weigh o Pa ame e s O e all Weigh o Pa ame e s O e all Ranking o Pa ame e s E gonomic aspec s(EAs) 0.44 1s Display Posi ion 0.023 0.023 17 h Display Type 0.069 0.069 6 h In e ac ion De ice 0.032 0.032 11 h Display Holde 0.024 0.024 16 h Weigh o Equipmen 0.151 0.151 1s Scanne Posi ion 0.076 0.076 4 h Ba e y Posi ion 0.030 0.030 12 h S o age le el 0.034 0.034 10 h Logis ics 2022,6, 84 8 o 26 Table 4. Con . Dimension Desc ip ion Weigh o Dimensions Rank Pa ame e s Desc ip ion Local Weigh o Pa ame e s O e all Weigh o Pa ame e s O e all Ranking o Pa ame e s Visualiza ion aspec s(VAs) 0.16 3 d Field o View 0.060 0.060 7 h Focal Dis ance 0.009 0.009 23 d Visualiza ion Op ics 0.006 0.006 24 h In o ma ion Mode 0.035 0.035 9 h In o ma ion A ailabili y 0.025 0.025 14 h Display View 0.013 0.013 22nd Exis ence o AR 0.004 0.004 26 h Di ec ion In e ace 0.003 0.003 28 h Display Se ings 0.018 0.018 20 h Technical aspec s(TAs) 0.08 5 h Ba code Type 0.029 0.029 13 h Scanning Dis ance 0.006 0.006 25 h Ba e y Li e 0.043 0.043 8 h Ex. o T acking Sys em 0.004 0.004 27 h Con i ma ion Equipmen 0.020 0.020 19 h Ope a ion aspec s(OAs) 0.11 4 h Picking S a egy 0.069 0.069 5 h Handling Uni 0.025 0.025 15 h Exis ence o Con i ma ion 0.013 0.013 21s O de -p o ile aspec s(OPAs) 0.21 2nd Numbe o O de s 0.079 0.079 3 d Lines pe O de 0.107 0.107 2nd I ems pe Line 0.021 0.021 18 h Fo he selec ion o ac o s o be in es iga ed ia labo a o y expe imen s, he anking om he AHP we e ac o ed in, bu he inal selec ion o pa ame e s was made by ac o - ing in also a ious limi a ions o he a ailable labo a o y layou (e.g., limi ed space o conduc ing he es s), lack o a ailabili y o u he de elop g aphical use in e ace (GUI) o he sys em, e c. A e aking in o accoun hese limi a ions, we decided o in es iga e ia labo a o y expe imen a ion he ollowing six pa ame e s: (a) ba e y posi ion, (b) ype o o de , (c) s o age le el, (d) con i ma ion equipmen , € i ems pe o de line, and ( ) o - de lines pe o de . A de ailed analysis o he selec ed pa ame e s will ake place in he ollowing sec ion. 3. Design o Expe imen s Du ing he s ages o he de elopmen and e alua ion o a p ocess o a sys em, i is impo an o adop a obus me hodology wi h speci ic s eps o he execu ion o he expe imen al p ocedu e (e.g., planning and conduc ing expe imen s, da a collec ion and analysis, e c.) in o de o achie e eliable and alid esul s [ 70 – 72 ]. To his end, o he expe - imen al design and he pe o mance e alua ion o he p oposed pick-by- ision sys em, he design o expe imen (DoE) me hodology was adop ed as p oposed by Mon gome y [ 68 ]. DoE was used in o de o in es iga e he e ec o he selec ed pa ame e s in e ms o o de -picking e iciency. Logis ics 2022,6, 84 15 o 26 o de -picking mo emen s, he SLS ® M-100/M-101 Wea able RFID Reade (Sma Label Solu ions LLC 1100 Du an D i e, Howell, MI 48843, Uni ed S a es) was selec ed. All he a o emen ioned equipmen was connec ed o a wa ehouse managemen sys em (WMS) ins alled in a compu e se e a he labo a o y. Each pa icipan was assigned o un a g oup o 12 picking lis s sepa a ed in six mul iple o de pickings (simul aneously) and six disc e e o de pickings (one by one). E e y picking lis consis ed o one o six i ems pe o de line and one o six o de lines pe o de , sha ed equally. The i ems on he shel es con ained simila p oduc ca ego ies wi h di e en sizes and weigh s. E e y i em could be handled wi h one hand. 3.3. Fo mula ion o Resea ch Hypo hesis As men ioned ea lie , he pe o mance o he pick-by- ision sys em was measu ed by aking in o accoun o de -picking e iciency (o de -picking ime pe o de line). O de - picking ime was measu ed by a ypical s opwa ch, and he ime da a we e p esen ed in minu es pe o de line. In o de o in es iga e whe he he pa ame e s unde conside a ion we e s a is ically signi ican , ce ain null hypo heses we e in oduced, as ollows. The i s null hypo hesis (H 0.1 ) s a es ha he pe o mance o he pick-by- ision sys em is he same when he ba e y posi ion is ei he weigh ed back o weigh ed side: H0.1: weigh ed-back = weigh ed-side (1) The second null hypo hesis (H 0.2 ) s a es ha he ime needed o he labo a o y expe imen is same when he ype o o de is ei he disc e e o de o mul iple o de : H0.2: disc e e = mul iple (2) The hi d null hypo hesis (H 0.3 ) s a es ha he ime needed o he labo a o y expe i- men is equal when he s o age le el is ei he low o high: H0.3: low = high (3) The ou h null hypo hesis (H 0.4 ) s a es ha he pe o mance o he pick-by- ision sys em emains he same when he con i ma ion equipmen is ei he a scanne o an RFID eade : H0.4: scanne = RFID (4) The i h null hypo hesis (H 0.5 ) s a es ha he o de -picking ime o he labo a o y expe imen is he same when i ems pe o de line is ei he ew o many: H0.5: ew = many (5) The six h null hypo hesis (H 0.6 ) s a es ha he o de -picking ime o he labo a o y expe imen is he same when o de lines pe o de is ei he ew o many: H0.6: ew = many (6) 3.4. Resul s om he S a is ical Analysis A e he comple ion o he es s and he collec ion o he da a, a quan i a i e anal- ysis was conduc ed in o de o e alua e he o de -picking ime o he pick-by- ision sys em. The de ailed esul s o he ANOVA analysis on o de -picking e iciency a e p esen ed in Table 6. Logis ics 2022,6, 84 16 o 26 Table 6. Resul s o s a is ical analysis (es ima ed e ec s) on o de -picking ime (e iciency). The symbol * ela es he main e ec s as a as hei in e ac ions a e conce ned. Sou ce o Va ia ion Te m p-Value Main E ec s Ba e y Posi ion 0.55 Type o O de 0.052 S o age Le el 0.732 Con i ma ion Equipmen 0 I ems pe O de Line 0 O de Lines pe O de 0.173 2-way in e ac ions Ba e y Posi ion*Type o O de 0.35 Ba e y Posi ion*S o age Le el 0.314 Ba e y Posi ion*Con i ma ion Equipmen 0.031 Ba e y Posi ion*I ems pe O de Line 0.925 Ba e y Posi ion*O de Lines pe O de 0.256 Type o O de *S o age Le el 0.042 Type o O de *Con i ma ion Equipmen 0.004 Type o O de *I ems pe O de Line 0.145 Type o O de *O de Lines pe O de 0.053 S o age Le el*Con i ma ion Equipmen 0.014 S o age Le el*I ems pe O de Line 0.621 S o age Le el*O de Lines pe O de 0.062 Con i ma ion Equipmen *I ems pe O de Line 0.008 Con i ma ion Equipmen *O de Lines pe O de 0.219 I ems pe O de Line*O de Lines pe O de 0.019 3-way in e ac ions Ba e y Posi ion*Type o O de *S o age Le el 0.513 Ba e y Posi ion*Type o O de *Con i ma ion Equipmen 0.085 Ba e y Posi ion*Type o O de *I ems pe O de Line 0.564 Ba e y Posi ion*Type o O de *O de Lines pe O de 0.839 Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen 0.411 Ba e y Posi ion*S o age Le el*I ems pe O de Line 0.77 Ba e y Posi ion*S o age Le el*O de Lines pe O de 0.891 Ba e y Posi ion*Con i ma ion Equipmen *I ems pe O de Line 0.374 Ba e y Posi ion*Con i ma ion Equipmen *O de Lines pe O de 0.251 Ba e y Posi ion*I ems pe O de Line*O de Lines pe O de 0.111 Type o O de *S o age Le el*Con i ma ion Equipmen 0.644 Type o O de *S o age Le el*I ems pe O de Line 0.003 Type o O de *S o age Le el*O de Lines pe O de 0.932 Type o O de *Con i ma ion Equipmen *I ems pe O de Line 0.436 Type o O de *Con i ma ion Equipmen *O de Lines pe O de 0.855 Type o O de *I ems pe O de Line*O de Lines pe O de 0.778 S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.386 S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.092 S o age Le el*I ems pe O de Line*O de Lines pe O de 0.128 Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.005 Logis ics 2022,6, 84 17 o 26 Table 6. Con . Sou ce o Va ia ion Te m p-Value 4-way in e ac ions Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen 0.125 Ba e y Posi ion*Type o O de *S o age Le el*I ems pe O de Line 0.815 Ba e y Posi ion*Type o O de *S o age Le el*O de Lines pe O de 0.729 Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *I ems pe O de Line 0.572 Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *O de Lines pe O de 0.181 Ba e y Posi ion*Type o O de *I ems pe O de Line*O de Lines pe O de 0.603 Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.552 Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.797 Ba e y Posi ion*S o age Le el*I ems pe O de Line*O de Lines pe O de 0.53 Ba e y Posi ion*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.111 Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.171 Type o O de *S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.963 Type o O de *S o age Le el*I ems pe O de Line*O de Lines pe O de 0.048 Type o O de *Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.422 S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.43 5-way in e ac ions Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.341 Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.608 Ba e y Posi ion*Type o O de *S o age Le el*I ems pe O de Line*O de Lines pe O de 0.156 Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.34 Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.035 Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.942 Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.371 Figu e 9p esen s a Pa e o cha ha e i ies he alidi y o he ANOVA analysis, in ha i dis inguishes he ac o s ha a e s a is ically signi ican . A e he ob ained esul s ha e been aken in o accoun , i can be seen ha he e a e a numbe o ac o s and combina ions o ac o s ha signi ican ly a ec he e iciency o he pick-by- ision sys em unde in es iga ion. The esul s ha e shown ha o cases H 0.1 , H 0.2 , H 0.3 , and H 0.6 , he null hypo hesis is accep ed, whe eas o cases H 0.4 and H 0.5 , he null hypo hesis has been ejec ed. To his end, i can be concluded ha he con i ma ion equipmen and i ems pe o de line a ec he pe o mance o pick-by- ision sys em. Logis ics 2022,6, 84 18 o 26 Logis ics 2022, 6, x FOR PEER REVIEW 17 o 26 Figu e 9. Pa e o cha o o de -picking e iciency. In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal p obabili y plo ollows a s aigh line and ha he e sus i s plo has andomly dis ibu ed esiduals a ound ze o. The his og am plo has a bell shape, and he e sus o de plo shape p esen s no speci ic pa e n. The e o e, he da a a e highly eliable. Figu e 10. Residual plo s o o de -picking e iciency. Figu e 9. Pa e o cha o o de -picking e iciency. In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal p obabili y plo ollows a s aigh line and ha he e sus i s plo has andomly dis ibu ed esiduals a ound ze o. The his og am plo has a bell shape, and he e sus o de plo shape p esen s no speci ic pa e n. The e o e, he da a a e highly eliable. Logis ics 2022, 6, x FOR PEER REVIEW 17 o 26 Figu e 9. Pa e o cha o o de -picking e iciency. In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal p obabili y plo ollows a s aigh line and ha he e sus i s plo has andomly dis ibu ed esiduals a ound ze o. The his og am plo has a bell shape, and he e sus o de plo shape p esen s no speci ic pa e n. The e o e, he da a a e highly eliable. Figu e 10. Residual plo s o o de -picking e iciency. Figu e 10. Residual plo s o o de -picking e iciency. Logis ics 2022,6, 84 19 o 26 Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed, we in es iga ed he le els o he s a is ically signi ican ac o s and he sys em con igu a ion ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding o Figu e 11, i can be obse ed ha using he scanne akes less o de -picking ime han using he RFID ag eade does. Fu he mo e, when i ems pe o de line a e ew, he picking e iciency is be e han when he i ems pe o de line a e many (Figu e 12). Logis ics 2022, 6, x FOR PEER REVIEW 18 o 26 Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed, we in es iga ed he le els o he s a is ically signi ican ac o s and he sys em con igu a ion ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding o Figu e 11, i can be obse ed ha using he scanne akes less o de -picking ime han using he RFID ag eade does. Fu he mo e, when i ems pe o de line a e ew, he picking e iciency is be e han when he i ems pe o de line a e many (Figu e 12). Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues). Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues). Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as signi ican . This is because he es s we e conduc ed in a labo a o y en i onmen whe e no signi ican a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may each he shel es om he a ea whe e he ca is by aking a ew s eps. The lack o a el dis ance a ec ed he signi icance o he ype o o de pa ame e , and hus, he di e ence, Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues). Logis ics 2022, 6, x FOR PEER REVIEW 18 o 26 Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed, we in es iga ed he le els o he s a is ically signi ican ac o s and he sys em con igu a ion ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding o Figu e 11, i can be obse ed ha using he scanne akes less o de -picking ime han using he RFID ag eade does. Fu he mo e, when i ems pe o de line a e ew, he picking e iciency is be e han when he i ems pe o de line a e many (Figu e 12). Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues). Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues). Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as signi ican . This is because he es s we e conduc ed in a labo a o y en i onmen whe e no signi ican a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may each he shel es om he a ea whe e he ca is by aking a ew s eps. The lack o a el dis ance a ec ed he signi icance o he ype o o de pa ame e , and hus, he di e ence, Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues). Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as signi ican . This is because he es s we e conduc ed in a labo a o y en i onmen whe e no signi ican a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may each he shel es om he a ea whe e he ca is by aking a ew s eps. The lack o a el dis ance a ec ed he signi icance o he ype o o de pa ame e , and hus, he di e ence, in e ms o picking e iciency, be ween disc e e o de picking and mul iple o de picking was elimina ed. Logis ics 2022,6, 84 20 o 26 Las bu no leas , acco ding o he esul s o he s a is ical analysis, i can be concluded ha he con igu a ion o he in es iga ed pick-by- ision sys em ha p o ides he mos encou aging esul s in e ms o o de -picking e iciency (i.e., o de -picking ime pe o de line) inco po a es he ollowing le els pe pa ame e : ba e y posi ion—weigh ed side; ype o o de —disc e e o de picking; s o age le el—high s o age le el; con i ma ion equipmen —scanne ”; i ems pe o de line— ew; and o de lines pe o de — ew. 4. Pe cei ed Wo kload E alua ion A pe cei ed wo kload e alua ion can be accomplished by many means. NASA TLX is a widely used subjec i e mul idimensional assessmen ool ha a es pe cei ed wo kload o assess a ask, sys em, o p ocess [ 74 ], and acco ding o he a ailable li e a u e, NASA TLX has achie ed some solid goals in human- ac o s esea ch, while assessing sys em design and de elopmen phases [ 75 ]. The NASA TLX is based on a weigh ed a e age o a ings on six subscales [74,76]. Th ee dimensions a e ela ed o he demands imposed on he subjec (men al demand, physical demand, and empo al demand) and h ee o he in e ac ion o a subjec wi h he ask (e o , us a ion, and pe o mance). Acco ding o a se ies o simila wo ks [ 25 , 30 , 33 , 37 , 56 , 65 ], i can be seen ha he NASA TLX me hodology is he mos sui able me hodology o e alua e he pe cei ed wo kload o he p oposed pick-by- ision echnology. Acco ding o NASA [ 74 ], he implemen a ion o NASA TLX ollows wo s eps. The i s s ep deals wi h he sou ce o load (weigh s) and he second s ep wi h he magni ude o loads ( a ing). Fu he in o ma ion on he implemen a ion s ep o NASA TLX me hodology is a ailable in he wo k o NASA [74]. Taking in o accoun he a o emen ioned s eps o NASA TLX me hodology, we e al- ua ed he p oposed pick-by- ision sys em’s pe cei ed wo kload. A e comple ing a ask (o de picking), e e y pa icipan illed he NASA TLX ques ionnai e, based on he a o emen ioned s eps and he expe imen e ’s ins uc ions. To his end, in Figu e 13, he inal esul s o he NASA TLX su ey a e p esen ed. The pick-by- ision sys em sco ed M = 32.8 (SD = 9.1). The indi idual ac o s p esen ed om low wo kload o high wo kload sco ed M = 23.4 (SD = 14.2) o pe o mance, M = 26.1 (SD = 17.5) o men al demand, M = 28.0 (SD = 14.6) o physical demand, M = 30.4 (SD = 20.5) o empo al demand, M = 33.0 (SD = 16.7) o e o , and M = 33.6 (SD = 19.1) o us a ion le el. Logis ics 2022, 6, x FOR PEER REVIEW 19 o 26 in e ms o picking e iciency, be ween disc e e o de picking and mul iple o de picking was elimina ed. Las bu no leas , acco ding o he esul s o he s a is ical analysis, i can be concluded ha he con igu a ion o he in es iga ed pick-by- ision sys em ha p o ides he mos encou aging esul s in e ms o o de -picking e iciency (i.e., o de -picking ime pe o de line) inco po a es he ollowing le els pe pa ame e : ba e y posi ion— weigh ed side; ype o o de —disc e e o de picking; s o age le el—high s o age le el; con i ma ion equipmen —scanne ”; i ems pe o de line— ew; and o de lines pe o de — ew. 4. Pe cei ed Wo kload E alua ion A pe cei ed wo kload e alua ion can be accomplished by many means. NASA TLX is a widely used subjec i e mul idimensional assessmen ool ha a es pe cei ed wo kload o assess a ask, sys em, o p ocess [74], and acco ding o he a ailable li e a u e, NASA TLX has achie ed some solid goals in human- ac o s esea ch, while assessing sys em design and de elopmen phases [75]. The NASA TLX is based on a weigh ed a e age o a ings on six subscales [74,76]. Th ee dimensions a e ela ed o he demands imposed on he subjec (men al demand, physical demand, and empo al demand) and h ee o he in e ac ion o a subjec wi h he ask (e o , us a ion, and pe o mance). Acco ding o a se ies o simila wo ks [25,30,33,37,56,65], i can be seen ha he NASA TLX me hodology is he mos sui able me hodology o e alua e he pe cei ed wo kload o he p oposed pick-by- ision echnology. Acco ding o NASA [74], he implemen a ion o NASA TLX ollows wo s eps. The i s s ep deals wi h he sou ce o load (weigh s) and he second s ep wi h he magni ude o loads ( a ing). Fu he in o ma ion on he implemen a ion s ep o NASA TLX me hodology is a ailable in he wo k o NASA [74]. Taking in o accoun he a o emen ioned s eps o NASA TLX me hodology, we e alua ed he p oposed pick-by- ision sys em’s pe cei ed wo kload. A e comple ing a ask (o de picking), e e y pa icipan illed he NASA TLX ques ionnai e, based on he a o emen ioned s eps and he expe imen e ’s ins uc ions. To his end, in Figu e 13, he inal esul s o he NASA TLX su ey a e p esen ed. The pick-by- ision sys em sco ed M = 32.8 (SD = 9.1). The indi idual ac o s p esen ed om low wo kload o high wo kload sco ed M = 23.4 (SD = 14.2) o pe o mance, M = 26.1 (SD = 17.5) o men al demand, M = 28.0 (SD = 14.6) o physical demand, M = 30.4 (SD = 20.5) o empo al demand, M = 33.0 (SD = 16.7) o e o , and M = 33.6 (SD = 19.1) o us a ion le el. Figu e 13. NASA TLX esul s o he p oposed pick-by- ision sys em. Figu e 13. NASA TLX esul s o he p oposed pick-by- ision sys em. Acco ding o hese esul s, he o e all NASA TLX sco e p o es ha he pe cei ed wo kload o each pa icipan is sa is yingly small. The o e iew o all he indi idual ac o s shows ha he pa icipan s had no signi ican p oblems; hus, hey escala ed he subscales Logis ics 2022,6, 84 21 o 26 wi h small di e ences in o de o dis inguish hem om each o he . Addi ionally, as Casne and Go e [ 77 ] suppo , people who a e o e wo king and people who a e unde wo king exhibi simila pe o mance as hey bo h commi e o s, ha e low e iciency, ge us a ed, and ha e poo awa eness o hei su oundings. To his end, a e he wo kload gauge om Casne and Go e [ 77 ] has been aken in o accoun , i is clea ha he o e all NASA TLX sco e o he p oposed sys em is kep on accep able le el. Mo e speci ically, he highes pe cei ed wo kload index came om he us a ion le el. Indeed, subjec s admi ed ha di icul y in he scanning p ocess was i i a ing hem, and sligh mo emen s o he headbands caused hem annoyance. Simila ly, e o is he second-highes ac o , as subjec s concluded ha compa ed wi h he es o he ac o s, ha d wo k came second. Addi ionally, he labo a o y en i onmen and he exis ence o a s opwa ch o ced he pa icipan s o pick in ensely. As hey admi ed, he mo e amilia hey go , he less in ensi y hey el . As a as he physical demand is conce ned, he lack o a el dis ances and o e all age a oided low sco es, as no a el ime exis ed and c ouching and bending we e no a p oblem o young people. In e ms o men al demand, he subjec s el ha hey didn’ need o hink abou wha o do, as hey we e ully guided h oughou all he p ocess. The lowes sco ing ac o was pe o mance, p o ing ha no special skills we e needed o conduc he es s. Las bu no leas , in eal wa ehouses, whe e p o es- sional picke s wo k o eigh hou s, a el dis ances a e longe , he a e age age is highe , and no labo a o y expe imen is conduc ed, an inc eased NASA TLX sco e is expec ed o be obse ed. 5. Discussion The objec i e o his s udy was o in es iga e ce ain ope a ional and echnical pa am- e e s ha a ec he pe o mance o pick-by- ision echnology in i em-le el o de picking ia a se ies o labo a o y es s. The indings o his s udy a e essen ial because hey show he pa ame e s ( echnical and ope a ional) ha a ec he pe o mance o pick-by- ision echnology and may esul in inc eased o de -picking e iciency and accu acy. A e he ob ained esul s ha e been accoun ed o , i can be seen ha he e we e a numbe o ac o s and combina ions o ac o s ha signi ican ly a ec ed he e iciency o he pick-by- ision sys em unde in es iga ion. The esul s showed ha he con i ma ion equipmen and i ems pe o de line a ec he pe o mance o he pick-by- ision sys em. In addi ion, i can be obse ed ha selec ing he scanne as he piece o o de con i ma ion equipmen educes o de -picking ime o e using an RFID ag eade . Fu he mo e, when i ems pe o de line a e ew, he picking e iciency is be e han when i ems pe o de line a e many. Acco ding o he esul s o he s a is ical analysis, he con igu a ion o he pick-by- ision sys em ha p o ided he mos encou aging esul s in e ms o o de -picking e - iciency (i.e., o de -picking ime pe o de line) inco po a ed he ollowing le els pe pa ame e : (a) ba e y posi ion—weigh ed side; (b) ype o o de —disc e e o de picking; (c) s o age le el—high s o age le el; (d) con i ma ion equipmen —scanne ; (e) i ems pe o de line— ew; and ( ) o de lines pe o de — ew. In he pe cei ed wo kload e alua ion, he o e all NASA TLX sco e p o ed ha he wo kload o each pa icipan was sa is yingly small. The o e iew o all he indi idual ac o s showed ha he pa icipan s had no signi ican p oblems; hus, hey escala ed he subscales wi h small di e ences in o de o dis inguish hem om each o he . 5.1. Con ibu ion o Theo y On he basis o hese indings, i may be a gued ha he esul s p o ide some key insigh s in o heo y. Fi s , a se ies o pa ame e s we e es ed, and hei ole conce ning he pe o mance o pick-by- ision echnology was assessed. O he s udies, such as [ 14 , 34 , 68 ], ha e also e alua ed a se ies o pa ame e s, bu no o he ex en o he analysis ha was made in his s udy. Fu he mo e, an e alua ion o he pa ame e s iden i ied by he SLR me hod was made by p ac i ione s o he logis ics ield. Mo e speci ically, he iew o logis ics manage s was aken in o accoun on p ac ical issues (i.e., o de picking p ocess) and was analyzed by Logis ics 2022,6, 84 22 o 26 using he AHP me hod. In addi ion, al hough o he s udies ha e p esen ed indings om labo a o y es s conce ning picking accu acy and e iciency when pick-by- ision echnology was implemen ed, his wo k adop ed he DoE me hodology and a s a is ical analysis o iden i y co ela ions be ween pa ame e s and he se -up o pa ame e s ha would p o ide he bes esul in pick-by- ision sys em pe o mance. Las ly, his s udy also p o ided use ul insigh s in heo ies on pe cei ed wo kload. By adop ing he NASA TLX me hodology, pick-by- ision echnology can be used by picke s wi h no signi ican p oblems. O he s udies, such as [ 21 , 27 ], ha e also e alua ed e gonomic pa ame e s wi hou , howe e , aking in o accoun o he echnical and ope a ional pa ame e s ha a e also impo an du ing an e alua ion o a pick-by- ision sys em. To his end, he main con ibu ion o his wo k ocused on he iden i ica ion o key pa ame e s ha a ec he pe o mance o pick-by- ision echnology and he de elopmen o a amewo k o he s uc u ed ca ego iza ion o hese pa ame e s. Fu he mo e, his wo k p esen ed signi ican esul s om labo a o y es ing and he bes se -up o he pick-by- ision sys em o inc eased picking e iciency. Las bu no leas , he pa ame e s es ed in his wo k we e selec ed on he basis o he answe s ecei ed om he ques ionnai es ( ia he AHP me hod) comple ed by logis ics manage s who ha e signi ican expe ience in he o de -picking p ocess. 5.2. P ac ical Implica ions F om he ob ained esul s and h ough in e iews wi h logis ics manage s, a numbe o use ul p ac ical implica ions a ise. The la e deal mainly wi h (a) o ganiza ional cul u e, (b) p ocess e-enginee ing, (c) s a esis ance o change, and d) mo i a ion o main aining he new way o doing business. These implica ions a e discussed below: • O ganiza ional cul u e: an e ec i e digi al ans o ma ion om ypical o de -picking me hods o he pick-by- ision sys em needs mo e han upda ing he cu en echnol- ogy. Au oma ion ools a e likely o c ea e dissa is ac ion among he wo k o ce i no managed p ope ly. An o ganiza ional cul u e is needed, he lack o which can cause he in es men o ail and educed pe o mance. • P ocess e-enginee ing: apa om he need o o ganiza ional cul u e, mal unc ion may be caused by a lack o necessa y p ocess e-enginee ing. I is hus c ucial o com- panies o iden i y hei needs and adjus hei o de -picking p ocesses in acco dance wi h he adop ed new echnology. • S a esis ance o change: s a end o esis o echnological change because hey belie e ha hei posi ion is in dange . Typical examples o such si ua ions a e me in he logis ics sec o when new o de -picking echniques/sys ems a e in oduced. In o de o keep he wo k o ce and managemen uni ed, con inuous s a aining, use - iendly sys ems, and echnologies acili a ing wo ke ’s li es a e essen ial. • Mo i a ion o main aining he new way o doing business: a e a comple e and mul ile el ins alla ion o an o de -picking sys em, i is impo an o main ain he new way o doing business. Con inuous imp o emen is equi ed in o de o main ain he in e es o he use , as a e sugges ions o imp o emen s om people wo king wi h he new sys em. 5.3. Limi a ions and Fu u e Resea ch Al hough, he labo a o y es s conduc ed ha e esul ed in encou aging esul s on he pe o mance o pick-by- ision echnology, he e a e always oppo uni ies o u u e esea ch, such as in he ollowing a eas, which we e no in es iga ed in his s udy: • Human ac o : he human ac o plays a c i ical ole when new echnologies a e adop ed, especially o he pick-by- ision sys em, whe e aining and amilia iza ion a e necessa y o i s use. Despi e he ac ha mos people can wo k o e long pe iods wi h pick-by- ision headbands o glasses wi hou being s ained, he e a e s ill some people who ind eading con inuously om a sma de ice di icul . Issues ha should be u he in es iga ed include e gonomics, men al and physical demand on use s, pe o mance, and us a ion le el. Logis ics 2022,6, 84 23 o 26 • Technical issues: Ano he c i ical issue o u u e in es iga ion is he echnical aspec o pick-by- ision, especially when i comes o he use in e ace (UI). Fu he mo e, a en ion should also be paid o issues om in eg a ing he pick-by- ision sys em in o o he sys ems, especially in he close in eg a ion o augmen ed eali y (AR) and wa ehouse managemen sys ems (WMSs), he inc easing com o o ha dwa e compo- nen s, and he po en ial connec ion o picking sys ems wi h au oma ic iden i ica ion sys ems, such as RFID ags. O he echnical issues o be u he in es iga ed may include ba e y li e and scanning dis ance. • Compa a i e assessmen wi h o he picking echnologies: La ely, a compa a i e as- sessmen o pick-by- ision wi h al e na i e picking echnologies, such as oice picking and pick o ligh , ha e come up, bu s ill he e a e many esea ch oppo uni ies in his a ea. Indeed, expe imen s should be conduc ed in o de o assess he accu acy and e iciency o di e en picking echnologies. Las bu no leas , i is wo h e alua ing in es men cos s in o de o compa e no only he pe o mance bu also he cos o ob aining an o de -picking sys em. 6. Conclusions The aim o his pape was ini ially o in es iga e ce ain pa ame e s ha a ec he ope a ional pe o mance o a pick-by- ision sys em ia a se ies o labo a o y es s. A o al o 27 pa ame e s and h ee pe o mance measu emen indices we e iden i ied ia a sys ema ic li e a u e e iew. Six o hem we e selec ed ia an analy ical hie a chy p ocess (AHP), o be in es iga ed u he ia a se ies o labo a o y expe imen s by adop ing he design o expe imen s (DoE) me hodology. The p oposed pick-by- ision sys em was in es iga ed in e ms o o de -picking ime and wo kload. The con igu a ion o he pick- by- ision sys em ha p o ided he mos encou aging esul s in e ms o o de -picking e iciency (i.e., o de -picking ime pe o de line) inco po a ed he ollowing le els pe pa ame e : (a) ba e y posi ion—weigh ed side; (b) ype o o de —disc e e o de picking; (c) s o age le el—high s o age le el; (d) con i ma ion equipmen —scanne ; (e) i ems pe o de line— ew; and ( ) o de lines pe o de — ew. The pe cei ed wo kload o he pick- by- ision sys em was e alua ed ia a NASA TLX su ey. The esul s a e encou aging, showing ha he pick-by- ision echnology can be used by picke s wi h no signi ican wo kload ine iciencies. Au ho Con ibu ions: Concep ualiza ion, V.Z. and A.G.; me hodology, A.G., V.Z.; o mal analysis, N.C., A.G.; in es iga ion, N.C., A.G., V.Z.; esou ces, V.Z.; da a cu a ion, N.C.; w i ing—o iginal d a p epa a ion, N.C., A.G., V.Z.; w i ing— e iew and edi ing, A.G., V.Z.; isualiza ion, N.C., A.G.; supe ision, V.Z.; p ojec adminis a ion, V.Z.; unding acquisi ion, V.Z. All au ho s ha e ead and ag eed o he published e sion o he manusc ip . Funding: This esea ch was unded by he In elligen Resea ch In as uc u e o Shipping, Sup- ply Chain, T anspo and Logis ics (ENIRISST+) p ojec (MIS 5027930), om he NSRF 2014-2020 Ope a ional P og amme “Compe i i eness, En ep eneu ship, Inno a ion”. Con lic s o In e es : The au ho s decla e no con lic o in e es . Re e ences 1. Ma che , G.; Melacini, M.; Pe o i, S. In es iga ing o de picking sys em adop ion: A case-s udy-based app oach. In . J. Logis . Res. Appl. 2015,18, 82–98. [C ossRe ] 2. Lu, W.; McFa lane, D.; Giannikas, V.; Zhang, Q. An algo i hm o dynamic o de -picking in wa ehouse ope a ions. Eu . J. Ope . Res. 2016,248, 107–122. [C ossRe ] 3. Van Gils, T.; Ramaeke s, K.; Ca is, A.; de Kos e , R.B.M. Designing e icien o de picking sys ems by combining planning p oblems: S a e-o - he-a classi ica ion and e iew. Eu . J. Ope . Res. 2018,267, 1–15. [C ossRe ] 4. F anzke, T.; G osse, E.H.; Glock, C.H.; Elbe , R. An in es iga ion o he e ec s o s o age assignmen and picke ou ing on he occu ence o picke blocking in manual picke - o-pa s wa ehouses. In . J. Logis . Manag. 2017,28, 841–863. [C ossRe ] 5. Theys, C.; B äysy, O.; Dullae , W.; Raa, B. Using a TSP heu is ic o ou ing o de picke s in wa ehouses. Eu . J. Ope . Res. 2010, 200, 755–763. [C ossRe ] Logis ics 2022,6, 84 24 o 26 6. Chen, F.; Wang, H.; Xie, Y.; Qi, C. An ACO-based online ou ing me hod o mul iple o de picke s wi h conges ion conside a ion in wa ehouse. J. In ell. Manu . 2016,27, 389–408. [C ossRe ] 7. Wang, S.; Wan, J.; Li, D.; Zhang, C. Implemen ing Sma Fac o y o Indus ie 4.0: An Ou look. In . J. Dis ib. Sens. Ne w. 2016 , 12, 3159805. [C ossRe ] 8. Hanson, R.; Falkens öm, W.; Mie inen, M. Augmen ed eali y as a means o con eying picking in o ma ion in ki p epa a ion o mixed-model assembly. Compu . Ind. Eng. 2017,113, 570–575. [C ossRe ] 9. S ol z, M.H.; Giannikas, V.; McFa lane, D.; S achan, J.; Um, J.; S ini asan, R. Augmen ed Reali y in Wa ehouse Ope a ions: Oppo uni ies and Ba ie s. IFAC-Pape sOnLine 2017,50, 12979–12984. [C ossRe ] 10. G osse, E.H.; Glock, C.H.; Neumann, W.P. Human Fac o s in O de Picking: A Con en Analysis o he Li e a u e. In . J. P od. Res. 2017,55, 1260–1276. [C ossRe ] 11. Van Gils, T.; Ramaeke s, K.; B aeke s, K.; Depai e, B.; Ca is, A. Inc easing o de picking e iciency by in eg a ing s o age, ba ching, zone picking, and ou ing policy decisions. In . J. P od. Econ. 2018,197, 243–261. [C ossRe ] 12. Boysen, N.; de Kos e , R.; Weidinge , F. Wa ehousing in he E-Comme ce E a: A Su ey. Eu . J. Ope . Res. 2019 ,277, 396–411. [C ossRe ] 13. Masae, M.; Glock, C.H.; G osse, E.H. O de picke ou ing in wa ehouses: A sys ema ic li e a u e e iew. In . J. P od. Econ. 2020 , 224, 107564. [C ossRe ] 14. Gialos, A.; Zeimpekis, V. De ining and es ing sys em pa ame e s o enhancing ision picking echnology in wa ehouse ope a ions. In Supply Chain 4.0: Imp o ing Supply Chains wi h Analy ics and Indus y 4.0 Technologies; Ak as, E., Bou lakis, M., Zeimpekis, V., Minis, I., Eds.; Kogan Page: London, UK, 2021; ISBN 978-1789660753. 15. T an ield, D.; Denye , D.; Sma , P. Towa ds a me hodology o de eloping e idence-in o med managemen knowledge by means o sys ema ic e iew. B . J. Manag. 2003,14, 207–222. [C ossRe ] 16. Touboulic, A.; Walke , H. Theo ies in Sus ainable Supply Chain Managemen : A S uc u ed Li e a u e Re iew. In . J. Phys. Dis ib. Logis . Manag. 2015,45, 16–42. [C ossRe ] 17. Lago io, A.; Pin o, R.; Golini, R. Resea ch in U ban Logis ics: A Sys ema ic Li e a u e Re iew. In . J. Phys. Dis ib. Logis . Manag. 2016,46, 908–931. [C ossRe ] 18. Gialos, A.; Zeimpekis, V. Vision picking echnology: De ining design pa ame e s ia a sys ema ic li e a u e e iew. In . J. Logis . Sys . Manag. 2020,37, 106–139. [C ossRe ] 19. Khan, M.; Pa aiz, G.S.; Ali, A.; Jehangi , M.; Hassan, N.; Bae, J. A model o unde s anding he media ing associa ion o anspa ency be ween eme ging echnologies and humani a ian logis ics sus ainabili y. Sus ainabili y 2022 ,14, 16917. [C ossRe ] 20. Glock, C.H.; G osse, E.H.; Neumann, W.P.; Feldman, A. Assis i e de ices o manual ma e ials handling in wa ehouses: A sys ema ic li e a u e e iew. In . J. P od. Res. 2021,59, 3446–3469. [C ossRe ] 21. Khan, M.; Pa aiz, G.S.; Tohi o ich-Dedahano , A.; Iqbal, M.; Junghan, B. Resea ch ends in humani a ian logis ics and sus ainable de elopmen : A bibliome ic analysis. Cogen Bus. Manag. 2022,9, 2143071. [C ossRe ] 22. Schwe d ege , B.; F imo , T.; Pus ka, D.; Klinke , G. Mobile In o ma ion P esen a ion Schemes o Sup a-adap i e Logis ics Applica ions. In Lec u e No es in Compu e Science (Including Subse ies Lec u e No es in A i icial In elligence and Lec u e No es in Bioin o ma ics); Sp inge : Cham, Swi ze land, 2006; pp. 998–1007. ISBN 3540497765. 23. Baumann, H.; S a ne , T.; Zschale , P. S udying o de picking in an ope a ing au omobile manu ac u ing plan . In P oceedings o he P oceedings—In e na ional Symposium on Wea able Compu e s, ISWC, Newcas le Upon Tyne, UK, 18–22 June 2012. 24. Funk, M.; Shi azi, A.S.; Maye , S.; Lischke, L.; Schmid , A. Pick om He e-An in e ac i e mobile ca using in-si u p ojec ion o o de picking. In P oceedings o he UbiComp 2015—P oceedings o he 2015 ACM In e na ional Join Con e ence on Pe asi e and Ubiqui ous Compu ing, Osaka, Japan, 7–11 Sep embe 2015. 25. Kim, S.; Nussbaum, M.A.; Gabba d, J.L. In luences o augmen ed eali y head-wo n display ype and use in e ace design on pe o mance and usabili y in simula ed wa ehouse o de picking. Appl. E gon. 2019,74, 186–193. [C ossRe ] [PubMed] 26. Ma sumo o, T.; Kosaka, T.; Saku ada, T.; Nakajima, Y.; Tano, S. Picking wo k using AR ins uc ions in wa ehouses. In P oceedings o he 2019 IEEE 8 h Global Con e ence on Consume Elec onics (GCCE), Osaka, Japan, 15–18 Oc obe 2019; pp. 31–34. [C ossRe ] 27. Smi h, E.; Bu ch, V.R.F.; S awde man, L.; Chande , H.; Smi h, B.K. A com o analysis o using sma glasses du ing “picking” and “pu ing” asks. In . J. Ind. E gon. 2021,83, 103133. [C ossRe ] 28. K ajco ic, M.; Gabajo a, G.; Micie a, B. O de picking using augmen ed eali y. Komunikacie 2014,16, 106–111. [C ossRe ] 29. Die e, A.; Sz yle , T.; Weiland, L.; S uckenschmid , H. Explo ing a mul i-senso picking p ocess in he u u e wa ehouse. In P oceedings o he UbiComp ’16: P oceedings o he 2016 ACM In e na ional Join Con e ence on Pe asi e and Ubiqui ous Compu ing: Adjunc , Heidelbe g, Ge many, 12–16 Sep embe 2016; pp. 1755–1758. [C ossRe ] 30. Renne , P.; P ei e , T. Augmen ed Reali y Assis ance in he Cen al Field-o -View Ou pe o ms Pe iphe al Displays o O de Picking: Resul s om a Vi ual Reali y Simula ion S udy. In P oceedings o he Adjunc P oceedings o he 2017 IEEE In e na ional Symposium on Mixed and Augmen ed Reali y, ISMAR-Adjunc , Nan es, F ance, 9–13 Oc obe 2017. 31. Elbe , R.; Knigge, J.K.; Sa now, T. T ans e abili y o o de picking pe o mance and aining e ec s achie ed in a i ual eali y using head moun ed de ices. IFAC-Pap. 2018,51, 686–691. [C ossRe ] 32. Rei , R.; Gün hne , W.A. Pick-by- ision: Augmen ed eali y suppo ed o de picking. Vis. Compu . 2009 ,25, 461–467. [C ossRe ]