Chond oma idis, Nikolaos; Gialos, Anas asios; Zeimpekis, Vasileios
A icle
In es iga ing he pe o mance o he o de -picking
p ocess by using sma glasses: A labo a o y expe imen al
app oach
Logis ics
P o ided in Coope a ion wi h:
MDPI – Mul idisciplina y Digi al Publishing Ins i u e, Basel
Sugges ed Ci a ion: Chond oma idis, Nikolaos; Gialos, Anas asios; Zeimpekis, Vasileios (2022) :
In es iga ing he pe o mance o he o de -picking p ocess by using sma glasses: A labo a o y
expe imen al app oach, Logis ics, ISSN 2305-6290, MDPI, Basel, Vol. 6, Iss. 4, pp. 1-26,
h ps://doi.o g/10.3390/logis ics6040084
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/310293
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by/4.0/
Ci a ion: Chond oma idis, N.;
Gialos, A.; Zeimpekis, V.
In es iga ing he Pe o mance o he
O de -Picking P ocess by Using
Sma Glasses: A Labo a o y
Expe imen al App oach. Logis ics
2022,6, 84. h ps://doi.o g/
10.3390/logis ics6040084
Academic Edi o : Robe Hand ield
Recei ed: 24 Oc obe 2022
Accep ed: 1 Decembe 2022
Published: 8 Decembe 2022
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2022 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
logis ics
A icle
In es iga ing he Pe o mance o he O de -Picking P ocess by
Using Sma Glasses: A Labo a o y Expe imen al App oach
Nikolaos Chond oma idis, Anas asios Gialos and Vasileios Zeimpekis *
Depa men o Financial and Managemen Enginee ing, School o Enginee ing, Uni e si y o he Aegean,
82100 Chios, G eece
*Co espondence: [email p o ec ed]
Abs ac :
Backg ound: O de picking p ocess is c i ical o accu a e and e icien o de ul ilmen . Pick-
by- ision is a p omising echnology ha may suppo o de picking p ocess, howe e he e is s ill a
limi ed amoun o esea ch conce ning he impac o his echnology on he pe o mance o o de -
picking. The pu pose o his pape is o in es iga e ce ain ope a ional and echnical pa ame e s
ha a ec he pe o mance o pick-by- ision echnology in i em-le el o de picking ia a se ies
o labo a o y es s. Me hods: A sys ema ic li e a u e e iew is conduc ed o he iden i ica ion o
pa ame e s ha a ec pick-by- ision pe o mance. Subsequen ly, he analy ical hie a chy p ocess
is adop ed o ank hese pa ame e s, conce ning hei impac on o de picking. Then, he design o
expe imen and NASA ask load index me hodology a e implemen ed o assessing pick-by- ision
e iciency and pe cei ed wo kload. Resul s: The esul s e eal he pa ame e s ha signi ican ly a ec
he pe o mance o he pick-by- ision sys em, as well as he bes con igu a ion o pa ame e s o
he implemen a ion o he p oposed sys em in eal en i onmen s. Conclusions: The esul s ob ained
a e encou aging, showing how pick-by- ision echnology can suppo o de picking e iciency.
Fu he mo e, p ac ical implica ions a e p esen ed ha deal wi h he o ganiza ional cul u e, p ocess
e-enginee ing, s a esis ance o change, and mo i a ion o main aining he new way o execu ing
o de -picking p ocesses.
Keywo ds:
pick by ision; sys ema ic li e a u e e iew; analy ical hie a chy p ocess; design o
expe imen s; labo a o y es ing
1. In oduc ion
The inc ease o e-comme ce sales, he globaliza ion o ade, he cus ome s’ demand
o equen and low- olume o de s, and he need o as e esponse imes a e he main
ac o s ha inc ease he complexi y o logis ics p ocesses [
1
,
2
]. The managemen o he
a o emen ioned challenges and he op imiza ion o wa ehouse ope a ions, coupled wi h
logis ics cos educ ion, a e complica ed asks o wa ehouse manage s o cope wi h,
because mos wa ehouses a e manually o semimanually ope a ed, esul ing in deli e ing
labo -in ensi e se ices o hei cus ome s [
3
]. Once he p ocesses o a s anda d wo k low
in a wa ehouse has been aken in o accoun , i can be a gued ha o de picking con ibu es
signi ican ly o logis ics cos s and cus ome se ice [
1
,
3
,
4
]. Indeed, in manual wa ehouses,
he o de -picking p ocess emb aces he mos labo -in ensi e ope a ion, esul ing om 55%
o 65% o he o al ope a ional wa ehouse cos [
5
], while in au oma ed wa ehouses, he
o de -picking cos becomes capi al in ensi e because o he high in es men cos [
6
]. This
is he main a gumen o logis ics p o essionals who p io i ize wa ehouse imp o emen s by
ocusing mainly on he o de -picking p ocess.
Focusing on he de elopmen o in o ma ion and communica ion echnology (ICT)
and a numbe o o he echnologies (e.g., pick by ligh , pick by oice, ing scanne s,
augmen ed eali y, RFID, e c.) ha e al eady been adop ed o o de picking. These ech-
nologies digi alize he adi ional pape -based picking lis , acili a ing in ha way he
Logis ics 2022,6, 84. h ps://doi.o g/10.3390/logis ics6040084 h ps://www.mdpi.com/jou nal/logis ics
Logis ics 2022,6, 84 2 o 26
ul illmen o mode n cus ome needs o inc eased e iciency and accu acy [
7
]. Pick by
ision h ough sma glasses is an inno a i e solu ion ha may imp o e bo h ime e i-
ciency and o de -picking accu acy [
8
]. Acco ding o S ol z e al. [
9
], pick by ision uses
wea able echnology and p o ides a as and hands- ee solu ion o he execu ion o he
o de -picking p ocess. This inno a i e echnology combines he bes o ision-guided pick-
ing so as o p oduce a mo e e icien and mo e accu a e ope a ion beyond he con en ional
o de -picking echnologies [7].
Despi e he gene al imp ession ha pick by ision is a p omising o de -picking ech-
nology, he e is s ill a limi ed amoun o esea ch conce ning he impac o his ech-
nology on he pe o mance o he o de -picking p ocess, which makes i di icul o
de i e solid esul s and make p ac ical ecommenda ions. Indeed, on he basis o he
a ailable li e a u e [
10
–
13
], a signi ican numbe o s udies ha e deal wi h he op imiza ion
o he o de -picking p ocess, by conside ing a se ies o s a egic, ac ical, and ope a ional
pa ame e s, bu only a small numbe o s udies ha e conside ed he pa ame e s ha a ec
o de -picking accu acy and e iciency du ing he design and op imiza ion phase o he
o de picking p ocess [
14
]. Indeed, in [
14
], he au ho s ake in o accoun ou pa ame e s:
(a) display holde , (b) ield o iew, (c) ba code ype, and (d) exis ence o con i ma ion, wi h-
ou , howe e , conside ing o he , equally impo an pa ame e s ha deal wi h e gonomic
pa ame e s, such as ba e y posi ion, and o de -p o ile pa ame e s, such as o de lines pe
o de , i ems pe o de line, e c. To his end, we a gue ha he e is s ill a need o u he
in es iga ion in pa ame e s ha a ec o de picking, in o de o ho oughly analyze hei
impac on o de -picking p oduc i i y, e iciency, and ope a ional cos .
To his end, he aim o his pape is o in es iga e ce ain pa ame e s ha a ec he
ope a ional pe o mance o a pick-by- ision sys em ia a se ies o labo a o y es s. Ini ially,
27 pa ame e s and h ee pe o mance measu emen indices a e iden i ied o he pick-by-
ision sys em design, de elopmen , and es ing by adop ing a sys ema ic li e a u e e iew
(SLR) me hodology. Six pa ame e s a e selec ed ia an analy ical hie a chy p ocess (AHP)
in o de o in es iga e hei impac on he p oposed pick-by- ision sys em. Subsequen ly,
a se ies o labo a o y es s a e conduc ed by adop ing he design o expe imen s (DoE)
me hodology. The p oposed pick-by- ision sys em is in es iga ed o assess i s o de -picking
ime and wo kload. The pe cei ed wo kload o he pick-by- ision sys em is e alua ed ia
NASA TLX su ey.
The emainde o his pape is s uc u ed as ollows. Sec ion 2p esen s he indings
om he li e a u e e iew and desc ibes he selec ed pa ame e s o he e alua ion o he
p oposed pick-by- ision sys em. Sec ion 3p esen s he necessa y s eps ha ha e been
adop ed o he design o he labo a o y expe imen s. In Sec ions 4and 5, he esul s o he
labo a o y es s and he discussion o he indings, coupled wi h heo e ical con ibu ion
and p ac ical implica ions, a e p esen ed. Finally, Sec ion 6concludes he pape .
2. Iden i ica ion and Selec ion o Pa ame e s
2.1. Subsec ion
The sys ema ic li e a u e e iew (SLR) me hodology, coupled wi h a se ies o esea ch
ques ions (RQs), was adop ed wi h he aim o iden i ying, de ec ing, and ca ego izing he
pa ame e s ha a ec he design and ope a ion o he p oposed pick-by- ision sys em.
Acco ding o Table 1, a o al o ou esea ch ques ions we e de eloped. In o de o
answe he abo e RQs, we use he sys ema ic li e a u e e iew (SLR) me hod. Mo e
speci ically, we ollowed a h ee-s ep p o ocol based on p e ious p ominen a icles [
15
–
21
],
in o de o come up wi h eliable and p o en wo k. The s eps o selec ing p o ocol a e
desc ibed as ollows.
Logis ics 2022,6, 84 3 o 26
Table 1.
Resea ch ques ions o he iden i ica ion, de ec ion, and ca ego iza ion o pa ame e s ha
a ec he design and ope a ion o he p oposed pick-by- ision sys em.
No Desc ip ion o Resea ch Ques ion
RQ1 Which a e he main pa ame e s ha should be aken in o conside a ion o a pick-by- ision de ice
pa ame e iza ion (sys em design)?
RQ2
Which a e he main pa ame e s ha should be aken in o conside a ion o he e alua ion and op imiza ion o
pick-by- ision sys ems in e ms o ope a ional pe o mance?
RQ3 Which a e he main pa ame e s ha should be aken in o conside a ion o he e alua ion and compa a i e
assessmen o pick-by- ision echnology in e ms o o de p o ile?
RQ4
Which a e he main pe o mance measu emen indices and side e ec s ha should be aken in o conside a ion
o he e alua ion pick-by- ision echnology in e ms o pe o mance and e gonomics?
In he i s s ep, a se ies o sea ch e ms/keywo ds and induc ion c i e ia we e de-
e mined, in o de o conduc comp ehensi e esea ch (Table 2). The esea ch ocused on
pape s published in pee - e iewed jou nals, a in e na ional con e ences, in disse a ions
and in echnical epo s on he ield o logis ics. The main eason o including a icles
om in e na ional con e ences, disse a ions, and epo s in his wo k is ha he num-
be o pape s in pee - e iewed jou nals ha deal wi h he e alua ion o pick-by- ision
sys ems is limi ed.
Table 2. Inclusion c i e ia o implemen ing SLR me hodology.
Inclusion C i e ia Desc ip ion
Sea ch e ms/Keywo ds Vision picking, pick-by- ision, wea able echnology, wea able compu e s, o de picking,
augmen ed eali y, head-moun ed displays, sma glasses, use in e ace, logis ics
Sou ce ypes (a) pee - e iewed jou nals, (b) in e na ional con e ences,
(c) epo s, (d) disse a ions
Language English
In he second s ep, a e iew o selec ed a icles ( om S ep 1) ook place on he basis
o he i les and abs ac s o he a icles. Du ing his e iew, a se ies o a icles ou o
he esea ch scope was excluded om ou lis . Mo e speci ically, 46 s udies ocused on
di e en aspec s o pick-by- ision echnology and ields. A e he comple ion o his s ep,
he emaining numbe o a icles was 31.
In he las s ep, he eading o ull e sions o a ailable s udies led o he inal selec ion
o he lis o s udies o be conside ed. Eigh s udies we e excluded, and by implemen ing he
snowballing me hod, we we e able o add o ou ini ial lis some addi ional pape s, epo s,
and disse a ions ha me he inclusion c i e ia. The inal co pus in ol ed 66 s udies. The
la e we e e iewed in o de o iden i y he key pa ame e s ha a ec he design and
implemen a ion o a pick-by- ision sys em.
2.2. Desc ip i e Analysis o he Co pus
A e he esul s o SLR we e aken in o accoun , i was e ealed ha 44.6% o he
e iewed s udies we e coming om jou nals, 40% om con e ences, 4.6% om epo s,
and 10.8% om disse a ions. The small numbe o published s udies and he e o e he
limi ed numbe o jou nal a icles we e ep esen a i e signs ha he ield is qui e new, om
a esea ch poin o iew.
The la e seems o be con i med i we ake in o accoun he ime dis ibu ion o he
e iewed s udies. Gi en he esul s o he ime dis ibu ion o he e iewed s udies, i is
e iden ha he yea s o publica ion among he iden i ied publica ions a y om 2001 o
2021. The numbe o s udies ac o ing in he design, de elopmen , and es ing o pick-by-
ision echnology has apidly g own du ing he pas se e al yea s. Almos , hal o he
Logis ics 2022,6, 84 4 o 26
conside ed s udies we e published wi hin he pas ou yea s, indica ing ha he a ea has
been signi ican ly expanding o e he pas ew yea s. The peak in he numbe o s udies is
obse ed du ing he wo-yea pe iod om 2018 o 2020, when 27 s udies we e published.
Acco ding o a geog aphical analysis o he s udy a eas, 70.5% o he s udies we e
conduc ed in Eu ope and 25.6% on he Ame ican con inen s. The majo i y o he Eu opean
s udies we e conduc ed in Ge many (60% o he s udies). Fu he mo e, 7.3% o he s udies
we e conduc ed in G eece, same as Sweden, and Slo enia is nex , a 5.4%. The emain-
ing 20% o he s udies we e conduc ed in a ious o he Eu opean coun ies, as shown
in Figu e 1.
Logis ics 2022, 6, x FOR PEER REVIEW 4 o 26
is obse ed du ing he wo-yea pe iod om 2018 o 2020, when 27 s udies we e
published.
Acco ding o a geog aphical analysis o he s udy a eas, 70.5% o he s udies we e
conduc ed in Eu ope and 25.6% on he Ame ican con inen s. The majo i y o he Eu opean
s udies we e conduc ed in Ge many (60% o he s udies). Fu he mo e, 7.3% o he s udies
we e conduc ed in G eece, same as Sweden, and Slo enia is nex , a 5.4%. The emaining
20% o he s udies we e conduc ed in a ious o he Eu opean coun ies, as shown in
Figu e 1.
Figu e 1. Geog aphical dis ibu ion o he e iewed s udies.
2.3. Iden i ica ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems
A e he s uc u e o he basic esea ch ques ions had been inalized, he iden i ied
pa ame e s we e classi ied in ou ca ego ies: (a) pa ame e s o sys em design, (b)
pa ame e s o sys em e alua ion in e ms o ope a ion, (c) pa ame e s o sys em
e alua ion in e ms o o de p o ile, and (d) pe o mance measu emen indices ha can
be used o he e alua ion o pick-by- ision echnology. All he e iewed pa ame e s pe
ca ego y a e p esen ed in Table 3.
Table 3. O e iew o he e iewed pa ame e s o he design and in es iga ion o he p oposed
sys em.
Ca ego y
Subca ego y
Pa ame e
Re e ences
A. Pa ame e s o
sys em design
E gonomic aspec s
Display Posi ion
[22–24]
Display Type
[24–27]
In e ac ion De ice
[24,25,28–31]
Ba e y Posi ion
[27]
Display Holde
[22,30,32]
Scanne Posi ion
[9,32–35]
Weigh o Equipmen
[27,34,36–38]
Visualiza ion aspec s
Field o View
[26,30,39–41]
Focal Dis ance
[26,39,42]
Visualiza ion Op ics
[29,38,41,43,44]
In o ma ion Mode
[8,22,25,26,45,46]
In o ma ion A ailabili y
[25,34]
Display View
[9,33,37,47,48]
Exis ence o AR
[45,46,49]
Di ec ion In e ace
[28,32,50–52]
1 1 1 1 1
33
4
1111131
4213
18
Figu e 1. Geog aphical dis ibu ion o he e iewed s udies.
2.3. Iden i ica ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems
A e he s uc u e o he basic esea ch ques ions had been inalized, he iden i ied
pa ame e s we e classi ied in ou ca ego ies: (a) pa ame e s o sys em design, (b) pa am-
e e s o sys em e alua ion in e ms o ope a ion, (c) pa ame e s o sys em e alua ion
in e ms o o de p o ile, and (d) pe o mance measu emen indices ha can be used o
he e alua ion o pick-by- ision echnology. All he e iewed pa ame e s pe ca ego y a e
p esen ed in Table 3.
Table 3.
O e iew o he e iewed pa ame e s o he design and in es iga ion o he p oposed sys em.
Ca ego y Subca ego y Pa ame e Re e ences
A. Pa ame e s o
sys em design E gonomic aspec s
Display Posi ion [22–24]
Display Type [24–27]
In e ac ion De ice [24,25,28–31]
Ba e y Posi ion [27]
Display Holde [22,30,32]
Scanne Posi ion [9,32–35]
Weigh o Equipmen [27,34,36–38]

Logis ics 2022,6, 84 5 o 26
Table 3. Con .
Ca ego y Subca ego y Pa ame e Re e ences
A. Pa ame e s o
sys em design
Visualiza ion aspec s
Field o View [26,30,39–41]
Focal Dis ance [26,39,42]
Visualiza ion Op ics [29,38,41,43,44]
In o ma ion Mode [8,22,25,26,45,46]
In o ma ion A ailabili y [25,34]
Display View [9,33,37,47,48]
Exis ence o AR [45,46,49]
Di ec ion In e ace [28,32,50–52]
Display Se ings [25,26,29]
Technical aspec s
Ba code Type [9,35,53–55]
Scanning Dis ance [8,44,56–58]
Ba e y Li e [27,38,59–61]
Exis ence o T acking Sys em [32,40]
Con i ma ion Equipmen [28,34,36,41,62]
B. Pa ame e s o sys em e alua ion in e ms o ope a ion
Picking S a egy [63,64]
Handling Uni [32,57]
Exis ence o Con i ma ion [14,28,29,37,65]
C. Pa ame e s o sys em e alua ion in e ms o o de p o ile
Numbe o O de s [33,36,37,47,49,56,59,66]
Lines pe O de [14,24,33,37,47,56,64,66–68]
I ems pe Line [14,32,33,37,47,49,56,64,66]
D. Pe o mance measu emen indices
E iciency (Time) [9,25,34,47,62,65]
Accu acy [23,25,34,39,47,65]
Wo kload [30,33,34,37,41,42,68]
The i s ca ego y comp ises he de ice design and de elopmen o he p oposed
sys em and includes 21 e iewed pa ame e s. Because o he high numbe o pa ame e s
in his ca ego y, hey we e u he classi ied in o h ee dis inc i e subca ego ies. The
i s subca ego y deals wi h he e gonomic aspec s and in ol es se en pa ame e s, he
second one ocuses on isualiza ion aspec s and includes nine pa ame e s, and he hi d
subca ego y is associa ed wi h echnical aspec s and encompasses i e pa ame e s. The
e gonomic aspec s o de ice pa ame e iza ion play a c i ical ole du ing he design and
de elopmen o he sys em in ha hey deal wi h pa ame e s ha de ine how com o able
a wo ke would eel while using he sys em. A c ucial issue is ha he wo ke has o
wea he equipmen needed o a pick-by- ision sys em du ing a shi . To his end, he
pick-by- ision equipmen mus be ligh , e gonomically designed, and sa e and mus
ha e an eigh -hou ba e y li e [
32
]. The isualiza ion aspec s du ing he design and
de elopmen o pick-by- ision echnology deal wi h he g aphical use in e ace (GUI)
o he de ice. Indeed, one o he mos impo an ea u es o a pick-by- ision sys em is
he GUI, because he i ual in o ma ion mus be displayed in he lens o he glasses a
he igh ime and a he igh posi ion [
49
,
59
]. Ne e heless, he display o necessa y
in o ma ion (i.e., s ock loca ion, a icle numbe , goods desc ip ion, equi ed quan i ies,
e c.) on picke s’ glasses o headbands does no always e icien ly appea , because o
a ious p oblems, such as eye s ain, di icul ies seeing he display image, eye pain, eye
concen a ion p oblems, and headaches, which ha e been obse ed du ing he es ing o
pick-by- ision echnology [
23
]. The echnical aspec s ocus mainly on he ha dwa e used
Logis ics 2022,6, 84 6 o 26
du ing pick by ision. This ca ego y is i al du ing he design and de elopmen o pick-by-
ision sys ems because i includes pa ame e s ha a ec he abili y o a sys em o ead and
ecognize he p oduc s, o moni o he p ocess and di ec he o de picke s, and o ensu e
o de -picking-p ocess accu acy.
The second ca ego y deals wi h h ee pa ame e s ha enable he ope a ional pe o mance
o he sys em o be e alua ed. Acco ding o he a ailable li e a u e [
4
,
22
,
28
,
29
,
37
,
57
,
63
–
65
],
his ca ego y includes a se ies o pa ame e s (e.g., picking s a egy, handling uni , e c.) ha
can be used by in es iga o s when hey assess he ope a ion pe o mance o he sys em in
he ield.
The hi d ca ego y includes h ee pa ame e s ha a e used o he compa ison as-
sessmen o he p oposed sys em wi h o he picking echnologies. This ca ego y de-
sc ibes a se ies o ac o s, such as he numbe o o de s, lines pe o de , e c., ha can
be used in o de o compa e he pick-by- ision sys em wi h o he con en ional o de -
picking echnologies and sys ems (e.g., ligh picking, oice picking, RF-scanning picking,
e c.) [14,24,33,37,47,56,64,66–68].
Las ly, he ou h ca ego y includes h ee pe o mance measu emen indices and deals
wi h he inal ou pu o he e alua ion p ocess. These indices a e used by p o essionals and
academics o e alua e pick-by- ision echnology and o he o de -picking echnologies in
o de o compa e hem [23,25,34,39,47,65].
2.4. Selec ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems
A e aking in o accoun he esul s o he SLR, we implemen ed he p ede ined
s eps o an analy ical hie a chy p ocess (AHP) [
69
] in o de o iden i y he mos impo an
pa ame e s ha a ec a pick-by- ision sys em. All he e iewed pa ame e s we e alida ed
om expe s’ opinions on i e aspec s: e gonomic aspec s (EAs), isualiza ion aspec s (VAs),
echnical aspec s (TAs), ope a ional aspec s (OAs), and o de -p o ile aspec s (OPAs). In
addi ion o he 27 pa ame e s ha we e iden i ied in he SLR p ocess, one mo e pa ame e ,
ha is s o age le el, was men ioned by he expe s and was aken in o accoun du ing he
AHP p ocess, esul ing in 28 pa ame e s in o al o be assessed. Following he me hodology
ecommended by Saa y [
69
], du ing he i s s ep, he cons uc ion o hie a chy s uc u e
ook place. Mo e speci ically, he AHP amewo k o e alua ing pick-by- ision pa ame e s
was s uc u ed in h ee le els (Figu e 2).
Logis ics 2022, 6, x FOR PEER REVIEW 6 o 26
echnologies and sys ems (e.g., ligh picking, oice picking, RF-scanning picking, e c.)
[14,24,33,37,47,56,64,66–68].
Las ly, he ou h ca ego y includes h ee pe o mance measu emen indices and
deals wi h he inal ou pu o he e alua ion p ocess. These indices a e used by
p o essionals and academics o e alua e pick-by- ision echnology and o he o de -
picking echnologies in o de o compa e hem [23,25,34,39,47,65].
2.4. Selec ion o Pa ame e s o he Design and In es iga ion o Pick-by-Vision Sys ems
A e aking in o accoun he esul s o he SLR, we implemen ed he p ede ined s eps
o an analy ical hie a chy p ocess (AHP) [69] in o de o iden i y he mos impo an
pa ame e s ha a ec a pick-by- ision sys em. All he e iewed pa ame e s we e
alida ed om expe s’ opinions on i e aspec s: e gonomic aspec s (EAs), isualiza ion
aspec s (VAs), echnical aspec s (TAs), ope a ional aspec s (OAs), and o de -p o ile
aspec s (OPAs). In addi ion o he 27 pa ame e s ha we e iden i ied in he SLR p ocess,
one mo e pa ame e , ha is s o age le el, was men ioned by he expe s and was aken
in o accoun du ing he AHP p ocess, esul ing in 28 pa ame e s in o al o be assessed.
Following he me hodology ecommended by Saa y [69], du ing he i s s ep, he
cons uc ion o hie a chy s uc u e ook place. Mo e speci ically, he AHP amewo k o
e alua ing pick-by- ision pa ame e s was s uc u ed in h ee le els (Figu e 2).
Figu e 2. AHP-based hie a chical model o e alua e pick-by- ision design and de elopmen
pa ame e s.
The i s le el includes he goal ( o p io i ize c i ical pick-by- ision design and
de elopmen pa ame e s), he second le el ocuses on he dimensions o pa ame e s ( i e
dimensions), and he hi d le el deals wi h he cons uc s o dimensions (28 pa ame e s).
Du ing he second s ep o he ecommended me hodology, he necessa y
ques ionnai e wi h he pai -wise compa ison ma ices (PWCM) was cons uc ed. Fo he
cons uc ion o he ques ionnai e, all dimensions and cons uc s o he AHP-based
hie a chical model we e aken in o accoun , and he scale o numbe s sugges ed by Saa y
[69] was used. A e he cons uc ion o he necessa y ques ionnai e, he anking o he
selec ed pa ame e s was comple ed by expe s. In his phase, a se ies o in e iews wi h
logis ics/wa ehouse manage s, specialis s, and execu i es ook place. The ques ionnai e
was comple ed by 15 expe s who wo k in logis ics se ice p o ide s and in comme cial
and manu ac u ing companies (wi h in-house logis ics) in G eece. The s eps o
comple ing he ques ionnai e we e speci ic and he same o all pa icipan s, and hey a e
p esen ed below.
• S ep 1: P esen a ion o he main aim and objec i e o his esea ch.
• S ep 2: De ailed desc ip ion o e iewed pa ame e s o he pa icipan s (expe s).
• S ep 3: Speci ic ins uc ions gi en o pa icipan s on how o comple e he
ques ionnai e.
Figu e 2.
AHP-based hie a chical model o e alua e pick-by- ision design and de elopmen pa ame e s.
The i s le el includes he goal ( o p io i ize c i ical pick-by- ision design and de-
elopmen pa ame e s), he second le el ocuses on he dimensions o pa ame e s ( i e
dimensions), and he hi d le el deals wi h he cons uc s o dimensions (28 pa ame e s).
Du ing he second s ep o he ecommended me hodology, he necessa y ques ionnai e
wi h he pai -wise compa ison ma ices (PWCM) was cons uc ed. Fo he cons uc ion
o he ques ionnai e, all dimensions and cons uc s o he AHP-based hie a chical model
we e aken in o accoun , and he scale o numbe s sugges ed by Saa y [
69
] was used. A e
he cons uc ion o he necessa y ques ionnai e, he anking o he selec ed pa ame e s
was comple ed by expe s. In his phase, a se ies o in e iews wi h logis ics/wa ehouse
Logis ics 2022,6, 84 7 o 26
manage s, specialis s, and execu i es ook place. The ques ionnai e was comple ed by
15 expe s who wo k in logis ics se ice p o ide s and in comme cial and manu ac u ing
companies (wi h in-house logis ics) in G eece. The s eps o comple ing he ques ionnai e
we e speci ic and he same o all pa icipan s, and hey a e p esen ed below.
•S ep 1: P esen a ion o he main aim and objec i e o his esea ch.
•S ep 2: De ailed desc ip ion o e iewed pa ame e s o he pa icipan s (expe s).
•
S ep 3: Speci ic ins uc ions gi en o pa icipan s on how o comple e he ques ionnai e.
•
S ep 4: Ra ing o pick-by- ision sys em design pa ame e s by expe s (comple ed
ia ques ionnai e).
A e he anking was comple ed, a sho discussion on he pa icipan s ook place
in o de o gi e us hei eedback abou he pick-by- ision echnology, he p e e able
ope a ional/ unc ional se ices, he challenges and ine iciencies, and he po en ial bene i s
o i s implemen a ion in eal-li e scena ios. A e he comple ion o in e iews and da a
collec ion (expe s’ inpu s), he da a analysis and he calcula ion o consis ency we e
accomplished acco ding o he me hodology ecommended by Saa y [69].
In he las s ep, he p io i ies we e calcula ed on he basis o he AHP me hodology by
aking in o accoun he hie a chical model and he a ings achie ed h ough he ques ion-
nai e. Fu he mo e, o each pai -wise compa ison ma ix (PWCM), he maximum eigen
alues (
λ
max), CI, and CR we e calcula ed. Mo eo e , alues o he consis ency a io (CR)
we e wi hin an accep able ange o all he pai -wise compa ison ma ices, ensu ing he
eliabili y o decision make s.
Taking in o accoun he esul s o he anking, we concluded ha he mos impo an
dimension o pick-by- ision echnology design and de elopmen was he e gonomic as-
pec s (EAs), ollowed by o de -p o ile aspec s (OPAs), isualiza ion aspec s (VAs), ope a ion
aspec s (OAs), and echnical aspec s (TAs).
In e ms o e gonomic aspec s (EAs), he weigh o equipmen (WE) and scanne
posi ion (SP) we e ound o be as he mos impo an cons uc s, while o he o de -
p o ile aspec s (OPAs) dimension, he lines pe o de (LO) and numbe o o de s (NOs)
we e anked as he mos impo an cons uc s. Finally, acco ding o he expe s, he
echnical aspec s (TAs) dimension was less impo an han he o he ou dimensions. The
comple e anking o c i ical cons uc s/pa ame e s o pick-by- ision echnology design
and de elopmen is p esen ed in Table 4.
Table 4. O e all weigh ing and anking o pick-by- ision design and de elopmen pa ame e s.
Dimension
Desc ip ion
Weigh o
Dimensions Rank Pa ame e s Desc ip ion Local Weigh
o Pa ame e s
O e all Weigh
o Pa ame e s
O e all
Ranking o
Pa ame e s
E gonomic
aspec s(EAs) 0.44 1s
Display Posi ion 0.023 0.023 17 h
Display Type 0.069 0.069 6 h
In e ac ion De ice 0.032 0.032 11 h
Display Holde 0.024 0.024 16 h
Weigh o Equipmen 0.151 0.151 1s
Scanne Posi ion 0.076 0.076 4 h
Ba e y Posi ion 0.030 0.030 12 h
S o age le el 0.034 0.034 10 h
Logis ics 2022,6, 84 8 o 26
Table 4. Con .
Dimension
Desc ip ion
Weigh o
Dimensions Rank Pa ame e s Desc ip ion Local Weigh
o Pa ame e s
O e all Weigh
o Pa ame e s
O e all
Ranking o
Pa ame e s
Visualiza ion
aspec s(VAs) 0.16 3 d
Field o View 0.060 0.060 7 h
Focal Dis ance 0.009 0.009 23 d
Visualiza ion Op ics 0.006 0.006 24 h
In o ma ion Mode 0.035 0.035 9 h
In o ma ion A ailabili y 0.025 0.025 14 h
Display View 0.013 0.013 22nd
Exis ence o AR 0.004 0.004 26 h
Di ec ion In e ace 0.003 0.003 28 h
Display Se ings 0.018 0.018 20 h
Technical
aspec s(TAs) 0.08 5 h
Ba code Type 0.029 0.029 13 h
Scanning Dis ance 0.006 0.006 25 h
Ba e y Li e 0.043 0.043 8 h
Ex. o T acking Sys em 0.004 0.004 27 h
Con i ma ion Equipmen
0.020 0.020 19 h
Ope a ion
aspec s(OAs) 0.11 4 h
Picking S a egy 0.069 0.069 5 h
Handling Uni 0.025 0.025 15 h
Exis ence o
Con i ma ion 0.013 0.013 21s
O de -p o ile
aspec s(OPAs) 0.21 2nd
Numbe o O de s 0.079 0.079 3 d
Lines pe O de 0.107 0.107 2nd
I ems pe Line 0.021 0.021 18 h
Fo he selec ion o ac o s o be in es iga ed ia labo a o y expe imen s, he anking
om he AHP we e ac o ed in, bu he inal selec ion o pa ame e s was made by ac o -
ing in also a ious limi a ions o he a ailable labo a o y layou (e.g., limi ed space o
conduc ing he es s), lack o a ailabili y o u he de elop g aphical use in e ace (GUI)
o he sys em, e c. A e aking in o accoun hese limi a ions, we decided o in es iga e
ia labo a o y expe imen a ion he ollowing six pa ame e s: (a) ba e y posi ion, (b) ype
o o de , (c) s o age le el, (d) con i ma ion equipmen ,
€
i ems pe o de line, and ( ) o -
de lines pe o de . A de ailed analysis o he selec ed pa ame e s will ake place in he
ollowing sec ion.
3. Design o Expe imen s
Du ing he s ages o he de elopmen and e alua ion o a p ocess o a sys em, i
is impo an o adop a obus me hodology wi h speci ic s eps o he execu ion o he
expe imen al p ocedu e (e.g., planning and conduc ing expe imen s, da a collec ion and
analysis, e c.) in o de o achie e eliable and alid esul s [
70
–
72
]. To his end, o he expe -
imen al design and he pe o mance e alua ion o he p oposed pick-by- ision sys em, he
design o expe imen (DoE) me hodology was adop ed as p oposed by Mon gome y [
68
].
DoE was used in o de o in es iga e he e ec o he selec ed pa ame e s in e ms o
o de -picking e iciency.
Logis ics 2022,6, 84 15 o 26
o de -picking mo emen s, he SLS
®
M-100/M-101 Wea able RFID Reade (Sma Label
Solu ions LLC 1100 Du an D i e, Howell, MI 48843, Uni ed S a es) was selec ed. All he
a o emen ioned equipmen was connec ed o a wa ehouse managemen sys em (WMS)
ins alled in a compu e se e a he labo a o y.
Each pa icipan was assigned o un a g oup o 12 picking lis s sepa a ed in six
mul iple o de pickings (simul aneously) and six disc e e o de pickings (one by one).
E e y picking lis consis ed o one o six i ems pe o de line and one o six o de lines pe
o de , sha ed equally. The i ems on he shel es con ained simila p oduc ca ego ies wi h
di e en sizes and weigh s. E e y i em could be handled wi h one hand.
3.3. Fo mula ion o Resea ch Hypo hesis
As men ioned ea lie , he pe o mance o he pick-by- ision sys em was measu ed
by aking in o accoun o de -picking e iciency (o de -picking ime pe o de line). O de -
picking ime was measu ed by a ypical s opwa ch, and he ime da a we e p esen ed in
minu es pe o de line. In o de o in es iga e whe he he pa ame e s unde conside a ion
we e s a is ically signi ican , ce ain null hypo heses we e in oduced, as ollows.
The i s null hypo hesis (H
0.1
) s a es ha he pe o mance o he pick-by- ision sys em
is he same when he ba e y posi ion is ei he weigh ed back o weigh ed side:
H0.1: weigh ed-back = weigh ed-side (1)
The second null hypo hesis (H
0.2
) s a es ha he ime needed o he labo a o y
expe imen is same when he ype o o de is ei he disc e e o de o mul iple o de :
H0.2: disc e e = mul iple (2)
The hi d null hypo hesis (H
0.3
) s a es ha he ime needed o he labo a o y expe i-
men is equal when he s o age le el is ei he low o high:
H0.3: low = high (3)
The ou h null hypo hesis (H
0.4
) s a es ha he pe o mance o he pick-by- ision
sys em emains he same when he con i ma ion equipmen is ei he a scanne o an
RFID eade :
H0.4: scanne = RFID (4)
The i h null hypo hesis (H
0.5
) s a es ha he o de -picking ime o he labo a o y
expe imen is he same when i ems pe o de line is ei he ew o many:
H0.5: ew = many (5)
The six h null hypo hesis (H
0.6
) s a es ha he o de -picking ime o he labo a o y
expe imen is he same when o de lines pe o de is ei he ew o many:
H0.6: ew = many (6)
3.4. Resul s om he S a is ical Analysis
A e he comple ion o he es s and he collec ion o he da a, a quan i a i e anal-
ysis was conduc ed in o de o e alua e he o de -picking ime o he pick-by- ision
sys em. The de ailed esul s o he ANOVA analysis on o de -picking e iciency a e
p esen ed in Table 6.

Logis ics 2022,6, 84 16 o 26
Table 6.
Resul s o s a is ical analysis (es ima ed e ec s) on o de -picking ime (e iciency). The
symbol * ela es he main e ec s as a as hei in e ac ions a e conce ned.
Sou ce o Va ia ion Te m p-Value
Main E ec s
Ba e y Posi ion 0.55
Type o O de 0.052
S o age Le el 0.732
Con i ma ion Equipmen 0
I ems pe O de Line 0
O de Lines pe O de 0.173
2-way in e ac ions
Ba e y Posi ion*Type o O de 0.35
Ba e y Posi ion*S o age Le el 0.314
Ba e y Posi ion*Con i ma ion Equipmen 0.031
Ba e y Posi ion*I ems pe O de Line 0.925
Ba e y Posi ion*O de Lines pe O de 0.256
Type o O de *S o age Le el 0.042
Type o O de *Con i ma ion Equipmen 0.004
Type o O de *I ems pe O de Line 0.145
Type o O de *O de Lines pe O de 0.053
S o age Le el*Con i ma ion Equipmen 0.014
S o age Le el*I ems pe O de Line 0.621
S o age Le el*O de Lines pe O de 0.062
Con i ma ion Equipmen *I ems pe O de Line 0.008
Con i ma ion Equipmen *O de Lines pe O de 0.219
I ems pe O de Line*O de Lines pe O de 0.019
3-way in e ac ions
Ba e y Posi ion*Type o O de *S o age Le el 0.513
Ba e y Posi ion*Type o O de *Con i ma ion Equipmen 0.085
Ba e y Posi ion*Type o O de *I ems pe O de Line 0.564
Ba e y Posi ion*Type o O de *O de Lines pe O de 0.839
Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen 0.411
Ba e y Posi ion*S o age Le el*I ems pe O de Line 0.77
Ba e y Posi ion*S o age Le el*O de Lines pe O de 0.891
Ba e y Posi ion*Con i ma ion Equipmen *I ems pe O de Line 0.374
Ba e y Posi ion*Con i ma ion Equipmen *O de Lines pe O de 0.251
Ba e y Posi ion*I ems pe O de Line*O de Lines pe O de 0.111
Type o O de *S o age Le el*Con i ma ion Equipmen 0.644
Type o O de *S o age Le el*I ems pe O de Line 0.003
Type o O de *S o age Le el*O de Lines pe O de 0.932
Type o O de *Con i ma ion Equipmen *I ems pe O de Line 0.436
Type o O de *Con i ma ion Equipmen *O de Lines pe O de 0.855
Type o O de *I ems pe O de Line*O de Lines pe O de 0.778
S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.386
S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.092
S o age Le el*I ems pe O de Line*O de Lines pe O de 0.128
Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.005
Logis ics 2022,6, 84 17 o 26
Table 6. Con .
Sou ce o Va ia ion Te m p-Value
4-way in e ac ions
Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen 0.125
Ba e y Posi ion*Type o O de *S o age Le el*I ems pe O de Line 0.815
Ba e y Posi ion*Type o O de *S o age Le el*O de Lines pe O de 0.729
Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *I ems pe O de Line 0.572
Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *O de Lines pe O de 0.181
Ba e y Posi ion*Type o O de *I ems pe O de Line*O de Lines pe O de 0.603
Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.552
Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.797
Ba e y Posi ion*S o age Le el*I ems pe O de Line*O de Lines pe O de 0.53
Ba e y Posi ion*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.111
Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line 0.171
Type o O de *S o age Le el*Con i ma ion Equipmen *O de Lines pe O de 0.963
Type o O de *S o age Le el*I ems pe O de Line*O de Lines pe O de 0.048
Type o O de *Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.422
S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe O de 0.43
5-way in e ac ions
Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line
0.341
Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *O de Lines pe
O de 0.608
Ba e y Posi ion*Type o O de *S o age Le el*I ems pe O de Line*O de Lines pe O de 0.156
Ba e y Posi ion*Type o O de *Con i ma ion Equipmen *I ems pe O de Line*O de
Lines pe O de 0.34
Ba e y Posi ion*S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines
pe O de 0.035
Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de Line*O de Lines pe
O de 0.942
Ba e y Posi ion*Type o O de *S o age Le el*Con i ma ion Equipmen *I ems pe O de
Line*O de Lines pe O de 0.371
Figu e 9p esen s a Pa e o cha ha e i ies he alidi y o he ANOVA analysis,
in ha i dis inguishes he ac o s ha a e s a is ically signi ican . A e he ob ained
esul s ha e been aken in o accoun , i can be seen ha he e a e a numbe o ac o s and
combina ions o ac o s ha signi ican ly a ec he e iciency o he pick-by- ision sys em
unde in es iga ion. The esul s ha e shown ha o cases H
0.1
, H
0.2
, H
0.3
, and H
0.6
, he
null hypo hesis is accep ed, whe eas o cases H
0.4
and H
0.5
, he null hypo hesis has been
ejec ed. To his end, i can be concluded ha he con i ma ion equipmen and i ems pe
o de line a ec he pe o mance o pick-by- ision sys em.
Logis ics 2022,6, 84 18 o 26
Logis ics 2022, 6, x FOR PEER REVIEW 17 o 26
Figu e 9. Pa e o cha o o de -picking e iciency.
In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal
p obabili y plo ollows a s aigh line and ha he e sus i s plo has andomly
dis ibu ed esiduals a ound ze o. The his og am plo has a bell shape, and he e sus
o de plo shape p esen s no speci ic pa e n. The e o e, he da a a e highly eliable.
Figu e 10. Residual plo s o o de -picking e iciency.
Figu e 9. Pa e o cha o o de -picking e iciency.
In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal p obabili y
plo ollows a s aigh line and ha he e sus i s plo has andomly dis ibu ed esiduals
a ound ze o. The his og am plo has a bell shape, and he e sus o de plo shape p esen s
no speci ic pa e n. The e o e, he da a a e highly eliable.
Logis ics 2022, 6, x FOR PEER REVIEW 17 o 26
Figu e 9. Pa e o cha o o de -picking e iciency.
In Figu e 10, he esidual plo s a e p esen ed. I can be seen ha he no mal
p obabili y plo ollows a s aigh line and ha he e sus i s plo has andomly
dis ibu ed esiduals a ound ze o. The his og am plo has a bell shape, and he e sus
o de plo shape p esen s no speci ic pa e n. The e o e, he da a a e highly eliable.
Figu e 10. Residual plo s o o de -picking e iciency.
Figu e 10. Residual plo s o o de -picking e iciency.
Logis ics 2022,6, 84 19 o 26
Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed, we
in es iga ed he le els o he s a is ically signi ican ac o s and he sys em con igu a ion
ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding o Figu e 11, i
can be obse ed ha using he scanne akes less o de -picking ime han using he RFID
ag eade does. Fu he mo e, when i ems pe o de line a e ew, he picking e iciency is
be e han when he i ems pe o de line a e many (Figu e 12).
Logis ics 2022, 6, x FOR PEER REVIEW 18 o 26
Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed,
we in es iga ed he le els o he s a is ically signi ican ac o s and he sys em
con igu a ion ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding
o Figu e 11, i can be obse ed ha using he scanne akes less o de -picking ime han
using he RFID ag eade does. Fu he mo e, when i ems pe o de line a e ew, he
picking e iciency is be e han when he i ems pe o de line a e many (Figu e 12).
Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues).
Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues).
Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking
sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as
signi ican . This is because he es s we e conduc ed in a labo a o y en i onmen whe e
no signi ican a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may
each he shel es om he a ea whe e he ca is by aking a ew s eps. The lack o a el
dis ance a ec ed he signi icance o he ype o o de pa ame e , and hus, he di e ence,
Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues).
Logis ics 2022, 6, x FOR PEER REVIEW 18 o 26
Now ha he in e ac ions be ween ac o s and con igu a ions ha e been analyzed,
we in es iga ed he le els o he s a is ically signi ican ac o s and he sys em
con igu a ion ha esul s in he sho es o de -picking ime. Mo e speci ically, acco ding
o Figu e 11, i can be obse ed ha using he scanne akes less o de -picking ime han
using he RFID ag eade does. Fu he mo e, when i ems pe o de line a e ew, he
picking e iciency is be e han when he i ems pe o de line a e many (Figu e 12).
Figu e 11. Boxplo o o de -picking ime o con i ma ion equipmen (* ep esen s ou lie alues).
Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues).
Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking
sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as
signi ican . This is because he es s we e conduc ed in a labo a o y en i onmen whe e
no signi ican a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may
each he shel es om he a ea whe e he ca is by aking a ew s eps. The lack o a el
dis ance a ec ed he signi icance o he ype o o de pa ame e , and hus, he di e ence,
Figu e 12. Boxplo o o de -picking ime o i ems pe o de line (* ep esen s ou lie alues).
Ano he impo an ac o ha usually a ec s he pe o mance o an o de -picking
sys em is he ype o o de . In ou expe imen , his ac o was no conside ed as signi ican .
This is because he es s we e conduc ed in a labo a o y en i onmen whe e no signi ican
a el dis ances exis . As i can be seen in Figu e 7, he pa icipan s may each he shel es
om he a ea whe e he ca is by aking a ew s eps. The lack o a el dis ance a ec ed
he signi icance o he ype o o de pa ame e , and hus, he di e ence, in e ms o picking
e iciency, be ween disc e e o de picking and mul iple o de picking was elimina ed.
Logis ics 2022,6, 84 20 o 26
Las bu no leas , acco ding o he esul s o he s a is ical analysis, i can be concluded
ha he con igu a ion o he in es iga ed pick-by- ision sys em ha p o ides he mos
encou aging esul s in e ms o o de -picking e iciency (i.e., o de -picking ime pe o de
line) inco po a es he ollowing le els pe pa ame e : ba e y posi ion—weigh ed side;
ype o o de —disc e e o de picking; s o age le el—high s o age le el; con i ma ion
equipmen —scanne ”; i ems pe o de line— ew; and o de lines pe o de — ew.
4. Pe cei ed Wo kload E alua ion
A pe cei ed wo kload e alua ion can be accomplished by many means. NASA TLX is
a widely used subjec i e mul idimensional assessmen ool ha a es pe cei ed wo kload
o assess a ask, sys em, o p ocess [
74
], and acco ding o he a ailable li e a u e, NASA
TLX has achie ed some solid goals in human- ac o s esea ch, while assessing sys em
design and de elopmen phases [
75
]. The NASA TLX is based on a weigh ed a e age o
a ings on six subscales [74,76]. Th ee dimensions a e ela ed o he demands imposed on
he subjec (men al demand, physical demand, and empo al demand) and h ee o he
in e ac ion o a subjec wi h he ask (e o , us a ion, and pe o mance). Acco ding o a
se ies o simila wo ks [
25
,
30
,
33
,
37
,
56
,
65
], i can be seen ha he NASA TLX me hodology
is he mos sui able me hodology o e alua e he pe cei ed wo kload o he p oposed
pick-by- ision echnology. Acco ding o NASA [
74
], he implemen a ion o NASA TLX
ollows wo s eps. The i s s ep deals wi h he sou ce o load (weigh s) and he second s ep
wi h he magni ude o loads ( a ing). Fu he in o ma ion on he implemen a ion s ep o
NASA TLX me hodology is a ailable in he wo k o NASA [74].
Taking in o accoun he a o emen ioned s eps o NASA TLX me hodology, we e al-
ua ed he p oposed pick-by- ision sys em’s pe cei ed wo kload. A e comple ing a
ask (o de picking), e e y pa icipan illed he NASA TLX ques ionnai e, based on he
a o emen ioned s eps and he expe imen e ’s ins uc ions. To his end, in Figu e 13, he
inal esul s o he NASA TLX su ey a e p esen ed. The pick-by- ision sys em sco ed
M = 32.8 (SD = 9.1). The indi idual ac o s p esen ed om low wo kload o high wo kload
sco ed M = 23.4 (SD = 14.2) o pe o mance, M = 26.1 (SD = 17.5) o men al demand,
M = 28.0 (SD = 14.6) o physical demand, M = 30.4 (SD = 20.5) o empo al demand,
M = 33.0 (SD = 16.7) o e o , and M = 33.6 (SD = 19.1) o us a ion le el.
Logis ics 2022, 6, x FOR PEER REVIEW 19 o 26
in e ms o picking e iciency, be ween disc e e o de picking and mul iple o de picking
was elimina ed.
Las bu no leas , acco ding o he esul s o he s a is ical analysis, i can be
concluded ha he con igu a ion o he in es iga ed pick-by- ision sys em ha p o ides
he mos encou aging esul s in e ms o o de -picking e iciency (i.e., o de -picking ime
pe o de line) inco po a es he ollowing le els pe pa ame e : ba e y posi ion—
weigh ed side; ype o o de —disc e e o de picking; s o age le el—high s o age le el;
con i ma ion equipmen —scanne ”; i ems pe o de line— ew; and o de lines pe
o de — ew.
4. Pe cei ed Wo kload E alua ion
A pe cei ed wo kload e alua ion can be accomplished by many means. NASA TLX
is a widely used subjec i e mul idimensional assessmen ool ha a es pe cei ed
wo kload o assess a ask, sys em, o p ocess [74], and acco ding o he a ailable li e a u e,
NASA TLX has achie ed some solid goals in human- ac o s esea ch, while assessing
sys em design and de elopmen phases [75]. The NASA TLX is based on a weigh ed
a e age o a ings on six subscales [74,76]. Th ee dimensions a e ela ed o he demands
imposed on he subjec (men al demand, physical demand, and empo al demand) and
h ee o he in e ac ion o a subjec wi h he ask (e o , us a ion, and pe o mance).
Acco ding o a se ies o simila wo ks [25,30,33,37,56,65], i can be seen ha he NASA
TLX me hodology is he mos sui able me hodology o e alua e he pe cei ed wo kload
o he p oposed pick-by- ision echnology. Acco ding o NASA [74], he implemen a ion
o NASA TLX ollows wo s eps. The i s s ep deals wi h he sou ce o load (weigh s) and
he second s ep wi h he magni ude o loads ( a ing). Fu he in o ma ion on he
implemen a ion s ep o NASA TLX me hodology is a ailable in he wo k o NASA [74].
Taking in o accoun he a o emen ioned s eps o NASA TLX me hodology, we
e alua ed he p oposed pick-by- ision sys em’s pe cei ed wo kload. A e comple ing a
ask (o de picking), e e y pa icipan illed he NASA TLX ques ionnai e, based on he
a o emen ioned s eps and he expe imen e ’s ins uc ions. To his end, in Figu e 13, he
inal esul s o he NASA TLX su ey a e p esen ed. The pick-by- ision sys em sco ed M
= 32.8 (SD = 9.1). The indi idual ac o s p esen ed om low wo kload o high wo kload
sco ed M = 23.4 (SD = 14.2) o pe o mance, M = 26.1 (SD = 17.5) o men al demand, M =
28.0 (SD = 14.6) o physical demand, M = 30.4 (SD = 20.5) o empo al demand, M = 33.0
(SD = 16.7) o e o , and M = 33.6 (SD = 19.1) o us a ion le el.
Figu e 13. NASA TLX esul s o he p oposed pick-by- ision sys em.
Figu e 13. NASA TLX esul s o he p oposed pick-by- ision sys em.
Acco ding o hese esul s, he o e all NASA TLX sco e p o es ha he pe cei ed
wo kload o each pa icipan is sa is yingly small. The o e iew o all he indi idual ac o s
shows ha he pa icipan s had no signi ican p oblems; hus, hey escala ed he subscales

Logis ics 2022,6, 84 21 o 26
wi h small di e ences in o de o dis inguish hem om each o he . Addi ionally, as Casne
and Go e [
77
] suppo , people who a e o e wo king and people who a e unde wo king
exhibi simila pe o mance as hey bo h commi e o s, ha e low e iciency, ge us a ed,
and ha e poo awa eness o hei su oundings. To his end, a e he wo kload gauge om
Casne and Go e [
77
] has been aken in o accoun , i is clea ha he o e all NASA TLX
sco e o he p oposed sys em is kep on accep able le el.
Mo e speci ically, he highes pe cei ed wo kload index came om he us a ion
le el. Indeed, subjec s admi ed ha di icul y in he scanning p ocess was i i a ing hem,
and sligh mo emen s o he headbands caused hem annoyance. Simila ly, e o is he
second-highes ac o , as subjec s concluded ha compa ed wi h he es o he ac o s,
ha d wo k came second. Addi ionally, he labo a o y en i onmen and he exis ence o a
s opwa ch o ced he pa icipan s o pick in ensely. As hey admi ed, he mo e amilia
hey go , he less in ensi y hey el . As a as he physical demand is conce ned, he lack o
a el dis ances and o e all age a oided low sco es, as no a el ime exis ed and c ouching
and bending we e no a p oblem o young people. In e ms o men al demand, he subjec s
el ha hey didn’ need o hink abou wha o do, as hey we e ully guided h oughou
all he p ocess. The lowes sco ing ac o was pe o mance, p o ing ha no special skills
we e needed o conduc he es s. Las bu no leas , in eal wa ehouses, whe e p o es-
sional picke s wo k o eigh hou s, a el dis ances a e longe , he a e age age is highe ,
and no labo a o y expe imen is conduc ed, an inc eased NASA TLX sco e is expec ed
o be obse ed.
5. Discussion
The objec i e o his s udy was o in es iga e ce ain ope a ional and echnical pa am-
e e s ha a ec he pe o mance o pick-by- ision echnology in i em-le el o de picking
ia a se ies o labo a o y es s. The indings o his s udy a e essen ial because hey show
he pa ame e s ( echnical and ope a ional) ha a ec he pe o mance o pick-by- ision
echnology and may esul in inc eased o de -picking e iciency and accu acy. A e he
ob ained esul s ha e been accoun ed o , i can be seen ha he e we e a numbe o ac o s
and combina ions o ac o s ha signi ican ly a ec ed he e iciency o he pick-by- ision
sys em unde in es iga ion. The esul s showed ha he con i ma ion equipmen and i ems
pe o de line a ec he pe o mance o he pick-by- ision sys em. In addi ion, i can be
obse ed ha selec ing he scanne as he piece o o de con i ma ion equipmen educes
o de -picking ime o e using an RFID ag eade . Fu he mo e, when i ems pe o de line
a e ew, he picking e iciency is be e han when i ems pe o de line a e many.
Acco ding o he esul s o he s a is ical analysis, he con igu a ion o he pick-by-
ision sys em ha p o ided he mos encou aging esul s in e ms o o de -picking e -
iciency (i.e., o de -picking ime pe o de line) inco po a ed he ollowing le els pe
pa ame e : (a) ba e y posi ion—weigh ed side; (b) ype o o de —disc e e o de picking;
(c) s o age le el—high s o age le el; (d) con i ma ion equipmen —scanne ; (e) i ems pe
o de line— ew; and ( ) o de lines pe o de — ew.
In he pe cei ed wo kload e alua ion, he o e all NASA TLX sco e p o ed ha he
wo kload o each pa icipan was sa is yingly small. The o e iew o all he indi idual
ac o s showed ha he pa icipan s had no signi ican p oblems; hus, hey escala ed he
subscales wi h small di e ences in o de o dis inguish hem om each o he .
5.1. Con ibu ion o Theo y
On he basis o hese indings, i may be a gued ha he esul s p o ide some key
insigh s in o heo y. Fi s , a se ies o pa ame e s we e es ed, and hei ole conce ning he
pe o mance o pick-by- ision echnology was assessed. O he s udies, such as [
14
,
34
,
68
], ha e
also e alua ed a se ies o pa ame e s, bu no o he ex en o he analysis ha was made in
his s udy. Fu he mo e, an e alua ion o he pa ame e s iden i ied by he SLR me hod was
made by p ac i ione s o he logis ics ield. Mo e speci ically, he iew o logis ics manage s
was aken in o accoun on p ac ical issues (i.e., o de picking p ocess) and was analyzed by
Logis ics 2022,6, 84 22 o 26
using he AHP me hod. In addi ion, al hough o he s udies ha e p esen ed indings om
labo a o y es s conce ning picking accu acy and e iciency when pick-by- ision echnology
was implemen ed, his wo k adop ed he DoE me hodology and a s a is ical analysis o
iden i y co ela ions be ween pa ame e s and he se -up o pa ame e s ha would p o ide
he bes esul in pick-by- ision sys em pe o mance. Las ly, his s udy also p o ided use ul
insigh s in heo ies on pe cei ed wo kload. By adop ing he NASA TLX me hodology,
pick-by- ision echnology can be used by picke s wi h no signi ican p oblems. O he
s udies, such as [
21
,
27
], ha e also e alua ed e gonomic pa ame e s wi hou , howe e ,
aking in o accoun o he echnical and ope a ional pa ame e s ha a e also impo an
du ing an e alua ion o a pick-by- ision sys em. To his end, he main con ibu ion o his
wo k ocused on he iden i ica ion o key pa ame e s ha a ec he pe o mance o pick-by-
ision echnology and he de elopmen o a amewo k o he s uc u ed ca ego iza ion
o hese pa ame e s. Fu he mo e, his wo k p esen ed signi ican esul s om labo a o y
es ing and he bes se -up o he pick-by- ision sys em o inc eased picking e iciency.
Las bu no leas , he pa ame e s es ed in his wo k we e selec ed on he basis o he
answe s ecei ed om he ques ionnai es ( ia he AHP me hod) comple ed by logis ics
manage s who ha e signi ican expe ience in he o de -picking p ocess.
5.2. P ac ical Implica ions
F om he ob ained esul s and h ough in e iews wi h logis ics manage s, a numbe
o use ul p ac ical implica ions a ise. The la e deal mainly wi h (a) o ganiza ional cul u e,
(b) p ocess e-enginee ing, (c) s a esis ance o change, and d) mo i a ion o main aining
he new way o doing business. These implica ions a e discussed below:
•
O ganiza ional cul u e: an e ec i e digi al ans o ma ion om ypical o de -picking
me hods o he pick-by- ision sys em needs mo e han upda ing he cu en echnol-
ogy. Au oma ion ools a e likely o c ea e dissa is ac ion among he wo k o ce i no
managed p ope ly. An o ganiza ional cul u e is needed, he lack o which can cause
he in es men o ail and educed pe o mance.
•
P ocess e-enginee ing: apa om he need o o ganiza ional cul u e, mal unc ion
may be caused by a lack o necessa y p ocess e-enginee ing. I is hus c ucial o com-
panies o iden i y hei needs and adjus hei o de -picking p ocesses in acco dance
wi h he adop ed new echnology.
•
S a esis ance o change: s a end o esis o echnological change because hey
belie e ha hei posi ion is in dange . Typical examples o such si ua ions a e me
in he logis ics sec o when new o de -picking echniques/sys ems a e in oduced.
In o de o keep he wo k o ce and managemen uni ed, con inuous s a aining,
use - iendly sys ems, and echnologies acili a ing wo ke ’s li es a e essen ial.
•
Mo i a ion o main aining he new way o doing business: a e a comple e and
mul ile el ins alla ion o an o de -picking sys em, i is impo an o main ain he new
way o doing business. Con inuous imp o emen is equi ed in o de o main ain he
in e es o he use , as a e sugges ions o imp o emen s om people wo king wi h
he new sys em.
5.3. Limi a ions and Fu u e Resea ch
Al hough, he labo a o y es s conduc ed ha e esul ed in encou aging esul s on
he pe o mance o pick-by- ision echnology, he e a e always oppo uni ies o u u e
esea ch, such as in he ollowing a eas, which we e no in es iga ed in his s udy:
•
Human ac o : he human ac o plays a c i ical ole when new echnologies a e
adop ed, especially o he pick-by- ision sys em, whe e aining and amilia iza ion
a e necessa y o i s use. Despi e he ac ha mos people can wo k o e long pe iods
wi h pick-by- ision headbands o glasses wi hou being s ained, he e a e s ill some
people who ind eading con inuously om a sma de ice di icul . Issues ha should
be u he in es iga ed include e gonomics, men al and physical demand on use s,
pe o mance, and us a ion le el.
Logis ics 2022,6, 84 23 o 26
•
Technical issues: Ano he c i ical issue o u u e in es iga ion is he echnical aspec
o pick-by- ision, especially when i comes o he use in e ace (UI). Fu he mo e,
a en ion should also be paid o issues om in eg a ing he pick-by- ision sys em
in o o he sys ems, especially in he close in eg a ion o augmen ed eali y (AR) and
wa ehouse managemen sys ems (WMSs), he inc easing com o o ha dwa e compo-
nen s, and he po en ial connec ion o picking sys ems wi h au oma ic iden i ica ion
sys ems, such as RFID ags. O he echnical issues o be u he in es iga ed may
include ba e y li e and scanning dis ance.
•
Compa a i e assessmen wi h o he picking echnologies: La ely, a compa a i e as-
sessmen o pick-by- ision wi h al e na i e picking echnologies, such as oice picking
and pick o ligh , ha e come up, bu s ill he e a e many esea ch oppo uni ies in his
a ea. Indeed, expe imen s should be conduc ed in o de o assess he accu acy and
e iciency o di e en picking echnologies. Las bu no leas , i is wo h e alua ing
in es men cos s in o de o compa e no only he pe o mance bu also he cos o
ob aining an o de -picking sys em.
6. Conclusions
The aim o his pape was ini ially o in es iga e ce ain pa ame e s ha a ec he
ope a ional pe o mance o a pick-by- ision sys em ia a se ies o labo a o y es s. A
o al o 27 pa ame e s and h ee pe o mance measu emen indices we e iden i ied ia a
sys ema ic li e a u e e iew. Six o hem we e selec ed ia an analy ical hie a chy p ocess
(AHP), o be in es iga ed u he ia a se ies o labo a o y expe imen s by adop ing he
design o expe imen s (DoE) me hodology. The p oposed pick-by- ision sys em was
in es iga ed in e ms o o de -picking ime and wo kload. The con igu a ion o he pick-
by- ision sys em ha p o ided he mos encou aging esul s in e ms o o de -picking
e iciency (i.e., o de -picking ime pe o de line) inco po a ed he ollowing le els pe
pa ame e : (a) ba e y posi ion—weigh ed side; (b) ype o o de —disc e e o de picking;
(c) s o age le el—high s o age le el; (d) con i ma ion equipmen —scanne ; (e) i ems pe
o de line— ew; and ( ) o de lines pe o de — ew. The pe cei ed wo kload o he pick-
by- ision sys em was e alua ed ia a NASA TLX su ey. The esul s a e encou aging,
showing ha he pick-by- ision echnology can be used by picke s wi h no signi ican
wo kload ine iciencies.
Au ho Con ibu ions:
Concep ualiza ion, V.Z. and A.G.; me hodology, A.G., V.Z.; o mal analysis,
N.C., A.G.; in es iga ion, N.C., A.G., V.Z.; esou ces, V.Z.; da a cu a ion, N.C.; w i ing—o iginal
d a p epa a ion, N.C., A.G., V.Z.; w i ing— e iew and edi ing, A.G., V.Z.; isualiza ion, N.C., A.G.;
supe ision, V.Z.; p ojec adminis a ion, V.Z.; unding acquisi ion, V.Z. All au ho s ha e ead and
ag eed o he published e sion o he manusc ip .
Funding:
This esea ch was unded by he In elligen Resea ch In as uc u e o Shipping, Sup-
ply Chain, T anspo and Logis ics (ENIRISST+) p ojec (MIS 5027930), om he NSRF 2014-2020
Ope a ional P og amme “Compe i i eness, En ep eneu ship, Inno a ion”.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
Re e ences
1.
Ma che , G.; Melacini, M.; Pe o i, S. In es iga ing o de picking sys em adop ion: A case-s udy-based app oach. In . J. Logis .
Res. Appl. 2015,18, 82–98. [C ossRe ]
2.
Lu, W.; McFa lane, D.; Giannikas, V.; Zhang, Q. An algo i hm o dynamic o de -picking in wa ehouse ope a ions. Eu . J. Ope .
Res. 2016,248, 107–122. [C ossRe ]
3.
Van Gils, T.; Ramaeke s, K.; Ca is, A.; de Kos e , R.B.M. Designing e icien o de picking sys ems by combining planning
p oblems: S a e-o - he-a classi ica ion and e iew. Eu . J. Ope . Res. 2018,267, 1–15. [C ossRe ]
4.
F anzke, T.; G osse, E.H.; Glock, C.H.; Elbe , R. An in es iga ion o he e ec s o s o age assignmen and picke ou ing on he
occu ence o picke blocking in manual picke - o-pa s wa ehouses. In . J. Logis . Manag. 2017,28, 841–863. [C ossRe ]
5. Theys, C.; B äysy, O.; Dullae , W.; Raa, B. Using a TSP heu is ic o ou ing o de picke s in wa ehouses. Eu . J. Ope . Res. 2010,
200, 755–763. [C ossRe ]
Logis ics 2022,6, 84 24 o 26
6.
Chen, F.; Wang, H.; Xie, Y.; Qi, C. An ACO-based online ou ing me hod o mul iple o de picke s wi h conges ion conside a ion
in wa ehouse. J. In ell. Manu . 2016,27, 389–408. [C ossRe ]
7.
Wang, S.; Wan, J.; Li, D.; Zhang, C. Implemen ing Sma Fac o y o Indus ie 4.0: An Ou look. In . J. Dis ib. Sens. Ne w.
2016
,
12, 3159805. [C ossRe ]
8.
Hanson, R.; Falkens öm, W.; Mie inen, M. Augmen ed eali y as a means o con eying picking in o ma ion in ki p epa a ion
o mixed-model assembly. Compu . Ind. Eng. 2017,113, 570–575. [C ossRe ]
9.
S ol z, M.H.; Giannikas, V.; McFa lane, D.; S achan, J.; Um, J.; S ini asan, R. Augmen ed Reali y in Wa ehouse Ope a ions:
Oppo uni ies and Ba ie s. IFAC-Pape sOnLine 2017,50, 12979–12984. [C ossRe ]
10.
G osse, E.H.; Glock, C.H.; Neumann, W.P. Human Fac o s in O de Picking: A Con en Analysis o he Li e a u e. In . J. P od. Res.
2017,55, 1260–1276. [C ossRe ]
11.
Van Gils, T.; Ramaeke s, K.; B aeke s, K.; Depai e, B.; Ca is, A. Inc easing o de picking e iciency by in eg a ing s o age, ba ching,
zone picking, and ou ing policy decisions. In . J. P od. Econ. 2018,197, 243–261. [C ossRe ]
12.
Boysen, N.; de Kos e , R.; Weidinge , F. Wa ehousing in he E-Comme ce E a: A Su ey. Eu . J. Ope . Res.
2019
,277, 396–411.
[C ossRe ]
13.
Masae, M.; Glock, C.H.; G osse, E.H. O de picke ou ing in wa ehouses: A sys ema ic li e a u e e iew. In . J. P od. Econ.
2020
,
224, 107564. [C ossRe ]
14.
Gialos, A.; Zeimpekis, V. De ining and es ing sys em pa ame e s o enhancing ision picking echnology in wa ehouse
ope a ions. In Supply Chain 4.0: Imp o ing Supply Chains wi h Analy ics and Indus y 4.0 Technologies; Ak as, E., Bou lakis, M.,
Zeimpekis, V., Minis, I., Eds.; Kogan Page: London, UK, 2021; ISBN 978-1789660753.
15.
T an ield, D.; Denye , D.; Sma , P. Towa ds a me hodology o de eloping e idence-in o med managemen knowledge by means
o sys ema ic e iew. B . J. Manag. 2003,14, 207–222. [C ossRe ]
16.
Touboulic, A.; Walke , H. Theo ies in Sus ainable Supply Chain Managemen : A S uc u ed Li e a u e Re iew. In . J. Phys. Dis ib.
Logis . Manag. 2015,45, 16–42. [C ossRe ]
17.
Lago io, A.; Pin o, R.; Golini, R. Resea ch in U ban Logis ics: A Sys ema ic Li e a u e Re iew. In . J. Phys. Dis ib. Logis . Manag.
2016,46, 908–931. [C ossRe ]
18.
Gialos, A.; Zeimpekis, V. Vision picking echnology: De ining design pa ame e s ia a sys ema ic li e a u e e iew. In . J. Logis .
Sys . Manag. 2020,37, 106–139. [C ossRe ]
19.
Khan, M.; Pa aiz, G.S.; Ali, A.; Jehangi , M.; Hassan, N.; Bae, J. A model o unde s anding he media ing associa ion o
anspa ency be ween eme ging echnologies and humani a ian logis ics sus ainabili y. Sus ainabili y
2022
,14, 16917. [C ossRe ]
20.
Glock, C.H.; G osse, E.H.; Neumann, W.P.; Feldman, A. Assis i e de ices o manual ma e ials handling in wa ehouses: A
sys ema ic li e a u e e iew. In . J. P od. Res. 2021,59, 3446–3469. [C ossRe ]
21.
Khan, M.; Pa aiz, G.S.; Tohi o ich-Dedahano , A.; Iqbal, M.; Junghan, B. Resea ch ends in humani a ian logis ics and
sus ainable de elopmen : A bibliome ic analysis. Cogen Bus. Manag. 2022,9, 2143071. [C ossRe ]
22.
Schwe d ege , B.; F imo , T.; Pus ka, D.; Klinke , G. Mobile In o ma ion P esen a ion Schemes o Sup a-adap i e Logis ics
Applica ions. In Lec u e No es in Compu e Science (Including Subse ies Lec u e No es in A i icial In elligence and Lec u e No es in
Bioin o ma ics); Sp inge : Cham, Swi ze land, 2006; pp. 998–1007. ISBN 3540497765.
23.
Baumann, H.; S a ne , T.; Zschale , P. S udying o de picking in an ope a ing au omobile manu ac u ing plan . In P oceedings o
he P oceedings—In e na ional Symposium on Wea able Compu e s, ISWC, Newcas le Upon Tyne, UK, 18–22 June 2012.
24.
Funk, M.; Shi azi, A.S.; Maye , S.; Lischke, L.; Schmid , A. Pick om He e-An in e ac i e mobile ca using in-si u p ojec ion o
o de picking. In P oceedings o he UbiComp 2015—P oceedings o he 2015 ACM In e na ional Join Con e ence on Pe asi e
and Ubiqui ous Compu ing, Osaka, Japan, 7–11 Sep embe 2015.
25.
Kim, S.; Nussbaum, M.A.; Gabba d, J.L. In luences o augmen ed eali y head-wo n display ype and use in e ace design on
pe o mance and usabili y in simula ed wa ehouse o de picking. Appl. E gon. 2019,74, 186–193. [C ossRe ] [PubMed]
26.
Ma sumo o, T.; Kosaka, T.; Saku ada, T.; Nakajima, Y.; Tano, S. Picking wo k using AR ins uc ions in wa ehouses. In P oceedings
o he 2019 IEEE 8 h Global Con e ence on Consume Elec onics (GCCE), Osaka, Japan, 15–18 Oc obe 2019; pp. 31–34. [C ossRe ]
27.
Smi h, E.; Bu ch, V.R.F.; S awde man, L.; Chande , H.; Smi h, B.K. A com o analysis o using sma glasses du ing “picking”
and “pu ing” asks. In . J. Ind. E gon. 2021,83, 103133. [C ossRe ]
28. K ajco ic, M.; Gabajo a, G.; Micie a, B. O de picking using augmen ed eali y. Komunikacie 2014,16, 106–111. [C ossRe ]
29.
Die e, A.; Sz yle , T.; Weiland, L.; S uckenschmid , H. Explo ing a mul i-senso picking p ocess in he u u e wa ehouse. In
P oceedings o he UbiComp ’16: P oceedings o he 2016 ACM In e na ional Join Con e ence on Pe asi e and Ubiqui ous Compu ing:
Adjunc , Heidelbe g, Ge many, 12–16 Sep embe 2016; pp. 1755–1758. [C ossRe ]
30.
Renne , P.; P ei e , T. Augmen ed Reali y Assis ance in he Cen al Field-o -View Ou pe o ms Pe iphe al Displays o O de
Picking: Resul s om a Vi ual Reali y Simula ion S udy. In P oceedings o he Adjunc P oceedings o he 2017 IEEE In e na ional
Symposium on Mixed and Augmen ed Reali y, ISMAR-Adjunc , Nan es, F ance, 9–13 Oc obe 2017.
31.
Elbe , R.; Knigge, J.K.; Sa now, T. T ans e abili y o o de picking pe o mance and aining e ec s achie ed in a i ual eali y
using head moun ed de ices. IFAC-Pap. 2018,51, 686–691. [C ossRe ]
32.
Rei , R.; Gün hne , W.A. Pick-by- ision: Augmen ed eali y suppo ed o de picking. Vis. Compu .
2009
,25, 461–467. [C ossRe ]