applied
sciences
A icle
A P ac ical App oach o Picking I ems in an Online
Shopping Wa ehouse
Na aliya Nechypo enko 1, An onio Mo ales 1,* , En ic Ce e a 1and Angel P. del Pobil 1,2


Ci a ion: Nechypo enko, N.;
Mo ales, A.; Ce e a, E.; del Pobil,
A.P. A P ac ical App oach o Picking
I ems in an Online Shopping
Wa ehouse. Appl. Sci. 2021,11, 5805.
h ps://doi.o g/10.3390/
app11135805
Academic Edi o : Manuel A mada
Recei ed: 4 May 2021
Accep ed: 8 June 2021
Published: 23 June 2021
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2021 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
1Robo ic In elligence Lab., Uni e si a Jaume I, 12071 Cas ellón, Spain;
[email p o ec ed] (N.N.); [email p o ec ed] (E.C.); [email p o ec ed] (A.P.d.P.)
2Depa men o In e ac ion Science, Sungkyunkwan Uni e si y, Seoul 03063, Ko ea
*Co espondence: [email p o ec ed]
Abs ac :
Comme cially iable au oma ed picking in uns uc u ed en i onmen s by a obo a m
emains a di icul challenge. The p oblem o obo g asp planning has long been a ound bu he
exis ing solu ions end o be limi ed when i comes o deploy hem in open-ended ealis ic scena ios.
P ac ical picking sys ems a e called o ha can handle he di e en p ope ies o he objec s o be
manipula ed, as well as he p oblems a ising om occlusions and cons ained accessibili y. This
pape p esen s a p ac ical solu ion o he p oblem o obo picking in an online shopping wa ehouse
by means o a no el app oach ha in eg a es a ca e ully selec ed me hod wi h a new s a egy, he
cen oid no mal app oach (CNA), on a cos -e ec i e dual-a m obo ic sys em wi h wo g ippe s
speci ically designed o his pu pose: a wo- inge g ippe and a acuum g ippe . Objec s iden i ied
in he scene poin cloud a e ma ched o he g asping echniques and g ippe s o maximize success.
Ex ensi e expe imen a ion p o ides clues as o wha a e he easons o success and ailu e. We chose
as benchma k he scena io p oposed by he 2017 Amazon Robo ics Challenge, since i ep esen s a
ealis ic desc ip ion o a e ail shopping wa ehouse case; i includes many challenging cons ain s,
such as a wide a ie y o di e en p oduc i ems wi h a di e si y o p ope ies, which a e also
p esen ed wi h es ic ed isibili y and accessibili y.
Keywo ds: obo ics; g asping; wa ehouse au oma ion; manipula ion
1. In oduc ion
La ge depo s ha con ain millions o di e en i ems a e becoming mo e common as
online e ail se ices o e huge ca alogs o wo ldwide po en ial cus ome s. This demands
no el solu ions o au oma ion on inc easingly a ied condi ions. Robo picking is no an
excep ion: incoming i ems need o be s owed, s o ed empo a ily and la e e ie ed o
a end o cus ome o de s. The au onomous manipula ion o a la ge a ie y o di e en
manu ac u ed p oduc s is s ill a majo challenge ha canno be easily achie ed wi h a
unique solu ion, ei he a uni e sal g ippe o a single-g asp planning algo i hm.
The p oblem o g asp planning has been a ound in he obo ics communi y o a long
ime [
1
]; howe e , he exis ing solu ions end o be limi ed when i comes o deploying
hem in open-ended ealis ic scena ios. P ac ical picking sys ems a e called o ha can
i s cope wi h he di e en p ope ies o he objec s o be manipula ed: di e en shapes,
igid s. a icula ed s. so objec s and ex u ed s. un ex u ed s. anspa en su aces.
Second, i i ems a e no s o ed sepa a ely bu in clu e s, hen ecogni ion and loca ion
sys ems ha e o deal wi h occlusions and cons ained accessibili y. These ypes o scena ios
equi e he solu ion o complex sub-p oblems such as eliable objec modeling, ecogni ion
and loca ion, g asp and pa h planning in he p esence o unce ain y and obs acles, obus
execu ion and many o he s.
Nechypo enko and co-wo ke s conduc ed a s udy compa ing he AGILE (An ipodal
G asp Iden i ica ion and LEa ning ) and HAF (Heigh Accumula ed Fea u es) me hods
o obo g asping [
2
]. This pape builds on ha p elimina y wo k; he e we ocus on he
Appl. Sci. 2021,11, 5805. h ps://doi.o g/10.3390/app11135805 h ps://www.mdpi.com/jou nal/applsci
Appl. Sci. 2021,11, 5805 2 o 24
p oblem o planning he g asp and i s execu ion, unde he condi ions men ioned abo e.
G asp syn hesis e e s o he p oblem o inding a g asp con igu a ion ha sa is ies a se
o c i e ia ele an o he g asping ask [
1
]. The gene a ion, e alua ion and selec ion o
g asps can be pe o med in di e en ways; in he ollowing pa ag aphs we e iew he
s a e-o - he-a on he opic o obo g asping bu limi ing i s scope o app oaches ha a e
pe inen o picking objec s om a con aine , such as a o e o a bin.
Task-based g asping has been sepa a ely s udied in he con ex o Bayesian ne wo ks
o encoding he p obabilis ic ela ions among a ious ask- ele an a iables [
3
]. The
syn hesis o he ca ego y and ask has been pe o med based on 2D and 3D da a om
low-le el ea u es [
4
,
5
]. Ins ead o elying on senso da a poin s, ano he p oposal is o
syn hesize g asps based on seman ic con en wi h he hope o yielding mo e s able g asps
ha a e unc ionally sui able o speci ic objec manipula ion asks [6,7].
Mille e al. used shape p imi i es such as sphe es, cones and boxes o app oxima e
objec shape in he G aspI simula ion en i onmen [
8
]. In ano he app oach, supe ised
lea ning wi h local pa ch-based image and dep h ea u es was used o g asping no el
objec s in clu e ed en i onmen s [
9
]. Heigh maps o he ep esen a ion o ea u es
ha e been p oposed o g asp planning wi h a epo ed 92% success a e o single objec
g asping and aking only 2–3 s [
10
,
11
]; he implemen a ion on he Bax e obo is ema kable
since i has a p ecision o only 1cm, and a simple g ippe is used. Using ea u es based on
geome y, a combina ion o analy ical and da a-d i en me hods was also p oposed [
12
].
Finally, ano he app oach also uses ea u es bu ins ead o a 3D senso , i elies on he
supe ised deep lea ning o 2D RGB images [
13
]; as is common o deep lea ning, his
me hod equi es bo h a la ge da ase and a long aining ime. A p oposal o alle ia e
his las d awback has been ecen ly pu o wa d o a ela ed p oblem in wa ehouse
au oma ion [14].
In ecen yea s, a solid end has eme ged o apply machine and deep lea ning o deal
wi h objec ecogni ion, g asp planning and o he componen s o he pick-and-place pipeline.
In some cases, con olu ional ne wo ks a e applied exclusi ely o image p ocessing [
15
],
whe eas in o he cases, lea ning echniques aim o ob ain models ha link isual pe cep ion
o g asp planning. Some exhaus i e echniques execu e in simula ion millions o g asps on
a i icially gene a ed i ual objec s in o de o lea n he ela ionship be ween shapes and
success ul g asps [
16
], o e en y o connec simula ed poin clouds o p omising suc ion
placemen s on objec su aces [
17
,
18
]. Rein o cemen lea ning app oaches ha e also p o ed
o be app op ia e o his kind o applica ions. In some cases, demons a ed ajec o ies
ha e been used as aining da a o he pa h planning o he a m [
19
,
20
]. In o he cases,
a isual o ask success ewa d has been used o adap he g asping s a egies [
21
,
22
].
In gene al, lea ning-based app oaches a e e y ask speci ic o equi e ex ensi e and
exhaus i e compu a ional aining.
S ill, comme cially iable au oma ed picking in uns uc u ed en i onmen s by a obo
a m emains a di icul challenge. This pape p oposes a new app oach o his p oblem
based on he expe ience ga he ed by ou pa icipa ion in wo edi ions o he Amazon
Robo ics Challenge. Indeed, a p elimina y e sion o ou sys em [
2
] was used in combina ion
wi h an objec ecogni ion module [
23
] o success ul pa icipa ion in he Amazon Robo ics
Challenge 2017, (ARC’17) [
24
]. As a es bed o he expe imen s desc ibed in his pape ,
we use he scena io de ined by ha edi ion o he compe i ion, which includes ealis ic
cons ain s o an online shopping wa ehouse.
The scena io o he Amazon Robo ics Challenge has been used as a amewo k o es a
numbe o ap oaches. Fou speci ic g asping s a egies we e p oposed depending on he
shape o he objec and he ype o end e ec o —ei he g ippe o suc ion cup [
18
]. Lea ning
om demons a ion was used o compu e he each- o-g asp ac ion and heu is ically
sugges he bes con ac poin s [
20
]. D’A ella e al. used a Bax e dual-a m obo o pick
objec s om a box by means o a dep h analysis o he RGB-D image o he scene, along
wi h a cus om designed jamming end-e ec o [25].
Appl. Sci. 2021,11, 5805 3 o 24
Ou solu ion le e ages s a e-o - he-a g asp planning me hods in eg a ed wi h a
new ad hoc algo i hm. The main con ibu ion o his pape is a p ac ical solu ion o he
p oblem o obo picking in uns uc u ed en i onmen s by means o a no el app oach ha
in eg a es a ca e ully selec ed me hod wi h a new s a egy, he cen oid no mal app oach
(CNA), on a cos -e ec i e dual-a m obo ic sys em wi h wo g ippe s speci ically designed
o his pu pose. Objec s a e ma ched o he g asping echniques and g ippe s o maximize
success, and ex ensi e expe imen a ion p o ides clues as o wha a e he easons o
success and ailu e. In e ms o he de eloped me hods, he main con ibu ion o ou
esea ch is he c ea ion and es ing o he CNA algo i hm ha , using he poin cloud and
he majo g aspable componen o he objec , is able o ind he cen oid and i s no mals
co esponding o he la es pa o he obje ’s poin cloud; hen, g asps a e o a ed a ound
he e ical z-axis so ha he inal g asp is mos adequa e o he obo ’s kinema ics.
The pape is o ganized as ollows. Sec ion 2desc ibes ou global sys em along wi h
he asks o be pe o med by his sys em in ag eemen wi h he speci ica ions o he ARC’17.
I has o be no ed ha ARC’17 was he las edi ion o he Amazon Robo ics Challenge. The
a ionale o choose his scena io is ha i has become a de ac o benchma k in he obo ics
communi y. Sec ion 3desc ibes in de ail he g asp planning algo i hms ha cons i u e
he co e o ou app oach. Sec ion 4desc ibes he expe imen s ca ied ou o e alua e he
sys em, and he esul s a e discussed in Sec ion 5. Sys ema ic es s unde he benchma king
condi ions o he Amazon Challenge a e desc ibed and discussed in Sec ion 6. Finally,
Sec ion 7summa izes ou con ibu ions.
2. Sys em Desc ip ion
Amazon Robo ics aims o au oma e he ask o cus ome o de placemen and deli e y
o i s p oduc s. Amazon’s au oma ed wa ehouses success ully emo e he walking and
sea ching o he objec , bu au oma ed picking s ill emains a di icul challenge. The
Amazon Robo ics Challenge was o ganized in o de o spu he ad ancemen o hese unda-
men al echnologies ha , in he end, could be used a wa ehouses all o e he wo ld [
26
].
The challenge en an s could use hei own obo ha dwa e and so wa e o a emp o
sol e somewha simpli ied e sions o he gene al asks o picking and s owing i ems on
wa ehouse shel es.
The challenge e en consis ed o wo asks: picking and s owing, i s independen ly
and hen combined in a inal ask. The obo s a e sco ed based on how many i ems a e
picked and s owed, in a ixed amoun o ime, om a s o age sys em in o a box/bin and
ice e sa. Some i ems a e known in ad ance, so hey a e a ailable o expe imen a ion and
aining be o ehand; some o he i ems a e unknown, meaning ha hey a e only shown
o he eams some minu es be o e s a ing he ac ual challenge. A e he compe i ion is
comple e, he eams sha e and dissemina e hei app oach o imp o e u u e challenge
esul s and indus ial implemen a ions [27–31].
The i ems o be picked by he obo ha e been selec ed no only because o hei
common occu ence in wa ehouses and households bu also because o hei a ied shape
and na u e. Figu e 1shows he ull se o o y benchma k i ems om ARC’17; all hese
i ems a e known in ad ance. In e ms o g asping, he di icul y lies in i em dimensions,
ex u e and poin cloud ep esen a ion. Some i ems, such as he ba h sponge can easily slip
h ough he inge s o he g ippe and o he s, such as he ma bles, le s h ough acuum ai
p essu e. These complica ions call o algo i hms and g ippe combina ions ha a e obus
o changes in objec o ien a ion, shape and ex u e.
Appl. Sci. 2021,11, 5805 4 o 24
Figu e 1. O icial se o benchma k i ems om ARC’17. All hese i ems a e known in ad ance.
The RobinLab sys em is based on a dual-a m Bax e manipula o by Re hink Robo ics
wi h wo cus om-made g ippe s (Figu e 2) and a shel ing design based on a eliable
indus ial solu ion [
24
]. Bins can smoo hly slide on a sys em o ee- o a ing olle s ha
a e ac ua ed by an ex e nal mechanism a ached o he obo sys em. The pu pose o his
s o age sys em is o allow a compac packing o he i ems on he bins and, a he same ime,
Appl. Sci. 2021,11, 5805 5 o 24
p o ide a con enien access o he g ippe s and came as om he op. The whole sys em
se up can be seen in Figu e 3.
The unc ion o he ision sys em is o iden i y and localize he objec s in he s o age
sys em in such a way ha he g asping sys em can e ie e speci ic a ge i ems. The
p ima y design goal was o con e ge in o an accu a e se o wo king algo i hms o objec
ecogni ion as well as c ea ing a poin cloud o one speci ic i em. Di e en lib a ies used o
accomplish hese asks a e in eg a ed in o he sys em. The ision sys em diag am is shown
in Figu e 4; he ha dwa e p o ides an RGB image om which objec s a e ecognized, along
wi h a dep h image om which a poin cloud is gene a ed. The ision pipeline module
handles he ecogni ion o he objec s by means o he combina ion o h ee me hods (SIFT,
SegNe and ResNe ) in a way ha maximizes he numbe o objec s ound and minimizes
he numbe o alse posi i es [23].
The ARC’17 ask is in line wi h applica ions in a wa ehouse, p oduc ion line, labo a o y
o household in which a obo has o analyze a scene in on o i and hen manipula e
objec s. The obo has o sa ely ope a e in a es ic ed wo k space, equi ing p ecise ye
simple kinema ic con igu a ions o allow o p edic abili y and ope abili y. The e is low
isibili y wi hin he en i onmen and he obo mus handle objec s in a clu e inside a
small box. Gi en ha he objec s will be bo h known and unknown and he obo needs o
ope a e in eal ime, he compu a ion has o be pe o med quickly and e icien ly. Low-cos
con enien g ippe s wi h a as -ac ing algo i hm has o be pai ed o he highes success o
objec picking.
In addi ion, we added a acuum sys em in o de o o e al e na i es when g ipping
is un eliable o no possible. This kind o mechanism has been p esen ed wi h comple e
p ac ical and heo e ical analyses o o he ela ed app oaches [27,30,31].
(a) Vacuum g ippe (b) Pinche g ippe
Figu e 2. End e ec o s.
The inpu ha he g asping module ecei es om he ision module is composed o
he objec ha has been iden i ied, along wi h i s loca ion as gi en by an app oxima e poin
cloud [
23
]. The g asping algo i hm does no pe o m any u he e i ica ion o he iden i y
o he objec . I s pu pose is o compu e he posi ion and o ien a ion o one o se e al g asp
con igu a ions ha can be ans e ed o manipula ion con ol.
The obo hen has o mo e he a m in such a way ha he e minal elemen ends
up in he desi ed loca ion o g asp he objec . The wo g ippe s ha ha e been moun ed
on he Bax e obo a e depic ed in Figu e 2. The i s g ippe is a acuum g ippe ha
uses ai p essu e o pick up objec s. The second g ippe is a wo- inge g ippe wi h a
limi ed opening wid h, ha has been named he Pinche . The goal now is o de elop a
g asp planning app oach ha op imizes he pe o mance o hese g ippe s in a wa ehouse
en i onmen .

Appl. Sci. 2021,11, 5805 6 o 24
Vacuum
G ippe Pinche
S o age
Sys em
Robo
Kinec 2.0
Figu e 3. UJI RobinLab obo pla o m se up.
Figu e 4. Vision sys em diag am.
Appl. Sci. 2021,11, 5805 7 o 24
3. G asp Planning
Two s a e-o - he-a algo i hms o he wo- inge g ippe ha e been selec ed, analyzed
and compa ed. The mos sui able one was subjec ed o u he es ing based on he analysis
o ime, obus ness and success.
In his sec ion we p o ide heo e ical knowledge o he wo algo i hms ha we e
conside ed. F om a bi d’s eye pe spec i e, he sepa a e algo i hms ha e hei indi idual
en i onmen s in which each one can exhibi i s s eng h. HAF akes in o accoun he heigh
o he objec s and is used wi h a op g asp he eby educing he dimensionali y o he
p oblem [
10
,
11
]. AGILE (an ipodal g asp iden i ica ion and lea ning) explo es he geome y
o he whole objec in o de o ind handle-like sec ions o exploi o g asping and hus i
o en aims a side g asps [
12
]. These ea u e-based algo i hms can easily be compa ed wi h
ou p oposed me hod, CNA (cen oid no mals app oach), which is compu a ionally less
hea y and hus as e and also mo e adep a g asping la objec s such as books o olde s
o which a wo- inge g ippe can only succeed wi h a side g ip [32].
3.1. Heigh Accumula ed Fea u es (HAF)
As he name sugges s, he HAF algo i hm u ilizes he heigh s o su ace poin s,
ga he ed om he poin cloud da a, ela i e o hei neighbo s in o de o lea n how
o g asp he objec s. The au ho s s ess h ee impo an ad an ages o he algo i hm;
segmen a ion independen ,in eg a ed pa h planning and use o known dep h egions [10,11].
The e m heigh e e s o he measu e o he pe pendicula dis ance om he able
plane o he poin s on he op su ace o he objec . The inpu poin cloud is i s disc e ized
and he heigh g id
H
now con ains a 1
×
1 cm
2
cell ha sa es he highes z- alued poin s
wi h co esponding x and y alues [
33
]. HAF ea u es a e de ined simila ly o Haa Basis
unc ions. All heigh g id alues o each egion,
Ri
, on a heigh g id
H
, a e summed up. The
sums
i
a e indi idually weigh ed by
wi
and hen summed up. The egions and weigh s a e
dependen on he HAF ea u e ha a e de ined by an SVM classi ica ion. A ea u e alue,
i, is de ined as he weigh ed sum o all egions. The j h HAF alue jis calcula ed as:
j=
n Regionsj
∑
i=1
wi,j· i,j(1)
i,j=∑
k,l∈N:H(k,l)∈Ri,j
H(k,l)(2)
The pape claims o ha e es ed 71,000 ea u es (70,000 o which a e au oma ically
gene a ed) and inally selec ed 300 o 325 wi h an F-sco e selec ion [
11
]. Figu e 5 isualizes
he calcula ions in p ocess o ou implemen a ion o he HAF algo i hm; i co esponds o
one o he objec s in ARC’17.
The g een ba s indica e he iden i ied po en ial g asps, whe eas he heigh o he ba s
ep esen s he g asp e alua ion sco e. The ame (g een, ed and blue segmen s) ep esen s
he inal g asp hypo hesis chosen by he algo i hm and indica es he inal posi ion whe e
he end e ec o should go.
Appl. Sci. 2021,11, 5805 8 o 24
Figu e 5. HAF isualiza ion o he ennis ball con aine : (le ) iew om he Kinec 2.0; ( igh ) iew om Bax e .
3.2. An ipodal G asp Iden i ica ion and Lea ning (AGILE)
AGILE g asping is an algo i hm ha uses a poin cloud o p edic he p esence o
geome ic condi ions ha a e indica i e o good g asps on an objec [
12
]. Fi s , geome y
is used o educe he size o he sample space by applying he condi ions equi ed o a
g asp o exis : he hand mus be collision- ee, pa o he objec su ace mus be con ained
be ween wo inge s and he g asp is an ipodal. A pai o poin con ac s wi h ic ion
is an ipodal i and only i he line connec ing he con ac poin s lies inside bo h ic ion
cones [
34
]. Then, he emaining g asps a e classi ied using machine lea ning; geome y is
used in o de o au oma ically label he aining se .
G asp geome y is quan i ied by ce ain pa ame e s. The eason why his algo i hm is
easy o implemen is ha hese pa ame e s a e easy o une depending on he dimensions
o he wo- inge g ippe . The g ippe is speci ied by he pa ame e s
θ= (θl
,
θw
,
θd
,
θ )
which, espec i ely, s and o g ippe leng h, wid h, he dis ance be ween wo inge s and
he hickness o inge s. The me hod elies on ea u es: classi ica ion o hand hypo hesis
uses a ea u e desc ip o o a hand hypo hesis as seen in Figu e 6. In he his og am o
g adien s (HOG) ea u e desc ip o , he dis ibu ion (his og ams) o di ec ions o g adien s
(o ien ed g adien s) a e used as ea u es. G adien s (x and y de i a i es) o an image a e
use ul because he magni ude o g adien s is la ge a ound edges and co ne s ( egions o
ab up in ensi y changes) and edges and co ne s pack in a lo mo e in o ma ion abou
objec shape han la egions [35].
Figu e 6. An example o he HOG ea u e ep esen a ion Adapa ed om [36].
3.3. Cen oid No mals App oach (CNA)
The idea behind he new cen oid no mals app oach (CNA) me hod comes om he
obse a ion ha many manu ac u ed objec s a e symme ic and ha e a la ge cen al su ace
ha is mos sui able o c ea ing an ai sealed g asp. The main idea is o ecei e a poin
cloud and downsample i using a oxel g id. Then, a cylinde o a plane is ex ac ed,
depending on he mos p ominen objec shape. Finally, using he ex ac ed shape, g asps
loca ed in i s cen e a e ound by using su ace no mals and Eule o qua e nion o a ions.
Qua e nions a e used o con enience wi h he pu pose o simpli ying he analysis o he
Appl. Sci. 2021,11, 5805 9 o 24
o ien a ion o an objec ha can be ei he lying on a la su ace o leaning agains a wall.
Figu e 7shows he logic low o he app oach.
Figu e 7. CNA logic low.
The PCL lib a y con ains he ollowing op ions o c ea ing a model om a se o poin
cloud da a poin s: andom sample consensus (RANSAC), leas median o squa es (LMEDS),
M-es ima o sample consensus (MSAC), andomized RANSAC (RRANSAC), andomized
MSAC (RMSAC), maximum likelihood es ima ion sample consensus (MLESA) and p og es-
si e sample consensus (PROSAC). A ho ough compa ison based on accu acy, compu ing,
ime and obus ness was pe o med be ween RANSAC and i s descendan s as well as o he
consensus models by [
37
,
38
]. Based on he analysis o his wo k, PROSAC has been chosen
o implemen a ion.
Once he ele an shape has been ex ac ed, which would be plana (in he case o a
book) o cylind ical (e.g., o he ennis ball con aine ), he algo i hm inds he cen oid o
he 3D a e age o all he poin s ed in o i . Then i calcula es he su ace no mals closes o
he cen oid.
3.3.1. Kinema ic Cons ain s
When he g ippe s we e moun ed on o he Bax e obo in such a way ha hey a e
pe pendicula wi h espec o he w is , he wis o he g ippe was mo ed om join W2
o join W1: Figu e 8shows he join s o he Bax e obo , he o iginal con igu a ion (using
W2 o wis ing a ound Z-axis) and he chosen con igu a ion (using W1).
Figu e 8.
Con igu a ion o he w is join s: (
le
) Bax e join s; (
cen e
) s anda d con igu a ion using W2 o Z-axis wis ;
( igh ) ou con igu a ion using W1 o Z-axis wis .
The es ic ion o wis means ha he angula oll along he z-axis o he g ippe is also
es ic ed. Table 1summa izes he angula ange in each join . No e ha he angula mo ion
has been educed by 140.5 deg ees by eo ien ing he g ippe as seen on
Figu e 3
. Ou side
o he mo emen ange, he obo will no be able o mo e and he in e se kinema ic (IK)
solu ion will no be ound.
Appl. Sci. 2021,11, 5805 16 o 24
Figu e 14.
Pe cen success pe g asping s a egy wi h isola ed objec s; each success a e has been
de e mined o he co esponding subse o objec s. The o al a e age is weigh ed wi h he pe cen age
o i ems pe s a egy, including he 6 blacklis ed no _possible i ems.
In he case o CNA wi h acuum, he main eason, as seen in Figu e 15a, is he
inconsis ency o objec ex u e. Fo example, he pie pla es i em has a back side ha is
pe ec ly la and has 100% chance o being picked up; howe e , es ing was pe o med
on all sides and he o he side is cu ed and ai p essu e slips be ween he holes. On he
o he side, he pie pla es i em has been picked up 0/5 imes; as such, he o al success can
only be 50%. Figu e 15b shows he eason behind he ailu e o CNA wi h Pinche . The
main eason is ha he objec slips. The ma e ial o an objec such as able clo h is e y hin
and hus d ops unexpec edly. Ano he majo eason is ha he ex u e is inconsis en . An
objec such as black ashion glo es has a label and i one o he inge s ouches his label
hen i glides ac oss he label wi hou g abbing any o he ma e ial. No e ha in he abo e
case he CNA algo i hm is success ul despi e he ha dwa e es ic ions. This is due o he
p eassigned angula olls a ound he z-axis o he obo w is . This echnique in CNA
allows o he con igu a ion o always be com o able o Bax e and he IK o always ind
a solu ion. Hence he s eng h o his algo i hm is i s independence o he o ien a ion o
he w is . Figu e 15c shows he easons o he ailu e o he HAF algo i hm wi h Pinche .
This combina ion shows a low success a e o 25%. The eason behind his low success a e
can be a ibu ed o he ha dwa e es ic ions: a g asp can always be calcula ed, e en i i
is no he bes , bu i can a ely be execu ed. Ano he eason, especially ele an o he
hand weigh i em, is obo imp ecision: he Pinche opening allows abou a millime e o
clea ance wi h he cen e o he hand weigh and he obo has 1cm accu acy, hence he
obo simply misses he co ec g asp loca ion.
(a) CNA wi h acuum (b) CNA wi h Pinche (c) HAF wi h Pinche
Figu e 15. Reasons o ailu e o he di e en me hods.

Appl. Sci. 2021,11, 5805 17 o 24
The condi ions o es ing ha e been as ealis ic as possible. I is e y impo an o
e alua e a g asp in his pe spec i e because g asping c i e ia o en only esul s in success
wi h ega ds o he objec , no o he scene. Despi e he se back desc ibed abo e, he o al
success a e, as shown in Figu e 14, shows a 69% success o objec g asping.
To illus a e he esul s, Figu e 16 shows he success ul g asps o h ee di e en i ems
wi h CNA and he acuum g ippe , Figu e 17 shows o he success ul g asps o lexible and
de o mable i ems wi h CNA and he Pinche , and, inally, a success ul g asp o one o he
di icul i ems ( he oile b ush) is achie ed wi h HAF and he Pinche (Figu e 18).
Figu e 16. CNA wi h acuum; success ul g asp o he Reynold’s w ap, ennis ball con aine and I ish sp ing soap.
Appl. Sci. 2021,11, 5805 18 o 24
Figu e 17. CNA wi h Pinche ; success ul g asp o balloons, ace clo h and able clo h.
Figu e 18. HAF wi h Pinche ; success ul g asp o he oile b ush.
6. Sys em Pe o mance Tes s Based on ARC’17 Benchma k
Eigh es s wi h di e en se ups we e pe o med in o de o es he sys em pe o -
mance in he picking ask, eplica ing he condi ions o he ARC’17 as a benchma k. Fo
each es , 10 a ge i ems, dis ibu ed among he 5 bins o he s o age sys em, had o be
Appl. Sci. 2021,11, 5805 19 o 24
iden i ied by he ision sys em, and a co ec g asp had o be planned o pick hem up.
In each es , he 32 i ems in he bins we e a combina ion o known i ems ( om he se o
40 objec s shown in Figu e 1) and unknown i ems. Since hese unknown i ems we e no
p o ided by Amazon, we used he 12 i ems shown in Figu e 19 as ou unknown se ; as in
he compe i ion, hey a e simila , bu di e en , o some o he known i ems. Fu he , hey
we e only a ailable o he sys em 30 min be o e each es .
The i s wo es s a e explained in de ail below o illus a e he es ing condi ions
and make obse a ions abou he sys em. The in o ma ion ga he ed om all he es s is
summa ized la e on.
Figu e 19. Se o unknown i ems. These i ems a e only a ailable 30 min be o e each es .
6.1. Tes 1
The ini ial se up o his es can be seen in Figu e 20. The 10 a ge i ems o be g asped
a e as ollows: In bin A he e is he unknown a ge i em ‘big duck’, and he known ‘glue
s icks’ and ‘balloons’. In bin B, he e is he unknown ‘glue’. In bin C, bo h he ‘c ayola ce a’
and ‘Ad en u es o Hucklebe y Finn’, which a e unknown. In bin D, he ‘glasses’ and
‘b ush’ a e unknown, and he ‘ba h sponge’ and ‘windex’ a e known. The e a e no a ge
i ems in bin E, he e o e i is igno ed o his es .
Figu e 20. Ini ial se up o Tes 1.
Appl. Sci. 2021,11, 5805 20 o 24
In his case, all he a ge objec s we e success ully picked up wi h he excep ion o he
‘big duck’ i em ha was no ecognized by he ision subsys em. The eason was ha i
was o e lapped by he ‘small duck’ i em ha was lipped o e i by he Pinche g ippe
when he ‘balloons’ we e picked up.
6.2. Tes 2
The ini ial se up o his es can be seen in Figu e 21. In bin A he e a e he known
a ge i ems ‘ lashligh ’ and ‘black ashion glo es’. In bin B, he e a e he unknown ‘big
duck’, and known ‘whi e aceclo h’ and ‘ma bles’. In bin C, bo h he ‘ uban isolan ’ and
‘balls’, which a e unknown. In bin D, he unknown ‘b ush’. In bin E, he known ‘mesh-cup’
and ‘duc - ape’.
In his case, h ee i ems we e no success ully picked up: ‘duc ape’ was no ecog-
nized, and e en hough ‘balls’ and ‘ma bles’ we e co ec ly iden i ied, picking up wi h he
acuum g ippe and he Pinche g ippe , espec i ely, ailed.
Figu e 21. Ini ial se up o Tes 2.
Appl. Sci. 2021,11, 5805 21 o 24
6.3. Summa y o Conduc ed Tes s
The summa y o he eigh conduc ed es s can be seen in Table 4. The numbe o i ems
ha we e success ully picked up is shown o each es . In he second and hi d columns,
espec i ely, he pe cen age o he a ge known i ems ha we e co ec ly picked is shown,
along wi h ha pe cen age o he unknown i ems. The ime o he ask o inish is also
shown (no e ha he e is a limi a ion o 15 min o each es ). Finally, he sco e acco ding o
he Amazon Robo ic Challenge ules is shown in he igh mos column.
A o al o 62 i ems we e success ully picked and placed in he eigh es s, ou o he
o e all 80 a ge i ems o be iden i ied among he 256 i ems in he bins (8
×
32)— he esul
was a 77.5% success a io. Some o he a ge i ems we e he same in a ious es s, bu unde
di e en condi ions (pose, isibili y). This a io is highe han 69% o objec s in isola ion,
bu i has o be no ed ha now a emp s o li an i em ha ailed we e epea ed up o h ee
imes as soon as he senso s de ec ed he e o ; i he second o hi d y we e success ul
i is coun ed as a success acco ding o he compe i ion ules. No signi ican di e ences
a e o be ound in he pe cen ages o known and unknown i ems; his is cong uen wi h
he ac ha he g asp planning app oach does no ely on p e ious knowledge abou an
objec , beyond he decision o which g ippe and algo i hm o use. Fo he known i ems,
his decision was aken by looking a he op imiza ion Table 3abo e. Fo unknown i ems
his in o ma ion was inpu manually in he a ailable minu es, and based on hei simila i y
wi h known i ems.
Table 4. Summa y da a o conduc ed es s.
Tes # Success ul I ems Known I ems (%) Unknown I em (%) Time (min:s) Amazon Sco e
1 9/10 100 83 8:56 135
2 7/10 66 75 14:29 80
3 8/10 66 100 8:40 105
4 8/10 75 83 8:27 95
5 8/10 83 75 11:18 90
6 7/10 50 100 11:11 110
7 9/10 100 75 10:15 100
8 6/10 60 60 14:41 75
A e ages 7.75/10 75 81 11:00 98
7. Conclusions
Al hough he p oblem o g asp planning has been a ound in he obo ics communi y
o a long ime [
1
], he exis ing solu ions end o be limi ed when i comes o deploying
hem in ealis ic scena ios. Ou main con ibu ion he e has been o y and apply p e ious
knowledge oge he wi h ou own me hods in a ealis ic scena io as p oposed by Amazon
o i s wa ehouses. The ha dwa e employed is e y cos e ec i e: om he Bax e obo
i sel o he in-house made suc ion-cup end-e ec o and he 3D-p in ed Pinche g ippe .
This adds ex a limi a ions on he p ecision and epea abili y o he sys em.
The UJI RobinLab eam ook pa in he ARC’17 in o de o au oma e his wa ehouse
en i onmen . One o he necessa y accomplishmen s o he obo is o be able o g asp an
objec , which equi es a so wa e app oach ha ma ches he obo ic ha dwa e. Based on
ou expe ience, we p esen he e a p ac ical solu ion o he p oblem o obo picking in his
uns uc u ed, ealis ic scena io by using an app oach based on wo algo i hms ha akes
a poin cloud as an inpu and ou pu s a posi ion and o ien a ion o he g ippe gi en he
ha dwa e speci ica ions and es ic ions.
Two s a e-o - he-a algo i hms o wo- inge g ippe s we e analyzed since hey bo h
use a poin cloud as an inpu , and a g asp posi ion and o ien a ion as ou pu . We ha e
implemen ed and sys ema ically s udied hem in e ms o compu a ion ime, obus ness
and success a e. I was concluded ha AGILE was less obus and ook longe han
HAF. The la e ga e ewe g asp op ions wi h only e ical g asps, bu showed be e

Appl. Sci. 2021,11, 5805 22 o 24
obus ness and compu a ional speed. Gi en he esul s, i was decided ha AGILE did no
add signi ican con ibu ion o he ask and was disca ded o he sys em.
Ano he con ibu ion o he wo k is he c ea ion and es ing o he cen oid no mals
app oach (CNA) algo i hm. I uses he poin cloud and he majo g aspable componen o
he objec in o de o ind he cen oid and i s no mals in he la es pa o he poin cloud
o he objec . Then, g asps a e o a ed a ound he e ical z-axis in such a way ha he inal
g asp is mos com o able o he Bax e obo . This app oach was he mos success ul in
combina ion wi h a wo- inge g ippe and a acuum g ippe .
The inal con ibu ion was o ma ch he compe i ion objec s o he g asping echniques
and g ippe s o maximize he numbe o g asped i ems, as shown in Table 3. This able
exposes he analysis ha one uni e sal algo i hm o one uni e sal g ippe has no been
ound and cu en ly he bes solu ion is o use a ious algo i hms o a ious g ippe s. This
is suppo ed by he ac ha e en hough ou o e all success a es o 69% and 77.5% o
ou wo se s o es s, espec i ely, may appea o no be high enough, ou de ailed ailu e
analysis p o ides impo an clues abou he easons why he sys em ails, which e y o en
a e un ela ed o he g asping algo i hms hemsel es (objec ex u e, imp ecision o he
con ol and/o poin cloud, kinema ic es ic ions, e c.). We belie e ha ou esul s will
p o ide a use ul addi ion o he li e a u e in he ield owa ds he deploymen o p ac ical
wo king sys ems.
Au ho Con ibu ions:
Concep ualiza ion, A.M.; Me hodology, N.N.; P ojec adminis a ion, A.P.d.P.;
Supe ision, A.M. and A.P.d.P.; Valida ion, E.C.; W i ing—o iginal d a , N.N.; W i ing— e iew and
edi ing, A.M., E.C. and A.P.d.P. All au ho s ha e ead and ag eed o he published e sion o he
manusc ip .
Funding:
This pape desc ibes esea ch conduc ed a he UJI Robo ic In elligence Labo a o y. Suppo
o his labo a o y is p o ided in pa by Minis e io de Economía y Compe i i idad (DPI2015-
69041-R, DPI2017-89910-R), by Uni e si a Jaume I (P1-1B2014-52) and by Gene ali a Valenciana
(PROMETEO/2020/034). The i s au ho was ecipien o an E asmus Mundus schola ship by he
Eu opean Commission o he EMARO+ Mas e P og am.
Ins i u ional Re iew Boa d S a emen : No applicable.
In o med Consen S a emen : No applicable.
Acknowledgmen s:
The au ho s would like o hank Monica A ias o he wo k on he sys em es s.
Con lic s o In e es : The au ho s decla e ha hey ha e no con lic o in e es .
Re e ences
1.
Bohg, J.; Mo ales, A.; As ou , T.; K agic, D. Da a-D i en G asp Syn hesis—A Su ey. IEEE T ans. Robo .
2013
,30, 289–309.
[C ossRe ]
2.
Nechypo enko, N.; Mo ales, A.; del Pobil, A.P. G asping S a egies o Picking I ems in an Online Shopping Wa ehouse. In
In elligen Au onomous Sys ems 15; S and, M., Dillmann, R., Menega i, E., Ghidoni, S., Eds.; Sp inge In e na ional Publishing:
Cham, Swi ze land, 2019; pp. 775–785.
3.
Song, D.; Ek, C.H.; Huebne , K.; K agic, D. Task-Based Robo G asp Planning Using P obabilis ic In e ence. IEEE T ans. Robo .
2015,31, 546–561. [C ossRe ]
4.
Mad y, M.; Song, D.; K agic, D. F om objec ca ego ies o g asp ans e using p obabilis ic easoning. In P oceedings o he 2012
IEEE In e na ional Con e ence on Robo ics and Au oma ion (ICRA), Sain Paul, MN, USA, 14–18 May 2012; pp. 1716–1723.
5.
Huebne , K.; K agic, D. Selec ion o obo p e-g asps using box-based shape app oxima ion. In P oceedings o he 2008 IEEE/RSJ
In e na ional Con e ence on In elligen Robo s and Sys ems, IROS, Nice, F ance, 22–26 Sep embe 2008; pp. 1765–1770.
6.
Dang, H.; Allen, P.K. Seman ic g asping: Planning ask-speci ic s able obo ic g asps. Au on. Robo .
2014
,37, 301–316. [C ossRe ]
7. Nikand o a, E.; Ky ki, V. Ca ego y-based ask speci ic g asping. Robo . Au on. Sys . 2015,70, 25–35. [C ossRe ]
8.
Mille , A.T.; Knoop, S.; Ch is ensen, H.I.; Allen, P.K. Au oma ic g asp planning using shape p imi i es. In P oceedings o he
Robo ics and Au oma ion, 2003. P oceedings. ICRA’03, IEEE In e na ional Con e ence, Taipei, Taiwan, 14–19 Sep embe 2003;
Volume 2, pp. 1824–1829.
9.
Saxena, A.; Wong, L.L.; Ng, A.Y. Lea ning G asp S a egies wi h Pa ial Shape In o ma ion. In P oceedings o he AAAI-08,
Chicago, IL, USA, 13–17 July 2008; Volume 3, pp. 1491–1494.
10.
Fischinge , D.; Vincze, M.; Jiang, Y. Lea ning g asps o unknown objec s in clu e ed scenes. In P oceedings o he 2013 IEEE
In e na ional Con e ence on Robo ics and Au oma ion (ICRA), Ka ls uhe, Ge many, 6–10 May 2013; pp. 609–616.
Appl. Sci. 2021,11, 5805 23 o 24
11.
Fischinge , D.; Weiss, A.; Vincze, M. Lea ning g asps wi h opog aphic ea u es. In . J. Robo . Res.
2015
,34, 1167–1194. [C ossRe ]
12.
Gual ie i, M.; en Pas, A.; Saenko, K.; Pla , R. High p ecision g asp pose de ec ion in dense clu e . In P oceedings o he
IEEE/RSJ In e na ional Con e ence on In elligen Robo s and Sys ems (IROS), Daejeon, Ko ea, 9–14 Oc obe 2016; pp. 598–605.
13.
Jiang, Y.; Moseson, S.; Saxena, A. E icien g asping om gbd images: Lea ning using a new ec angle ep esen a ion. In
P oceedings o he 2011 IEEE In e na ional Con e ence on Robo ics and Au oma ion (ICRA), Shanghai, China, 9–13 May 2011;
pp. 3304–3311.
14.
Yoneyama, R.; Du an, A.J.; del Pobil, A.P. In eg a ing Senso Models in Deep Lea ning Boos s Pe o mance: Applica ion o
Monocula Dep h Es ima ion in Wa ehouse Au oma ion. Senso s 2021,21, 1437. [C ossRe ] [PubMed]
15.
Hasegawa, S.; Wada, K.; Nii ani, Y.; Okada, K.; Inaba, M. A h ee- inge ed hand wi h a suc ion g ipping sys em o picking
a ious objec s in clu e ed na ow space. In P oceedings o he 2017 IEEE/RSJ In e na ional Con e ence on In elligen Robo s
and Sys ems (IROS), Vancou e , BC, Canada, 24–28 Sep embe 2017; pp. 1164–1171. [C ossRe ]
16.
Tobin, J.; Biewald, L.; Duan, R.; And ychowicz, M.; Handa, A.; Kuma , V.; McG ew, B.; Ray, A.; Schneide , J.; Welinde , P.;
e al.
Domain Randomiza ion and Gene a i e Models o Robo ic G asping. In P oceedings o he 2018 IEEE/RSJ In e na ional
Con e ence on In elligen Robo s and Sys ems (IROS), Mad id, Spain, 1–5 Oc obe 2018; pp. 3482–3489.
17.
Mahle , J.; Ma l, M.; Liu, X.; Li, A.; Gealy, D.; Goldbe g, K. Dex-Ne 3.0: Compu ing Robus Vacuum Suc ion G asp Ta ge s in
Poin Clouds Using a New Analy ic Model and Deep Lea ning. In P oceedings o he 2018 IEEE In e na ional Con e ence on
Robo ics and Au oma ion (ICRA), B isbane, QLD, Aus alia, 21–25 May 2018; pp. 5620–5627.
18.
Zeng, A.; Song, S.; Yu, K.T.; Donlon, E.; Hogan, F.R.; Bauza, M.; Ma, D.; Taylo , O.; Liu, M.; Romo, E.; e al. Robo ic Pick-and-Place
o No el Objec s in Clu e wi h Mul i-A o dance G asping and C oss-Domain Image Ma ching. In P oceedings o he 2018
IEEE In e na ional Con e ence on Robo ics and Au oma ion (ICRA), B isbane, QLD, Aus alia, 21–25 May 2018; pp. 3750–3757.
19.
De Coninck, E.; Ve belen, T.; Van Molle, P.; Simoens, P.; Dhoed , B. Lea ning obo s o g asp by demons a ion. Robo . Au on. Sys .
2020,127, 103474. [C ossRe ]
20.
Schwa z, M.; Lenz, C.; Ga cía, G.M.; Koo, S.; Pe iyasamy, A.S.; Sch eibe , M.; Behnke, S. Fas Objec Lea ning and Dual-a m
Coo dina ion o Clu e ed S owing, Picking, and Packing. In P oceedings o he 2018 IEEE In e na ional Con e ence on Robo ics
and Au oma ion (ICRA), B isbane, QLD, Aus alia, 21–25 May 2018; pp. 3347–3354.
21.
Kalashniko , D.; I pan, A.; Pas o , P.; Iba z, J.; He zog, A.; Jang, E.; Quillen, D.; Holly, E.; Kalak ishnan, M.; Vanhoucke, V.; e al.
Scalable Deep Rein o cemen Lea ning o Vision-Based Robo ic Manipula ion. P oc. Mach. Lea n. Res. 2018,87, 651–673.
22.
Be scheid, L.; Meißne , P.; K öge , T. Sel -Supe ised Lea ning o P ecise Pick-and-Place Wi hou Objec Model. IEEE Robo .
Au om. Le . 2020,5, 4828–4835. [C ossRe ]
23.
Mallick, A.; del Pobil, A.P.; Ce e a, E. Deep Lea ning Based Objec Recogni ion o Robo Picking Task. In P oceedings o he 12 h
In e na ional Con e ence on Ubiqui ous In o ma ion Managemen and Communica ion, IMCOM ’18; ACM: New Yo k, NY, USA, 2018;
pp. 30:1–30:9.
24.
del Pobil, A.P.; Kassawa , M.; Du an, A.J.; A ias, M.A.; Nechypo enko, N.; Mallick, A.; Ce e a, E.; Subedi, D.; Vasile ,
I.; Ca din, D.; e al. UJI RobInLab’s app oach o he Amazon Robo ics Challenge 2017. In P oceedings o he 2017 IEEE
In e na ional Con e ence on Mul isenso Fusion and In eg a ion o In elligen Sys ems (MFI), Daegu, Ko ea, 16–18 No embe
2017; pp. 318–323.
25.
D’A ella, S.; T ipicchio, P.; A izzano, C.A. A s udy on picking objec s in clu e ed en i onmen s: Exploi ing dep h ea u es o a
cus om low-cos uni e sal jamming g ippe . Robo . Compu . In eg . Manu . 2020,63, 101888. [C ossRe ]
26. Wu man, P.R.; Romano, J.M. The amazonpicking challenge 2015. IEEE Robo . Au om. Mag. 2015,22, 10–12. [C ossRe ]
27.
Yu, K.T.; Fazeli, N.; Cha an-Da le, N.; Taylo , O.; Donlon, E.; Lankenau, G.D.; Rod iguez, A. A Summa y o Team MIT’s App oach
o he Amazon Picking Challenge 2015. a Xi 2016, a Xi :1604.03639.
28.
He nandez, C.; Bha a heesha, M.; Ko, W.; Gaise , H.; Tan, J.; an Deu zen, K.; de V ies, M.; Van Mil, B.; an Egmond, J.; Bu ge , R.;
e al. Team del ’s obo winne o he amazon picking challenge 2016. In RoboCup 2016: Robo Wo ld Cup XX; Sp inge : Cham,
Swi ze land, 2016; pp. 613–624.
29.
Mo ison, D.; Tow, A.W.; McTagga , M.; Smi h, R.; Kelly-Boxall, N.; Wade-McCue, S.; E skine, J.; G ino e , R.; Gu man, A.; Hunn,
T.; e al. Ca man: The low-cos ca esian manipula o ha won he amazon obo ics challenge. a Xi 2017, a Xi :1709.06283.
30.
Co ell, N.; Bek is, K.E.; Be enson, D.; B ock, O.; Causo, A.; Hause , K.; Okada, K.; Rod iguez, A.; Romano, J.M.; Wu man, P.R.
Analysis and Obse a ions om he Fi s Amazon Picking Challenge. IEEE T ans. Au om. Sci. Eng.
2018
,15, 172–188. [C ossRe ]
31.
Eppne , C.; Hö e , S.; Jonschkowski, R.; Ma ín-Ma ín, R.; Sie e ling, A.; Wall, V.; B ock, O. Fou Aspec s o Building Robo ic
Sys ems: Lessons om he Amazon Picking Challenge 2015. Au on. Robo . 2018, 1–17. [C ossRe ]
32. P a s, M.; Ma inez, E.; Sanz, P.; del Pobil, A. The UJI Lib a ian Robo . J. In ell. Se . Robo . 2008,1, 321–335. [C ossRe ]
33.
Fischinge , D.; Vincze, M. Emp y he baske -a shape based lea ning app oach o g asping piles o unknown objec s. In
P oceedings o he In elligen Robo s and Sys ems (IROS), IEEE/RSJ In e na ional Con e ence, Vilamou a-Alga e, Po ugal,
7–12 Oc obe 2012; pp. 2051–2057.
34.
Nguyen, V.D. Cons uc ing o ce-closu e g asps. In P oceedings o he 1986 IEEE In e na ional Con e ence on Robo ics and
Au oma ion, San F ancisco, CA, USA, 7–10 Ap il 1986; Volume 3, pp. 1368–1373.
35.
Dalal, N.; T iggs, B. His og ams o o ien ed g adien s o human de ec ion. In P oceedings o he 2005 IEEE Compu e Socie y
Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR’05), San Diego, CA, USA, 20–25 June 2005; Volume 1, pp. 886–893.
Appl. Sci. 2021,11, 5805 24 o 24
36.
en Pas, A.; Pla , R. Using Geome y o De ec G asps in 3d Poin Cloud. In P oceedings o he In e na ional Symposium on
Robo ics Resea ch (ISRR), Geno a, I aly, 12–15 Sep embe 2015.
37.
Choi, S.; Kim, T.; Yu, W. Pe o mance E alua ion o RANSAC Family. In P oceedings o he B i ish Machine Vision Con e ence
2009, London, UK, 7–10 Sep embe 2009.
38.
Chum, O.; Ma as, J. Ma ching wi h PROSAC-P og essi e Sample Consensus. In P oceedings o he 2005 IEEE Compu e
Socie y Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR’05)-Volume 1-Volume 01; CVPR ’05; IEEE Compu e Socie y:
Washing on, DC, USA, 2005; pp. 220–226.