

Ci a ion: Heczko, D.; Ošˇcádal, P.; Ko ,
T.; Bolesla ský, A.; K ys, V.; Bém, J.;
Vi gala, I.; Bobo ský, Z. Finding he
Op imal Pose o 2D LLT Senso s o
Imp o e Objec Pose Es ima ion.
Senso s 2022,22, 1536. h ps://
doi.o g/10.3390/s22041536
Academic Edi o : Cosimo Dis an e
Recei ed: 7 Janua y 2022
Accep ed: 15 Feb ua y 2022
Published: 16 Feb ua y 2022
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2022 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
senso s
A icle
Finding he Op imal Pose o 2D LLT Senso s o Imp o e Objec
Pose Es ima ion
Dominik Heczko 1,* , Pe Ošˇcádal 1, Tomáš Ko 1, Adam Bolesla ský1, Václa K ys 1, Jan Bém1,
I an Vi gala 2and Zdenko Bobo ský1,*
1Depa men o Robo ics, Facul y o Mechanical Enginee ing, VSB—Technical Uni e si y o Os a a,
70800 Os a a, Czech Republic; pe [email p o ec ed] (P.O.); [email p o ec ed] (T.K.);
[email p o ec ed] (A.B.); acla [email p o ec ed] (V.K.); [email p o ec ed] (J.B.)
2Depa men o Indus ial Au oma ion and Mecha onics, Facul y o Mechanical Enginee ing,
Technical Uni e si y o Kosice, 04200 Kosice, Slo akia; [email p o ec ed]
*Co espondence: [email p o ec ed] (D.H.); [email p o ec ed] (Z.B.)
Abs ac :
In his pape , we examine a me hod o imp o ing pose es ima ion by co ec ly posi ioning
he senso s ela i e o he scanned objec . Th ee objec s made o di e en ma e ials and using di e en
manu ac u ing echnologies we e selec ed o he expe imen . To collec inpu da a o o ien a ion
es ima ion, a simula ion en i onmen was c ea ed whe e each objec was scanned a di e en poses.
A simula ion model o he lase line iangula ion senso was c ea ed o scanning, and he op ical
su ace p ope ies o he scanned objec s we e se o simula e eal scanning condi ions. The simula ion
was e i ied on a eal sys em using he UR10e obo o o a e and mo e he objec . The p esen ed
esul s show ha he simula ion ma ches he eal measu emen s and ha he app op ia e placemen
o he senso s has imp o ed he o ien a ion es ima ion.
Keywo ds: o ien a ion es ima ion; lase scanning; LLT senso ; i ual scanning; op imal con igu a ion;
op imal pose; ICP algo i hm; pose es ima ion
1. In oduc ion
In indus ial manu ac u ing, accu acy and p ecision equi emen s in assembly and
manipula ion o objec s keep inc easing. Fo a p ecise assembly pe o med by an indus ial
obo , i is impo an o de ine he objec ’s picking poin and o ien a ion so ha he obo
can accu a ely g asp he objec and pe o m he assembly. T adi ionally, his is done
using p ocess palle s, jigs, and o he equipmen . Howe e , hese jigs end o be expensi e
and mus be unique o each objec . The e a e sys ems ha pe o m assembly based on
au oma ic guidance o la ge objec s [
1
], whe e he obo end-e ec o mus ha e a senso y
subsys em o ensu e su icien p ecision. In [
2
], he au ho s deal wi h assembly o small
componen s, whe e hey use one came a in he end-e ec o o guide he obo o g asp
objec s, wo came as in he alignmen phase o he objec be o e assembly, ollowed by
eedback om he o ce senso du ing assembly; howe e , he objec was co ec ly o ien ed
o obo g asping. In he s udy [
3
], he au ho s achie e signi ican assembly accu acy
wi h an indus ial obo o hund ed hs o a deg ee by on-line mo emen compensa ion.
Howe e , hey used a e y expensi e lase acke senso and we e no dealing wi h
g asping o objec s.
P ecise assembly is especially di icul in applica ions whe e objec s in a box o bin
a e o ien ed chao ically. This bin-picking p oblem is sol ed in di e en ways, such as
inding he posi ion o g asp an objec using a 3D came a [
4
] o objec de ec ion and pose
es ima ion using a dep h map [
5
]. An impo an pa o bin-picking is image p ocessing
and ecogni ion; nowadays, he use o neu al ne wo ks has shown g ea p omise in his
ask [
6
,
7
]. Al hough he pose es ima ion o he neu al ne wo k is becoming mo e accu a e,
he e is s ill an e o in he es ima ion o o ien a ion [
8
]. These solu ions a e su icien o
Senso s 2022,22, 1536. h ps://doi.o g/10.3390/s22041536 h ps://www.mdpi.com/jou nal/senso s
Senso s 2022,22, 1536 2 o 18
pick-and-place applica ions [
9
], such as unloading pa s om a box on o a con eyo o o he
de ice. Howe e , in o de o achie e an exac g asping posi ion o u u e manipula ion, he
pa mus be aligned wi h a ious jigs and ix u es. The addi ional alignmen o pa s o
g asping by he obo is ime-consuming and i would be mo e bene icial o elimina e his
in e media e s ep. This can be achie ed by pe o ming he assembly p ocess di ec ly a e
g asping he objec ia bin-picking, hus elimina ing he need o addi ional equipmen o
align objec s and sa ing space by excluding he con eyo ha is o en used o anspo
objec s o he nex s a ion.
Fo p ecise manipula ion o objec s g asped by bin-picking, i is necessa y o de e mine
hei posi ion and o ien a ion in he obo ’s g ippe . This can be achie ed by a ious
me hods, such as pose es ima ion om a 2D image (mainly RGB image) using neu al
ne wo ks [
10
,
11
]. RANdom SAmple Consensus (RANSAC) me hod is o en used o i 3D
shape p imi i es and oughly es ima es he pose o a ca ied objec analyzing a 3D image
o a dep h came a o lase senso da a—poin cloud [
12
]. To e ine pose es ima ion, he
I e a i e Closes Poin (ICP) algo i hm [
13
] is used, which is e y s able and obus and has
been g ea ly imp o ed in ecen yea s [14].
Da a collec ion can be p o ided by a ious senso s, gene ally di ided in o wo g oups:
con ac and non-con ac . Fo con ac senso s, ac ile senso s a e used which a e placed on
he inge s o he end-e ec o [
15
]. E en ually, he obo ha holds he g asped objec is
d i en in o a me ology sys em [
16
] whe e he measu emen will be pe o med. Tac ile
senso s a e e y accu a e and insensi i e o he op ical p ope ies o he scanned su aces.
Howe e , ob aining a 3D shape o a complex pa using a ac ile senso is ime-consuming
and con ac wi h he measu ed pa could lead o su ace deg ada ion. The e o e, con ac -
less senso s a e he p e e ed choice in oday’s indus y. Time-o - ligh [
17
], iangula ion
(1D, 2D, and 3D) [
18
], o classic RGB came as image analysis [
19
] a e he mos commonly
used me hods. When using non-con ac senso s, he measu ed objec canno be damaged in
any way and he measu emen is as e ; senso s can sense se e al housand poin s wi hin
milliseconds [
20
]. Howe e , con ac less senso s a e sensi i e o op ical p ope ies o he
scanned objec s, especially shiny, e y smoo h, and anspa en su aces a e p oblema ic,
and scanning da a may be dis o ed o e en absen . Fo example, in lase line iangula ion
(LLT) scanning, he lase beam may e lec o shiny su aces ou side he senso and he lase
line is no de ec ed. Senso ics o de ec ing an objec in he g ippe is ei he placed di ec ly
in he end-e ec o [21] o he objec is scanned by a senso placed in he wo ks a ion [22].
In his wo k, we ocused on he app op ia e placemen o LLT senso s wi h espec
o a scanned objec o p o ide ele an inpu da a o pose es ima ion using RANSAC
and ICP algo i hms and hus imp o e he accu acy o o ien a ion es ima ion, because he
co ec inpu da a a e essen ial o pose es ima ion me hods. Finding he op imal senso
pose would be e y ime-consuming in a eal en i onmen , so a simula ion en i onmen
was c ea ed o speed up he p ocess. The op ical p ope ies o scanned objec s we e
implemen ed in he simula ion en i onmen o simula e he eal condi ions du ing scanning.
Th ee objec s we e selec ed o he expe imen , each made wi h a di e en echnology and
ma e ial. The simula ion esul s we e e i ied on a eal sys em. Besides, he simula ion
en i onmen could be used o es he LLT senso and i s beha io on di e en ma e ials
du ing scanning be o e pu chasing he ac ual senso .
2. Me hodology
As men ioned ea lie , he da a acqui ed by LLT senso s could be a ec ed by he geo-
me ical and op ical p ope ies o he sensed su aces. Di use ligh e lec ion is impo an
o LLT senso scanning, which is he sca e ing o e lec ed ligh in o he su oundings
and hus in o he cha ge-coupled de ice (CCD), which de ec s he lase line on he scanned
su ace. In p e ious esea ch, we ha e in es iga ed he e ec o he angle o incidence (AoI)
o he lase beam on he in ensi y o e lec ed ligh o ensu e su icien e lec ed ligh o he
CCD senso [
23
]. The esea ch was pe o med o ma e ials commonly used in he indus y
(plas ics, me als, and o he s). These da a we e implemen ed in a simula ion o simula e
Senso s 2022,22, 1536 3 o 18
eal condi ions du ing scanning wi h an LLT senso (speci ically he Keyence LJ-X8080
senso [24]) in a i ual en i onmen .
The simula ion model was c ea ed in he CoppeliaSim en i onmen [
25
]. The sen-
so simula ion model e u ns he same da a o ma as he ac ual senso (3200 alues
o he heigh —Z-coo dina e o each poin ); i a poin was no de ec ed, i s alue is
−9999.9999 (emp y
poin ). The X-coo dina e mus be calcula ed acco ding o he se esolu-
ion o he CCD came a. The simula ion uses he a ailable elemen s modi ied by cus om
sc ip s. Figu e 1desc ibes he simula ion model o he LLT senso , whe e he ision senso
elemen was used o simula e he CCD came a; he p oximi y lase senso (PLS) elemen
was used o simula e he lase sou ce.
Figu e 1.
Simula ion model o he LLT senso LJ-X8080. Middle ange is deno ed as R and ep esen s
he middle ange o he senso — he coo dina es a his poin a e X, Z (0, 0).
The p oximi y lase senso is con olled using he sc ip desc ibed in Algo i hm 1. The
PLS is posi ioned using a e olu e join (as can be seen in Figu e 1). I he lase ay (beam)
hi s he scanned objec a a gi en posi ion, i will de ec i s ID and su ace p ope ies (i he e
we e mul iple objec s in he simula ion, each may ha e di e en p ope ies). The angle
o incidence o he lase ay (beam) ela i e o he su ace no mal is calcula ed allowing
o e alua e he in ensi y o he e lec ed lase ay. The ob ained da a a e sen o he CCD
came a sc ip o u he p ocessing.
Algo i hm 1. E alua ion o he e lec ed lase ay (beam) in ensi y.
o i = 0 o m // m[1:3200] ep esen s he posi ion o PLS
se _lase _posi ion (i)
i sensing
ge _ma e ial_pa am (ID) // The ID ep esen s he ID o he objec on which he
lase beam is inciden
compu e_angle_o _incidence (i)
compu e_in ensi y (AoI)
end i
end o
e u n in ensi y // in ensi y ep esen s an a ay o 3200 in ensi y alues
The CCD came a is used o de ec he lase line on he su ace o he scanned objec .
The con ol sc ip o he CCD came a is desc ibed by Algo i hm 2. Fi s , he in ensi y da a
om he PLS sc ip a e eques ed. Then he image is p ocessed: he pixel columns a e
sequen ially checked and he lase colo is sea ched. I he colo has been de ec ed and
he in ensi y o a gi en pixel is g ea e han he minimal de ec able in ensi y, he heigh
o he poin (Z-coo dina e) is w i en in o he a ay. I he colou has no been de ec ed o
Senso s 2022,22, 1536 4 o 18
he in ensi y is below he h eshold, he alue
−
9999.9999 (blank poin ) is w i en in o he
a ay o coo dina e alues.
Algo i hm 2. Lase line de ec ion by CCD ision senso .
ge _in ensi y () // in ensi y ep esen a ay o in ensi y o each poin
o i = 0 o m // m[1:3200] ep esen s a column in he came a image
o j = 0 o n // n[1:2230] ep esen s a ow in he came a image
i RGB is de ec ed
i in ensi y > min_in ensi y_ alue
ge _coo dian e_o _poin (i)
end i
end i
end o
end o
e u n coo dina es // coo dina es ep esen s an a ay o 3200
The ICP algo i hm a ailable om he Open3D [
26
] and PCL [
27
] lib a ies was used
o calcula e he o ien a ion o he objec . Bo h lib a ies a e open sou ce and ha e been
in con inuous de elopmen . The inpu da a gi en o he ICP algo i hm a e wo poin
clouds— loa ing and ixed. The loa ing poin cloud is i ed ( egis e ed) o he ixed poin
cloud, which is a e e ence poin cloud. In ou case, poin clouds o h ee di e en ly placed
senso s ela i e o he scanned objec and hei con igu a ions a e used as he inpu ( loa ing
poin clouds) o he algo i hm.
To accu a ely egis e poin clouds, i is essen ial o cap u e e e ence ea u es (geo-
me ic p imi i es) in he loa ing poin cloud such as demanded su aces, edges, cu es,
cylinde s, e c. An example o senso placemen is shown in Figu e 2a, whe e he anges o
h ee di e en ly placed LLT senso s a e ma ked. Senso 1 (S1) senses he le su ace, pa
o he uppe su ace, he inne su ace, and he oundness; Senso 2 (S2) senses he uppe
su ace and he holes; Senso 3 (S3) senses he igh su ace, pa o he uppe su ace, and
he oundness. Figu e 2b shows he posi ioned e e ence coo dina e sys ems o he LLT
senso s ela i e o he base coo dina e sys em o he scanned objec (O
BASE
). Leng h o he
componen is deno ed as l and in his case, i is he same as he scanning leng h. De ails o
he expe imen and i s e i ica ion a e desc ibed in Sec ion 3.
Figu e 2.
Me hodology o LLT senso placemen (S1 is senso no. 1, S2 is senso no. 2, S3 is senso
no. 3): (
a
) he measu emen anges o senso s S1, S2, and S3; (
b
) coo dina e sys ems S1, S2, and S3
ep esen he e e ence coo dina e sys ems o LLT senso s, O
BASE
is he objec coo dina e sys em; l is
he scanning leng h.
3. Selec ion o Senso Con igu a ions and Thei Ve i ica ion
Th ee di e en objec s we e chosen o he expe imen , which can be seen in
Figu e 3a,b
.
Each objec had a di e en shape and ma e ial and was made wi h a di e en echnology.
Figu e 3a shows Pa A, which was made o aluminum alloy by milling, he su ace
Senso s 2022,22, 1536 5 o 18
oughness is Ra1.6; he dimensions o his pa a e 54
×
45
×
32 mm (leng h
×
wid h
×
heigh ). Figu e 3b shows Objec B, which was made o plas ic using addi i e manu ac u ing;
he dimensions o his componen a e 60
×
41
×
17.6 mm. Fu he mo e, Figu e 3c shows
Pa C, which was made by bending a s eel cu ou ; he dimensions o his componen a e
42.5
×
55
×
32.5 mm. The ma e ial p ope ies o Pa C we e no measu ed; howe e ,
he su ace o his objec is e y smoo h. The e o e, he ma e ial o he simula ion was
es ima ed as an aluminum alloy wi h he su ace oughness o Ra0.8. Fo Objec s A and
B, he scanning leng h was iden ical o he leng h o he objec s, while o Objec C he
scanning leng h was 12 mm, which is he wid h o he ben elemen s.
Figu e 3.
Objec s o he expe imen and hei CAD models. All objec s a e connec ed o a lange ha
a aches o he UR10e o eal measu emen s: (
a
) Pa A and i s CAD model (aluminum, oughness
Ra1.6); (b) Pa B and i s CAD model (black PLA); (c) Pa C and i s CAD model (s eel).
In he simula ion en i onmen , he e lec ion p ope ies ( he dependence o he AoI o
he lase beam on he e lec ed in ensi y) we e se acco ding o [
23
] o each scanned objec .
Senso s we e placed ela i e o he objec coo dina e sys em manually based on he use ’s
decision so ha hey cap u ed as many e e ence ea u es as possible du ing he scanning
p ocess (simila o Figu e 2b). Fo each pa , h ee senso poses we e p ede ined, om which
he op imal ones we e ound (wi h he lowes pose es ima ion e o ). Howe e , mul iple
senso poses could be p ede ined o each objec as well. Each componen had a di e en
shape, so he senso s we e placed indi idually o each objec . An example o he placemen
o senso s a ound Pa B shows Equa ions (1)–(3), which ep esen a igid ans o ma ion
be ween he coo dina e sys em o he objec and he senso s coo dina e sys em.
MOBase→S1=




0 1 0 0.03
0.7071 0 0.7071 −0.023
0.7071 0 −0.7071 0.015
0 0 0 1




(1)
MOBase→S2=




0 1 0 0.03
1 0 0 0
0 0 −1 0.175
0 0 0 1




(2)

Senso s 2022,22, 1536 6 o 18
MOBase→S3=




0 1 0 0.03
0.7071 0 −0.7071 0.014
−0.7071 0 −0.7071 0.015
0 0 0 1




(3)
Then, a se ies o measu emen s was aken o each objec as desc ibed in
Algo i hm 3
.
Fi s , he o ien a ion o he objec was se and hen he scanning was pe o med. The
o ien a ion was se o all possible o a ion combina ions using RPY angles (Roll, Pi ch,
and Yaw):
•Ro a ion a ound a sepa a e axis (Roll only, Pi ch only, Yaw only);
•
Ro a ion a ound wo axes (Roll and Pi ch only, Pi ch and Yaw only, Yaw and Roll
only);
•Ro a ion a ound all axes a once (Roll, Pi ch, and Yaw).
Algo i hm 3. Da a collec ion algo i hm.
se _objec _o ien a ion(Roll, Pi ch, Yaw)
o i = 0 o m // m ep esen s scanning s eps (acco ding o scanning leng h)
se _objec _posi ion(i)
ge _senso _da a()
ans o m_da a()
end o
sa e_da a()
The Roll, Pi ch, and Yaw angles we e se om
−
2
◦
o +2
◦
in inc emen s o 0.4
◦
. This
bounda y was se based on he accu acy o he pose es ima ion o he bin-picking sys ems.
The scanning leng h was se acco ding o he leng h o he pa o he wid h o he scanned
ea u e, and he scanning s ep was se o 0.5 mm. Du ing scanning, he objec always
mo ed o wa d in he x-axis di ec ion o he O coo dina e sys em ( ixed coo dina e sys em
o he scanned objec ), as shown in Figu e 4in de ail.
Figu e 4.
Scanning o Objec B. The igu e de ail shows he coo dina e sys ems O ( he objec is no
o ien ed) and O’ ( he objec is o ien ed a a yaw angle o
−
2
◦
). The scanning di ec ion is always in
he x-axis di ec ion o he O coo dina e sys em.
To c ea e a 3D poin cloud, he sequen ially collec ed senso da a wi h espec o he
senso e e ence coo dina e sys ems (ma ked as S1 e in Figu e 4 o senso 1) mus be
exp essed wi h espec o he objec coo dina e sys em (ma ked as O in Figu e 4), which is
achie ed by he ans o ma ion in (4):
PiO =T−1
Sj e →O·PiSj e (4)
Senso s 2022,22, 1536 7 o 18
whe e
PiSj e
is he i- h poin exp essed in he j- h senso coo dina e sys em,
T−1
Sj e →O
is
he ans o ma ion ma ix om he j- h senso coo dina e sys em o he objec coo dina e
sys em, and PiO is he i- h poin exp essed in he objec coo dina e sys em.
3.1. Selec ion o Senso Con igu a ion in Vi ual En i onmen
F om he da ase ob ained om he simula ions, he o ien a ion was calcula ed as
desc ibed in Algo i hm 4. The poin clouds o each senso we e en e ed in o he calcula ion
acco ding o he con igu a ion. A e p e-p ocessing, he poin clouds we e egis e ed.
F om he ob ained ans o ma ion ma ix, he RPY angles (Roll, Pi ch, and Yaw) and he
de ia ion om he ac ual (se ) o ien a ion we e calcula ed.
Algo i hm 4. Poin cloud egis a ion.
o i = 0 o m // m [1:7] ep esen s he senso con igu a ion numbe
o j = 0 o n // n [1:11] ep esen s he poin clouds o each o a ion ( om −2◦ o +2◦)
me ge_poin _clouds() // acco ding o senso con igu a ion
p ocess_poin _cloud() // downsample, emo e ou lie s
poin _cloud_ egis a ion()
ge _ py_angles()
compu e_e o ()
sa e_da a()
end o
end o
Based on he esul s, he bes senso con igu a ions we e selec ed— he senso s ha
had he smalles de ia ion in he calcula ion. The median o o al e o and he e o ange
(75 h pe cen ile) we e used o selec he con igu a ions. The o ien a ion de ia ions a e
shown in he ollowing igu es using a boxplo , whe e he ho izon al line in he boxplo
ep esen s he median e o and he whi e do ep esen s he mean e o .
Figu e 5a–c show he de ia ions o he calcula ed o ien a ion o all senso con igu-
a ions o Objec A using he Open3D lib a y; Figu e 5d– show he de ia ions o he
calcula ed o ien a ion using he PCL lib a y. The o ien a ion es ima ion esul s we e simila ;
howe e , he Open3D lib a y pe o med be e . Senso S1 showed he bes esul s using
one senso . Senso s S1 and S3 showed he bes esul s using wo senso s, whe e Senso S1
da a was used o calcula e Roll and Yaw; Senso S3 da a we e inpu o calcula e Pi ch. The
bes esul s using he h ee senso s we e me ged da a o senso s S1, S2, and S3 o calcula e
Roll; o he calcula ion o Pi ch da a o Senso S3; and o he calcula ion o Yaw da a o
Senso S1.
Figu e 6a–c show he de ia ions o he calcula ed o ien a ion o all senso con ig-
u a ions o Objec B using he Open3D lib a y and Figu e 6d– using he PCL lib a y.
Acco ding o he o ien a ion es ima ion esul s, he Open3D lib a y achie ed mo e accu a e
esul s. Senso S1 showed he bes esul s using one senso . Senso s 1 and 2 showed he
bes esul s using wo senso s, whe e he me ged da a om Senso s S1 and S2 we e used o
calcula e Roll and Pi ch; Senso S1 was inpu o he calcula ion o Yaw. The bes esul s
using all h ee senso s we e he me ged da a o Senso s S1, S2, and S3 o he calcula ion
o Roll; he me ged da a o Senso s S2 and S3 o he calcula ion o Pi ch; and he da a o
Senso S1 o he calcula ion o Yaw.
Figu e 7a–c show he de ia ions o he calcula ed o ien a ion o all senso con igu a-
ions o Objec C using he Open3D lib a y and Figu e 7d– using he PCL lib a y. Again,
he Open3D lib a y p o ided mo e accu a e o ien a ion es ima ion esul s. The S2 senso
showed he bes esul s using a single senso . Senso s 2 and 3 showed he bes esul s using
wo senso s, whe e Senso S1 da a was inpu o Roll calcula ion; Senso S3 da a was inpu
o Pi ch and Yaw calcula ion. The esul s using all h ee senso s we e wo se han using
wo senso s.
Senso s 2022,22, 1536 8 o 18
Figu e 5.
Objec A o ien a ion de ec ion e o by simula ion using Open3D and PCL lib a y: (
a
) sen-
so s sepa a ely (Open3D); (
b
) wo senso s con igu a ion (Open3D); (
c
) h ee senso s con igu a ion
(Open3D); (
d
) senso s sepa a ely (PCL); (
e
) wo senso s con igu a ion (PCL); (
) h ee senso s con ig-
u a ion (PCL).
Figu e 6. Objec B o ien a ion de ec ion e o by simula ion using Open3D and PCL lib a y: (a) sen-
so s sepa a ely (Open3D); (
b
) wo senso s con igu a ion (Open3D); (
c
) h ee senso s con igu a ion
(Open3D); (
d
) senso s sepa a ely (PCL); (
e
) wo senso s con igu a ion (PCL); (
) h ee senso s con ig-
u a ion (PCL).
Senso s 2022,22, 1536 9 o 18
Figu e 7.
Objec C o ien a ion de ec ion e o by simula ion using Open3D and PCL lib a y: (
a
) sen-
so s sepa a ely (Open3D); (
b
) wo senso s con igu a ion (Open3D); (
c
) h ee senso s con igu a ion
(Open3D); (
d
) senso s sepa a ely (PCL); (
e
) wo senso s con igu a ion (PCL); (
) h ee senso s con ig-
u a ion (PCL).
Based on he simula ion esul s, he senso con igu a ions wi h he smalles de ia ion
o he calcula ed o ien a ion we e selec ed. Con igu a ions using one, wo, and h ee
senso s we e used. The chosen con igu a ions we e e i ied on a eal sys em. Fo Objec C,
only con igu a ions using one and wo senso s we e selec ed because he h ee-senso con-
igu a ion pe o med wo se han he wo-senso con igu a ion. The o ien a ion es ima ion
esul s o bo h lib a ies we e simila , bu in all cases he esul s using he PCL lib a y we e
wo se. The e o e, in he nex sec ion, only he Open3D lib a y was used o he calcula ion.
The con igu a ions we e selec ed acco ding o he esul s using he Open3D lib a y, bu he
same con igu a ions would ha e been selec ed using he PCL lib a y (wi h wo se esul s).
3.2. Measu emen on a Real Sys em
A wo ks a ion wi h he UR10e obo was used o e i y he simula ion esul s. The
expe imen al wo ks a ion can be seen in Figu e 8a. The UR10e obo was used o o a e he
objec s and o pe o m linea mo ion o he objec s o scanning. Du ing he measu emen
p ocess, he obo hea s up, which causes posi ioning inaccu acies, so we compensa ed he
obo posi ioning accu acy acco ding o [28].
Since we only had wo LLT senso s, when measu ing a h ee-senso con igu a ion,
he da a om wo senso s we e collec ed, hen he da a om he missing senso we e
measu ed sepa a ely. Fo each objec , a housing was c ea ed o hold he senso s in a de ined
posi ion, as shown in Figu e 8b–d. The housings we e made by an addi i e manu ac u ing
echnique using PLA ma e ial, which is commonly used and has e y good mechanical
p ope ies [29].
Senso s 2022,22, 1536 16 o 18
5. Conclusions
Tigh ening up ole ances in indus ial manu ac u ing and assembly is essen ial o
imp o e p oduc quali y and du abili y. One o he easons o igh ening is less s ess on
pa s du ing p ecision assembly. Howe e , p ecision assembly is mo e ime-consuming, as
pa s mus be p ecisely posi ioned a he pick poin s so a machine (mainly an indus ial
obo ) can accu a ely g asp hem. To speed up he p ocess, bin-picking sys ems a e used o
es ima e he pose o he objec in a bin. Howe e , he accu acy o pose es ima ion is abou
±
2
◦
. Objec s g asped in his way mus be aligned o he exac posi ion using o he de ices
o he posi ion o he g asped objec in he obo ’s end-e ec o mus be es ima ed.
In his pape , we ha e in es iga ed he me hodology o selec ing he con igu a ion
o LLT senso s ela i e o he scanned objec based on simula ion. When he senso s a e
co ec ly placed, ele an da a a e collec ed o he objec o ien a ion es ima ion algo i hm,
hus imp o ing he o ien a ion es ima ion.
Fo ou expe imen we selec ed h ee objec s. Each objec had a di e en shape and
ma e ial and was made wi h a di e en echnology. Objec A was made o aluminum by
milling, Objec B was made o PLA plas ic by addi i e echnology (3D p in ing), and objec
C was made by bending a s eel cu -ou .
Based on he i ual scanning, a da ase was p epa ed o es ima e he o ien a ion o
objec s in 3D space. Fo each objec , he senso con igu a ions wi h he smalles o ien a ion
es ima ion e o we e selec ed using one, wo, and h ee senso s. The simula ion esul s
we e e i ied using ac ual measu emen s. To place he LLT senso s in 3D space, housings
we e made using 3D p in ing and a UR10e obo was used o o a e and mo e he objec s.
A compa ison o he simula ed and ac ual measu emen s is shown in Table 1, which
shows he de ec ion e o (median) and he e o ange (75 h pe cen ile) o all objec s and
selec ed senso con igu a ions.
The p esen ed da a show ha he posi ion o he senso s ela i e o he scanned objec
has a signi ican e ec on he accu acy o he o ien a ion es ima ion. The esul s show ha
he numbe o senso s can be minimized by app op ia e placemen o senso s. Using inpu
da a om all h ee senso s may no always lead o imp o ed o ien a ion es ima ion, bu
inpu da a om one senso may show be e esul s. When he co ec op ical p ope ies
o he scanned objec a e se , he simula ion ma ches he ac ual measu emen . Thanks o
he i ual en i onmen , we can ind he op imal posi ion o he senso s and imp o e he
o ien a ion es ima ion sys em. This also allows wo king wi h LLT senso s in he simula ion,
whe e he use can es hei scanning beha io be o e acqui ing he eal senso s.
Now, i is necessa y o place senso s o each objec manually, based on he use ’s
decision. This opens up he possibili y o u u e wo k by c ea ing an algo i hm ha
au oma ically selec s he op imal senso placemen o a gi en objec o manipula ion.
Au ho Con ibu ions:
Concep ualiza ion, D.H. and Z.B.; me hodology, D.H., V.K. and Z.B.; so wa e,
D.H. and P.O.; alida ion, D.H., P.O. and T.K.; o mal analysis, A.B. and J.B.; in es iga ion, D.H.,
T.K., P.O., A.B. and J.B.; w i ing—o iginal d a p epa a ion, D.H., A.B. and T.K.; w i ing— e iew
and edi ing, P.O., T.K., V.K., I.V., A.B., J.B. and Z.B.; isualiza ion, D.H.; supe ision, V.K. and Z.B.;
p ojec adminis a ion, I.V. and Z.B. All au ho s ha e ead and ag eed o he published e sion o
he manusc ip .
Funding:
This wo k was suppo ed by he Resea ch Pla o m ocused on Indus y 4.0 and Robo ics
in Os a a Agglome a ion p ojec , p ojec numbe CZ.02.1.01/0.0/0.0/17_049/0008425 wi hin he
Ope a ional P og amme Resea ch, De elopmen and Educa ion. This a icle has been also suppo ed
by speci ic esea ch p ojec SP2022/67 and inanced by he s a e budge o he Czech Republic. This
esea ch was also unded by Slo ak KEGA 030TUKE-4/2020 “T ans e o knowledge om he ield
o indus ial au oma ion and obo ics o eaching in he ield o Mecha onics”.
Ins i u ional Re iew Boa d S a emen : No applicable.
In o med Consen S a emen : No applicable.
Da a A ailabili y S a emen : No applicable.

Senso s 2022,22, 1536 17 o 18
Con lic s o In e es :
The au ho s decla e no con lic o in e es . The unde s had no ole in he design
o he s udy; in he collec ion, analyses, o in e p e a ion o da a; in he w i ing o he manusc ip , o
in he decision o publish he esul s.
Abb e ia ions
The ollowing abb e ia ions a e used in his manusc ip :
RANSAC RANdom SAmple Consensus
ICP I e a i e Closes Poin
AoI Angle o Incidence
LLT Lase Line T iangula ion
CCD Cha ge Coupled De ice
Re e ences
1.
Qin, Z.; Wang, P.; Sun, J.; Lu, J.; Qiao, H. P ecise Robo ic Assembly o La ge-Scale Objec s Based on Au oma ic Guidance and
Alignmen . IEEE T ans. Ins um. Meas. 2016,65, 1398–1411. [C ossRe ]
2.
Ma, Y.; Du, K.; Zhou, D.; Zhang, J.; Liu, X.; Xu, D. Au oma ic p ecision obo assembly sys em wi h mic oscopic ision and o ce
senso . In . J. Ad . Robo . Sys . 2019,16, 172988141985161. [C ossRe ]
3.
Jiang, Y.; Huang, X.; Li, S. An on-line compensa ion me hod o a me ology-in eg a ed obo sys em o high-p ecision assembly.
Ind. Robo . In . J. Robo . Res. Appl. 2016,43, 647–656. [C ossRe ]
4.
Ma inez, C.; Boca, R.; Zhang, B.; Chen, H.; Nidama hi, S. Au oma ed bin picking sys em o andomly loca ed indus ial pa s.
In P oceedings o he 2015 IEEE In e na ional Con e ence on Technologies o P ac ical Robo Applica ions (TePRA), Wobu n,
MA, USA, 11–12 May 2015; pp. 1–6. [C ossRe ]
5.
Kuo, H.-Y.; Su, H.-R.; Lai, S.-H.; Wu, C.-C. 3D objec de ec ion and pose es ima ion om dep h image o obo ic bin picking. In
P oceedings o he 2014 IEEE In e na ional Con e ence on Au oma ion Science and Enginee ing (CASE), New Taipei, Taiwan,
18–22 Augus 2014; pp. 1264–1269. [C ossRe ]
6.
Wang, J.; Sun, K.; Cheng, T.; Jiang, B.; Deng, C.; Zhao, Y.; Liu, D.; Mu, Y.; Tan, M.; Wang, X.; e al. Deep High-Resolu ion
Rep esen a ion Lea ning o Visual Recogni ion. IEEE T ans. Pa e n Anal. Mach. In ell.
2021
,43, 3349–3364. [C ossRe ] [PubMed]
7.
Chen, L.-C.; Papand eou, G.; Kokkinos, I.; Mu phy, K.; Yuille, A.L. DeepLab: Seman ic Image Segmen a ion wi h Deep
Con olu ional Ne s, A ous Con olu ion, and Fully Connec ed CRFs. IEEE T ans. Pa e n Anal. Mach. In ell.
2017
,40, 834–848.
[C ossRe ] [PubMed]
8.
Khalid, M.U.; Hage , J.M.; K aus, W.; Hube , M.F.; Toussain , M. Deep Wo kpiece Region Segmen a ion o Bin Picking. In
P oceedings o he 2019 IEEE 15 h In e na ional Con e ence on Au oma ion Science and Enginee ing (CASE), Vancou e , BC,
Canada, 22–26 Augus 2019. [C ossRe ]
9.
Dolezel, P.; S u sa, D.; Honc, D. Rapid 2D Posi ioning o Mul iple Complex Objec s o Pick and Place Applica ion Using
Con olu ional Neu al Ne wo k. In P oceedings o he 2020 24 h In e na ional Con e ence on Sys em Theo y, Con ol and
Compu ing (ICSTCC), Sinaia, Romania, 8–10 Oc obe 2020; pp. 213–217. [C ossRe ]
10.
Liu, Y.; Zhang, Q.; Zhang, D.; Han, J. Employing Deep Pa -Objec Rela ionships o Salien Objec De ec ion. 2019, pp. 1232–1241.
A ailable online: h ps://openaccess. hec .com/con en _ICCV_2019/h ml/Liu_Employing_Deep_Pa -Objec _Rela ionships_
o _Salien _Objec _De ec ion_ICCV_2019_pape .h ml (accessed on 20 Decembe 2021).
11.
Qi, Q.; Zhao, S.; Shen, J.; Lam, K.-M. Mul i-scale Capsule A en ion-Based Salien Objec De ec ion wi h Mul i-c ossed Laye
Connec ions. In P oceedings o he 2019 IEEE In e na ional Con e ence on Mul imedia and Expo (ICME), Shanghai, China,
8–12 July 2019; pp. 1762–1767. [C ossRe ]
12.
Kaise , A.; Zepeda, J.A.Y.; Boubekeu , T. A Su ey o Simple Geome ic P imi i es De ec ion Me hods o Cap u ed 3D Da a.
Compu . G aph. Fo um 2018,38, 167–196. [C ossRe ]
13.
Li, P.; Wang, R.; Wang, Y.; Tao, W. E alua ion o he ICP Algo i hm in 3D Poin Cloud Regis a ion. IEEE Access
2020
,8,
68030–68048. [C ossRe ]
14.
Shi, X.; Peng, J.; Li, J.; Yan, P.; Gong, H. The I e a i e Closes Poin Regis a ion Algo i hm Based on he No mal Dis ibu ion
T ans o ma ion. P ocedia Compu . Sci. 2019,147, 181–190. [C ossRe ]
15.
Bimbo, J.; Luo, S.; Al hoe e , K.; Liu, H. In-Hand Objec Pose Es ima ion Using Co a iance-Based Tac ile o Geome y Ma ching.
IEEE Robo . Au om. Le . 2016,1, 570–577. [C ossRe ]
16. Thalmann, R.; Meli, F.; Küng, A. S a e o he A o Tac ile Mic o Coo dina e Me ology. Appl. Sci. 2016,6, 150. [C ossRe ]
17.
Pasine i, S.; Hassan, M.M.; Ebe ha d , J.; Lancini, M.; Docchio, F.; Sansoni, G. Pe o mance Analysis o he PMD Camboa d
Pico lexx Time-o -Fligh Came a o Ma ke less Mo ion Cap u e Applica ions. IEEE T ans. Ins um. Meas.
2019
,68, 4456–4471.
[C ossRe ]
18.
Schla p, J.; Csencsics, E.; Schi e , G. Design and e alua ion o an in eg a ed scanning lase iangula ion senso . Mecha onics
2020,72, 102453. [C ossRe ]
Senso s 2022,22, 1536 18 o 18
19.
Liu, M.-Y.; Tuzel, O.; Vee a agha an, A.; Taguchi, Y.; Ma ks, T.K.; Chellappa, R. Fas objec localiza ion and pose es ima ion in
hea y clu e o obo ic bin picking. In . J. Robo . Res. 2012,31, 951–973. [C ossRe ]
20.
Ko , T.; Bobo ský, Z.; Heczko, D.; Vysocký, A.; Vi gala, I.; P ada, E. Using Vi ual Scanning o Find Op imal Con igu a ion o a
3D Scanne Tu n able o Scanning o Mechanical Pa s. Senso s 2021,21, 5343. [C ossRe ] [PubMed]
21.
Wen, B.; Mi ash, C.; Soo ian, S.; Kimmel, A.; Sin o , A.; Bek is, K.E. Robus , Occlusion-awa e Pose Es ima ion o Objec s G asped
by Adap i e Hands. In P oceedings o he 2020 IEEE In e na ional Con e ence on Robo ics and Au oma ion (ICRA), Pa is, F ance,
31 May–31 Augus 2020; pp. 6210–6217. [C ossRe ]
22.
Ta yudi; Wang, M.-S. 3D objec pose es ima ion using s e eo ision o objec manipula ion sys em. In P oceedings o he 2017
In e na ional Con e ence on Applied Sys em Inno a ion (ICASI), Sappo o, Japan, 13–17 May 2017; pp. 1532–1535. [C ossRe ]
23.
Heczko, D.; Ošˇcádal, P.; Ko , T.; Huczala, D.; Semjon, J.; Bobo ský, Z. Inc easing he Reliabili y o Da a Collec ion o Lase Line
T iangula ion Senso by P ope Placemen o he Senso . Senso s 2021,21, 2890. [C ossRe ] [PubMed]
24.
Senso Head—LJ-X8080|KEYENCE UK & I eland. A ailable online: h ps://www.keyence.co.uk/p oduc s/measu e/lase -2d/
lj-x8000/models/lj-x8080/ (accessed on 21 Decembe 2021).
25.
Robo Simula o CoppeliaSim: C ea e, Compose, Simula e, any Robo —Coppelia Robo ics’. A ailable online: h ps://www.
coppelia obo ics.com/ (accessed on 1 July 2020).
26.
Zhou, Q.-Y.; Pa k, J.; Kol un, V. Open3D: A Mode n Lib a y o 3D Da a P ocessing. a Xi
2018
, a Xi :1801.09847. A ailable
online: h p://a xi .o g/abs/1801.09847 (accessed on 21 Decembe 2021).
27.
Rusu, R.B.; Cousins, S. 3d is he e: Poin cloud lib a y (pcl). In P oceedings o he 2011 IEEE In e na ional Con e ence on Robo ics
and Au oma ion, Shanghai, China, 9–13 May 2011; pp. 1–4.
28.
Voce ka, M.; Bobo ský, Z.; Babjak, J.; Sude , J.; G ushko, S.; Mlo ek, J.; K ys, V.; Haga a, M. In luence o D i on Robo
Repea abili y and I s Compensa ion. Appl. Sci. 2021,11, 10813. [C ossRe ]
29.
Sude , J.; Bobo sky, Z.; Zeman, Z.; Milo ek, J.; Voce ka, M. The in luence o annealing empe a u e on ensile s eng h o polylac ic
acid. MM Sci. J. 2020,2020, 4132–4137. [C ossRe ]
30.
Huczala, D.; Ošˇcádal, P.; Spu ný, T.; Vysocký, A.; Voce ka, M.; Bobo ský, Z. Came a-Based Me hod o Iden i ica ion o he
Layou o a Robo ic Wo kcell. Appl. Sci. 2020,10, 7679. [C ossRe ]
31.
Ošˇcádal, P.; Heczko, D.; Vysocký, A.; Mlo ek, J.; No ák, P.; Vi gala, I.; Sukop, M.; Bobo ský, Z. Imp o ed Pose Es ima ion o
A uco Tags Using a No el 3D Placemen S a egy. Senso s 2020,20, 4825. [C ossRe ] [PubMed]