Recei ed 6 May 2024, accep ed 27 May 2024, da e o publica ion 30 May 2024, da e o cu en e sion 6 June 2024.
Digi al Objec Iden i ie 10.1109/ACCESS.2024.3407341
Alignmen and Imp o emen
o Shape-F om-Silhoue e
Recons uc ed 3D
Objec s
ALBERTO J. PEREZ 1, JAVIER PEREZ-SOLER 2, JUAN-CARLOS PEREZ-CORTES2,
AND JOSE-LUIS GUARDIOLA2
1Depa amen o de In o má ica de Sis emas y Compu ado es (DISCA), Uni e si a Poli ècnica de València, 46022 Valencia, Spain
2Ins i u o Tecnológico de In o má ica (ITI), Uni e si a Poli ècnica de València, 46022 Valencia, Spain
Co esponding au ho : Albe o J. Pe ez ([email p o ec ed].es)
This wo k was suppo ed in pa by Eu opean Union Ho izon Eu ope P og amme ‘‘A i icial In elligence D i en Indus ial Equipmen
P oduc Li e Cycle Boos ing Agili y, Sus ainabili y and Resilience’’ (AIDEAS) unde G an 101057294; and in pa by he Gene ali a
Valenciana h ough Ins i u o Valenciano de Compe i i idad Emp esa ial [Valencian Ins i u e o Business Compe i i eness (IVACE)]
Dis ibu ed Nomina i ely o Valencian Technological Inno a ion Cen es unde P ojec IMAMCA/2023/11.
ABSTRACT 3D objec alignmen is essen ial in mul iple ields. Fo ins ance, o allow p ecise measu emen s
in me ology, o pe o m su ace/ olume ic checks o quali y con ol in indus ial inspec ion, o align pa ial
cap u es o a 3D objec du ing objec scanning, o simpli y objec ecogni ion o classi ica ion in pa e n
ecogni ion, accu acy and speed, being opposed, a e desi able ea u es o hose algo i hms. Ne e heless,
hey can be mo e o less c i ical depending on he applica ion a ea. In he p esen wo k, we p opose a
me hodology o imp o e he alignmen o 3D objec s econs uc ed using shape- om-silhoue e echniques.
This econs uc ion echnique p oduces objec s wi h small syn he ic bulges, making hem mo e di icul
o align accu a ely. On he one hand, p ealignmen and b anch-and-bound echniques a e used o imp o e
he con e gence and speed o he alignmen algo i hms. On he o he hand, a me hod o ob ain a p ecise
alignmen e en in he p esence o bulges is p esen ed. Finally, a e inemen o he shape- om-silhoue es
echnique is shown. This echnique uses mul iple cap u es o e ine objec econs uc ion and educe o
elimina e, among o he imp o emen s, syn he ic bulges.
INDEX TERMS 3D alignmen , 3D econs uc ion, shape- om-silhoue e, b anch-and-bound, indus ial
inspec ion.
I. INTRODUCTION
3D econs uc ion om images is a powe ul g oup o ech-
niques o cap u ing and ep esen ing he h ee-dimensional
s uc u e o objec s and scenes using a se o wo-dimensional
images. Those echniques a e widely used in ields such
as compu e ision, obo ics, augmen ed eali y, human
pose es ima ion, and medical imaging [1],[2],[3],[4].
Binocula dispa i y, mo ion, silhoue es, linea pe spec i e,
a mosphe e sca e ing, shading, ex u es, occlusions, and
bila e al symme ies a e among o he ea u es used o induce
3D s uc u es om 2D images [2],[5].
The associa e edi o coo dina ing he e iew o his manusc ip and
app o ing i o publica ion was Joewono Widjaja .
The shape- om-silhoue es (SFS) app oach [6],[7],[8]
ob ains a 3D model om he silhoue es o an objec ob ained
om images aken om di e en posi ions. This me hod
does no equi e de ailed ex u e in o ma ion, making i
applicable in scena ios whe e ex u e in o ma ion is lacking
o un eliable. Howe e , i elies hea ily on an accu a e
silhoue e ex ac ion and came a calib a ion o success ul
econs uc ion [7],[9],[10]. This echnique is cu en ly
applied in se e al ields, such as indus ial inspec ion [11],
[12], human pose ecogni ion [13],[14], medical imaging
[4],[15].
In SFS, he objec ’s 3D shape is cons uc ed h ough
he in e sec ion o silhoue e cones de i ed om mul iple
images. Each silhoue e cone o igina ed om he union o he
VOLUME 12, 2024
2024 The Au ho s. This wo k is licensed unde a C ea i e Commons A ibu ion-NonComme cial-NoDe i a i es 4.0 License.
Fo mo e in o ma ion, see h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/ 76975
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
FIGURE 1. Gene a ion o he silhoue e cone o an objec o each came a
in ol es he ollowing s eps: Fi s , an image o he objec is cap u ed (a).
Nex , he objec is segmen ed in he image o p oduce a bina y silhoue e.
Finally, a silhoue e cone (d) is c ea ed by combining he p ojec ion cones
o all he ‘‘objec ’’ pixels (c) wi hin he silhoue e, using he came a
pa ame e s.
p ojec ion o pixels iden i ied as ‘‘objec ’’ in he segmen ed
image (Figu e 1). The in e sec ion o hese silhoue e cones
ep esen s he isual hull [16], de ining he la ges shape
consis en wi h he objec ’s silhoue es obse ed om any
iewpoin wi hin a speci ied a ea. This compu a ion elies
on in insic and ex insic came a pa ame e s [6]. Hence,
as p e iously no ed, ensu ing accu a e segmen a ion and
came a calib a ion is essen ial o main aining econs uc ion
p ecision.
To ob ain he isual hull, i s an oc ee s uc u e is
gene a ed by a ca ing p ocess o an ini ial 3D cube using he
space ou side he silhoue e cone o each image [17]. Nex a
polygoniza ion o he oc ee s uc u e is pe o med by using
a ma ching cubes algo i hm [18].
FIGURE 2. Visual hull ob ained om silhoue e cone in e sec ions.
Syn he ic bulges appea on he objec econs uc ion (in ed) depending
on he numbe o came as and hei posi ions.
FIGURE 3. Cube econs uc ion om 16, 24, and 48 came as (le o igh ).
This echnique allows o econs uc he 3D shape o
an objec only om images aken om calib a ed came as
posi ioned a ound he objec . The mo e came as a e used, he
be e he econs uc ion accu acy (Figu e 3).
Al hough simple concep ually, his econs uc ion me hod
p esen s wo main d awbacks. On he one hand, he conca e
su ace egions can ne e be dis inguished using silhoue e
in o ma ion alone, hus making his unsui able o hose
objec s a p io i. On he o he hand, syn he ic bulges can
appea depending on he numbe o came as and hei
posi ions (see Figu e 2). Those bulges can complica e
he alignmen ope a ions c i ical in many applica ions:
me ology, indus ial inspec ion, quali y con ol, 3d objec
ecogni ion, and classi ica ion.
This wo k p esen s se e al me hodologies o add ess
syn he ic bulges and 3D alignmen . A echnique is p oposed
o imp o e econs uc ion accu acy in shape- om-silhoue e
me hods by u ilizing mul iple cap u e se s.
The pape is o ganized as ollows: sec ion II p esen s
o he ela ed wo ks, sec ion III in oduces p ealignmen
echniques o ob ain good ini ial guesses on ICP, sec ion IV
p esen s an accu a e me hod o align e e ence models and
econs uc ed objec s. In sec ion V, a me hodology is shown
o e ine econs uc ion using di e en se s o cap u es o
76976 VOLUME 12, 2024
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
he same objec . In sec ion VI, some esul s a e p esen ed,
in sec ion VII he esul s a e summa ized, and inally,
in sec ion VIII he conclusions o ou wo k a e exposed.
II. RELATED WORKS
A. ALIGNMENT
Among he alignmen algo i hms, he Poin - o-Plane I e a i e
Closes Poin (ICP) [19],[20] o e s in gene al a p ecise,
obus , and e icien solu ion o ma ch igid su aces o poin
clouds i a good ini ial guess is p o ided. In p esence o
noise, Spa se ICP [21] o EM-ICP [22] can be employed.
Ne e heless, hose me hods equi e mo e compu a ional
ime and some ex a pa ame e s ha e o be es ima ed o ejec
ou lie s co ec ly o o es ima e he exis ing noise [23].
Fea u ed-based alignmen [24],[25] ies o iden i y
su ace o geome ical ea u es o es ablish a cons ella ion o
ea u es ha allow an alignmen ans o ma ion o be ound.
Those me hods ha e subs an ial limi a ions i no ex u e o
dis inguishable ea u es exis . The compu a ional cos o he
ea u e sea ch can be high i he ea u es a e complex.
O he s a egies based on P incipal Componen Analysis
(PCA) [26], Deep Lea ning [27] o objec symme ies [28],
[29] do no need a coa se alignmen o a good alignmen
ini ial guess, as ICP me hods, bu hey do no p o ide,
in gene al, an accu a e alignmen . Those s a egies can be
use ul o ob ain an ini ial guess o he ICP algo i hms o
o o he s asks whe e an exac ma ching is no necessa y, o
example o objec classi ica ion o ecogni ion.
B. SHAPE-FROM-SILHOUETTES
The SFS me hod belongs o he mul i- iew econs uc ion
me hods. Those me hods y o econs uc he 3d s uc u e
o an objec based on 2D images. Among hose me hods we
can ind:
•S uc u e- om-Mo ion (SFM) [30],[31],[32]: objec
ea u es mus be iden i ied in di e en cap u es whe e
an objec is in mo ion o he came a mo es. A ma ching
p ocess uses hose ea u es and he came a model o
econs uc he scene.
•Mul i- iew S e eo (MVS) [33]: using images om
wo di e en calib a ed came as, objec ea u es a e
iden i ied and iangula ed o c ea e a poin -cloud
ep esen a ion o he objec o su aces using, o
example, pho oconsis ency [34].
•Deep Lea ning me hods (DL): deep con olu ional
neu al ne wo ks (CNN) o s e eo econs uc ion
(DeepMVS [35]) o isual hull lea ning (SiINET [36]).
Objec s o econs uc ha e o be p esen ed o he CNN
i s .
As commen ed be o e, SFS me hods do no equi e
ex u ized objec s o in e he 3D s uc u e o an objec ,
as SFM o MVS me hods o e en DeepMVS, because hey
a e based on silhoue es. Besides, compu ing silhoue es is,
in gene al, a mo e s aigh o wa d and less ime consuming
p ocess han sea ching and iden i ying ea u es on images.
Ne e heless, inconsis en silhoue es pose a po en ial
challenge, mainly when dealing wi h poo calib a ion o inac-
cu a e o noisy silhoue es [10]. The econs uc ion quali y
hinges on se e al ac o s, including calib a ion p ecision,
silhoue e accu acy, and he quan i y o came as employed.
I ’s wo h no ing ha he numbe o came as u ilized di ec ly
impac s he occu ence o bulges, as p e iously discussed.
Ou wo k was mainly de eloped in he con ex o indus ial
inspec ion using a well-calib a ed de ice [11]. The desc ibed
de ice econs uc s ee- alling objec s using he images
aken by a cons ella ion o 16 came as. Ligh condi ions and
backg ound a e con olled. Thus, objec segmen a ion can be
done accu a ely.
III. PREALIGNMENT
The con e gence speed o aligning 3D objec s using ICP
depends on he ini ial objec o ien a ions. The mo e he
o ien a ions di e , he mo e compu a ion powe and ime a e
equi ed. I is possible e en ha he ICP does no con e ge in
some si ua ions, ypically i o ien a ions a e oo di e en [37]
being unable o align he objec s. To a oid his p oblem and o
educe compu a ional powe , ough alignmen echniques a e
commonly employed o ob ain ini ial guesses (Sec ion II-A).
Some o hose echniques o e one o mo e hypo heses ha
he ICP should explo e o ensu e con e gence. Tha implies
mo e compu a ional cos , which will be add essed la e in his
sec ion.
FIGURE 4. P incipal axis compu ed by PCA o e he poin cloud o a 3D
objec :
1( ed),
2(g een) and
3(blue).
In he p esen wo k, p ealignmen based on PCA is
employed because i is s aigh o wa d and as o compu e
om a poin -cloud ep esen a ion o a 3d objec [26]. The
PCA analysis gi es h ee o hogonal axes (p incipal axis)
ep esen ing he di ec ions whe e he objec poin s p esen
maximal a iance. As shown in Figu e 4, aligning an objec
along i s p incipal axis o e s a as me hod o no malize
objec o ien a ion and, hus, simpli y objec alignmen . This
app oxima ion can no achie e a p ecise alignmen i objec s
VOLUME 12, 2024 76977
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
FIGURE 5. A e compu ing PCA, ou p ealignmen hypo heses ha e o
be conside ed, aking in o accoun bo h o ien a ions o
1and
2.
p esen de ec s, acquisi ion e o s, noise, o , as in ou case,
econs uc ion bulges. Fo his pu pose, he ICP algo i hm is
employed.
I is impo an o no e ha he p incipal axes ep esen
he maximal a iance di ec ions; hus, i objec s a e no
symme ic, bo h o ien a ions mus be conside ed o he wo
i s p incipal axes, his leads o 4 possible p ealignmen
hypo heses.
Being 1, 2and 3 he eigen ec o s (p incipal axis)
compu ed o he poin se o an objec wi h eigen alues e1>
e2>e3, he objec can be aligned using he ans o ma ion
ma ix,
h1=
1
2
3
conside ing each possible o ien a ion o 1and 2, he
ollowing ans o ma ion mus be conside ed equally (see
Figu e 5),
h2=
1
− 2
− 3
,h3=
− 1
2
− 3
,h4=
− 1
− 2
3
P incipal axes a e o de ed by i s eigen alues ha ep esen
he a iance explained in each axis, bu i wo o mo e eigen-
alues a e simila , se e al hypo heses mus be conside ed
because he o de is no de ined, and eigen ec o s can be
selec ed in se e al combina ions (see Table 1). Fo example in
he Figu e 6eigen alues a e e1≈e2≈e3because p incipal
axes ha e simila a iance.
The ICP (I e a i e Closes Poin ) algo i hm i e a es o
ob ain he o a ion ans o m ha aligns a pai o poin se s,
which, in ou case, is ob ained om a couple o objec s. Each
i e a ion minimizes a simpli ied and linea ised exp ession o
he quad a ic e o [19] using leas squa es un il con e gence.
TABLE 1. PCA hypo hesis o conside in non-symme ic objec s
depending on he eigen alues.
FIGURE 6. The eigen alues o his objec ’s p incipal axis (ei) gi e a
simila alue. Thus, 24 hypo heses should be conside ed i he objec is
no symme ic.
The simpli ied exp ession is alid i objec s a e no oo much
misaligned. Fo his eason, a good ini ial guess o p ealign
hypo hesis is necessa y o ob ain good con e gence.
E e y hypo hesis has o be e alua ed by he ICP i
objec s a e no symme ical; hus, he alignmen cos can
inc ease signi ican ly. To minimize he cos o e alua ing
e e y hypo hesis, he au ho s p opose a b anch and bound
algo i hm o p une he hypo heses ha a e no con e ging as
enough. Du ing a ew numbe s o i e a ions, I0( one o wo
a e enough in ou expe imen s), all hypo heses a e conside ed
in pa allel. Then, he bes esul is used o compu e a bound
(e ∗B), and hypo heses pe o ming wo s a e p uned (See
Algo i hm 1).
Those echniques sol e o ien a ion. T ansla ion is easily
sol ed b inging he objec ’s cen e o mass o he o igin.
IV. ACCURATE ALIGNMENT WITH BULGES
The p esence o bulges complica es ICP ask because hei
loca ion depends on he objec ’s o ien a ion du ing cap u e.
A good designed cap u e sys em will y o minimize hose
a i ac s bu in some si ua ions ha can a ec he alignmen .
Fo example, i a cap u ed objec has o be aligned wi h
i s CAD e e ence o check dimensions, su ace de ec s,
o ien a ion, o wha e e (see Figu e 7) bulges in he cap u ed
image can educe alignemen p ecission.
76978 VOLUME 12, 2024
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
Algo i hm 1 B&B ICP
1: INPUT: Obj,Re , 1E ,B,I0,IT
2: Obj′=PCA(Obj)
3: {Re ′
i:i=1. . . NH} = Hypo hesysPCA(Re )
4: e 0=0
5: epea
6: n=n+1
7: o all Re ′
ido
8: {e in,Re ′
i} = ICPi e a ion(Obj′,Re ′
i)
9: end o
10: un il i<Io
11: epea
12: e =mini=1...NH(e in)
13: n=n+1
14: o all Re ′
ido
15: i ∥e in−e in−1∥> 1E hen
16: i e in<B∗e hen
17: {Re ′
i,e in} = ICPi e a ion(Obj′,Re ′
i)
18: else
19: {Hypo hesis P unned}
20: end i
21: else
22: {Hypo hesis Con e ged}
23: end i
24: end o
25: un il i<IT
26: s=a g mini=1...NH(e in)
27: OUTPUT: Re ′
s
To sol e his p oblem, an i e a i e ICP has been p oposed
(see Algo i hm 2). Exac ma ching is no possible because
o he bulges, hen a e inding he a ine ans o ma ion
(M) ha bes align he e e ence objec (Re ) wi h he
econs uc ed objec (Obj) using ICP, a i ual se o images
(IRe ) o he e e ence objec in he aligned posi ion (Re ′) is
ob ained. This can be done using he calib a ion pa ame e s
(Calib) o he came a se up used o econs uc he objec
and using z-bu e echniques [38] o p ojec he e e ence
on he came as. This se o images allows us o c ea e
a econs uc ed e sion o he e e ence (Re ′
) ha will
p esen bulges mo e o less in he same posi ions ha he
econs uc ed objec , and hus applying ICP again wi h he
new e e ence a mo e p ecise alignmen should be ob ained.
This is epea ed un il con e gence.
V. REFINEMENT FROM MULTIPLE CAPTURES
The mos impo an d awback o he shape- om-silhoue e
econs uc ion me hod is he p esence o bulges. Thei p es-
ence educes he accu acy o measu emen s and complica es
alignmen , bu mo e in e es ingly, i a oids he possibili y o
ob aining p ecise models o cap u ed objec s.
As commen ed, hose bulges can be minimized using mo e
came as (see Figu e 3) bu inc easing he numbe o came as
is no always easible due o cos o complexi y easons.
Ob aining mo e images om a mo ing came a implies, on he
FIGURE 7. Cap u e bulges (le ) can educe alignmen p ecision when
aligning wi h CAD models ( igh ).
Algo i hm 2 I e a i e ICP
1: INPUT: Obj,Re ,Calib, 1E
2: n=0
3: {M,e n} = ICP ans (Obj,Re )
4: Re ′=T ans (Re ,M)
5: epea
6: n=n+1
7: IRe ′= {i1,i2,...,ic} = Zbu e (Calib,Re ′)
8: Re ′
=Recons uc 3D(IRe ′)
9: {M ,e n} = ICP ans (Obj,Re ′
)
10: Re ′=T ans (Re ,M )
11: un il |e n−e n−1|< 1E
12: OUTPUT: Re ′
one hand, mo e cap u e ime and, on he o he hand, and
mo e impo an ly, i adds complexi y because o he came a
posi ion and o ien a ion ha e o be known e y accu a ely o
he econs uc ion algo i hm o wo k [11].
In his sec ion, we p opose o e ine econs uc ion using
se e al se s o cap u es o he same objec . We assume objec s
a e cap u ed in di e en posi ion each ime, ei he because
hey a e p esen ed o he cap u e sys em so o , as in [11],
because hey a e cap u ed on ee all h ough he sys em.
As explained in Sec ion I, o each se o cap u es, an oc ee
s uc u e is gene a ed by ca ing an ini ial 3D cube using he
ou side o he silhoue e cone o each image (see Figu e 9).
Wi h each image, he econs uc ion o he cap u ed objec
is e ined. The exac came a posi ions, o ien a ions, and
in insic pa ame e s mus be known o compu e he silhoue e
cone. Those a e ob ained in a calib a ion p ocess o a eal
sys em [11].
A second se o images can no be used o keep
ca ing he oc ee because he objec is in a di e en
posi ion and he ca ing p ocess will no e ode in he igh
places. Ne e heless, we can align he objec econs uc ions
ob ained om each se , change he came a posi ion in one
o hem o ma ch he objec ’s posi ion and o ien a ion, and
edo he ca ing p ocess wi h bo h se s o images and he
new came a pa ame e s (see Figu e 8). The The esul will
esemble ha o a sys em wi h double came as. In he
same way, se e al se s can be used o e ine i e a i ely he
VOLUME 12, 2024 76979
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
econs uc ion o an objec , allowing us o elimina e bulges
wi h no ex a ha dwa e.
The me hod wo ks as ollow (Algo i hm 3): being
{I1,I2,...,Ik},kse s o cap u es o he same objec , k econ-
s uc ions a e pe o med ob aining {Obj1,Obj2,...,Objk},
a se o econs uc ed objec s. I a CAD e sion o he
objec exis s, i can be used as a e e ence; i his is no
he case, he i s objec , Obj1, is chosen wi hou loss
o gene ali y. Each objec Objiis hen aligned wi h he
e e ence ob aining an a ine ans o ma ion (Mi) use o
modi y he came a posi ion o cap u es Ii(see Figu e 8)
o ma ch he poin o iew o he e e ence. Ini ializing
he se Ci= {[R1,P1],[R2,P2],...,[Rc,Pc]}wi h he
o iginal o ien a ions (Rj) and posi ions (Pj) o each came a
(calib a ion da a), he alues o he cap u e se ia e
ecompu ed as,
R′
j=[MiRT
j]T
P′
j=MiPj
Using he pai s {[Ii,Ci]|i∈[1,k]}, a new oc ee is
compu ed ob aining in o ma ion om all he cap u e se s.
The new oc ee is polygonized using he ma ching cubes
algo i hm [18] o ob ain a 3D objec as in he simple case.
Algo i hm 3 SFS Re inemen
1: INPUT: {I1,I2,...,Ik}
2: INPUT: Calib = {[R1,P1],[R2,P2],...,[Rc,Pc]}
3: o all Iido
4: Obji=Recons uc 3D([Ii,Calib]
5: end o
6: i CADmodel hen
7: Re =CADmodel
8: else
9: Re =Obj1
10: end i
11: o all Objido
12: Mi=ICP ans (Obji,Re )
13: o j=1. . . cdo
14: R′
j=[MiRT
j]T
15: P′
j=[MiPj]
16: end o
17: Ci= {[R′
1,P′
1],[R′
2,P′
2],...,[R′
c,P′
c]}
18: end o
19: Obj =Recons uc 3D([I1,C1],[I2,C2],), . . . [Ik,Ck])
20: OUTPUT: Obj
When implemen ed o educe memo y usage, he p ocess
can be done i e a i ely using a cap u e se each ime, econ-
s uc ing, aligning, modi ying came a calib a ion, e ining
he oc ee, and hen disca ding all his in o ma ion o he nex
i e a ion wi h a new cap u e se .
VI. EXPERIMENTS
To es he p oposed algo i hms, a se o syn he ic objec s
(See Figu e 10 and Table 2) is used. A model o a cap u e
FIGURE 8. In a sys em wi h ou came as, wo se s o cap u es o he
same objec in di e en o ien a ions a e aken ( op). A e aligning bo h
econs uc ions (black a ow), one o he came a se s is eo ien ed
acco dingly, and a new econs uc ion can be pe o med using bo h
cap u e se s (bo om).
FIGURE 9. A ca ing p ocess is pe o med wi h he silhoue e cone o
each came a o e a 3d cube. A sphe e is cap u ed (a), he ini ial cube
(b) and he ca ing p ocess (c)-( ).
de ice (see Figu e 11 [11]) is de ined, and he syn he ic
cap u e se s we e ob ained using a z-bu e echniques [38]
76980 VOLUME 12, 2024
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
FIGURE 10. Tes se o objec s: ha d A, ha d C, ninja A, sp ing, ninja B
and duck (le - igh , up-bo om).
FIGURE 11. Model o he cap u e sys em. A cons ella ion o 16 came as is
a anged in a sphe e poin ing o i s cen e .
FIGURE 12. An example o syn he ic cap u e se .
(see Figu e 12). In he model, he came as a e a anged in a
sphe e o adius 560 cm, wi h 2D senso o 2448×2048 pixels
o size 3.45µmand wi h op ics o ocal leng h o 50 mm.
TABLE 2. Syn he ic objec s s a is ics.
FIGURE 13. E olu ion o he di e en e sions o he ICP algo i hm o
he objec sp ing: s anda d e sion ( op), bound a e he i s i e a ion
(cen e ), bound a e he second i e a ion (bo om). A alue in b acke s
means minimal e o a ained. A Xmeans hypo hesis bounded.
Fo each objec , 20 syn he ic se s o cap u es a e gene a ed.
Each objec was p esen ed in a andom posi ion and
o ien a ion nea he cen e o he cap u e sys em. Posi ion and
o ien a ion we e sa ed as g ound u h o ou expe imen s.
I is wo h commen ing ha syn he ic cap u es a e used,
among o he easons, because posi ion and o ien a ion could
no be a ailable o eal cap u es.
Using hose cap u e se s, 20 objec econs uc ions ha e
been compu ed using he shape- om-silhoue e me hod;
bulges appea on hem in di e en posi ions. Each
econs uc ion was labeled wi h i s posi ion and o ien a ion.
A. PREALIGNEMENT AND BOUNDED ICP
Figu e 13 shows he e olu ion o he ICP algo i hm while
conside ing simul aneously he 4 PCA hypo hesis o he
objec sp ing. Hypo hesis 3 ob ains he co ec alignmen ,
while he o he s lead o inco ec alignmen because he
ICP algo i hm eaches local minima. This example shows
he impo ance o s a ing ICP wi h a good guess, as s a ed
in [37]. The same example illus a es he impo ance o
compu ing enough i e a ions be o e s a ing p uning in he
bounded ICP. In Figu e 13, i can be seen ha p uning a e
VOLUME 12, 2024 76981
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
TABLE 3. Simple e sus bounded ICP: alignmen e o and compu a ional
ime.
FIGURE 14. Alignmen e o s and compu a ional ime compa ison: simple
and bounded ICP.
he i s i e a ion (Io=1) elimina es he co ec hypo hesis;
in his case, a leas wo i e a ions a e necessa y o allow he
ICP o con e ge adequa ely. In ou expe imen s, an I0 alue
o 2 was enough o assu e con e gence o all he objec s.
To es he algo i hms, each econs uc ed objec has been
aligned agains i s o iginal e e ence using bo h simple ICP
and bounded ICP. Being Mo he labeled o a ion, and Mi he
o a ion ob ained wi h he ICP, he alignmen e o is de ined
as he di e ence ans o m angle, ae , compu ed om he
di e ence ans o m, Mdi , as ollows [39],
Mdi =MoM−1
i⇐⇒ Mdi ∗Mi=Mo
ae =(180.0/π)∗acos( (Mdi )−1)/2))
The esul s can be seen in Table 3and Figu e 14. The
alignmen e o mean and s anda d de ia ion among he
FIGURE 15. Alignmen e o s and compu a iona ime on bounded ICP.
20 econs uc ions a e p esen ed o each objec . Alignmen
e o s a e simila o he s anda d and he bounded ICP,
while he bounded ICP has a smalle compu a ional cos .
Objec s ha dA and ha dC p esen mo e signi ican e o s and
dispe sion. In pa because bulges complica e alignmen and
in pa because o hei geome y.
B. ACCURATE ALIGNMENT WITH BULGES
As in he p e ious sec ion, i e a i e ICP has been applied
o he econs uc ed objec s, and alignmen e o s a is ics
ha e been calcula ed. Table 4and Figu e 15 p esen he
esul s. The i e a i e algo i hm signi ican ly imp o es he
alignmen e o mean o e e y objec . Objec s such as duck
o ha dC ob ain he bes esul s, wi h imp o emen s o e
200% e o alignmen . Dispe sion is also g ea ly educed o
hose objec s and ha dA. Remo ing he in luence o bulges,
bo h alignmen e o s and esul dispe sion a e imp o ed.
Ne e heless, he compu a ional cos inc eases signi i-
can ly. On he one hand because he alignmen is done se e al
imes, a mean o 5 i e a ions a e needed o he algo i hm o
con e ge. And on he o he hand, because o he ex a cos o
compu ing he p ojec ion and econs uc ion o he e e ence
wi h bulges in each i e a ion.
C. MULTIPLES CAPTURES
In his sec ion, we show how he mul icap u e econs uc ion
imp o es objec econs uc ion jus in a couple o cases.
76982 VOLUME 12, 2024
A. J. Pe ez e al.: Alignmen and Imp o emen o Shape-F om-Silhoue e Recons uc ed 3D Objec s
TABLE 4. I e a i e alignemen e o s and compu a ional ime.
FIGURE 16. Objec econs uc ion imp o emen h ough mul iple cap u e
e inemen . (le o igh and op o bo om) One cap u e, wo cap u es,
ou cap u es, and 16 cap u es).
In Figu es 16 and 17 i is possible o app ecia e quali a i ely
he imp o emen ob ained using 1, 2, 4, and 16 cap u es o
he same objec o he case o he objec duck and o a
new objec , ube. Especially in e es ing is he case o ube he
hole h ough he objec can be econs uc ed accu a ely using
se e al cap u es.
VII. DISCUSSION
The p esen ed bounded ICP has a smalle compu a ional cos
han he s anda d ICP while o e ing he same alignmen
p ecision. This app oach allows us o conside mo e p ealign-
men hypo heses o ob ain a be e p ecision and con e gence
wid h o he ICP s anda d algo i hm.
On he o he hand, he i e a i e ICP algo i hm signi i-
can ly imp o es he alignmen e o o e e y es ed objec ,
minimizing o emo ing he in luence o bulges on SFS
econs uc ed objec s. This imp o emen is ob ained a he
expense o a highe compu a ional cos , bu i alignmen
accu acy is impo an , he ex a cos is jus i ied.
Finally, he p oposed SFS e inemen me hod allows o
con ol he econs uc ion accu acy i se e al se s o cap u es
FIGURE 17. Ano he example o objec econs uc ion imp o emen
h ough mul iple cap u es (le o igh and op o bo om: 1, 2, 4, and 16
cap u es).
o he same objec in di e en posi ions can be ob ained. Wi h
each new se o cap u es, he econs uc ion can be e ined
educing o elimina ing bulges. This me hod allows a mo e
accu a e SFS econs uc ion wi h less ha dwa e cos .
VIII. CONCLUSION
In he p esen wo k, se e al echniques o wo k wi h objec s
econs uc ed using he shape- om-silhoue es me hod ha e
been p esen ed.
Fi s , a bounded e sion o he ICP alignmen algo i hm
was p esen ed o speed up alignmen when se e al hypo heses
ha e o be conside ed. Expe imen s in Sec ion VI-A show
ha he algo i hm can educe he compu a ional cos , bu
special a en ion should be paid conside ing in which
i e a ion (I0) he bound p ocess s a s o a oid p uning good
solu ions.
Nex , o minimize alignmen p oblems ela ed o syn he ic
bulges appea ing wi h he shape- om-silhoe e me hod,
an i e a i e ICP algo i hm is p esen ed in Sec ion IV. The
co esponding expe imen s in Sec ion VI-B show ha he
p oposed me hod signi ican ly educes alignmen e o a
he cos o inc easing he compu a ional cos app eciably.
The algo i hm was ini ially designed o use CAD models as
e e ences. These kinds o e e ences do no ha e syn he ic
bulges, and he algo i hm can wo k op imally, bu econ-
s uc ed e e ences can also ake ad an age o he algo i hm.
To ha e a CAD model o an objec is common, o example,
in a quali y con ol con ex whe e a pa has o be 3D scanned
and checked agains a model o de ec geome ic o su ace
e o s.
Finally, a mul icap u e econs uc ion app oach is p e-
sen ed in Sec ion V. This p ocess e ines objec econ-
s uc ion wi h each new cap u e, educing bulges on he
VOLUME 12, 2024 76983
