F anke, Jan; Hein ich, Flo ian; Reisch, Ra en T.
A icle — Published Ve sion
Vision based p ocess moni o ing in wi e a c addi i e
manu ac u ing (WAAM)
Jou nal o In elligen Manu ac u ing
P o ided in Coope a ion wi h:
Sp inge Na u e
Sugges ed Ci a ion: F anke, Jan; Hein ich, Flo ian; Reisch, Ra en T. (2024) : Vision based p ocess
moni o ing in wi e a c addi i e manu ac u ing (WAAM), Jou nal o In elligen Manu ac u ing, ISSN
1572-8145, Sp inge US, New Yo k, NY, Vol. 36, Iss. 3, pp. 1711-1721,
h ps://doi.o g/10.1007/s10845-023-02287-x
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/319241
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h p://c ea i ecommons.o g/licenses/by/4.0/
Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721
h ps://doi.o g/10.1007/s10845-023-02287-x
Vision based p ocess moni o ing in wi e a c addi i e manu ac u ing
(WAAM)
Jan F anke1·Flo ian Hein ich1·Ra en T. Reisch2,3
Recei ed: 16 Janua y 2023 / Accep ed: 23 No embe 2023 / Published online: 1 Ma ch 2024
© The Au ho (s) 2024
Abs ac
As ableweldingp ocessisc ucial oob ainhighquali ypa sinwi ea caddi i emanu ac u ing.Thecomplexi yo hep ocess
makes i inhe en ly uns able, which can cause a ious de ec s, esul ing in poo geome ic accu acy and ma e ial p ope ies.
This demands o in-p ocess moni o ing and con ol mechanisms o indus ialize he echnology. In his wo k, p ocess
moni o ing algo i hms based on welding came a image analysis a e p esen ed. A neu al ne wo k o seman ic segmen a ion
o he welding wi e is used o moni o he wo king dis ance as well as he ho izon al posi ion o he wi e du ing welding and
classic image p ocessing echniques a e applied o cap u e spa e o ma ion. Using hese algo i hms, he p ocess s abili y
is e alua ed in eal ime and he analysis esul s enable he di ec ion independen closed-loop-con ol o he manu ac u ing
p ocess. This signi ican ly imp o es geome ic ideli y as well as mechanical p ope ies o he ab ica ed pa and allows
he au oma ed p oduc ion o pa s wi h complex deposi ion pa hs including weld bead c ossings, cu a u es and o e hang
s uc u es.
Keywo ds Wi e a c addi i e manu ac u ing ·Vision based moni o ing ·Machine lea ning ·Nozzle- o-wo k dis ance
moni o ing ·Con ac ube wea o de ec ion ·Spa e de ec ion
In oduc ion
Wi ea caddi i emanu ac u ing(WAAM)isap ocess o 3D
p in ing o la ge, nea -ne -shape me al pa s laye -by-laye ,
using a c welding echnologies. I o e s signi ican ime and
cos ad an ages o a ious applica ions compa ed o con-
en ional sub ac i e me hods (Ma ina & Williams, 2015;
Williams e al., 2016) and has been widely used wi h s eel,
aluminium, i anium, and nickel-based alloys (Rod igues e
Flo ian Hein ich and Ra en T. Reisch ha e con ibu ed equally o his
wo k.
BJan F anke
[email p o ec ed]
Flo ian Hein ich
[email p o ec ed]
Ra en T. Reisch
a[email p o ec ed]
1FORWISS, Uni e si y o Passau, Passau, Ge many
2Technical Uni e si y o Munich (TUM), Munich, Ge many
3Siemens AG, Munich, Ge many
al., 2019). Cu en ly mos WAAM p ocesses a e based on
Cold Me al T ans e (CMT), a modi ied Me al Ine Gas
(MIG) welding p ocess based on sho -ci cui ing ans e
(Sel i e al., 2018). Due o an oscilla ing wi e, i allows
a lowe hea inpu in compa ison o o he welding ech-
nologies, and hus educes esidual s esses. Besides se e al
ad an ages, WAAM is a mul i-scale and mul i-physics cou-
pling p ocess wi h complica ed and unbalanced physical,
chemical, he mal and me allu gical cha ac e is ics (Chen e
al., 2021). The o ming o a wo kpiece is in ensely associ-
a ed wi h he dynamic luid cha ac e is ic o he mel pool,
which is signi ican ly in luenced by se e al p ocess pa am-
e e s such as welding cu en and ol age, welding-speed,
welding-angle o wi e eed a e. Imp ope pa ame iza ion
leads o a se ies o de ec s such as luc ua ion e ec s, oxi-
da ion, po es, lack o usion o slag inclusion (Hause e al.,
2020,2021a,2021b; Wu e al., 2018). Also, a ying hea
dissipa ion condi ions may esul in mo phological changes
o deposi ed beads (Li e al., 2021). Fu he mo e, o ce ain
ma e ials such as s eel, he wi es’ con ac ube wea s o o e
ime, which could lead o ma e ial deposi ion de ec s o spa -
e ing. He e ogeneous e o sou ces lead o a ious de ec s,
123
1712 Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721
nega i ely a ec ing bo h geome ic accu acy and ma e ial
p ope ies. Mo phological e o s p opaga e o he nex laye
and accumula e o e ime. This may lead o a con inuous
de e io a ion o he o e all geome ic accu acy o he pa ,
so ha i e en ually has o be ejec ed. In o de o ensu e high
componen quali y and p oduc i i y, in-p ocess moni o ing
and con ol s a egies a e indispensable.
P ocess de ia ions a e associa ed wi h he change o a -
ious signals, which can be used o online de ec de ec ion.
Se e al senso da a e alua ion algo i hms we e p oposed in
ecen yea s. Tang e al. (2017) used an indus ial came a
behind he welding o ch o cap u e weld bead images and
ained a deep con olu ional neu al ne wo k (DCNN) com-
bining i wi h a suppo ec o machine (SVM) o classi y
weld bead images in o i e pa e ns as no mal, po e, hump,
dep essionandunde cu .In a ed he mog aphyda awasu i-
lized by Chen e al. (2019) o iden i y weld bead de ia ion,
hump and low de ec s using a neu al ne wo k. Reisch e al.
(2020) de eloped a dis ance based anomaly de ec ion based
on he e alua ion o cu en , ol age and welding came a
images wi h a LSMT model, a con olu ion neu al ne wo k
and an au oencode , espec i ely. The p oposed app oach
showed he capabili y o de ec ing anomalies due o oxida-
ion, pollu ed su aces and o m de ia ions. Lee e al. (2021)
used a high dynamic ange came a o ex ac ea u es me al
ans e and weld-pool ha a e used o an a i icial in el-
ligence model o classi y no mal and abno mal s a uses o
a c welding. Li (2021) de eloped wo in elligen WAAM
de ec de ec ion modules. The i s module akes welding
a c cu en and ol age signals du ing he deposi ion p ocess
as inpu s and uses algo i hms such as SVM and inc emen al
SVM o iden i y dis u bances and con inuously lea n new
de ec s. The second module akes CCD images as inpu s and
uses objec de ec ion algo i hms o p edic he un used de ec
du ing he WAAM manu ac u ing p ocess.
Al hough hese s udies con ibu e o he unde s anding
o he complex p ocess dynamics and p o ide me hods
o de ec ing ypical de ec s, i is c ucial o au oma ically
compensa e o weld bead de ia ions du ing p oduc ion o
indus ialize he echnology. Especially, luc ua ions in he
weld bead heigh a e c i ical when accumula ing o e ime.
Since he welding o ch is aised by a p ede ined laye heigh
a e each applied bead, la ge de ia ions in he dis ance
be ween he nozzle o he welding o ch and he wo kpiece,
he nozzle- o-wo k (N W) dis ance dN W, can occu a e
se e al applied laye s, which des abilizes he welding p o-
cess.Alongdis ance may gene a e bad gas p o ec i ee ec s,
esul ing in po osi ies o bad o ma ion o he laye . In con-
as , a sho dis ance can lead o a highe spa e a e and
cause weld spa e o s ick o he nozzle o e en cause a col-
lision be ween he welding nozzle and he wo kpiece (Xiong
& Zhang, 2014). Thus, i is c ucial o ha e a s able laye
heigh esp. dN W h oughou he whole build p ocess.
In ecen yea s, he e has been inc easing esea ch in o me h-
ods o measu ing and con olling he laye heigh . Mos ly a
lase scan is used a e each deposi ed laye o deduce heigh
in o ma ions, which inc eases he in e -laye wai ing ime.
Han e al. (2018) used he ex ac ed ea u es o con ol he
laye heigh by adjus ing he ol age, based on expe ience
ules o a speci ic s eel. The de eloped heigh con olle
was es ed o a mul ilaye mul i bead cuboid s uc u e. Mu
e al. (2022) compa ed he measu emen s agains he CAD
model. Geome ic e o s a e hen compensa ed and a new se
o welding pa ame e s o he nex laye is c ea ed. Li e al.
(2021) in oduced an in e laye closed-loop-con ol (ICLC)
algo i hm o mul i-laye mul i-bead deposi ion o cuboid
componen s wi h s aigh , pa allel pa hs. Heigh e o s a e
de e mined by compa ing he measu emen s o he bead
geome ymodel.Tange al.(2021)es ablishedamul i-senso
sys em o moni o p ocess pa ame e and p oposed a me hod
o weld bead modelling by means o a deep neu al ne wo k.
The eedback o he weld bead shape was ecei ed by means
o an o line weld bead scan.
Šˇce inec e al. (2021) p oposed a laye heigh con olle
based on he a e age a c cu en o he las deposi ed laye .
Heigh de ia ions a e compensa ed by eplanning he ool
pa h. Xiong e al. (2021) de eloped a closed-loop-con ol
(CLC) o he deposi ion heigh by isual inspec ion o p e i-
ous and cu en laye using a passi e ision sys em. Classic
image p ocessing echniques a e used o ack he heigh .
De ia ionsa eau oma icallycompensa ed iacon olling he
wi e eed speed. The solu ion was es ed o a single ack
unidi ec ional wall in one welding di ec ion. Kissinge e al.
(2019)and Hallam e al.(2022) used cohe en ange- esol ed
in e e ome y(CO-RRI) o laye heigh measu emen when
building s aigh walls made o s eel. The senso is moun ed
di ec ly o he welding o ch and measu es he dis ance o
he weld bead immedia ely behind he mel pool. The e is
no inhe en sensi i i y o he a c ligh and i is a cos e ec-
i e and simple o se up solu ion. Mah udian o e al. (2019)
de eloped an a i icial neu al ne wo k wi h one hidden laye
using se e al cha ac e is ic p ocess pa ame e such as a el
and eeding speed, cu en , ol age, e c. as inpu sou ces o
es ima e he dis ance o he wi es’ con ac ube o he wo k
piece in eal ime. Hölsche e al. (2022) ma ched di e -
en p ocess pa ame e s o ha dis ance. I has been shown
ha he elec ical esis ance du ing sho ci cui moni o ed
he dis ance bes o WAAM wi h and wi hou using a CMT
sou ce. Bo h solu ions e i ied hei measu emen echniques
on single- ack expe imen s by con inuously modi ying he
weld dis ance.
The abo e s udies p esen di e en solu ions o ei he
measu ing he weld bead heigh o de e mining he weld
dis ance. Also, laye heigh con ol s a egies a e p oposed,
mos ly alida ed by ab ica ing s aigh walls o cuboids.
Mu e al. (2022) and Šˇce inec e al. (2021) e i ied he
123
Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721 1713
adap i eness o he designed con ol s a egy by ab ica ing
inclined and ee- o m s uc u es. Howe e , when p in ing
complex geome ies, ma e ial agglome a ions o example
a weld bead c ossings could lead o sudden changes in he
welddis ance,se e elyin luencing he o mingquali y,when
building up o e ime. I such sho - e m de ia ions a e no
p ope ly accoun ed o in he con ol s a egy, his can lead
oinaccu a ecompensa ion.Fu he mo e,op ical sys ems o
di ec measu emen o he weld bead dimensions behind he
mol en pool ha e an eccen ic guidance and hus a di ec-
ional dependency on he welding pa h. Reisgen e al. (2019)
u ilized isual moni o ing o he p ocessing zone o de e -
mine heigh in o ma ions in-si u, independen ly o p ocess
pa ame e s, he selec ed ma e ial and he welding di ec ion,
by measu ing he leng h o he isible wi e (s ickou ), using
no CMT welding echnology. Wo kpiece and welding o ch
heigh con ol s a egies we e implemen ed based on an ele-
a ion map and showed he capabili y o compensa e o
su ace e o s and o con ol he dis ance be ween welding
o ch and wo kpiece. Reisch e al. (2021) in es iga ed and
e alua ed he usage o se e al senso s o in-p ocess mea-
su emen o he nozzle- o-wo k dis ance dN W,usingaCMT
welding sou ce. I has been shown, ha a welding came a is
mos sui able o p ocess anddi ec ion independen measu e-
men s. A CLC wi h in-si u deposi ion con ol was de eloped,
whe e laye heigh de ia ions a e compensa ed by adjus ing
he welding speed acco ding o he e alua ions o welding
came a ames wi h a DCNN, p edic ing he isible leng h
o he oscilla ing wi e (s ickou ). Valida ion-pa s showed
signi ican ly educed weld bead de ia ions and hus a highe
geome ical accu acy. Humping e ec s a weld bead c oss-
ings in pa icula could be compensa ed. E en a ee- o m
objec wi h incline s uc u e was p in ed co ec ly on he i s
y wi hou manual in e en ion, while he build up wi hou
deposi ion con ol had o be s opped a e 20 deposi ed laye s
due o se e e oxida ion, misalignmen and uns able p ocess
beha iou .
This pape p esen s image based WAAM p ocess moni-
o ing algo i hms o he CMT welding echnology. The goal
is o ob ain in o ma ion abou he p ocess s abili y in eal
ime. Moni o ing o he dN W is ealized by measu ing he
wi e s ickou . The e o e, he la e p ocedu e using a DCNN,
is ex ended o a wi e segmen a ion. The addi ional seman-
ic in o ma ion makes he p ocedu e mo e obus and no
only he wi e leng h bu also he ho izon al posi ion can be
ex ac ed, which is used o iden i y de lec ion o he wi e.
De lec ion occu s i he wi e con ac ube is wo n o o
dN W is oo la ge. This could lead o uns able a c beha iou ,
inc eased spa e ing o weld seam i egula i ies such as lack
o usion (Henckell e al., 2020) and he esul ing o se
a ec s he so wa e suppo ed pa h planning wi h he isk o
dimensional de ia ions om he CAD model and he wo k
piece (Zhan e al., 2017). Spa e ing indica es ins abili ies in
he welding p ocess o a ious easons, like imp ope eed-
s ock, inco ec welding pa ame e s, insu icien shielding
gas o an inco ec dN W. This may esul in educed su -
ace quali y, s eng h, du abili y and unc ionali y (Se a i
e al., 2023). To cap u e his phenomenon, a no el spa e
moni o ing algo i hm, based on de ec ing changes be ween
consecu i e ames, is p esen ed. The collec ed spa e s a is-
ic is a aluable da a-sou ce o ex-si u es s o he ab ica ed
pa and op imiza ion s a egies conce ning design and depo-
si ion pa hs.
The eminde o his pape iso ganizedas ollows:Sec ion
“Vision sys em” desc ibes he expe imen al se up. WAAM-
and moni o ing-componen s as well as a semi-au oma ic
sys em calib a ion me hod a e in oduced. Sec ion “Wi e
moni o ing” ocuses on he deduc ion o he dN W om
wi e s ickou and discusses in de ail he de eloped machine
lea ning solu ion o wi e moni o ing. A seman ic wi e seg-
men a ion and a classi ica ion o he esul s as well as a
ollowing ea u e ex ac ion is p oposed. Sec ion “Spa e
moni o ing”p esen s a wo-s agealgo i hm o obus lyde ec
spa e o ma ion in welding came a ames. Finally, he ind-
ings o his pape a e summa ized in sec ion “Conclusion”.
Vision sys em
Expe imen al se up
Expe imen s and alida ion es s we e conduc ed on a obo -
based WAAM se up wi h F onius TPS CMT 4000 ad anced
sys em as welding sou ce which is capable o using CMT
wi e eed echnology. The welding o ch was a ached o he
head o a calib a ed six axis obo (COMAU NJ130 2.0), see
Fig. 1. An AlSi12 wi e ( 1.2 mm ) and A gon as ine gas
a a gas low a e o 10 Lmin−1in an ai -condi ioned oom
was used.
Came a
To isually moni o he p ocessing zone, a welding came a
(Ca i a C300) was moun ed o he obo head, see Fig. 2.I
enables he isualiza ion o build-pa and welding p ocess
a he same ime, using ac i e lase -illumina ion. I p o ides
a 8-bi g ayscale ideo s eam and was ope a ed a a sample
a e o 20 Hz wi h a esolu ion o 960 ×740 pixels.
Co ne de ec ion on a checke boa d pa e n was used o
calib a e he welding came a o he p ocessing zone, see Fig.
3. This au oma ically compensa es o he moni o ing angle.
123
1714 Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721
Fig. 1 WAAM sys em o e iew: CMT welding sou ce, obo wi h
a ached welding o ch and subs a e able
Fig. 2 Welding-came a a a wo king dis ance o 200 mm and an angle
o 10◦ o he ho izon al
Wi e moni o ing
Welding s abili y o he CMT p ocess wi h espec o uni-
o m ma e ial deposi ion is moni o ed by obse ing he
oscilla ing wi e. The pe spec i e on he p ocessing zone is
ixed h oughou he build p ocess and p ocess ela ed, he
oscilla ing wi e dips in o he weld bead. Then, he s ickou
lwco esponds o he dN W when added-o a known hidden
po ion lh. This enables he measu emen o he dN W by
measu ing he s ickou lw, see Fig. 4.
F om a classic image p ocessing poin o iew, measu -
ing he s ickou is di icul o achie e due o pe manen ly
Fig. 3 Came a calib a ion: a 3 mm ×3 mm checke boa d pa e n
a wo king dis ance and he calib a ion esul s (blue) - De ec ed inne
checke boa d co ne s
Fig. 4 Welding came a ame. Weld dis ance dN W composed o s ick-
ou lwand hidden po ion lh
changing specula e lec ions and hus badly posed, see Fig.
5.
To o e come he p oblem o op ical measu emen in an
en i onmen wi h highly luc ua ing ligh ing condi ions, a
da a-d i en app oach was adop ed. To obus ly ex ac wi e
in o ma ion in eal ime, a h ee s age a chi ec u e is p o-
posed. Fi s , a neu al ne wo k o seman ic segmen a ion
localizes he wi e, ollowed by a classi ica ion ha checks
he in eg i y o he p edic ion, alida ing i o he inal ea-
u e ex ac ion.
Wi e segmen a ion
Seman ic segmen a ion is used o loca e objec s in an image.
The goal is o label each pixel o an image wi h a co e-
sponding class o wha is being ep esen ed. This can be
123
Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721 1715
Fig. 5 P ocessingzoneo a es -p in a h eedi e en imesdu ingp o-
duc ion. Mo ion and con inuously changing ligh ing condi ions cause
unp edic able specula e lec ions om me al su aces. I is no easy,
e en o he human eye, o see he s ickou
achie ed using he well known U-ne a chi ec u e, o iginally
de eloped o biomedical image p ocessing by Ronnebe g
e al. (2015). The p oposed a chi ec u e was modi ied o
de ec and loca e he isible wi e, u ilizing a ea u e ec-
o =4,8,16,32,64, a single ou pu class (wi e), ze o
padding and addi ional d opou laye s wi h 25% d opou ,
as modi ica ions, see he le side o Fig. 6. I consis s o a
downsampling pa (le ) and a symme ic upsampling pa
( igh ). A downsampling s ep consis s o a back o back exe-
cu ion o wo con olu ions, a ec i ied linea uni (ReLU),
max pooling o downsampling and d opou o p e en o e -
i ing and imp o e he gene aliza ion e o . By epea ing
his, mo e compac and inc easingly abs ac ep esen a ions
( ea u e maps) o he inpu image a e c ea ed, acco ding
o he ea u e ec o . The ne wo k lea ns o de ec a wi e
using hese de ailed ea u e maps. As a esul , he in o -
ma ion abou whe e he wi e is loca ed is los . The spa ial
in o ma ion is eco e ed by upsampling, i.e. by con e -
ing he low- esolu ion ep esen a ions in o a high- esolu ion
image. E e y up s ep consis s o a ansposed con olu ion
ha hal es he numbe and doubles he dimensions o he ea-
u e maps, d opou , a conca ena ion wi h he co esponding
ea u e maps om he downsampling pa and wo con olu-
ions, each ollowed by a ReLU. The co esponding ea u e
maps om he downsampling s ep help o localize he ea-
u es mo e p ecisely du ing upsampling. A he inal laye
a 1x1 con olu ion is used o map each 4-componen ea u e
ec o o one class. The a chi ec u e e u ns a ull esolu ion
p edic ion o a wi e. Tha means, each pixel is assigned a
p obabili y, called p ecision sco e, o belonging o a wi e.
The seman ic segmen a ion model is ained using segmen-
a ion maps. The e o e, welding came a ideos o a ious
es -p in s se ed as da a sou ce. In e es ing subse s om di -
e en p in ing s ages we e anno a ed using a semi-au oma ic
anno a ion ool. In o al, 3000 ames we e labelled, whe e
20% o hem we e se aside o alida ion. The ne wo k
was ained on cu -ou s o dimensions 256x512 pixels which
con ain he immedia e p ocessing zone, oge he wi h hei
co esponding segmen a ion maps wi h alue 1 inside an
enclosing quad angle o he isible wi e and 0 elsewhe e,
see Fig. 7.
σ>ρ TRUEFALSE
b eak Decision h esholding: τ
Segmen a ion
Inpu : Image 512x256 Ou pu :P obabili y P
512x256
14 4 84 4 1
256x128
8 8 16 8
128x64
16 16 32 16
64x32
32 32 64 32
32x16
64
con 1x1
copy
con 3x3, ReLU
max pool 2x2, d opou
ans-con 2x2, d opou
Classi ica ion
Inpu :
P obabili y P
His og am analysis o P
Ou pu :
Bina y wi e
Fig. 6 The p oposed a chi ec u e o compu ing a alid wi e ep esen a ion o ea u e ex ac ion wi h segmen a ion ne wo k and subsequen
classi ica ion
123
1716 Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721
Fig. 7 T aining da a: Sample da um (le ) and co esponding anno a ed
segmen a ion map ( igh )
Fig. 8 Lea ning cu e o he p oposed U-Ne implemen a ion
Thene wo kwasimplemen edwi h heTenso low ame-
wo k1using i s Ke as API and ained o 80 epochs wi h 32
samples pe i e a ion (ba ch size), bina y c oss-en opy loss
and de aul RMSp op op imize achie ing a aining and al-
ida ion accu acy o ∼99.5%. Figu e 8shows he lea ning
cu e.
Classi ica ion
The quali y o a speci ic wi e p edic ion is de e mined by
he accu acy o he ne wo k and he ac ual isual condi ions
p esen . The da a si ua ion can occasionally be poo , leading
hene wo k o p o idewi e p edic ions con aining a eas wi h
low con idence, whe eas in o he egions, he con idence is
high. To decide i he in o ma ion con en o a p edic ion is
sui able o ea u e ex ac ion, i is classi ied in o one o h ee
ca ego iesbymeasu ingi sce ain y.The e o e, hep obabil-
i y dis ibu ion o he p edic ion is conside ed o cons uc a
con idence ma ke . I is assumed ha p ecision sco es below
0.1 a e i ele an o he de e mina ion o a wi e ins ance.
To measu e he ce ain y o he p edic ion, a dicho omy o
1h ps://www. enso low.o g/.
he ele an p ecision sco es is pe o med, using he s an-
da d decision h eshold o seman ic segmen a ion τ=0.5.
A alue abo e τis classi ied as high, o he wise as low. High
ce ain y o a p edic ion is indica ed by a high p opo ion o
high p ecision sco es. Fo a p edic ion P, conside he se s
M={P(x,y)|P(x,y)≥0.1}and (1)
N={P(x,y)|P(x,y)>τ}.(2)
Then σ=|N|
|M|is used as a con idence ma ke and Pis
alida ed by se ing a sui able h eshold ρ o σ.I
σ>ρ, (3)
hen he co esponding p edic ion Pis conside ed o be o
high ce ain y and a bina y wi e ep esen a ion is compu ed
by h esholding Pusing τ. This esul s ei he in a agmen ed
o a ull wi e ep esen a ion, as depic ed wi hin he yellow
and g een ec angle in Fig. 6.I Eq.3does no hold, his indi-
ca es ha he co esponding p edic ion Pis o low ce ain y
and h esholding P esul s in an unsui able wi e ep esen-
a ion o ea u e ex ac ion, see he ed ec angle in Fig. 6.
Images co esponding o in alid classi ied wi es se ed as
da a sou ce o e- eaching o he model. In his way, he ne -
wo k became mo e obus wi h espec o he pe manen ly
changing specula e lec ions om he wi e.
Fea u e ex ac ion
The s ickou is de e mined by he lowes de ec ed pixel in
a bina y wi e e u ned by he classi ica ion. Figu e 9shows
he moni o ing esul o he s ickou s o a speci ic build job
pa wi h ad e se isual condi ions along wi h σ, iden i y-
ing phases du ing ab ica ion, whe e he ea u e ex ac ion
should be suspended.
The bene i o his app oach is ha , compa ed o he
DCNN eg essionmodel used inReisch e al. (2021),now he
p edic ion esul s a e explainable. This b eaks up he black
box cha ac e is ic o he eg ession model and only sui able
p edic ions a e used o deduce he s ickou o p ocess con-
ol. Figu e 10 illus a es he p oblem o using he eg ession
model o p edic he s ickou . Unde ad e se isual condi-
ions, i may e u n plausible alues, bu hey a e no co ec .
Now ha a p edic ion is comp ehensible, no only he
s ickou can be obus ly de e mined, bu also he cen e o
g a i y and hus he posi ion o he wi e. I is deduced om
non- agmen ed wi e ep esen a ions, which is ensu ed by an
addi ional connec ed componen analysis (CCA). Figu e 11
shows h ee alid wi e ep esen a ions o he same build job
and he de i ed ea u es, supe imposed on he co esponding
welding came a ames.
The wi e posi ion is used o ack he wea o o he
con ac ube by measu ing de ia ions in i s ho izon al com-
123
Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721 1717
Fig. 9 Con idence ma ke σ( ed) and s ickou s (blue), deduced wi h
he p oposed app oach. The p opo ion o high con idence alues d ops
when en e ing ad e se isual condi ions a he end o he obse a ion,
iden i ied using a con idence h eshold ρ=0.75, indica ed by a ho i-
zon al line (Colo igu e online)
Fig. 10 S ickou s p edic ed wi h he eg ession model and an unde es-
ima ed s ickou o a wi e, indica ed by a ho izon al line
ponen . Wea o causes he wi e o lose guidance o e ime.
This leads o inc eased deposi ion de ec s, which becomes
unaccep able a a ce ain poin du ing ab ica ion. Then, he
p ocess mus be s opped o exchange he wea pa . The li e
cycle o ha speci ic wea pa can be de e mined adap i ely
in-p ocess independen o a speci ic build job. Figu e 12
shows wo con ac ube examples wi h he wi e guide wo n
o di e en deg ees.
To moni o he posi ional o se o he wi e and conse-
quen ly he wea s a e o he con ac ube, he median o he
posi ional de ia ions om he o iginal is calcula ed o e e y
200 ames along he p ocessing. Figu e 13 shows he mon-
i o ing esul o a 45 minu e build job.
This app oach was es ed wi h a single came a on indus-
ial WAAM da a as s a ed in sec ion “Vision sys em”. In
gene al, i is a p io i no clea in which di ec ion he wea o
will e ol e. In he wo s case, i e ol es in he iew di ec-
ion o he moni o ing sys em and hus canno be de ec ed.
Fig. 11 Robus mapping o he wi e. Moni o ing esul s (blue) and
ex ac ed ea u es: ma ke o end o wi e ( ed) and ho izon al ba ycen-
e (g een) (Colo igu e online)
Fig. 12 Wi e con ac ubes, 50% wo n o (le ) and 100% ( igh )
Fig. 13 De lec ion de ec ion. The median o posi ional o se s, mea-
su ed o e e y 200 images, inc eases o e ime
In mos cases, a sui able posi ioning o he came a is su -
icien . Fo an op imal solu ion, a second came a could be
used, moun ed ho izon al pe pendicula , o ensu e he cap-
u ing o posi ional de ia ions. Howe e , his is accompanied
by highe cos s.
123
1718 Jou nal o In elligen Manu ac u ing (2025) 36:1711–1721
Fig. 14 Spa e de ec ion: image p ocessing pipeline. The inal esul con ains only po en ial spa e componen s
Compu a ional aspec
The e alua ion o a welding came a ame, using he p o-
posed me hod, akes 40 ms on an In el Co e i7-10700K CPU
and 25 ms on he N idia GeFo ce RTX 3070 GPU, wi h a
se up as desc ibed in sec ion “Vision sys em”. The eal ime
moni o ing o he welding p ocess could be ealized and he
de eloped wi e moni o ing sys em enables he in-si u con ol
o he weld bead heigh , when using he CLC o he dN W
designed by Reisch e al. (2021).
Spa e moni o ing
Spa e s in welding migh indica e p ocess ins abili ies. In
he ollowing, an au oma ed spa e de ec ion and quan i i-
ca ion is desc ibed. Moni o ing spa e o ma ion does no
ace he same image p ocessing challenges like moni o -
ing he wi e. Spa e componen s ha e simila shapes and
b igh ness and hus a e, as objec s, easie o de ec . Bu , spec-
ula e lec ions can ha e he same isual cha ac e is ics may
be de ec ed alsely as spa e . These a e iden i ied by de e -
mining changes be ween consecu i e ames. The algo i hm
consis s o wo s ages, spa e de ec ion and a e inemen
s age o emo e alse posi i es.
Spa e de ec ion
Spa e appea s as small sized b igh spo s, is a empo a y
phenomenon and dis ibu es beyond he immedia e p ocess-
ingzone.Thus he ull amesa einspec ed.The ideos eam
is a unc ion in ime wi h disc e e a gumen s. The e o e, he
i s s ep is o compu e he ini e di e ence be ween consecu-
i e ames o emo ing backg ound and low in ensi y a eas.
Nex , clus e s o specula e lec ions, mos ly esiding a ound
he immedia e p ocessing zone a e iden i ied using mo pho-
logical closing. Finally, a CCA keeps only a eas ep esen ing
po en ial spa e . The image p ocessing pipeline o a ame,
oge he wi h heco espondingin e media e esul sdepic ed
o a cu -ou (blue) o isualiza ion easons, is shown in
Fig. 14 .
Backg ound sub ac ion:Fo a ame i,i=0,1, ... conside
i s successo i+1and he pixel-wise ini e di e ence
D= i+1− i.(4)
Using a sui able h eshold T>0, his yields bina y images
+ esp. −con aining high in ensi y a eas only isible in
i+1 esp. i, de ined by:
+(x,y)=1, D(x,y)>T
0,else (5)
−(x,y)=1, D(x,y)<−T
0,else .(6)
Spa e de ec ion mus no be pe o med wice o i, since
hein o ma ionabou po en ial spa e isal eadyknown om
he p e ious i e a ion, excep i i=0. In he la e case, he
es o he spa e de ec ion pipeline is pe o med o bo h
+and −, else i is pe o med only o +. The algo i hm
is insensi i e o he choice o T. Any b igh objec s le a e
he backg ound sub ac ion ha do no ep esen spa e a e
emo ed by he ollowing p ocessing s eps.
Clus e ing: Assuming ha indi idual spa e ins ances ha e
a minimum dis ance dmin o each o he , clus e o specula
e lec ions, mos ly esiding on weld bead and mel pool, can
beiden i iedusingmo phologicalclosing.Thisconnec shigh
in ensi y a eas which a e close oge he , wi h espec o dmin
and lea es he es . This esul s in he mo phological closu e
C( +)o he bina y image +.
Spa e iden i ica ion: Non-spa e -like a eas a e inally
emo ed conside ing hei mo phology and size, using a
CCA. This yields a bina y image S( +)con aining only
spa e -like componen s isible in i+1.
123
