elec onics
A icle
S eel Ba Coun ing om Images wi h Machine Lea ning
Ana Ca en He nández-Ruiz * , Ja ie Alejand o Ma ínez-Nie o and Julio Da id Buldain-Pé ez


Ci a ion: He nández-Ruiz, A.C.;
Ma ínez-Nie o, J.A.; Buldain-Pé ez,
J.D. S eel Ba Coun ing om Images
wi h Machine Lea ning. Elec onics
2021,10, 402. h ps://doi.o g/
10.3390/elec onics10040402
Academic Edi o : Gwanggil Jeon
Recei ed: 18 Decembe 2020
Accep ed: 3 Feb ua y 2021
Published: 7 Feb ua y 2021
Publishe ’s No e: MDPI s ays neu-
al wi h ega d o ju isdic ional clai-
ms in published maps and ins i u io-
nal a ilia ions.
Copy igh : © 2021 by he au ho s. Li-
censee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and con-
di ions o he C ea i e Commons A -
ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
Depa men o Elec onic Enginee ing and Communica ions, Uni e si y o Za agoza, 50018 Za agoza, Spain;
alma inez@uniza .es (J.A.M.-N.); buldain@uniza .es (J.D.B.-P.)
*Co espondence: anaaca en@uniza .es; Tel.: +34-602-803-025
Abs ac :
Coun ing has become a undamen al ask o da a p ocessing in a eas such as mic o-
biology, medicine, ag icul u e and as ophysics. The p oposed SA-CNN-DC (Scale Adap i e—
Con olu ional Neu al Ne wo k—Dis ance Clus e ing) me hodology in his pape is designed o
au oma ed coun ing o s eel ba s om images. I s design consis s o wo Machine Lea ning ech-
niques: Neu al Ne wo ks and Clus e ing. The sys em has been ained o coun ound and squa ed
s eel ba s, ob aining an a e age de ec ion accu acy o 98.81% and 98.57%, espec i ely. In he s eel
indus y, coun ing s eel ba s is a ime consuming ask which highly elies on human labou and is
p one o e o s. Reduc ion o coun ing ime and esou ces, sa e y and p oduc i i y o employees
and high con idence o he in en o y a e some o he ad an ages o he p oposed me hodology in a
s eel wa ehouse.
Keywo ds: con olu ional neu al ne wo ks; s eel ba s; coun ing; clus e ing; machine lea ning
1. In oduc ion
Coun ing is a ime-consuming ask and a key ac o in keeping ack o he in en o y o
any ma e ial. When alking abou objec s wi h di e en shapes and sizes, he ask becomes
mo e challenging. In he s eel indus y, he s eel ba is one o he mos widely used p oduc
in he wo ld o building cons uc ion and o ge. Du ing he manu ac u ing p ocess,
he ba s a e usually coun ed by using images, hus allowing o dis ance, ligh ing and
angle con ol. Howe e , once hey lea e he ac o y, hese hea y and la ge ma e ials mus
be s acked and s o ed in wa ehouses o e ails, whe e a hos ile en i onmen p e ails in
o de o ack a smoo h and eliable in en o y.
T adi ional s eel ba coun ing is based on human calcula ion; howe e , due o he
shi ing condi ions and low manoeu abili y, he manual coun ing is qui e slow and
labou -in ensi e wi h low accu acy a e. The e o e, an au oma ic sys em capable o coun
hese ma e ials ega dless o he physical condi ions is equi ed in o de o imp o e he
e ec i eness and eliabili y o he s eel ba s coun ing p ocess.
Basically, i is possible o dis inguish wo main app oaches o coun ing asks. Image
p ocessing echniques implemen algo i hms based on ma hema ical unc ions o ans o m
an image. Fil e s, h eshold segmen a ion, edge de ec ion and ma ching a e commonly
used echniques [
1
–
4
]. Al hough hese echniques a e highly accu a e, hey a e bounded
o speci ic condi ions such as cons an ligh ning and backg ound, o special came a e-
qui emen s. Mo eo e , hey a e limi ed o ound s eel ba s wi h ixed shape and size,
assuming hei shape is quasici cula , hus lacking obus ness [
5
]. No e ha some o hese
me hods a e bounded o he p oduc ion line in s eel ab ics whe e physical sepa a ion o
he ma e ials is iable [6,7].
Simila ly, o he s eel ba coun ing algo i hms based on image p ocessing a e mainly
based on a ea and empla e algo i hms. Bo h o hese me hods a e easible, bu he e a e
some disad an ages. The esul s o he i s me hod canno di ec ly loca e he s eel ba in
he coun ing esul , so i makes g ea incon enience in e o analysis o he algo i hm [
8
,
9
].
Elec onics 2021,10, 402. h ps://doi.o g/10.3390/elec onics10040402 h ps://www.mdpi.com/jou nal/elec onics
Elec onics 2021,10, 402 2 o 19
Templa e ma ching me hod hea ily depends on he shape o he empla e and a ge objec ,
so he adap i e abili y is limi ed [10,11].
In he case o machine lea ning (ML) echniques, con olu ional neu al ne wo ks
(CNN) ha e demons a ed o be highly accu a e and as enough o image p ocessing.
Well-known CNN a chi ec u es such as Fea u e Py amid Ne wo k, Visual Geome ic
G oup and Incep ion-ResNe a e used as classi ie s and eg esso s in o de o ob ain he
o al numbe o elemen s [
12
–
16
]. As o indus ial applica ions, neu al ne wo ks ha e
p o en o achie e high accu acy and p o ide a as - ack solu ion o p oblems in hos ile
en i onmen s [17,18].
A me hodology ha esembles how humans coun is p esen ed in [
19
]. A CNN im-
plemen ed as a Bina y Classi ie is used o de ec each ba in he image and ma k i as a
candida e cen e. Once e e y candida e cen e is de ec ed, a clus e ing echnique is applied
in o de o ex ac he ac ual geome ic cen e o he ba s. The inal coun is he numbe o
cen es de ec ed. This me hodology, e e ed o as CNN-DC, al hough i achie es 99.26%
o accu acy in 3.58 s, i is es ic ed o a cons an backg ound among he images and a ixed
pa ch size, hus implying ha he dis ance om he came a is cons an .
On he o he hand, a deep lea ning usion model o de ec ing objec s is p oposed
in [
20
]. Localisa ion and segmen a ion o s eel ba s is done by using a combined model.
The model achie es a 98.17% in F1 sco e (ha monic mean alue o p ecision and ecall) in
objec de ec ion in 0.03 s; howe e , he p oposed Incep ion-RFB-FPN a chi ec u e is qui e
complex wi h many laye s and equi es high compu a ional esou ces o i s deploymen ,
hus making i una o dable o an embedded and po able sys em.
Mos o he implemen a ions ocus on coun ing a single objec kind wi h speci ic cha -
ac e is ics and cons an backg ound. Al hough some machine lea ning based wo ks ha e a
high pe o mance, a po able sys em migh no be iable due o he equi ed amoun o
compu a ional esou ces. By conside ing hese issues, a CNN-based coun ing me hodology
app oach, namely Scale Adap i e Con olu ional Neu al Ne wo k Dis ance Clus e ing
(SA-CNN-DC), is p esen ed in his pape in o de o coun s eel ba s ega dless o hei
size and shape by adop ing a compac design wi h a minimum numbe o pa ame e s.
The p oposed coun ing me hodology is desc ibed in Sec ion 2. Sec ion 3summa ises he
image p ocessing echniques applied o gene a e he da ase o each ne wo k in he sys em.
A desc ip ion o he implemen ed neu al ne wo k a chi ec u es and dis ance clus e ing
algo i hm is shown in Sec ion 4, whe e a aining pe o mance o each one is also p e-
sen ed. The mos ele an esul s and he SA-CNN-DC o e all pe o mance alida ion a e
p esen ed in Sec ion 5. A compa ison wi h simila implemen a ions is also made in his
sec ion. The desk op app implemen a ion is desc ibed in Sec ion 6. Finally, conclusions a e
d awn in Sec ion 7.
2. P oposed SA-CNN-DC Coun ing App oach
A a ie y o s eel ba shapes can be ound in he s eel indus y, and each ba kind
usually is manu ac u ed wi h di e en dimensions. The di e si y o sizes and shapes is a
challenge ha he p oposed me hodology add esses by adding an ex a inpu called densi y.
This pa ame e is a undamen al ac o in o de o ob ain an e ec i e ba classi ica ion as
well as i s accu a e localisa ion.
The p oposed me hodology a emp s o imp o e he CNN-DC amewo k p esen ed
in [
19
] which equi es speci ic condi ions o he images conside ed, as well as a ixed
size and shape ba . In addi ion, he p oposed me hod is designed o be po able, epli-
cable and obus o na u al a iabili y o he condi ions in he wa ehouse, such as noise,
ligh and scale.
Basically, he p oposed SA-CNN-DC me hodology au oma ically classi ies he s eel
ba ype, loca es each ba cen e and p o ides an ou pu wi h he o al amoun o ba s.
Bo h he ma e ial image and he densi y a e he inpu s equi ed o he sys em. I is wo h
men ioning ha based on an analysis o he s o ing ma e ial physical condi ions wi hin
he wa ehouse, h ee ypes o densi y we e de ined (low, medium and high) in o de o
Elec onics 2021,10, 402 3 o 19
ensu e obus ness o he a iabili y o he s eel ba dimensions. Figu e 1shows an example
o each o he possible s eel ba densi ies.
(a) (b) (c)
Figu e 1. Possible s eel ba densi ies: (a) Low, (b) medium and (c) high.
The SA-CNN-DC main co e consis s o h ee neu al ne wo ks and a clus e ing ech-
nique. Each ne wo k sol es an speci ic ask: ba classi ica ion, image esizing and cen e
localisa ion. The modula design no only allows po abili y bu also i is eplicable,
hus p o iding he capabili y o add new ma e ials by using he same me hodology. Bea -
ing his in mind, ound and squa ed ba s we e conside ed o coun ing pu poses; howe e ,
angled and ec angula ba s we e also used o aining.
A g aphical ep esen a ion o he p oposed me hodology is shown in Figu e 2.
Thei main s ages a e desc ibed as ollows:
Image Densi y
O iginal
Image
Inpu s Resized Image Fi e
C ops Fi e Resized
C ops Class
Resizing
Fac o
Candida e
Cen e s
Dis ance
Clus e ing
Pa ches
64 ×64
Final
Coun
Cen e s
Visualiza ion
Slidding
Window
22
Isola ion
Fil e
Scale Adap ed
Image
MEDIUMLOW
44
66
55
11
33
HIGH
Bina y
Classi ie
Linea
Reg esso
Classi ie
Bina y
Classi ie
Linea
Reg esso
Classi ie
Bina y
Classi ie
Linea
Reg esso
Classi ie
Fla en o
Op imal
C op
Fla en o Op imal C op
0 / 1
CNN
CNN
64 ×64
Figu e 2. SA-CNN-DC me hodology block diag am.
1.
P ep ocessing: The inpu image is esized o a size limi be ween 3500 and 1700 pixels.
Acco ding o i s densi y ype, i e c ops o di e en sizes a e ex ac ed om he cen e
Elec onics 2021,10, 402 4 o 19
o he image and esized o he CNN inpu size. Fo high densi y, he c ops ex ac ed
a e smalle and ice- e sa o he low densi y.
2.
Classi ica ion: These i e c ops a e eed- o wa d in o he classi ie o ob ain he so max
ou pu . The class is chosen acco ding o he i e p edic ions ob ained h ough o ing,
hus he class wi h he highes numbe o p edic ions is selec ed. F om he co ec
p edic ions he one wi h he highes p obabili y is selec ed. A highe p obabili y
means a mo e eliable p edic ion and i implies he ne wo k is able o de ec ea u es
wi h high ce ain y. The la en ec o o he esponse wi h he highes p obabili y,
which is gene a ed a e he con olu ional laye s and con ains all he in o ma ion o
he image condensed, is s o ed as he op imal la en o he nex s ep.
3.
Linea Reg esso : This ne wo k ou pu s a esizing ac o ,
R
, which is used o esize he
image. This ac o de e mines how much an image mus be esized so ha a single
ba comple ely i s in a pa ch o a ixed size, as seen in Figu e 3. Then, he scale o
he esul ing image is adap ed o he nex s age. The con olu ional laye s o he
classi ie a e used o ain a simple mul ilaye pe cep on wi h a linea ou pu ha
ac s as eg esso o he esizing ac o . This p ocess, called
ans e lea ning
, helps o
d as ically educe he aining da a and ime o he eg esso ne wo k.
4.
Bina y Classi ica ion: A ixed-size sliding window mo es h ough he scale adap ed
image wi h a small s ide. A bina y-ou pu ne wo k classi ies whene e he esul ing
pa ch con ains a ba o no . I a posi i e de ec ion is made, he cen e coo dina e o
he pa ch is s o ed as a candida e cen e.
5.
Dis ance Clus e ing: Candida e cen es a e il e ed acco ding o hei ho izon al and
e ical p oximi y wi h o he candida e cen es. I a candida e cen e is comple ely
isola ed, i is dele ed. The dis ance clus e ing algo i hm measu es he Euclidean
dis ance be ween candida e cen es and g oups hem wi hin a h eshold dis ance.
Fo each clus e , a cen e coo dina e is s o ed, ideally his is he geome ic cen e o
he ma e ial [19].
6.
Ou pu : The inal coun is he o al numbe o cen e coo dina es. Fo isualisa ion
pu poses, he cen es a e ma ked wi h a colou do in he image.
(a) (b) (c)
Figu e 3. Resized image: (a,b) show a w ong esizing, while (c) a co ec ly esized image.
3. Da ase Acquisi ion
Since each neu al ne wo k has an speci ic ask wi hin he p oposed sys em, h ee di -
e en da ase s we e buil o each ne wo k: Classi ie -Da ase , Reg esso -Da ase and
Bina y-Da ase . I is wo h men ioning ha a single inpu size was ixed o he ne wo ks,
hus ob aining a modula design. This inpu size de e mines how much GPU memo y
will be equi ed o he ke nel weigh s in he ne wo k laye s, as well as he ba ch size and
he aining ime. When an image wi h la ge dimensions and high esolu ion is esized
o a conside ably smalle size, i ends up dis o ed and he s eel ba s lose hei shape.
In consequence, he CNN is unable o lea n cha ac e is ics o he ma e ials, ins ead i lea ns
o ex ac in o ma ion om he noise in he image. By conside ing a ade-o be ween he
equi ed GPU memo y and image dis o ion, he inpu size was se o 64
×
64 pixels. Mo e-
Elec onics 2021,10, 402 5 o 19
o e , since he RGB channels do no con ain ele an in o ma ion abou he shape and hey
also equi e a la ge memo y consump ion, he images we e con e ed o g ayscale.
3.1. Classi ie -Da ase
The classi ie was buil o dis inguish be ween s eel ba ypes. Fo his ne wo k
i is impo an ha he samples con ain he ele an shape cha ac e is ics o he ba s.
No e ha an angled, ec angula , ound and squa ed s eel ba s wi h di e en size and
colou , as shown in Figu e 4, we e conside ed as possible classes. I is impo an o
emphasise ha only he ound and squa ed ba s we e used o coun ing, while he angled
and ec angula ca ego ies we e in oduced as ejec ion classes, bu hey will be conside ed
o u u e esea ch.
(a) (b) (c) (d)
Figu e 4. S eel ba classes o he classi ie : (a) angled, (b) ec angula , (c) ound and (d) squa ed.
Fi s , s eel ba pho og aphs wi h di e en dimensions we e manually collec ed in an
o dina y wa ehouse. To ensu e an app op ia e ep esen a ion o he possible condi ions
in he place, i was equi ed o collec se e al images wi h a ia ions in he ligh , ame,
angle and posi ion. In his way, a ound 400 pho og aphs we e aken om each single
ma e ial pile, hus ob aining a small se o 12,793 images.
F om hese collec ed images and by conside ing he a o emen ioned densi y pa ame e ,
a iable size c ops (wi h a andom inc ease o dec ease) we e ex ac ed om di e en
coo dina es wi hin he images. Once ex ac ed, hey we e esized o 64
×
64 pixels and
con e ed o g ayscale. This echnique helps o inc ease he amoun o da a and p e en s
dis o ion due o esizing. Figu e 5shows an example o he mos app op ia e esized
c op (Figu e 5d) ob ained om he o iginal image (Figu e 5a). The c op size was selec ed
acco ding o bo h he densi y and he image dimensions. Table 1shows he da ase c ea ed
o he CNN. The ca ego ies a e qui e balanced and he amoun o da a is enough o a oid
o e i ing when using a i ed CNN size.
(a) (b) (c) (d)
Figu e 5.
Squa ed s eel ba s c ops wi h di e en dimensions: (
a
) O iginal image wi h la ge dimensions and high esolu ion
image, (b) esized small c op, (c) esized big c op and (d) esized co ec c op.

Elec onics 2021,10, 402 6 o 19
Table 1. Classi ie -Da ase sample dis ibu ion.
S eel Ba Numbe o Samples Pe cen (%)
Angled 15,204 25.14
Rec angula 14,995 24.80
Round 15,329 25.35
Squa ed 14,945 24.71
To al 60,473 100
3.2. Reg esso -Da ase
As men ioned be o e, only ound and squa ed ba s we e conside ed o coun ing.
Fo he Linea Reg esso , new pho og aphs wi h a iable dimensions we e aken ho izon-
ally in on o he ma e ial piles whe e he s eel ba s we e uni o mly pain ed. The esul ing
263 images we e manually labelled wi h he ba size in pixels and hei co esponding
densi y. The ba size is de e mined by he heigh and diame e o wid h o he ba ’s sec ion
and i is a single numbe . In his way, he esizing ac o could be compu ed wi h he
equa ion shown in Equa ion (1). Mo eo e , o a ions and shi s we e used o conside ably
inc ease he amoun o da a. The esul ing sample dis ibu ion is summa ised in Table 2
and some samples a e p esen ed in Figu e 6.
R=ba size
64 (1)
Table 2. Reg esso -Da ase sample dis ibu ion o ound and squa ed s eel ba s.
S eel Ba Numbe o Samples
Round 5318
Squa ed 2114
(a) (b)
Figu e 6.
Samples o he Reg esso -Da ase o ound s eel ba s wi h (
a
)
R=
1.18 and (
b
)
R=
3.37.
3.3. Bina y-Da ase
The Bina y Classi ie classi ies each ba in he image. Inpu pa ches gene a ed by
a sliding window we e classi ied in o wo classes: ze os and ones. The i s one e e s o
images con aining backg ound, incomple e elemen s o join s be ween hem, as shown
in Figu e 7a, and he g oup labelled as ones con ains images wi h cen ed and comple e
elemen s, as shown in Figu e 7b.
An image o each size o he ound and squa ed s eel ba s we e conside ed o he
da ase building (12 images in o al). Ba cen es we e manually ma ked by using an image
edi o wi h a 10% b ush size o he ba size. No e ha his size de e mines how many pixels
will be conside ed as cen e. Nex , a sliding window o he s eel ba sizes pass h ough he
image wi h a s ide o 5% in o de o ex ac he equi ed pa ches. I he cen e o he pa ch
Elec onics 2021,10, 402 7 o 19
ma ches a cen e o an elemen , which is ecognisable by i s ma ke colou , he pa ch is
sa ed as one in g ayscale. On he con a y, i he pa ch cen e is no a s eel ba cen e, he
pa ch is sa ed as ze o.
(a) (b)
Figu e 7. Samples om he Bina y Classi ie Da ase o ound s eel ba s: (a) Ze os and (b) Ones.
The sample dis ibu ion o he Bina y-Da ase is p esen ed in Table 3. I is wo h men-
ioning ha he unbalanced da a shown o he ound ba s is no an issue o his ne wo k
because he da a can be chosen andomly in o de o balance bo h classes. Mo e impo -
an ly, he addi ion o new ma e ials is easily done because he wo equi ed da ase s a e
au oma ically p ocessed wi h he implemen ed algo i hms. Labelling he images is also a
simple ask jus by changing he ilename and using a simple as e g aphics edi o , such as
Mic oso Pain . This ensu es he eplicabili y o he me hodology and p o ides a as - ack
addi ion o di e en s eel ba ypes.
Table 3. Bina y-Da ase sample dis ibu ion o ound and squa ed s eel ba s.
S eel Ba Ones Samples Ze os Samples
Round 25,397 45,446
Squa ed 26,399 24,398
4. Neu al Ne wo ks Pe o mance
The p oposed ne wo ks we e e icien ly designed wi h he leas numbe o neu ons
and laye s in o de o c ea e a modula and po able a chi ec u e. The mo e con olu ion
laye s a e added, he mo e abs ac in o ma ion is ex ac ed. Howe e , i he numbe
o con olu ional laye s exceeds he one equi ed, ea u es wi h new in o ma ion a e no
c ea ed because he e is no u he in o ma ion o lea n. The e o e, i is no ecommended
o add a la ge numbe o con olu ional laye s. In 1987, Lippmann demons a ed ha
a mul ilaye pe cep on wi h wo hidden laye s is enough o o m a bi a y decision
egions [
21
]. These simple guidelines we e aken in o conside a ion o he a chi ec u es
design.
The classi ie p o ides he con olu ional laye s o he o he wo ne wo ks: he
Linea Reg esso and he Bina y Classi ie , which we e ained o each single s eel ba
ype wi h wo new da ase s. These wo ne wo ks a e smalle and equi e less da a and
aining ime. This cha ac e is ic is an ad an age i new s eel ba ypes need o be added.
Ins ead o aining he whole model, only aining o wo new mul ilaye pe cep ons is
equi ed. This me hodology o eusing he ea u e-ex ac ion pa o a ained model wi h a
pa icula goal o be used in o he model wi h di e en goal is commonly known as ans e
lea ning [
22
,
23
]. Figu e 8shows how his p ocess is ca ied ou : he classi ie con olu ional
laye s emains he same and only he small mul ilaye pe cep ons co esponding o bo h,
he Reg esso and he Bina y Classi ie , a e ained.
Elec onics 2021,10, 402 8 o 19
Inpu CNN
Fea u e
Ex ac ion
Fla en o
Op imal C op
Classi ie Class
0 / 1
Bina y
Classi ie
Linea
Reg esso
Con olu ional
Laye s
Fully
Connec ed
Laye s
Figu e 8.
Block diag am o he implemen ed Neu al Ne wo ks a chi ec u es. Fi s , he classi ie
is ained o classi y angled, ec angula , ound and squa ed s eel ba s. Then, ans e lea ning is
applied o ain he Linea Reg esso and he Bina y Classi ie o ound and squa ed ba s. No e ha
he CNN ea u e ex ac ion pa (con olu ional laye s) o he classi ie , is ozen and eused o ain
he p e iously men ioned ne wo ks.
Fo each ne wo k, he da a was di ided in o 70% o aining, 20% o alida ion
and 10% o es in o de o c oss- alida e esul s. Se e al simula ions we e ca ied ou
o de e mine he bes a chi ec u e and he hype pa ame e s o each ne wo k by using a
NVIDIA GeFo ce GTX 1050 GPU wi h Ke as [24,25].
4.1. Classi ie
The classi ie was designed and ained o classi y he ou di e en s eel ba ypes.
This ne wo k also wo ks as a ea u e ex ac o o encode , which means i educes he
in o ma ion o a la ge inpu in o a compac ( la en) ec o .
I was ained o 12 epochs wi h a ba ch size o 108 samples and i s a chi ec u e
consis s o h ee con olu ional laye s and wo ully connec ed laye s, as shown in Figu e 9.
The alida ion and es accu acies a e 99.28% and 99.35%, espec i ely, and he aining
ime was ba ely 84 s.
Inpu Con olu ional
Laye 1 Con olu ional
Laye 2 Con olu ional
Laye 3
kk k
Fla en Fully
Connec ed
Laye 1
Fully
Connec ed
Laye 2
Ou pu
RELU RELU RELU RELU So max
64 ×64
5 ×514 ×14 6 ×6
2304 ×1
12
4
Classes
Image
64
channels
64
con olu ional
il e s
64
channels
64
channels
64
con olu ional
il e s
64
con olu ional
il e s
30 ×30
s = 2 s = 2
3 ×3
s = 2
3 ×3
Figu e 9.
Classi ie a chi ec u e. Th ee con olu ional laye s and wo ully connec ed laye s, wi h 12 and 4 neu ons, a e used
o classi y he ou s eel ba ypes.
Elec onics 2021,10, 402 9 o 19
The con usion ma ix is used as pe o mance me ic o he classi ica ion ne wo k.
The class p edic ions made by he ne wo k wi h he es da a a e compa ed wi h he ac ual
class in Table 4. No e ha ound and squa ed ba s a e he classes mos likely o be con used.
Table 4. Con usion ma ix o he classi ie .
P edic ed Class
Angled Squa ed Round Rec angula
Ac ual Class
Angled 1554 1 3 2
Squa ed 1 1554 8 3
Round 0 9 1544 0
Rec angula 9 4 0 1477
4.2. Linea Reg esso
The selec ed a chi ec u e o he Linea Reg esso ne wo k consis s o one hidden
laye wi h ou neu ons and an ou pu laye wi h a single neu on, as shown in Figu e 10.
The inpu co esponds o he la en ec o gene a ed by he con olu ional encode o he
classi ie and he linea ou pu is a nume ical alue which ep esen s he esizing ac o .
The loss is calcula ed by using he Mean Squa ed E o (MAE), while he Mean Absolu e
E o (MAE) is he me ic conside ed o his ne wo k.
Fla en Hidden
Laye 1
Ou pu
Laye
Ou pu
ReLU Linea
2304 × 1
4
1
Inpu
Figu e 10. Linea Reg esso modula a chi ec u e.
A ba ch size o 256 ound and squa ed ba samples was conside ed, esul ing in a
aining ime o 80 and 100 epochs, espec i ely. Once ained, p edic ions o he alida ion
and es se s we e compu ed by using he Sciki -Lea n’s linea eg ession algo i hm [
26
],
hus ob aining bo h, he linea eg ession (
W
) and de e mina ion (
R2
) coe icien s. The bes
possible sco e o he
R2
coe icien is 1, which means ha he p edic ions a e equal o he
eal alues and is calcula ed by:
R2=1−
∑(y ue −yp ed)2
∑(y ue −y ue)2(2)
whe e
y ue
co esponds o he eal alues and
yp ed
a e he alues p edic ed by he ne wo k.
Figu e 11 shows he eal and p edic ed alues o he es se o bo h ma e ial ypes,
while Table 5summa ises hei esul ing me ics and coe icien s.
Elec onics 2021,10, 402 16 o 19
Table 11. Compa ison wi h o he implemen a ions by conside ing di e en ound ba s da ase s.
Me hod P ecision Recall F1 Accu acy (%) In e ence Time (s)
Zhang e .al. [1] 0.9360 0.8864 0.9103 94.69 0.3023
Ying e .al. [28] 0.8417 0.9617 0.8975 85.68 0.2404
Liu e .al. [11] 0.6833 0.8123 0.7420 80.99 0.0313
Fan e .al. [19] 0.9976 0.9951 0.9963 99.26 3.5862
Zhu e .al. [20] 0.9753 0.9881 0.9817 98.72 0.0306
P oposed * 0.9926 0.9888 0.9906 98.81 25
* NVIDIA GeFo ce GTX 1050 GPU.
Table 12.
Compa ison be ween he p oposed me hodology and he CNN-DC p esen ed in [
19
] by
conside ing bo h, he same compu a ional esou ces and same images.
Me hod Pa ame e s Accu acy (%) In e ence Time (s)
Fan [19] * 2,899,138 91.15 96
P oposed * 149,356 98.81 25
* NVIDIA GeFo ce GTX 1050 GPU.
6. Desk op App
A use iendly desk op app was de eloped in o de o implemen he p oposed
me hodology. Mino imp o emen s we e conside ed wi hin he app o c ea e a p ac ical
use in e ace. The possibili y o de ine he elemen size as well as he a ea o in e es
wi hin he aw image was added. The pa ch size used o he bina y ne wo k can be
de ined jus by selec ing he elemen size wi h a bounding box in he image (Figu e 14a).
While educing he a ea o in e es (Figu e 14b), he image is c opped so ha he possible
noise in he o iginal image ames can be educed.
On he o he hand, once he p ocessing is comple ed and he inal coun is ob ained,
he app shows he possible coun ing e o s highligh ed in ed colou (Figu e 14c), so he
use can easily e i y i hose e o s a e ela ed wi h clu e ing o clus e ing p oblems.
(a) (b) (c)
Figu e 14. Sc een-sho s o he app: (a) elemen selec ion, (b) a ea o in e es selec ion and (c) possible e o s highligh ed.
Figu e 15 shows he low diag am o he desk op app implemen a ion. Fi s , he app
asks o he use name and passwo d o login. The use can access he ile di ec o y and
selec s he image. The image is displayed in he densi y selec ion window whe e he use
mus choose he image densi y. A his s age, he use has he op ion o ei he selec a
bounding box o an elemen o o he s ack o pile o ba s, o bo h. Once he p ocessing is
comple ed, an ou pu window p o ides he image wi h ma ked cen es (whe e he possible
e o s also appea ), he ma e ial ype, he inal coun and he p ocessing ime, as shown in
Figu e 16.

Elec onics 2021,10, 402 17 o 19
Login Image
selec ion Densi y
selec ion Image
analysis
Final coun and
image wi h
ma ked cen e s
A ea and
elemen
c op
Figu e 15. Flow diag am o he de eloped desk op app whe e he g ey s ep is op ional.
Figu e 16.
Desk op app ou pu window. The ex on op indica es he ba ype and he inal coun
while he p ocessing ime is shown a he bo om. Red ma ke s a e displayed due o ma e ial
clu e ing.
7. Conclusions
Ba coun ing is a ime-consuming and edious ask o he wo ke s in he s eel wa e-
house and i becomes mo e complica ed due o he di e en sizes o each ba ype. Con olu-
ional neu al ne wo ks ha e become sui able o his ask because hey a e able o pe o m
as ea u e ex ac o s, jus exploi ing hei inhe en capabili y o lea n abs ac concep s and
de ec ing di e en shape objec s. The e o e, he p oposed wo k is a machine lea ning based
me hodology capable o iden i y he ba ype and coun he numbe o elemen s om an
image, which has been used in a eal s eel wa ehouse wi h high le el o use sa is ac ion.
The addi ion o challenging s eel ba s shapes, such as angled and ec angula ba s, has been
conside ed o u u e wo k. Collec ing mo e images, gene al imp o emen s and in e ence
ime educ ion ha e been also aken in o accoun . Mo e impo an ly, he implemen a ion
o he cu en ne wo ks in an embedded sys em is a high p io i y goal.
The SA-CNN-DC me hodology shows a modula and po able design implemen ed
by h ee mul ilaye ne wo ks, p ocessing he esponses o a common con olu ional encode
and a clus e ing echnique. A compac design app oach was adop ed wi h a minimum
numbe o pa ame e s in o de o educe he compu a ional esou ces bu wi hou comp o-
mising i s pe o mance and accu acy.
Tes simula ions we e ca ied ou in o de o alida e i s p ope pe o mance, so ha
di e en wa ehouses images we e conside ed o e i y he gene alisa ion capabili y. Simu-
la ion esul s showed a good pe o mance by conside ing ound and squa ed ba s coun ing,
Elec onics 2021,10, 402 18 o 19
wi h an accu acy o 98.81% and 98.57% espec i ely. Compa ed wi h he implemen a ions
ound in li e a u e, he p oposed me hodology is capable o achie ing compe i i e esul s
wi h he minimum compu a ional esou ces. Mo eo e , i s modula design allows he
addi ion o new ba ypes in a simple way, hus ul illing he eal wa ehouses expec a ions.
The usage o he desk op app in he s eel wa ehouse has d as ically educed he
coun ing ime o ound and squa ed ba s. Mo eo e , he e is a highe con idence in he
in en o y due o he low e o a e and he possible e o s ma ks. Pain ing ma e ials o
manual coun ing is no longe necessa y which esul s in a signi ican educ ion o esou ces
and en i onmen al damage. Finally, p oduc i i y is imp o ed since employees can ocus
on o he ac i i ies, while a oiding he exhaus i e ac i i y o coun ing a la ge amoun o
elemen s in a hos ile en i onmen .
Au ho Con ibu ions:
Concep ualisa ion, A.C.H.-R. and J.D.B.-P.; Me hodology, A.C.H.-R. and
J.D.B.-P.; So wa e, A.C.H.-R.; Valida ion, A.C.H.-R.; Fo mal analysis, A.C.H.-R. and J.D.B.-P.; In-
es iga ion, A.C.H.-R. and J.D.B.-P.; Resou ces, A.C.H.-R. and J.D.B.-P.; Da a Cu a ion, A.C.H.-R.;
W i ing—o iginal d a p epa a ion, A.C.H.-R., J.D.B.-P. and J.A.M.-N.; W i ing— e iew and edi ing,
A.C.H.-R., J.D.B.-P. and J.A.M.-N.; Visualisa ion, A.C.H.-R., J.D.B.-P. and J.A.M.-N.; and Supe ision,
A.C.H.-R. and J.D.B.-P.; Funding Acquisi ion, A.C.H.-R. and J.D.B.-P. All au ho s ha e ead and
ag eed o he published e sion o he manusc ip .
Funding:
This esea ch wo k has been pa ially suppo ed by he Na ional Council o Science and
Technology (CONACYT) by he M.Sc. G an wi h CVU No. 855821.
Acknowledgmen s:
The au ho s would like o hank DIACO and TEX s eel wa ehouses o hei
suppo du ing his esea ch, especially o he employees who helped ake pho og aphs when needed.
Con lic s o In e es :
The au ho s decla e no con lic o in e es . The unde s had no ole in he design
o he s udy; in he collec ion, analyses, o in e p e a ion o da a; in he w i ing o he manusc ip ,
o in he decision o publish he esul s.
Re e ences
1.
Zhang, D.; Xie, Z.; Wang, C. Ba sec ion image enhancemen and posi ioning me hod in on-line s eel ba coun ing and au oma ic
sepa a ing sys em. In P oceedings o he 2008 Cong ess on Image and Signal P ocessing, Sanya, China, 27–30 May 2008;
pp. 319–323. [C ossRe ]
2.
Zhao, J.; Xia, X.; Wang, H.; Kong, S. Design o eal- ime s eel ba s ecogni ion sys em based on machine ision. In P oceedings
o he 2016 8 h In e na ional Con e ence on In elligen Human-Machine Sys ems and Cybe ne ics (IHMSC), Hangzhou, China,
27–28 Augus 2016; pp. 505–509. [C ossRe ]
3. O su, N. A h eshold selec ion me hod om g ay-le el his og ams. IEEE T ans. Sys . Man Cybe n. 1979,9, 62–66. [C ossRe ]
4.
Wu, Y.; Zhou, X.; Zhang, Y. S eel ba s coun ing and spli ing me hod based on machine ision. In P oceedings o he 2015 IEEE
In e na ional Con e ence on Cybe Technology in Au oma ion, Con ol, and In elligen Sys ems (CYBER), Shenyang, China,
8–12 June 2015; pp. 420–425. [C ossRe ]
5.
Wang, J.; Hao, C.; Xiaoqing, X. Pa e n ecogni ion o coun ing o bounded ba s eel. In P oceedings o he Fou h In e na ional
Con e ence on he Applica ions o Digi al In o ma ion and Web Technologies (ICADIWT 2011), S e ens Poin , WI, USA,
4–6 Augus 2011; pp. 173–176. [C ossRe ]
6.
Su, Z.; Fang, K.; Peng, Z.; Feng, Z. Reba au oma ically coun ing on he p oduc line. In P oceedings o he 2010 IEEE In e na ional
Con e ence on P og ess in In o ma ics and Compu ing, Shanghai, China, 10–12 Decembe 2010; pp. 756–760. [C ossRe ]
7.
Nie, Z.; Hung, M.-H.; Huang, J. A no el algo i hm o eba coun ing on con eyo bel based on machine ision. J.
In . Hiding Mul imed. Sign. P ocess 2016,7, 425–437.
8.
Youlian, Z. Resea ch o image ecogni ion a i hme ic in au oma ic coun ing sys em o s eel ba s. J. Wuhan Eng. Ins .
2008
,20,
31–34.
9.
Chi, L. A coun ing me hod o bundled s eel ba s based on image p ocessing. J. Shenyang Uni . Technol.
2016
,38, 551–554.
[C ossRe ]
10.
Dahou, Z.; Sba aï, Z.M.; Cas el, A.; Ghoma i, F. A i icial neu al ne wo k model o s eel–conc e e bond p edic ion. Eng. S uc .
2009,31, 1724–1733. [C ossRe ]
11.
Xiaohu, L.; Jineng, O. Resea ch on s eel ba de ec ion and coun ing me hod based on con ou s. In P oceedings o he 2018
In e na ional Con e ence on Elec onics Technology (ICET), Chengdu, China, 23–27 May 2018; pp. 294–297. [C ossRe ]
12.
Cohen, J.P.; Bouche , G.; Glas onbu y, C.A.; Lo, H.Z.; Bengio, Y. Coun -cep ion: Coun ing by Fully Con olu ional Redundan
Coun ing. In P oceedings o he 2017 IEEE In e na ional Con e ence on Compu e Vision Wo kshops (ICCVW), Venice, I aly,
22–29 Oc obe 2017; pp. 18–26. [C ossRe ]
Elec onics 2021,10, 402 19 o 19
13.
Rahnemoon a , M.; Sheppa d, C. Deep Coun : F ui Coun ing Based on Deep Simula ed Lea ning. Senso s
2017
,17, 905.
[C ossRe ] [PubMed]
14.
Sam, D.B.; Su ya, S.; Babu, R.V. Swi ching Con olu ional Neu al Ne wo k o C owd Coun ing. In P oceedings o he 2017 IEEE
Con e ence on Compu e Vision and Pa e n Recogni ion (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 4031–4039. [C ossRe ]
15.
Zeng, L.; Xu, X.; Cai, B.; Qiu, S.; Zhang, T. Mul i-scale con olu ional neu al ne wo ks o c owd coun ing. In P oceedings o he
2017 IEEE In e na ional Con e ence on Image P ocessing (ICIP), Beijing, China, 17–20 Sep embe 2017; pp. 465–469. [C ossRe ]
16.
Seguí, S.; Pujol, O.; Vi ià, J. Lea ning o coun wi h deep objec ea u es. In P oceedings o he 2015 IEEE Con e ence on Compu e
Vision and Pa e n Recogni ion Wo kshops (CVPRW), Bos on, MA, USA, 7–12 June 2015; pp. 90–96. [C ossRe ]
17.
Li, X.; Jia, X.; Wang, Y.; Yang, S.; Zhao, H.; Lee, J. Indus ial Remaining Use ul Li e P edic ion by Pa ial Obse a ion Using Deep
Lea ning Wi h Supe ised A en ion. IEEE/ASME T ans. Mecha on. 2020,25, 2241–2251. [C ossRe ]
18.
Li, X.; Jia, X.; Yang, Q.; Lee, J. Quali y analysis in me al addi i e manu ac u ing wi h deep lea ning. J. In ell. Manu .
2020
,31,
2003–2017. [C ossRe ]
19.
Fan, Z.; Lu, J.; Qiu, B.; Jiang, T.; An, K.; Joseph aj, A.N.; Wei, C. Au oma ed s eel ba coun ing and cen e localiza ion wi h
con olu ional neu al ne wo ks. a Xi 2019, a Xi :1906.00891.
20.
Zhu, Y.; Tang, C.; Liu, H.; Huang, P. End-Face Localiza ion and Segmen a ion o S eel Ba Based on Con olu ion Neu al Ne wo k.
IEEE Access 2020,8, 74679–74690. [C ossRe ]
21. Lippmann, R. An in oduc ion o compu ing wi h neu al ne s. IEEE ASSP Mag. 1987,4, 4–22. [C ossRe ]
22. Weiss, K.; Khoshgo aa , T.M.; Wang, D. A su ey o ans e lea ning. J. Big Da a 2016,3, 9. [C ossRe ]
23.
Alom, M.Z.; Taha, T.M.; Yakopcic, C.; Wes be g, S.; Sidike, P.; Nas in, M.S.; Hasan, M.; Van Essen, B.C.; Awwal, A.A.S.; Asa i, V.K.
A S a e-o - he-A Su ey on Deep Lea ning Theo y and A chi ec u es. Elec onics 2019,8, 292. [C ossRe ]
24.
Bell, N.; Ga land, M. E icien Spa se Ma ix-Vec o Mul iplica ion on CUDA; NVIDIA Technical Repo NVR-2008-004; NVIDIA
Co po a ion: San a Cla a, CA, USA, Decembe 2008.
25. Ke as: Simple. Flexible. Powe ul. A ailable online: h ps://ke as.io (accessed on 12 Oc obe 2020).
26.
Ped egosa, F.; Va oquaux, G.; G am o , A.; Michel, V.; Thi ion, B.; G isel, O.; Blondel, M.; P e enho e , P.; Weiss, R.; Dubou g, V.;
e al. Sciki -lea n: Machine lea ning in Py hon. J. Mach. Lea n. Res. 2011,12, 2825–2830.
27.
Sa ang Na khede. Unde s anding AUC-ROC Cu e. 2018. A ailable online: h ps:// owa dsda ascience.com/unde s anding-
auc- oc-cu e-68b2303cc9c5 (accessed on 12 Oc obe 2020).
28.
Ying, X.; Wei, X.; Pei-xin, Y.; Qing-da, H.; Chang-hai, C. Resea ch on an Au oma ic Coun ing Me hod o S eel Ba s’ Image.
In P oceedings o he 2010 In e na ional Con e ence on Elec ical and Con ol Enginee ing, Wuhan, China, 25–27 June 2010;
pp. 1644–1647. [C ossRe ]