A novel urban mobility classification approach based on convolutional neural networks and mobility-to-image encoding
Abstract
Over the last few decades, the classification and prediction of mobility trajectories in dynamic networks have become major research topics. Switching of mobility areas (hand-over) in modern cellular networks is frequent due to restricted coverage area and node speeds (urban, highway, etc.). Accurate management of hand-over events is highly desirable to improve the system’s quality of service. We have exploited the high accuracy of machine learning to classify user mobility from mobility traces which we encoded into images. The method delivers high performance in mobility classification/prediction (exceeding 95 ) and avoids the need to study and implement a dedicated neural network structure. The technique requires the conversion of mobility traces into image structures and the subsequent application of a convolutional neural network. We propose a novel approach to classifying mobility that involves data-to-image encoding and machine learning for image classification. Numerous simulations were performed to demonstrate the benefits of the proposed technique and to illustrate the variance in the accuracy of the functions of many encoding/classification parameters. The work represents a first preliminary step towards a new mobility prediction approach. We demonstrate that it is possible to achieve a very high level of prediction accuracy with low computational complexity, exploiting the strength of neural networks in image recognition.
Full text
A no el u ban mobili y classi ica ion app oach based on con olu ional
neu al ne wo ks and mobili y- o-image encoding
Peppino Fazio
a,b,
⇑
, Mi alem Mehic
b,c
, Mi osla Voznak
b
a
DSMN, Ca’ Fosca i Uni e si y o Venice, Via To ino 155, 30172 Mes e, VE, I aly
b
VSB – Technical Uni e si y o Os a a, 17. lis opadu 2172/15, 70800 Os a a, Czechia
c
Depa men o Telecommunica ions, Facul y o Elec ical Enginee ing, Uni e si y o Sa aje o, Zmaja od Bosne bb, 71000 Sa aje o, Bosnia and He zego ina
a icle in o
A icle his o y:
Recei ed 21 No embe 2022
Re ised 22 Ma ch 2023
Accep ed 13 Ap il 2023
A ailable online 27 Ap il 2023
Keywo ds:
Con olu ional neu al ne wo ks
Da a-2-image con e sion
Machine lea ning
Mobili y classi ica ion
Pa e n p edic ion
abs ac
O e he las ew decades, he classi ica ion and p edic ion o mobili y ajec o ies in dynamic ne wo ks
ha e become majo esea ch opics. Swi ching o mobili y a eas (hand-o e ) in mode n cellula ne wo ks
is equen due o es ic ed co e age a ea and node speeds (u ban, highway, e c.). Accu a e managemen
o hand-o e e en s is highly desi able o imp o e he sys em’s quali y o se ice. We ha e exploi ed he
high accu acy o machine lea ning o classi y use mobili y om mobili y aces which we encoded in o
images. The me hod deli e s high pe o mance in mobili y classi ica ion/p edic ion (exceeding 95%) and
a oids he need o s udy and implemen a dedica ed neu al ne wo k s uc u e. The echnique equi es
he con e sion o mobili y aces in o image s uc u es and he subsequen applica ion o a con olu ional
neu al ne wo k. We p opose a no el app oach o classi ying mobili y ha in ol es da a- o-image encod-
ing and machine lea ning o image classi ica ion. Nume ous simula ions we e pe o med o demons a e
he bene i s o he p oposed echnique and o illus a e he a iance in he accu acy o he unc ions o
many encoding/classi ica ion pa ame e s. The wo k ep esen s a i s p elimina y s ep owa ds a new
mobili y p edic ion app oach. We demons a e ha i is possible o achie e a e y high le el o p edic ion
accu acy wi h low compu a ional complexi y, exploi ing he s eng h o neu al ne wo ks in image
ecogni ion.
Ó2023 The Au ho (s). Published by Else ie B.V. on behal o King Saud Uni e si y. This is an open access
a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
1. In oduc ion
The ad en o 5G echnology and ela ed s udies conce ning i s
po en ial la ency and bandwid h pe o mance (Liu e al., 2020)in
mobile ne wo ks ha e demons a ed ha i is possible o a ain a
good le el o Quali y o Se ice (QoS), especially i p edic i e
app oaches a e in eg a ed in o he sys em (Fazio e al., 2017;
Fazio e al., 2023) a e co ec ly modeling a ic lows (K ome
e al., 2020; K ome e al., 2018; De Rango e al., 2005). Machine
and deep lea ning applica ions (Ma in e al., 2021; Singh e al.,
2021; Xu e al., 2021) ha e also s eeply p oli e a ed in he las
ew yea s and added eno mous alue o he so wa e which can
exploi hem. In he cu en wo k, we especially highligh he pos-
sibili y o using well-known Con olu ional Neu al Ne wo ks
(CNNs) (Khan e al., 2018) o classi y mobili y a e ansposing
he mobili y da a in o sui able mobili y images (ins ead o eal-
wo ld images). Ou wo k does no p opose a new neu al ne wo k
laye scheme o image classi ica ion me hod. S ill, we demons a e
he s eng h o CNNs (Luo e al., 2018) in hei accu acy and how
hey can be applied o classi ying/p edic ing mobili y. Neu al-
based image classi ica ion algo i hms a e known o achie e an
accu acy o 95–98 %whe eas mobili y classi ica ion o p edic ion
schemes such as hose desc ibed in Jin e al. (2001), Gaiduchenko
and G i syk (2019),Fazio e al. (2017), Zhang e al. (2018) can
ob ain, o he bes o ou knowledge, a maximum accu acy o abou
85–87 %, which is well below 90 %.Fig. 1 shows a gene ic e e -
ence scena io: mobile hos s a e ee o mo e in any geog aphical
a ea, each one co e ed by a 5G mic o/ em o-cell. Fo example,
o he black pa h, a mobili y p edic i e app oach can be in eg a ed
wi h 5G a chi ec u e, in o de o ese e esou ces (bandwid h
channels) in-ad ance, a oiding se ice dis up ions o he mobile
hos (da ke cells). The encoding can be made locally, on- he- ly,
h ps://doi.o g/10.1016/j.jksuci.2023.101561
1319-1578/Ó2023 The Au ho (s). Published by Else ie B.V. on behal o King Saud Uni e si y.
This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/).
⇑
Co esponding au ho .
E-mail add esses: [email p o ec ed] (P. Fazio), [email p o ec ed]
(M. Mehic), [email p o ec ed] (M. Voznak).
Pee e iew unde esponsibili y o King Saud Uni e si y.
P oduc ion and hos ing by Else ie
Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
Con en s lis s a ailable a ScienceDi ec
Jou nal o King Saud Uni e si y –
Compu e and In o ma ion Sciences
jou nal homepage: www.sciencedi ec .com
o emo ely, by a dedica ed se e , while he emo e CNN ecei es
he images and he ela ed belonging cell, ha ing he possibili y o
be ained and, hen, alida ed. S a ing om his obse a ion, a
new me hod o con e ing mobili y-da a in o image-da a is p o-
posed; in pa icula , he main con ibu ions o he cu en wo k
can be summa ised as:
A p oposed inno a i e me hod o encoding mobili y da a in o
images, aking in o accoun he en opy me ic (F ank and
F ank, 2020), o assis he CNN in eaching a be e alida ion
accu acy;
No need o he design and implemen a ion o a new neu al ne -
wo k: he mos s aigh o wa d CNN s uc u es a e aken in o
accoun (we do no design a new laye ing s uc u e, we demon-
s a e how i can pe o m in i s simples con igu a ion);
E alua ion o CNN pe o mance in he unc ion o encoding
pa ame e s such as sampling equency, encoding algo i hm,
image size, e c.;
Base s a ion capabili y o ecognizing he geog aphical a ea
whe e a mobile hos is mo ing wi hou he aid o GPS o o he
posi ioning de ices. Mobile nodes and base s a ions can apply
Angle-o -A i al (AoA), P oximi y, T ila e a ion, o Time Di e -
ence o A i al (TDoA) me hods. Ou encoding scheme unc ions
wi h no malized mobili y da a so he ained CNN ecognizes
he mobili y egion wi hou accu a e GPS da a.
The emainde o he pape is s uc u ed as ollows: Sec ion 2
in oduces some ecen wo ks which examine mobili y classi ica-
ion and p edic ion; Sec ion 3desc ibes he main p oposed algo-
i hm, speci ying su icien de ail ega ding he encoding
app oach; Sec ion 4desc ibes in de ail he main esul s ob ained
h ough he p oposed app oach; Sec ion 5concludes he pape ,
ema king on he main ad an ages o he p oposed algo i hm.
2. Rela ed wo k
This sec ion e iews he main li e a u e con ibu ions conce n-
ing mobili y classi ica ion, p edic ion, and image ecogni ion. We
examined he exis ing li e a u e and ound ha no s udies ha e
in eg a ed hese ea u es and exploi ed he s eng h o image-
classi ica ion CNNs o sugges ed me hods o encoding mobili y
aces in images. We ho oughly e iew he mos signi ican wo ks
ela ed o ou p oposal. In pa icula , ehicle ajec o y is consid-
e ed as a ime se ies (coo dina es, speeds, and ehicle pa ame e s
a e ime-dependen ). The g ea pe o mance o CNNs (Ka im e al.,
2019) is exploi ed o mul i a ia e ime se ies classi ica ion: he
au ho s in oduced mul iple con olu ional laye s o ea u es
ex ac ion, showing he classi ica ion accu acy o he p oposed
me hod in he unc ion o he da ase noise and ob aining an accu-
acy ha anges om 30%and 100%. The au ho s o Asad e al.
(2020) ocused on a ele s p o iling in ain s a ions o analyze
how humans beha e acco ding o hei age ( wo age classes a e
conside ed: 16–59 and 60-and-o e ) du ing he Co id-19 disease
pandemic. In pa icula , he au ho s employed six di e en classi-
ie s (Logis ic Reg ession, Mul i-laye Pe cep on, Suppo Vec o
Machine, Random Fo es , K-nea es Neighbou , and Decision T ee)
o es ablish au oma ion in in elligen decisions while lea ning
om his o y and adap ing o he es ing en i onmen . Gi en he
desc ibed da ase (London Unde g ound and O e g ound - LUO),
he p oposed idea can moni o po en ial con ac s/p oximi y a el-
e s and ad ise hem o sa egua d ulne able age-g oup a ele s.
Simula ion esul s eached an accu acy o abou 82%and 86% o
he wo age classes.
The impo ance o mobili y p edic ion is s a ed by se e al li e -
a u e con ibu ions, such as he ones in Fazio e al. (2017), Zhang
e al. (2018), whe e he au ho s su ey he main con ibu ions, also
in e ms o di e en me hodologies and app oaches, in he wo ld
o mobili y p edic ion. I has been s udied o decades, and he e
is a wide a ie y o p edic i e models (Kalman il e s, Ma ko
chains, Neu al Ne wo ks, Au o Reg essi e, e c.). In he men ioned
wo ks, he au ho s compa e he di e en app oaches, showing
he po en iali ies o each p edic o . I is also unde lined ha he
eached accu acy is gene ally below 90%. The wo k in Wang
e al. (2021) p oposes an inno a i e a en ional Ma ko model,
conside ing he long- e m co ela ion wi h his o ical ajec o ies
and con ex in o ma ion and p edic ing u u e hos posi ions. The
au ho s conside ed an ex ensi e da ase (mo e han 20000 use s),
ob aining an imp o emen o he classical machine lea ning
app oaches (such as he Hidden Ma ko Model - HMM, Recu en
Neu al Ne wo k - RNN, F iendship, and Mobili y - MF, e c.), wi h a
as e execu ion ime.
Fo many yea s, machine lea ning and neu al ne wo ks ha e
been widely used in image ecogni ion. The con olu ional laye
o CNNs can disco e some hidden ea u es o he inpu images
while he subsequen laye s p oduce he co ec ou pu . The wo k
in Chen e al. (2019) ex ends he well-known LeCun’s CNN (Lecun
e al., 1998), adding con olu ional and pooling laye s and p opos-
ing a Mul i-Con olu ion NN (MCNN). The au ho s es ed he new
CNN on he Ca s s. Dogs (Dogs s Ca s da ase , 2023), Ci a -10
(K izhe sky, 2023), and Fe 2013 (Facial Exp ession Recogni ion
da ase , 2013) da ase s, showing ha he p oposed MCNN ou pe -
o ms he classical one in e ms o complex- ex u e ea u es ecog-
ni ion. The au ho s o Tiwa i e al. (2020) p oposed he Visual
Geome y G oup 16 (VGG16) model o classi y images in o wo
addi ional ca ego ies ins ead o pe o ming ea u e ex ac ion o
segmen a ion. VGG16 o e s an accu acy o 99%, and images a e
u he ca ego ized in o addi ional sub-ca ego ies. The pape (Xu
e al., 2020) is ela ed o he comp essed-domain image classi ica-
ion: images a e conside ed be o e encoding in he desi ed o ma
( he econs uc ion s ep is bypassed), and he aining is made
wi h a dynamic Measu emen Ra e (MR) by selec ing he needed
MR wi h he help o a sensing ma ix. The au ho s es ed hei p o-
posal on a massi e se o da ase s (e.g. Ci a -10 (K izhe sky, 2023),
and Coil-100 (Nene e al., 2023)), and he pe o mance has been
e y sa is ac o y, also in e ms o noise obus ness.
Taking in o accoun he main esul s discussed abo e, we will
illus a e, in he nex sec ion, ou p oposal, o design a new mobil-
i y classi ica ion algo i hm based on CNNs and mobili y- o-image
encoding.
3. Mobili y classi ica ion h ough CNNs imaging
This sec ion is dedica ed o ou main p oposal. Fi s o all, we
will in oduce he main issue. Then we will gi e an in-dep h o e -
iew o mobili y encoding and he ype o used CNN. We a e no
p oposing a new CNN s uc u e bu a new app oach o mobili y
classi ica ion, which can exploi he high accu acy o CNNs in
image classi ica ion. In addi ion, we a e no conside ing any pa ic-
ula kind o node (a ehicle, a human, a mobile senso , an
Unmanned Ae ial Vehicle, e c.), so he app oach is comple ely gen-
e al (simula ion esul s will be specialized o a pa icula case).
Fig. 2 illus a es he main s eps o ou p oposed idea h ough
Da a-Flow Diag am s uc u es. As shown in igu e (a), mobili y
aces a e gene a ed o all he conside ed geog aphical a eas
based on eal maps and eal a ic beha io s; hen, we p opose
wo possible algo i hms o con e ing he c ea ed da a in o ma i-
ces and, in he end, in o images. In his way, each pa h o a mobile
use is s o ed as a se o images, so he use s mo ing in o he same
a ea c ea e images belonging o he same class. Then (b), a classical
CNN is ained based on he p e iously gene a ed se o images; we
ca ied ou se e al simula ions o a oid o e - i ing and ind he
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
2
bes ade-o be ween he subse o he images o aining and
he subse o he images o alida ion. Once he CNN has been
ained (c), each new mobile ace can be classi ied adequa ely, gi -
ing he CNN a sho po ion o he ace as inpu . All he implemen-
a ion de ails a e gi en in he ollowing subsec ions and as
simula ion esul s.
3.1. The gene al model o mobili y classi ica ion
We a e conside ing a gene al se o Mobili y A eas (e.g., a squa e
a ea on he Ea h’s su ace, a po ion o he sky, a olume in he
sea/ocean, e c.) MAs ¼ma
1
;ma
2
;...;ma
n
g
, wi h jMAsj¼n. Each
ma
i
2MAs, can be adjacen o ano he ma
j
2MAs, wi h i–jand
i;j¼1;...;n, o i can be loca ed in a comple ely di e en pa o
he Ea h/sky.
Le us indica e wi h V he se o mobile nodes which a e mo ing
in o MAs: we assume ha V¼
1
;...;
m
g
and jVj¼m. So, ou
sys em has nmobili y a eas and mmobile nodes. Each
k
2V
mo es in o only one ma
i
2MAs, so we can use he no a ion
k;i
o indica e he k h mobile node is mo ing in ma
i
2MAs.
Unde hese assump ions, we canno ha e
k;i
and
k;j
a he same
ime.
Wi hou loss o gene ali y, we assume ha each mobile node
mo es in a 3D en i onmen , so each ma
i
2MAs is cha ac e ized
by a cen e c
i
¼cx
i
;cy
i
;cz
i
ðÞand an ex ension adius
i
, ha is
ma
i
¼c
i
;
i
ðÞ, wi h cx
i
;cy
i
;cz
i
2R. So, each ma
i
2MAs can be ep e-
sen ed as a sphe e (o a ci cle in 2D mobili y). Ou model is alid i
a cube o a squa e is conside ed.
Each node
k;i
, by i s mobili y in ma
i
, c ea es a pa e n
P
k
¼
x
k
;
y
k
;
z
k
, whe e ¼1;2;...is he disc e e ime index,
wi h
l
l1
ðÞ¼T, he sampling pe iod ( he pe iod a which
mobili y posi ion is sampled and s o ed). We can e e o P
k
also
by he no a ion P
k
¼
x
1
k
;
y
1
k
;
z
1
k
;
x
2
k
;
y
2
k
;
z
2
k
;...;
x
k
;
y
k
;
z
k
;...g, gi en ha i is composed by a sequence o i-
ple s (in he gene al case o 3D space). An o de ed subse o P
k
;p
0
k
,
o leng h pis de ined as p
0
k
#P
k
:
Fig. 1. An example o 5G cellula co e age and use s ajec o ies encoded in o images, able o ain a emo e se e and i s CNN.
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
3
p
0
k
¼x
1
;y
1
;z
1
ðÞ;x
2
;y
2
;z
2
ðÞ;...;x
p
;y
p
;z
p
i 9
0
j...
x
1
;y
1
;z
1
ðÞ
¼
x
0
k
;
y
0
k
;
z
0
k
;...
x
2
;y
2
;z
2
ðÞ¼
x
0
þ1
k
;
y
0
þ1
k
;
z
0
þ1
k
;...
x
p
;y
p
;z
p
¼
x
0
þp
k
;
y
0
þp
k
;
z
0
þp
k
:
ð1Þ
So, unde he assump ions abo e, i we e e o
k;i
, i means ha :
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
cx
i
x
k
2
þcy
i
y
k
2
þcz
i
z
k
2
hi
6
i
8 2N:ð2Þ
Eq. 2indica es ha each poin o he mobili y pa e n o
k
is
bounded o he ex ension a ea (o olume) o ma
i
.
The goal o ou en i e a icle is: gi en an o de ed subse p
0
k
(ex ac ed om P
k
), a CNN should classi y i by e u ning he
ma
i
in which node
k
is mo ing.
To each his aim, i s o all, we p opose a mobili y- o-image
encoding scheme, and hen, he CNNs a e ained and alida ed
by he ob ained encoded da ase s.
3.2. Mobili y o ma ix encoding: MME, an al e na i e o aw da a
ep esen a ion
Based on he de ini ions abo e, we will p opose a new way o
encode mobili y da a in o lossless images (basically, a ma ix is
ob ained, hen s o ed as an image h ough a p ope lossless codec).
Le us s a wi h a mobili y pa e n subse
p
0
k
¼
x
k
;
y
k
;
z
k
j 2
0
;
0
þp
½
. As ea lie de ined, i is a
se o iple s x;y;zðÞ(in he gene al case o a 3D mobili y). We
can e alua e he ela ed speed s
0
k
o node
k
du ing he ime ange
0
;
0
þp½as:
s
0
k
¼
1
T
x
0
þlþ1
k
x
0
þl
k
;
y
0
þlþ1
k
y
0
þl
k
;...
z
0
þlþ1
k
z
0
þl
k
no
;
ð3Þ
wi h l¼0;1;...;p1, and sx
0
k
¼ x
0
þlþ1
k
x
0
þl
k
;
sy
0
k
¼ y
0
þlþ1
k
y
0
þl
k
;sz
0
k
¼ z
0
þlþ1
k
z
0
þl
k
. The leng h o s
0
k
will be p1.
We can also e alua e he accele a ion/decele a ion a
0
k
o node
k
du ing he ime ange
0
;
0
þp½as:
a
0
k
¼
1
T
sx
0
þqþ1
k
sx
0
þq
k
;sy
0
þqþ1
k
sy
0
þq
k
;...sz
0
þqþ1
k
sz
0
þq
k
no
;ð4Þ
wi h q¼0;1;...;p2, and ax
0
k
¼sx
0
þqþ1
k
sx
0
þq
k
;ay
0
k
¼
sy
0
þqþ1
k
sy
0
þq
k
Þ;az
0
k
¼sz
0
þqþ1
k
sz
0
þq
k
. The leng h o a
0
k
will be
p2.
In he ollowing, we will use he sho e no a ions o p
0
k
;s
0
k
and
a
0
k
. In he case o Raw Mobili y Da a (RMD) ep esen a ion, he
in o ma ion is con e ed om a ec o o a ma ix o ma . Th ee
ma ices a e ob ained o p
0
k
;s
0
k
and a
0
k
. Wi hou loss o gene ali y,
we assume ha , o RMD, pis a squa e alue, so i is easy o de ine
¼ffiffiffi
p
p. Fo speed and accele a ion ec o s, he missing elemen s (1
and 2, espec i ely) can be padded du ing he ma ix cons uc ion
(we assume ha , o example, he alues a e eplaced by ze os).
Ma ix dimensions a e x3 , because columns a e dedica ed o
each coo dina e.
Fig. 2. The Da a-Flow Diag ams o he h ee s eps p o ided in ou p oposal.
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
4
Algo i hm 1 ep esen s jus a da a-s uc u e ans o ma ion
wi h a compu a ional complexi y h
2
. I is applied o posi ion,
speed, and accele a ion ec o s, accep ing as inpu he ec o
ec
0
k
con aining posi ion, speed, o accele a ion mobili y in o ma-
ion (speci ied in he inpu pa ame e ype) ela ed o mobile hos
k
, ha is one o he e ms in Eq. 5. Fi s ly, he algo i hm e alua es
he igh ma ix dimension . Then i execu es wo nes ed Fo
cycles o ill up he ows and columns o he ou pu ma ix M
0
k
,
which has been ini ially se o he null ma ix and no malized in
he ange 0;1½a he end. Inside he nes ed loops, he no a ion
ec
0
k
c;ind
ðÞ, wi h c= 1,2,3 e e s o he x;yand zcomponen s o
he
ec
0
k
cðÞ iple .
Le us see how he aw da a can be e ec i ely encoded while
pu ing i in o a ma ix s uc u e. We can s a o build h ee sep-
a a e ma ices om he h ee ec o s. A sp eading ac o s is
de ined, and each ma ix will be squa e: se ing ¼ps ðÞ, hen
he dimensions will be x . All he elemen s o p
0
k
;s
0
k
and a
0
k
a e
ini ially e-scaled ( s) be ween 0 and 1 ( hey can also con ain neg-
a i e alues), so we can w i e:
p
0
k
¼ s p
0
k
;s
0
k
¼ s s
0
k
;a
0
k
¼ s a
0
k
:ð5Þ
The s ope a ion di ides all he elemen s o a ec o by he maxi-
mum one i he e a e no nega i e alues; o he wise, he absolu e
alue o he lowes nega i e elemen is added o each alue be o e
selec ing he maximum and no malizing he elemen s. Then he
s uc u e o he ma ices is de ined as ollows. Fi s o all, he x
ma ices Mp
0
k
;Ms
0
k
and Ma
0
k
a e c ea ed as emp y ma ices, ha
is, each elemen is equal o ze o.
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
5
Fo all o hem, he xcomponen is encoded as a sequence o
columns om op o bo om, he ycomponen is encoded as col-
umns om he bo om o he op, and he zcomponen is encoded
as ows, om le o igh . The idea is o associa e a ow o a col-
umn o ”ones” which is p opo ional o he con en o p
0
k
;s
0
k
;a
0
k
.
We unde line ha ou p oposal is only one o he possible ways
o ill he ma ices: Howe e , by knowing which kind o esul s can
be ob ained, one can ind di e en ways o c ea e images. So, he
Algo i hm 2 speci ies how he h ee ma ices a e illed up; we illus-
a e he algo i hm o he gene ic ma ix M. Bo h RMD and MME
algo i hms a e execu ed o :
Posi ion p
0
k
:Mp
0
k
¼MME p
0
k
;s ;p;
’posi ion
0
Þ;o Mp
0
k
¼
RMD p
0
k
;
0
posi ion
0
;
Speed s
0
k
:Ms
0
k
¼MME s
0
k
;s ;p;
’speed
0
Þ;o Ms
0
k
¼RMD s
0
k
;
’speed
0
Þ;
Accele a ion a
0
k
:Ma
0
k
¼MME a
0
k
;s ;p;
’accel
0
Þ;o
Ma
0
k
¼RMD a
0
k
;
’accel
0
Þ;
Re e ing o Algo i hm 2, i accep s as inpu he no malized ec-
o
ec
0
k
con aining posi ion, speed o accele a ion mobili y in o -
ma ion (speci ied in he inpu pa ame e ype) ela ed o mobile
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
6
hos
k
, ha is one o he e ms in Eq. 5; he leng h o he subse p
and he sp eading ac o s a e also gi en as inpu . The alues o s
indica e he numbe o columns assigned o each componen (a
leas i should be se o 3).
Fi s ly, he MME e alua es he ma ix dimension as de ined
ea lie and c ea es he emp y ma ix M
0
k
. Then i compu es he
igh dimension o he inpu ec o : om he de ini ions o Eqs.
(3) and (4), we know ha o speed and accele a ion ec o s, he
leng hs a e p1 and p2 espec i ely. A his poin he algo i hm
execu es he i s cycle (h a iable), conside ing o he xcompo-
nen s he columns (index
x
)1,1+s , 1+2 s , and so on, o he y
componen s he columns (index
y
), 3, 3 + s , 3+2 s , and so on, while
o he zcomponen s he ows (index
z
)1,1+s , 1+2 s , and so on.
A his poin , each ow/column is illed wi h se e al ones, which is
p opo ional o he con en o he elemen s o
ec
0
k
c;hðÞ. Fo he z
componen , du ing he hi d in e nal cycle, i he gene ic ma ix
elemen M
0
k
index
z
; ðÞhas al eady been modi ied by he wo p e i-
ous cycles ( o he xand ycomponen s), no u he ac ions a e
made.
Recalling ha he elemen s o
ec
0
k
a e no malized om 0 o 1,
hen he e m b
ec
0
k
c;hðÞ ewill be bounded o he maximum size
o M
0
k
. The symbol be indica es he ounding ope a ion ( he nea es
in ege is chosen). Compu a ional complexi y is accep able
because he ini ial ope a ions ha e cons an complexi y (negligi-
ble), he p ima y cycle is execu ed OpðÞ imes. In con as , he
in e nal cycles a e execu ed O
ðÞ imes, so we can w i e ha he
o e all compu a ional complexi y is Op3 ðÞ=Op3ps ðÞ
which is bounded by Op
3
and pdoes no depend on he leng h
o he o e all pa e n. A his poin , i is use ul o make an example
o how he ma ices a e c ea ed om eal alues wi h MME (RMD
is mo e in ui i e since i is only a con e sion be ween ec o s and
ma ices). Le pbe equal o 4, s be equal o 4 and p
0
k
be
12:735;87:675;10:22ðÞ;18:143;79:884;ð11:31Þ;25:750;71:943;ð
12:7Þ;30:564;63:345;11:8ðÞg, wi h jp
0
k
j¼p¼4. By applying
Eqs. (3) and (4) hen we will ha e s
0
k
=
5:408;7:791;1:09ðÞ;7:607;7:941;1:39ðÞ;4:814;8:598;0:9ðÞ;g,
wi h js
0
k
j¼p1¼3 and a
0
k
= 2:199;0:15;0:3
ðÞ
;
2:793;ð0:657;2:29Þg, wi h ja
0
k
j¼p2¼2. A his poin , by
he inpu alue o s , we will ha e =4 4¼16, so he h ee ma ices
will ha e dimensions o 16 x 16. The no malized ec o s will be:
p
0
k
: 0:145;1;0:116ðÞ;0:207;0:911;0:129ðÞ;0:294;0:820;ð
0:145Þ;0:349;0:722;0:134ðÞg;
s
0
k
= 0:864;0:04978;0:598ðÞÞ;1;0:040;0:616ðÞ;0:827;0;0:475ðÞg;
a
0
k
= 1;0:529;0:619ðÞ;0;0:427;0:101ðÞg.
The MME is applied o each no malized ec o a his poin , and
he ma ices illus a ed in Fig. 3 a e ob ained.
F om Fig. 3, i can be seen how he xcomponen s a e added by
he uni a y elemen s om op o bo om ( ed colo ) in he columns
indexed as in he MME (indexes 1, 5, 9, 13), he uni a y elemen s
om he bo om add he ycomponen s o he op (g een colo )
in he columns indexed as in he MME (3, 7, 11, 15) and he z
Fig. 3. An example o he h ee ma ices as ou pu o MME.
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
7
componen s a e added by he uni a y elemen s om le o igh
(da k blue colo ) in he ows indexed as in he MME (1, 5, 9, 13).
O ange alues ep esen he p esence o mo e componen s in he
same ma ix elemen s: e.g., he elemen Mp
0
k
1;1ðÞis gene a ed
by bo h he xcomponen and zcomponen .
The ollowing (and las ) s ep is o encode he ma ices in o an
image: o his aim, we associa e he Mp
0
k
ma ix o he RED image
channel, he Ms
0
k
ma ix o he GREEN image channel and he Ma
0
k
ma ix o he BLUE image channel, a e mul iplying each ma ix
by 2
8
-1 (we a e conside ing 8-bi s esolu ion pe laye ), indepen-
den ly om RMD o MME. We did no ca e abou in es iga ing
he pa icula image o ma . S ill, we ocused on a lossless encod-
ing, such as he Po able Ne wo k G aphics (PNG) (The Po able
Ne wo k G aphics speci ica ion, 2023), o a oid he c ea ion o a i-
ac s (JPG o ma , o example, is no sui able o ou app oach).
In pa icula , we chose he RGB 24bi o ma o PNG, so each
elemen in each ma ix is ep esen ed by 8 bi s (256 le els): a
de ail o he h ee ma ices o Fig. 3 in a isual ep esen a ion is
gi en in Fig. 4 (please, no e ha he images ha e been magni ied,
because he wid h o each ow/column is 1 pixel).
3.3. The image en opy: A possible e alua ion me ic
I is known om he li e a u e ha he en opy concep plays a
i al ole in image classi ica ion (Gowd a e al., 2020). The classical
me ic used o e alua e he goodness o a slicing ecogni ion and
classi ica ion is he Maximum En opy (ME): he majo i y o classi-
ica ion/ ecogni ion machine lea ning app oaches y o di ide big
images in o smalle ones, inding he slices which a e cha ac e ized
by he highes ME (i.e., he maximum le el o in o ma ion, he
mos isually ep esen a i e po ions o he image), which can be
used o aining he CNN, whose con olu ion ope a ions can dis-
co e and ex ac he hidden in o ma ion. En opy, independen ly
om Ha ley’s (Ha ley, 1928) o Shannon’s (Shannon, 1953) de i-
ni ions, ep esen s he amoun o in o ma ion con ained in he
image o be classi ied and he deg ee o andomness o he pixel
alues. I is in ended ha he mo e in o ma ion is included in
he image, he highe will be bo h he en opy and, also, he s uc-
u al complexi y o he CNN: gene ally, mo e han one con olu ion
laye is needed o ex ac he igh ea u es om da a. In ou expe -
imen s, we e alua e he colo image en opy as ollows (assuming
ha is he numbe o image pixels and 8-bi alues ep esen
each colo channel):
EIm½¼
1
3E
c
Im
RED
ðÞþE
c
Im
GREEN
ðÞþE
c
Im
BLUE
ðÞ½;ð6Þ
ha is he a e age o he en opy o each colo channel E
c
de ined
as:
E
c
Im
ch
½
¼X
2
8
1
pix
ch
¼0
pIm
ch
;pix
ch
ðÞ
log
2
pIm
ch
;pix
ch
ðÞ½
;ð7Þ
wi h:
pIm
ch
;pix
ch
ðÞ¼
coun Im
ch
;pix
ch
ðÞ
ð8Þ
wi h he unc ion coun In
ch
;pix
ch
ðÞcoun ing he numbe o imes a
pixel in Im
ch
assumes he alue pix
ch
. The ocus o ou wo k is no
ela ed o he p oposal o a new CNN s uc u e bu only o show
ha good esul s can be achie ed in mobili y classi ica ion wi h a
no el app oach. Fo hese easons, we a e p oposing a new MME
algo i hm o main ain he CNN complexi y as low as possible, so
images will be cha ac e ized by ela i ely low en opy alues.
As s a ed in he p e ious sub-sec ion, he p oposed MME is jus
one o he possible algo i hms o da a- o-image encoding. In his
wo k, we show, o he i s ime, ha his app oach is sui able
o mobili y classi ica ion. The easies way o encode mobili y in o
an image is o pu he aw da a di ec ly in o a ma ix and hen
encode i as an image. Bu his las app oach has se e al
d awbacks:
Ve y iny images will be gene a ed: e e y CNN, as shown in he
nume ical esul s sec ion, needs a minimum image size o ec-
ognize he image and classi y i ; i we do no hink abou an e i-
cien encoding scheme, a huge numbe o samples is needed.
Conside he p e ious example: we had p= 4, which means
we ha e only ou iple s (4 alues o each coo dina e compo-
nen ). I also means ha we ob ain h ee alues o speed and
wo alues o accele a ion: we ha e nine di e en da a o each
componen , o a o al o 27 ma ix elemen s. Wi h an encoding
scheme (as MME), we ha e 3
2
=768 ma ix elemen s, s a ing
om 27 da a poin s. So, a p ope ep esen a ion o he da a is
manda o y o build an adequa e aining se ;
F om an en opy poin o iew: i we ep esen he aw da a
di ec ly in o an image, he en opy alue will be high due o
he eno mous colo changes wi h high g anula i y (especially
i mobili y is ep esen ed by la and lon couples); he MME,
ins ead, as demons a ed la e , will educe he en opy sensibly.
Fo example, we show he di e ence be ween RMD and MME
encoding in e ms o en opy alues dis ibu ion. Fig. 5 shows an
example o he esul s ob ained by encoding 2D mobili y da a by
MME o jus lea ing he aw alues in o a single RGB ma ix
(RMD). RMD and MME images con ain 8192 and 7744 pixels,
espec i ely (compa able sizes). S ill, he RMD image is c ea ed
by exac ly 4096 samples (mobili y poin s, which a e doubled o
xand ycomponen s), while MME needs only 484 mobili y poin s
(wi h a gain o abou 88.1%).
We used 2500 2D mobili y pa e n subse s p ocessed wi h RMD
and MME o see an example o how he en opy is dis ibu ed. The
subse leng h phas been se o 22; Fig. 6 shows he massi e di e -
ence in e ms o en opy be ween RMD and MME encoded images:
o MME, he mean en opy is 1.22 J/K, while o RMD i is 4.26 J/K.
In he case o MME i is almos cons an (due o he p esence o
high black spaces), while o RMD i has a subs an ial oscilla o y
end. I is also in e es ing o see how he en opy is dis ibu ed:
Fig. 7 shows he pd (E) end. I con i ms wha was obse ed in
Fig. 6, a e y low de ia ion a ound he mean alue o MME, while
mo e sp ead alues o RMD. In he nume ical esul s sec ion, we
will show wha happens o En opy based on he pa ame e s
ela ed o image encoding. In he nex sec ion, ins ead, he CNN
model is deeply discussed.
Fig. 4. A isual ep esen a ion o Mp
0
k
;Ms
0
k
;Ma
0
k
( op) and he ela ed RGB laye s
(bo om).
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
8
3.4. The conside ed con olu ional neu al ne wo k
Ou wo k shows which pe o mance can o e a CNN-based
image classi ica ion model i ained on mobili y da a (p e iously
encoded in o images). So, ou aim does no ega d he p oposal
o a new CNN model bu a new way o using i in a di e en
esea ch con ex . So, in his sub-sec ion, we a e gi ing jus he
main desc ip ion o he used LeCun-like (Lecun e al., 1998) CNN
model. Using i s Machine Lea ning Toolbox, we used MATLAB
(MATLAB, 2021) o ou implemen a ion. In pa icula , we s uc-
u ed he CNN, ollowing LeCun’s p oposal, as ollows:
Inpu laye : i is designed o accep ing an image as inpu ; i is
c ea ed by de ining he size o he inpu image and he numbe
o i s laye s (3 in ou case: ed, g een, and blu); by de aul , in
MATLAB, he image inpu laye no malizes pixel alues by sub-
s ac ing hei mean alue; we can speci y pixel alues o be
no malized be ween 0 and 1;
Con olu ion laye : i applies sliding con olu ional il e s o he
2D inpu . The inpu is con ol ed by mo ing he speci ied il e s
along he 2D inpu (in ho izon al and e ical di ec ions) and
e alua ing he do p oduc o he weigh s and he inpu ;
Ba ch No maliza ion laye : i no malizes a da a ba ch ac oss
all obse a ions. I is gene ally used o speed up he aining
o he CNN, educing he sensi i i y o he beginning
unce ain y;
Rec i ied Linea Uni (ReLU) laye : i is one o he possible ac i-
a ion laye s and applies a h eshold ope a ion o i s inpu (in
simple wo ds, nega i e alues a e se o 0);
Fully Connec ed laye : i is a dense laye o neu ons and mul i-
plies he inpu by a weigh ma ix;
So Max laye : i applies a so max unc ion o he inpu . So -
max is a gene aliza ion o he logis ic unc ion, which can com-
p ess a k-leng h and a bi a y con en ec o o ano he k-
leng h ec o , wi h he sum o elemen s equal o 1;
Classi ica ion laye : i can ecei e an inpu and e u n he clas-
si ied ou pu (in ou case, he ma
i
2MAs).
Fig. 8 shows he comple e laye ing o he conside ed CNNs in
he MATLAB window, wi h he main used pa ame e s. Resul s
can be, o cou se, enhanced. S ill, in his pape , we a e in e es ed
nei he in in es iga ing he s uc u e o he CNN no in inc easing
i s complexi y (on he con a y, one o ou goals is o main ain he
CNN s uc u e as simple as possible).
Fo mo e de ails abou he possible CNN op imiza ion, please
e e o Bishop (2006).
Fig. 7. P obabili y Densi y Func ion o he en opy alues o Fig. 7.
Fig. 6. En opy end o RMD and MME images, wi h p= 22 and T= 1s.
Fig. 5. T ue images ob ained by using Raw Mobili y Da a (RMD), size 64 128, o
he MME algo i hm, size 88 88.
Fig. 8. The laye ing s uc u e o he simple CNN used in ou wo k.
P. Fazio, M. Mehic and M. Voznak Jou nal o King Saud Uni e si y – Compu e and In o ma ion Sciences 35 (2023) 101561
9