INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
Linking Bi s eam In o ma ion o QoE: A S udy on
S ill Images Using HEVC In a Coding
Tomas MIZDOS1, Ma cus BARKOWSKY 2, Mi osla UHRINA1, Pe e POCTA1
1Depa men o Mul imedia and In o ma ion-Communica ion Technologies, Facul y o Elec ical Enginee ing
and In o ma ion Technology, Uni e si y o Zilina, Uni e zi na 1, 010 26 Zilina, Slo ak Republic
2Depa men o In e ac i e Sys ems and In e ne o Things, Facul y o Applied Compu e Science,
Deggendo Ins i u e o Technology, Die e -Go li z-Pla z 1, 94469 Deggendo , Ge many
[email p o ec ed], ma cus.ba ko[email p o ec ed], mi osla[email p o ec ed],
pe e .po[email p o ec ed]
DOI: 10.15598/aeee. 17i4.3625
Abs ac . The coding ools used in image and ideo
encode s aim a high pe cep ual quali y o low bi-
a es. Analyzing he esul s o he encode s in e ms
o quan iza ion pa ame e , image pa i ioning, p edic-
ion modes o esiduals may p o ide impo an insigh
in o he link be ween hose ools and he human pe cep-
ion. As a i s s ep, his con ibu ion analyzes he pos-
sibili y o anscode e e ence images o h ee well-
known image da abases, i.e. IRCCyN/IVC, LIVE and
TID2013, om hei o iginal, olde o ma s o HEVC;
hus c ea ing a homogeneous da abase o 327 HEVC
encoded images accompanied wi h bi s eam pa ame e s
and alues ob ained om objec i e and subjec i e as-
sessmen s. Secondly, i analyzes some o he HEVC
in a coding pa ame e s ega ding hei in luence on
he image quali y by using machine lea ning, namely
Suppo Vec o Machine - Reg ession.
Keywo ds
HEVC, HEVC in a coding pa ame e s, image
quali y.
1. In oduc ion
Since he in oduc ion o i s s anda dized ideo cod-
ing algo i hm ITU-T H.261, he ideo coding expe s
ha e imp o ed he pe o mance by hal ing he lowe
bi a e a he same pe cep ual quali y e e y ew yea s.
Mos o his gain is due o imp o ed algo i hms o in-
a and in e ame p edic ion and u he e inemen
o esidual coding. The esul s also show ha he hu-
man isual sys em is as sa is ied wi h an HEVC bi -
s eam as i was wi h H.261 o mo e han en imes
he size. The ques ion o he QoE communi y is hen:
Wha can we lea n om he in o ma ion educ ion p o-
cess in he ideo encode o he analysis o he image
and ideo quali y?
This pape may se e as a i s s ep owa ds ha
ques ion by c ea ing a da ase o HEVC in a coded
images om olde s ill image coding s anda ds such
as JPEG. This s ep p o ides us wi h su icien ly la ge
g oup o sou ce images, oughly associa ed o sco e ob-
ained om subjec i e es s. Then, some i s p ope -
ies o he HEVC bi s eam a e analyzed owa ds iden-
i ying he impo ance o he highly lexible pa i ion-
ing p ocess which is one o he s eng hs o HEVC.
While he mo i a ion in his endea o is di e en ,
he wo k is also closely ela ed o No-Re e ence qual-
i y measu emen and also can se es as a base o de-
elopmen o new No-Re e ence quali y measu emen s.
Fo his eason, we p o ide a b ie o e iew o hem.
In ecen yea s many app oaches o link bi s eam
pa ame e s o QoE we e in oduced [1], [2], [3], [4],
[5], [6], [7], [8], [9], [10], [11], [12] and [13]. The quali y
p edic ion app oaches de eloped o H.264/AVC coded
ideos, especially he bi s eam based ones, a e no
applicable o HEVC coded ideos [2]. A ew HEVC
bi s eam based quali y es ima ion models ha e been
al eady p oposed in he li e a u e. The model p o-
posed by Lei e al. in [3] is based on a linea e-
g ession and ex ac s Quan iza ion Pa ame e A e age
and Skip Coding Uni Pe cen om HEVC bi s eam
o es ima e a ideo quali y p edic ed by PSNR. In [4],
Anegekuh e al. de eloped he eg ession model, based
on a mou ion amoun me ic (a me ic desc ibed in
his pape de e mining ideo con en ype) and Quan-
iza ion Pa ame e (QP), being able o es ima e ideo
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 436
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
quali y p edic ed again by PSNR. An ex ended e -
sion o his model aking in o accoun also a complex-
i y o ideo sequences was p oposed in [5]. Simila ly
as in [4] and [5], Anegekuh e al. p oposed he e-
g ession model based on he QP and con en ype in
his wo k cha ac e ized by he con en ype classi i-
ca ion me ic, es ima ing quali y alues p edic ed by
PSNR. Shahid e al. designed in [6] he model, based
on a wo-laye eed o wa d a i icial neu al ne wo k and
in ol ing 43 HEVC bi s eam pa ame e s, being able
o es ima e a ideo quali y p edic ed by PSNR, VQM,
VIF, and PVQM. In [7] Izumi e al. p oposed an objec-
i e pe cep ual ideo-quali y-measu emen o HEVC.
They in oduced wo pa ame ic NR me hods o es i-
ma e a pe cep ual pic u e quali y o HEVC. In [8], He
e al. p oposed No-Re e ence model which conside s
bi s eam and display pa ame e s, o be mo e gene -
alized o di e en e minal scena ios. The ollowing
pa ame e s we e conside ed as inpu s o he model:
Quan isa ion Pa ame e , mo ion ec o s, ideo com-
plexi y and key- ame indica o and display pa ame-
e s ( esolu ion, PPI, . . . ). The au ho s ied o make
gene al model o bo h H.264/AVC and H.265/HEVC
s anda ds. In [9] Fazliani e al. p oposed a nea eal-
ime No-Re e ence ideo quali y assessmen me hod.
They ained a ully connec ed neu al ne wo k wi h
ea u es ex ac ed om bo h bi s eam and pixel do-
mains along wi h hei espec i e subjec i e quali y
sco es. In [10], Alizadeh e al. p esen ed a no el
No-Re e ence Video Quali y Assessmen (NR-VQA)
based on Con olu ional Neu al Ne wo k (CNN) o
he HEVC. In [11], Ren e al. p esen ed a No-Re e ence
quali y assessmen algo i hm o UHD HEVC encoded
ideos. The algo i hm di ec ly ex ac s he speci ic
ideo cha ac e is ics om he HEVC bi s eam and
es ablishes he ma hema ical model be ween he ideo
cha ac e is ics and he inal ideo quali y h ough a lin-
ea eg ession. The algo i hm ex ac s h ee ideo ea-
u es, i.e. quan iza ion pa ame e s o he measu emen
comp ession damage, numbe o CTUs o di e en sizes
(8×8 & 32×32), and numbe s o he SAO blocks.
Huang e al. p oposed in [12] a No-Re e ence (NR)
Video Quali y Assessmen (VQA) me hod o ideos
dis o ed by he HEVC. The assessmen was pe o med
wi hou an access o a bi s eam. The p oposed analy-
sis was based on he ans o m coe icien s es ima ed by
he decoded ideo pixels, which a e used o es ima e
a le el o quan iza ion. In [13], Nawada e al. p e-
sen ed and desc ibed mo e quali y indica o s ha can
be used in a No-Re e ence QoE calcula ion, since some
o hem de ec speci ic e o s. Such e o s a e di icul
o include in a global QoE model bu a e impo an
om he ope a ion poin o iew.
In his pape , we i s map HEVC Quan iza ion Pa-
ame e s (QPs) using h ee well-known image qual-
i y da abases, i.e. he IRCCyN/IVC [14] da abase,
he LIVE image da abase [15], and he TID2013
da abase [16], o MOS. As a esul o he mapping
p ocess, a new me ged da ase con aining 327 HEVC
encoded images accompanied wi h he co esponding
bi s eams and second o de MOS alues is c ea ed.
Second o de MOS means indica i e quali y alues
de i ed om he MOSs included in he men ioned
da ase s by linea alignmen . In he second s ep,
he new da ase is used o analyze an impac o some
HEVC in a coding pa ame e s, i.e. QPs, dis ibu-
ion o Coding Uni (CU), P edic ion Uni (PU) and
T ans o m Uni (TU), on image quali y expe ienced
by he end use by deploying Suppo Vec o Machine
- Reg ession (SVM).
The emainde o he pape is o ganized as ollows.
Sec ion 2. desc ibes he expe imen dealing wi h
he mapping o he HEVC QPs o MOS. In Sec. 3. ,
he analysis o he selec ed HEVC in a coding pa am-
e e s impac on image quali y using SVM is p esen ed.
Sec ion 4. p o ides he inal conclusions and sugges s
u u e wo k.
2. Mapping o HEVC Images
o Subjec i e Quali y o
JPEG Images
2.1. Desc ip ion o he Used
Da ase s
We would like o ind ou whe he he e exis s a ela-
ionship be ween bi s eam pa ame e s o in a coded
H.265/HEVC images and hei subjec i e quali y.
Un o una ely, he e is no su icien ly la ge da ase
o HEVC in a coded images anno a ed by subjec i e
quali y es s. We ha e decided o euse olde anno a ed
da ase s wi h simila ype o dis o ions like he HEVC
coding p oduces. Sui able da ase s should be hose
which con ain images wi h JPEG deg ada ion because
i in oduces simila ypes o dis o ions as HEVC
coding. In ou expe imen s, h ee well-known image
quali y da abases, namely IRCCyN/IVC [14] da abase,
LIVE image da abase [15], and TID2013 da abase [16],
con aining non-deg aded and also JPEG-deg aded im-
ages, besides o he deg ada ions, we e used. All o
he abo e men ioned da ase s in ol e esul s om
subjec i e es s as well. In he case o he IRC-
CyN/IVC da abase, he Double S imulus Impai men
Scale (DSIS) [17] me hod was used o ob ain quali y
sco es. This me hod uses a i e-g ade impai men scale
whe e one means e y annoying and i e impe cep ible
di e ence be ween he deg aded and e e ence image.
The Absolu e Ca ego y Ra ing (ACR) [18] me hod was
employed o a subjec i e e alua ion in he case o
he LIVE image da abase. The g ading scale was di-
ided in o i e linea equal egions anging om low
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 437
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
IVC/IRCYN
LIVE
TID 2013
HM e e ence
so wa e
QP: 0-51
Well known da a
Objec i e
measu eme s
(PSNR, SSIM, VIF)
Compa ison o
objec i e alues
min(VQMHEVC-VQMJPEG)2
Median o VQM
alues
327 HEVC
encoded images
INLSA
MOS scaling
o 1-5
Objec i e
measu eme s
(PSNR, SSIM, VIF)
New ILT-HEVC
da ase
327 images
wi h second
o de MOS 1-5
Re e ence images
(Non-deg aded)
JPEG dis o ed
images PSNR, SSIM,
VIF alues
SSIM alues
Subjec i e sco es
(IVC/IRCyN MOS 1-5, LIVE MOS 1-100, TID2013 0-9)
HEVC dis o ed
images
PSNR, SSIM,
VIF alues
PSNR
minial di e ence
SSIM
minial di e ence
VIF
minial di e ence
Fig. 1: Basic schema o da a p epa a ion p ocess.
o excellen quali y. The scale was hen linea ly con-
e ed in o 1–100. A pai wise so ing me hodology de-
sc ibed in [16] was used in he case o he TID2013
da abase o ob ain a isual quali y. The subjec i e
es s we e conduc ed in i e coun ies, i.e. Finland,
Uk aine, F ance, I aly, and USA. The MOS alues ob-
ained by his me hodology a y om 0 o 9 whe e
he la ge alues co espond o be e isual quali y.
Mo e de ails, like esolu ion, numbe o he used e e -
ence and deg aded images a e p esen ed in he Tab. 1.
Tab. 1: Pa ame e s o he used image da ase s.
Da ase IRCCyN/IVC LIVE TID2013
Name
Numbe 10 29 25
o images
Numbe
50 175 125
o JPEG
dis o ed
images
Resolu ion 512×512 768×512 512×384
( ypically)
2.2. Da a P epa a ion o
Expe imen
Applying machine lea ning algo i hms equi e a la ge
amoun o inpu da a. Due o his ac , we ha e me ged
he h ee abo e men ioned da ase s in o la ge one.
Sou ce ( e e ence) images wi hou any dis o ions se e
as a base o he new bigge da ase . Whole p ocess
o da a p epa a ion and c ea ion o new da ase sui -
able o machine lea ning is depic ed in Fig. 1. Fi s ly,
a colou space o each e e ence image was con e ed
om RGB o YUV420p. Secondly, all he e e ence
images om he da ase s we e encoded by he HEVC
comp ession s anda d using he HM e e ence so wa e
( e sion 16.20) [19]. All he encoding se ings we e kep
de aul , excep Quan isa ion Pa ame e . The Quan i-
sa ion Pa ame e (QP) has been con inuously changed
om 1 o 51 du ing he encoding p ocess. As a esul ,
a new da ase con aining 3 264 HEVC encoded images
accompanied wi h he co esponding bi s eams was
c ea ed. This da ase in ol es 51 le els o comp ession
deg ada ion o each e e ence image. To in es iga e
he ela ionship be ween HEVC s eam pa ame e s and
quali y o image, i is necessa y o ha e quali y assess-
men s o he co esponding images a hand. The bes
way o ge hem is o execu e subjec i e es s bu his
is also a e y expensi e and ime consuming app oach.
Fo ou pu pose, an app oxima e alue o subjec i e
sco e is su icien jus o see whe he i is possible o
map HEVC in a coding pa ame e s o image quali y.
So, ins ead o assessing he quali y o all he images
subjec i ely, an equi alen quali y o he deg aded im-
ages om he o iginal da abases was sough in o de o
assign hei subjec i e MOS alues o he HEVC in a-
coded images. In o he wo ds, we mapped a ailable
quali y sco es o he JPEG image o he coun e pa
HEVC coded image. In o de o ind ou a ela ionship
be ween he HEVC and JPEG deg ada ions, h ee di -
e en objec i e measu es, namely Peak Signal o Noise
Ra io (PSNR), S uc u al SIMila i y index, and Visual
In o ma ion Fideli y (VIF), we e compu ed. Acco d-
ing o [20], he e is a ela ionship be ween objec i e
measu e alues and subjec i e sco e. Men ioned objec-
i e measu emen s we e done on bo h he new HEVC
da ase and JPEG deg aded images. Fo calcula ing
he PSNR, SSIM, and VIF alues, a Video Quali y
Measu emen Tool (VQMT) de eloped by Mul imedia
Signal P ocessing G oup ( e sion 1.1) was used in [21].
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 438
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
Dispesion 4; (Selec ed QP 23)
23
20
24
PSNR SSIM VIF
0
5
10
15
20
25
(a)
Dispesion 4; (Selec ed QP 48)
49 48
45
PSNR SSIM VIF
0
10
20
30
40
50
(b)
Dispesion 3; (Selec ed QP 42)
43 42 40
PSNR SSIM VIF
0
10
20
30
40
50
(c)
Dispesion 3; (Selec ed QP 42)
43 42 40
PSNR SSIM VIF
0
10
20
30
40
50
(d)
Fig. 2: Dis ibu ions o he QP alues ob ained by he measu es o he dispe sion wo s cases.
2.3. Mapping HEVC Deg ada ion o
JPEG Subjec i e Sco e
Residues be ween he HEVC and JPEG me ics esul s
we e compu ed acco ding o ollowing Eq. (1):
HEV C_QP = min(((V QM(JP EGimage)+
−V QM(HEV Cimage))2),(1)
whe e HEV C_QP is he HEVC image wi h he clos-
es objec i e quali y o he JPEG image and
V QM ={P SNR, SSIM, V IF }. Each esul om
he HEVC da ase was compa ed o he esul s om
he JPEG da ase . This compa ison was done o each
objec i e me hod sepa a ely. Fo he HEVC encoded
image wi h he QP alue whe e he objec i e measu es
ma ched bes (minimum esidual), he MOS alue
o he JPEG deg aded image was assigned. As h ee
di e en measu es (PSNR, SSIM, VIF) we e used,
some imes happened ha h ee di e en candida es o
he single JPEG sco e we e deno ed. I was caused
by di e en app oaches and sensi i i y o he used ob-
jec i e measu emen s. The di e en candida es co e-
spond o HEVC images wi h sligh ly di e en alues
o QP. The e o e, we ha e calcula ed dispe sion be-
ween images selec ed by each objec i e measu emen
acco ding o he ollowing Eq. (2):
QP _Dispe sionp s = max(selec edQP (V QM))+
−min(selec edQP (V QM)),
(2)
whe e QP _Dispe sionp s is a dispe sion o each im-
age and selec edQP (V QM)is he QP o he images
selec ed by he co ensponding VQM. A maximum dis-
pe sion ob ained by he h ee objec i e me ics was
4 QP uni s and ha e appea ed only wice. A di e ence
o 4 QP uni s is isually ha dly no iceable. I is wo h
no ing ha he di e ence is a he small conside ing
he ac ha PSNR, SSIM, and VIF a e based on qui e
di e en app oaches. The dispe sion and equency o
occu ence o di e en QP alues a e depic ed in Fig. 3.
Dis ibu ion o QP dispe sion
69
188
58
10
2
01234
QP dispe sion
0
20
40
60
80
100
120
140
160
180
200
Numbe o occu ance
Fig. 3: Size o QP dispe sion s numbe o occu ence.
We also analysed dis ibu ions o all he objec i e
measu es deployed in ou expe imen . Figu e 2 depic s
ou wo s cases in e ms o he QP dispe sion. As you
can see om his pic u e, he alues ob ained om
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 439
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
he objec i e measu es sligh ly di e . This is caused
by di e en app oaches used by he measu es and also
by a con en o he es images.
In o de o ge only one image associa ed o he sub-
jec i e quali y, a median alue o QPs p o ided by di -
e en objec i e measu emen s was chosen and he co -
esponding HEVC image was agged wi h he MOS
sco e om he o iginal JPEG da ase . Figu e 4 depic s
a pe cen age o he cases when he alue ob ained by
he measu e di e s om he selec ed median alue.
I is ob ious ha he esul s ob ained by PSNR mos ly
di e om he selec ed median alue. The eason can
be ha PSNR is a a he simple measu e which
does no conside p ope ies o human isual sys em.
As he MOS sco es we e no di ec ly ob ained by sub-
jec i e es ing bu by he alignmen using he objec i e
measu es, we will e e o hem as a second o de MOS
in his wo k.
45%
34%
16%
PSNR SSIM VIF
0
5
10
15
20
25
30
35
40
45
Fig. 4: Pe cen age o he cases when he measu es p o ided
he di e en QP alue om he selec ed median alue.
012345
MOS
25
30
35
40
45
50
Median QP
IVC MOS o median QP
A ion
Ba ba
Boa s
Clown
F ui
House
Isabe
Lena
Mand il
Pimen
Fig. 5: Rela ionship be ween he HEVC QP and JPEG MOS
o he IRCCyN/IVC da abase wi h deno ed di e en
con en o images.
By he abo e speci ied app oach, we map he Quan-
isa ion Pa ame e o he images o alues desc ibing
hei quali y, i.e. JPEG MOS. As he eused olde
image quali y da ase s ha e included di e en MOS
scales, we ha e execu ed a mapping o QP o quali y
sco e on each da ase sepa a ely. A ela ionship be-
ween QP and JPEG MOS is depic ed in Fig. 5, Fig. 6
and Fig. 7.
0 10 20 30 40 50 60 70 80 90 100
MOS
0
10
20
30
40
50
60
Median QP
LIVE MOS o median QP
Ligh house
Sailing2
Sailing3
S a ue
Woman
Womanha
Building2
Coinsin oun ain
Flowe sonih35
Bikes
Buildings
Caps
house
ligh house2
mona ch
ocean
pain edhouse
pa o s
plane
apids
sailing1
sailing4
st eam
ca ni aldolls
ceme y
chu chandcapi ol
dance s
manfishing
studen sculp u e
Fig. 6: Rela ionship be ween he HEVC QP and JPEG MOS
o he LIVE image da abase wi h deno ed di e en con-
en o images.
0123456789
MOS
0
10
20
30
40
50
60
Median QP
TID MOS o median QP
I01
I02
I03
I04
I05
I06
I07
I08
I09
I10
I11
I12
I13
I14
I15
I16
I17
I18
I19
I20
I21
I22
I23
I24
I25
Fig. 7: Rela ionship be ween he HEVC QP and JPEG MOS
o he TID2013 da abase wi h deno ed di e en con en
o images.
Figu e 5, Fig. 6 and Fig. 7 show a ypical beha iou
o JPEG MOS e sus QP, showing ha he mapping
may be conside ed easonable.
Howe e , he h ee dis inc da ase s a e s ill consid-
e ed small in e ms o machine lea ning needs.
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 440

INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
2.4. Da ase s Me ging
Due o he equi emen o la ge da ase o mo e eli-
able esul s, he MOS alues coming om he h ee
da ase s we e aligned and me ged. To me ge he
da ase s, an I e a ed Nes ed Leas -Squa es Algo i hm
(INLSA) p oposed in [22] was deployed. Inpu s o
he algo i hm we e objec i e alues o SSIM and
subjec i e JPEG MOSs om all he h ee da ase s.
The LIVE da ase was used as he e e ence one when
i comes o he me ging p ocess. I means ha e-
maining da ase s we e aligned acco ding o he LIVE
da abase. The eason why he LIVE as e e ence was
chosen is ha he LIVE da ase is he bigges one and
con ains a highe ange o subjec i e MOS.
As he INLSA u ilizes he unc ional ela ionships
be ween an objec i e quali y me ic (ex ac ed om
he images) and he co esponding subjec i e MOS,
SSIM was used in he me ging p ocess as he inpu
objec i e quali y me ic. In ou case, he INLSA al-
go i hm calcula ed weigh s and scaling pa ame e s on
he basis o SSIM alues and hen scaled MOS linea ly
o 1–5 scale. Fi s ly, he o iginal MOS alues we e lin-
ea ly ans o med o he in e al [0, 1] in each da ase
sepa a ely. I is so called dis o ion domain, whe e 0
ep esen s no-impai men and 1 ep esen s se e e im-
pai men . Then he scaling p ocess ollows a ypical
linea Eq. (3):
scaledMOS =a·MOS0+b, (3)
whe e amean gain, bshi and MOS0is MOS in
he dis o ion domain. As i was al eady men ioned
abo e, he LIVE da abase was used as he e e ence
da ase . I means ha a= 1 and b= 0. Fo
he IRCCyN/IVC and he TID2013, we ha e ob ained
a= 0.7696 and b= 0.2160 and a= 1.0475 and
b=−0.1168 by he INLSA espec i ely. As a esul
o he me ging p ocess, he new da ase en i led ILT-
HEVC con aining 327 HEVC encoded images accom-
panied wi h he co esponding bi s eams and second
o de MOS alues was made.
2.5. Possible Limi a ions o
he App oach
I is wo h no ing he e ha he p oposed app oach
migh ha e been posi i ely/nega i ely in luenced by
a couple o e ec s. The me ging p ocess o he da ase s
can be one o hem. We ha e used he INLSA o me ge
he alues o he MOS by using he objec i e alues bu
an unde s anding o he ask by he obse e s can be,
a he end, di e en . Secondly, di e en subjec i e es
me hodologies we e deployed o ge he MOS sco es
when i comes o he da ase s. The di e en me hod-
ologies led o he MOS sco es wi h di e se meaning
and in di e en scales. Thi dly, he mapping based on
he objec i e alues can be seen as one o he p ospec-
i e sou ces o noise in his con ex . Despi e he ac
ha we ha e deployed he ull e e ence objec i e me h-
ods, which a e conside ed a he eliable, hey a e s ill
a om being pe ec . I is wo h o ei e a e he e ha
we ha e used h ee di e en ull e e ence measu es and
selec ed he inal quali y on he basis o he median
alue. Finally, a di e en appea ance o he JPEG and
HEVC dis o ions can also ha e some in luence in his
case. I is wo h no ing ha he JPEG comp ession
uses ixed size o blocks (8×8 pixel) unlike he HEVC,
which uses adap i e size. Highe size o blocks can lead
o blu ing impo an lines and can subsequen ly cause
wo se subjec i e assessmen .
3. Analysis o he Selec ed
HEVC In a Coding
Pa ame e s Impac on
Image Quali y by
SVM-Reg ession
3.1. Expe imen Desc ip ion
Fi s , he ILT-HEVC da ase con aining 327 HEVC
encoded images accompanied wi h he co espond-
ing bi s eams and second o de MOS alues was di-
ided in o wo pa s by he 80/20 a io, ypically de-
ployed when i comes o he machine lea ning ap-
p oaches. The da ase di ision was andom while con-
side ing some es ic ions o a oid a model o e i ing.
Thi een di e en con en s co e ing all he quali y le -
els, anging om 4 o 7, we e andomly selec ed o
a alida ion subse . I means ha 68 images, ou o
he 327 dis o ed images, we e selec ed o a alida ion
and he emaining images we e used o a aining o
SVM model. I should be men ioned ha he con en
deployed in he aining da ase was no eplica ed in
he alida ion da ase . This ac should a oid o e i -
ing. The SVM was selec ed as a machine lea ning
echnique o be used o his analysis. Fo a SVM
aining, we ha e used a unc ion included in MAT-
LAB, en i led “ i s m”, which ains a Suppo Vec-
o Machine (SVM) eg ession model on a low- h ough
mode a e-dimensional p edic o da a se . Rega ding
a ke nel unc ion, as second o de polynomial unc-
ion has led o he bes esul s, his unc ion was used
in his analysis as he ke nel unc ion. The epsilon
pa ame e was se o 0.75. O he SVM se ings e-
main unchanged (de aul ). Inpu o he SVM we e
he QP, he Coding, T ans o m, and P edic ion Uni
Size (CU, TU, PU). The a ge alue was he second
o de MOS. The used unc ion has e u ned a ull SVM
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 441
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
eg ession model ained by he selec ed HEVC in a
coding pa ame e s and he co esponding e e ence o
quali y alues he second o de MOS. The aining p o-
cess was epea ed 9 imes wi h di e en combina ions
o he HEVC in a coding pa ame e s in o de o ind
ou how di e en HEVC in a coding pa ame e s in-
luence image quali y. Pea son Linea Co ela ion Co-
e icien (PLCC), Spea man Rank-O de Co ela ion
Coe icien (SROCC), and Roo Mean Squa e E o
(RMSE) we e used as pe o mance indica o s in his
analysis.
3.2. Expe imen al Resul s
Figu e 8, Fig. 9 and Fig. 10 compa e he second
o de MOS alues wi h he p edic ions p o ided by
he SVM eg ession model o he selec ed combina-
ions o he HEVC in a coding pa ame e s in ol ed in
he SVM-based eg ession p ocess. I can be obse ed
om Fig. 8 ha he co ela ion esul s a e a he good
when all he selec ed HEVC in a coding pa ame e s
a e used. Mo eo e , he epo ed RMSE alue is also
a he good.
1 1.5 2 2.5 3 3.5 4 4.5 5
Real MOS
1
1.5
2
2.5
3
3.5
4
4.5
5
SVR p edic ed MOS
Real MOS o SVR p edic ed MOS (Polynomial ke nel q=2)
PLCC = 0.9029
SROCC = 0.9226
RMSE = 0.4167
Fig. 8: Co ela ion be ween he second o de MOS alues and
p edic ions p o ided by he SVM eg ession model o
all he selec ed HEVC in a coding pa ame e s.
The esul s o all he in es iga ed combina ions a e
summa ized in Tab. 2. As i can be clea ly seen om
his able, he bes esul s in e ms o RMSE we e
achie ed o he combina ion in ol ing he QP, CU,
and PU, ollowed by he QP, CU, and TU combina ion
as well as he QP and CU combina ion. I is wo h no -
ing he e ha he epo ed RMSE alue is e en smalle
han ha ob ained o he combina ion in ol ing all
he pa ame e s. Figu e 9 depic s he co ela ion be-
ween he second o de MOS alues and p edic ions
p o ided by he SVM eg ession model o he bes
pe o ming pa ame e s combina ion in e ms o RMSE.
When i comes o he combina ion in ol ing he CU,
TU and PU pa ame e s and a numbe o he suppo
Tab. 2: Resul s ob ained o he di e en combina ions o
he in es iga ed HEVC in a coding pa ame e s.
QP CU TU PU Pe o mance
indica o s
X
PLCC = 0.90
SROCC = 0.92
RMSE = 0.45
Numbe o SVs = 28
X X
PLCC = 0.92
SROCC = 0.93
RMSE = 0.37
Numbe o SVs = 26
X X
PLCC = 0.92
SROCC = 0.93
RMSE = 0.38
Numbe o SVs = 26
X X
PLCC = 0.90
SROCC = 0.93
RMSE = 0.38
Numbe o SVs = 24
X X X
PLCC = 0.92
SROCC = 0.93
RMSE = 0.37
Numbe o SVs = 26
X X X
PLCC = 0.92
SROCC = 0.93
RMSE = 0.37
Numbe o SVs = 24
X X X
PLCC = 0.91
SROCC = 0.93
RMSE = 0.38
Numbe o SVs = 24
X X X
PLCC = 0.85
SROCC = 0.86
RMSE = 0.52
Numbe o SVs = 48
X X X X
PLCC = 0.90
SROCC = 0.92
RMSE = 0.42
Numbe o SVs = 27
1 1.5 2 2.5 3 3.5 4 4.5 5
Real MOS
1
1.5
2
2.5
3
3.5
4
4.5
5
SVR p edic ed MOS
Real MOS o SVR p edic ed MOS (Polynomial ke nel q=2)
Pa ame e s wi hou TU size
PLCC = 0.9152
SROCC = 0.9284
RMSE = 0.3688
Fig. 9: Co ela ion be ween he second o de MOS alues and
p edic ions p o ided by he SVM eg ession model o
he QP, CU and PU dis ibu ions.
ec o s, he numbe doubled in compa ison o he o he
in es iga ed combina ions.
We we e in e es ed in he dependency o he esul -
ing pe o mance on he inpu alues. E en only by us-
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 442
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
ing QP, we disco e ed ha he anking o de was s ill
easonable as i can be seen in Fig. 10. I should be
no ed he e ha he esul s ob ained in his s udy we e
achie ed by a a he small da ase . So, a s a is ical
signi icance o he esul s is a he limi ed.
1 1.5 2 2.5 3 3.5 4 4.5 5
Real MOS
1
1.5
2
2.5
3
3.5
4
4.5
5
SVR p edic ed MOS
Real MOS o SVR p edic ed MOS (Polynomial ke nel q=2)
QP only
PLCC = 0.8971
SROCC = 0.9218
RMSE = 0.4533
Fig. 10: Co ela ion be ween he second o de MOS alues and
p edic ions p o ided by he SVM eg ession model o
he QP only.
4. Conclusions and Fu u e
Wo k
In his con ibu ion, we ha e analyzed he possibili y
o c ea e a da abase wi h a mo e ecen coding s an-
da d (HEVC) om he exis ing anno a ed da abases
o olde coding s anda d, i.e. JPEG, by using di e en
objec i e measu es. While he esul s could no ye
be alida ed by subjec i e es ing, some e idence was
p esen ed ha he app oach is easonable. In addi ion,
machine lea ning, in pa icula SVM, was used o e-
la e he bi s eam in o ma ion o HEVC o subjec i e
quali y.
While his is a common app oach o aining No-
Re e ence bi s eam models, ou wo k is ocused on un-
de s anding he link be ween he encoding p ocess and
he human pe cep ion. Fi s esul s conce ning he im-
age pa i ioning used by he HEVC in a codec we e
p esen ed in his pape . Fu he analysis o he encod-
ing p ocess and he esul ing bi s eam in o ma ion is
equi ed, also ex ending om s ill image o ideo in
u u e esea ch.
Acknowledgmen
This publica ion is he esul o he p ojec imple-
men a ion: Cen e o excellence o sys ems and se -
ices o in elligen anspo II, ITMS 26220120050
suppo ed by he Resea ch & De elopmen Ope a ional
P og amme unded by he ERDF.
Re e ences
[1] SEVCIK, L., M. VOZNAK and J. FRNDA.
QoE p edic ion model o mul imedia se ices
in IP ne wo k applying queuing policy. In: In-
e na ional Symposium on Pe o mance E al-
ua ion o Compu e and Telecommunica ion
Sys ems (SPECTS 2014). Mon e ey: IEEE,
2014, pp. 593–598. ISBN 978-1-4799-5745-3.
DOI: 10.1109/SPECTS.2014.6879998.
[2] VAN WALLENDAEL, G., N. STAELENS,
L. JANOWSKI, J. DE COCK, P. DEMEESTER
and R. VAN DE WALLE. No- e e ence bi s eam-
based impai men de ec ion o high e iciency
ideo coding. In: 2012 Fou h In e na ional Wo k-
shop on Quali y o Mul imedia Expe ience. Ya a
Valley: IEEE, 2012, pp. 7–12. ISBN 978-1-4673-
0725-3. DOI: 10.1109/QoMEX.2012.6263845.
[3] LEI, X., X. JIANG and X. MA. A no- e e ence
ideo quali y e alua ion me hod based on HEVC
bi s eam. In: Eigh h In e na ional Con e ence on
Digi al Image P ocessing (ICDIP 2016). Chengdu:
SPIE, 2016, pp. 1068–1074. ISBN 978-1-5106-
0503-9. DOI: 10.1117/12.2243822.
[4] ANEGEKUH, L., L. SUN and E. IFEA-
CHOR. Encoded bi s eam based ideo con en
ype de ini ion o HEVC ideo quali y p e-
dic ion. In: 2014 IEEE In e na ional Con e -
ence on Communica ions (ICC). Sydney: IEEE,
2014, pp. 1296–1301. ISBN 978-1-4799-2003-7.
DOI: 10.1109/ICC.2014.6883500.
[5] ANEGEKUH, L., L. SUN and E. IFEACHOR.
Encoding and ideo con en based HEVC ideo
quali y p edic ion. Mul imedia Tools and Appli-
ca ions. 2015, ol. 74, iss. 11, pp. 3715–3738.
ISSN 1380-7501. DOI: 10.1007/s11042-013-1795-z.
[6] SHAHID, M., J. PANASIUK, G. VAN WALLEN-
DAEL, M. BARKOWSKY and B. LOVSTROM.
P edic ing ull- e e ence ideo quali y measu es
using HEVC bi s eam-based no- e e ence
ea u es. In: 2015 Se en h In e na ional
Wo kshop on Quali y o Mul imedia Expe-
ience (QoMEX). Pylos-Nes o as: IEEE,
2015, pp. 1–2. ISBN 978-1-4799-8958-4.
DOI: 10.1109/QoMEX.2015.7148118.
[7] IZUMI, K., K. KAWAMURA, T. YOSHINO and
S. NAITO. No e e ence ideo quali y assessmen
based on pa ame ic analysis o HEVC bi s ea.
In: 2014 Six h In e na ional Wo kshop on Quali y
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 443
INFORMATION AND COMMUNICATION TECHNOLOGIES AND SERVICES VOLUME: 17 |NUMBER: 4 |2019 |DECEMBER
o Mul imedia Expe ience (QoMEX). Singapo e:
IEEE, 2014, pp. 49–50. ISBN 978-1-4799-6536-6.
DOI: 10.1109/QoMEX.2014.6982287.
[8] HE, T., R. XIE, J. SU, X. TANG and L. SONG.
A no e e ence bi s eam-based ideo quali y as-
sessmen model o h.265/he c and h.264/a c.
In: 2018 IEEE In e na ional Symposium on
B oadband Mul imedia Sys ems and B oadcas ing
(BMSB). Valencia: IEEE, 2018. ISBN 978-1-5386-
4729-5. DOI: 10.1109/BMSB.2018.8436879.
[9] FAZLIANI, Y., E. ANDRADE and S. SHIRANI.
Lea ning based hyb id no- e e ence ideo qual-
i y assessmen o comp essed ideos. In: 2019
IEEE In e na ional Symposium on Ci cui s and
Sys ems (ISCAS). Sappo o: IEEE, 2019, pp. 1–
5. ISBN 978-1-7281-0397-6. DOI: 10.1109/IS-
CAS.2019.8702584.
[10] ALIZADEH, M., A. MOHAMMADI and
M. SHARIFKHANI. No- e e ence deep
comp essed-based ideo quali y assessmen .
In: 2018 8 h In e na ional Con e ence on Com-
pu e and Knowledge Enginee ing (ICCKE).
Mashhad: IEEE, 2018, pp. 130–134. ISBN 978-1-
5386-9569-2. DOI: 10.1109/ICCKE.2018.8566395.
[11] REN, J., X. JIANG, L. XU and X. GUO.
No- e e ence quali y assessmen o UHD
ideos based on HEVC encoded bi s eam.
In: 2017 4 h In e na ional Con e ence on Sys ems
and In o ma ics (ICSAI). Hangzhou: IEEE,
2017, pp. 1319–1323. ISBN 978-1-5386-1107-4.
DOI: 10.1109/ICSAI.2017.8248490.
[12] HUANG, X., J. SOGAARD and S. FORCHHAM-
MER. No- e e ence pixel based ideo quali y as-
sessmen o HEVC decoded ideo. Jou nal o Vi-
sual Communica ion and Image Rep esen a ion.
2017, ol. 43, iss. 1, pp. 173–184. ISSN 1047-3203.
DOI: 10.1016/j.j ci .2017.01.002.
[13] NAWADA, J., L. JANOWSKI and
M. LESZCZUK. Modeling o quali y o ex-
pe ience in no- e e ence mode. Jou nal o
Telecommunica ions and In o ma ion Technology.
2017, ol. 2017, iss. 2, pp. 11–17. ISSN 1509-4553.
DOI: 10.26636/j i .2017.114517.
[14] NINASSI, A., P. LE CALLET and
F. AUTRUSSEAU. Pseudo no e e ence
image quali y me ic using pe cep ual da a
hiding. In: Human ision and elec onic imag-
ing XI. San Jose: SPIE, 2006, pp. 146–157.
DOI: 10.1117/12.650780.
[15] SHEIKH, H. R., Z. WANG, L. CORMACK
and A. C. BOVIK. Li e image quali y assess-
men da abase 2003. In: Labo a o y o Image
&Video Enginee ing [online]. 2006. A ail-
able a : h p://li e.ece.u exas.edu/
esea ch/Quali y/subjec i e.h m.
[16] POMONARENKO, N., L. JIN, O. IEREMEIEV,
V. LUKIN, K. EGIAZARIAN, J. ASTOLA,
B. VOZEL, K. CHEHDI, M. CARLI, F. BAT-
TISTI and C.-C. J. KUO. Image da abase
TID2013: Peculia i ies, esul s and pe spec i es.
Signal P ocessing: Image Communica ion. 2015,
ol. 30, iss. 1, pp. 57–77. ISSN 0923-5965.
DOI: 10.1016/j.image.2014.10.009.
[17] ITU-R BT.500-13. Me hodology o he subjec i e
assessmen o he quali y o ele ision pic u es.
Gene a: ITU, 2012.
[18] ITU-T P.910. Subjec i e ideo quali y assessmen
me hods o mul imedia applica ions. Gene a:
ITU, 2008.
[19] HEVC. High E iciency Video Coding. Be lin:
HHI, 2015.
[20] SEVCIK, L., L. BEHAN, J. FRNDA,
M. UHRINA, J. BIENIK and M. VOZNAK.
P edic ion o subjec i e ideo quali y based on
objec i e assessmen . In: 2018 26 h Telecom-
munica ions Fo um (TELFOR). Belg ade:
IEEE, 2018, pp. 1–4. ISBN 978-1-5386-7171-9.
DOI: 10.1109/TELFOR.2018.8612127.
[21] VQMT: Video Quali y Measu emen Tool.
In: L’Ecole poly echnique ede ale de Lau-
sanne (EPFL) [online]. 2019. A ailable
a : h ps://mmspg.ep l.ch/ qm .
[22] PINSON, M. H. and S. WOLF. An objec i e
me hod o combining mul iple subjec i e da a
se s. In: Visual Communica ions and Image P o-
cessing 2003. Lugano: SPIE, 2003, pp. 583–593.
ISBN 0-8194-5023-5. DOI: 10.1117/12.509909.
Abou Au ho s
Tomas MIZDOS was bo n in 1993 in Pop ad,
Slo ak Republic. He ecei ed his B.Sc. deg ee in
Mul imedia echnologies a he o Mul imedia and
In o ma ion-Communica ion Technologies, Facul y o
Elec ical Enginee ing and In o ma ion Technology,
Uni e si y o Zilina in 2015. Cu en ly he is a Ph.D.
s uden a he same depa men . His main a ea o
in e es is Func ionali y and Quali y o Mul imedia
Se ices, Quali y o Expe ience, Video Coding and
Digi al Signal P ocessing.
Ma cus BARKOWSKY ecei ed his D .-Ing.
deg ee om he Uni e si y o E langen-Nu embe g
c
2019 ADVANCES IN ELECTRICAL AND ELECTRONIC ENGINEERING 444