Mas e Deg ee P og am in
In o ma ion Managemen
Fac o s d i ing engagemen in social media – an analysis o he
cogni i e app aisal o ideo con en .
Younes Abouljid
Disse a ion
p esen ed as pa ial equi emen o ob aining he Mas e Deg ee P og am in In o ma ion Managemen
NOVA In o ma ion Managemen School
Ins i u o Supe io de Es a ís ica e Ges ão de In o mação
Uni e sidade No a de Lisboa
MGI
NOVA In o ma ion Managemen School
Ins i u o Supe io de Es a ís ica e Ges ão de In o mação
Uni e sidade No a de Lisboa
FACTORS DRIVING ENGAGEMENT IN SOCIAL MEDIA – AN ANALYSIS
OF THE COGNITIVE APPRAISAL OF VIDEO CONTENT.
By
Younes Abouljid
Mas e Thesis p esen ed as pa ial equi emen o ob aining he Mas e ’s deg ee in In o ma ion
Managemen , wi h a specializa ion in In o ma ion Sys ems and Technologies Managemen
Supe iso : D . Nuno An ónio
Co-Supe iso s: D . Ju ij Jaklič
No embe 2023
ii
STATEMENT OF INTEGRITY
I he eby decla e ha ing conduc ed his academic wo k wi h in eg i y. I con i m ha I ha e no used
plagia ism o any o m o undue use o in o ma ion o alsi ica ion o esul s along he p ocess leading
o i s elabo a ion. I u he decla e ha I ha e ully acknowledge he Rules o Conduc and Code o
Hono om he NOVA In o ma ion Managemen School.
Younes Abouljid
Lisbon, 30 No embe 2023
iii
ACKNOWLEDGEMENTS
I would like o exp ess my since e g a i ude o my supe iso D . Nuno An ónio om NOVA IMS and
co-supe iso D . Ju ij Jaklič om Uni e si y o Ljubljana, School o Economics and Business o hei
sugges ions and cu a ion o he p ojec idea and scope. Thei help and guidance h oughou he hesis
concep ion and implemen a ion, and o he p e ious semes e s as a compe en p o esso in hei
espec i e ields as I ha e lea ned a lo om hei knowledge. In addi ion o all he p o esso s whom I
lea ned om and helped me de elop bo h academically and pe sonally.
I would also like o hank ou esea ch pa icipan s om a ious coun ies o sha ing hei opinions
wi h us and aking he ime o be in ol ed in he esea ch.
Finally, I humbly exp ess my g a i ude o my amily, iends, and men o s ha suppo ed my jou ney
in his Eu opean mas e s, helped me g ow, disco e my po en ials, and con ibu ed o wha I’ e
achie ed so a . You a e uly app ecia ed!
i
ABSTRACT
This esea ch aims o elucida e he ac o s ha d i e engagemen wi h ideo-cen ic social media
con en , ocusing on he cogni i e app aisal o such con en . This s udy employs sen imen analysis,
u ilizing he analy ical capabili ies o Powe BI o in e p e da a ga he ed h ough OpenCV machine
lea ning models and use eedback su eys. The objec i e is o disce n he emo ional esponses elici ed
by di e en ypes o engagemen . The da a collec ed om a ious sou ces was syn hesized and
s anda dized o unco e pa e ns, co ela ions, and key de e minan s o social media engagemen and
he emo ions i igge s. By gaining insigh s in o he emo ional aspec s igge ed by media con en ,
campaign manage s can mo e e ec i ely design and es hei campaigns p io o deploymen , he eby
enhancing engagemen and mi iga ing po en ial sho comings.
KEYWORDS
Sen imen analysis, cogni i e app aisal, social media, cus ome engagemen , emo ional esponse,
ideo con en
Sus ainable De elopmen Goals (SGD):

INDEX
1. In oduc ion .................................................................................................................. 1
2. Li e a u e e iew .......................................................................................................... 2
2.1. Emo ional esponse ............................................................................................... 2
2.1.1. Russel’s “Ci cumplex model o a ec ” ........................................................... 2
2.1.2. Mul imodal Sen imen Analysis ..................................................................... 3
2.1.3. The six basic emo ions.................................................................................... 3
2.2. Use s Engagemen ................................................................................................. 5
2.3. Typology o engaged use s .................................................................................... 6
2.4. Opinion mining ...................................................................................................... 8
3. Me hodology ................................................................................................................ 9
3.1. Da ase ................................................................................................................. 10
3.2. Machine Lea ning models ................................................................................... 12
3.2.1. OpenCV ......................................................................................................... 12
3.2.2. Su ey ........................................................................................................... 13
3.3. Empi ical S udy .................................................................................................... 15
4. Resul s and discussion ................................................................................................ 20
5. Conclusions and u u e wo ks .................................................................................... 27
5.1. Theo e ical con ibu ion and business implica ions ........................................... 27
5.2. Limi a ions and Fu u e esea ch ......................................................................... 28
6. Bibliog aphical REFERENCES ....................................................................................... 30
7. Appendix ..................................................................................................................... 33
7.1. Appendix A........................................................................................................... 33
7.2. Appendix B:.......................................................................................................... 34
7.3. Appendix C:.......................................................................................................... 37
8. Annexes (op ional)...................................................................................................... 38
i
LIST OF FIGURES
Figu e 2.1 - Russel’s ci cumplex model o emo ions. (Russel, 2005) ........................................ 3
Figu e 2.2 - Typology o i al ad sha e s (Kulka ni, Ka lo, Sha ma, & Sha ma, 2019) ............... 7
Figu e 3.1 - Social Media engagemen by 1s qua ile o 2023 (Senso Towe , 2023) ................ 9
Figu e 3.2 -Diag am o he esea ch design ............................................................................. 10
Figu e 3.3 - Zach King TikTok p o ile (Tik ok, 2023) ................................................................. 10
Figu e 3.4 - lowcha desc ibes he Haa cascade classi ie design (Mi al, 2020)................. 13
Figu e 3.5 - he su ey g id o collec emo ions om ideos wa ched. .................................. 14
Figu e 3.6 - plo o he emo ional analysis da a ame. ........................................................... 16
Figu e 3.7 - Plo s o he emo ional ecogni ion calcula ed by OpenCV model o he selec ed
ideos. .............................................................................................................................. 17
Figu e 3.8 – Da a model ........................................................................................................... 18
Figu e 4.1 - Clus e ed column cha o su ey s model alues 1. .......................................... 20
Figu e 4.2 - Clus e ed column cha o su ey s model alues 2. .......................................... 21
Figu e 4.3 - Engagemen ques ion in he su ey. .................................................................... 22
Figu e 4.4 - Ma ix o engagemen me ics om he su ey s he pla o m. ........................ 23
Figu e 4.5 - able ep esen s imes o consume media con en .............................................. 24
Figu e 4.6 - s acked ba cha o desi able con en . ................................................................ 24
Figu e 4.7 - s acked ba cha o exposu e o new con en . .................................................... 25
Figu e 4.8 – Reasons o engagemen by emo ions p edic ed 1............................................... 25
Figu e 4.9 – Reasons o engagemen by emo ions p edic ed 2............................................... 26
LIST OF TABLES
ii
Table 3.1 - Videos selec ed o analysis om Zach King p o ile ac oss he mon h o Ap il..... 11
Table 3.2 - Example o a inal s a e o p ocessed da a ame. ................................................. 15
iii
LIST OF ABBREVIATIONS AND ACRONYMS
BI: Business In elligence
EDA: Explo a o y Da a Analysis
ETL: Ex ac , T ans o m, Load
HCI: Human-Compu e In e ac ion
ML: machine lea ning
NLP: na u al language p ocessing
SNS: social ne wo k si es
SVM: Suppo Vec o Machine
TLM: hough lis ing me hod.
7
Figu e 2.2 - Typology o i al ad sha e s (Kulka ni, Ka lo, Sha ma, & Sha ma, 2019)
Ac i e sha e s ca y ou he mos sen imen o bo h he ad and he b and, while b and- ana ic sha e s
ca e mo e abou he b and han he ad, and ice e sa o he con en hung y sha e s. The do man
sha e s simply do no ca e abou he b and o he ad (Kulka ni, Ka lo, Sha ma, & Sha ma, 2019).
The ac i e sha e s, o he mos a o able clus e , which bes seeds he i al campaign, a e ha d o ind
and ela i ely low compa ed o he es o he use s. In his s udy, 17.2% o he popula ion was
a ec ed; howe e , he choice o hese seed consume s who will po en ially d i e he i ali y o he ad
campaign has o be done wi h p ecision. Acco ding o he classical Pa e o p inciple ( he law o he i al
ew), 20% o use s a e expec ed o ca y 80% o he load o p opaga e he message. Some imes
companies epidemic i al campaigns end o ail due o he b oadness o he sp ead o he message
and a gene al lack o p ope segmen a ion (Kulka ni, Ka lo, Sha ma, & Sha ma, 2019).
P edic o s o i ali y a e he impo ance o “message-con en ” pa icula ly he emo ional con en ,
psychological ( he need o belong), and ie s eng h as a ea u e o SNS.
Also, exis ing li e a u e sugges s ha he inco po a ion o an embedded b and may dis ac he
consume om he message o he ad. Too much in o ma ion ega ding he b and embedded in an ad
may lead o a nega i e ou come om sha ing he i al ad; mos consume s end o ocus on he
con en o he ad.
Newe li e a u e sugges s he opposi e. A s ong b and sen imen enhances he ecep ion o he ad
message, he e o e inc easing he sha ing p obabili y. I is also suppo ed by he “classical ‘Recip ocal
Media ion’ model o ad e ising e ec i eness p oposed by MacKenzie e al. (1986), which
hypo hesizes a ecip ocal ela ionship be ween a consume 's ‘ad’ and ‘b and’ in o ma ion p ocessing
beha io .”
The insigh s om his s udy help b and manage s, ad e ise s, and ma ke e s unde s and hei use -
gene a ed con en and de ise e ec i e s a egies o mee he p o ile o he seede s in o de o design
a success ul i al ad campaign.

8
2.4. OPINION MINING
Sen imen analysis, o opinion mining, s a ed as ea ly as he 1950s in he analysis o w i en
documen s (Rao, Ahuja, Kansa a, & Pa el, 2021). In his book Bing Liu desc ibed sen imen analysis as
he “ ield o s udy ha analyzes people's opinions, sen imen s, e alua ions, app aisals, a i udes, and
emo ions owa ds en i ies such as p oduc s, se ices, o ganiza ions, indi iduals, issues, e en s, opics,
and hei a ibu es” (Liu, 2012).
Bu since hen, sen imen analysis has de eloped o analyze di e en a i ac s using new echniques
such as Na u al Language P ocessing (NLP) and Machine Lea ning (ML), which ha e inc eased
pe o mance and accu acy. O ganiza ions use sen imen analysis o gain compe i i e ad an age,
inc ease sales, and de elop s a egies ha a e mo e ailo ed and ocused based on he opinions o
hei cus ome s (Rao, Ahuja, Kansa a, & Pa el, 2021). The e a e wo ypes o sen imen analysis
echniques. Fea u e-based echnique ha ex ac s a special ea u e om a ex ual documen .
Documen -based echnique ha de i es he o e all pola i ies o a ex documen (Kulka ni, Ka lo,
Sha ma, & Sha ma, 2019).
Ini ially, he adi ional way o analyze he cogni i e esponse is he ‘ hough -lis ing me hod’ (TLM).
While he mode n way o analyze he cogni i e esponse is h ough sen imen analysis d i en by
psycholinguis ics and NLP (Kulka ni, Ka lo, Sha ma, & Sha ma, 2019).
The TLM me hod is he adi ional way o measu e emo ional esponse o ideo con en o social
media con en in gene al. I is ad hoc based and has a subjec i e na u e due o i s indica o s and
in ui ion. The way i wo ks is o w i e immedia e hough s du ing o a e exposu e o media con en .
Then, he expe imen e classi ies hese hough s based on p e-se c i e ia and sees which ca ego y
hey i he mos . Then, hey p oceed o calcula e he sco e o each ca ego y, ei he by a simple
addi ion o , i he e a e le els o impo ance, hey use a model ha weigh s each esponse o an
indica o and e en ually sums up and a e ages he sco es o p edic he a i ude. Typically, he TLM
has been used o compu e a “ alence-weigh ed agg ega e index o cogni i e esponses." The
sho coming o his me hod is ha i has measu emen p oblems. Mainly, i cap u es he di ec ion o
he a i ude, whe he posi i e o nega i e, bu ails o cap u e he in ensi y o he cogni i e hough
(Kulka ni, Ka lo, Sha ma, & Sha ma, 2019).
Following he li e a u e e iew explo ing emo ions, emo ional esponse, use engagemen , and
opinion mining and hei in e ela ion, his esea ch pape aims o in es iga e whe he he emo ions
de ec ed ia sen imen analysis me hodologies align wi h hose pe cei ed by human obse e s. The
me hodology p oposed will es he ollowing hypo heses: H0 sugges s ha he e is no signi ican
di e ence be ween he emo ions iden i ied h ough sen imen analysis and hose expe ienced by
human obse e s, while H1 p oposes he opposi e. Th oughou he esea ch pape me hodology and
esul s discussion, we will y o alida e he e icacy and accu acy o sen imen analysis echniques in
cap u ing he nuances o human emo ions. Th ough his in es iga ion, we aim o con ibu e o a
deepe unde s anding o emo ional esponses o ideo media con en and hei implica ions o use
engagemen in digi al en i onmen s.
9
3. METHODOLOGY
This esea ch design was designed o de e mine he eason behind he engagemen success o social
media ideos. The ocus is on he sho - o m ideo con en ha go hea ily popula ized a e he 2020
lockdown. Amongs he op h ee leading pla o ms in his s yle o ideo con en (Ins ag am eels,
YouTube sho s, and Tik Toks), Tik Tok emains he leade in e ms o downloads, usage, and
engagemen . Acco ding o Senso owe , a ma ke -Leading Digi al In elligence in he i s qua ile o
he yea 2023, his pla o m has eco ded he highes use engagemen , wi h o e 7 hou s/week and
120 sessions/week pe use , as shown in igu e 3.1. (Ab aham Youse , 2023)
Figu e 3.1 - Social Media engagemen by 1s qua ile o 2023 (Senso Towe , 2023)
The sho - o m ideo con en o mula is cha ac e ized by a sho du a ion, ypically be ween 15
seconds and 1 minu e, wi h hea y sound and ideo e ec s, bold ex s, and c ea i e il e s, as well as
c ea ing a collabo a i e ecosys em be ween i s use s. (Singh, 2023)
This s yle o con en go hea ily popula ized wi h he cu en as pace o li e ha limi s he use ’s
a en ion span o consume sho and diges ible con en ha o e s a a ie y o opics in a un and
exp essi e way (Singh, 2023). In o de o unde s and he eason behind he success o his o mula as
well as how o su i e in his cu en landscape, his s udy ocuses on he analysis o he cogni i e
app aisal ha is gene a ed om his con en .
The diag am shown in Figu e 3.2 desc ibes he esea ch design p ocess used o pe o m a compa a i e
analysis be ween he machine lea ning models ha analyze he ideos selec ed h ough he TikTok
pla o m and a su ey-gene a ed da ase ha humanly analyzes he ideo by ocusing on he
emo ional esponse. A py hon sc ip is de eloped in Jupy e no ebook o un he machine lea ning
models agains he ideos selec ed o analysis. The esul s a e expo ed in o a CSV ile which will be
uploaded in o Powe Que y a buil -in ETL whi in he Powe BI sui e. The da a models consis o su ey
gene a ed esponses and machine lea ning models p edic ed emo ional esul s. Then, de i e he
ac o s om he isual dashboa ds ha d i e he engagemen s, es he accu acy o he cogni i e
10
app aisal, and in e con en cha ac e is ics ac o s o he b and-new s yle o ideos domina ing he
social ne wo k si es.
Figu e 3.2 -Diag am o he esea ch design
3.1. DATASET
Pa o his s udy is o ocus on a single en i y. In o he wo ds, he choice o a single p o ile o in luence
and seeing he luc ua ion in engagemen o hei con en o e a speci ic pe iod o ime. Then, h ough
sen imen analysis o he cogni i e app aisal, in e he ac o s ha esul ed in he di e ence in he
le el o engagemen s.
The choice made in his s udy was o an in luence who is he holde o he Guinness wo ld eco d o
he mos iewed ideo on he Tik Tok pla o m, eaching 2.2 billion iews as o Ma ch 15, 2022
(Rod iguez, 2022). The p o ile goes by he name o Zach King, as seen in igu e 3.3.
Figu e 3.3 - Zach King TikTok p o ile (Tik ok, 2023)
The da a was ex ac ed du ing he mon h o Ap il 2023, in he i s mon h o his s udy. Fo ha ime
pe iod, 6 ideos we e ex ac ed wi h di e en le els o engagemen (see able 3.1), which would be
11
he basis o his analysis bo h h ough he machine lea ning models de eloped by Oc a io e al. (2019)
and he su ey designed and sha ed ac oss di e en pa icipan s om di e en coun ies.
Table 3.1 - Videos selec ed o analysis om Zach King p o ile ac oss he mon h o Ap il.
Zach King Videos o he mon h o Ap il 2023 S a is ics
Videos (o de ed by ch onological o de )
Engagemen s s a is ics
(no i le)
da e: 05/04/2023
iews: 25 800 000
likes: 1 300 000
commen s: 3 372
sa ed: 38 700
When he day needs me anywhe e and
e e ywhe e, #Ce aVe SPF is wha I wea so I can
#FaceI LikeADe m #Ce aVePa ne
da e: 07/04/2023
iews: 97 100 000
likes: 907 500
commen s: 2 627
sa ed: 14 100
This is Puzzling
da e: 14/04/2023
iews: 135 600 000
likes: 8 200 000
commen s: 17 500
sa ed: 366 800
Can someone please explain how he mi o
knows he e is an egg
da e: 19/04/2023
iews: 11 200 000
likes: 696 200
commen s: 3 899
sa ed: 23 300
Li e isn always as i seems @so yank96
da e: 27/04/2023
12
iews: 80 900 000
likes: 5 300 000
commen s: 16 800
sa ed: 164 600
spen he las 6 mon hs making his, hope you
enjoy he aile . Wa ch he en i e sho ilm in
my p o ile
da e: 28/04/2023
iews: 2 400 000
likes: 181 800
commen s: 1 497
sa ed: 11 500
This da ase o ideos also exp essed a a ie y o con en cha ac e is ics, which we discussed in he
p e ious sec ion, such as in ol ing celeb i ies and o he in luence s, b and ad e isemen s, and isual
and sound e ec s. In addi ion o di e en opics ha a y om p omo ional, sales, in o ma i e, and
en e ainmen .
3.2. MACHINE LEARNING MODELS
3.2.1. OpenCV
The choice o use OpenCV echnology as he main ML model o ex ac emo ions based on acial
exp essions om ideos came a e se e al i e a ions and echnological ools you s aced wi h dead
ends such as expe imen ing wi h Mic oso Azu e Cogni i e Se ices. OpenCV e e s o he Open
Sou ce Compu e Vision Lib a y. I is an open-sou ce p ojec licensed unde he Apache 2 licensed
p oduc s which makes i a ailable o use o indi idual and comme cial pu poses (OpenCV eam,
2023). The applica ion o OpenCV algo i hms as desc ibed in hei o icial documen a ion on he
Gi Hub pla o m could be used o he ollowing: “ o de ec and ecognize aces, iden i y objec s,
classi y human ac ions in ideos, ack came a mo emen s, ack mo ing objec s, ex ac 3D models
o objec s, p oduce 3D poin clouds om s e eo came as, s i ch images oge he o p oduce a high
esolu ion image o an en i e scene, ind simila images om an image da abase, emo e ed eyes om
images aken using lash, ollow eye mo emen s, ecognize scene y and es ablish ma ke s o o e lay
i wi h augmen ed eali y, e c” (OpenCV eam, 2023)
Fo his s udy, he model chosen was de eloped by b-i Bonn-Aachen In e na ional Cen e o
In o ma ion Technology and Hochschule Bonn-Rhein-Sieg unde he esea ch pape published “Real-
ime Con olu ional Neu al Ne wo ks o Emo ion and Gende Classi ica ion” The epo ed accu acy o
his model was 70% o he emo ional da ase and 96% o he gende da ase , wi h he goal o a eal-
ime in e ac i i y ha ma ches human pe o mance in analyzing emo ions based on acial exp essions
by c ea ing he bes accu acy o e a numbe o pa ame e s (Oc a io, Ploge , & Ma ias, 2019).

13
Rega ding he emo ional da ase , he de elope s used he Paul Ekman classi ica ion o he six basic
uni e sal emo ions, which a e Happy, Sad, Ang y, Disgus , Su p ise, Fea and he se en h added was a
Neu al s a e o emo ion.
Howe e , he e a e some sho comings desc ibed in he esea ch pape desc ibing he model. A majo
sho coming is ela ed o he accu acy o he model since, acco ding o he au ho s, h ough he
isualiza ion o he con usion ma ix, he e is a aul y in e p e a ion o he ang y emo ion wi h he
disgus emo ion, as well as he occu ence o con usion be ween he ea and sad emo ions. This
con usion ypically happens when he subjec is wea ing glasses mo e equen ly, especially da k
glasses, since he model classi ies i as a own, which belongs o he ang y emo ion. Ano he
sho coming exp essed by he au ho s is he bias o he model since i was ained using mos ly
wes e n popula ions, which could e lec a classi ica ion ha is a he inaccu a e using eas e n
popula ions wi h eas e n cul u e cha ac e is ics. (Oc a io, Ploge , & Ma ias, 2019)
The models used ollow a Haa classi ie algo i hm. An Haa classi ie “is a machine lea ning objec
de ec ion p og am ha iden i ies objec s in an image and ideo” (Mi al, 2020). I was i s in oduced
by Paul Viola and Michael Jones in hei 2001 pape , "Rapid Objec De ec ion using a Boos ed Cascade
o Simple Fea u es" (OpenCV, 2023). The algo i hm needs bo h posi i e images, which a e images wi h
aces, and nega i e images wi hou aces o ain he classi ie . The way he Haa classi ie wo ks is
desc ibed in igu e 3.4, whe e i mo es h ough an objec de ec o in a se ies o s ages whe e, a each
s age, he classi ie decides i a egion o pixels is posi i e o nega i e. I hen disca ds he nega i e
egions in which he e is no hing o in e es and mo es on o he nex s age. A he end, along wi h
he model, i c ea es a sys em ision objec able o analyze he image ames.
Figu e 3.4 - lowcha desc ibes he Haa cascade classi ie design (Mi al, 2020).
3.2.2. Su ey
The su ey was gene a ed h ough he Google Fo ms applica ion and hen manipula ed o an ini ial
phase h ough Mic oso Excel o clean up da a ela ed o sco ing he emo ions. Thus, simpli ying he
esul s and p epa ing hem o impo in he Jupy e no ebook and Powe BI o u he analysis.
14
This su ey was conduc ed ia con enience sampling in which 1274 pa icipan s esponded o he
su ey om di e en coun ies and a ious age demog aphics. The o de o he ideos was se by he
esea che and he esponses we e all ea ed in a syn hesized manne .
The pu pose o he su ey is o se e as a compa a i e elemen in he s udy be ween he sco ing o
he ML model o he emo ions de ec ed h ough ideo ames and he human emo ions ha a e
collec ed a e iewing he ideos h ough he ques ion ha uses a Like scale om 1 o 7 o eco d
he a ousal and in ensi y o he emo ion el by he iewe . This ques ion is displayed in igu e 3.5.
Figu e 3.5 - he su ey g id o collec emo ions om ideos wa ched.
The esul s collec ed h ough he su ey we e manipula ed in an Excel ile in o de o pe o m he i s
s eps o cleaning he da a be o e impo ing i in o he Powe BI pla o m and pe o ming u he
explo a o y da a analysis echniques and in e ing esul s om i .
In addi ion o he sco ing eco ded by he Like scale, o he ques ions we e asked in o de o ge a
comp ehensi e unde s anding o he s a e o emo ions expe ienced, i s con ex , he easons behind
i , and he ela ionship o he iewe wi h he use o social media ideo con en .
The su ey empla e is a ached in he Appendix A sec ion o e e ence.
15
3.3. EMPIRICAL STUDY
The i s pa o his s udy was o analyze he emo ional esponse h ough an ML model and ex ac
emo ions in e ms o he ame coun ha exis s in each ideo. To achie e hese esul s, we c ea ed a
sepa a e i ual en i onmen ha includes ele an packages and lib a ies such as CV2, Ke as, Pandas,
Numpy, and Ma plo lib.
In he no ebook o analysis, we i s loaded he emo ional ecogni ion model and he ace de ec ion
model. To analyze he ideos men ioned in he da ase , a cons uc ed loop block b eaks down he
ideo in o ames and p ocesses each one using p e- ained ace de ec ion o ex ac he acial
ea u es, hen p edic s he emo ion using an emo ion ecogni ion model, and hen s o es he emo ion
wi h he associa ed ame in a Pandas da a ame, which will be u he manipula ed o analysis.
Se e al s eps we e aken o achie e his esul :
1. The model con e s he ame o g ayscale o as e p ocessing.
2. I uses hen he Haa cascade classi ie o de ec aces in he ame.
3. A loop will un h ough he aces ha exis in each ame and p edic he emo ions ou o
hem.
4. I adds he emo ional analysis o he da a ame.
5. A e i uns h ough each ame, he ideo ile is eleased, and all windows a e des oyed.
So now we ha e a da a ame ha has all he ames ead om he ideo wi h an emo ion associa ed
wi h hem. Howe e , he e is a huge edundancy p oblem ha needs o be sol ed since mul iple simila
emo ions we e de ec ed ac oss di e en ideo ames. To o e come his, we c ea ed ano he da a
ame ha g ouped he ames by he emo ions and added hem in a column called ame coun s.
La e , he column ame was d opped om he o iginal da a ame, and we me ged he wo o hem
in o he emo ion’s column. Finally, we ese he index o make 'Emo ion' a column again, which gi es
a inal da a ame as shown in Table 3.2, eady o be plo ed and expo ed o be used in Powe BI o
u he analysis. This p ocess was epea ed o all six ideos.
Table 3.2 - Example o a inal s a e o p ocessed da a ame.
Emo ions
F ame Coun
Ang y
66
Disgus
1
Fea
17
Happy
205
Neu al
100
Sad
46
Su p ise
2
16
To be e isualize he esul s, we used he Ma plo lib lib a y as exp essed in igu e 3.6 o see he
p esence o emo ions in ela ion o he ame coun s. This plo helps as well in compa ing he indings
wi h he su ey esul s, which we will discuss la e .
Figu e 3.6 - plo o he emo ional analysis da a ame.
The same analysis p ocess was i e a ed h ough all six ideos, esul ing in a ying esul s on emo ion
ecogni ion. The esul s a e plo ed in igu e 3.7.
23
Figu e 4.4 - Ma ix o engagemen me ics om he su ey s he pla o m.
Wha is wo h men ioning in his ma ix is he sha ing wi h iends and ollowe s' engagemen me ic
eco ded by he su ey esponden s. This me ic, as we saw in he ypology o ad sha e s in he
li e a u e e iew, is one o he impo an beha io al esponses ha ma ke s a e looking o in e e y
ad e isemen campaign. F om he su ey esul s, he ideo wi h he mos sha es was he ou h ideo,
which was anked 5 h in likes om he pla o m s a s. I we go back o he compa ison o he emo ions
clus e ed in he column cha , we can no ice ha bo h happy and sad emo ions a e dominan in he
su ey eco ds and he ML model p edic ion. This con as in emo ions shows an insigh ha i could
be one o he d i ing ac o s o he quali y engagemen o he use s. A ideo ha con ains bo h
con as ing emo ions could also mean ha he e could be an a c o emo ions in he ideo. Ei he he
ideo s a s om a sad scena io in o a happy ending, o he opposi e s a s om a happy scena io in o
a d ama ic melancholic ending.
The second-mos sa ed ideos a e bo h he i s and he i h. Looking back a he emo ional
compa ison, we can see ha i we elimina e he neu al emo ion om he cha om bo h ideos, we
can see ha he op h ee emo ions om bo h ideos would be happy, su p ise, and ang y. Two o
hese emo ions a e cha ac e ized by high in ensi y emo ion as we ha e seen be o e om he Russel
Ci cumplex model o he 2D emo ional analysis. This p o es ha a ideo ha con eys a message
h ough in ense emo ions ca ches he a en ion o iewe s and causes hem o engage in he mos
desi able beha io , sha ing. So, we can also say ha an in ense momen , acial exp ession, o ideo
e ec ha could display an in ense emo ion is also a d i ing ac o in a o able engagemen .
O he insigh s in o he d i ing ac o s o engagemen s could be gained om he demog aphic and
beha io al analysis done h ough he su ey da a collec ed ia a se ies o ques ions.
The mos a o able ime o consume social media con en is du ing b eaks and commu ing, which is
conside ed killing ime. Bo h accoun o 79%. Thus, choosing a con enien ime o display he
ma ke ing campaigns h ough social media should be ca e ully chosen and all unde one o hese
imes: la e mo ning, mid-day, o ea ly e ening. This will gi e mo e exposu e o he con en and
con ibu e o i s i ali y and engagemen . The esul is shown in Figu e 4.5.

24
Figu e 4.5 - able ep esen s imes o consume media con en .
Ano he me ic ha was eco ded ia he su ey esponses was he ype o desi able con en . Figu e
4.6 shows he pe cen age o pa icipan s esponses ega ding he ype o con en . 82.42% o
esponden s a e d awn o sho , en e aining ideos. This inding also suppo s he emo ional
esponses collec ed and p edic s ha he happy and su p ise emo ions we e p esen in all success ul
ideos, as well as he deg ee o hei in ensi y.
Figu e 4.6 - s acked ba cha o desi able con en .
The las me ic measu ed was exposu e o media con en . Figu e 4.7 demons a es he anking and
pe cen ages o he a ibu es and hei exposu e le el. The dominan one is he ecommenda ion o
he pla o m. This a ibu e is ha d o measu e since i depends on he algo i hm o he pla o m, which
is no open sou ce and changes ac oss ime. So, by omi ing his choice, we a e le wi h a dominan
pe cen age o 12.63% ha belongs o sha ing me hods bo h h ough ins an messaging and he ac i i y
o hei social ne wo k, compa ed o 6.59% ha comes om ollowing speci ic c ea o s. This insigh
also p o es ha he sha ing beha io discussed is he mos a o able due o i s abili y o sp ead
con en , inc ease exposu e, and esul in a success ul ma ke ing campaign.
25
Figu e 4.7 - s acked ba cha o exposu e o new con en .
We saw p e iously ha he mos desi able con en eco ded by use s was en e aining ideos, wi h a
high pe cen age o 82%. Thus, in an a emp o au oma e he p edic ion o emo ions so ha ma ke e s
could un pilo es s o hei campaigns and unde s and hei emo ional impac on engagemen ,
Figu es 4.8 and 4.9 ou line he incen i es o engagemen ha ma ch he emo ions de ec ed by he
model unning h ough he es ideos.
Figu e 4.8 – Reasons o engagemen by emo ions p edic ed 1
26
Figu e 4.9 – Reasons o engagemen by emo ions p edic ed 2
The esul s show ha ac ually en e aining iends o ollowe s is he mos sough -ou eason o
engage, as eco ded h ough all he emo ions p edic ed. Howe e , we see a spike in his ype o
engagemen in he Happy and he Su p ise emo ions. These wo emo ions we e e y dominan in he
es ideos, which can be jus i ied by he niche o ideos he con en c ea o chosen o his es made.
As a coun e example, he six h ideo was supposed o aise awa eness o a aile o a new mo ie.
Despi e he ac ha i had amazing audio and isual e ec s and he in ol emen o celeb i ies and
in luence s, i aced he lowes a es and numbe s o engagemen ( e e o Table 3.1 o s a is ics).
This could be in e p e ed as he con en c ea o ying o aise awa eness and d i e his ype o
engagemen by using he w ong emo ions o suppo i in his ideo campaign, esul ing in an
engagemen ailu e.
The second mos desi able incen i e o engagemen is o connec o ne wo k wi h o he s who sha e
simila in e es s. This was high in he happy and su p ise emo ions bu also in he sad and ang y
emo ions. Bo h o hese emo ions show di e en in ensi ies o opposi e alences. Hence, d ama izing
emo ions in sho ideos will push he use s o engage by sha ing wi h hei ne wo k, esul ing in a
wild i e e ec and desi able esul s o he ma ke ing campaigns.
I is impo an o ac o all o hese nuances in o he c ea ion o a ma ke ing campaign. The acial
ecogni ion and emo ional p edic ion models de eloped and used by OpenCV, along wi h he da a
modeling and dashboa ding, will c ea e an ecosys em o so wa e ha applies sen imen analysis
me hodology and simula es use beha io al and emo ional esponses ia an unde s anding o hei
emo ional expe ience owa ds a ious ideo-cen ic ma ke ing campaigns. Enabling an au oma ed and
dynamic p ocess ha acili a es he making and es ing o ma ke ing campaigns wi h an emphasis on
he emo ional aspec o hem and he engagemen hey d i e.
27
5. CONCLUSIONS AND FUTURE WORKS
Th oughou his s udy, we ound ha he emo ional esponse o ideo o ma media con en can
in luence engagemen a es and beha io s owa ds i . Thus, unde s anding he emo ions gene a ed
by media con en is essen ial in designing and es ing a ideo-cen ic ma ke ing campaign. This s udy
used a sen imen analysis app oach o pe o m a compa a i e analysis be ween Haa classi ie emo ion
ecogni ion models and su ey-gene a ed esponses owa ds di e en ideo media con en s. Then,
om he p edic ions, modeling he da a gene a ed and isualizing he esul s using dashboa ding
so wa e.
The emo ions p edic ed om he acial de ec ion and emo ional ecogni ion models show up in
di e en ways in he es ideos. One way o exp essing p esence is o ha e wo opposing emo ions
ha occu in he o m o an a c, p e e ably elling a s o y, as in, s a wi h one and inish wi h ano he
in he opposi e alence. This me hod o u ilizing emo ions igge s an engaging beha io in he iewe ,
and e y o en i is he sha ing beha io ha is mos desi able by campaign leads. Ano he way is o
d ama ize he p esence o he emo ions by inc easing hei in ensi y in he ideo while playing wi h
di e en pa ame e s o con en cha ac e is ics.
O he ac o s a e also key in d i ing engagemen s, such as exposu e o media con en and choosing
he igh iming a which a la ge po ion o use s will be online and wai ing o diges he con en . An
example o his would be du ing wo k o s udy b eaks and in commu e hou s, as su ey da a shows. In
addi ion o ha , ocus on he igh ype o con en ha use s p e e ably consume and end o sha e
h ough hei pages o ins an messaging. The mos dominan one om his s udy was en e ainmen ,
whe e he ocus was on he posi i e and in ense emo ions, which we e also eco ded he mos by ou
machine lea ning models and su ey esponden s.
5.1. THEORETICAL CONTRIBUTION AND BUSINESS IMPLICATIONS
The con ibu ion o his s udy s ems om i s unique combina ion o sen imen analysis, su ey insigh s,
machine lea ning models, and a ocus on op imizing ma ke ing campaigns h ough pe sonalized
emo ional engagemen .
This s udy con ibu es o he exis ing li e a u e by p o iding a new and di e en me hod o analyze
social media ideos using sen imen analysis. This s udy uniquely in eg a es sen imen and emo ion
analysis o digi al con en wi h su ey da a collec ed di ec ly om use s. While exis ing li e a u e may
ocus on one o hese aspec s o use a iangula ion me hod as an app oach o hei mul imodal
sen imen analysis owa ds media con en (Ba dzell, Ba dzell, & Pace, 2009). The syn hesis o bo h
p o ides a mo e comp ehensi e unde s anding o use emo ions and p e e ences.
By inco po a ing machine lea ning models o emo ion p edic ion, his s udy ex ends he li e a u e on
emo ion ecogni ion in digi al con en . The compa ison o model-p edic ed emo ions wi h use -
epo ed emo ions p o ides aluable insigh s in o he eliabili y and accu acy o such models, which
could be explo ed u he o imp o e hei accu acy o ind new ways o disco e pa e ns and
co ela ions be ween he con en a ibu es o media con en and he emo ions i igge s.
Fu he mo e, his s udy has implica ions o designing, es ing, and ad e ising ma ke ing campaigns.
Insigh s in o how use s disco e and engage wi h con en p o ide aluable in o ma ion o imp o ing
28
con en disco e y me hods. This can lead o enhancemen s in ecommenda ion algo i hms, sea ch
unc ionali ies, and o e all pla o m usabili y. Also, le e aging emo ional analysis o in o m ma ke ing
s a egies p o ides a compe i i e edge. B ands ha can connec wi h hei audience on an emo ional
le el a e mo e likely o build las ing ela ionships and loyal y, di e en ia ing hemsel es om
compe i o s.
The s udy also p omo es da a-d i en decision-making ha helps ma ke e s make in o med decisions
o op imize hei campaigns. Unde s anding when use s a e mos emo ionally engaged wi h con en
allows o he scheduling o campaigns a op imal imes. This can lead o highe isibili y, as con en is
mo e likely o be no iced and sha ed du ing pe iods o peak emo ional engagemen . Ma ke ing e hics
a e also implica ed he e. A oiding con en ha e okes nega i e emo ions o exploi s emo ions in an
une hical manne is essen ial o main aining b and us and in eg i y.
5.2. LIMITATIONS AND FUTURE RESEARCH
E e y esea ch p ojec comes wi h limi a ions ha allow u u e esea che s o exploi hese gaps, build
on op o he exis ing li e a u e, and imp o e he quali y o hei wo k and insigh s.
One o he limi a ions o his s udy was he uncon olled en i onmen in which he su ey was
conduc ed. Due o he online na u e o he su ey, we could no de e mine o pe o m an ini ial es
o assess he s a ing emo ional condi ion o each pa icipan , and hus an emo ional baseline was
ha d o de e mine. Pa icipan s come om di e en emo ional backg ounds, which could cloud hei
cogni i e app aisal and in luence he esponse hey would gi e o he media con en .
I is also ha d o exp ess emo ions, especially when hey a e in e ela ed. In he case whe e I p oposed
he su ey in pe son o a pa icipan , a lack o unde s anding o hei sel -emo ional s a e was e y
clea o ce ain media con en . Which in u n causes andomness while choosing a sco e o gi e o
he emo ions.
Ano he limi a ion ha was ine i able since he s a o he s udy was he lack o esou ces, mainly he
du a ion o he s udy and he choice o echnologies. The use o comme cial so wa e comes wi h a lo
o suppo om he manu ac u e and is eady o deploy APIs and code, while open sou ce lacks some
documen a ion on ce ain speci ic issues. Then he e is he hea y ask o de eloping an algo i hm ha
in eg a es he model in o he local ecosys em and ge s alid esul s ou o i . The ime cons ain was
also a no iceable limi a ion, since a he easibili y s age mo e da a was p ojec ed o be collec ed om
pa icipan s, which would allow mo e accu a e esul s om he compa a i e analysis. In addi ion o
inding dead ends wi h he coding pa and he choice o echnologies o be implemen ed.
This s udy is no comp ehensi e. The ield o opinion mining is as and keeps on g owing a a as pace
wi h he apid echnological de elopmen ha occu s. Rega ding he con inui y o his esea ch,
se e al poin s could be imp o ed, such as:
Analyzing he p ose e iews and aligning he esul s wi h he ML model, su ey esponses, and
pla o m s a is ics. This ques ion om he su ey was in ended o ex analysis, so as o implemen a
mul imodal app oach o opinion mining (see Appendix B ques ion 13 o e e ence). The idea was o
analyze he images om he ideo ames and p edic he emo ions. Then, by using Py hon and he
suppo ec o machine algo i hm (SVM), we analyze he ex ex ac ed om he p ose e iews. The
SVM me hod analyzes ex by using a unc ion ha con e s he wo ds in o ec o s in nume ic o m.

29
The ou pu is i ed in o an SVM pipeline. (Rao, Ahuja, Kansa a, & Pa el, 2021) The emo ional sco e
would hen be a a io o ex analyzed and ideo analyzed wi h a pe cen age ha is based on hei
accu acy. Then a e age hem based on he numbe o clips analyzed. This me hod will add a suppo ing
modal o ensu e he accu acy o he emo ions p edic ed.
Ano he u u e wo k would be o es hese esul s wi h di e en emo ional ecogni ion models and
compa e model esul s wi h su ey esul s. So a , we only used he model de eloped by b-i Bonn-
Aachen In e na ional Cen e o In o ma ion Technology and Hochschule Bonn-Rhein-Sieg. This model
had an accu acy o 70%, which is a he high compa ed o o he models bu s ill low in e ms o ge ing
i e u able esul s.
.
30
6. BIBLIOGRAPHICAL REFERENCES
Ab aham Youse . (2023, Ap il 01). The La es T ends on Social Media Apps - Q1 2023. Récupé é su
senso owe : h ps://senso owe .com/blog/ he-la es - ends-on-social-media-apps-q1-2023
Bal usai is, T., Zadeh, A., Lim, Y. C., & Mo ency, L. P. (2018). OpenFace 2.0: Facial beha io analysis
oolki . In 2018 13 h IEEE In e na ional Con e ence on Au oma ic Face & Ges u e Recogni ion
(FG 2018) (pp. 59-66). IEEE
Ba dzell, S., Ba dzell, J., & Pace, T. (2009). Unde s anding A ec i e In e ac ion: Emo ion, Engagemen ,
and In e ne Videos. IEEE, 1-8.
DOLAN, B. (2023, Feb ua y 09). wha is Tik ok? Récupé é su in es opedia:
h ps://www.in es opedia.com/wha -is- ik ok-6826240
D'SOUZA, D. (2023, Oc obe 22). TikTok: Wha I Is, How I Wo ks, and Why I ’s Popula . Récupé é su
in es opedia: h ps://www.in es opedia.com/wha -is- ik ok-
4588933#:~: ex =TikTok%20allows%20use s%20 o%20make, he%2018%E2%80%9324%20ag
e%20 ange.
Ekman, P. (1992). An A gumen o Basic Emo ions. Cogni ion and Emo ion, 169-200.
F esh Essays. (Feb ua y 2023). Psychological Senso s o Facial Exp ession Recogni ion. Re ie ed om
h ps://samples. eshessays.com/psychological-senso s- o - acial-exp ession- ecogni ion.h ml
Geyse , W. (2023, Ap il 27). The Ul ima e Guide o Sho -Fo m Video Con en . Récupé é su
in luence ma ke inghub: h ps://in luence ma ke inghub.com/sho - o m- ideo-con en /
Gup a, S. (2022, 01 06). How o apply Min-Max No maliza ion o you da a. Récupé é su Medium:
h ps://sou abha sh.medium.com/how- o-apply-min-max-no maliza ion- o-you -da a-
976d1633d2b
Gup a, Y., Aga wal, S., & Singh, P. (2020). TO STUDY THE IMPACT OF INSTAFAMOUS CELEBRITIES ON
CONSUMER BUYING BEHAVIOR. Academy o Ma ke ing S udies Jou nal Volume 24, 1528-
2678.
Henkel, I. (2023, Oc obe 11). Azu e AI Video Indexe documen a ion. Récupé é su lea n.mic oso :
h ps://lea n.mic oso .com/en-us/azu e/azu e- ideo-indexe / ideo-indexe -use-apis
Huang, M., Yang, Y., & Zhu, X. (2020). Sho Video Unde s anding: Challenges, Me hodology, and
Applica ions. IEEE T ansac ions on Mul imedia, 23, 6.
Kulka ni, K., Ka lo, A., Sha ma, D., & Sha ma, P. (2019). A ypology o i al ad sha e s using sen imen
analysis. Jou nal o Re ailing and Consume Se ices, 10.
Liu, B. (2012). Sen imen Analysis and Opinion Mining . Mo gan & Claypool Publishe s.
Mi al, A. (2020, 12 20). Haa Cascades, Explained. Récupé é su Medium:
h ps://medium.com/analy ics- idhya/haa -cascades-explained-
31
38210e57970d#:~: ex =A%20Haa %20classi ie %2C%20o %20a,in%20an%20image%20and%
20 ideo
Oc a io, A., Ploge , P. G., & Ma ias, V. (2019). Real- ime Con olu ional Neu al Ne wo ks o Emo ion
and Gende Classi ica ion. B uges: Eu opean Symposium on A i icial Neu al Ne wo ks,
Compu a ional In elligence.
Oli ei a, T., A aujo, B., & Tam, C. (2020). Why do people sha e hei a el expe iences on social
media? Tou ism Managemen 78, 1-14.
OpenCV. (2023, 10 30). Cascade Classi ie . Récupé é su docs.openc :
h ps://docs.openc .o g/3.4/db/d28/ u o ial_cascade_classi ie .h ml
OpenCV eam. (2023). Abou . Récupé é su openc : h ps://openc .o g/abou /
Palalic, R., Ramadani, V., Gilani, M., Ge gu i-Rashi i, S., & Dana, L.–P. (2021). Social media and
consume buying beha io decision: wha en ep eneu s should know? Managemen
Decision, 1249-1270.
Po ia, S., Camb ia, E., Haza ika, D., Majumde , N., Zadeh, A., & Mo ency, L. P. (2017). A e iew o
a ec i e compu ing: F om unimodal analysis o mul imodal usion. In o ma ion Fusion, 37,
98-125.
Posne , J., Russel, J. A., & Pe e son, B. S. (2005). The ci cumplex model o a ec : An in eg a i e
app oach o a ec i e neu oscience, cogni i e de elopmen , and psychopa hology.
De elopmen and Psychopa hology, 715-734.
Raine Lienha and Jochen Mayd . An ex ended se o haa -like ea u es o apid objec de ec ion. In
Image P ocessing. 2002. P oceedings. 2002 In e na ional Con e ence on, olume 1, pages I–
900. IEEE, 2002.
Rambocas, M., & Pacheco, B. (2018). Online sen imen analysis in ma ke ing esea ch: a e iew.
Jou nal o Resea ch in In e ac i e Ma ke ing, 146-163.
Rao, A., Ahuja, A., Kansa a, S., & Pa el, V. (2021). Sen imen Analysis on Use -gene a ed Video, Audio
and Tex . In e na ional Con e ence on Compu ing, Communica ion, and In elligen Sys ems
(ICCCIS) (p. 5). Mumbai: IEEE Xplo e.
Rod iguez, A. (2022, Decembe 01). In e ne illusionis Zach King b eaks eco d o mos iewed ideo
on TikTok. Récupé é su guinnesswo ld eco ds:
h ps://www.guinnesswo ld eco ds.com/news/2022/12/in e ne -illusionis -zach-king-b eaks-
eco d- o -mos - iewed- ideo-on- ik ok-724834
SAMBARE, M. (2020, 10 16). FER-2013. Récupé é su kaggle:
h ps://www.kaggle.com/da ase s/msamba e/ e 2013/da a
Sch eine , M., Fische , T., & Riedl, R. (2019). Impac o con en cha ac e is ics and emo ion on
beha io al engagemen in social media: li e a u e e iew and esea ch agenda. Elec onic
Comme ce Resea ch, 17.
32
Singh, A. (2023, Augus 7). The Rise o Sho -Fo m Video Con en : Reshaping Digi al Media
Consump ion. Récupé é su Linkedin: h ps://www.linkedin.com/pulse/ ise-sho - o m- ideo-
con en - eshaping-digi al-media-aas ha-singh/
Viola , P., & Jones, M. (2004). Robus eal- ime ace de ec ion. In e na ional Jou nal o Compu e
Vision, 137–154.
Wes , T. (2011). Going Vi al: Fac o s Tha Lead Videos o Become In e ne Phenomena. B oadcas
Jou nalism and In e na ional S udies, 76-84.
Zadeh, A., Po ia, S., Camb ia, E., & Hussain, A. (2018). Mul imodal sen imen analysis in he wild. In
Mul imodal Beha io Analysis in he Wild (pp. 135-155). Sp inge .
39