scieee Science in your language
[en] (orig)

A study on contextual influences on automatic playlist continuation

Abstract

Recommender systems still mainly base their reasoning on pairwise interactions or information on individual entities, like item attributes or ratings, without properly evaluating the multiple dimensions of the recommendation problem. However, in many cases, like in music, items are rarely consumed in isolation, thus users rather need a set of items, selected to work well together, serving a specific purpose, while having some cognitive properties as a whole, related to their perception of quality and satisfaction, under given circumstances. In this paper, we introduce the term of playlist concept in order to capture the implicit characteristics of joint music item selections, related to their context, scope and general perception by the users. Although playlist consumptions may be associated with contextual attributes, these may be of various types, differently influencing users' preferences, based on their character and emotional state, therefore differently reflected on their final selections. We highlight on the use of this term in HybA, our hybrid recommender system, to identify clusters of similar playlists able to capture inherit characteristics and semantic properties, not explicitly described in them. The experimental results presented, show that this conceptual clustering results in playlist continuations of improved quality, compared to using explicit contextual parameters, or the commonly used collaborative filtering technique.

Read accessible full text

A study on contextual influences on automatic playlist continuation

Author: Gkatzioura, Anna,Sànchez-Marrè, Miquel,Jorge, Alípio Mário
Year: 2018
DOI: 10.3233/978-1-61499-918-8-156
Source: https://upcommons.upc.edu/bitstream/2117/131202/1/paper38-vdef.pdf
A S udy on Con ex ual In luences on
Au oma ic Playlis Con inua ion
Anna GATZIOURA a,1, Miquel SÀNCHEZ-MARRÈaand Alípio Má io JORGE b,c
aKnowledge Enginee ing & Machine Lea ning G oup (KEMLG)
In elligen Da a Science and A i icial In elligence Cen e (IDEAI)
Uni e si a Poli ècnica de Ca alunya (UPC), Ba celona, Spain
bLabo a o y o A i icial In elligence and Decision Suppo (LIAAD)
INESC TEC, Po o, Po ugal
cFaculdade de Ciências da Uni e sidade do Po o (FCUP), Po o, Po ugal
Abs ac . Recommende sys ems s ill mainly base hei easoning on pai wise in-
e ac ions o in o ma ion on indi idual en i ies, like i em a ibu es o a ings, wi h-
ou p ope ly e alua ing he mul iple dimensions o he ecommenda ion p oblem.
Howe e , in many cases, like in music, i ems a e a ely consumed in isola ion, hus
use s a he need a se o i ems, selec ed o wo k well oge he , se ing a speci ic
pu pose, while ha ing some cogni i e p ope ies as a whole, ela ed o hei pe -
cep ion o quali y and sa is ac ion, unde gi en ci cums ances.
In his pape , we in oduce he e m o playlis concep in o de o cap u e he
implici cha ac e is ics o join music i em selec ions, ela ed o hei con ex , scope
and gene al pe cep ion by he use s. Al hough playlis consump ions may be asso-
cia ed wi h con ex ual a ibu es, hese may be o a ious ypes, di e en ly in lu-
encing use s’ p e e ences, based on hei cha ac e and emo ional s a e, he e o e
di e en ly e lec ed on hei inal selec ions. We highligh on he use o his e m in
HybA, ou hyb id ecommende sys em, o iden i y clus e s o simila playlis s able
o cap u e inhe i cha ac e is ics and seman ic p ope ies, no explici ly desc ibed
in hem. The expe imen al esul s p esen ed, show ha his concep ual clus e ing
esul s in playlis con inua ions o imp o ed quali y, compa ed o using explici
con ex ual pa ame e s, o he commonly used collabo a i e il e ing echnique.
Keywo ds. hyb id ecommende sys ems, au oma ic playlis con inua ion, con ex ual
dimensions, case-based easoning, la en opic models
1. In oduc ion
Music i ems a e a ely consumed in isola ion bu a he as sequences aiming o c ea e a
pa icula a mosphe e [15]. Mo e speci ic, playlis s a e se s o music i ems designed o be
consumed as a sequence, wi h speci ic p ope ies as a whole, simila o adi ional adio
b oadcas s [3].
The e o e, in playlis ecommenda ions, and simila domains, mo e han ecom-
mending isola ed i ems, o p esen ing an o de ed lis o he mos p omising al e na i es,
1Co esponding Au ho : Anna Ga ziou a, Knowledge Enginee ing & Machine Lea ning G oup (KEMLG),
In elligen Da a Science and A i icial In elligence Cen e (IDEAI), Uni e si a Poli ècnica de Ca alunya
(UPC), C/Jo di Gi ona 1-3, 08034, Ba celona, Spain; E-mail: [email p o ec ed].
like he majo i y o ecommende sys ems (RSs) do, he unde lying s uc u e o join i em
selec ions should be e alua ed, as i em in e ac ions wi hin a se may hea ily in luence
he esul [19]. The p esence o an i em wi hin a conc e e concep should be cap u ed,
in o de o ecommend se s o i ems, add essing quali y ela ed a ibu es, like cohe ence
and di e si y, while being ele an o he playlis ’s pu pose and c ea ion momen [18].
Popula ecommenda ion echniques, like collabo a i e il e ing (CF) and con en -
based (CB), mainly use he use -i em ma ix o explode pas in e ac ions and p edic
he sui abili y o an i em o a use , and no he sui abili y o an i em o a pa icula
concep . Mo e speci ic, CF echniques a e domain independen , ocusing only on use
a ings on i ems, while CB app oaches, when applied o music base hei analysis mainly
on sound ela ed a ibu es [4]. They nei he explo e he join i em selec ions no he
ci cums ances unde which hose we e pe o med. Ne e heless, use s may cons uc
di e en playlis s unde di e en ci cums ances, no always made o hei a ou i e o
he mos popula i ems. Thus, simply p edic ing i a song o an a is would be liked
by a use , wi hou e alua ing he whole concep , is no enough and usually esul s in
lowe pe o mance, especially in domains whe e use s pe o m a lo o ansac ions.
Fu he mo e, he in luence e en o e y simila con ex ual si ua ions, on use s’ music
pe cep ion and p e e ences, has been ound o hea ily depend on he use ’s cha ac e and
emo ional s a e [12] ha need o be cap u ed app op ia ely.
1.1. Mo i a ion
The mo i a ion o his wo k a ises om he lack o RSs o e icien ly ecommend se s
o i ems ha would i wi hin a s a ed concep , while add essing addi ional seman ic
cha ac e is ics, like cohe ence and di e si y, simila o au oma ic playlis con inua ion
(APC).
Mo e speci ic, we e alua e he in luence o explici and implici con ex ual dimen-
sions on playlis con inua ion ecommenda ions. HybA, a hyb id RS o APC, has been
designed wi h aim o gene a e ecommenda ions o “se s” o music i ems, no ela ed o a
speci ic use , bu o a speci ic concep . This RS uses Case-Based Reasoning (CBR) wi h
en i e playlis s modelled as cases, o iden i y hei s uc u es, combined wi h a La en
Di ichle Alloca ion (LDA) opic model, o cap u e he i ems’ s yles appea ing wi hin
di e en concep s. In his pape , we highligh on he way ha implici con ex ual ac o s
a e used in his RS o iden i y addi ional playlis cha ac e is ics. We emphasize mainly
on he e ie al s ep o he CBR cycle and complemen ou p e ious wo k [8]. The basic
con ibu ions o his wo k, can be summa ized as:
•The e m playlis concep , ex ending he explici con ex , is in oduced.
•We p esen an app oach o cap u ing implici playlis con ex h ough i s concep .
•We e alua e he abili y o explici and implici con ex ual ac o s o iden i y clus-
e s o simila playlis s, and hei sui abili y o he designed hyb id RS.
The es o he pape is s uc u ed as ollows: in he nex sec ion an o e iew o he
backg ound on music consump ion and he ela ed con ex ual dimensions can be ound.
Following, HybA is p esen ed, wi h emphasis on he e m o playlis concep and i s use
in he sys em. Finally, he e alua ion esul s, showing ha he p oposed concep ual il e -
ing is able o be e cap u e playlis simila i ies and p o ide imp o ed ecommenda ions,
compa ed o he use o explici con ex and use based CF echniques, can be ound.
2. Backg ound
2.1. Music Consump ion
When e e ing o music i ems hose can be songs,gen es,a is s,albums and adio
s a ions. The e o e, music ecommenda ions can be add essed a di e en le els o ab-
s ac ion [18]. As music i ems esul om a complica ed syn hesis p ocess, hei analy-
sis in e ms o con en cha ac e is ics, equi es deepe domain knowledge. In addi ion,
songs a e a ely lis ened o in isola ion [14]. Use s a he c ea e playlis s/sessions, being
sequences o songs, placing mo e impo ance on he songs and hei ela i e o de [19].
Playlis s con ain he no ion o i em sequences and se cha ac e is ics, while being
highly a ec ed by he in en and he con ex in which hey we e gene a ed and consumed
[11]. In addi ion, music is well known o e oke emo ions while a he same ime use s’
music needs a e in luenced by hei ac ual emo ional si ua ion [7]. Howe e , he e is
s ill a lack o solid me hods combining use s’ cogni i e pe cep ion o music wi h sound
cha ac e is ics, he e o e i becomes e en mo e di icul o cap u e hei pe cep ion o
a playlis and speci y he cha ac e is ics ha a “good” playlis should ha e. This no-
ion can be highly subjec i e, depending on pa ame e s like he use ’s music knowledge,
p e e ences, pe sonali y, emo ional s a e, con ex and in en [19].
As cu en ly mo e and mo e online si es ei he inco po a e some music ep oduc ion
in o hei en i onmen , o ocus pu ely on p esen ing music se s, au oma ic playlis gen-
e a ion (APG) and ecommenda ion has eme ged as among he in e es ing issues in he
music ecommenda ion domain. APG e e s o he au oma ic c ea ion o sequences o
music i ems based on some a ge cha ac e is ics. On he o he hand, au oma ic playlis
con inua ion (APC) which is a a ia ion, o sub-case, o APG, consis s in adding a se
o music i ems o a playlis in a way ha i would ma ch i s ini ial a ge cha ac e is ics.
The e o e, APC consis s in he selec ion o he mos app op ia e music i ems, and he
cons uc ion o a sequence o imp o ed quali y acco ding o cha ac e is ics in e ed om
a s a ed playlis [3,19].
Mo e p ecisely, gi en a s a ed lis , he aim is ecommending se s o songs able o
comple e i , while p o iding a mo e exci ing use expe ience. These ecommenda ions
a e gene a ed based on he cha ac e is ics o he s a ed lis , in e ms o music s yles
and a is a ie y, independen ly o he use who made i . The e o e, mo e han simply
p edic ing whe he a music i em would be highly a ed, he unde lying s uc u e o join
selec ions should be e alua ed, in o de o ecommend se s o songs sa is ying a he same
ime ele ance, and o he beyond accu acy dimensions, like cohe ence and di e si y [18].
2.2. Con ex ual Fac o s
Inco po a ing in o he ecommenda ion p oblem con ex ual in o ma ion ela ed o he
ecommenda ion momen , inc eases he inpu da a dimensions o h ee, namely use s,
i ems and con ex . Howe e , when co ec ly cap u ed, music con ex , may lead o a sig-
ni ican inc ease in ecommenda ion accu acy [13].
When e e ing o music, con ex , ini ially de ined as any “in o ma ion desc ibing
whe e you a e, whom you a e wi h, and wha esou ces a e nea by” [20], is mapped o
he use ’s si ua ion when consuming he music i ems, in e ms o ime, mood, ac i i y
and o he people’s p esence. I can be ca ego ized acco ding o se e al c i e ia, as:
•Fully obse able, pa ially obse able and unobse able
•P ima y and seconda y
•En i onmen al and use ela ed
Whe e p ima y and en i onmen al dimensions (loca ion, ime and wea he ) can be used
o de i e he seconda y and use ela ed ones (ac i i y, mood, social and cul u al) [6,11].
A con ex -awa e ecommenda ion p ocess e e s o he es ima ion o use con ex ual
p e e ences, and based on hose, on he gene a ion o he mos ele an ecommenda-
ions. Depending on he pa o he easoning p ocess ha he con ex ual in o ma ion is
aken in o accoun , i can be desc ibed as [1]:
•Con ex ual p e- il e ing: selec ion o he ele an da a, ha will be u he used
o he ecommenda ions’ gene a ion, based on a speci ic con ex .
•Con ex ual pos - il e ing: a ecommenda ion se is gene a ed om he en i e da a
se and is hen adjus ed based on he con ex ual in o ma ion o he ac i e use .
•Con ex ual modelling: con ex ual pa ame e s a e inse ed in o he ecommenda-
ion model o a e used o ans o m he i ems in o a di e en dimension.
Gillho e and Schedl [10] analyze he ela ionships be ween a ious con ex ual di-
mensions o iden i y whe he hose pe mi he accu a e p edic ion o use lis ening p e -
e ences. They e alua e hei in luence on he di e en ca ego ies o music i ems, namely
songs, gen es, a is s and mood. De ice, ask, wea he and ime ha e been ound as he
mos impo an a ibu es, being able o cap u e almos he same in o ma ion as all con-
ex ual ca ego ies when combined. Howe e , due o he da a spa si y in music ecom-
menda ion p oblems, s ill when ecommending songs he pe o mance is e y low, while
when i comes o gen e o a is p edic ions, he addi ional con ex ual in o ma ion indeed
imp o es p edic ion accu acy. In ou p e ious wo k [9], we ha e also ound ha e al-
ua ing he playlis c ea ion ime (hou o he day) leads o imp o ed ecommenda ion
accu acy o a is s and gen es.
As explici ly de ined con ex ela ed o playlis s may be o di e en ypes, o di -
e en ly e lec ed on use s’ selec ions, Pichl e al. [16] p opose he use o playlis names
o implici ly ex ac con ex ual in o ma ion. They c ea e con ex ual clus e s based on
playlis names, ha a e hen inco po a ed in o he ecommenda ion p ocess. Howe e ,
he e iciency o his app oach hea ily depends on he da a quali y, as highly subjec i e
names like “my a ou i e”, “bes ”, e c., lead o spa se clus e s ha a e no aluable o
he ecommenda ion p ocess.
3. Recommenda ion App oach
The designed RS aims o gene a e ecommenda ions o se s o i ems able o comple e
he ac i e use ’s expe ience, based on he ypes o i ems appea ing in simila concep s.
HybA aims o add ess simila seman ic concep s, a he han simila use s and elies on
he basic CBR idea, ha “Simila p oblems ha e simila solu ions” [17]. An o e iew
o i s easoning p ocess, ha ollows he gene al CBR Cycle, is shown in igu e 1.
Gi en a new playlis , HybA i s compa es he al eady selec ed i ems wi h hose in
he playlis s s o ed in he case base and e ie es he kmos simila o hem. Based on he
i ems ound in hose, weigh ed by he simila i y deg ee o he playlis (s) in which hey
appea wi h he new one, he con inua ion is cons uc ed and ecommended.
Figu e 1. HybA Reasoning and Recommenda ion Model
The p oblem en i ies can be desc ibed as:
•a se o songs I={i1,...,iz}
•a se o p e iously ep oduced playlis s L={l1,...,lk}whe e each can be w i en
as he se o songs ha i consis s o , being lj={ij1,...,ijn},ij ∈I, =1,...,n
•a se o use s U={u1,...,u } ha ha e o med hose playlis s
Each o hese en i ies may be associa ed wi h addi ional cha ac e is ics, like me a-
da a o edi o ial in o ma ion, empo al o con ex ual da a, p e e ences and demog aphic
cha ac e is ics, espec i ely, used o compu e hei simila i y deg ees, like in [8].
3.1. Playlis Gene al Concep
Aplaylis is a collec ion o music i ems, ep oduced as a meaning ul sequence ha
should ha e some special cha ac e is ics as a whole [3]. We p opose he ca ego iza ion
o playlis s’ cha ac e is ics in o:
•In e nal (o con en ela ed): he exac music i ems ha a playlis consis s o . The
s yles o hose i ems (which may be songs, a is s o gen es), de ine o a g and
ex end he cha ac e is ics o a playlis , like i s cohe ence o di e si y le el.
•Ex e nal (o en i onmen al ela ed): he pa ame e s ela ed o he playlis bu no
di ec ly desc ibed in i . Fo ins ance, he c ea ion pu pose, con ex , e c., and he
in luence o hose ac o s on he use ’s emo ional s a e and music pe cep ion.
Al hough no always explici ly e lec ed in e ms o o mula ion, he con ex , and
o he ex e nal pa ame e s ela ed o a playlis , ha e been ound o in luence i s s yle and
he i ems inally placed in i [10]. Howe e , as music pe cep ion is highly subjec i e, and
may be hea ily a ec ed by ci cums ances unde which is consumed, he pe sonali y and

he emo ional s a e o he use , i is ha d o es ablish a di ec connec ion be ween use s
and hei p e e ed playlis s unde a gi en con ex . Ne e heless, i has been ound [7]
ha depending on hei cha ac e , use s pe o m di e en music selec ions e en unde
simila ci cums ances. Fo ins ance, mo e ex o e people end o ely on “happy” music
unde sad o s ess ul si ua ions, in o de o “chee ” hemsel es up, while in o e s end
o lis en o “sad” o “dep essi e” music simila o hei emo ional s a e.
The e m playlis concep is in oduced as a wide e m o desc ibe he gene al cha -
ac e is ics o a playlis , being he esul o he combina ion o bo h in e nal and ex e nal
pa ame e s, beyond explicil y de ined con ex . Fu he mo e, as playlis s a e o mula ed
o se e a speci ic seman ic concep , hey a e ea ed as dis ibu ions o e music s yles.
The e o e each playlis can be w i en as lj={sj1,...,sjn}whe e each sj is he music
s yle o he song ij . In o de o cap u e he gene al concep o playlis s, wi hou ocusing
on hei speci ic con en , o when no addi ional in o ma ion, like hei names o explici ly
de ined con ex , is a ailable, he playlis s’ la en opic dis ibu ion, based on he music
s yles in hem, is p oposed. Thus, be o e analysing he playlis -song dis ibu ions, i s
he playlis -music s yles dis ibu ions a e iden i ied, wi h aim o cap u e he endencies
and pa e ns p esen in he playlis s, a he han inding he exac songs.
Figu e 2. Playlis s’ la en dis ibu ions o e music s yles
Ha ing playlis s desc ibed as dis ibu ions o e music s yles, a La en Di ichle Al-
loca ion (LDA) opic model [2] is buil o ex ac he unde lying la en opics, as in igu e
2. Playlis s a e hen cha ac e ized by hei dominan opic(s) as a ep esen a ion o hei
gene al concep , closely ela ed o hei cen al idea and cogni i e pe cep ion. We ha e
ound his a ibu e o be capable o cap u ing he gene al endencies in playlis s, and
hei simila i ies, and o imp o e ecommenda ion accu acy.
3.2. Candida es Re ie al
An impo an p ocess o a CBR sys em ha highly a ec s i s pe o mance, is i s abili y
o iden i y he cha ac e is ics o a new case, and e ie e, wi hin an accep able ime, he
pas cases ha seem as mos app op ia e o i s solu ion. The e o e, in o de o iden i y
globally accep able candida es o a new playlis , while educing he compu a ional ime,
i s a p ope clus e ing and p e- il e ing o he case base is pe o med.
Especially when applied o la ge scale da ase s he selec ion o he pa ame e s ha
mos ly cha ac e ize he playlis s and enable hei p ope clus e ing, is o high impo ance.
To his di ec ion, a con ex ual p e- il e ing based on he playlis gene a ion momen (hou
o he day) was ini ially es ed, and was ound o p o ide imp o ed esul s [9]. Howe e ,
as many imes con ex ual in o ma ion is no explici ly desc ibed, o e en i so i may
a ec di e en ly use s’ selec ions, a mo e gene al clus e ing would be mo e app op ia e.
The e o e, we ha e ex ended he con ex ual o a concep ual p e- il e ing, e alua ing i s
playlis s’ concep ual simila i ies.
Mo e speci ic, based on he music s yles’ dis ibu ion in a new playlis being lN=
{sN1,...,sNn}, i s opic dis ibu ion, speci ying i s concep , is iden i ied. Then he se
o ins ances wi h he same dominan opic,LC⊆L, ha add ess he same concep , a e
ini ially e ie ed and hen used o u he compu a ions, inally leading o he e ie al
o he kmos simila pas playlis s.
Gi en a e ie ed playlis lR={sR1,...,sRm},lR∈LC, i s simila i y Sim(lN,lR)wi h
lN, e e ed o as global simila i y, is compu ed as a unc ion o he local simila i y le els
o he i ems in hem. HybA inally e ie es he kmos simila playlis (s), hose whose
global simila i y ul ils equa ion (1), and cons uc s he playlis con inua ion using he
songs ha ini ially appea ed in hem, as in [8].
l0∈LC:∀lR∈LC,l0=a gmax{Sim(lN,lR)}(1)
4. E alua ion
In his sec ion we p esen g aphically he compa a i e esul s o HybA, using he p o-
posed concep ual clus e ing and p e- il e ing, wi h he use o h ee di e en con ex ual
clus e ings and a CF app oach, on h ee eal music da ase s.
Mo e speci ic, o a new playlis , cn xT, uses he clus e LHo playlis s gene a ed
in close hou s, cn xM, he clus e LMo playlis s ep oduced in close mon hs and cn xC
e alua es bo h he day hou and he mon h when he playlis was made. Fu he mo e, a
use -based CF2using Euclidean simila i y and he 10 nea es neighbou s o each use
has been es ed.
The da abases used o e alua ion come om he Po uguese music social ne wo k
Palco P incipal3and con ain in o ma ion on use s adding acks o hei lis ening sessions
o hei playlis s, on a gi en momen (e en ). Mo e de ails can be ound in able 1.
Da ase Palco Lis en1 Palco Lis en2 Palco Playlis s
E en s 1171849 295044 111942
Songs 29786 22986 26117
Use s 21815 5543 10392
Playlis s 86174 22108 22132
Table 1. Palco P incipal da ase s
To e alua e bo h ecommenda ion accu acy and quali y we ha e used he ecom-
menda ions’ a e age p ecision and cohe ence. Cohe en playlis s a e gene ally consid-
e ed as o “be e quali y”, while he di e si y and no el y deg ee ha a use enjoys may
be subjec o his/he ac ual music p e e ences [5,19].
•P ecision is he a io o co ec ecommenda ions: he missing i ems ha we e
success ully iden i ied, o e he o al numbe o ecommenda ions, calcula ed as:
P ecision =#Rele an RecommendedI ems/#RecommendedI ems
2h ps://mahou .apache.o g/use s/ ecommende / ecommende -documen a ion.h ml
3h p://palcop incipal.com/
•Cohe ence e alua es he homogeni y o a playlis R, and can be calcula ed as he
a e age simila i y (sim) among he pai s o consequen i ems in he lis , being:
Cohe ence =1
|R|−1∑
i∈R
sim(i,i+1)
Figu e 3. Recommenda ions’ p ecision and cohe ence on Palco Lis en1 da ase
Figu e 4. Recommenda ions’ p ecision and cohe ence on Palco Lis en2 da ase
Figu e 5. Recommenda ions’ p ecision and cohe ence on Palco Playlis s da ase
As i can be seen om igu es 3–5, CF shows a low pe o mance due o i s sligh ly
di e en scope, as i does no ake in o accoun any addi ional in o ma ion ela ed o
join i em selec ions. In addi ion, in he majo i y o cases including in o ma ion on i ems’
join selec ions and basic ime con ex , imp o es he esul s o e he CF app oach, while
going beyond explici con ex leads o highe imp o emen s. Mo e speci ic, he p oposed
concep ual clus e ing and p e il e ing p o ides an impo an imp o emen o bo h ec-
ommenda ion p ecision and cohe ence.
5. Conclusions
Al hough ecen ly con ex -awa e me hods ha e gained space among ecommenda ion
echniques, s ill hey ocus mainly on explici ly de ined con ex ual pa ame e s. Howe e ,
con ex may be o di e en ypes, di e en ly e lec ed in use s’ beha iou , especially in
domains ela ed o emo ional esponses, like in music consump ion.
In his pape we ha e p esen ed he e m o playlis concep , o implici ly cap u ing
he con ex o playlis gene a ions. This e m is used o cha ac e ize playlis s, cons uc ed
unde gi en ci cums ances, based on he music s yle combina ions in hem. Fu he mo e,
i has been ound o be e iden i y simila playlis s, compa ed o explici con ex ual
dimensions, and o p o ide imp o ed APC ecommenda ions compa ed wi h he widely
used CF app oach. As pa o ou u u e wo k, he exploi a ion o he la en concep s wi h
aim o pe o m a p ope “ ansla ion”, enabling he explana ion o he esul ing playlis
clus e s, will ake place.
Re e ences
[1] Gediminas Adoma icius and Alexande Tuzhilin. Con ex -awa e ecommende sys ems. In Recom-
mende sys ems handbook, pages 217–253. Sp inge , 2011.