Ci a ion: Pape i, S e ano, Hanna
Jä eläinen, and Fede ico Fon ana.
2023. Design and Assessmen o
Digi al Musical De ices Yielding
Vib o ac ile Feedback. A s 12: 143.
h ps://doi.o g/10.3390/
a s12040143
Academic Edi o s: Jus in Pa e son
and Ma celo M. Wande ley
Recei ed: 14 Janua y 2023
Re ised: 2 May 2023
Accep ed: 1 June 2023
Published: 7 July 2023
Copy igh : © 2023 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
a s
A icle
Design and Assessmen o Digi al Musical De ices Yielding
Vib o ac ile Feedback
S e ano Pape i 1,* , Hanna Jä eläinen 1and Fede ico Fon ana 2
1
Ins i u e o Compu e Music and Sound Technology, Zu ich Uni e si y o he A s, 8005 Zu ich, Swi ze land
2Depa men o Compu e Science, Ma hema ics and Physics, Uni e si y o Udine, 33100 Udine, I aly
*Co espondence: [email p o ec ed]
Abs ac :
Touch has a pi o al impo ance in de e mining he exp essi i y o musical pe o mance
o a numbe o musical ins umen s. Howe e , mos digi al musical de ices p o ide no in e ac i e
o ce and/o ib a o y eedback o he pe o me , hus dep i ing he soma osenso y channel o a
numbe o cues. Is he lack o hap ic eedback only an aes he ic issue, o does i emo e cues essen ial
o digi al ins umen playing? I so, a which le el is he in e ac ion objec i ely impo e ished? Wha
a e he e ec s on musical pe o mance? In his su ey a icle we illus a e ou ecen esea ch abou
he use o ib o ac ile eedback in h ee digi al ins umen in e aces, using ools ha we de eloped
o e se e al yea s and made a ailable o he communi y in open-sou ce o m. These in e aces span
a wide ange o amilia i y and ges u al oppo uni ies, enabling us o explo e he impac o hap ic
eedback on di e en ypes o digi al ins umen s. We conduc ed expe imen s wi h p o essional
musicians o assess he impac o ib a o y cues on bo h he pe cei ed quali y o he ins umen and
he playing expe ience, as well as on musical pe o mance. Pa icula a en ion was paid o scien i ic
igo and epea abili y o he esul s, so as o se e as a e e ence o esea che s and p ac i ione s
o he musical hap ics communi y. Ou esul s sugges a signi ican ole o ib o ac ile eedback in
shaping he pe cep ion o digi al musical ins umen s, al hough he e ec s on musical pe o mance
a ied depending on he in e aces es ed.
Keywo ds:
ib a ion; musical ins umen ; digi al musical in e ace; quali y pe cep ion; pe o mance
assessmen ; musical p ac ice
1. In oduc ion
Du ing ins umen al pe o mance, musicians a e exposed o audi o y, isual, and also
soma osenso y cues. This mul isenso y expe ience has been s udied since long ago Camp-
bell (2014); Coch an (1931); Galembo and Asken el (2003); Hodges e al. (2005); Palme
e al. (1989); Tu ne (1939); howe e , he speci ic in e ac ion be ween sound and ib a ions
in musical ins umen playing has only been he objec o sys ema ic esea ch since he
1980s Asken el and Jansson (1992); Fon ana e al. (2017); Keane and Dodd (2011); Ma shall
(1986); Sai is (2013); Suzuki (1986); Wollman e al. (2014), wi h an inc easing ecogni ion
o he p ominen ole o ac ile and o ce eedback cues in he complex pe cep ion–ac ion
mechanisms in ol ed O’Modh ain and Gillespie (2018). Mo e ecen ly, esea ch on he
soma osenso y pe cep ion o musical ins umen s has been consolida ed, as es i ied by
he eme ging “musical hap ics” opic Pape i and Sai is (2018a). This g owing in e es is
in pa d i en by he inc eased a ailabili y o accu a e senso s capable o eco ding ouch
ges u es and ib a o y cues, and a o dable and e icien ac ua o s ende ing ib a ions
o o ce. Touch senso s and ac ua o s can be employed i s o in es iga e he join ole
o he audi o y and soma osenso y modali ies in he pe cep ion o musical ins umen s,
and hen o ealize no el musical in e aces and ins umen s, building on he lessons o
he p e ious in es iga ion Ma shall and Wande ley (2006); O’Modh ain and Cha e (2000);
O e hol e al. (2011). Th ough his p ocess, iche o e en uncon en ional eedback cues
A s 2023,12, 143. h ps://doi.o g/10.3390/a s12040143 h ps://www.mdpi.com/jou nal/a s
A s 2023,12, 143 2 o 25
can be con eyed o he pe o me , wi h he goal o enhancing engagemen and imp o -
ing he ini ial accep abili y and playabili y o he new musical ins umen Bi nbaum and
Wande ley (2007); Fon ana e al. (2015); Michailidis and Bullock (2011); Young e al. (2017).
In his pe spec i e, se e al musical hap ics s udies aspi ed, on he one hand, o
design nex -gene a ion digi al musical in e aces yielding hap ic eedback Pape i and
Sai is (2018b), and, on he o he hand, o assess i and how hap ic eedback is ele an o he
pe cei ed quali y o musical ins umen s, o he pe o me ’s expe ience and pe o mance,
and o he esul ing musical ou come Fon ana e al. (2018); Sai is e al. (2018); Schmid (2014);
Young e al. (2018).
Among adi ional musical ins umen s, s udies ha e mainly ocused on he hap ic
esponse o he piano and he iolin, gene ally inding ha hap ic cues a e ele an o
hei pe cei ed quali y, and may be e en mo e impo an han audi o y eedback o hei
iden i ica ion Fon ana e al. (2017); Galembo and Asken el (2003); Keane and Dodd (2011);
Wollman e al. (2014).
While he in e es on hap ic human–compu e in e aces has g own exponen ially in
ecen yea s, he ansi ion om a human–compu e o a pe o me –digi al ins umen
in e ac ion o en equi es a shi o he a en ion on di e en se s o design equi emen s
and aspec s. In addi ion, musicians es ablish an in ima e physical connec ion wi h hei
ins umen s, and ha e a heigh ened sensi i i y o he hap ic eedback hey ecei e om
hem. As a esul , c ea ing e ec i e machine in e ac ions wi h musicians in ol es much
mo e han simply ailo ing he design con ex o music. I equi es implemen ing sensing
and ac ua ion echnology wi h a high deg ee o accu acy and eliabili y, capable o econ-
ex ualizing he in e ac ion in a way ha enables making a . Achie ing his equi es an
unde s anding o he complex in e play be ween echnical aspec s and a is ic conside -
a ions, and a mul idisciplina y app oach ha d aws on expe ise om a ange o ields,
including enginee ing, musicology, and human–compu e in e ac ion. Only by aking his
holis ic app oach can we c ea e digi al musical ins umen s ha o e a uly exp essi e
and engaging hap ic expe ience o musicians.
While looking a his pe asi e o e lap be ween accu a e physical in e ac ion and
c ea i e con ex , his pape summa izes esul s p oposed by he au ho s o e ecen yea s
in o m o design, applica ion, and e alua ion o a ious digi al musical in e aces Fon ana
e al. (2015); Jä eläinen e al. (2022); Pape i e al. (2019,2021). In con as o mos p e ious
s udies on he opic, ou esea ch aimed o achie e a high le el o scien i ic igo , wi h he
goal o p oducing a solid ounda ion o empi ical e idence ha can be eplica ed by o he
esea che s and p ac i ione s. In pa icula , we employed ela i ely la ge sample sizes and
mode n s a is ical analysis echniques, and implemen ed s ic con ol o e expe imen al
condi ions. Addi ionally, we de eloped measu emen ools and conduc ed se e al psy-
chophysical expe imen s o in es iga e ib a ion pe cep ion unde ac i e ouch condi ions
commonly ound in musical pe o mance De P a e al. (2021,2022); Pape i e al. (2017),
o en ob aining adically di e ging esul s when compa ed o he exis ing li e a u e Asken-
el and Jansson (1992); Ve illo (1992). Finally, we ensu ed accu a e measu emen and
cha ac e iza ion o he hap ic eedback p o ided by ou in e aces, on he one hand o
comply wi h psychophysical esul s, and on he o he hand o enable he in e p e a ion o
expe imen al esul s based on objec i e quan i ies.
While limi ed o i s speci ic scope, ou esea ch p o ides insigh in o he ole o hap ic
eedback in a b oad domain o musical in e ac ions, co e ing a wide ange o in e ace
designs o digi al sound gene a ion and con ol: Sec ion 2p esen s e idence o he signi -
icance o inco po a ing ib a ions on a amilia class o in e aces as uni e sal as digi al
piano keyboa ds; Sec ion 3ex ends hose indings o a less amilia , bu mo e exp essi e,
mul idimensional musical in e ac ion scena io h ough o ce-sensi i e mul i ouch su aces;
Sec ion 4 es ic s he ocus o a speci ic musical con ol ask such as pi ch-bending, so as o
in es iga e he e ec s o hap ic eedback on pe o mance accu acy. Finally, hese a gumen s
a e discussed oge he in Sec ion 5.
A s 2023,12, 143 3 o 25
The au ho s hope ha , due o he scope o he p oposed in e ac ions and he e o
o p esen hei cha ac e is ics in a cohe en de elopmen , his su ey helps disambigua e
wha pe ains o a —on which we will no da e o claim any esul de i ing om ou
wo k—and wha ins ead conce ns science and echnology. Such a disambigua ion is
necessa y, in ou opinion, o econcile, a he han di ide, a and science wi hin he
musical hap ics domain.
2. Augmen ing he Digi al Piano wi h Vib a ion
When conside ing digi al musical in e aces augmen ed wi h hap ic eedback, he
piano ep esen s an especially ele an case s udy, no only o i s impo ance in he his o y
o Wes e n musical adi ion, bu also o i s po en ial in he musical ins umen s ma ke
and o he uni e sali y o he keyboa d in e ace—pe haps he mos widely adop ed
musical in e ace, which is s anda d e en on mode n syn hesize s Moog and Rhea (1990).
When playing an acous ic piano, he pe o me is exposed o a a ie y o audi o y, isual,
kines he ic, and ib o ac ile cues ha a e combined and in eg a ed o shape he pianis ’s
pe cep ion–ac ion loop. F om he e, a ques ion abou he ole o soma osenso y eedback in
o ming he pe cep ion o a keyboa d ins umen na u ally a ises. His o ically, he hap ic
p ope ies o he musical keyboa d we e i s ende ed om a kinema ic pe spec i e, wi h
he aim o ep oducing he mechanical esponse o he keys Cadoz e al. (1990); Oboe
and De Poli (2006), also in ligh o expe imen s emphasizing he sensi i i y o pianis s
o he keyboa d mechanics Galembo and Asken el (2003). Only ecen ly—also pas he
lesson lea ned abou he echnological challenges posed by ac i e con ol o he keyboa d
esponse using o ce- eedback de ices—ha e esea che s s a ed o analyze he ole o
ib o ac ile eedback as a po en ial con eyo o salien cues. Indeed, he la e is easie o
implemen in p ac ice, as es i ied by ea ly indus ial ou comes in digi al pianos (e.g., he
Yamaha A an G and se ies), howe e s ill missing documen ed eedback om cus ome s
Guizzo (2010).
An ea ly implemen a ion a emp by some o he p esen au ho s claimed possible qual-
i a i e ele ance o ib o ac ile cues Fon ana e al. (2011). La e , along he same line, g ound
o a subs an ial s ep o wa d was se when he p esen au ho s no only ound signi ican
sensi i i y o such cues, bu also hypo hesized ha pianis s a e sensi i e o key ib a ions
also when hei ampli ude is below he s anda d subjec i e h esholds ob ained by pas-
si ely s imula ing he human inge ip wi h pu ely sinusoidal s imuli Fon ana e al. (2017);
Ve illo (1971). This conclusion only appa en ly con adic s p e ious expe imen s Asken el
and Jansson (1992): In ac , some o he au ho s o his pape demons a ed ha ib o ac ile
sensi i i y h esholds measu ed du ing ac i e inge p essing a e signi ican ly lowe han
p e iously epo ed o passi e ouch Pape i e al. (2017).
In his con ex , an expe imen was se up, on he one hand, o assess he pe cei ed
quali y o a digi al piano depending on di e en ypes o ib o ac ile eedback p o ided a
he keyboa d, and, on he o he hand, o s udy possible e ec s o ib a ion on pe o mance,
namely, accu acy in playing iming and dynamics gi en a e e ence. This esea ch is
epo ed in Sec ion 2.1.
The keyboa d ib a ions ha we e eco ded on a g and piano o he pu pose o
he men ioned expe imen we e la e complemen ed by ib a ion eco dings aken on an
up igh piano and by binau al eco dings aken om he pianis ’s lis ening poin . The
eco ded ma e ial was hen packaged in a lib a y compa ible wi h a popula so wa e
sample , and eleased in open-access o m. The lib a y is desc ibed in Sec ion 2.4.
2.1. The Vib oPiano
The keyboa d o a Viscoun Galileo VP-91 digi al piano was de ached om i s me al
casing, also con aining he elec ic and elec onic ha dwa e, and hen ixed o a hick
plywood boa d (see Figu e 1).
A s 2023,12, 143 4 o 25
Figu e 1. The expe imen al se up used in he Vib oPiano expe imen .
Two Cla k Syn hesis TST239 Sil e Tac ile T ansduce s we e a ached o he bo om o
he wooden boa d, espec i ely, in co espondence wi h he lowe and middle oc a es, in
his way enabling o con ey ib a ions a he mos ele an a eas o he keyboa d Fon ana
e al. (2017). The keyboa d was hen laid on an X-shaped keyboa d s and, in e posing oam
ubbe a he con ac poin s o minimize ib a ion p opaga ion o he loo while a he
same ime educing ene gy dissipa ion and op imizing he o e all ib a o y esponse.
The ansduce s we e d i en by a Yamaha P2700 ampli ie ed by an RME Fi e ace
800 audio in e ace connec ed o a lap op. Sound and ib o ac ile eedback was gene a ed
ia so wa e using he Reape digi al audio wo ks a ion,
1
hos ing an ins ance o he Na i e
Ins umen s Kon ak sample
2
in se ies wi h MeldaP oduc ion MEqualize pa ame ic
equalize 3, as well as an ins ance o he Moda Piano eq 4.5 piano syn hesize 4.
A schema ic o he se up is shown in Figu e 2.
Figu e 2. The Vib oPiano implemen a ion.
A s 2023,12, 143 5 o 25
Thanks o i s physics-based sound engine allowing di ec con ol o se e al acous ic
and mechanical p ope ies, he piano syn hesize was con igu ed o ma ch he sound o
he g and piano used as e e ence (Yamaha Diskla ie DC3 M4). Keyboa d ib a ions
we e eco ded on he same piano (see Sec ion 2.4 o mo e de ails on he p ocedu e), and
hei ep oduc ion on he Vib oPiano was made o spec ally ma ch he o iginal eco ded
ib a ions by uning he pa ame ic equalize as explained below.
Since wo ansduce s a e no su icien o dis ibu ing a uni o m equaliza ion ac oss
he en i e keyboa d ex ension, a la ening cha ac e is ic was a e aged o e equency
esponses compu ed o all he A keys (see Figu e 3).
0 0.5 1 1.5 2
x 104
−200
−150
−100
−50
0
50
100
150
200
F equency (Hz)
Phase (deg ees)
0 100 200 300 400 500 600 700 800 900 1000
−5
0
5
10
15
F equency (Hz)
Magni ude (dB)
Figu e 3. Spec al la ening: a e age equaliza ion cu e.
A his poin , impo an dependencies o he e e ence we e s ill missing, he mos
impo an o which we e loudness ma ching and key eloci y calib a ion. Thanks o he
ac ha he e e ence piano was a Yamaha Diskla ie —an MIDI-complian acous ic pi-
ano equipped wi h senso s o eco ding keys okes and pedaling, and elec omechanical
mo o s o ac ua ing he keys and pedals—i was possible o di ec ly ma ch MIDI eloci y
and he esul ing loudness on he Vib oPiano. Conce ning key eloci y calib a ion, i
mus be no ed ha he keys o he Diskla ie and he Vib oPiano ha e di e en esponse
dynamics because o hei mechanics. Since pianis s adap hei s yle in consequence o
hese di e ences, he digi al keyboa d had o be subjec i ely calib a ed, aiming a equal-
izing i s dynamics wi h ha o he Diskla ie . To his end, he eloci y cu e calib a ion
ou ine a ailable in Piano eq was ollowed by an expe pianis , i s pe o ming on he
Diskla ie and hen on he Vib oPiano. As expec ed, wo ai ly di e en eloci y maps we e
ob ained. Then, he eloci y cu e o he Vib oPiano was p ojec ed poin -by-poin on o
he co esponding map o he Diskla ie : he esul ing key eloci y ans e cha ac e is ics
he e o e allowed playing o he Vib oPiano wi h he dynamics o he Diskla ie keyboa d,
hus ensu ing ha when a pianis played he digi al keyboa d a desi ed dynamics, he
co esponding ib a ion samples eco ded on he Diskla ie would be igge ed. This cu e
was independen ly checked by wo mo e pianis s o alida e i s eliabili y and neu ali y.
Conce ning loudness ma ching, he one p oduced by each A key o he Diskla ie a
a ious eloci ies was eco ded using a KEMAR head posi ioned a he pianis ’s loca ion.
Equi alen ones syn hesized by Piano eq we e eco ded wi h he KEMAR mannequin
wea ing a pai o Sennheise CX 300-II ea phones and, on op o hem, a pai o 3M Pel o X5
ea mu s. Ea mu s, in ac , ensu ed su icien isola ion om sounds coming as a byp oduc
o he ib a ing se up in Figu e 2. A his poin , he loudness o he Vib oPiano was ma ched
o ha o he Diskla ie by using he olume mapping ea u e o Piano eq, which allows
one o independen ly se he olume o each key ac oss he keyboa d.
2.2. Expe imen : Pe cei ed Quali y and Pe o mance
Ele en pianis s in p o essional aining (5F/6M, a e age age 26 yea s) pa icipa ed in a
quali y e alua ion es and in a iming pe o mance and dynamic s abili y es . Vib o ac ile
s imuli we e p oduced by he Vib oPiano. In addi ion o he ib a ions eco ded om he
Diskla ie , a second se o ib a ion signals was syn he ically gene a ed wi h he pu pose
o ep oducing he same empo al en elope o he o me , howe e wi h ma kedly di e en
spec al con en . To his end, whi e noise was gene a ed and hen bandpass- il e ed in he
A s 2023,12, 143 6 o 25
equency ange 20–500 Hz—i.e., he ange o in e es o ib o ac ile pe cep ion Ve illo
(1971). Se e al noise signals we e hen gene a ed—one o each key—by passing hem
h ough a esonan il e cen e ed a he undamen al equency o he espec i e key. The
esul ing signal—ha ing a la empo al en elope—was hen modula ed by he ampli ude
en elope o he co esponding eco ded no e ib a ion, which was es ima ed om he
ene gy decay cu e o he espec i e piano sample. Finally, he ene gy o he syn he ic
ib a ion was equalized o ha o he co esponding eal sample. The wo se s o ib a o y
signals we e loaded as wo dis inc ins ances o he Kon ak plug-in.
Th ee ib a ion condi ions we e assessed in compa ison o a non ib a ing condi ion
labeled A; hey we e
B:
Reco ded ib a ions;
C:
Reco ded ib a ions wi h 9 dB boos ;
D:
Syn he ic ib a ions.
Audi o y eedback was always p o ided by he Piano eq syn hesize , wi h he same
se ings.
In he i s es , he ask was o play eely on he Vib oPiano and assess he playing
expe ience on i e a ibu e a ing scales: dynamic con ol, ichness, engagemen , na u al-
ness, and gene al p e e ence. The dynamics and ange o playing we e no es ic ed in any
way; pa icipan s could swi ch eely be ween A and he cu en es condi ion un il hey
we e eady o make a judgmen . Ra ings we e gi en on a con inuous compa ison ca ego y
a ing scale (CCR), anging om
−
3 ( ib a ing se up much wo se han he non ib a ing
one) o +3 ( ib a ing se up much be e han he non ib a ing one).
In he second es , pa icipan s could swi ch eely only be ween condi ions A and
B. In addi ion, a me onome sound a 120 BPM was deli e ed h ough he ea phones.
Pa icipan s we e asked o play an ascending and hen a descending D-majo scale by
playing a no e a e e y second me onome bea , and o keep cons an dynamics in he
mean ime. The a ionale o using such simple ask was o main ain maximum con ol
o e he expe imen al condi ions. I any sco e o musical signi icance was pe o med
ins ead, he elemen o exp essi i y would come in o play, in insically in oducing indi-
idual a ia ions o dynamics and iming, hus making i di icul o achie e he desi ed
expe imen al goal. Only he h ee lowes oc a es we e conside ed, so as o maximize
ib o ac ile eedback Fon ana e al. (2017), his equi ing each pa icipan o play wi h
hei le hand only. Each pa icipan epea ed he ask wi h h ee dynamic le els (pp,m ,
) h ee imes each in bo h condi ions, o a o al o 18 andomized ials. MIDI da a o
“no e ON”, “no e du a ion”, and “key eloci y” messages we e eco ded ac oss his es
o subsequen analysis. The hypo hesis was ha i he pa icipan s’ iming and dynamic
beha io was a ec ed by key ib a ions, hen di e ences would be seen in means and
s anda d de ia ions o key eloci ies and in e -onse in e als (IOIs).
2.3. Resul s
Resul s om he i s es on pe cei ed quali y a e plo ed in Figu e 4.
On a e age, all ib a ing condi ions we e p e e ed o e A (non ib a ing), he only
excep ion being D (syn he ic ib a ions) o na u alness. Fo condi ions B and C ( eal
and boos ed eal ib a ions), na u alness ecei ed sligh ly posi i e sco es. The s onges
p e e ences we e o dynamic ange and engagemen . Gene al p e e ence and ichness
had e y simila mean sco es, al hough somewha lowe han engagemen and dynamic
con ol. Gene ally, C was p e e ed he mos : i sco ed highes on ou ou o i e scales,
al hough B was conside ed he mos na u al. In e es ingly enough, B sco ed lowes in all
o he scales. The analysis is p esen ed in mo e de ail in Fon ana e al. (2015).
A s 2023,12, 143 7 o 25
−3
−2
−1
0
1
2
3
dynamic.con olengagemen ichness na u alness p e e ence
A ibu e
Ra ing
ib a ion
B
C
D
All subjec s (N=11)
Figu e 4.
Resul s o he quali y expe imen . Boxplo p esen ing median and qua iles o each
a ibu e scale and ib a ion condi ion. Posi i e alues indica e p e e ence o a ib a ing se up o e
he non ib a ing se up.
Segmen a ion pe o med a pos e io i e ealed di e ences be ween pa icipan s: eigh
o hem p e e ed ib a ions and h ee p e e ed he non ib a ing se up. The di e ence
be ween hese g oups is e iden : he median a ings o he mos p e e ed condi ion (C)
we e nea ly +2 in he posi i e g oup and
−
1.5 in he nega i e g oup conce ning gene al
p e e ence. Figu e 5p esen s he di e ences be ween such g oups.
−2
−1
0
1
2
3
dynamic.con olengagemen ichness na u alness p e e ence
A ibu e
Ra ing
ib a ion
B
C
D
Posi i e g oup (N=8)
−3
−2
−1
0
1
2
3
dynamic.con olengagemen ichness na u alness p e e ence
A ibu e
Ra ing
ib a ion
B
C
D
Nega i e g oup (N=3)
Figu e 5. Quali y esul s o he posi i e and nega i e g oups.
O e all, hese esul s show ha key ib a ions inc ease he pe cei ed quali y o a
digi al piano. Al hough he eco ded ib a ions we e pe cei ed as he mos na u al, ampli-
ied na u al ib a ions we e o e all p e e ed and ecei ed he highes sco es on all o he
scales as well. Ano he in e es ing ou come is ha he se up using syn he ic ib a ions was
conside ed in e io o he non ib a ing se up only in e ms o na u alness: his sugges s ha
pianis s a e indeed sensi i e o he ma ch be ween he audi o y and ib o ac ile eedback.
The a ibu e scales wi h he highes co ela ion o gene al p e e ence we e engagemen
(
ρs=
0.75) and ichness (
ρs=
0.72). A simila esul was ob ained in a s udy on iolin
e alua ion, whe e ichness was signi ican ly associa ed wi h p e e ence Sai is e al. (2012).
In he second es , mean key eloci ies we e compu ed o each subjec as he a e age
o e he h ee epea ed uns o each condi ion. Resul s a e p esen ed in Figu e 6.
A s 2023,12, 143 8 o 25
m
pp
0 1 0 1 0 1
54.5
55.0
55.5
56.0
56.5
57.0
57.5
93
94
95
119
120
121
Vib o ac ile eedback
Key eloci y
ibes 0 1
Figu e 6.
Mean key eloci ies o e h ee oc a es in Expe imen 2, wi h 95% CI e o ba s as gi en
in Mo ey (2008).
A he
dynamic le el, subjec s played jus sligh ly loude in p esence o ib a ions
han wi hou ; con e sely, a he
m
le el hey played sligh ly so e . Howe e , a epea ed
measu es ANOVA did no e eal a signi ican e ec o ei he ib a ions (
F(
1, 2826
) =
2.27,
p>
0.05) o he in e ac ion be ween ib a ions and dynamic le el (
F(
2, 2826
) =
0.83,
p>
0.05). No e ec was obse ed by s udying he lowes oc a e alone, whe e he ib a ions
would be el s onges , no was he e a signi ican di e ence in he s anda d de ia ions
be ween condi ions A and B.
IOIs we e, likewise, s able ac oss he wo condi ions. Gene ally, hey we e sligh ly
mo e sca e ed a he
pp
le el; howe e , no e ec o ib a ions was obse ed (see Figu e 7).
0. 1. 0.m 1.m 0.pp 1.pp
0.9 1.0 1.1 1.2
Scale expe imen , IOI
IOI [s]
Vib a ions [0/1] / dynamics
Figu e 7. Boxplo o IOIs in he scale expe imen .
No es du a ion was also s able i espec i e o ib a ions, sugges ing ha he e was no
signi ican di e ence in a icula ion o no e o e lap.
2.4. BiVib
The de elopmen o he Vib oPiano equi ed he eco ding o accu a e keyboa d
ib a ion by means o calib a ed de ices. In addi ion o ha , ou ecen esea ch on
piano sound localiza ion Fon ana e al. (2017,2018) b ough a numbe o piano sound
eco dings aken a he pianis ’s lis ening poin . In an aim o p o ide public access o
such da a, an anno a ed da ase o audio– ac ile piano samples was o ganized as a lib a y
o synch onized binau al piano sounds and keyboa d ib a ions. The goal o sha ing an
open-access lib a y was o os e esea ch on he ole o ib a ions and one localiza ion
A s 2023,12, 143 9 o 25
in he pianis ’s pe cei ed ins umen quali y, as well as adding knowledge abou he
impo ance a cogni i e le el o mul isenso y eedback o i s use in he design o no el
musical keyboa d in e aces.
The BiVib (
Bi
nau al and
Vib
a o y) sample lib a y is a collec ion o high- esolu ion
audio iles (
.wa
o ma , 24-bi @ 96 kHz) con aining binau al piano sounds and keyboa d
ib a ions, along wi h documen a ion and p ojec iles, o hei ep oduc ion h ough a
ee music so wa e sample . The da ase —whose co e s uc u e is illus a ed in Table 1—is
made a ailable h ough an open-access da a eposi o y
5
and eleased unde a C ea i e
Commons (CC BY-NC-SA 4.0) license o di ec playback wi h he Kon ak ( e sion 5 and
abo e) so wa e sample a ailable o Windows and Mac OS sys ems. Fu he di ec ions
on how o load BiVib in Kon ak o use i as a s andalone da ase in he compu e a e
illus a ed in Pape i e al. (2019).
Table 1. BiVib co e s uc u e. Piano lid con igu a ions a e gi en in squa e b acke s.
Diskla ie G and Diskla ie Up igh
Sample se s
(.wa iles)
Binau al [closed]
Binau al [open]
Binau al [ emo ed]
Keyboa d ib a ion
Binau al [closed]
Binau al [semi-open]
Binau al [open]
Keyboa d ib a ion
Sample p ojec s
(Kon ak mul is)
Binau al [closed] + ib a ion
Binau al [open] + ib a ion
Binau al [ emo ed] + ib a ion
Binau al [closed] + ib a ion
Binau al [semi-open] + ib a ion
Binau al [open] + ib a ion
The da ase con ains samples eco ded on wo Yamaha Diskla ie pianos—a g and
DC3 M4 loca ed in Pado a (PD), I aly, and an up igh model DU1A wi h con ol uni
DKC-850 loca ed in Zu ich (ZH), Swi ze land—a se e al dynamic le els o all 88 keys.
The g and piano was loca ed in a la ge labo a o y space (app oxima ely 6
×
4 m), while he
up igh piano was in an acous ically ea ed small oom (app oxima ely 4 ×2 m).
Fo he eco dings sessions, each key was igge ed ia MIDI con ol a 10 le els o
dynamics, chosen be ween MIDI eloci y 12 and 111 by e enly spli ing his ange (i.e., 12,
23, 34, 45, 56, 67, 78, 89, 100, 111): his choice was mo i a ed by a p e ious s udy by he
p esen au ho s, which epo ed ha bo h Diskla ie s p oduced inconsis en dynamics
ou side such eloci y ange Fon ana e al. (2017).
Binau al audio eco dings made use o a dummy head wi h binau al mic ophones o
acous ic measu emen s. The mannequin was placed in on o he piano, app oxima ely
whe e he pianis ’s head is loca ed on a e age (see Figu e 8). The binau al mic ophones
we e connec ed o a p o essional audio in e ace. Th ee con igu a ions o he keyboa d lid
we e selec ed o each piano. The g and piano (PD) was measu ed wi h he lid closed, ully
open, and emo ed (i.e., de ached om he ins umen ). The up igh piano was eco ded
wi h he lid closed, in semi-open posi ion (see Figu e 8), and ully open. The eco ding o
binau al samples was d i en by an au oma ic p ocedu e p og ammed in Supe Collide .
6
The eco ding sessions ook place a nigh ime, hus minimizing unwan ed noise coming
om human ac i i y in he building. On he g and piano, no e leng hs we e de e mined
algo i hmically depending on hei dynamics and pi ch, anging om 30 s when A0 was
played a key eloci y 111, o 10 s when C8 was played a key eloci y 12.
7
These du a ions
allowed each no e o ade ou comple ely, while minimizing silen eco dings and he
o e all du a ion o he eco ding sessions—s ill, each session las ed app oxima ely 6 h.
On he con a y, an undocumen ed p o ec ion mechanism on he up igh piano p e en s
i s elec omechanical sys em om holding down he keys o mo e han abou 17 s, hus
making a comple e decay impossible o some no es, especially a low pi ches and high
dynamics. In his case, o he sake o simplici y, all ones we e eco ded o as long as
possible. Because o he mechanics o piano keyboa ds and he in insic limi a ions o
elec omechanical ac ua ion, a sys ema ic delay is in oduced while ep oducing MIDI
A s 2023,12, 143 16 o 25
wheel, which equi es playe s o dedica e hei le hand o ope a e. All o his makes he
e gonomics o pi ch-bending e y di e en on such de ices, as compa ed o adi ional
musical ins umen s. No el digi al musical in e aces such as hose men ioned abo e ha e
now ein oduced mo e na u al and di ec ways o modula e/bend pi ch, o ins ance,
by sliding up/down a inge on he same su ace ha is being played. Howe e , despi e
he abili y o ine- une mapping sensi i i y, pe o ming such ges u es on digi al de ices
poses a majo challenge o con ol accu acy. This may be due, in pa , o he lack o hap ic
eedback p o ided by mos cu en digi al ins umen s, among o he ac o s Pape i and
Sai is (2018b).
I is known om he li e a u e ha young no mal-hea ing adul s each an audi o y
pi ch disc imina ion accu acy o 0.5% in a wide equency ange, while musicians can
each e en 0.1% Moo e and Pe e s (1992); Spiegel and Wa son (1984). Al hough limi ed
in equency compa ed o he audible ange, ib o ac ile s imuli may also exci e a pi ch
sensa ion which depends on equency and ampli ude (o ene gy) Ha is e al. (2006);
Ve illo (1992). Disc imina ion accu acy be ween 18% and 3% has been epo ed F anzén
and No dma k (1975); Pong ac (2006). Audio– ac ile in e ac ions exis in pe cep ion o
consonance, loudness, and pi ch h ough a ious mechanisms and depending on he
ask Okazaki e al. (2013); Yau e al. (2010), bu i is unclea i and how a noisy ib o ac ile
signal migh dis ac audi o y pi ch con ol pe o mance, o whe he ma ched audi o y
and ib o ac ile signals migh e en enhance i .
In he con ex o no el digi al musical in e aces and ges u e mapping o pi ch con ol,
we se ou o in es iga e he ac ion o pi ch-bending.
4.1. Se up
The expe imen made use o a sel -de eloped hap ic de ice called TouchBox: his
o e s a Plexiglas ouch panel measu ing 3D o ces (i.e., along no mal, longi udinal, and
ans e se di ec ions), and p o ides ich ib o ac ile eedback. The de ice is he la es
i e a ion o a o me design p e iously published in open-access o m Pape i e al. (2019).
As shown in Figu e 13, pa icipan s sa a a able and p essed a inge on op o he
TouchBox, which was placed in on o hem. In o de o com o ably pe o m his, hey
could es hei o ea m on a suppo and adjus he heigh o hei sea . A piece o adhesi e
wi h ine sandpape back was s uck a he cen e o he op panel o he TouchBox, so as o
p e en he pa icipan s’ inge s om slipping while pushing o wa d o pulling backwa d.
Figu e 13. Expe imen al se up o he pi ch-bending expe imen .
Audi o y eedback, gene a ed by a so wa e syn hesize implemen ed wi h Cycling
’74 Max, was p o ided h ough closed-back headphones (Beye dynamic DT 770 PRO)
A s 2023,12, 143 17 o 25
connec ed o a MOTU M4 USB audio in e ace. The syn hesize ep oduced a saw oo h
wa e o m wi h low-pass il e ing.
Vib o ac ile eedback—also gene a ed in Max in he o m o audio signals—was
ende ed by d i ing a oice-coil ac ua o , a ached o he bo om o he ouch panel, ei he
wi h he syn hesize ’s signal o wi h noise. In he o me se ing, he goal was o simula e
wha happens when playing acous ic o elec oacous ic musical ins umen s, whe e he
sou ces o ib a ion and sound coincide. Since humans a e maximally sensi i e o ib a ion
in he 200–300 Hz ange Ve illo (1992), in o de o p o ide uni o mly e ec i e ib o ac ile
eedback, he sound syn hesize was limi ed o a pi ch ange o
±
2 semi ones a ound
C4 = 261.6 Hz; he e o e, he o e all ange was 233.08–293.66 Hz (Bb3–D4). Al hough
na ow, he chosen pi ch ange did no comp omise he pi ch ma ching ask, as human
equency disc imina ion is s able ac oss a wide ange om 200 Hz o se e al kHz Dai and
Micheyl (2011). Fo he noise eedback, whi e noise bandpassed in he 40–360 Hz ange
was used, again wi h he goal o maximizing he pe cei able eedback, his ime lea ing
he audi o y and ac ile channels unco ela ed. Vib a ion ampli ude in bo h con igu a ions
was no malized o 120 dB RMS accele a ion ( e 10
−6m/s2
) so as o be always clea ly
pe cei able Ve illo (1992). Any sound spillage gene a ed by he ac ua o was adequa ely
masked by he headphones wo n by pa icipan s.
A swi ch pedal, connec ed ia MIDI o he audio in e ace, allowed pa icipan s o
eco d he cu en pi ch o he syn hesize and ad ance he expe imen .
Finally, he able housed a compu e sc een and a mouse, which we e used o gi e
a ings, as desc ibed below.
4.2. Expe imen : Accu acy o Pi ch-Bending
The ask was o ep oduce a a ge e e ence pi ch as accu a ely as possible by con ol-
ling a sound syn hesize ia he TouchBox. I was ca ied ou unde wo c ossed ac o s:
ges u e and ib o ac ile eedback. Ges u e had wo condi ions: push o wa d o pull
backwa d he de ice’s op panel. The wo ges u es we e, espec i ely, eques ed o in-
c easing o dec easing he gi en ini ial pi ch owa d he a ge . Th ee ib o ac ile eedback
condi ions we e possible: no ib a ion, noise ib a ion, and pi ched ib a ion ma ching he
audi o y eedback. I ib o ac ile eedback was o e ed in a ial, i was only on while he
syn hesize sound was playing.
Ta ge pi ches we e andomized in a con inuous ange o
±
2 semi ones a ound
C4 = 261.6 Hz. Ini ial pi ches o he s imuli we e one o h ee semi ones abo e o be-
low he a ge pi ch.
The measu ed a iables we e ela i e pi ch accu acy and sel - epo ed a ings on
agency (“I el ha I p oduced he sound”), con idence (“I el ha I ga e a co ec esponse”),
and pleasan ness (“The ask el pleasan ”).
Thi y-one no mal-hea ing, musically ained subjec s ook pa in he expe imen
(14 F and 17 M, a e age age 25 yea s, musical aining: m = 16 yea s). In a p elimina y
phase, hey lea ned o execu e he ask co ec ly and o use one and he same inge o
hei dominan hand h oughou he expe imen . Du ing each ial, a e hea ing he a ge
pi ch o 3 s, pa icipan s adjus ed he s imulus pi ch o ma ch he a ge . T ials we e
o ganized in wo blocks by ges u e in andom o de , and he same ib o ac ile eedback
was p esen ed in a block o ou successi e ials. A e each block o ou ials, a ings on
agency,pleasan ness, and con idence we e gi en by ope a ing isual analog slide s shown on
sc een wi h he mouse, in he ange [0, 1] (0 = e y low, 1 = e y high).
4.3. Resul s
Pi ch adjus men e o (in cen s, whe e cen =
1
100
o a semi one)
16
was p edic ed om
eedback, ges u e, and hei in e ac ion.
S a is ical models we e i o bo h accu acy and a ing da a, and hei espec i e
pa ame e s we e es ima ed by Bayesian in e ence. The models and esul ing i s a e
A s 2023,12, 143 18 o 25
desc ibed in de ail in Jä eläinen e al. (2022). Resul s a e p esen ed in Figu e 14 o he
pi ch accu acy ask.
−0.5
0.0
0.5
0 noise pi ch
Feedback
E o %
ges u e
pull (y−)
push (y+)
−40
−20
0
20
40
0 20 40 60
T ial numbe
E o [cen s]
eedback
0
noise
pi ch
ges u e
pull (y−)
push (y+)
Figu e 14.
Le : es ima ed e o s and espec i e 95% c edible in e al (CI); Righ : accu acy as a
unc ion o ial numbe ( aw da a and linea end).
Sligh ly la ge e o s we e measu ed in he push ges u e. No ably, pa icipan s pe -
o med he ask somewha as e wi h ha ges u e han wi h pulling, wi h mean imes pe
ial being 10.6 s and 12.0 s, espec i ely. Howe e , as Figu e 14 shows, pa icipan s demon-
s a ed sligh ly mo e lea ning wi h pushing, as bo h ges u es app oach 0.5% accu acy in
he las ials, in spi e o la ge di e ences be ween hem a he beginning o sessions.
The di e ences in signed e o s a e indeed ob ious, as he es ima ed e o is an o e -
shoo o 8.74 cen s (+0.53% o he a ge equency) o he pull ges u e and an unde shoo
o
−
12.1 cen s (
−
0.68% o he a ge equency) o push. To in es iga e he signi icance
o he di e ence wi hou conside ing he e ec o he opposi e signs, a second model was
i o e o alues mi o ed a ound ze o in he push condi ion. This e o es ima e was
3.4 cen s highe o he push ges u e; howe e , s a is ical analysis did no p oduce e idence
o a c edible e ec .
Es ima ed e ec s o eedback, ges u e, and a ibu e on mean a ibu e a ings a e
p esen ed in Figu e 15: Noise ib a ion was c edibly a ed lowe , as we e he a ibu es
con idence and pleasan ness; pushing ges u e, in u n, was a ed c edibly highe .
0.80
0.82
0.84
0.86
0.88
0 noise pi ch
Feedback
Es ima e [0,1]
0.825
0.850
0.875
0.900
pull (y−) push (y+)
Ges u e
Es ima e [0,1]
0.76
0.80
0.84
0.88
agency con idence pleasan ness
A ibu e
Es ima e [0,1]
Figu e 15. Es ima ed condi ional e ec s o eedback, ges u e, and a ibu e on sel - epo ed a ings.
5. Discussion
The i s wo expe imen s (see Sec ions 2.3 and 3.3) add essed he quali a i e impac o
ib o ac ile eedback. Thei esul s show ha ca e ully designed ib o ac ile eedback can
enhance he pe cei ed quali y o digi al piano keyboa ds and o ce-sensi i e mul i ouch
su aces. These indings a e consis en wi h p e ious s udies ha examined he quali a i e
e ec s o hap ic eedback in digi al musical in e aces Kalan a i e al. (2017); Ma shall and
Wande ley (2011); Tache e al. (2012).
A s 2023,12, 143 19 o 25
In he case o he Vib oPiano, al hough he ealis ic (i.e., as eco ded) ib a ions we e
pe cei ed as he mos na u al, ampli ied na u al ib a ions we e o e all p e e ed and
ecei ed highes sco es o o he a ibu es as well. Mo eo e , he se up using syn he ic
ib a ions was conside ed in e io o he non ib a ing se up only in e ms o na u alness.
In he case o he HSoundplane, he measu ed e ec o he ib a ion ca ying clea pi ch
and dynamics in o ma ion (i.e., he sine and audio ib a ion) consis en wi h sound was
app eciably posi i e, as compa ed o he non ib a ing condi ion; con e sely, he noise
ib a ion did no enhance he subjec i e quali y o he in e ace. In pa icula , ib o ac ile
eedback inc eased he pe cei ed exp essi eness o he in e ace and he enjoymen o
playing. Ou expe imen s o e all sugges ha ensu ing consis ency be ween ib a ion and
sound in e ms o spec al con en and dynamics is a well- ounded s a egy o he design
o a hap ic digi al music in e ace.
Howe e , in a- and in e indi idual consis ency is an impo an issue in ins umen
e alua ion expe imen s. In he case o he Vib oPiano, oughly wo- hi ds o he subjec s
clea ly p e e ed he ib a ing se up, pe haps less ewa ded by he syn he ic ib a ions,
while he emaining one- hi d had qui e he opposi e opinion. I is wo h obse ing ha
he wo pa icipa ing jazz pianis s we e bo h in he “nega i e” mino i y. While jazz pianis s
a e likely o ha e mo e expe ience o digi al pianos han classical pianis s, who a e ins ead
used o ich ib o ac ile cues Fon ana e al. (2017), he epo ed s udy could no speci ically
assess i pianis s used o s anda d (i.e., non ib a ing) digi al pianos would a e hem be e
han a ib a ing one.
Unde s anding why di e en ypes o ib a o y eedback on he HSoundplane led o
a ying le els o accep ance among pe o me s is mo e challenging. Despi e he c edible
e ec s obse ed in he es popula ion, a numbe o inconsis en esponses we e also
eco ded. One possible explana ion o his a iabili y is indi idual di e ences in sensi i i y
o ib o ac ile s imuli among he pa icipan s. I is possible ha some pa icipan s did
no eel he ib a ions as s ongly o a all du ing some ials. Howe e , since we did no
sc een pa icipan s o ib o ac ile sensi i i y, i is di icul o de ini i ely de e mine he
exac cause o hese inconsis en esponses.
The possibili y ha some pa icipan s had lowe ac ile sensi i i y would also con-
ibu e o explaining he gene ally highe sco es ecei ed by he “boos ed ib a ions” se ing
on he Vib oPiano. A any a e, also aking in o accoun ha a ew pa icipan s (likely
indi iduals wi h high sensi i i y) ound ib a ions on he HSoundplane oo s ong, one can
in e ha ib a ion magni ude is a c ucial ac o in he design o hap ic eedback o digi al
music in e aces. I s ole appea s easie o iden i y (and, hence, o ep oduce) when he
in e ac ion is cons ained wi hin i m ges u al p imi i es, such as hose o e ed by he piano
keyboa d; con e sely, assessing he impac o ib a ion in ensi y becomes mo e challenging
when he in e ace is un amilia and/o a o ds mul idimensional ac ile in e ac ion.
The pe cep ual a iabili y o ib a ion s eng h and audio– ac ile cong uence may
ha e been modula ed by whe e and how he pa icipan s we e playing ac oss he keyboa d
o su ace, hence exace ba ing he numbe o inconsis en a ings. Fo ins ance, sensa ion
magni ude would inc ease when he undamen al equency o he played one o e lapped
wi h he ange o ib o ac ile sensi i i y Ve illo (1992), o i highe p essing o ces we e
applied Pape i e al. (2017). This could ha e been especially ue o he la e in e ace,
which, unlike he piano, p o ides localized ib o ac ile eedback a each pad. Mo eo e ,
ee playing was a necessa y expe imen al condi ion, as i allowed o e alua e he impac
o ib o ac ile eedback on a ious aspec s o he playing expe ience. Cons aining he ask
would ha e limi ed he na u al exp ession and c ea i i y o he musicians, which would
ha e impac ed he alidi y o he s udy.
The hi d expe imen epo ed in his pape (Sec ion 4.2) and, o a limi ed ex en , he
i s one oo (Sec ion 2.2), aimed a assessing how ib o ac ile eedback may suppo some
aspec s o musical pe o mance. As al eady discussed, he ac ile pe cep ion o pe o ming
musicians can be g ea ly in luenced by a ious ac o s, making i challenging o con ol
expe imen ally. Mo eo e , in o de o make pe o mance measu emen s eliable, exp essi e
A s 2023,12, 143 20 o 25
modula ion o playing dynamics and iming had o be a oided. Fo he abo e easons,
he impac o ib a ion on musical pe o mance was s udied by asking pa icipan s o
execu e musical ye basic asks, and by measu ing hei accu acy and s abili y. Howe e ,
his coun e measu e had he unin ended consequence o making he asks oo easy o
ained musicians, who we e ou p ima y pool o pa icipan s. In o he wo ds, sac i icing
pe o mance complexi y o expe imen al con ol aises ques ions abou he powe o ou
pe o mance es s.
On he Vib oPiano, he ask was o play h ee oc a es o a D-majo scale wi h he le
hand a ela i ely slow pace. No di e ences we e obse ed in iming pe o mance and
dynamic s abili y, ega dless o he p esence o ib a ion. Al hough his esul is likely due
o he ask being oo easy o ained pianis s, i is no ewo hy ha he pa icipan s a ed
he pe cei ed dynamic con ol highly o all ib a ion se ings. Hence, i is easonable o
expec ha mo e challenging asks may e eal an e ec o key ib a ions on pe o mance,
besides making he digi al piano mo e enjoyable o play. Indeed, ecen esea ch shows ha
pianis s do use ac ile in o ma ion as a means o iming egula ion Goebl and Palme (2008,
2009), e en hough he exac ole o keyboa d ib a ions emains unknown. The e is also
e idence ha ib o ac ile eedback helps o ce accu acy in inge -p essing asks Ahmaniemi
(2012); Jä eläinen e al. (2013). Whe he ib a ions el on cu en ly dep essed key(s)
could aid in planning ahead, o example, he dynamics o upcoming key p ess, is an
in e es ing ques ion ha he epo ed expe imen could no answe , un o una ely. As
an addi ional no e, du ing he expe imen wi h he HSoundplane we obse ed a ce ain
ade-o be ween execu ion speed and consis ency o esponses, which is ypical o decision-
making asks Hei z (2014). Ne e heless, hese indings alone a e insu icien o con i m
ha ib a ions had a subs an ial impac on he playing ask.
In he hi d expe imen (Sec ion 4), emphasis was on sound con ol a he han p o-
duc ion, speci ically on he accu acy o pi ch con ol h ough inge pushing and pulling.
Though o e all ib a ions had basically no demons able e ec on he ask, he esul s show
ha pi ched ib a ion led o sligh ly lowe e o s, while noise ib a ion caused sligh ly
highe e o s han no ib a ion. These e ec s we e a guably no null; howe e , hei e i i-
ca ion would equi e a la ge -scale expe imen . Addi ionally, noise ib a ion had a c edibly
nega i e impac on all o he a ed a ibu es. The a e age pi ch accu acy in all condi ions
was oughly 0.6%. This is in line wi h pi ch disc imina ion h esholds o complex ha monic
ones o young no mal-hea ing adul s Moo e and Pe e s (1992), al hough musicians can a
bes each a 0.1% accu acy Spiegel and Wa son (1984). A gene al inding was a di e ence
in accu acy be ween pulling and pushing: pulling (app oaching a ge om a highe pi ch)
p oduced, on a e age, +0.53% o e shoo , while pushing (app oaching a ge om a lowe
pi ch) esul ed in
−
0.68% unde shoo . The da a sugges a speed–accu acy ade-o : e en
hough sligh ly less accu a e, pushing ook less ime and was a ed c edibly highe in
agency,con idence, and pleasan ness. Fu he mo e, mean e o s we e equalized owa ds he
end o he session, as pa icipan s demons a ed ha hey lea ned sligh ly mo e by pushing
han by pulling. All his conside ed, pushing migh be mo e e icien in asks ha a e
lea ned and hen pe o med equen ly and equi e as e execu ion, such as con olling
a musical in e ace. I he ask is new o al oge he a e, maximal accu acy seems o be
achie ed by pulling. Once again, since he ask gi en o pa icipan s in he expe imen
did no equi e in ensi e use o hei long-p ac iced musical skills, i is possible ha he
simplici y o he ask i sel p e en ed he measu emen o he e ec o hap ic eedback
on hei pe o mance. A he same ime, he choice o an isola ed simple musical ask
was mo i a ed by a bo om-up app oach, while lea ing open he possibili y o ex end o
ull- ledged musical ges u es in he u u e.
In gene al, a mo i a ion o he lack o measu ed e ec s on pe o mance migh be
ound in he ac ha ou expe imen s in ol ed only highly ained musicians—a choice
mo i a ed by ou p elimina y goal o ob aining eedback om expe s, as well as due o
he a ailable pool o subjec s a he Zu ich Uni e si y o he A s, whe e all expe imen s
we e pe o med. Indeed, expe musicians’ mo o and audi o y skills and memo y may
A s 2023,12, 143 21 o 25
ha e been good enough o allow pe o ming he eques ed asks while comple ely igno ing
any ib o ac ile eedback.
6. Conclusions
The augmen a ion o digi al musical de ices wi h ib a o y eedback has he po en ial
o e-es ablish a consis en physical exchange be ween musicians and hei digi al musical
de ices—simila o wha is na u ally ound on acous ic musical ins umen s, whe e he
sou ce o sound and ib a ion coincides—wi h he demons a ed e ec o enhance he
playing expe ience and he pe cei ed quali y o he in e ace. On op o ha , almos all
pa icipan s in ou expe imen s in o mally epo ed ha ing enjoyed mo e playing he
ins umen s when ib a ion was p esen , highligh ing hei “ali eness”. We ound his
imp ession o be consis en ac oss all o he in e aces used in ou s udy, which oge he
co e ed a ange o exp essi e possibili ies om amilia o explo a i e, and om sound
syn hesis o con ol.
Howe e , i is ye o be seen i and how such subjec i e enhancemen s may be e-
lec ed in he quali y o playing, and musical pe o mance al oge he . Making objec i e
measu emen s o hese aes he ic aspec s, howe e , poses a majo esea ch challenge, and
he epo ed s udies only sc a ched he su ace o his issue.
Gi en he lack o signi ican e ec s o ib o ac ile eedback on pe o mance in speci ic
musical asks, i would be wo h in es iga ing whe he ea ly-s age lea ne s, child en,
o indi iduals wi h, e.g., soma osenso y o audi o y impai men s migh bene i mo e
om ib a o y cues. Fo hese g oups, in ac , i would be easie o iden i y challenging
asks while gua an eeing he necessa y expe imen al con ol. Fu he mo e, mul isenso y
enhancemen is known o be s onges in condi ions whe e unisenso y in o ma ion is
weakly e ec i e Spence (2012). Fo expe musicians—whose pe o mance le el in he
speci ic asks es ed was una ec ed by he p esence o ib a o y cues— hei esilience o
ib a o y dis u bance migh b ing bene i s in ce ain pe o mance si ua ions, such as in
ensemble playing, o when he e is backg ound noise.
The ib o ac ile ende ing s a egies adop ed in he epo ed s udies we e a he basic
by design: his allowed us o highligh which ea u es (e.g., spec um and dynamics) o
ib a o y signals would esul in s onge e ec s. F om a echnological s andpoin , he
abili y o syn hesize simple ib a ions—as long as hey do no dis up he audio– ac ile
cohe ence equi ed o enhance he pe cei ed quali y—may esul in a echnical ad an age
when designing hap ic musical in e aces. Fo example, i is a he i ial o gene a e
sinusoidal signals a audio a e on low-cos embedded sys ems, and in his way, he
e ec i eness o ib o ac ile eedback would be always op imized o endi ion. Mo e
complex app oaches we e ecen ly p oposed in he li e a u e, aiming o en ich he music
lis ening expe ience Me chel and Al insoy (2018); Okazaki e al. (2015)— hese may be
es ed in u u e expe imen s.
The key inding o ou s udy is ha , by ho oughly es ing and e alua ing ac ile
augmen a ions, we can de elop mo e ewa ding musical in e ac ions and po en ially
disco e imp o emen s on musical pe o mance ha a e ye o be explo ed.
Au ho Con ibu ions:
Concep ualiza ion, S.P. and F.F.; me hodology, H.J., S.P., and F.F.; in es-
iga ion, S.P., H.J., and F.F.; so wa e, S.P. and H.J.; o mal analysis, H.J.; esou ces, S.P. and F.F.;
w i ing—o iginal d a p epa a ion, S.P.; w i ing— e iew and edi ing, S.P., F.F., and H.J.; p ojec ad-
minis a ion, S.P. and F.F. All au ho s ha e ead and ag eed o he published e sion o he manusc ip .
Funding:
The esea ch epo ed in his pape has been suppo ed by p ojec AHMI (g an 150107)
and p ojec HAPTEEV (g an 178972), unded by he Swiss Na ional Science Founda ion.
In o med Consen S a emen :
In o med consen was ob ained om all subjec s in ol ed in he s udy.
The s udy was conduc ed in acco dance wi h he Decla a ion o Helsinki, and he p o ocols we e
app o ed by he Swiss Associa ion o Resea ch E hics Commi ees as nonclinical s udies in ol ing
human subjec s when equi ed by na u e o he da a.
A s 2023,12, 143 22 o 25
Acknowledgmen s:
The au ho s wish o hank F ancesco Zanini, Vale io Zanini, And ea Ghi o o,
De id Bianco, Lo enzo Mala ol a, Debo a Scappin, Ma ia Be na di, and F ancesca Minchio, o me
s uden s a he Conse a o y o Padua and Uni e si y o Padua, I aly, o con ibu ing o he piano
esea ch o e he yea s; and Ma in F öhlich, E ic La ieux, Thomas Pe e , and Sébas ien Schiesse a
he Zu ich Uni e si y o he A s o helping wi h he implemen a ion o he ha dwa e and so wa e
sys ems o he HSoundplane and TouchBox.
Con lic s o In e es : The au ho s decla e no con lic o in e es .
No es
1www. eape . m, accessed on 9 June 2023.
2www.na i e-ins umen s.com, accessed on 9 June 2023.
3www.meldap oduc ion.com, accessed on 9 June 2023.
4www.moda .com/piano eq, accessed on 9 June 2023.
5doi.o g/10.5281/zenodo.1213210, accessed on 9 June 2023.
6
A p og amming en i onmen o sound p ocessing and algo i hmic composi ion: supe collide .gi hub.io, accessed on 9 June
2023.
7In ac , no es o inc easing pi ch and/o dec easing dynamics ha e sho e decay imes.
8en.wikipedia.o g/wiki/Lemu _(inpu _de ice), accessed on 9 June 2023
9opensoundcon ol.s an o d.edu, accessed on 9 June 2023.
10 A isual p og amming en i onmen o mul imedia: cycling74.com, accessed on 9 June 2023.
11 inyu l.com/HS-sounds, accessed on 9 June 2023.
12 C edible in e al is he Bayesian equi alen o he con idence in e al in equen is analysis.
13 C edible e ec is he Bayesian equi alen o signi ican e ec in equen is analysis.
14 oli.com/p oduc s/seaboa d/ ise2, accessed on 9 June 2023.
15 www. oge linndesign.com/linns umen , accessed on 9 June 2023.
16 The e o is compu ed as 1200 ·log2( esponse equency
a ge equency ), whe e 1200 ep esen s an oc a e ange in cen s.
Re e ences
Ahmaniemi, Teemu. 2012. E ec o Dynamic Vib o ac ile Feedback on he Con ol o Isome ic Finge Fo ce. IEEE T ansac ions on
Hap ics 6: 376–80. [C ossRe ]
Al insoy, M. E can, and Sebas ian Me chel. 2018. Touchsc eens and musical in e ac ion. In Musical Hap ics. Edi ed by S e ano Pape i
and Cha alampos Sai is. Cham: Sp inge In e na ional Publishing, pp. 239–55.
Asken el , Ande s. 1993. Obse a ions on he ansien componen s o he piano one. STL-QPSR 34: 15–22.
Asken el , Ande s, and E ik V. Jansson. 1992. On ib a ion sensa ion and inge ouch in s inged ins umen playing. Music
Pe cep ion 9: 311–49. [C ossRe ]
Basdogan, Caga ay, F ede ic Gi aud, Vincen Le esque, and Seungmoon Choi. 2020. A e iew o su ace hap ics: Enabling ac ile
e ec s on ouch su aces. IEEE T ansac ions on Hap ics 13: 450–70. [C ossRe ]
Bi nbaum, Da id M., and Ma celo M. Wande ley. 2007. A sys ema ic app oach o musical ib o ac ile eedback. Pape p esen ed a
In e na ional Compu e Music Con e ence (ICMC), Copenhagen, Denma k, Augus 27–31.
Cadoz, Claude, Leszek Lisowski, and Jean-Loup Flo ens. 1990. A Modula Feedback Keyboa d Design. Compu e Music Jou nal 14:
47–51. [C ossRe ]
Campbell, D. Mu ay. 2014. E alua ing musical ins umen s. Physics Today 67: 35–40. [C ossRe ]
Coch an, Ma y. 1931. Insensi i eness o one quali y. The Aus alasian Jou nal o Psychology and Philosophy 9: 131–33. [C ossRe ]
Dai, Huanping, and Ch is ophe Micheyl. 2011. Psychome ic unc ions o pu e- one equency disc imina ion. Jou nal o he Acous ical
Socie y o Ame ica 130: 263–72. [C ossRe ]
De P a, Yu i, S e ano Pape i, Fede ico Fon ana, and Emidio Tibe i. 2021. An open-sou ce obo ic ool o he simula ion o quasi-s a ic
inge p essing on s a iona y and ib a ing su aces. IEEE T ansac ions on Hap ics 14: 273–78. [C ossRe ] [PubMed]
De P a, Yu i, S e ano Pape i, Hanna Jä eläinen, Ma eo Bianchi, and Fede ico Fon ana. 2022. E ec s o ib a ion di ec ion and
p essing o ce on inge ib o ac ile pe cep ion and o ce con ol. IEEE T ansac ions on Hap ics 16: 23–32. [C ossRe ]
Fon ana, Fede ico, Debo a Scappin, Fede ico A anzini, Ma ia Be na di, De id Bianco, and Gio gio Klaue . 2017. Audi o y, isual and
soma osenso y localiza ion o piano ones: A p elimina y s udy. Pape p esen ed a In e na ional Con e ence on Sound and
Music Compu ing (SMC), Espoo, Finland, July 5–8, pp. 254–60.
Fon ana, Fede ico, Fede ico A anzini, and S e ano Pape i. 2018. E idence o la e aliza ion cues in g and and up igh piano sounds.
Pape p esen ed a 15 h In e na ional Con e ence on Sound and Music Compu ing (SMC2018), Limassol, Cyp us, July 4–7,
pp. 80–84.
A s 2023,12, 143 23 o 25
Fon ana, Fede ico, Fede ico A anzini, Hanna Jä eläinen, S e ano Pape i, Gio gio Klaue , and Lo enzo Mala ol a. 2015. Rende ing
and subjec i e e alua ion o eal s. syn he ic ib o ac ile cues on a digi al piano keyboa d. Pape p esen ed a In e na ional
Con e ence on Sound and Music Compu ing (SMC), Maynoo h, I eland, July 26–Augus 1, pp. 161–67.
Fon ana, Fede ico, S e ano Pape i, Hanna Jä eläinen, and Fede ico A anzini. 2017. De ec ion o keyboa d ib a ions and e ec s on
pe cei ed piano quali y. Jou nal o he Acous ical Socie y o Ame ica 142: 2953–67. [C ossRe ] [PubMed]
Fon ana, Fede ico, S e ano Pape i, Hanna Jä eläinen, Fede ico A anzini, and B uno L. Gio dano. 2018. Pe cep ion o ib o ac ile cues
in musical pe o mance. In Musical Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Cham: Sp inge In e na ional
Publishing, pp. 49–72.
Fon ana, Fede ico, S e ano Pape i, Ma co Ci olani, Valen ina dal Bello, and Balázs Bank. 2011. An explo a ion on he in luence
o ib o ac ile cues du ing digi al piano playing. Pape p esen ed a In e na ional Con e ence on Sound Music Compu ing
(SMC2011), Padua, I aly, July 6–9, pp. 273–78.
F anzén, O e, and Jan No dma k. 1975. Vib o ac ile equency disc imina ion. Pe cep ion & Psychophysics 17: 480–84.
Galembo, Alexande , and Ande s Asken el . 2003. Quali y assessmen o musical ins umen s—E ec s o mul imodali y. Pape
p esen ed a ESCOM Con e ence, Hanno e , Ge many, Sep embe 8–13, pp. 441–44.
Gillmeis e , Helge, and Ma in Eime . 2007. Tac ile enhancemen o audi o y de ec ion and pe cei ed loudness. B ain Resea ch 1160:
58–68. . [C ossRe ] [PubMed]
Goebl, We ne , and Ca oline Palme . 2008. Tac ile eedback and iming accu acy in piano pe o mance. Expe imen al B ain Resea ch 186:
471–79. [C ossRe ]
Goebl, We ne , and Ca oline Palme . 2009. Finge mo ion in piano pe o mance: Touch and empo. Pape p esen ed a In e na ional
Symposium on Pe o mance Science, U ech , The Ne he lands, Decembe 15–18.
Guizzo, E ico. 2010. Keyboa d maes o. IEEE Spec um 47: 32–33. [C ossRe ]
Ha is, Jus in A., Ehsan A abzadeh, Ad ienne L. Fai hall, Clai e Beni o, and Ma hew E. Diamond. 2006. Fac o s a ec ing equency
disc imina ion o ib o ac ile s imuli: Implica ions o co ical encoding. PLoS ONE 1: e100. [C ossRe ]
Hei z, Richa d P. 2014. The speed-accu acy adeo : His o y, physiology, me hodology, and beha io . F on ie s in Neu oscience 8: 150.
[C ossRe ]
Hodges, Donald A., W. Da id Hai s on, and Jona han H. Bu de e. 2005. Aspec s o Mul isenso y Pe cep ion: The In eg a ion o Visual
and Audi o y In o ma ion in Musical Expe iences. Annals o he New Yo k Academy o Sciences 1060: 175–85. [C ossRe ]
Jä eläinen, Hanna, S e ano Pape i, and E ic La ieux. 2022. Accu acy o musical pi ch con ol h ough inge pushing and pulling.
In Hap ic and Audio In e ac ion Design. Edi ed by Cha alampos Sai is, Ilda Fa kha dino and S e ano Pape i. Cham: Sp inge
In e na ional Publishing, pp. 125–34. [C ossRe ]
Jä eläinen, Hanna, S e ano Pape i, Sebas ien Schiesse , and Tobias G osshause . 2013. Audio- ac ile eedback in musical ges u e
p imi i es: Finge p essing. Pape p esen ed a In e na ional Con e ence on Sound and Music Compu ing (SMC2013), S ockholm,
Sweden, July 30–Augus 3.
Jones, Randy, Pe e D iessen, And ew Schloss, and Geo ge Tzane akis. 2009. A Fo ce-Sensi i e Su ace o In ima e Con ol. Pape
p esen ed a New In e aces o Musical Exp ession (NIME), Pi sbu gh, PA, USA, June 4–6.
Jo dá, S., M. Kal enb unne , G. Geige , and R. Bencina. 2005. The eacTable. Pape p esen ed a In e na ional Compu e Music
Con e ence (ICMC), Ba celona, Spain, Sep embe 4–10, pp. 579–82.
Kalan a i, Fa zan, Flo en Be hau , and Lau en G isoni. 2017. En iching Musical In e ac ion on Tac ile Feedback Su aces wi h
P og ammable F ic ion. Pape p esen ed a In e na ional Symposium on Compu e Music Mul idisciplina y Resea ch (CMMR),
Ma osinhos, Po ugal, Sep embe 25–28.
Kayse , Ch is oph, Ch is ophe I. Pe ko , Ma k Auga h, and Nikos K. Logo he is. 2005. In eg a ion o Touch and Sound in Audi o y
Co ex. Neu on 48: 373–84. [C ossRe ]
Keane, Michael, and Geo ge Dodd. 2011. Subjec i e Assessmen o Up igh Piano Key Vib a ions. Ac a Acus ica Uni ed wi h Acus ica 97:
708–13. [C ossRe ]
K uschke, John K. 2014. Doing Bayesian Da a Analysis—A Tu o ial wi h R, JAGS, and S an, 2nd ed. Camb idge: Academic P ess.
Liu, Fang, and Yunchuan Kong. 2015. zoib: An R Package o Bayesian In e ence o Be a Reg ession and Ze o/One In la ed Be a
Reg ession. R Jou nal 7: 34–51. [C ossRe ]
Ma shall, Kenne h D. 1986. The musician and he ib a ional beha io o a iolin. Jou nal o he Ca gu Acous ical Socie y 45: 28–33.
Ma shall, Ma k T., and Ma celo M. Wande ley. 2006. Vib o ac ile eedback in digi al musical ins umen s. Pape p esen ed a New
In e aces o Musical Exp ession (NIME), Pa is, F ance, June 4–8, pp. 226–29.
Ma shall, Ma k T., and Ma celo M. Wande ley. 2011. Examining he E ec s o Embedded Vib o ac ile Feedback on he Feel o a
Digi al Musical Ins umen . Pape p esen ed a New In e aces o Musical Exp ession (NIME), Oslo, No way, May 30–June 1,
pp. 399–404.
Me chel, Sebas ian, and M. E can Al insoy. 2018. Audi o y- ac ile expe ience o music. In Musical Hap ics. Edi ed by S e ano Pape i
and Cha alampos Sai is. Cham: Sp inge In e na ional Publishing, pp. 123–48. [C ossRe ]
Michailidis, Tychonas, and Jamie Bullock. 2011. Imp o ing pe o me s’ musicali y h ough li e in e ac ion wi h hap ic eedback: A
case s udy. Pape p esen ed a In e na ional Con e ence on Sound and Music Compu ing (SMC2011), Padua, I aly, July 6–9.
Moog, Robe A., and Thomas L. Rhea. 1990. E olu ion o he Keyboa d In e ace: The Bösendo e 290 SE Reco ding Piano and he
Moog Mul iply-Touch-Sensi i e Keyboa ds. Compu e Music Jou nal 14: 52–60. [C ossRe ]
A s 2023,12, 143 24 o 25
Moo e, B ian C. J., and Robe W. Pe e s. 1992. Pi ch disc imina ion and phase sensi i i y in young and elde ly subjec s and i s
ela ionship o equency selec i i y. Jou nal o he Acous ical Socie y o Ame ica 91: 2881. [C ossRe ]
Mo ey, Richa d D. 2008. Con idence in e als om no malized da a: A co ec ion o Cousineau (2005). Tu o ial in Quan i a i e Me hods
o Psychology 4: 61–64. [C ossRe ]
Oboe, Robe o, and Gio anni De Poli. 2006. A Mul i-Ins umen , Fo ce-Feedback Keyboa d. Compu e Music Jou nal 30: 38–52.
[C ossRe ]
Okazaki, Ryu a, Hideno i Ku ibayashi, and Hi oyuki Kajimo o. 2015. The E ec o F equency Shi ing on Audio–Tac ile Con e sion
o En iching Musical Expe ience. In Hap ic In e ac ion: Pe cep ion, De ices and Applica ions. Volume 277 o Lec u e No es in
Elec ical Enginee ing. Edi ed by Hi oyuki Kajimo o, Hideyuki Ando and Ki-Uk Kyung. Tokyo: Sp inge , pp. 45–51. [C ossRe ]
Okazaki, Ryu a, Taku Hachisu, Michi Sa o, Shogo Fukushima, Vincen Haywa d, and Hi oyuki Kajimo o. 2013. Judged Consonance o
Tac ile and Audi o y F equencies. Pape p esen ed a 2013 Wo ld Hap ics Con e ence, Daejeon, Republic o Ko ea, Ap il 14–17,
pp. 663–66.
O’Modh ain, Sile, and B en R. Gillespie. 2018. Once mo e, wi h eeling: The dynamics o pe o me -ins umen in e ac ion. In Musical
Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Cham: Sp inge .
O’Modh ain, Sile, and Ch is Cha e. 2000. Inco po a ing Hap ic Feedback in o In e aces o Music Applica ions. Pape p esen ed a
ISORA, Wo ld Au oma ion Cong ess, Maui, Hawaii, USA, June 11–16.
Ospina, Raydonal, and Sil ia L. P. Fe a i. 2010. In la ed be a dis ibu ions. S a is ical Pape s 51: 111–26. [C ossRe ]
O e hol , Dan, Edga Be dahl, and Robe Hamil on. 2011. Ad ancemen s in ac ua ed musical ins umen s. O ganised Sound 16:
154–65. [C ossRe ]
Palme , Ca olyn F., Rebecca K. Jones, Be h L. Hennessy, Ma sha G. Unze, and Anne D. Pick. 1989. How is a umpe known? The
“basic objec le el” concep and pe cep ion o musical ins umen s. The Ame ican Jou nal o Psychology 102: 17–37. [C ossRe ]
Pape i, S e ano, and Cha alampos Sai is. 2018a. Musical Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Sp inge Se ies on
Touch and Hap ic Sys ems. Cham: Sp inge In e na ional Publishing.
Pape i, S e ano, and Cha alampos Sai is. 2018b. Musical hap ics: In oduc ion. In Musical Hap ics. Cham: Sp inge In e na ional
Publishing, pp. 1–7.
Pape i, S e ano, Fede ico A anzini, and Fede ico Fon ana. 2019. Design and applica ion o he bi ib audio- ac ile piano sample lib a y.
Applied Sciences 9: 914. [C ossRe ]
Pape i, S e ano, Hanna Jä eläinen, and Sébas ien Schiesse . 2021. In e ac i e ib o ac ile eedback enhances he pe cei ed quali y o a
su ace o musical exp ession and he playing expe ience. IEEE T ansac ions on Hap ics 14: 635–45. [C ossRe ]
Pape i, S e ano, Hanna Jä eläinen, B uno L. Gio dano, Sébas ien Schiesse , and Ma in F öhlich. 2017. Vib o ac ile sensi i i y in
ac i e ouch: E ec o p essing o ce. IEEE T ansac ions on Hap ics 10: 113–22. [C ossRe ]
Pape i, S e ano, Ma in F ohlich, and Sebas ien Schiesse . 2019. The TouchBox: An open-sou ce audio-hap ic de ice o inge -based
in e ac ion. Pape p esen ed a IEEE Wo ld Hap ics Con e ence, Tokyo, Japan, July 9–12, pp. 491–96.
Pape i, S e ano, Ma in F öhlich, Fede ico Fon ana, Sébas ien Schiesse , and Fede ico A anzini. 2018. Implemen a ion and cha -
ac e iza ion o ib o ac ile in e aces. In Musical Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Cham: Sp inge
In e na ional Publishing, pp. 257–82.
Pong ac, Helena. 2006. Vib o ac ile Pe cep ion: Di e en ial E ec s o F equency, Ampli ude, and Accele a ion. Pape p esen ed a
2006 IEEE In e na ional Wo kshop on Hap ic Audio Visual En i onmen s and Thei Applica ions (HAVE 2006), O awa, ON,
Canada, No embe 4–5, pp. 54–59.
Ro, Tony, Johanan Hsu, Na i E. Yasa , L. Cai lin Elmo e, and Michael S. Beauchamp. 2009. Sound enhances ouch pe cep ion.
Expe imen al B ain Resea ch 195: 135–43. [C ossRe ]
Sai is, Cha alampos. 2013. E alua ing Violin Quali y: Playe Reliabili y and Ve baliza ion. Ph.D. hesis, Depa men o Music Resea ch,
McGill Uni e si y, Mon eal, QC, Canada.
Sai is, Cha alampos, B uno L. Gio dano, Claudia F i z, and Ga y P. Sca one. 2012. Pe cep ual e alua ion o iolins: A quan i a i e
analysis o p e e ence judgmen s by expe ienced playe s. The Jou nal o he Acous ical Socie y o Ame ica 132: 4002–12. [C ossRe ]
Sai is, Cha alampos, Hanna Jä eläinen, and Claudia F i z. 2018. The ole o hap ic cues in musical ins umen quali y pe cep ion. In
Musical Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Cham: Sp inge In e na ional Publishing, pp. 73–93.
Schmid, Gian-Ma co. 2014. Measu ing Musician’s Playing Expe ience: De elopmen o a ques ionnai e o he e alua ion o musical
in e ac ion. Pape p esen ed a P ac ice-Based Resea ch Wo kshop a NIME, London, UK, June 30–July 4.
Schwa z, Diemo, Wanyu Liu, and F édé ic Be ilacqua. 2020. A Su ey on he Use o 2D Touch In e aces o Musical Exp ession.
Pape p esen ed a New In e aces o Musical Exp ession (NIME), Bi mingham, UK, July 21–25.
Spence, Cha les. 2012. Mul isenso y pe cep ion, cogni ion, and beha io : E alua ion he ac o s modula ing mul isenso y in eg a ion.
In New Handbook o Mul isenso y P ocessing. Edi ed by Ba y E. S ein. Camb idge: MIT P ess.
Spiegel, Mu ay F., and Cha les S. Wa son. 1984. Pe o mance On F equency-disc imina ion Tasks By Musicians And Nonmusicians.
Jou nal o he Acous ical Socie y o Ame ica 76: 1690–95. [C ossRe ]
Suzuki, Hideo. 1986. Vib a ion and sound adia ion o a piano soundboa d. Jou nal o he Acous ical Socie y o Ame ica 80: 1573–82.
[C ossRe ]
A s 2023,12, 143 25 o 25
Tache, Oli ie , Jean-loup Flo ens, S ephen Sinclai , and Ma celo M. Wande ley. 2012. Explo ing audio and ac ile quali ies o
ins umen ali y wi h bowed s ing simula ions. Pape p esen ed a New In e aces o Musical Exp ession (NIME), Ann A bo ,
MI, USA, May 21–23.
Tu ne , E ica O. 1939. Touch and Tone-Quali y: The Pianis ’s Illusion. The Musical Times 80: 173–76. [C ossRe ]
Ve illo, Ronald T. 1971. Vib o ac ile h esholds measu ed a he inge . Pe cep ion and Psychophysics 9: 329–30. [C ossRe ]
Ve illo, Ronald T. 1992. Vib a ion sensa ion in humans. Music Pe cep ion 9: 281–302. [C ossRe ]
Wollman, Indiana, Claudia F i z, and Jacques Poi e ineau. 2014. In luence o ib o ac ile eedback on some pe cep ual ea u es o
iolins. Jou nal o he Acous ical Socie y o Ame ica 136: 910–21. [C ossRe ] [PubMed]
Yau, Je ey M., Alison I. Webe , and Sliman J. Bensmaia. 2010. Sepa a e mechanisms o audio- ac ile pi ch and loudness in e ac ions.
F on ie s in Psychology 1: 160. [C ossRe ] [PubMed]
Young, Ga e h W., Da id Mu phy, and Je ey Wee e . 2017. A Quali a i e Analysis o Hap ic Feedback in Music Focused Exe cises.
Pape p esen ed a New In e aces o Musical Exp ession (NIME’17), Copenhagen, Denma k, May 15–18, pp. 204–9.
Young, Ga e h W., Da id Mu phy, and Je ey Wee e . 2018. A unc ional analysis o hap ic eedback in digi al musical ins umen
in e ac ions. In Musical Hap ics. Edi ed by S e ano Pape i and Cha alampos Sai is. Cham: Sp inge In e na ional Publishing,
pp. 95–122.
Zappi, Vic o , and And ew P. McPhe son. 2014. Dimensionali y and app op ia ion in digi al musical ins umen design. Pape
p esen ed a 14 h In e na ional Con e ence on New In e aces o Musical Exp ession, NIME 2014, London, UK, June 30–July 4,
pp. 455–60.
Disclaime /Publishe ’s No e:
The s a emen s, opinions and da a con ained in all publica ions a e solely hose o he indi idual
au ho (s) and con ibu o (s) and no o MDPI and/o he edi o (s). MDPI and/o he edi o (s) disclaim esponsibili y o any inju y o
people o p ope y esul ing om any ideas, me hods, ins uc ions o p oduc s e e ed o in he con en .
