The TouchBox MK3: An Open-Sou ce De ice o Finge -Based
In e ac ion wi h Ad anced Audi o y and Vib o ac ile Feedback
S e ano Pape i
E ic La ieux
Ma in F öhlich
[email p o ec ed]
[email p o ec ed]
[email p o ec ed]
Zu ich Uni e si y o he A s
Zu ich, Swi ze land
ABSTRACT
The TouchBox is an open-sou ce de ice aimed a nge -based in-
e ac ion, o e ing ad anced audi o y and ib o ac ile eedback. I s
capabili ies encompass acking mul iple nge ges u es, measu ing
exe ed o ces, and con ac a eas, while p o iding ich in e ac i e
audio and ib o ac ile expe iences. P e ious p o o ypes ha e been
u ilized in psychophysical expe imen s and musical pe o mance as-
sessmen s. The la es MK3 e sion in oduces no el enhancemen s,
including a new mechanical design, onboa d compu a ion o n-
ge acking using he OpenMV pla o m, upg aded ib o ac ile
ac ua o s, and imp o ed audio ampli e s. The de ice is suppo ed
by an open-access eposi o y, o e ing ools and documen a ion
o easy eplica ion and u he de elopmen . The TouchBox MK3
p esen s a e sa ile and a o dable solu ion, well-sui ed o edu-
ca ional and esea ch pu poses, and holds po en ial o a ious
mul imodal human-machine in e ac ion applica ions.
KEYWORDS
nge -based in e ac ion, 3D o ce sensing, nge acking, ib o ac-
ile eedback, audi o y eedback, hap ic de ice, open-sou ce, DIY
ACM Re e ence Fo ma :
S e ano Pape i, E ic La ieux, and Ma in F öhlich. 2023. The Touch-
Box MK3: An Open-Sou ce De ice o Finge -Based In e ac ion wi h Ad-
anced Audi o y and Vib o ac ile Feedback. In INTERNATIONAL CON-
FERENCE ON MULTIMODAL INTERACTION (ICMI ’23 Companion), Oc o-
be 09–13, 2023, Pa is, F ance. ACM, New Yo k, NY, USA, 4 pages. h ps:
//doi.o g/10.1145/3610661.3616181
1 INTRODUCTION
The TouchBox is an open-ended de ice designed o ack a ious
nge -based ges u es, p ecisely measu e exe ed h ee-dimensional
o ces, and de e mine nge con ac a eas, while deli e ing ich
in e ac i e audi o y and ib o ac ile eedback. I s ea lie i e a-
ions [
3
] ha e success ully con ibu ed o psychophysical expe i-
men s and musical pe o mance assessmen s [
2
,
4
]. Building upon
This wo k is licensed unde a C ea i e Commons A ibu ion In e na ional
4.0 License.
ICMI ’23 Companion, Oc obe 09–13, 2023, Pa is, F ance
© 2023 Copy igh held by he owne /au ho (s).
ACM ISBN 979-8-4007-0321-8/23/10.
h ps://doi.o g/10.1145/3610661.3616181
hese achie emen s, he TouchBox MK3 ep esen s a signi can
ad ancemen wi h se e al no el ea u es. No ably, he MK3 e -
sion inco po a es a new mechanical design ha includes a la ge
ouch su ace, expanding he scope o in e ac ion possibili ies. Also,
nge - acking now elies on he OpenMV pla o m
1
o onboa d
compu a ion, imp o ing i s esponsi eness and capabili ies. Mo e-
o e , he MK3 e sion inco po a es mo e powe ul and e cien
ib o ac ile ac ua o s based on oice-coil echnology, along wi h an
upg aded audio ampli e , o e ing a mo e ealis ic use expe ience.
An ex ensi e open-access eposi o y is made a ailable, o e ing
comp ehensi e documen a ion and so wa e ools o easy eplica-
ion and u he de elopmen .
2
This aspec signi can ly con ibu es
o he de ice’s sui abili y o educa ional pu poses. As a low-cos
ye accu a e and sophis ica ed ool, he TouchBox MK3 o e s an
ideal pla o m o lea ning he p inciples o ib o ac ile pe cep ion,
nge -based in e ac ion, and mul imodal eedback in educa ional
se ings. I s e sa ili y and accessibili y make i a aluable asse o
s uden s, educa o s and esea che s alike.
In his pape , we p esen he echnical de ails o he Touch-
Box MK3, elabo a ing on i s no el ea u es and enhancemen s.
Addi ionally, we explo e po en ial applica ions in educa ion and
esea ch, highligh ing i s capaci y as a calib a ed measu emen de-
ice and i s signi cance in os e ing mul imodal human-machine
in e ac ion s udies.
2 DESIGN
Figu e 1 p esen s a de ailed ende ing o he MK3 de ice, wi h
key componen s labeled. Addi ionally, Fig. 2 showcases he ully
implemen ed p o o ype o he TouchBox MK3, p o iding a isual
glimpse o i s unc ional componen s wi h he enclosu e opened.
The MK3 design le e ages a combina ion o ma e ials and ech-
nologies, allowing o a cos -e ec i e ye high-pe o ming de ice.
The ma e ial cos s, excluding manu ac u ing expenses, o aled ap-
p oxima ely 350 USD, wi h aw ma e ials like Plexiglas and alu-
minum cons i u ing nea ly 100 USD o he o e all expenses. This
budge - iendly app oach ensu es accessibili y, making he Touch-
Box MK3 pa icula ly well-sui ed o educa ional se ings.
The cen al componen o he TouchBox MK3 is he Plexiglas
op panel (1), which se es as he p ima y ouch su ace o use
1h ps://openm .io/
2h ps://gi hub.com/ICST-AHMI/TouchBox
302
ICMI ’23 Companion, Oc obe 09–13, 2023, Pa is, F ance Pape i e al.
(1)
(5)
(4)
(3)
(6)
(2)
(7)
(8)
(9)
Figu e 1: Rende ing o he TouchBox MK3, wi h ele an
componen s labeled: (1) Plexiglas op panel, (2) no mal o ce
senso (1x, a he bo om o he s uc u e), (3) la e al o ce
senso (4x, one pe side), (4) IR LEDs s ip (2x, a opposi e
sides), (5) OpenMV came a, (6) ib a ion ac ua o (2x, a op-
posi e sides), (7) A duino mic ocon olle boa d, (8) audio
ampli e , (9) MDF enclosu e.
Figu e 2: The implemen ed p o o ype o he TouchBox MK3.
in e ac ions. The panel is connec ed o he senso s and ac ua o s
esponsible o cap u ing and deli e ing in e ac i e eedback.
An A duino
3
mic ocon olle boa d (7) se es as he cen al
con ol uni , enabling da a p ocessing and communica ion be ween
he senso s, ac ua o s, and he hos compu e .
To co e hese in ica e componen s secu ely, he TouchBox MK3
is housed in an MDF enclosu e (9). This p o ides s abili y and
p o ec ion, making he de ice sui able o use in a ious educa ional
se ings. I s compac and e gonomic design encou ages s uden s
and educa o s o explo e nge -based in e ac ion and mul imodal
eedback wi h ease.
The TouchBox MK3’s hough ul design, coupled wi h i s a -
o dabili y and accessibili y, posi ions i as an ideal educa ional
ool.
2.1 3D-Fo ce Sensing Solu ion
The TouchBox MK3 o e s p ecise and eliable sensing o h ee-
dimensional o ces applied o i s op panel. To achie e his, he
3h ps://www.a duino.cc/
de ice is equipped wi h e Mic o Load Cell (MLC) senso s. A
CZL635 MLC (2), si ua ed a he bo om o he s uc u e, accu a ely
measu es he no mal o ce applied o he op panel, wi h a capaci y
o up o 50 N. Fou CZL616C MLCs (3), placed a each side o he op
pane, measu e la e al o ces, wi h a maximum capaci y o 7.65 N
pe senso . This inno a i e 3D- o ce sensing solu ion e ec i ely
cap u es a wide ange o ouch in e ac ions, enabling he use o
engage wi h he TouchBox MK3 in a highly in ui i e manne .
The e MLCs a e p iced a abou 7 USD each, he espec i e
op-amps a e wo h also 7 USD each, and he A duino boa d cos s
40 USD. The e o e, he o e all cos o ou 3D- o ce sensing solu ion
is only a small ac ion o he p ice o compa able comme cial
sys ems (usually in he 700–900 USD ange).
2.2 Finge T acking using OpenMV
The TouchBox MK3’s nge acking capabili ies a e acili a ed by
image p ocessing echniques implemen ed on an OpenMV Cam
M7 boa d (5) unning Mic oPy hon OS. The came a sys em, wi h
mwa e e sion 4.4.2, o e s wo esolu ion modes: low- esolu ion
(160
×
120) and high- esolu ion (320
×
240). To e ec i ely cap u e a
la ge po ion o he op panel despi e he sho dis ance, he de ice
u ilizes an ul a-wide-angle ( sheye) lens. The cos s a e espec i ely
65 USD o he came a and 15 USD o he lens.
In conjunc ion wi h he came a se up, wo PCB s ips equipped
wi h in a ed (IR) LEDs (4) a e s a egically placed on opposi e sides
o he op panel. These IR LEDs emi ho izon al beams ha a el
h ough he Plexiglas panel, illumina ing nge -pads ouching i s
su ace. This illumina ion enables e cien de ec ion and acking
o nge posi ions and con ac a eas h ough a ideo p ocessing
algo i hm (see Sec. 3). The combined cos o he IR LEDs amoun s
o app oxima ely 12 USD.
2.3 Audio-Tac ile Rende ing
The TouchBox MK3 is complemen ed by a sophis ica ed audio-
ac ile ende ing sys em ha en iches he use expe ience wi h
dynamic and ealis ic ib o ac ile eedback. A i s co e a e wo
Lo el L5 ad anced oice-coil ac ua o s (6), p o iding powe ul ac-
ile sensa ions based on incoming audio signals. The mo ing masses
o hese ac ua o s displace angen ially o he op panel, op imiz-
ing he hap ic expe ience and adding dep h o he in e ac ions [
1
].
To d i e each ac ua o indi idually, he wo channels o a Day on
Audio DTA-2 s e eo ampli e (8) a e u ilized. The cos -e ec i e na-
u e o each ac ua o (app oxima ely 10 USD wi h edu-p icing) and
he ampli e (abou 35 USD) ensu es a high-quali y audio- ac ile
ende ing sys em ha emains budge - iendly.
The de ice is supplied wi h open-sou ce so wa e o syn hesize
audi o y and ib a ion signals based on he incoming senso da a.
This so wa e is g ounded in physical modeling, enabling a espon-
si e and dynamic in e ac ion, pe ec ly sui ed o he capabili ies
o e ed by he TouchBox MK3. Th ough he o e ed examples, use s
can explo e a wide ange o audi o y and ib o ac ile eedback
possibili ies.
3 SENSORS DATA PROCESSING
Fo ce da a a e uni o mly sampled a a high a e o 1176 Hz wi h
10-bi esolu ion, using an A duino Mega 2560 boa d connec ed
303
The TouchBox MK3: An Open-Sou ce De ice o Finge -Based In e ac ion wi h
Ad anced Audi o y and Vib o ac ile Feedback ICMI ’23 Companion, Oc obe 09–13, 2023, Pa is, F ance
ia USB o a hos compu e . To achie e such synch onous da a
acquisi ion, a cus om mwa e was de eloped, u ilizing low-le el
ha dwa e in e up s ins ead o he na i e API.
To ensu e accu a e measu emen s, a calib a ion p ocedu e is
a ailable h ough he p o ided so wa e. Fo he o ce senso s, e.g.
es weigh s can be used o calib a e he no mal o ce, while a
o ce gauge dynamome e applied o he op panel can be used o
calib a e he la e al o ces.
Fo he image da a ob ained om he OpenMV Cam M7, a ious
p ep ocessing s ages a e applied o enhance he quali y o he nge
acking. Fi s , he dis o ion in oduced by he ul a-wide-angle
( sheye) lens is co ec ed. Finge acking is achie ed using he
ind_blobs
unc ion o he OpenMV
Image
class
4
. The algo i hm
sea ches o con iguous g oups o pixels ha pass a h eshold es
based on speci c pa ame e s, such as minimum and maximum colo
channel in ensi y, minimum numbe o pixels, and minimum blob
a ea. The numbe o de ec ed blobs is cu en ly limi ed o h ee,
e ec i ely acking he posi ions o up o h ee nge s.
The compu ed nge posi ions a e ep esen ed as dis ances om
he cen e o he panel in millime e s, p o iding accu a e spa ial
in o ma ion. Addi ionally, he con ac a eas o he nge s wi h
he op panel a e compu ed in squa e millime e s. To achie e his,
an addi ional calib a ion p ocedu e is employed: Sample objec s
wi h known a eas a e placed on he panel, allowing he sys em
o compu e he e ec i e pixel size in millime e s, which is hen
u ilized o accu a ely calcula e he con ac a eas o he nge s. In
ou p o o ype he e ec i e pixel size is 0
.
450 mm in low- es, and
0.225 mm in hi- es mode.
A so wa e applica ion, de eloped using he Max g aphical p o-
g amming en i onmen
5
u he p ocesses he da a om he sen-
so s and p esen s hem in a use - iendly g aphical in e ace (see
Fig. 3).
The Max applica ion p ocesses he senso da a ecei ed h ough
he de ice’s USB connec ions and p esen s hem in a clea and
in ui i e manne . The GUI displays nume ical in o ma ion abou
he nge s, such as hei con ac a eas and posi ions ela i e o he
cen e o he op panel. Addi ionally, he GUI isually ep esen s
he o ces applied o he panel, wi h he o al no mal o ce shown
as a da k ed disc, and he la e al o ces illus a ed by a ed line
indica ing hei ec o ial sum. The Open Sound Con ol p o ocol
6
is
also implemen ed, allowing easy in e acing wi h ex e nal sys ems.
O e all, he supplied so wa e enables use s o in e ac wi h he
TouchBox MK3, in e p e and o wa d he senso y da a in eal- ime,
os e ing a seamless and imme si e use expe ience.
4 EDUCATIONAL SCENARIOS
While he TouchBox MK3 has no ye been o mally es ed in edu-
ca ional en i onmen s, i s e sa ile ea u es, a o dabili y and open-
sou ce na u e make i an ideal ool o in oducing s uden s o he
p inciples o hap ic in e ac ion and mul imodal eedback. Below a e
se e al educa ional scena ios whe e he de ice can p o e aluable:
4h p://docs.openm .io/lib a y/om .image.h ml#image.Image. nd_blobs
5
Max is a mul imedia p ocessing en i onmen a ailable o Windows and Mac OS
compu e s.
6h ps://cc ma.s an o d.edu/osc
Figu e 3: G aphical Use In e ace (GUI) o he so wa e ap-
plica ion. The sc eensho e ec s he da a om wo nge s
p essing down wi h an o e all no mal o ce o 6
.
83 N, and
pushing o wa d and le wi h o ces o 1
.
41 N and 1
.
39 N,
espec i ely. The le mos sec ion o he GUI displays nume -
ical da a o up o h ee nge s, including hei con ac a eas
and posi ions ela i e o he cen e o he op panel. In he
cen e sec ion, he alues o o ces applied o he panel a e
g aphically ep esen ed: he o al no mal o ce is depic ed
as a da k ed disc, while he la e al o ces a e illus a ed by
a ed line, o igina ing om he cen e o he disc, which
shows hei ec o ial sum. The igh mos sec ion o he GUI
displays he a eas and posi ions o he con ac ing nge s as
seen by he OpenMV blob acke .
Hap ic Pe cep ion S udies. The TouchBox MK3 can se e as an
e ec i e ool o conduc ing hap ic pe cep ion s udies, allowing
s uden s o explo e he ela ionship be ween o ce exe ion, nge
con ac a eas, and pe cei ed ib o ac ile eedback. In psychology
cou ses, he de ice could be used in expe imen s in es iga ing
e.g. how di e en ex u es simula ed ia ib a ion a ec human
emo ional esponses, he eby deepening s uden s’ unde s anding o
senso y psychology and p o iding hem wi h p ac ical expe ience
in expe imen al design.
Mul imodal In e ac ion Expe imen s. Wi h i s capabili y o p o-
ide bo h ib o ac ile and audi o y eedback, he TouchBox MK3
acili a es expe imen s ha in es iga e he in eg a ion o mul i-
ple senso y modali ies in human pe cep ion. S uden s can explo e
how he b ain combines hap ic and audi o y s imuli, leading o
no el insigh s in o c oss-modal pe cep ion and he de elopmen
o mul imodal in e ac i e sys ems. This ea u e could be especially
bene cial in special educa ion se ings, whe e cus omized educa-
ional con en could be c ea ed o make lea ning mo e accessible
and engaging o s uden s wi h senso y impai men s.
Hap ic Technology and Design P ojec s. The open-ha dwa e na-
u e and low-cos o he TouchBox MK3 empowe s s uden s o
engage in hands-on hap ic echnology and design p ojec s. They
could o ins ance explo e di e en ma e ials, ac ua o s, and sen-
so s o c ea e cus omized hap ic in e aces o speci c applica ions.
The expe ience o building and i e a ing upon he TouchBox MK3
304
ICMI ’23 Companion, Oc obe 09–13, 2023, Pa is, F ance Pape i e al.
o add new ea u es o unc ionali ies would allow s uden s gaining
eal-wo ld enginee ing expe ience in a well-de ned con ex .
Compu e Science and Enginee ing Educa ion. The Touch-
Box MK3 can be used o each opics ela ed o physical compu ing,
da a p ocessing, and senso in eg a ion. S uden s can gain p ac ical
expe ience in p og amming mic ocon olle s, implemen ing eal-
ime algo i hms o senso da a p ocessing, and building in e ac i e
hap ic sys ems. In a emo e lea ning con ex , he de ice could sub-
s i u e a ious lab equipmen ha s uden s migh no o he wise
ha e access o, he eby b eaking down ba ie s in hei educa ional
expe ience.
In e ac i e Media and Design S udies. Fo s uden s in in e ac i e
media and design, he TouchBox MK3 o e s a angible pla o m o
p o o yping and es ing no el in e ac ion concep s. I s capabili y o
deli e bo h audi o y and ib o ac ile eedback opens up possibili-
ies o designing imme si e and engaging in e ac i e expe iences.
S uden s can explo e no el ways o communica e in o ma ion and
emo ions h ough ib o ac ile and audi o y cues.
Accessible Educa ion and Inclusi e Design. The de ice’s a -
o dabili y and open-sou ce na u e make i a iable op ion
o p omo ing accessibili y and inclusi e design in educa ion.
Educa o s could use he TouchBox MK3 o de elop educa-
ional con en ha in eg a es ich audi o y and ib o ac ile
eedback, as well as ex ensi e physical in e ac ion possibili ies.
Tha would enhance he lea ning expe ience especially o
s uden s wi h senso y impai men s o physical disabili ies, en-
su ing ha he lea ning expe ience is inclusi e and engaging o all.
O e all, he TouchBox MK3 is a aluable addi ion o se e al ed-
uca ional en i onmen s, o e ing s uden s a p ac ical and engaging
pla o m o explo ing hap ic in e ac ion, mul imodal eedback,
and in e ac i e design concep s. I s open-sou ce na u e encou ages
collabo a ion and knowledge sha ing, os e ing a communi y o
educa o s and s uden s who can collec i ely ad ance he eld o
hap ic in e ac ion and mul imodal in e aces.
5 CONCLUSIONS
The TouchBox MK3 s ands as an open-sou ce audio-hap ic de ice
designed o nge -based in e ac ion, o e ing ad anced audi o y
and ib o ac ile eedback. Wi h i s enhanced mechanical design,
imp o ed nge acking capabili ies using he OpenMV pla o m,
and upg aded ib o ac ile ac ua o s, he MK3 e sion ep esen s a
signi can ad ancemen o e i s p edecesso s.
The de ice’s abili y o p ecisely sense h ee-dimensional o ces,
ack mul iple nge s’ posi ions and con ac a eas, and p o ide in-
e ac i e ib o ac ile and audi o y eedback opens up a wide ange
o applica ions, spanning om scien i c esea ch o c ea i e in e ac-
ions. Mo eo e , he TouchBox MK3’s a o dabili y and accessibili y
make i an excellen choice o educa ional se ings.
ACKNOWLEDGMENTS
This esea ch was suppo ed by p ojec HAPTEEV (g an n° 178972,
2018-2022), unded by he Swiss Na ional Science Founda ion.
REFERENCES
[1]
Yu i De P a, S e ano Pape i, Hanna Jä eläinen, Ma eo Bianchi, and Fede ico
Fon ana. 2023. E ec s o Vib a ion Di ec ion and P essing Fo ce on Finge Vib o-
ac ile Pe cep ion and Fo ce Con ol. IEEE T ans. Hap ics 16, 1 (jan 2023), 23–32.
h ps://doi.o g/10.1109/TOH.2022.3225714
[2]
Hanna Jä eläinen, S e ano Pape i, Sébas ien Schiesse , and Tobias G osshause .
2013. Audio-Tac ile Feedback in Musical Ges u e P imi i es: Finge P essing. In
Sound Music Compu . Con . S ockholm, Sweden, 109–114. h ps://doi.o g/10.5281/
zenodo.850222
[3]
S e ano Pape i, Ma in F öhlich, and Sébas ien Schiesse . 2019. The TouchBox: an
open-sou ce audio-hap ic de ice o nge -based in e ac ion. In IEEE Wo ld Hap ics
Con . IEEE, Tokyo, Japan, 491–496. h ps://doi.o g/10.1109/WHC.2019.8816172
[4]
S e ano Pape i, Hanna Jä eläinen, B uno L. Gio dano, Sébas ien Schiesse , and
Ma in F öhlich. 2017. Vib o ac ile Sensi i i y in Ac i e Touch: E ec o P essing
Fo ce. IEEE T ans. Hap ics 10, 1 (jan 2017), 113–122. h ps://doi.o g/10.1109/TOH.
2016.2582485
305
