Why methods matter: Approaching multimodality in translation research.

Tuominen, T., Jiménez Hu ado, C., Ke ola, A. (2018). Why me hods ma e : App oaching mul imodali y in ansla ion esea ch. Linguis ica An e piensia, New Se ies: Themes in T ansla ion S udies, 17, 1–21. Why me hods ma e : App oaching mul imodali y in ansla ion esea ch Tiina Tuominen Uni e si y o Glasgow, Uni ed Kingdom [email p o ec ed] Ca alina Jiménez Hu ado Uni e si y o G anada, Spain [email p o ec ed] Anne Ke ola Tampe e Uni e si y, Finland [email p o ec ed] The s udy o mul imodal phenomena calls upon ansla ion schola s o c oss disciplina y bounda ies and adop a ange o heo e ical and me hodological app oaches. The di e si y o he mul imodal landscape b ings abou esea ch challenges ha mus be ca e ully add essed o ensu e ha hese esea ch e o s yield use ul and c edible esul s. This special issue is dedica ed o a discussion on how o engage in mul imodal ansla ion esea ch: how adi ional esea ch me hods can be adap ed and wha kinds o no el app oaches can be adop ed o de eloped in o de o deal wi h a di e si y o mul imodal da a. In his in oduc ion, we i s discuss de ini ions o mode and mul imodali y and e lec on he na u e o mul imodali y as a opic o esea ch wi hin T ansla ion S udies. We hen explain ou a ionale o dedica ing he special issue o esea ch me hods and in oduce h ee a eas o mul imodal ansla ion esea ch ha , in ou iew, me i pa icula a en ion om a me hodological poin o iew. Finally, we in oduce he a icles con ained in his special issue. 1. In oduc ion Communica ion in oday’s socie y has become p ima ily mul imodal: we encoun e combina ions o modes – wo ds, images, sounds and so on – whene e we look a ou compu e s, sma phones o ele ision sc eens. Mul imodali y is also highly ele an o ansla ion: a signi ican numbe o he ex s being ansla ed nowadays a e mul imodal, anging om use manuals, websi es, ex books and comics o audio isual p oduc s such as ilms and ideogames. The ansla ion and he ansla ional enqui y o such ex s needs o include a ca e ul conside a ion o bo h he indi idual meaning-making esou ces in ol ed and he way hey ie oge he o o m a mul imodal en i y. Accessible ansla ion, such as audio desc ip ion (AD), is also essen ially mul imodal, because i in ol es ansla ing in o ma ion om one mode o ano he , om isually pe cei ed s imuli o e bal in o ma ion. Mul imodali y o e s an in iguing special case o me hodological deba e in T ansla ion S udies, because i equi es us o ace up o he ask o ede ining wha we a e esea ching. The dis inguishing ea u e o mul imodal ma e ials is he p esence and in e ac ion o he di e en modes, and i is he e o e o en necessa y o place added Tuominen, Jiménez Hu ado & Ke ola 2 emphasis in esea ch on he non- e bal elemen s o a mul imodal message ins ead o ocusing exclusi ely on he e bal ex . T ansla ion S udies has a long adi ion o explo ing e bal ex s, bu when o he modes a e in oduced in o he esea ch design, we a e aced wi h his ques ion: How do we ind he op imal me hods o deal wi h hem all and in pa icula wi h hei in e play wi h one ano he . The e is s ill much o disco e abou he me hods and ools ha can help us explain how ansla ion ope a es in mul imodal con ex s. This special issue aims o con ibu e o he dissemina ion o in o ma ion abou di e en ways o app oach mul imodali y in ansla ion esea ch and also o he encou agemen o inc easingly anspa en and igo ous esea ch in he ield. The con ibu o s o his special issue acco dingly explo e how mul imodal opics ha e been and can be app oached in T ansla ion S udies, wha he bene i s and challenges associa ed wi h a ious me hods a e and how he me hods can be applied o di e en con ex s. In wha ollows, we i s in oduce mul imodali y, discuss he a ious de ini ions ha exis o he concep o “mode” and explo e he kinds o modes ha a e ypically p esen in mul imodal ex s. Then we discuss he na u e o mul imodali y and mul imodal ansla ion as opics o esea ch. We also explain why we chose me hods as he ocus o his special issue and in oduce a ew a eas o mul imodal ansla ion esea ch ha a e pa icula ly in e es ing om a me hodological pe spec i e and in he con ex o his special issue. Finally, we in oduce he a icles con ained in his special issue. 2. Mul imodal s udies and mul imodal ansla ion s udies 2.1 Mul imodali y and modes The concep o mul imodali y has a ac ed g owing esea ch in e es since he ea ly 2000s. Mul imodally o ien ed esea ch—o “mul imodal s udies” (O’Hallo an & Smi h, 2011, p. 1)—has i s oo s in Gun he K ess and Theo an Leeuwen’s seminal wo ks, Reading Images: The G amma o Visual Design (1996) and Mul imodal Discou se: The Modes and Media o Con empo a y Communica ion (2001), bo h o which buil on M.A.K. Halliday’s (1978) no ion o language as a social–semio ic sys em and a sys emic unc ional g amma . As i s s a ing poin , his app oach akes he h ee me a unc ions Halliday p oposed o e bal language and ex ends hem in o de o apply hem o o he o ms o communica ion, such as images and music (Jewi , 2014, p. 129). The sys emic unc ional pe spec i e on mul imodali y is s ill p ominen in mul imodal s udies (see sec ion 3.2.1). Mul imodali y i s became a ocus o in e es in a eas o esea ch such as semio ics and applied linguis ics, bu i has since become an inc easingly impo an pe spec i e o esea ch in p ac ically all disciplines ha examine communica ion, including s udies in isual a s, design, media and educa ion. As Jewi (2013) desc ibes i , mul imodal s udies aims o unde s and “communica ion and ep esen a ion as mo e han language” (p. 250) and, he e o e, o challenge he p e ious p edominance o language in esea ch. One could he e o e a gue ha mul imodal s udies has eme ged om he need o unde s and how e bal meaning-making esou ces ela e o and a e a ec ed by esou ces ha a e non- e bal—meaning all he o he ypes o esou ces ha can be employed o con ey meaning. Mul imodali y is ypically de ined om one o wo pe spec i es: i can be desc ibed ei he as he coexis ence o mul iple modes wi hin a pa icula con ex (e.g., Gibbons, 2012, p. 8) o as he p ocess o decoding he coexis ing modes om a iewe ’s o a eade ’s s andpoin (e.g., E e e , 2015, p. 3). The la e emphasises ha coexis ing modes do no ac ually in e ac unless hey a e being in e p e ed by someone; mul imodali y is unde s ood as being he in e ac ion o modes in he cogni i e sys em o he iewe o he eade . Why me hods ma e 3 In o de o unde s and ei he o hese de ini ions, we mus e lec on wha we mean by mode. Acco ding o Gun he K ess, a mode is a esou ce o meaning-making, one used o ep esen a ion and communica ion (K ess, 2010, p. 82). Mul imodal s udies ypically p oposes ha e bal language cons i u es a mode ( o a discussion o whe he spoken language and w i en language can be unde s ood as sepa a e modes, see, o example, Ke ola, 2018, p. 31). Ye , he ques ion o wha else cons i u es a mode is no a s aigh o wa d one: his ypically depends on he na u e o he mul imodal ma e ial ha is being analysed as well as on he esea ch ques ions ha a e being asked. Ins ead o o e ing ixed classi ica ions wi h a se numbe o modes, de ini ions o mode in he mul imodal s udies li e a u e he e o e end o o e open-ended lis s o examples o modes. Those ha a e o en men ioned in hese lis s include images, mo ing images, layou , ges u es, music, and sound (see, o example, Adami, 2017, p. 451; K ess, 2010, p. 79; Page, 2010, p. 1). In pa , his lack o de ini eness is a unc ion o he di e si y o mul imodal a e ac s and e en s, bu i also s ems om he social–semio ic o igins and o ien a ions p e alen in mul imodal s udies: he social–semio ic app oach o mul imodali y de ines modes om a socially speci ic pe spec i e. In o he wo ds, wha cons i u es a mode “depends on he social g oup who uses i and he ange o meanings ha he g oup can exp ess h ough i s esou ces” (Adami, 2017, p. 464). K ess uses on as an example in his e lec ion o wha cons i u es a mode: o o he s, on migh be a me e p ope y o sub-mode o w i en ex (c . Pauwels, 2012, p. 250), bu o a communi y o g aphic designe s i is likely o be a “ ull means o ep esen a ion” and i is he e o e a mode o ha communi y (K ess, 2010, p. 87). Adami’s (2017, p. 464) e lec ion on he opic o e s an e en mo e speci ic example: she p oposes ha o a communi y o wine as e s, wine cons i u es a ully a icula e mode wi h modal esou ces such as colou , a oma and as e. In mul imodal s udies, he e m mode is some imes also used in connec ion wi h physiological o senso y channels such as seeing ( he isual mode), hea ing (au al/audi o y mode), ouching ( ac ile mode), smelling ( he ol ac o y mode) and as ing (gus a o y mode). O he s e e o hese physiological channels as senso y modali ies. Fo he eade , he un o una e o e lap be ween hese e ms can be sligh ly misleading. I is he e o e wo h emphasising ha , o ins ance, he e m isual mode akes on di e en de ini ions depending on he ma e ial ha is being analysed, as we can see in he a icles con ained in his special issue: i may e e o s ill images (a icles examining illus a ed ex s), mo ing images (a icles examining audio isual ma e ial) o in o ma ion ha is pe cei ed wi h sigh (a icles examining AD). The non- e bal mode ha has so a spa ked he mos esea ch in e es in mul imodally o ien ed ansla ion esea ch is p obably he s ill image. The ole o images in ansla ion has been examined, o ins ance, in p in ad e isemen s (e.g., To esi, 2007; Millán-Va ela, 2004), child en’s pic u e books (e.g., Oi inen, Ke ola, & Ga a ini, 2018; Van Mee be gen, 2010), comics (e.g., Kaindl, 2004) and illus a ed echnical ex s (e.g., Ke ola, 2016, 2018). Na u ally, audio isual ansla ion is also an a ea o esea ch whe e i is ex emely in e es ing o conside he unc ions o modes and he o e all modal con igu a ion o a mul imodal ex . In ilmic AD, o ins ance, isually pe cei ed in o ma ion is con e ed in o an o al e bal o m. In dubbing, he ansla ion needs o i in o s ic limi a ions posed by he o he modes o he sou ce ex (ST; o ins ance, isual synch ony). In sub i ling, he ansla ion is p esen ed as an addi ional mode—w i en ex —which compe es o he iewe ’s isual a en ion. Audio isual ansla ion has, gene ally speaking, a p ominen p esence in s udies on mul imodal ansla ion. Because i in eg a es bo h au al and isual modes, i p o ides a good example o many o he cha ac e is ics and challenges ypical o mul imodali y. As such, i could e en be called an “a che ypal” (Pe ego, 2012, p. 7) o m o mul imodal ansla ion. O cou se, esea ch should no ep esen only one a ea o mul imodali y and audio isual ansla ion alone canno answe all he ques ions ega ding mul imodali y. Howe e , as audio isual ansla ions and hei esea ch can add ess ques ions o Tuominen, Jiménez Hu ado & Ke ola 4 mul imodali y in an enligh ening way, we hink i is jus i iable o use hem as example cases o ep esen mul imodali y whene e hey illumina e a ele an poin . 2.2 Mul imodal ansla ion as a clus e concep As he mul iplici y and en a i eness o he p eceding de ini ions demons a es, mul imodali y canno be desc ibed as a monoli hic concep . Ins ead, i co e s a wide a ie y o gen es, o ms o communica ion, and combina ions o modes and semio ic esou ces. This means ha mul imodali y can also be associa ed wi h many kinds o ansla ion. I would he e o e be almos impossible o gi e a single, all-encompassing de ini ion o wha mul imodal ansla ion means. Acco dingly, a he han labelling i a simple, easily dis inguishable phenomenon, mul imodal ansla ion could be called a “clus e concep ”. Ma ia Tymoczko (2007, pp. 83–90) has p oposed he idea o a clus e concep as a way o make sense o all ansla ion in an inclusi e way. In he wo ds, ansla ion “canno [simply be de ined in e ms o necessa y and su icien ea u es” (Tymoczko, 2007, p. 85). T ansla ions canno be dis inguished om o he p oduc s o phenomena by some cha ac e is ic ha all ansla ions would ha e in common, bu hey do ha e “pa ial and o e lapping simila i ies” which allow us o use and unde s and he concep o ansla ion (Tymoczko, 2007, p. 85). Simila ly, Luis Pé ez-González (2014a, pp. 141–142) poin s ou ha e en he ield o audio isual ansla ion on i s own is so di e se ha i equi es mul iple de ini ions, and he consequen ly endo ses a lexible heo e ical unde s anding o ansla ion as a clus e concep . The same a gumen s apply o mul imodal ansla ion: i ce ainly co e s a mul iplici y o ac i i ies ha do no i unde a single de ini ion wi h clea bounda ies. I would he e o e no be a p ac ical goal o y o o mula e a uni ied de ini ion o mul imodal ansla ion. Ins ead, esea ch could aim o explo e as many con ex s and gen es as possible in o de o de elop he mos comp ehensi e unde s anding possible o he a ie y o ways in which his clus e concep becomes ac ualized. Desc ibing mul imodal ansla ion as a clus e concep leads us o a undamen al ques ion: Do we unde s and mul imodal ansla ion in he same way as any o he kind o ansla ion o is i a di e en phenomenon? The ac ha a a ie y o modes a e in ol ed means ha he ansla ional ac i i ies do become a ied. Fo example, in e semio ic ansla ion, such as ansla ing isual in o ma ion in o e bal in o ma ion in AD, is a special case o mul imodal ansla ion ha expands he de ini ion o ansla ion, because i akes place be ween e bal and non- e bal modes. In ac , all mul imodal ansla ions mus ake non- e bal modes in o accoun o some ex en , and hey may in oduce new challenges o he ansla o ’s ask, such as modi ying isual in o ma ion o in oducing shi s in he ansla ion o he e bal con en so as o emain consis en wi h he non- e bal modes. The ield o mul imodal ansla ion he e o e inhe en ly encompasses many special o ma ginal cases o ansla ion. The e is no simple answe o he ques ion whe he mul imodal ansla ion can be conside ed a dis inc phenomenon. Bu i is undeniable ha he de ini ion o ansla ion mus be pa icula ly lexible, and he bo de s a ound his clus e concep mus be especially hazy, in o de o accommoda e all he ac i i ies ha can be e med “mul imodal ansla ion”. Ano he ele an ques ion is whe he his lexible pe spec i e on ansla ion leads o a iew o esea ch on mul imodal ansla ion as a dis inc sub ield o T ansla ion S udies, some hing we migh call Mul imodal T ansla ion S udies. Al hough we do no a emp o answe his ques ion conclusi ely, he b ead h o esea ch conduc ed in he a ea in ecen yea s does sugges ha esea ch in o mul imodal ansla ion is expanding and di e si ying. Fu he mo e, while many cu en app oaches a e indeb ed o T ansla ion S udies and s em om a long esea ch adi ion in he discipline, some de elopmen s owa ds de ining he objec o esea ch may sugges depa u es unique o mul imodal Why me hods ma e 5 esea ch con ex s. Pe haps i would be mos ealis ic o see a possible subdiscipline o Mul imodal T ansla ion S udies as a way o challenging any na ow de ini ions o ansla ion and o b oadening he scope o T ansla ion S udies. In his in oduc ion, we use he e m “mul imodal ansla ion s udies”, a ansla ion-speci ic de i a ion om he e m “mul imodal s udies”, o o eg ound mul imodally o ien ed ansla ion esea ch as a esea ch o ien a ion wi h a speci ic objec i e o add essing he p esence and in e play o di e en modes in ansla ional con ex s. I emains o be seen whe he i de elops in o a genuine, p ominen sub ield o T ansla ion S udies. In o de o ha o happen, mul imodal ansla ion s udies should demons a e i s dis inc iden i y by inding ways o add essing ques ions ha ha e been less equen ly co e ed in T ansla ion S udies, ques ions such as he in e play o e bal and non- e bal modes in he cons uc ion o meaning, and hus in oduce new esea ch ques ions, new me hods and new in e disciplina y connec ions in o T ansla ion S udies. The na u e o mul imodal ansla ion as a clus e concep will become appa en only i , as Tymoczko (2007, pp. 86–87) ecommends, we obse e and desc ibe many examples o i and obse e hei simila i ies and di e ences. The e o e, while much in e es ing esea ch has al eady been p oduced in ecen yea s, he clus e concep na u e o mul imodal ansla ion ce ainly wa an s ye u he con inuing esea ch. Fu he mo e, i con i ms he need o es new me hods ha can unco e di e en aspec s o mul imodal ansla ion and o m a deepe unde s anding o i s na u e. The e o e, in he ollowing sec ion, we con inue o explo e he na u e o mul imodal ansla ion s udies by discussing some o he me hodological challenges ela ed o he ield. 3. Me hods in mul imodal ansla ion s udies 3.1 In e disciplina i y in mul imodal ansla ion s udies E en hough we end o sha e an unde s anding o he na u e o schola ly in es iga ion, we do no always unde s and he e ms me hod, me hodology o e en esea ch app oach in he same way. The e ms used o dis inguish be ween he di e en aspec s and applica ion phases o a me hod a e some imes ambiguous. Wha some e e o as a me hod o a me hodology is e med a me hodological design o a scien i ic app oach by o he s. Academic knowledge o e s heo ies o explain he phenomena ha su ound us. We equi e in o ma ion ha is sys ema ically acqui ed o desc ibe exis ing pa e ns and disco e new ones. The e o e, in he i s place, i is o i al impo ance o adop a me hodology o a esea ch app oach ha allows he esea che o ga he ep esen a i e da a. In he second place, i is essen ial o employ a me hod ha desc ibes and explains he da a and he ela ionships ha exis be ween pa s o he da a. These explana ions gi e ise o models and heo ies ha desc ibe he phenomenon unde s udy. In his sense, a me hod is a collec ion o ope a ions and p ocesses which a e ca ied ou so ha he equi ed da a can be sys ema ically analysed and so ha eliable and e i iable a gumen s can hen be d awn om i . Saldanha and O’B ien (2013) desc ibe me hodology as “a gene al app oach o s udying a phenomenon” whe eas me hod “is a speci ic esea ch echnique” (p. 13). The au ho s desc ibe ools, in u n, as echnological ins umen s designed o di e en me hods (2013), such as “a oice eco ding de ice, sc een eco ding so wa e and a keys oke logging ool” (p. 13). In his sense, eye acking, o ins ance, is a speci ic me hod whe eas an eye acke is a ool: he so wa e used du ing he applica ion o he me hod. I is use ul o keep he de ini ions o hese e ms in mind when conside ing he s a e o esea ch in T ansla ion S udies. The ield is he e ogeneous in he way esea ch is conduc ed, and because T ansla ion S udies is inhe en ly in e disciplina y in i s me hodology, desc ip ions o he ools and me hods used can a y g ea ly. T ansla ion Tuominen, Jiménez Hu ado & Ke ola 6 S udies i sel has adop ed a numbe o app oaches om o he disciplines: om he adi ionally p e alen Li e a y C i icism and Linguis ics o mo e ecen ly es ablished links o disciplines such as Psychology and Sociology (Saldanha & O’B ien, 2013, pp. 1–2). The e o e, as Saldanha and O’B ien (2013) poin ou , T ansla ion S udies does no come ac oss as a discipline wi h a “single cohe en me hodology” (p. 2), e en hough many o he me hods bo owed om o he disciplines ha e been modi ied o i he special ci cums ances o T ansla ion S udies and many ools a e used in a speci ically ailo ed way. This in e disciplina i y and he cons an modi ica ion o ools and me hods explain why me hodological discussion plays an impo an ole in he discipline. I he e is no o e a ching, es ablished me hodology, i is impo an o explain and c i ically e alua e he b ead h o op ions o in e disciplina y c oss-o e s and hei implemen a ions speci ic o T ansla ion S udies. Fu he mo e, i is always possible o explo e new sou ces o me hodological inspi a ion and po en ially use ul ools. A deepe unde s anding o he a ailable esea ch me hods and hei applica ions can acili a e a mo e disciplined app oach o esea ch design and encou age a mo e nuanced deba e on he na u e o ansla ion as an ac i i y and T ansla ion S udies as a discipline. As is he case wi h T ansla ion S udies gene ally, i is impossible o delinea e a single, cohe en me hodological amewo k o mul imodal ansla ion s udies. In o he wo ds, bo h T ansla ion S udies and mul imodal ansla ion s udies a e also clus e concep s in a me hodological sense. Di e en ypes o mul imodal ansla ion may o m in e disciplina y connec ions in di e en di ec ions—audio isual ansla ion wi h ilm s udies, echnical ansla ion wi h echnical communica ion and ideogames wi h human– compu e in e ac ion. The need o pe cei e mul imodal ansla ion as a me hodological clus e concep is ein o ced by he c i icism di ec ed by some schola s a p ominen ends in mul imodal ansla ion s udies—namely, ha he esea ch me hods adop ed o mul imodal con ex s ha e ended o be simila o hose used wi h e bal ex s—which has led o a en ion being ocused on language a he expense o o he modes (Kaindl, 2013, p. 257). In his ega d, se e al schola s ha e called o mo e a ied me hods ha ake accoun o bo h he non- e bal and he e bal aspec s o mul imodal messages (Kaindl, 2013, p. 266; Kokkola & Ke ola, 2015, pp. 226–227; Pé ez-González, 2014a, p. 185). Following hese ecommenda ions and de eloping a b oade me hodological oolki would mean, i s , placing emphasis on wha is unique and speci ically challenging abou mul imodal ansla ion s udies and, secondly, shi ing away om he e bal ocus o T ansla ion S udies. In o de o accomplish his mo e a ied me hodological amewo k, i would be help ul o look o models om o he disciplines and mul idisciplina y inno a ions ha ha e no p e iously been ha nessed commonly in ansla ion esea ch. Emb acing a combina ion o new in e disciplina y connec ions and a solid unde s anding o he na u e o ansla ion as a a ge o esea ch could help o mo e esea ch o wa d in inno a i e and ambi ious ways. As we see i , he co e me hodological challenge acing mul imodal ansla ion esea ch is in eg a ing he analysis o non- e bal modes wi h ha o e bal in o ma ion. This challenge can ake on di e en o ms depending on he na u e o he esea ch being ca ied ou . In sec ions 3.2.1 o 3.2.3, we in oduce h ee a eas o mul imodal ansla ion s udies ha , in ou iew, me i pa icula e lec i e a en ion om a me hodological poin o iew. The i s o hese is he applica ion o he me a unc ional analysis o ansla ional enqui y. Me a unc ional analysis o e s a eady-made ool o analysing non- e bal modes. Ye , as we discuss in he ollowing sec ion, he in eg a ion o analyses o e bal and non- e bal in o ma ion poses pa icula challenges in his ype o esea ch. The second a ea o esea ch, desc ibed in sec ion 3.2.2, is co pus-based me hods in mul imodal ansla ion s udies, pa icula ly in he analysis o audio isual ex s. As ad oca ed by Baños, B u i and Zano i (2013, p. 488), in o de o deal wi h he complex semio ic ab ic o audio isual ex s, co pus-based audio isual ansla ion esea ch should Why me hods ma e 7 in eg a e mul imodal analysis wi h a co pus linguis ics me hodology. Bu he p ac ical applica ion o his idea encoun e s a numbe o concep ual and echnological challenges. The hi d a ea o esea ch we wish o d aw a en ion o is ecep ion esea ch. The ecep ion o ansla ed mul imodal ex s is o pa icula in e es , since he eade ’s o iewe ’s in e p e a ion e lec s he way in which he ansla ion in e ac s wi h o he modes. Howe e , as we w i e in sec ion 3.2.3, ou iew is ha mul imodali y has o da e no been conside ed su icien ly o explici ly in ecep ion esea ch. 3.2 Cu en and eme ging opics in mul imodal ansla ion s udies 3.2.1 Sys emic unc ional analyses in mul imodal ansla ion s udies The sys emic unc ional app oach o mul imodali y is linguis ically inspi ed: as i s s a ing poin , i adop s he h ee me a unc ions Halliday (1978) p oposed o e bal language— he idea ional, he in e pe sonal and he ex ual—and ex ends hem o apply o all modes. In his iew o mul imodali y, “common semio ic p inciples ope a e in and ac oss di e en modes” (K ess & an Leeuwen, 2001, p. 2). The app oach, he e o e, ea s all modes as iden ical in hei meaning-making s a egies. In mul imodal s udies, he me a unc ional analysis is pa icula ly popula in he analysis o illus a ed ex s, in which such an analysis is ca ied ou on bo h he e bal and he isual pa s o he ex , he wo ds and he images. The me a unc ional analysis o images is ounded on he social–semio ic accoun o how meaning is ealized isually ha was in oduced by K ess and an Leeuwen (1996). In he me a unc ional analysis o e bal language, he idea ional me a unc ion e e s o he way in which ex e nal eali y is ep esen ed in he ex : he people, objec s and places— e e ed o as “ he pa icipan s”—as well as ac ions, ci cums ances and so on being discussed (e.g., Halliday & Ma hiessen 2004/2013, p. 29). Simila ly, image analysis a he idea ional le el includes iden i ying bo h he pa icipan s in he image and he ci cums ances be ween he pa icipan s (K ess & an Leeuwen, 1996, p. 42). The in e pe sonal me a unc ion o e bal language e e s o he ela ionship ha exis s be ween he au ho o he ex and he eade o he ex (Halliday & Ma hiessen 2004/2013, pp. 29–30). The au ho may add ess he eade in a ious ways— o ins ance, by ins uc ing, nego ia ing, pe suading o gi ing o de s. Image analysis a he in e pe sonal le el e e s o iden i ying he p esumed ela ionships be ween he c ea o o he image, i s iewe and he pa icipan s in he image. This me a unc ion is said o be ealised a h ee le els: eye con ac , size o ame (close sho , medium sho , long sho ) and pe spec i e (K ess & an Leeuwen, 1996, pp. 42–43, pp. 116–129). The ex ual me a unc ion o e bal language e e s o he p ope y o a ex ha links indi idual wo ds in o cohe en clauses. I in ol es aspec s such as cohesion, in o ma ion s uc u e and hema ic s uc u e (Halliday & Ma hiessen 2004/2013, p. 30). In image analysis, his me a unc ion is e e ed o as composi ional and i e e s o iden i ying he composi ion o he image, he colou (s) and he ela i e sizes o he pa icipan s and hei posi ioning (up, down, le , igh , cen e) in ela ion o he o he pa icipan s (K ess & an Leeuwen, 1996, p. 43, pp. 177–179). Sys emic unc ional analysis has equen ly been employed in mul imodal ansla ion s udies. Examples include analyses o sub i les (e.g., Espindola, 2012; Mubenga, 2009, 2010), ansla ions o ad e isemen s (e.g., Feng & Espindola, 2013; Millán-Va ela, 2004), child en’s pic u e books (Van Mee be gen, 2010) and magazines (Chueasuai, 2013). The p ominence o he sys emic unc ional app oach o mul imodali y in ansla ion esea ch is also emphasised in Luis Pé ez-González’s a icle Mul imodali y in T ansla ion and In e p e ing S udies, w i en as a gene al in oduc ion o he opic o A Companion o T ansla ion S udies (2014b). In he a icle, he au ho in oduces he a i al and he p omise o he sys emic unc ional app oach, e e ing o i h oughou he Tuominen, Jiménez Hu ado & Ke ola 8 a icle as “mul imodal heo y”, which could e en be in e p e ed as sugges ing ha mul imodali y is synonymous wi h he sys emic unc ional app oach. Sys emic unc ional analysis can be complimen ed o o e ing sys ema ic p ocedu es o analysing mul imodal ma e ial (Ma inec & Salway, 2005, p. 341). A possible limi a ion o he me hod, howe e , is ha , based on exis ing s udies, he me a unc ional analyses o di e en modes, pe o med sepa a ely, canno always be combined in a p oduc i e manne ( o de ailed e iews o me a unc ional analyses in mul imodal ansla ion s udies, see, o example, Ke ola, 2018). I he analyses canno be combined, wha we a e discussing is a p ocedu e o p oducing sepa a e, monomodal analyses ins ead o a mul imodal analysis o he in e ac ion o he modes. Ano he conce n ha has been epo ed abou he analyses o mul imodal ex s is he p oblem o “in ini e de ail”: me a unc ional analyses o en p oduce pains akingly de ailed desc ip ions o he componen s o mul imodal ex s bu hey a ely o mula e gene aliza ions abou mul imodal meaning cons uc ion (Fo ce ille, 2007, p. 1236). 3.2.2 Co pus-based mul imodal ansla ion esea ch When we e e o he quan i y and ep esen a i eness o he da a being o pa amoun impo ance in da a collec ion, we a e e e ing o co pus-based T ansla ion S udies. Gene ally speaking, co pus analysis is ha dly new in T ansla ion S udies, since i has a long and ui ul his o y as a esea ch me hod (Saldanha & O’B ien, 2013). This is e lec ed in he conside able numbe o pape s ha p esen s udies on how o sol e di e en ansla ion p oblems o ha con i m di e en hypo heses ega ding ansla ion as ei he a p ocess o a p oduc . Co pus analysis has been success ully applied o he acquisi ion o ansla ion compe ence (Quinci, 2015; Rod íguez Inés, 2013), o ansla ion uni e sals (Hal e son, 2007), o he ansla ion o s yle, and o he desc ip ion o ex gen es and e en o specialised ex s (Jiménez C espo, 2008; López Rod íguez, 2007. Howe e , all o hese s udies ha e a ge ed monomodal ex s o only one mode o a mul imodal ex . When co pus analysis is applied o mul imodal ex s, howe e , he scena io unde goes a comple e ans o ma ion. Because o he di e en modes in mul imodal ex s, i is s ill no possible o align an ST au oma ically wi h a a ge ex (TT), o e en o do so wi h he di e en modes in he ex . Despi e he apid p og ess being made wi h echnology, compu e scien is s ha e s ill no been able o c ea e a p og am capable o achie ing his ea . Indeed, a e a somewha a duous adap a ion p ocess, ce ain so wa e applica ions, such as A las. i, An Conc and MQDA, a e able only o ela e isual and audio uni s semi-au oma ically wi h he TTs ha esul om he ansla ion o hese modes. In he co pus-based analysis o mul imodal ansla ion, howe e , hese applica ions a e ye unable o acili a e he segmen a ion o he ex in o uni s o analysis, no a e hey able o acili a e he subsequen seman ic anno a ion o hese uni s. These limi a ions cons i u e a bo leneck in his inno a i e ype o co pus esea ch. In con as , adi ional co pus s udies ha e managed o mee hese challenges success ully (Rod íguez-Inés, 2017, p. 268). This is because he segmen a ion o monomodal ex ual co po a is based ei he on di e en le els o linguis ic s uc u e (mo pheme, lexeme, syn agm, e c.) o on s uc u al aspec s o a ex (pa ag aphs, lines, e c.). Un o una ely, his is no he case wi h co pus-based s udies o mul imedia o mul imodal ex s, in which segmen a ion is conside ably mo e complex. Wha has s ill no been s udied su icien ly in co pus-based s udies a e he g amma s o ce ain semio ic modes and he ex s in which hey ope a e— o ins ance, dynamic o s a ic images, ges u es and sound ypes. Acco dingly, when he esea che ’s objec i e is o di ide a mul imodal ST in o isual, sound and/o ac ile uni s, his segmen a ion mus be based on a de ailed analysis o i s componen s, namely, he semio ic codes i is composed o . Why me hods ma e 9 Fo example, in ilm s udies, he Eu opean (Case i & di Chio, 1990), and Ame ican schools (Bo dwell & Thompson, 1986) ha e p oposed in e es ing heo ies ha a e cu en ly being applied o he segmen a ion o audio isual co po a. In heo ies abou he in e p e a ion o a and a is ic educa ion he e is a long adi ion o segmen ing and decomposing s a ic wo- and h ee-dimensional images in o codes and smalle componen s in o de o be able o in e p e hei a is ic composi ion and unc ion (Dondis, 1973). Howe e , hese ad ances in di e en disciplines ha ocus on he g amma ical aspec s o hei own ypes o ex ha e s ill no been ecu en ly o sys ema ically applied in co pus-based s udies o mul imodal ansla ed ex s, al hough some a emp s ha e been made o do so (Pé ez Payá, 2010). Ano he ea u e o co pus s udies is he complex agging sys em ha mus be applied o mul imodal ex s when s udying a ce ain phenomenon. This way o a aching in o ma ion o ex segmen s o o he objec o analysis is ex emely p oduc i e in monomodal co po a, because i is mainly au oma ed. Bu when a co pus is mul imodal, ex agging is a ely pe o med because i is ex ao dina ily di icul o do. The e ha e been a emp s o ag ges u es and unc ions in o he languages, howe e . A case in poin is he agging done by he ELAN POS agge o B i ish sign language (i.e. he Digging o Signs P ojec /BSL Co pus P ojec ). E en co pus-based in e p e ing s udies do no use any ype o agging o he sup a-segmen al elemen s o discou se. In his sense, mos mul imodal co po a a e o he ype desc ibed in Abuczki and Ghazaleh (2013): Simila ly o co pus-d i en app oaches ha s udy lexical bundles (mul i-wo d sequences) (Bibe 2010: 170–172), some o he mul imodal co pus esea ches a e inspi ed by he no ion o semio ic bundles (A za ello e al. 2005) whe e modeling language p oduc ion includes he manipula ion o esou ces as well as ges u e and alk. Some unc ional anno a ion schemes (Allwood e al. 2007) y o code he meaning ela ions be ween ges u es and co-occu ing speech in a sys ema ic way, and label communica i e e en s acco ding o he alignmen o speech and ges u e. (p. 89) In some mul imodal ex s, such as audio isual ex s o hose in which s a ic images a e pa o he mul imodali y, no sys ema ic agging o he cha ac e is ics ha de ine di e en ex gen es is being done ye . Finally, because o he p oblems and limi a ions ou lined abo e, mul imodal ex agging is gene ally pe o med manually o semi-au oma ically. Needless o say, his p ocess is slow and complica ed. I also equi es he agging se o be p o essionally and concep ually jus i ied, he selec ion o which p esumably depends on he esea che ’s objec i e. Besides, hese agging se s should be based on heo ies ha ha e al eady explained he issue unde analysis (Göp e ich, 2010). Fu he mo e, he agging p ocess mus be subjec o a s ic ly egula ed ag eemen be ween hose who pe o m he agging (an in e agge ag eemen ). All o his e iden ly equi es consensus be ween di e en esea ch ini ia i es so ha a cohe en agging sys em can be adap ed speci ically o T ansla ion S udies. Only in his way will analyses be mo e p oduc i e. Du ing he agging p ocess, o ins ance, agge s may consis en ly disag ee on ce ain exempla s o ypes o exempla . Disag eemen may be conside ed o be an indica ion ha he heo e ical de ini ion o an ins ance lacks cla i y and accu acy and he e o e equi es ede ini ion (o e en u he heo ising) (Ho y & La id, 2010, p. 30). S ill, he e a e c i ical oices which claim ha co pus s udies decon ex ualise he objec o s udy. Some academics claim ha co pus s udies do no o e a eal bo om-up app oach; mo eo e , he lack o a su icien ly la ge o ep esen a i e co pus can also in alida e esul s. Acco ding o E ison (2010, p. 132), combining quali a i e and quan i a i e da a is pa icula ly sui able o small domain-speci ic co po a. O he schola s a i m ha a smalle sample pe mi s a mo e in-dep h in e p e a ion o he esul s (Sole Gallego, 2013, p. 192); i can also pe mi a bo om-up analysis based on co pus da a ha Tuominen, Jiménez Hu ado & Ke ola 16 4.4 In e p e ing and unde s anding ansla ed mul imodal messages As we discussed in sec ion 3.2.3, mul imodali y p esen s an added dimension o meaning- making o hose who a e ying o unde s and he ansla ed mul imodal message. The e o e, he pe spec i e o ecep ion o e s impo an opics o esea ch, as we see in he i e a icles in his sec ion. The me hods used o engage wi h he opic ange om hose employed in Cogni i e Neu oscience o ocus g oups; hey demons a e he a ie y o ele an esea ch ques ions and me hods ha ela e o mul imodali y. Mul imodal communica ion is also cen al o knowledge ans e in di e en a eas and domains o specialised knowledge, as e lec ed in he ex gen es ypical o science and echnology. In ac , adi ional ins umen s o knowledge access, such as me apho s, a e also being s udied om a mul imodal pe spec i e. The i s wo a icles in his sec ion deal wi h he accessibili y o isual in o ma ion in he specialised ield o medical communica ion. The a icle by Ma ibel Te cedo and Alicia Casado-Valenzuela examines he ole o mul imodal communica ion as a way o accessing medical knowledge. Mo e speci ically, hey analyse isual images ha ac i a e a clea ly me apho ical s uc u e. The au ho s examine how his acili a es he gene al public’s access o medical in o ma ion. Fo his pu pose, hei assump ion is ha in mul imodal ex p oduc ion he selec ion o sui able images is c ucial o knowledge ans e in he sphe e o specialised ansla ion. This s udy seeks o ind answe s o wo esea ch ques ions. The i s seeks o iden i y he ype o images wi h a me apho ical s uc u e ha a e mos equen ly used in medical ex s o he gene al public and illus a e complex medical concep s. The second esea ch ques ion aims o asce ain whe he me apho ical images a e p e e ed o non- me apho ical ones and also whe he he me apho s in hese images acili a e use s’ access o knowledge. To add ess he i s ques ion, he au ho s base hei s udy on cogni i e heo ies ega ding me apho ical images and demons a e hei use ulness o he ansla ion p ocess by p o iding a ypology o hese images o en also p esen in o he cul u es. Fu he mo e, he s udy uses a da a-d i en app oach. This was possible hanks o Va iMed, a da abase o mul imodal ex s. The esul s show ha me apho is a key elemen in he ansmission o expe knowledge. The second esea ch ques ion is add essed expe imen ally. The au ho s he e o e analyse he eal- ime knowledge access o he a ge ed ecei e s by means o isual me apho ical elemen s. This s ep is essen ial be o e deciding which images a e mos sui able in mul imodal communica ion (including mul ilingual con ex s). Fo his pu pose, hey adminis e a double ques ionnai e o a se o esponden s, paying pa icula a en ion o i s ecological alidi y. The opic o he a icle by Juan An onio P ie o-Velasco and Vicen Mon al - Resu ecció is he comp ehensibili y o mul imodal pa ien in o ma ion guides, which a e o en he esul o he in alingual and in e modal ansla ion o specialised medical ex s in o a non-specialis in o ma ional ex . Illus a ions a e ypically used o suppo he e bal message and make i mo e comp ehensible in hese kinds o con ex s, bu we know li le abou how isual in o ma ion may imp o e comp ehensibili y. This a icle add esses ha gap in he esea ch. P ie o-Velasco and Mon al -Resu ecció conduc ed a pilo s udy in which hey used a su ey and a ocus g oup o collec bo h quan i a i e and quali a i e da a om ac ual eade s o pa ien in o ma ion guides; his hey did in o de o ind ou wha kinds o illus a ions bes suppo pa ien s’ unde s anding o he in o ma ion p o ided o hem. Thei indings sugges ha illus a ions do indeed play a ole in he comp ehensibili y o hese medical ex s, and u he esea ch is needed o unde s and his comp ehension p ocess be e . Whe eas he i s wo a icles in his sec ion employ bo h mul imodal analysis me hods and empi ical esea ch wi h use s o explo e he accessibili y o medical ex s, he o he h ee a icles concen a e mo e speci ically on he p ocesses o eading, Why me hods ma e 17 in e p e ing and unde s anding ansla ed mul imodal ex s. The i s is Olli Philippe Lau enbache ’s heo e ical p oposal o he manne in which mul imodal ex s a e ead and in e p e ed. The a icle in es iga es he unc ion o edundancy in mul imodal communica ion and he way he phenomenon o edundancy can be used as a c ucial elemen in mul imodal meaning-making. The me hodological amewo k o he a icle is based on ele ance heo y and Kin sch’s Cons uc ion-In eg a ion Model. Lau enbache p oposes a model o a eading p ocess o mul imodal ex s ha is based on he de ec ion o igge s imuli ha unc ion pa ly h ough edundancy. This model has implica ions o he ansla ion p ocess, because any changes in he edundancy s uc u e may also in luence ecep ion. Eliisa Pi käsalo’s a icle explo es he use o ocus-g oup discussions as a da a- collec ion me hod o examining he mul imodal eading expe ience o comics o , mo e pa icula ly, g aphic no els. The da a o he s udy we e collec ed om eade s om wo cul u al backg ounds: one da a-collec ion session was held in I aly, whe e comics ha e a long adi ion, and he o he in Hunga y, whe e comic-s yle li e a u e has no adi ionally been popula . Pi käsalo’s s udy indica es ha he cul u al backg ound o comic eade s, including he adi ion o eading comics, may ha e an e ec on he way eade s unde s and he isual language o comics, which should also a ec he wo k o comic ansla o s. The con ibu ion o An onio Ja ie Chica Núñez connec s h ee axes a ound which T ansla ion S udies migh e ol e in he u u e. Fi s o all, mul imodali y and i s semio ic implica ions a ST le el may de e mine he mac o- and mic o-s uc u al p ope ies o he TT and in luence i s seman ic ea u es. Secondly, he implica ions o mul imodali y can be be e pe cei ed a wha we may call he “ ansla ion e en ”, namely he ansla o ’s p oduc ion phase plus he ecep ion phase on he side o he audience. T ansla o s need o emb ace a mo e complex and in e wined semio ic con igu a ion o he ST. The e, a ious communica ion modes shape a mul ilaye ed discou se whose meanings, nuances and in e ac ions need o be ully con eyed in o he TT. Finally, Cogni i e S udies may cons i u e an indispensable heo e ical and me hodological pa adigm acco ding o which o conduc eal and si ua ed analyses o he mul imodal ansla ion p ocess and o accoun o i s in icacies. In his sense, his a icle d aws on se e al con ibu ions om Social Neu oscience and Cogni i e Neu oscience o ocus on he analysis o emo ions in AD. Those heo e ical models allow Chica o es ablish a me hodological app oach based on psychophysics in o de o measu e he eac ion o blind and isually impai ed subjec s o mul imodal emo ional s imuli in AD. As a esul , his app oach may con ibu e o assessing he alidi y and e ec i eness o ansla ion quan i a i ely, ha is, o assess he capaci y o any gi en AD o communica e emo ions o ecipien s in cogni i e e ms. This ype o app oach could be applied equally o o he audio isual ansla ion ypologies whe e mul imodali y plays a decisi e ole. The a icles in his inal sec ion demons a e jus how many open ques ions emain abou he ways in which ansla ions and ansla o s could bes se e hei a ge audiences, especially in si ua ions whe e hose audiences encoun e he mul i-laye ed meaning-making o a mul imodal ansla ion. I is, o cou se, impossible o know how indi idual iewe s, eade s o lis ene s in e p e a message, bu inno a i e heo e ical and expe imen al app oaches a e c ucial in inc easing ou unde s anding and p o iding new ways o explo ing wha ac ually happens when audiences p ocess ansla ions in a mul imodal se ing. 5. Epilogue This collec ion o a icles demons a es how esea che s a e heeding he call o igo ous and inno a i e app oaches ha ake in o accoun all modes o mul imodal ex s and hei Tuominen, Jiménez Hu ado & Ke ola 18 ansla ions, as well as he in e ac ion be ween modes. This includes si ua ions whe e one mode mus ake he place o ano he in ansla ion in o de o acili a e accessibili y. A consis en ocus on mul imodali y ensu es ha he e bal elemen s o a mul imodal message a e no he only ones analysed and ha he special cha ac e is ics o mul imodali y a e also aken in o accoun . Fu he mo e, one o ou main conce ns in selec ing he a icles o his special issue has been hei explici and e lec i e ocus on esea ch me hods. In line wi h his, all he a icles con ain some c i ical discussion on ques ions conce ning esea ch me hodology. Howe e , a icles on a me hodological heme can assume mul iple o ma s. Some o he a icles a e mo e heo e ical in na u e, explo ing he backg ound and he disciplina y unde pinnings o a speci ic me hodological app oach, whe eas o he s p o ide examples o how a ce ain me hod can be applied in p ac ice and o e c i ical e lec ion on he use ulness and applicabili y o his me hod in a pa icula con ex . The key elemen in all o hem is he anspa en desc ip ion and discussion o he me hods used. We hope ha his combina ion o longs anding and new me hods, heo e ical and applied app oaches, and p oduc and p ocess o ien a ion will o e a well- ounded o e iew o he me hodological mul iplici y ha cha ac e ises mul imodal ansla ion s udies oday. Re e ences Abuczki, A., & Ghazaleh, E. B. (2013). An o e iew o mul imodal co po a, anno a ion ools, and schemes. A gumen um, 9, 86–98. Adami, E. (2017). Mul imodali y. In O. Ga cía, N. Flo es, & M. Spo i (Eds.), The Ox o d handbook o language and socie y (pp. 451–472). Ox o d: Ox o d Uni e si y P ess. Al es Veiga, M. (2006). Sub i ling eading p ac ices. In J. Fe ei a Dua e, A. Assis Rosa, & T. Se uya (Eds.), T ansla ion S udies a he in e ace o disciplines (pp. 161–168). Ams e dam: John Benjamins. An onini, R. (2007). SAT, BLT, Spi i Biscui s, and he Thi d Amendmen : Wha I alians make o cul u al e e ences in dubbed ex s. In Y. Gambie , M. Shlesinge , & R. S olze (Eds.), Doub s and di ec ions in T ansla ion S udies: Selec ed con ibu ions om he EST Cong ess, Lisbon 2004 (pp. 153–167). Ams e dam: John Benjamins. Bai s ow, D. (2011). Audio isual p ocessing while wa ching sub i led ilms: A cogni i e app oach. In A. Şe ban, A. Ma amala, & J.-M. La au (Eds.), Audio isual ansla ion in close-up: P ac ical and heo e ical app oaches (pp. 205–219). Be n: Pe e Lang. Bald y, A., & Thibaul , P.J. (2006). Mul imodal ansc ip ion and ex analysis: A mul imedia oolki and cou sebook. London: Equinox. Baños, R., B u i S., & Zano i, S. (2013). Co pus linguis ics and audio isual ansla ion: In sea ch o an in eg a ed app oach. Pe spec i es, 21(4), 483–490. Bo dwell, D., & Thompson, K. (1986). Film a . New Yo k, NY: Al ed A. Knop . B ems, E., & Ramos-Pin o, S. (2013). Recep ion and ansla ion. In Y. Gambie & L. an Doo slae (Eds.), Handbook o ansla ion s udies, Vol. 4, (pp. 142–147). Ams e dam: John Benjamins. Cabezas-Cáce es, C. (2013). Audiodesc ipció i ecepció. E ec e de la eloci a de na ació l'en onació i l'explici ació en la comp ensió ílmica. Tesis doc o al. Re ie ed om h p://hdl.handle.ne / 10803/113556. Cania o, M. (2014). Double meaning e-nego ia ion: I alian ilms in Flande s as cul u al objec s. inTRAlinea Special Issue: Ac oss Sc eens Ac oss Bounda ies. Re ie ed om h p://www.in alinea.o g/ specials/a icle/2075. Case i, F., & Di Chio, F. (1990). Analisi del Film. Milan: Bompiani. Chia o, D., & An onini, R. (2005). The quali y o dubbed ele ision p og ammes in I aly: The expe imen al design o an empi ical s udy. In M. B. Paganelli & N. Maxwell (Eds.), C oss-cul u al encoun e s: Linguis ic pe spec i es (pp. 33–44). Rome: O icina Edizioni. Why me hods ma e 19 Chia o, D. (2007). The e ec o ansla ion on humou esponse: The case o dubbed comedy in I aly. In Y. Gambie , M. Shlesinge , & R. S olze (Eds.), Doub s and di ec ions in ansla ion s udies: Selec ed con ibu ions om he EST Cong ess, Lisbon 2004 (pp. 137–152). Ams e dam: John Benjamins. Chmiel, A., & Mazu , I. (2012). AD ecep ion esea ch: some me hodological conside a ions. In E. Pe ego (Ed.), Eme ging opics in ansla ion: Audio desc ip ion (pp. 57–80). T ies e: EUT. Chueasuai, P. (2013). T ansla ion shi s in mul imodal ex : A case o he Thai e sion o Cosmopoli an. JoST ans, The Jou nal o Specialised T ansla ion, 20, 107–121. C o , W., & C use, D. A. (2004). Cogni i e linguis ics. Ox o d: Ox o d Uni e si y P ess. Desilla, L. (2014). Reading be ween he lines, seeing beyond he images: An empi ical s udy on he comp ehension o implici ilm dialogue meaning ac oss cul u es. The T ansla o , 20(2), 194–214. Dondis, D.A (1973). A p ime o isual li e acy. Camb idge, MA: MIT P ess. Espindola, E. (2012). Sys emic unc ional linguis ics and audio isual ansla ion s udies: A concep ual basis o he s udy o he language o sub i les. DELTA: Documen ação de Es udos em Lingüís ica Teó ica e Aplicada, 28, 495–513. E e e , Y.U. (2015). Recon igu ing My h and Na a i e in Con empo a y Ope a. Blooming on: Indiana Uni e si y P ess. E ison, J. (2010). Wha a e basics o analyzing a co pus? In A. O´Kee e & M. McCa hy (Eds.), The Rou ledge handbook o co pus linguis ics (pp. 122–135). London: Rou ledge. Feng, D., & Espindola, E. (2013). In eg a ing sys emic unc ional and cogni i e app oaches o mul imodal discou se analysis. Ilha do Des e o, 64, 85–110. Fo ce ille, C. (2007). Book e iew: Mul imodal T ansc ip ion and Tex Analysis: A Mul imedia Toolki and Cou sebook. Jou nal o P agma ics, 39(6), 1235–1238. Fuen es Luque, A. (2003). An empi ical app oach o he ecep ion o AV ansla ed humou : A case s udy o he Ma x B o he s’ “Duck Soup”. The T ansla o , 9(2), 293–306. Gene e, G., & Maclean, M. (1991). In oduc ion o he pa a ex . New Li e a y His o y, 22(2), 261–272. Gibbons, A. (2012). Mul imodali y, cogni ion and expe imen al li e a u e. New Yo k, NY: Rou ledge. Ghia, E. (2012). The impac o ansla ion s a egies on sub i le eading. In E. Pe ego (Ed.), Eye acking in audio isual ansla ion (pp. 157–182). Rome: A acne Edi ice. Göp e ich, S. (2010), Da a documen a ion and da a accessibili y in ansla ion p ocess esea ch. The T ansla o , 16(1), 93–124. Halliday, M.A.K. (1978). Language as social semio ic: The social in e p e a ion o language and meaning. London: Edwa d A nold. Halliday, M.A.K. (1985). An in oduc ion o unc ional g amma . London: Edwa d A nold. Halliday, M.A.K., & C. Ma hiessen. (2004) 2013. An In oduc ion o Func ional G amma . 3 d ed. New Yo k: Rou ledge. Hal e son, S.L. (2007). In es iga ing g a i a ional pool in ansla ion: The case o English p og essing cons uc ion. In R. Jääskeläinen, T. Puu inen, & H. S o esbu y (Eds.), Tex , p ocesses, and co po a: Resea ch inspi ed by Sonja Ti kkonen-Condi (pp. 175–196). Joensuu: Joensuu Yliopis opaino. Hand o d, M. (2010). Wha can a co pus ell us abou specialis gen es? In A. O’Kee e & M. McCa hy (Eds.), The Rou ledge handbook o co pus linguis ics (pp. 254–269). London: Rou ledge. Ho y, E., & La id, J. (2010). Towa ds a ‘science’ o co pus anno a ion: A new me hodological challenge o co pus linguis ics. In e na ional Jou nal o T ansla ion, 22(1), 13–36. Jewi , C. (2014). Mul imodal app oaches. In S. No is & C. D. Maie (Eds.), In e ac ions, images and ex s: A eade in mul imodali y (pp. 127–136). Bos on, MA: De G uy e . Jiménez C espo, M.A. (2012). Los o los in ansla ion: a con as i e co pus-based s udy o o iginal and localized US websi es. JoST ans, The Jou nal o Specialised T ansla ion, 17, 126–173. Kaindl, K. (2004). Mul imodali y in he ansla ion o humo in comics. In E. Ven ola, C. Cha les, & M. Kal enbache (Eds.), Pe spec i es on mul imodali y (pp. 173–192). Ams e dam: John Benjamins. Kaindl, K. (2013). Mul imodali y and ansla ion. In C. Millán & F. Ba ina (Eds.), The Rou ledge handbook o ansla ion s udies (pp. 257–269). London: Rou ledge. Ke ola, A. (2016). Towa ds a mul imodally o ien ed heo y o ansla ion: A cogni i e amewo k o he ansla ion o illus a ed echnical ex s. T ansla ion S udies, 9(1), 67–81. Tuominen, Jiménez Hu ado & Ke ola 20 Ke ola, A. (2018). Wo d–image in e ac ion in echnical ansla ion: S uden s ansla ing an illus a ed ex . Ac a Uni e si a is Tampe ensis 2364. Tampe e: Tampe e Uni e si y P ess. Kokkola, S., & Ke ola, A. (2015). Thinking ou side he “Me hods Box”: New a enues o esea ch in mul imodal ansla ion. In D. Rells ab & N. Siponkoski (Eds.), Rajojen dynamiikkaa, G änse nas dynamik, Bo de s unde nego ia ion, G enzen und ih e Dynamik. VAKKI-symposiumi XXXV 12.– 13.2.2015 (pp. 219–228). Vaasa: VAKKI Publica ions 4. Re ie ed om h p://www. akki.ne / publica ions/2015/VAKKI2015_Kokkola&Ke ola.pd . K ess, G. (2010). Mul imodali y: A social semio ic app oach o con empo a y communica ion. New Yo k, NY: Rou ledge. K ess, G., & an Leeuwen, T. (1996) 2006. Reading images: The g amma o isual design. London: Rou ledge. K ess, G., & an Leeuwen, T. (2001). Mul imodal discou se: The modes and media o con empo a y communica ion. London: A nold. K uge , J.-L., & Dohe y, S. (2018). T iangula ion o online and o line measu es o p ocessing and ecep ion in AVT. In E. Di Gio anni & Y. Gambie (Eds.), Recep ion s udies and audio isual ansla ion (pp. 91–109). Ams e dam: John Benjamins. Lång, J., Mäkisalo, J., Gowases, T., & Pie inen S. (2013). Using eye acking o s udy he e ec o badly synch onized sub i les on he gaze pa hs o ele ision iewe s. New Voices in T ansla ion S udies, 10, 72–86. Langacke , R. W. (1987/2008). Cogni i e g amma : A basic in oduc ion. Ox o d: Ox o d Uni e si y P ess. López Rod iguez, C. I. (2007). Unde s anding scien i ic communica ion h ough he ex ac ion o he concep ual and he o ical in o ma ion codi ied by e bs. Te minology, 13(1), 61–84. Mangi on, C. (2016). Recep ion o game sub i les: An empi ical s udy. The T ansla o , 22(1), 72–93. Mangi on, C. (2018). Recep ion s udies in game localisa ion: Taking s ock. In E. Di Gio anni & Y. Gambie (Eds.), Recep ion s udies and audio isual ansla ion (pp. 277–296). Ams e dam: John Benjamins. Ma inec, R., & Salway, A. (2005). A sys em o image– ex ela ions in new (and old) media. Visual Communica ion, 4(3), 339–374. Millán-Va ela, C. (2004). Explo ing ad e ising in a global con ex : Food o hough . The T ansla o , 10(2), 245–267. Mubenga, K. S. (2009). Towa ds a mul imodal p agma ic analysis o ilm discou se in audio isual ansla ion. Me a, 54(3), 466–484. Mubenga, K. S. (2010). In es iga ing no ms in in e lingual sub i ling: A sys emic unc ional pe spec i e. Pe spec i es: S udies in T ansla ology, 18(4), 251–274. O’Hagan, M. (2009). Towa ds a c oss-cul u al game design: An explo a i e s udy in unde s anding he playe expe ience o a localised Japanese ideo game. JosT ans. The Jou nal o Specialised T ansla ion, 11, 211–233. O’Hallo an, K. L., & Smi h, B.A. (2011). Mul imodal s udies. In K. L. O’Hallo an & B. A. Smi h (Eds.), Mul imodal s udies: Explo ing issues and domains (pp. 1–13). New Yo k, NY: Rou ledge. Oi inen, R., Ke ola A., & Ga a ini, M. (2018). T ansla ing pic u ebooks: Re oicing he e bal, he isual and he au al o a child audience. New Yo k, NY: Rou ledge. O e o, P., Dohe y, S., K uge , J.-L., Ma amala, A., Pede sen, J., Pe ego, E., Rome o-F esco, P., Ro i a- Es e a, S., Sole -Vilageliu, O., & Sza kowska, A. (2018). Conduc ing expe imen al esea ch in audio isual ansla ion (AVT): A posi ion pape . JoST ans, The Jou nal o Specialised T ansla ion, 30, 105–126. Page, R. (2010). In oduc ion. In R. Page (Ed.), New pe spec i es on na a i e and mul imodali y (pp. 1– 13). New Yo k, NY: Rou ledge. Pauwels, L. (2012). A mul imodal amewo k o analyzing websi es as cul u al exp essions. Jou nal o Compu e -Media ed Communica ion, 17, 247–265. Pe ego, E., Del Missie , F., Po a, M., & Mosconi, M. (2010). The cogni i e e ec i eness o sub i le p ocessing. Media Psychology, 13(3), 243–272. Pe ego, E. (2012). In oduc ion. In E. Pe ego (Ed.), Eye acking in audio isual ansla ion (pp. 7–11). Rome: A acne Edi ice. Why me hods ma e 21 Pe ego, E., Laskowska, M., Ma amala, A., Remael, A., Robe , I. S., Sza kowska, A., Vila ó, A., & Bo i oli, S. (2016). Is sub i ling equally e ec i e e e ywhe e?: A i s c oss-na ional s udy on he ecep ion o in e lingually sub i led messages. Ac oss Languages and Cul u es, 17(2), 205–229. Pé ez Payá, M. (2010). Reco es de cine audiodesc i o: El lenguaje cinema og á ico en Tagge i Imagen y su e lejo en la audiodesc ipción. In C. Jiménez Hu ado, A. Rod íguez, & C. Seibel (Eds.), Un co pus de cine: Teo ía y p ác ica de la audiodesc ipción (pp. 111–179). G anada: T agacan o. Pé ez-González, L. (2014a). Audio isual ansla ion: Theo ies, me hods and issues. London: Rou ledge. Pé ez-González, L. (2014b). Mul imodali y in ansla ion and in e p e ing s udies: Theo e ical and me hodological pe spec i es. In S. Be mann & C. Po e (Eds.), A companion o ansla ion s udies (pp. 119–131). Chiches e : Wiley-Blackwell. Quinci, C. (2015). T ansla o s in he making: An empi ical longi udinal s udy o ansla ion compe ence and i s de elopmen (Unpublished doc o al disse a ion). Uni e si á degli S udi di T ies e, T ies e. Ramos Ca o, M. (2018). Los es udios de ecepción en T aducción Audio isual: Aspec os me odológicos. In A. Rojo (Ed.), Diseños y mé odos de in es igación en aducción (pp. 99–124). Mad id: An h opos. Rod íguez-Inés, P. (2013). ¿Cómo aducen aduc o es y p o eso es de idiomas?: Es udio de co pus. Me a, 58(1), 165–190. Rod íguez-Inés, P. (2017). Co pus-based insigh s in o cogni ion. In J. W. Schwie e & A. Fe ei a (Eds.), The handbook o ansla ion and cogni ion (pp. 265–289). Hoboken, NJ: John Wiley & Sons. Saldanha, G., & O’B ien, S. (2013). Resea ch me hodologies in ansla ion s udies. Manches e : S . Je ome. Se a ini, F. (2010). Reading mul imodal ex s: Pe cep ual, s uc u al and ideological pe spec i es. Child en’s Li e a u e in Educa ion, 41, 85–104. Sole Gallego, S. (2013). La aducción accesible en el espacio mul imodal museog á ico (Unpublished doc o al disse a ion). Uni e sidad de G anada, G anada. So o Almela, J. (2013). La aducción de é minos cul u ales en el con ex o u ís ico español-inglés: Recepción eal en usua ios angló onos. Quade ns. Re is a de T aducció, 20, 235–250. Sza kowska, A., & Ge be -Mo ón, O. (2018). Viewe s can keep up wi h as sub i les: E idence om eye mo emen s. PLoS ONE, 13(6), e0199331. To esi, I. (2007). T ansla ing he isual: The impo ance o isual elemen s in he ansla ion o ad e ising ac oss cul u es. In D. Kenny & K. Ryou (Eds.), Ac oss bounda ies: In e na ional pe spec i es on ansla ion s udies (pp. 38–55). Newcas le: Camb idge Schola s. Tuominen, T. (2012). The a o acciden al eading and inciden al lis ening: An empi ical s udy on he iewing o sub i led ilms. Ac a Uni e si a is Tampe ensis 1794. Tampe e: Tampe e Uni e si y P ess. Tuominen, T. (2018). Mul i-me hod esea ch: Recep ion in con ex . In E. Di Gio anni & Y. Gambie (Eds.), Recep ion s udies and audio isual ansla ion (pp. 69–89). Ams e dam: John Benjamins. Tymoczko, M. (2007). Enla ging ansla ion, empowe ing ansla o s. Manches e : S . Je ome. Van Mee be gen, S. (2010). Nede ländska bilde böcke bli s enska: En mul imodal ö e sä ningsanalys [Du ch pic u ebooks become Swedish: A mul imodal ansla ion analysis]. S ockholm: Uni e si y o S ockholm P ess. Widle , B. (2004). A su ey among audiences o sub i led ilms in Viennese cinemas. Me a, 49(1), 98–101. Yayoi, U.E. (2015). Recon igu ing my h and na a i e in con empo a y ope a. Blooming on, IN: Indiana Uni e si y P ess. d’Ydewalle, G., P ae , C., Ve aillie, K., & Van Rensbe gen, J. (1991). Wa ching sub i led ele ision: Au oma ic eading beha io . Communica ion Resea ch, 18(5), 650–666.