Picture Books as Multimodal Source Texts : Contrastive Analysis as a Tool for Depicting Image-text Interaction

Ke ola, A. (2017). “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”. Es udios de Teo ía Li e a ia. Re is a digi al: a es, le as y humanidades, 6 (12), 49-60. Es a ob a se encuen a bajo una Licencia C ea i e Commons A ibución-NoCome cial-Compa i Igual 4.0 In e nacional. 49 Lib os ilus ados como ex os de pa ida mul imodales El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o Pic u e Books as Mul imodal Sou ce Tex s Con as i e Analysis as a Tool o Depic ing Image- ex In e ac ion Anne Ke ola 1 Recibido: 17/08/2017 Acep ado: 30/08/2017 Publicado: 08/09/2017 Resumen En es e a ículo nos p oponemos: i) mos a el ápido c ecimien o de un nue o ámbi o de es udios den o del amplio ango cubie o po los Es udios de T aducción, conocido como Es udios Mul imodales; ii) desc ibi la in e acción imagen- ex o de las aducciones inlandesas de The Tale o Pe e Rabbi de Bea ix Po e , desde un pun o de is a mul imodal; iii) indaga en el modo en que las imágenes en las aducciones de es e lib o ilus ado pueden habe a ec ado las opciones léxicas ealizadas po su aduc o ; i ) analiza la elación en e las unidades léxicas en los ex os de o igen y de pa ida, y luego compa a es as obse aciones con la in o mación o ecida po la imagen que co esponde a cada unidad; ) inalmen e, de e mina si es posible a i ma que las imágenes mo i an los modos en que las unidades léxicas se han is o aducidas. Palab as cla e T aducción; es udios mul imodales; lib os ilus ados; imagen- ex o. Abs ac The pu poses o his a icle a e: i) o show he apid inc ease o in e es in a new ield o s udies wi hin he la ge scope o T ansla ion S udies, known as Mul imodal S udies; ii) o desc ibe he image- ex in e ac ion o he Finnish ansla ions o Bea ix Po e ’s The Tale o Pe e Rabbi , om a mul imodal poin o iew; iii) o look in o how he images in he ansla ions o his pic u e book may ha e a ec ed i s ansla o ’s wo d choices; i ) o analyze he ela ionship be ween lexical i ems in he sou ce and a ge ex s, and hen compa e hese obse a ions agains he in o ma ion o e ed by he image wi h which he i em is p esen ed; ) inally, o de e mine whe he an explana ion can be de i ed om he images as o why a ce ain i em was ansla ed he way i was. Keywo ds T ansla ion; mul imodal s udies; pic u e books; image- ex . 1 PhD S uden . Uni e si y o Tampe e, School o Language, T ansla ion and Li e a y S udies. Finland. Con ac : anne.ke ol[email p o ec ed] Anne Ke ola 50 Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 In oduc ion he s udy o mul imodali y app oaches communica ion, ep esen a ion and in e ac ion as always consis ing o di e en modes, such as spoken o w i en language, s ill o mo ing images, ges u es, sound, o gaze. Du ing he las decade o so, mul imodali y has been he subjec o apidly inc easing esea ch in e es in a a ie y o ields o s udy, including T ansla ion S udies [TS] (see e.g. Kaindl 2004, Taylo 2003, 2004, and Hi onen y Tii ula 2010). Pic u e books [PB] a e p onouncedly mul imodal; hei message is c ea ed in he in e ac ion o wo ds and images. Howe e , he mul imodali y o PB has only e y ecen ly begun o be add essed in esea ch (Moya 2010, 2013). Ex ensi e PB esea ch ca ied ou o e he las decades, ecen de elopmen s in he s udy o mul imodali y and TS a e ields ha need o be linked h ough con e ging ques s. A mul imodal sou ce ex [ST] consis ing o w i en wo d and image, such as a PB, can be hough o consis ing o wo sepa a e pa s, he e bal ST and he isual ST, which none heless ope a e seamlessly o c ea e he ex ’s o e all meaning. The ansla o s o such ex s in e p e in o ma ion o e ed by wo sepa a e modes. Fu he mo e, he in o ma ion con eyed by hese wo modes is always di e en in a leas some aspec s. The wo ds o a PB can e o lessly exp ess wha hings and cha ac e s a e called, wha he cha ac e s hink and say, when e en s ake place, and so on (G aham 2004: 211). Unlike images, wo ds may also con ey causal and empo al in o ma ion (Nikolaje a 2002: 97). Images, o hei pa , can be used o exp ess wha he cha ac e s and loca ions look like exac ly. They may also qui e accu a ely con ey he cha ac e s’ emo ions and he ela ionships be ween hem by showing hei acial exp essions, body language and he dis ances be ween hem (Nikolaje a 2002: 93). In ac , wo ds and images may con ey he same in o ma ion only e y supe icially (Nodelman 1988: 193). When p esen ed oge he , wo ds and images may a o d new kinds o meaning. As PB heo is Pe y Nodelman (1988: 220) s a es, he in o ma ion o e ed by he wo ds o a PB change he way he eade cons ues he meaning o he images, and he images change he way he eade in e p e s he wo ds. This gi es us a eason o suspec ha isually p esen ed in o ma ion migh , in some cases, al e he way in which he wo ds a e ansla ed in mul imodal ex s combining wo ds and images. I he image, in one way o ano he , changes he meaning o a ce ain wo d wi h which i is p esen ed, he ansla ion o he wo d migh no longe be wha could be conside ed as i s mos ob ious “wo d- o -wo d” ansla ion. The pu pose o his a icle is o disco e whe he he images o a ansla ed PB may ha e a ec ed i s ansla o ’s wo d choices. I analyze he ela ionship be ween lexical i ems in he sou ce and a ge ex s [TT] o iden i y any al e a ions and di e ences, and hen compa e hese obse a ions agains he in o ma ion o e ed by he image wi h which he i em is p esen ed. In o he wo ds, I aim o de e mine whe he he images p o ide an explana ion as o why a ce ain i em was ansla ed he way i was. The da a o he s udy is comp ised o he Finnish ansla ions o The Tale o Pe e Rabbi by Bea ix Po e . The Tale o Pe e Rabbi is an excep ional PB in he sense ha i has been e-illus a ed dozens o imes. The publishe o he book ailed o egis e he book o copy igh s in he Uni ed S a es when hey i s s a ed ma ke ing he book in he coun y in ea ly 20 h cen u y. The e o e, anyone could compile a new se o illus a ions and ma ke he book wi hou es ic ions (Mackey 1998: xix). Ou o he a ious illus a ed e sions, h ee ha e been ansla ed in o Finnish. The o iginal book illus a ed by Po e he sel was ansla ed in 1963 by Rii a Bjö klund. In 2003, Ri a Toi ola ansla ed a e sion illus a ed by Cyndy Szeke es and, in 2005, Ri a B ande ansla ed a sligh ly ab idged e sion o he s o y illus a ed by Lisa McCue. Reading he same e bal s o y coupled wi h h ee di e en T Anne Ke ola Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 51 illus a ions p o ides qui e an excep ional oppo uni y o examine whe he ce ain wo ds a e a o ded di e en kinds o meanings when p esen ed oge he wi h di e en images. Theo e ical backg ound Gun he K ess and Theo an Leeuwen (1998: 186), wo o he mos no able schola s in he ield o mul imodal esea ch, s a e ha all ex s a e mul imodal. A pe son who alks c ea es hei message using no only wo ds bu also in ona ion, hy hm, acial exp essions, ges u es and pos u es. I one would ecei e he same e bal message in a w i en o m, he ma e ial on which he message is w i en, he on o he handw i ing as well as he layou used would all a ec he way in which he message is in e p e ed. Each mode used in he c ea ion o a message impa s some hing abou he in en ions and he pe sonali y o i s sende (Lemke 2002: 302). I is he e o e impossible o c ea e a message using only one indi idual mode. As K ess and an Leeu en (1998: 186) unde line, no ex analysis ha ails o accoun o he ex ’s mul imodal aspec s can explain all laye s o meaning p esen in he ex . One o he cen al claims o mul imodal esea ch is ha all modes a e o equal impo ance (Jewi 2009b: 13). Wi hin mul imodal esea ch, language –al hough ha ing been o emendous esea ch in e es in he pas – is me ely one meaning making esou ce amongs many. This means ha wi hin a mul imodal ex combining wo ds and images, as i is he case in PB, wo ds and images play equally impo an oles in c ea ing he o e all message o he book. I may hence be claimed ha when ansla ing a PB, he in o ma ion o e ed by he e bal and he isual modes a e o equal impo ance. Ano he impo an p inciple o mul imodal esea ch is ha each mode wi hin a mul imodal ex pa icipa es in c ea ing i s o e all meaning by ealizing hei own communica i e ask, exp essing wha each mode exp esses bes (Jewi 2009c: 15). Analogically, his means ha when s udied sepa a ely, each mode o a mul imodal ensemble is incomple e and canno ep esen all meanings p esen in he mul imodal whole. Resea ch add essing he ansla ion o mul imodal ex s should, o his e y eason, always co e he analysis o each o he modes in ol ed. As s a ed abo e, PB combine wo di e en modes, he isual and he e bal, images and w i en language, bu how does his a ec he eade s o PB? Jay Lemke (2002: 303) discusses wha he calls “ he essen ial incommensu abili y” be ween di e en modes: A e bal ex is ne e able o p oduce he same and only he same meanings as an image, and ice e sa. In Lemke’s wo ds, his incommensu abili y ine i ably leads o genuinely new meanings being c ea ed in he combina ion o modes (2002). The meanings c ea ed in he combina ion o modes a e he e o e always g ea e han he sum o he meanings c ea ed by he same modes indi idually (Flewi , Hampel, Hauck & Lancas e 2009: 46). In acco dance wi h he abo e obse a ion, he combina ion o wo ds and images in a PB always c ea es genuinely new meanings. When we ead PB, i is possible o us o ind meanings ha a e p esen only when he wo modes a e p esen ed oge he ; meanings ha a e no p esen in ei he mode alone. I migh hen be possible ha he genuinely new meanings c ea ed wi hin he mul imodal ST a e con eyed o he ansla o ’s in e p e a ion o he ex and, mo eo e , occasionally o he ansla ion choices he ansla o makes. Resea ch me hod The aim o his s udy is o disco e whe he he images o a ansla ed PB may ha e a ec ed i s ansla o ’s wo d choices. Wi h a iew o de e mining his, I i s con as he wo ds o exp essions ha can be hough o occupying iden ical posi ions in he ST and TT and analyze Anne Ke ola 52 Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 how hey co espond o each o he . I hen compa e hese obse a ions o he in o ma ion he isual ST exp esses abou a gi en i em o ind ou whe he he images can be used o explain why he i ems we e ansla ed as hey we e. In he i s s age o my analysis I di ide he TT ph ases in o lexical i ems (wo ds, ph asal e bs, colloca ions, e c.) and compa e hem wi h hei coun e pa s in he e bal ST (in his a icle, I e e o hese wo i ems as “equi alen s” e en in cases whe e no imaginable o m o equi alence exis s be ween hem). I hen aim o iden i y simila i ies and di e ences be ween he meanings he con as ed i ems ca y. The esea ch could hence be conside ed as a o m o c oss-linguis ic con as i e analysis wi h a ocus on analyzing ansla ional equi alence. One may easily hink o di e en ela ionships exis ing be ween a ST and a TT i em. The pe haps mos common –and, in mos cases, desi able– si ua ion would be a wo d- o - wo d ansla ion. By his somewha assailable e m I mean a ela ionship whe e he wo i ems co espond o each o he as closely as i is possible be ween wo languages o , in Eugene Nida’s (1969, 495) e ms, whe e he ansla ion is “ he closes na u al equi alen o he message o he sou ce language” (a simple example o his could be he English wo d “ca ” ansla ed in o Spanish as “ga o”). Ano he possible ela ionship be ween he wo i ems is a si ua ion whe e he TT i em is mo e p ecise in meaning han he e bal ST i em, in o he wo ds, whe e he ansla o mo es om hype nyms o hyponyms, om supe o dina e o subo dina e e ms ( o ins ance, he English wo d “animal” ansla ed in o Spanish as “ga o”). In he opposi e case, he ansla o mo es om hyponyms o hype nyms, he TT i em being mo e gene al in meaning han he e bal ST i em ( he English wo d “ca ” being ansla ed in o Spanish as “animal”). I migh also be possible ha he lexical i em is changed in ansla ion in a way ha co espondence be ween he a ge and e bal ST i ems can ha dly be said o exis ( o example, he English wo d “ca ” ansla ed in o Spanish as “pe o”). I may also be ha a lexical i em in he TT has no equi alen in he e bal ST, in o he wo ds, some hing has been added in he ansla ion p ocess. The opposi e case would be a lexical i em being le ou in he ansla ion p ocess al oge he , and he e bal ST i em would hen ha e no equi alen in he TT. We may now compile he ollowing six ca ego ies o desc ibe he ela ionship be ween each elemen o he TT and i s equi alen in he e bal ST: A. The TT i em co esponds o i s e bal ST equi alen wo d- o -wo d. B. The TT i em is mo e p ecise in meaning han i s e bal ST equi alen . C. The TT i em is mo e gene al in meaning han i s e bal ST equi alen . D. The TT i em does no co espond o i s e bal ST equi alen . E. The TT i em has no equi alen in he e bal ST. F. The e bal ST i em has no equi alen in he TT. I p esen his s age o he analysis in a able whe e I di ide he ansla ed sen ence in o lexical i ems placed on he ows o he i s column (wi h English back- ansla ions in squa e b acke s). The e bal ST equi alen s o hese elemen s a e placed on he ows o he second column. The hi d column desc ibes how he Finnish and English exp essions co espond o each o he (op ions A–F). To illus a e he me hod, I p esen an analysis o a sen ence om my da a, aken om he ansla ion o Cyndy Szeke es’ illus a ion e sion. The e bal ST goes: “M . McG ego hung up he li le jacke and he shoes o a sca ec ow o igh en he blackbi ds” (Po e 2002) and i s Finnish ansla ion goes “He a Jy y ipus i Pe e in pikku akin ja kengä puu a haansa linnunpelä imeksi as aille” [M . Jy y hung up Pe e ’s li le jacke and shoes in his ga den o a sca ec ow o he h ushes] (Po e 2003). Anne Ke ola Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 53 Table 1. An example o a con as i e analysis be ween a ge and e bal ST i ems. TT i em Ve bal ST equi alen How i ems co espond He a Jy y [M . Jy y] M . McG ego A ipus i [hung up] hung up A Pe e in [Pe e ’s] E pikku [li le] Li le A akin [jacke ] he - - jacke A ja [and] And A kengä [ he shoes] he shoes A puu a haansa [in his ga den] E linnunpelä imeksi [ o a sca ec ow] o a sca ec ow A o igh en F as aille [( o he) h ushes] he blackbi ds C As i can be seen om he able, mos TT i ems co espond o hei e bal ST equi alen s wo d- o -wo d (op ion A). Howe e , wo o he elemen s (Pe e in, meaning “Pe e ’s” and puu a haansa, “in his ga den”) do no ha e equi alen s in he e bal ST (op ion E), and he e b “ o igh en” does no ha e an equi alen in he TT (op ion F). Also, he wo d “blackbi d” (mus a as as, “ he black h ush” in Finnish) has been ansla ed wi h an exp ession wi h a mo e gene al meaning (op ion C), as as, meaning “ h ush” ( he blackbi d is a subspecies o h ush). A e analyzing how he ST and TT i ems co espond o each o he , I hen compa ed hese esul s agains he in o ma ion he isual ST p o ided he ansla o s. The second s age o my analysis is wha I call a mul imodal ST analysis, whe e I con as he isual and e bal ST, in o he wo ds, I go h ough he da a sen ence by sen ence and compa e each one o he image wi h which i is p esen ed. The ques ions 2 I pose in he mul imodal ST analysis a e he ollowing: Wha is p esen ed edundan ly be ween isual and ex ual elemen s? Wha is p esen ed only in he isual mode? Wha is p esen ed only in he e bal mode? Wha is p esen ed di e en ly in he wo modes? Which elemen s change hei meaning in he combina ion o he wo modes? The las ques ion o he lis e e s o cases whe e he isual and e bal ST con ey sligh ly di e en bu closely ela ed aspec s o meaning abou a gi en i em. The eade hen ge s addi ional in o ma ion abou he i em, o example in cases whe e he e bal ST exp esses wha is done and he isual ST exp esses exac ly how i is done. To illus a e he me hod, I p esen he analysis o he sen ence used in he example abo e. Since I was denied he pe mission o p in he images o my da a, I am limi ed o desc ibing he isual in o ma ion e bally. In his case, he isual ST o e s an image o a sca ec ow made o wo long s icks in he shape o a c oss. The sca ec ow has a la ge onion o a head, and is wea ing a small, blue jacke as well as black shoes whe e i s hands supposedly a e. The sca ec ow is placed in he middle o a ga den. The e a e six b own bi ds lying a ound he sca ec ow and one o hem is si ing on i s a m. 2 The ques ions posed a his s age o he analysis a e based on a se o ques ions ha Lemke sugges s should be asked abou he combina ion o he isual and e bal modes in his a icle Doing Mul imedia Analysis o Visual and Ve bal Da a: A Guide (2011). Anne Ke ola 54 Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 Table 2. An example o a mul imodal ST analysis Wha is p esen ed edundan ly be ween isual and ex ual elemen s? Wha is p esen ed only in he isual mode? Wha is p esen ed only in he e bal mode? Wha is p esen ed di e en ly in he wo modes? Which elemen s change hei meaning in he combina ion o he wo modes? Small jacke and shoes ha e been hung o a sca ec ow. The e a e six b own bi ds, one o which is si ing on op o he sca ec ow. The sca ec ow has been placed in a ga den. The jacke and shoes ha e been hung by M . McG ego . The colo o he bi ds: b own acco ding o isual, black acco ding o e bal ST. Sca iness o he sca ec ow: no sca y o he bi ds acco ding o isual, sca y acco ding o e bal ST. – The isual ST epea s some o he in o ma ion o e ed by he e bal ST, bu i also o e s some in o ma ion ha is no a ailable e bally. The e a e wo de ails ha a e p esen ed di e en ly be ween he wo modes. The e bal ST s a es ha he sca ec ow is mean o igh en he bi ds in he ga den, ye , acco ding o he isual ST, he bi ds do no ind he sca ec ow sca y a all. The colo o he bi ds in ques ion is also p esen ed di e en ly in he wo modes: hey a e black acco ding o e bal in o ma ion, bu p esen ed as b own in he image. I may now compa e he obse a ions made in he wo s ages o analysis. I do his by aking he able o he con as i e analysis and adding o ha i e new columns which co espond o he ques ions posed in he mul imodal ST analysis (Was he elemen p esen ed edundan ly be ween isual and ex ual elemen s? Was i p esen ed only in he isual mode o he e bal mode? Was i p esen ed di e en ly in he wo modes? Was i s meaning changed in he combina ion o he wo modes?) In he i e new columns, I ick wi h an X he op ion ha is alid o each elemen , lea ing ou connec o s and o he wo ds ha a e impossible o be exp essed isually and a e he e o e o no in e es in a mul imodal analysis. The able may hen be used o de e mine wha kind o in o ma ion he isual ST o e ed abou he i ems analyzed in s age 1. To illus a e his, I p esen he combina ion o he ables used in he examples abo e: Anne Ke ola Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 55 Table 3. An example o combining he wo s ages o analysis. These ables may now be used o de ec , o ins ance, whe he i ems ha do no ha e an equi alen in he e bal ST (op ion E in he hi d column) a e p esen ed in he isual mode and he e o e ha e an equi alen in he isual ST, as is he case wi h he wo d puu a haansa (“in his ga den”). The ables may also be used o de e mine wha happened in he ansla ion o he elemen s ha a e p esen ed di e en ly in he di e en modes. Do hey co espond o he e bal ST elemen s wo d- o -wo d (op ion A), o did he con lic ing isual in o ma ion somehow a ec he ansla o ’s in e p e a ion? In Table 3, an example o a con adic o ily p esen ed elemen is he wo d “ o igh en”, which has been le ou o he ansla ion al oge he (op ion F). The wo d “blackbi d” is also p esen ed di e en ly in he wo modes. This elemen , howe e , has been ansla ed wi h an exp ession mo e gene al in meaning (op ion C). Resea ch esul s The main objec i e o his s udy has been o elici whe he he images o a ansla ed PB may ha e a ec ed i s ansla o ’s wo d choices. As migh be expec ed, he analysis indica ed ha mos TT i ems we e di ec “wo d- o -wo d” ansla ions o hei e bal ST equi alen s. Whe he isually o e ed in o ma ion a ec ed hese choices (o no ) is beyond he scope o his kind o a esea ch me hod and may only be specula ed on in his con ex . The e o e, my main esea ch in e es lies on he o he - han-no mal ansla ion choices, in o he wo ds, ca ego ies B–F p esen ed in he p e ious chap e . The analysis e ealed ha hese a ypical TT i em Ve bal ST equi alen How i ems co espon d P esen ed edundan ly be ween isual and ex ual el. P esen ed only in he isual mode P esen ed only in he e bal mode P esen ed di e en ly in he wo modes Elemen changed i s meaning in he comb. He a Jy y [M . Jy y] M . McG ego A X ipus i [hung up] hung up A X Pe e in [Pe e ’s] E - - - - - pikku [li le] li le A X akin [jacke ] he - - jacke A X ja [and] and A - - - - - kengä [ he shoes] he shoes A X puu a haansa [in his ga den] E X linnunpelä imeksi [ o a sca ec ow] o a sca ec ow A X o igh en F X as aille [( o he) h ushes] he blackbi ds C X Anne Ke ola 56 Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 ansla ion choices could, o g ea ex en , be explained by he in o ma ion o e ed by he isual ST o he in e play o isual and e bal in o ma ion in he mul imodal ST. I will p esen he indings by going h ough he six ca ego ies o possible ela ionships be ween a ge and e bal ST i ems p esen ed in he p e ious chap e and compa ing hem o how he isual ST ela ed o hese cases. As men ioned abo e, ca ego y A, whe e he TT i em co esponds o i s e bal ST equi alen wo d- o -wo d, is o li le in e es in his s age o he analysis, since i ep esen s wha is “supposed o happen” in a ansla ion p ocess, and o e s no insigh in o he cen al esea ch ques ions. Ca ego ies C, D, and F, ha is, TT i ems ha we e mo e gene al han hei e bal ST equi alen s, TT i ems ha did no co espond o hei e bal ST equi alen s and e bal ST i ems ha had no equi alen s in he TT, we e o en linked o a con adic ion be ween he in o ma ion p esen ed in he isual and e bal modes. I may hen be sugges ed ha in cases whe e he wo ds and images o he mul imodal ST p esen incohe en in o ma ion, he ansla o s migh eel he need o modi y he e bal in o ma ion o a oid con adic ion in he ansla ion. As he da a o he s udy demons a es, one s a egy ha may be used o a oid con adic ion in some cases is mo ing om hyponyms o hype nyms, ansla ing he i em wi h an equi alen so gene al in meaning ha con adic ion be ween image and e bal ex no longe exis s. The ansla ion o he wo d “blackbi d” in Table 3 is an example o such a s a egy. The bi ds a e p esen ed as b own in he isual ST bu desc ibed as black in he e bal ST. By ansla ing “blackbi d” wi h he name o he supe o dina e species, he ansla o a oids using he name o he colo all oge he , and con adic ion be ween wo d and image is a oided in he ansla ion. Ano he a ailable s a egy o a oid con adic ion is o ollow he in o ma ion p esen ed in he isual ST and igno e in o ma ion p esen ed e bally. This ob iously esul s in he ac ha he gi en TT i em no longe co esponds o i s e bal ST equi alen . The hi d s a egy used in he da a was simply lea ing ou he i em p esen ed di e en ly by he wo modes o he mul imodal ST ( he e bal ST i em is now ob iously le wi hou an equi alen in he TT). Examples o ca ego y E, whe e he TT i ems did no ha e an equi alen in he e bal ST, u ned ou o be closely associa ed wi h isually p esen ed in o ma ion, since mos o such i ems p o ed o ha e an equi alen in he isual ST. Wha his means is ha he ansla o s, ei he consciously o unconsciously, e balized in o ma ion ha was p esen ed only isually in he mul imodal ST. An example o such an addi ion can be seen in Table 3, whe e he Finnish wo d puu a haansa (“in his ga den”) ha does no ha e an equi alen in he e bal ST, is p esen ed in he isual mode, in o he wo ds, he added i em has an equi alen in he isual ST. Images o e a signi ican amoun o addi ional, (seemingly) supe luous in o ma ion abou he cha ac e s, se ings and ac ions in he book, and some o his in o ma ion had been explica ed in o e bal in o ma ion in all o he h ee ansla ions. These addi ions a e ob iously no necessa y o he a ge language eade o unde s and he ex . Mo eo e , one could e en ask whe he changing he ex in such a way in he ansla ion p ocess is accep able. Ca ego y B, whe e he TT i ems a e mo e p ecise in meaning han hei e bal ST equi alen s, is p obably he mos in e es ing o all om he poin o iew o mul imodal esea ch. Nea ly all hese ansla ion choices we e linked o elemen s whose meaning was changed in he combina ion o he wo modes, in o he wo ds, ha we e p esen ed sligh ly di e en ly in he wo modes. An ob ious example o such a change a e he ansla ions o he wo d “plan ” in a scene whe e Pe e Rabbi knocks o e a couple o plan s. In wo o he illus a ion e sions, he wo d is p esen ed oge he wi h an image o dis inc i ely small plan s ha g ow in po s o clay (in he hi d illus a ion e sion, his scene has been le ou ). Anne Ke ola Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción imagen- ex o”: 49-60 57 Visually o e ed in o ma ion may be sugges ed o ha e in luenced bo h ansla o s’ in e p e a ions o he i em, since he wo d “plan ” has been ansla ed in o Finnish as aimi (“seedling”) in one ansla ion and a aimi uukku (“seedling po ”) in he o he . He e he combina ion o e bal in o ma ion (“plan ”) and isual in o ma ion (“small” / “g owing in a po ”) has clea ly de ined he meaning o he gi en i em. As well as nouns, he da a o e ed a ious examples o e bs ha had been ansla ed wi h mo e p ecise exp essions. A pa icula ly illus a i e example o such modi ica ions a e he ansla ions o he e b “ o ea ” in a scene whe e Pe e Rabbi ea s adishes. The ac ion o ea ing is p esen ed in di e en ways in he h ee illus a ions. In Po e ’s o iginal illus a ion, he abbi is ea ing wo adishes a a ime, wi h i s head il ed back and eyes hal -closed wi h pleasu e. Szeke es has depic ed he scene e y di e en ly: he abbi si s holding a adish om which only one bi e has been aken. I s a es in o dis ance and i s mou h is igh ly closed. In McCue’s illus a ed e sion he ac ion o ea ing is depic ed as he mos anima ed o all. The abbi is leaning agains a cabbage wi h i s ace beaming wi h deligh . I is holding wo adishes, one o which i is jus abou o ake a la ge bi e o . The di e en a mosphe es o he illus a ions ha e also been con eyed o he ansla ions: in he ansla ion o Szeke es’ illus a ed e sion, whe e he ac ion o ea ing pe se was le o li le a en ion, he e b “ o ea ” is ansla ed wi h he exp ession mais ella hieman (“ o as e a li le”). In he ansla ions o McCue’s and Po e ’s illus a ed e sions, he e b has been ansla ed as ahmia (“ o gobble”) and as he ku ella (“ o elish”) espec i ely. One could claim ha he combina ion o e bal in o ma ion (“ o ea ”) combined wi h isual in o ma ion (“ ake pleasu e in ea ing”) esul s in a ansla ion mo e p ecise in meaning (“ o elish”). I p opose ha in hese cases, he in o ma ion coming om he isual and he e bal ST combine o de ine and delimi he meaning o a gi en elemen . The isual and ex ual elemen s become seman ically associa ed, and hei concu ence delimi s he meaning o he whole. In Lemke’s wo ds (2002), he combina ion o isual and e bal in o ma ion esul s in some hing genuinely new; a meaning mo e p ecise han wha ei he o he modes exp ess alone. Conclusions The eade s –including he ansla o s– o PB ead bo h wo ds and isual images. Thei unde s anding o he book’s o e all meaning is c ea ed in he in e ac ion o hese wo modes. As e bal and isual modes a e ne e able o con ey he same and only he same in o ma ion, he combina ion o he wo esul s in some hing genuinely new in he eade ’s in e p e a ion (Lemke 2002: 303). The obse a ion is pa icula ly in e es ing when modeling he p ocess o PB ansla ion. I a gi en wo d is a o ded genuinely new meanings when p esen ed oge he wi h an image, his migh also a ec he way in which he wo d is ansla ed. The analysis e ealed ha in cases whe e he images o he PB con ey in o ma ion ha is, in one way o ano he , con adic o y o he in o ma ion con eyed e bally, he ansla o s seemed o ha e el he need o modi y he e bal in o ma ion o a oid con adic ion in hei ansla ion. The s a egies used o accomplish his in he da a o his s udy we e ansla ing he gi en wo d wi h an equi alen mo e gene al in meaning, igno ing he e bal in o ma ion and con eying wha is exp essed isually, o omi ing he wo d al oge he . Ano he way in which isually o e ed in o ma ion seemed o a ec he ansla o s’ wo d choices was he explica ion o isually o e ed in o ma ion in o e bal. All h ee ansla o s had, ei he consciously o unconsciously, ex ac ed de ails om he illus a ions and e balized his in o ma ion in hei ansla ion. The hi d way in which isually o e ed in o ma ion seemed o ha e a ec ed he ansla o s’ wo d choices had o do