Ke ola, A. (2017). “Lib os ilus ados como ex os de pa ida mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción
imagen- ex o”. Es udios de Teo ía Li e a ia. Re is a digi al: a es, le as y humanidades, 6 (12), 49-60.
Es a ob a se encuen a bajo una Licencia C ea i e Commons
A ibución-NoCome cial-Compa i Igual 4.0 In e nacional.
49
Lib os ilus ados como ex os de pa ida mul imodales
El análisis con as i o como he amien a pa a mos a
la in e acción imagen- ex o
Pic u e Books as Mul imodal Sou ce Tex s
Con as i e Analysis as a Tool o Depic ing Image- ex In e ac ion
Anne Ke ola
1
Recibido: 17/08/2017
Acep ado: 30/08/2017
Publicado: 08/09/2017
Resumen
En es e a ículo nos p oponemos: i) mos a el
ápido c ecimien o de un nue o ámbi o de
es udios den o del amplio ango cubie o po
los Es udios de T aducción, conocido como
Es udios Mul imodales; ii) desc ibi la
in e acción imagen- ex o de las aducciones
inlandesas de The Tale o Pe e Rabbi de
Bea ix Po e , desde un pun o de is a
mul imodal; iii) indaga en el modo en que las
imágenes en las aducciones de es e lib o
ilus ado pueden habe a ec ado las opciones
léxicas ealizadas po su aduc o ; i ) analiza
la elación en e las unidades léxicas en los
ex os de o igen y de pa ida, y luego compa a
es as obse aciones con la in o mación o ecida
po la imagen que co esponde a cada unidad;
) inalmen e, de e mina si es posible a i ma
que las imágenes mo i an los modos en que las
unidades léxicas se han is o aducidas.
Palab as cla e
T aducción; es udios mul imodales; lib os
ilus ados; imagen- ex o.
Abs ac
The pu poses o his a icle a e: i) o show he
apid inc ease o in e es in a new ield o
s udies wi hin he la ge scope o T ansla ion
S udies, known as Mul imodal S udies; ii) o
desc ibe he image- ex in e ac ion o he
Finnish ansla ions o Bea ix Po e ’s The Tale
o Pe e Rabbi , om a mul imodal poin o
iew; iii) o look in o how he images in he
ansla ions o his pic u e book may ha e
a ec ed i s ansla o ’s wo d choices; i ) o
analyze he ela ionship be ween lexical i ems
in he sou ce and a ge ex s, and hen
compa e hese obse a ions agains he
in o ma ion o e ed by he image wi h which
he i em is p esen ed; ) inally, o de e mine
whe he an explana ion can be de i ed om
he images as o why a ce ain i em was
ansla ed he way i was.
Keywo ds
T ansla ion; mul imodal s udies; pic u e books;
image- ex .
1
PhD S uden . Uni e si y o Tampe e, School o Language, T ansla ion and Li e a y S udies. Finland. Con ac :
anne.ke ol[email p o ec ed]
Anne Ke ola
50
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la
in e acción imagen- ex o”: 49-60
In oduc ion
he s udy o mul imodali y app oaches communica ion, ep esen a ion and in e ac ion
as always consis ing o di e en modes, such as spoken o w i en language, s ill o
mo ing images, ges u es, sound, o gaze. Du ing he las decade o so, mul imodali y
has been he subjec o apidly inc easing esea ch in e es in a a ie y o ields o s udy,
including T ansla ion S udies [TS] (see e.g. Kaindl 2004, Taylo 2003, 2004, and Hi onen y
Tii ula 2010). Pic u e books [PB] a e p onouncedly mul imodal; hei message is c ea ed in
he in e ac ion o wo ds and images. Howe e , he mul imodali y o PB has only e y ecen ly
begun o be add essed in esea ch (Moya 2010, 2013). Ex ensi e PB esea ch ca ied ou o e
he las decades, ecen de elopmen s in he s udy o mul imodali y and TS a e ields ha
need o be linked h ough con e ging ques s.
A mul imodal sou ce ex [ST] consis ing o w i en wo d and image, such as a PB,
can be hough o consis ing o wo sepa a e pa s, he e bal ST and he isual ST, which
none heless ope a e seamlessly o c ea e he ex ’s o e all meaning. The ansla o s o such
ex s in e p e in o ma ion o e ed by wo sepa a e modes. Fu he mo e, he in o ma ion
con eyed by hese wo modes is always di e en in a leas some aspec s. The wo ds o a PB
can e o lessly exp ess wha hings and cha ac e s a e called, wha he cha ac e s hink and
say, when e en s ake place, and so on (G aham 2004: 211). Unlike images, wo ds may also
con ey causal and empo al in o ma ion (Nikolaje a 2002: 97). Images, o hei pa , can be
used o exp ess wha he cha ac e s and loca ions look like exac ly. They may also qui e
accu a ely con ey he cha ac e s’ emo ions and he ela ionships be ween hem by showing
hei acial exp essions, body language and he dis ances be ween hem (Nikolaje a 2002:
93). In ac , wo ds and images may con ey he same in o ma ion only e y supe icially
(Nodelman 1988: 193).
When p esen ed oge he , wo ds and images may a o d new kinds o meaning. As PB
heo is Pe y Nodelman (1988: 220) s a es, he in o ma ion o e ed by he wo ds o a PB
change he way he eade cons ues he meaning o he images, and he images change he
way he eade in e p e s he wo ds. This gi es us a eason o suspec ha isually p esen ed
in o ma ion migh , in some cases, al e he way in which he wo ds a e ansla ed in
mul imodal ex s combining wo ds and images. I he image, in one way o ano he , changes
he meaning o a ce ain wo d wi h which i is p esen ed, he ansla ion o he wo d migh no
longe be wha could be conside ed as i s mos ob ious “wo d- o -wo d” ansla ion.
The pu pose o his a icle is o disco e whe he he images o a ansla ed PB may
ha e a ec ed i s ansla o ’s wo d choices. I analyze he ela ionship be ween lexical i ems in
he sou ce and a ge ex s [TT] o iden i y any al e a ions and di e ences, and hen compa e
hese obse a ions agains he in o ma ion o e ed by he image wi h which he i em is
p esen ed. In o he wo ds, I aim o de e mine whe he he images p o ide an explana ion as o
why a ce ain i em was ansla ed he way i was.
The da a o he s udy is comp ised o he Finnish ansla ions o The Tale o Pe e
Rabbi by Bea ix Po e . The Tale o Pe e Rabbi is an excep ional PB in he sense ha i has
been e-illus a ed dozens o imes. The publishe o he book ailed o egis e he book o
copy igh s in he Uni ed S a es when hey i s s a ed ma ke ing he book in he coun y in
ea ly 20 h cen u y. The e o e, anyone could compile a new se o illus a ions and ma ke he
book wi hou es ic ions (Mackey 1998: xix). Ou o he a ious illus a ed e sions, h ee
ha e been ansla ed in o Finnish. The o iginal book illus a ed by Po e he sel was
ansla ed in 1963 by Rii a Bjö klund. In 2003, Ri a Toi ola ansla ed a e sion illus a ed
by Cyndy Szeke es and, in 2005, Ri a B ande ansla ed a sligh ly ab idged e sion o he
s o y illus a ed by Lisa McCue. Reading he same e bal s o y coupled wi h h ee di e en
T
Anne Ke ola
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción
imagen- ex o”: 49-60
51
illus a ions p o ides qui e an excep ional oppo uni y o examine whe he ce ain wo ds a e
a o ded di e en kinds o meanings when p esen ed oge he wi h di e en images.
Theo e ical backg ound
Gun he K ess and Theo an Leeuwen (1998: 186), wo o he mos no able schola s in he
ield o mul imodal esea ch, s a e ha all ex s a e mul imodal. A pe son who alks c ea es
hei message using no only wo ds bu also in ona ion, hy hm, acial exp essions, ges u es
and pos u es. I one would ecei e he same e bal message in a w i en o m, he ma e ial on
which he message is w i en, he on o he handw i ing as well as he layou used would all
a ec he way in which he message is in e p e ed. Each mode used in he c ea ion o a
message impa s some hing abou he in en ions and he pe sonali y o i s sende (Lemke
2002: 302). I is he e o e impossible o c ea e a message using only one indi idual mode. As
K ess and an Leeu en (1998: 186) unde line, no ex analysis ha ails o accoun o he
ex ’s mul imodal aspec s can explain all laye s o meaning p esen in he ex .
One o he cen al claims o mul imodal esea ch is ha all modes a e o equal
impo ance (Jewi 2009b: 13). Wi hin mul imodal esea ch, language –al hough ha ing been
o emendous esea ch in e es in he pas – is me ely one meaning making esou ce amongs
many. This means ha wi hin a mul imodal ex combining wo ds and images, as i is he case
in PB, wo ds and images play equally impo an oles in c ea ing he o e all message o he
book. I may hence be claimed ha when ansla ing a PB, he in o ma ion o e ed by he
e bal and he isual modes a e o equal impo ance.
Ano he impo an p inciple o mul imodal esea ch is ha each mode wi hin a
mul imodal ex pa icipa es in c ea ing i s o e all meaning by ealizing hei own
communica i e ask, exp essing wha each mode exp esses bes (Jewi 2009c: 15).
Analogically, his means ha when s udied sepa a ely, each mode o a mul imodal ensemble
is incomple e and canno ep esen all meanings p esen in he mul imodal whole. Resea ch
add essing he ansla ion o mul imodal ex s should, o his e y eason, always co e he
analysis o each o he modes in ol ed.
As s a ed abo e, PB combine wo di e en modes, he isual and he e bal, images
and w i en language, bu how does his a ec he eade s o PB? Jay Lemke (2002: 303)
discusses wha he calls “ he essen ial incommensu abili y” be ween di e en modes: A e bal
ex is ne e able o p oduce he same and only he same meanings as an image, and ice
e sa. In Lemke’s wo ds, his incommensu abili y ine i ably leads o genuinely new meanings
being c ea ed in he combina ion o modes (2002). The meanings c ea ed in he combina ion
o modes a e he e o e always g ea e han he sum o he meanings c ea ed by he same
modes indi idually (Flewi , Hampel, Hauck & Lancas e 2009: 46).
In acco dance wi h he abo e obse a ion, he combina ion o wo ds and images in a
PB always c ea es genuinely new meanings. When we ead PB, i is possible o us o ind
meanings ha a e p esen only when he wo modes a e p esen ed oge he ; meanings ha a e
no p esen in ei he mode alone. I migh hen be possible ha he genuinely new meanings
c ea ed wi hin he mul imodal ST a e con eyed o he ansla o ’s in e p e a ion o he ex
and, mo eo e , occasionally o he ansla ion choices he ansla o makes.
Resea ch me hod
The aim o his s udy is o disco e whe he he images o a ansla ed PB may ha e a ec ed
i s ansla o ’s wo d choices. Wi h a iew o de e mining his, I i s con as he wo ds o
exp essions ha can be hough o occupying iden ical posi ions in he ST and TT and analyze
Anne Ke ola
52
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la
in e acción imagen- ex o”: 49-60
how hey co espond o each o he . I hen compa e hese obse a ions o he in o ma ion he
isual ST exp esses abou a gi en i em o ind ou whe he he images can be used o explain
why he i ems we e ansla ed as hey we e. In he i s s age o my analysis I di ide he TT
ph ases in o lexical i ems (wo ds, ph asal e bs, colloca ions, e c.) and compa e hem wi h
hei coun e pa s in he e bal ST (in his a icle, I e e o hese wo i ems as “equi alen s”
e en in cases whe e no imaginable o m o equi alence exis s be ween hem). I hen aim o
iden i y simila i ies and di e ences be ween he meanings he con as ed i ems ca y. The
esea ch could hence be conside ed as a o m o c oss-linguis ic con as i e analysis wi h a
ocus on analyzing ansla ional equi alence.
One may easily hink o di e en ela ionships exis ing be ween a ST and a TT i em.
The pe haps mos common –and, in mos cases, desi able– si ua ion would be a wo d- o -
wo d ansla ion. By his somewha assailable e m I mean a ela ionship whe e he wo i ems
co espond o each o he as closely as i is possible be ween wo languages o , in Eugene
Nida’s (1969, 495) e ms, whe e he ansla ion is “ he closes na u al equi alen o he
message o he sou ce language” (a simple example o his could be he English wo d “ca ”
ansla ed in o Spanish as “ga o”). Ano he possible ela ionship be ween he wo i ems is a
si ua ion whe e he TT i em is mo e p ecise in meaning han he e bal ST i em, in o he
wo ds, whe e he ansla o mo es om hype nyms o hyponyms, om supe o dina e o
subo dina e e ms ( o ins ance, he English wo d “animal” ansla ed in o Spanish as “ga o”).
In he opposi e case, he ansla o mo es om hyponyms o hype nyms, he TT i em being
mo e gene al in meaning han he e bal ST i em ( he English wo d “ca ” being ansla ed in o
Spanish as “animal”). I migh also be possible ha he lexical i em is changed in ansla ion
in a way ha co espondence be ween he a ge and e bal ST i ems can ha dly be said o
exis ( o example, he English wo d “ca ” ansla ed in o Spanish as “pe o”). I may also be
ha a lexical i em in he TT has no equi alen in he e bal ST, in o he wo ds, some hing has
been added in he ansla ion p ocess. The opposi e case would be a lexical i em being le ou
in he ansla ion p ocess al oge he , and he e bal ST i em would hen ha e no equi alen in
he TT.
We may now compile he ollowing six ca ego ies o desc ibe he ela ionship be ween
each elemen o he TT and i s equi alen in he e bal ST:
A. The TT i em co esponds o i s e bal ST equi alen wo d- o -wo d.
B. The TT i em is mo e p ecise in meaning han i s e bal ST equi alen .
C. The TT i em is mo e gene al in meaning han i s e bal ST equi alen .
D. The TT i em does no co espond o i s e bal ST equi alen .
E. The TT i em has no equi alen in he e bal ST.
F. The e bal ST i em has no equi alen in he TT.
I p esen his s age o he analysis in a able whe e I di ide he ansla ed sen ence in o
lexical i ems placed on he ows o he i s column (wi h English back- ansla ions in squa e
b acke s). The e bal ST equi alen s o hese elemen s a e placed on he ows o he second
column. The hi d column desc ibes how he Finnish and English exp essions co espond o
each o he (op ions A–F). To illus a e he me hod, I p esen an analysis o a sen ence om
my da a, aken om he ansla ion o Cyndy Szeke es’ illus a ion e sion.
The e bal ST goes: “M . McG ego hung up he li le jacke and he shoes o a
sca ec ow o igh en he blackbi ds” (Po e 2002) and i s Finnish ansla ion goes “He a
Jy y ipus i Pe e in pikku akin ja kengä puu a haansa linnunpelä imeksi as aille” [M . Jy y
hung up Pe e ’s li le jacke and shoes in his ga den o a sca ec ow o he h ushes] (Po e
2003).
Anne Ke ola
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción
imagen- ex o”: 49-60
53
Table 1. An example o a con as i e analysis be ween a ge and e bal ST i ems.
TT i em
Ve bal ST
equi alen
How i ems
co espond
He a Jy y [M . Jy y]
M . McG ego
A
ipus i [hung up]
hung up
A
Pe e in [Pe e ’s]
E
pikku [li le]
Li le
A
akin [jacke ]
he - - jacke
A
ja [and]
And
A
kengä [ he shoes]
he shoes
A
puu a haansa [in his ga den]
E
linnunpelä imeksi [ o a sca ec ow]
o a sca ec ow
A
o igh en
F
as aille [( o he) h ushes]
he blackbi ds
C
As i can be seen om he able, mos TT i ems co espond o hei e bal ST
equi alen s wo d- o -wo d (op ion A). Howe e , wo o he elemen s (Pe e in, meaning
“Pe e ’s” and puu a haansa, “in his ga den”) do no ha e equi alen s in he e bal ST (op ion
E), and he e b “ o igh en” does no ha e an equi alen in he TT (op ion F). Also, he wo d
“blackbi d” (mus a as as, “ he black h ush” in Finnish) has been ansla ed wi h an
exp ession wi h a mo e gene al meaning (op ion C), as as, meaning “ h ush” ( he blackbi d
is a subspecies o h ush).
A e analyzing how he ST and TT i ems co espond o each o he , I hen compa ed
hese esul s agains he in o ma ion he isual ST p o ided he ansla o s. The second s age
o my analysis is wha I call a mul imodal ST analysis, whe e I con as he isual and e bal
ST, in o he wo ds, I go h ough he da a sen ence by sen ence and compa e each one o he
image wi h which i is p esen ed. The ques ions
2
I pose in he mul imodal ST analysis a e he
ollowing: Wha is p esen ed edundan ly be ween isual and ex ual elemen s? Wha is
p esen ed only in he isual mode? Wha is p esen ed only in he e bal mode? Wha is
p esen ed di e en ly in he wo modes? Which elemen s change hei meaning in he
combina ion o he wo modes? The las ques ion o he lis e e s o cases whe e he isual
and e bal ST con ey sligh ly di e en bu closely ela ed aspec s o meaning abou a gi en
i em. The eade hen ge s addi ional in o ma ion abou he i em, o example in cases whe e
he e bal ST exp esses wha is done and he isual ST exp esses exac ly how i is done.
To illus a e he me hod, I p esen he analysis o he sen ence used in he example
abo e. Since I was denied he pe mission o p in he images o my da a, I am limi ed o
desc ibing he isual in o ma ion e bally. In his case, he isual ST o e s an image o a
sca ec ow made o wo long s icks in he shape o a c oss. The sca ec ow has a la ge onion o
a head, and is wea ing a small, blue jacke as well as black shoes whe e i s hands supposedly
a e. The sca ec ow is placed in he middle o a ga den. The e a e six b own bi ds lying
a ound he sca ec ow and one o hem is si ing on i s a m.
2
The ques ions posed a his s age o he analysis a e based on a se o ques ions ha Lemke sugges s should be
asked abou he combina ion o he isual and e bal modes in his a icle Doing Mul imedia Analysis o Visual
and Ve bal Da a: A Guide (2011).

Anne Ke ola
54
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la
in e acción imagen- ex o”: 49-60
Table 2. An example o a mul imodal ST analysis
Wha is
p esen ed
edundan ly
be ween isual
and ex ual
elemen s?
Wha is p esen ed
only in he isual
mode?
Wha is
p esen ed only in
he e bal mode?
Wha is p esen ed
di e en ly in he
wo modes?
Which elemen s
change hei
meaning in he
combina ion o he
wo modes?
Small jacke and
shoes ha e been
hung o a
sca ec ow.
The e a e six b own
bi ds, one o which
is si ing on op o
he sca ec ow. The
sca ec ow has been
placed in a ga den.
The jacke and
shoes ha e been
hung by M .
McG ego .
The colo o he
bi ds: b own
acco ding o isual,
black acco ding o
e bal ST.
Sca iness o he
sca ec ow: no sca y
o he bi ds
acco ding o isual,
sca y acco ding o
e bal ST.
–
The isual ST epea s some o he in o ma ion o e ed by he e bal ST, bu i also
o e s some in o ma ion ha is no a ailable e bally. The e a e wo de ails ha a e p esen ed
di e en ly be ween he wo modes. The e bal ST s a es ha he sca ec ow is mean o
igh en he bi ds in he ga den, ye , acco ding o he isual ST, he bi ds do no ind he
sca ec ow sca y a all. The colo o he bi ds in ques ion is also p esen ed di e en ly in he
wo modes: hey a e black acco ding o e bal in o ma ion, bu p esen ed as b own in he
image.
I may now compa e he obse a ions made in he wo s ages o analysis. I do his by
aking he able o he con as i e analysis and adding o ha i e new columns which
co espond o he ques ions posed in he mul imodal ST analysis (Was he elemen p esen ed
edundan ly be ween isual and ex ual elemen s? Was i p esen ed only in he isual mode o
he e bal mode? Was i p esen ed di e en ly in he wo modes? Was i s meaning changed in
he combina ion o he wo modes?) In he i e new columns, I ick wi h an X he op ion ha
is alid o each elemen , lea ing ou connec o s and o he wo ds ha a e impossible o be
exp essed isually and a e he e o e o no in e es in a mul imodal analysis. The able may
hen be used o de e mine wha kind o in o ma ion he isual ST o e ed abou he i ems
analyzed in s age 1. To illus a e his, I p esen he combina ion o he ables used in he
examples abo e:
Anne Ke ola
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción
imagen- ex o”: 49-60
55
Table 3. An example o combining he wo s ages o analysis.
These ables may now be used o de ec , o ins ance, whe he i ems ha do no ha e
an equi alen in he e bal ST (op ion E in he hi d column) a e p esen ed in he isual mode
and he e o e ha e an equi alen in he isual ST, as is he case wi h he wo d puu a haansa
(“in his ga den”). The ables may also be used o de e mine wha happened in he ansla ion
o he elemen s ha a e p esen ed di e en ly in he di e en modes. Do hey co espond o he
e bal ST elemen s wo d- o -wo d (op ion A), o did he con lic ing isual in o ma ion
somehow a ec he ansla o ’s in e p e a ion? In Table 3, an example o a con adic o ily
p esen ed elemen is he wo d “ o igh en”, which has been le ou o he ansla ion
al oge he (op ion F). The wo d “blackbi d” is also p esen ed di e en ly in he wo modes.
This elemen , howe e , has been ansla ed wi h an exp ession mo e gene al in meaning
(op ion C).
Resea ch esul s
The main objec i e o his s udy has been o elici whe he he images o a ansla ed PB may
ha e a ec ed i s ansla o ’s wo d choices. As migh be expec ed, he analysis indica ed ha
mos TT i ems we e di ec “wo d- o -wo d” ansla ions o hei e bal ST equi alen s.
Whe he isually o e ed in o ma ion a ec ed hese choices (o no ) is beyond he scope o
his kind o a esea ch me hod and may only be specula ed on in his con ex . The e o e, my
main esea ch in e es lies on he o he - han-no mal ansla ion choices, in o he wo ds,
ca ego ies B–F p esen ed in he p e ious chap e . The analysis e ealed ha hese a ypical
TT i em
Ve bal ST
equi alen
How i ems
co espon
d
P esen ed
edundan ly
be ween
isual and
ex ual el.
P esen ed
only in he
isual
mode
P esen ed
only in
he e bal
mode
P esen ed
di e en ly
in he wo
modes
Elemen
changed
i s
meaning
in he
comb.
He a Jy y
[M . Jy y]
M .
McG ego
A
X
ipus i
[hung up]
hung up
A
X
Pe e in
[Pe e ’s]
E
-
-
-
-
-
pikku
[li le]
li le
A
X
akin
[jacke ]
he - -
jacke
A
X
ja
[and]
and
A
-
-
-
-
-
kengä
[ he shoes]
he shoes
A
X
puu a haansa
[in his ga den]
E
X
linnunpelä imeksi
[ o a sca ec ow]
o a
sca ec ow
A
X
o igh en
F
X
as aille
[( o he)
h ushes]
he
blackbi ds
C
X
Anne Ke ola
56
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la
in e acción imagen- ex o”: 49-60
ansla ion choices could, o g ea ex en , be explained by he in o ma ion o e ed by he
isual ST o he in e play o isual and e bal in o ma ion in he mul imodal ST.
I will p esen he indings by going h ough he six ca ego ies o possible ela ionships
be ween a ge and e bal ST i ems p esen ed in he p e ious chap e and compa ing hem o
how he isual ST ela ed o hese cases. As men ioned abo e, ca ego y A, whe e he TT i em
co esponds o i s e bal ST equi alen wo d- o -wo d, is o li le in e es in his s age o he
analysis, since i ep esen s wha is “supposed o happen” in a ansla ion p ocess, and o e s
no insigh in o he cen al esea ch ques ions.
Ca ego ies C, D, and F, ha is, TT i ems ha we e mo e gene al han hei e bal ST
equi alen s, TT i ems ha did no co espond o hei e bal ST equi alen s and e bal ST
i ems ha had no equi alen s in he TT, we e o en linked o a con adic ion be ween he
in o ma ion p esen ed in he isual and e bal modes. I may hen be sugges ed ha in cases
whe e he wo ds and images o he mul imodal ST p esen incohe en in o ma ion, he
ansla o s migh eel he need o modi y he e bal in o ma ion o a oid con adic ion in he
ansla ion.
As he da a o he s udy demons a es, one s a egy ha may be used o a oid
con adic ion in some cases is mo ing om hyponyms o hype nyms, ansla ing he i em
wi h an equi alen so gene al in meaning ha con adic ion be ween image and e bal ex no
longe exis s. The ansla ion o he wo d “blackbi d” in Table 3 is an example o such a
s a egy. The bi ds a e p esen ed as b own in he isual ST bu desc ibed as black in he
e bal ST. By ansla ing “blackbi d” wi h he name o he supe o dina e species, he
ansla o a oids using he name o he colo all oge he , and con adic ion be ween wo d and
image is a oided in he ansla ion. Ano he a ailable s a egy o a oid con adic ion is o
ollow he in o ma ion p esen ed in he isual ST and igno e in o ma ion p esen ed e bally.
This ob iously esul s in he ac ha he gi en TT i em no longe co esponds o i s e bal
ST equi alen . The hi d s a egy used in he da a was simply lea ing ou he i em p esen ed
di e en ly by he wo modes o he mul imodal ST ( he e bal ST i em is now ob iously le
wi hou an equi alen in he TT).
Examples o ca ego y E, whe e he TT i ems did no ha e an equi alen in he e bal
ST, u ned ou o be closely associa ed wi h isually p esen ed in o ma ion, since mos o
such i ems p o ed o ha e an equi alen in he isual ST. Wha his means is ha he
ansla o s, ei he consciously o unconsciously, e balized in o ma ion ha was p esen ed
only isually in he mul imodal ST. An example o such an addi ion can be seen in Table 3,
whe e he Finnish wo d puu a haansa (“in his ga den”) ha does no ha e an equi alen in
he e bal ST, is p esen ed in he isual mode, in o he wo ds, he added i em has an
equi alen in he isual ST. Images o e a signi ican amoun o addi ional, (seemingly)
supe luous in o ma ion abou he cha ac e s, se ings and ac ions in he book, and some o
his in o ma ion had been explica ed in o e bal in o ma ion in all o he h ee ansla ions.
These addi ions a e ob iously no necessa y o he a ge language eade o unde s and he
ex . Mo eo e , one could e en ask whe he changing he ex in such a way in he ansla ion
p ocess is accep able.
Ca ego y B, whe e he TT i ems a e mo e p ecise in meaning han hei e bal ST
equi alen s, is p obably he mos in e es ing o all om he poin o iew o mul imodal
esea ch. Nea ly all hese ansla ion choices we e linked o elemen s whose meaning was
changed in he combina ion o he wo modes, in o he wo ds, ha we e p esen ed sligh ly
di e en ly in he wo modes. An ob ious example o such a change a e he ansla ions o he
wo d “plan ” in a scene whe e Pe e Rabbi knocks o e a couple o plan s. In wo o he
illus a ion e sions, he wo d is p esen ed oge he wi h an image o dis inc i ely small plan s
ha g ow in po s o clay (in he hi d illus a ion e sion, his scene has been le ou ).
Anne Ke ola
Es udios de Teo ía Li e a ia, 6 (12), “Lib os ilus ados como ex os de pa ida
mul imodales. El análisis con as i o como he amien a pa a mos a la in e acción
imagen- ex o”: 49-60
57
Visually o e ed in o ma ion may be sugges ed o ha e in luenced bo h ansla o s’
in e p e a ions o he i em, since he wo d “plan ” has been ansla ed in o Finnish as aimi
(“seedling”) in one ansla ion and a aimi uukku (“seedling po ”) in he o he . He e he
combina ion o e bal in o ma ion (“plan ”) and isual in o ma ion (“small” / “g owing in a
po ”) has clea ly de ined he meaning o he gi en i em.
As well as nouns, he da a o e ed a ious examples o e bs ha had been ansla ed
wi h mo e p ecise exp essions. A pa icula ly illus a i e example o such modi ica ions a e
he ansla ions o he e b “ o ea ” in a scene whe e Pe e Rabbi ea s adishes. The ac ion o
ea ing is p esen ed in di e en ways in he h ee illus a ions. In Po e ’s o iginal illus a ion,
he abbi is ea ing wo adishes a a ime, wi h i s head il ed back and eyes hal -closed wi h
pleasu e. Szeke es has depic ed he scene e y di e en ly: he abbi si s holding a adish
om which only one bi e has been aken. I s a es in o dis ance and i s mou h is igh ly closed.
In McCue’s illus a ed e sion he ac ion o ea ing is depic ed as he mos anima ed o all. The
abbi is leaning agains a cabbage wi h i s ace beaming wi h deligh . I is holding wo
adishes, one o which i is jus abou o ake a la ge bi e o . The di e en a mosphe es o he
illus a ions ha e also been con eyed o he ansla ions: in he ansla ion o Szeke es’
illus a ed e sion, whe e he ac ion o ea ing pe se was le o li le a en ion, he e b “ o
ea ” is ansla ed wi h he exp ession mais ella hieman (“ o as e a li le”). In he ansla ions
o McCue’s and Po e ’s illus a ed e sions, he e b has been ansla ed as ahmia (“ o
gobble”) and as he ku ella (“ o elish”) espec i ely. One could claim ha he combina ion o
e bal in o ma ion (“ o ea ”) combined wi h isual in o ma ion (“ ake pleasu e in ea ing”)
esul s in a ansla ion mo e p ecise in meaning (“ o elish”).
I p opose ha in hese cases, he in o ma ion coming om he isual and he e bal ST
combine o de ine and delimi he meaning o a gi en elemen . The isual and ex ual
elemen s become seman ically associa ed, and hei concu ence delimi s he meaning o he
whole. In Lemke’s wo ds (2002), he combina ion o isual and e bal in o ma ion esul s in
some hing genuinely new; a meaning mo e p ecise han wha ei he o he modes exp ess
alone.
Conclusions
The eade s –including he ansla o s– o PB ead bo h wo ds and isual images. Thei
unde s anding o he book’s o e all meaning is c ea ed in he in e ac ion o hese wo modes.
As e bal and isual modes a e ne e able o con ey he same and only he same in o ma ion,
he combina ion o he wo esul s in some hing genuinely new in he eade ’s in e p e a ion
(Lemke 2002: 303). The obse a ion is pa icula ly in e es ing when modeling he p ocess o
PB ansla ion. I a gi en wo d is a o ded genuinely new meanings when p esen ed oge he
wi h an image, his migh also a ec he way in which he wo d is ansla ed.
The analysis e ealed ha in cases whe e he images o he PB con ey in o ma ion
ha is, in one way o ano he , con adic o y o he in o ma ion con eyed e bally, he
ansla o s seemed o ha e el he need o modi y he e bal in o ma ion o a oid
con adic ion in hei ansla ion. The s a egies used o accomplish his in he da a o his
s udy we e ansla ing he gi en wo d wi h an equi alen mo e gene al in meaning, igno ing
he e bal in o ma ion and con eying wha is exp essed isually, o omi ing he wo d
al oge he . Ano he way in which isually o e ed in o ma ion seemed o a ec he
ansla o s’ wo d choices was he explica ion o isually o e ed in o ma ion in o e bal. All
h ee ansla o s had, ei he consciously o unconsciously, ex ac ed de ails om he
illus a ions and e balized his in o ma ion in hei ansla ion. The hi d way in which
isually o e ed in o ma ion seemed o ha e a ec ed he ansla o s’ wo d choices had o do