Does the Timing of Visual Support Affect Sentence Comprehension? An Eye-Tracking Study

1 oes the Timing of Visual Support Affect Sentence Comprehension? An Eye-Tracking Study Miriam Rivero-Contreras1, Paul E. Engelhardt2, Pablo Delgado1, and David Saldaña1 1Universidad de Sevilla, Sevilla, Spain 2University of East Anglia, Norwich, UK Address for correspondence: [email protected] Miriam Rivero-Contreras Author Note: Miriam Rivero-Contreras is now at the Department of Psychology and Anthropology, University of Extremadura, Spain. 2 Does the Timing of Visual Support Affect Sentence Comprehension? An Eye-Tracking Study Purpose: Recent research suggests that visual elements improve sentence processing for students, even at the university level. However, few studies have systematically examined the timing of visual support in reading. Method: We examined the impact of visual support and its timing on sentence comprehension in a sample of 40 typically developing university students. Across 60 sentences, half with images and half without, participants either viewed images simultaneously with sentences or before sentences. Word frequency was also manipulated. Results: Results showed that visual support facilitated sentence processing and that participants who viewed images before sentences exhibited a lower probability of regressions. Conclusion: In conclusion, incorporating images with text can benefit language comprehension. Moreover, the results suggest implications regarding the timing of visual support. Keywords: visual support; image timing; sentence processing; eye movements Introduction Visual Support and Reading Many reading books for young children consist of a single (simple) sentence accompanied by an image described by the sentence. Pictures in these books, aimed at pre-readers and young children, play an important role in enhancing text comprehension (Strouse et al., 2018), improving vocabulary learning (Wasik et al., 2016), and fostering parent-child interaction during shared reading (Dowdall et al., 2020; Fletcher & Reese, 2005). However, although pictures do not disappear from books for older children and adults, the role of visual support for narrative text comprehension beyond early primary school becomes increasingly smaller. In fact, most pre-teen reading books do have illustrations and pictures but, in most cases, they are considered text “enrichment” rather than text “support”. However, illustrations and pictures can serve to provide knowledge to older students who are expected to read about things that are not part of 3 their daily experience (Hibbing & Rankin-Erickson, 2003). Visual support has also been shown to facilitate reading in cases of low-frequency or unfamiliar words, and when the text contains complex syntactic structures (Montag et al., 2015). In one particularly influential study, Brookshire et al. (2002) found better reading comprehension for books with text and illustrations, compared to illustrations with no text or text with no illustrations. More recently, Takacs and Bus (2018) showed that a congruent image contributed substantially to children's story retelling accuracy, suggesting that copresent visual representations lead to deeper/better encoding of linguistic information (Mayer, 2009). Visual Support for Adults It is generally believed that visual support is not necessary for understanding written information in adults (outside learning materials specifically designed in a multimedia format), because they are (in most cases) experienced readers. However, some recent research suggests that images do facilitate lexical access even in adults (Qu et al., 2016). A recent meta-analysis examined 39 experimental studies (21 with college students or older adults) published between 1985 and 2018 (Guo et al., 2020). The authors found that the inclusion of graphs had a moderate overall positive effect (Hedges' g = 0.39) on students' reading comprehension regardless of grade level (elementary, secondary, and undergraduate students and above), suggesting that visual elements improve reading comprehension for all students. Likewise, they found no significant differences between pictures, pictorial diagrams, and flow diagrams. In addition, in two prior studies from our lab, we found that visual support facilitated sentence processing in groups of adult readers with reading difficulties (dyslexia and a low level of education) and without them, even at university level (Rivero-Contreras et al., 2021, 2023). 4 Many studies carried out within the framework of the cognitive theory of multimedia learning also support the view that people learn better from words and images than from words alone (Mayer, 2009). However, cognitive limitations must be carefully considered (Mayer & Moreno, 2003). Multiple information sources, such as a text and an image, being processed in the same (visual) modality must be carefully designed to avoid cognitive overloading, split attention, and unnecessary incidental processing. But most studies under this model have focused on materials with substantial and converging overlap in content between different visual and auditory content. In our case, we explore the role of pictures as used in many narrative texts, in which the visual information rather provides a context for comprehension of written information. Although there exists a considerable amount of research focused on the enhancing effect of the presence of visual support on text comprehension, as noted in a review by Eitel and Scheiter (2015), research in this field has yielded inconclusive results. Some studies have found that presenting a picture prior to textual information enhances comprehension, whereas other found the opposite. In addition, little work has directly experimentally examined whether the timing of presentation of pictorial support plays any moderating role in the enhancing effect of picture on text comprehension. Except for some studies focused on complex textual and pictorial materials in which the textual information is needed to understand a graph or figure, or vice versa, in the context of multimedia learning, to date, there is a lack of studies that have manipulated experimentally the timing in which supporting pictures are presented (i.e., whether the picture precedes the text or both elements appear concurrently. An exception is a study by Eitel et al. (2013), in which the information was presented to participants in six conditions. One condition contained only text. Four conditions varied the length of 5 picture presentation before the text: 150 milliseconds, 600 milliseconds, 2 seconds, or participant determined (i.e., self-paced). Finally, there was a self-paced concurrent presentation of text and picture. The authors found that the self-paced before and selfpaced concurrent conditions led to better memory performance, better comprehension, and faster processing compared to the text alone condition. In addition, they found that presenting the image 600 milliseconds or 2 seconds before the text improved comprehension and resulted in faster reading. Therefore, images which are presented before the text for even a short period of time can facilitate processing of verbal information, and thus, promote better comprehension. Nevertheless, Eitel et al. (2013) used offline measures, which do not provide an accurate measure of text processing itself. In this regard, as also noted by Eitel and Scheiter (2015), there is a need for analysing processing data to explore the interaction between text and picture. Also except for the study of multimedia learning (for a review, see Alemdag & Cagiltay, 2018), most of the studies in the use of visual support to understand text have examined offline data, typically comprehension outcomes. Processing data is crucial not only to examine the existence of any effect, but also to provide an explanation for it. To fill this gap, Eitel and Scheiter called for increasing research analysing readers’ eye movements during the reading course. Two exceptions to the lack of this type of research are the studies by Ferreira et al. (2013) and, more recently, by Wassenburg et al. (2020). Ferreira et al. (2013) established that a preview of objects is critical to the observance of garden-path effects in the visual world paradigm. In their study, participants were often misled by Put the apple on the towel in the box, into initially thinking that they must move the apple onto a towel (as evidenced by frequent looks to the towel in the object array). However, this pattern of eye movements was only 6 observed when objects were previewed before the sentence was heard, but not in a concurrent (or simultaneous) viewing condition. The authors argued that the preview of visual information (consecutive presentation) allowed comprehenders to generate certain expectations or predictions regarding the upcoming linguistic information (i.e., an effect of one information source on another). In contrast, when both information sources have to be processed simultaneously, there is competition for attention, and the incremental processing that would normally occur for either visual or linguistic information is disrupted. Incremental processing for linguistic input (visual or auditory) typically occurs one word at a time and after each word the reader/listener derives an interpretation of the input to that point (Altmann & Kamide, 1999). Importantly, the study by Ferreira et al. (2013) did not explore the processing of texts but of auditory information. In the study by Wassenburg et al. (2020), the authors examined whether a picture presented before reading facilitates comprehension of a 404-word narrative text by fostering situation model construction, as compared to the same text without the picture. The authors reported that the eye-movement patterns of the participants who visualized the supporting picture reflected increased effort in processing information to generate inferences, whereas those who did not visualize the picture were more focused on processing literal information. Although there were no differences in text comprehension outcomes between both conditions (for tentative explanations see Wassenburg et al., 2020), this result indicates that visualizing a supporting image shapes subsequent reading behaviour. Building on the studies above, we examined the impact of lexical frequency (high vs. low) and visual support that was presented simultaneously with the sentence (image vs. no image) on sentence processing, specifically in individuals with low educational levels and with dyslexia (Rivero-Contreras et al., 2021, 2023). Word 7 frequency is a lexical variable that has been extensively studied using eye tracking (e.g., Ashby et al., 2005; Hyönä & Olson, 1995; Inhoff & Rayner, 1986; Rayner & Duffy, 1986; Rayner & Fischer, 1996; Rayner et al., 1996; Staub et al., 2010). In short, more frequent words (high frequency) are read/processed faster than low frequency words. Consistent with this, our prior work showed a large effect of word frequency, particularly between less skilled readers (low education and dyslexics) and typically developing controls; and increased lexical access and word integration speed, along with a reduction in the number of regressions, facilitated by visual support (Rivero- Contreras et al., 2021, 2023). Thus, supporting images seem to positively affect text processing. However, both in the study by Wassenburg et al. (2020) and in those by Rivero-Contreras et al. (2021, 2023), the presentation timing of the pictures was not manipulated, as they were presented only prior or simultaneously to the text, respectively. To fill this gap in research and provide cumulative evidence on online text processing, we conducted a study in which we experimentally compared the effect of presenting supporting images prior vs concurrent with the text on readers’ eye movements. Current Study In the present study, we aimed to build on the lines of research presented above (Eitel et al., 2013; Eitel & Scheiter; 2015; Rivero-Contreras et al., 2021, 2023); Wassenburg et al., 2020) We manipulated word frequency (high vs. low) and visual support (image vs. no image) in a sentence-reading task, which involved eye tracking. Importantly, we also manipulated a third between-subjects variable related to the timing of the images. Half of the participants viewed the image (for 1 second) prior to the sentence (consecutive condition), and half of the participants viewed the image simultaneously with the sentence (concurrent condition). The manipulation of the 8 timing of visual support was based on the idea that previewing a scene may impact differently on text comprehension processes than having it present at the same time as written content, as found by Eitel et al. (2013) in offline comprehension outcomes. We suggest that if linguistic and contextual visual information must be processed simultaneously, then incremental processing could be disrupted, leading to a more superficial and piecemeal processing strategy. It is important to understand that in our case pictures provide a context for interpretation, rather than specific converging content. If linguistic and visual information are processed consecutively, then there is much less risk of unnecessary incidental processing during reading the text, and performance might be enhanced. In short, we hypothesized that the timing of the presentation of visual support (the between-subjects variable) would lead to significantly shorter sentence reading times and a lower probability of regressions out in consecutive compared to concurrent visual support. In the current study, we recorded eye movements of typically developing university students while they read four blocks of 15 sentences (60 in total for the whole experiment). As we have indicated, the three variables (word frequency, visual support, and timing) were manipulated in a 2 × 2 × 2 design. To examine visual support, half the sentences contained an image related to the sentence, while the other half did not contain an image. To examine word frequency (lexical simplification), half the sentences contained a high-frequency target word and half a low-frequency target word. (We expected faster reading times on high-frequency words). Visual support (image vs. no image) and lexical frequency (high vs. low) were manipulated within subjects. To examine the impact of the timing of visual support, half the participants viewed the image simultaneously with sentence (concurrent presentation), while the other half were presented with the image before the sentence (consecutive presentation). The effect of 9 timing was only evaluated in the sentences including an image (i.e., 30 items in total). The possible confounding effects of participants’ age, level of receptive vocabulary, and reading comprehension skill were controlled by including them as covariates when they showed to be significant predictors of any of the dependent variables (i.e., eyemovement measures). Two final points need to be addressed. The first concerns the dependent measures. We examined the reading times and regressions for the entire sentence, which we refer to as the global text processing measures, and we also examined reading times and regressions for the manipulated (high vs. low) target word within each sentence, which we refer to as local text processing measures. The second is to further explicate the mechanism by which we expected the timing of visual support to facilitate processing. Given a 1-second preview of the visual scene, participants can determine the objects in the scene and their relations. Moreover, they very likely access the names of at least some of the objects in the scene, which is accomplished in part by the activation of semantic knowledge. There is also the possibility of constructing a partial situation model or event representation based on proposition-level knowledge (Kintsch, 1998; McRae et al., 2021). The extent of this processing will naturally vary by item and individual. However, the more representations that are activated/constructed upon viewing the picture will naturally lead to more predictions/expectations about the content of the upcoming sentence (i.e., there should be a facilitation effect of visual information on sentence processing). Consistently with previous studies, and our expectations with respect to the timing in the sentences with an image, we expected shorter reading times both in global and local measures, less regressions, and greater accuracy for sentences with high- 16 of participants and number of items) on a dataset from a previous similar study by Rivero-Contreras et al. (2021) that also manipulated the presence of picture and word frequency to examine their effects on the same eye-movement measures. Although the study was performed on a sample of young adults with typical development and young adults with dyslexia, given that no differences were found between both groups, we used the whole dataset to simulate power. Simulations were performed separately for each eye-movement measure to determine the a priori statistical power in the present study for each variable based on the effects found in Rivero-Contreras et al. (2021). Different combinations of both random effects were performed as follows. Firstly, the number of items were fixed at 60 to determine the minimum number of participants to reach a beta value of .20 when analysing the effect of Picture and Frequency on each eye-movement measure. As can be seen in Table 2, the appropriate minimum sample size in the cases of the effect of Frequency on Text regressions out and the effect of Picture on Word gaze duration exceeded 40. Therefore, we will avoid concluding on the absence of these effects. Secondly, since the factor Timing was nested within the sentence-with-picture condition, such as only 30 items were involved, we fixed the number of items at 30, to simulate power to find an effect of this factor (on each eye-movement measure) like the one found in Rivero-Contreras et al. (2021). For those eye-movement variables that were affected by Picture in Rivero-Contreras et al. (2021), these were the effects used to simulate power because this factor is related closely to Timing (i.e., the moment in which the picture is presented). In the remaining cases, we based the simulations on the effects of Frequency. Results showed that the appropriate minimum sample size exceeded 40 in the cases of the possible effects of Timing on Text regressions out, Word regressions out, Word reading time, Word gaze duration, and Word regression 17 path duration (see also Table 2). We will thus avoid arriving at conclusions, in any case, on the absence of these effects of Timing. The dataset and syntax used to perform all the a priori statistical power analyses can be found at https://osf.io/s28f5/?view_only=b5ce705485b04684bd4074cd61f7ee0a. (Insert Table 2) The LMMs to examine the effects of the experimental factors on the eyemovement measures in the present study were built as follows. Prior to entering the fixed effects in each model, we tested for possible covariation between the eyemovement measures and participants’ scores on PPVT-4, on ART, and on participants’ age. To that end, these three variables were added separately as continuous fixed factors to the null model for each eye-movement measure. When any of them proved to be a significant predictor, it was kept in subsequent models including the experimental factors (i.e., Picture and Frequency or Timing). Thus, the LMM for each eye-movement measure was built by sequentially adding terms to the null model as follows: (1) null model showing best goodness of fit, (2) null model + covariates (if any), and (3) null model + covariates (if any) + fixed effects. The model that was finally selected in each case was also the one that showed best goodness of fit (see Table 5), which was tested using the “anova” function from “stats” package v.4.0.2 for R (R Core Team, 2020). All eye-movement measures were normalized and centered using the “normalize” function with standardization method included in the “BBmisc” package v.1.11 for R (Bischl et al., 2017). 18 Results Covariates The model estimates of participants’ ART score, PPTV-4 score, and age for each eye-movement measure can be found in Table 3. ART score significantly predicted Target-word gaze duration, Target-word reading time, and Regression path duration in the full dataset, and only Target-word gaze duration and Target-word reading time in the data subset of models for Timing and Frequency. In addition, participants’ age significantly predicted Text regressions out. Therefore, ART score and participants’ age were included (centered; Cohen et al., 2013) as covariates in these cases. Thus, the marginal significant difference in ART score and the significant difference in age between Timing conditions (see Table 1) were controlled. Lastly, participants’ scores on PPTV-4 did not significantly predict any of the eye-movement measures so that this covariate was not included in any of the models. (Insert Table 3 here) Eye-movement Measures Table 4 and Table 5 show a summary of means for global and local eyemovement measures, respectively. An overview of the models that were finally selected for each eye-movement measure for Picture and Frequency and for Timing, respectively, can be found in Table 6 and Table 7. Given that these measures were standardized, the estimate values in subsequent analyses show proportion of standard deviation in each case (i.e., standardized beta coefficient). Reference values for Picture, Frequency, and Timing was set at no picture, high frequency, and concurrent presentation, respectively. (Insert Table 4, Table 5, Table 6, and Table 7 here) 19 Global Eye-movement Measures As can be seen in Table 8, Text gaze duration showed significant effects of Picture and Frequency, indicating that gaze duration was shorter on sentences with picture (2616 vs. 2854 ms.) and sentences with a high-frequency target-word (2651 vs. 2729 ms.; see Table 4). There were also significant effects of Picture and Frequency on Text reading time. Participants spent less time reading the sentences with picture (3625 vs. 4077 ms.) and when the target-word frequency was high (3647 vs. 3905 ms.; see Table 4). In addition, despite the lack of appropriate statistical power (see Analyses above), we found a significant effect of Picture and Frequency on Text regressions out indicating that the probability of regressions at text level was significantly lower also in sentences with picture (.05 vs .09) and with a high-frequency target-word (low: .05 vs high: .06; see Table 4). In the case of the models including Timing as fixed effect, we found an effect of this factor on Text regressions out (see Table 9), indicating that the probability of regressions out was higher when the picture was concurrent with the sentence (.08 vs .02, see Table 4). Lastly, we found no significant effect of Timing on Text gaze duration and on Text reading time. (Insert Table 8 and Table 9) Local Eye-movement Measures Models for the effects of Picture and Frequency on the eye-movement measures at target-word level are depicted in Table 8. Target-word gaze duration showed a significant effect of Picture, with shorter gaze duration in sentences with picture (252 vs. 279 ms), and an effect of Frequency, showing shorter gaze duration when the target- 20 word frequency was high (242 vs. 280 ms; see Table 5). It is noteworthy that this effect was found despite the insufficient statistical power of our sample size for this eyemovement measure. Moreover, Picture and Frequency also significantly affected Target-word reading time and Regression path duration. The results indicated that, in sentences with picture, reading time of the target-word was shorter than in no-picture sentences (429 vs 445 ms) and regression path duration was also shorter (446 vs 519 ms). Similarly, when target-word frequency was high (vs low), reading time of the target word was shorter (366 vs 502 ms) and regression path duration was shorter (437 vs 503 ms; see Table 5). Lastly, Target-word regressions out was significantly affected only by Frequency, as the probability of regressions on the target-word was lower when target-word frequency was high (.10 vs .14). Although this latter measure was also lower when the picture was present (.11 vs .15), the difference did not reach statistical significance (see Table 8). Regarding the models for Timing, they also revealed a significant effect of this factor on Target-word regressions out (see Table 9), indicating that the target word had higher probability of regressions out when the picture was concurrent with the sentence than when picture and sentence appeared consecutively (.18 vs .03, see Table 5). As found for global eye-tracking measures, there was no significant effect of Timing on Target-word gaze duration, Target-word reading time, and on Regression path duration. More detailed information including statistical data for random effects and model fit for each of the LMMs reported above can be found in the Appendix. Discussion The primary aim of the study was to examine the effect of visual support and how varying the timing of visual support affected sentence processing. We used eye 21 tracking to investigate online sentence processing, that is, reading comprehension processes as they occur in real time through fixation durations and regressions. We had expected that presenting visual support prior to the sentence would have a facilitating effect on reading and comprehension. In the remainder of the present section, we first go through the results of the global text eye-movement measures, and then, the local target-word eye-movement measures. Effects of Frequency, Picture, and Timing on global eye-movement measures As expected, all the global text measures (i.e., gaze duration, reading time, and regressions on the entire sentence) were affected by the presence of a picture supporting the sentences, regardless of the presentation timing (i.e., consecutive or concurrent with the sentence). In such cases, participants spent shorter time in reading the sentence and the probability of regressions was lower. A similar pattern was found for target-word frequency. When frequency was high, the participants also spent shorter times in reading the sentence and their probability of regressions was also lower than when frequency was low. It is noteworthy that, contrary to our prior work (Rivero-Contreras et al., 2021, 2023), whose findings showed that the effects of the supporting picture on global measures were not consistently significant either in university students with dyslexia and with typical development, the effects of picture on gaze duration and total reading time at text level in the present study were significant and larger than the effects of target-word frequency (standardized estimates: -0.28 and -.027 vs -0.11 and -0.16, respectively). However, these results are consistent with the idea that the effect of picture is expected to be larger than the one of frequency, as it has the potential to affect the processing of the sentence in its entirety, whereas the effect of frequency would be necessarily limited to a smaller portion of the sentence (i.e., one, two, or at most three 22 words). A possible explanation for the differences between present findings and those in Rivero-Contreras et al. (2021, 2023) is that the samples in these latter studies included adult readers with dyslexia and with low levels of education. Regarding the effect of Timing, we found a large effect (standardized estimate: 0.83; Cohen, 1988) showing that when the pictures were presented before the sentences, the probabilities of regressions at text level was lower. We provide two possible complementary explanations to this finding. On the one hand, it could be the case that presenting a picture before the sentence facilitates the construction of a situation model or the event mental representation (Kintsch, 1998; McRae et al., 2021). Visual information would provide a context for understanding written information, thus fostering the activation of students’ prior knowledge which, in turn, would speed up the generation of inferences facilitating comprehension. This would explain why participants’ probability of regressions was lower on sentences that were presented after the participants visualized the picture. On the other hand, it could be also possible that the lower probability of regressions when the picture is presented before the sentence was at least partially due to participants adapting their reading strategy to the experimental materials. Being able to look at the picture concurrent with the sentence could have led the participants to make an increased number of regressions, as they could made gaze transitions between the text and the picture. Although regressions on the sentences that occurred immediately after picture-to-text transitions were excluded from the analyses, it is possible that these transitions led the participants to make more subsequent regressions within the sentences. However, we see these two possible explanations as complementary rather than exclusive. If this is the case, it would explain the larger size of the effect, as it would be the consequence of summing both the effect of activating 23 prior knowledge before reading the sentence, and the influence of our experimental materials. Effects of Frequency, Picture, and Timing on local eye-movement measures The local target-word measures were also affected by the presence visual support, the target-word frequency, and the timing of visual support, showing the same direction that those found for the entire sentences. When visualizing the supporting picture, the students’ spent less time in reading the target-word, which was reflected both in shorter gazed duration and reading time. However, in this case, the effect of visual support on the probability of regressions in the target word did not reach statistical significance, and the effect of frequency was larger than the effect of picture both on gaze duration (standardized estimates: -0.24 vs. -0.16) and on reading time (- 0.36 vs. -0.12). Given that frequency is a factor inherent to the target-word, contrary to what was found among the global eye-movement measures, its influence was more salient on the target than on the time spent reading the whole sentence. These findings corroborate previous evidence indicating that visual support facilitates lexical access and comprehension of unfamiliar words (Huettig et al., 2011; Montag et al., 2015; Qu et al., 2016). Altogether, our results demonstrate that the use of pictures supporting textual information facilitates text processing, especially when the text contains low-frequency vocabulary. In addition, we also found a main effect of the timing of picture presentation on participants’ probability of regressions on the target-word. Similar to what was found in the global measures, presenting the picture before the sentence decreased participants’ regressions. As suggested above, this result could indicate that the students behaved differently when the picture was concurrent with the text, which supports Ferreira et al. 24 (2013), who indicate that previewing visual information (consecutive presentation) allows readers to generate certain expectations or predictions in relation to the linguistic information in the text. However, as also argued, we cannot exclude the possibility that the presence of a concurrent picture increased the number of regressions as students made more gaze transitions between the picture and the text, which could increase rereading. Before turning to the limitations of the current study, we should point out that, in general, most of the effects on the local measures found in the present study replicate findings in prior works (Rivero-Contreras et al., 2021, 2023). For the local measures, there is a clear and robust effect of word frequency, and picture has positive effects on reading time, gaze duration and regression path durations. In addition, we also found a robust effect of the timing of presentation of visual support on reading behaviour, both at text and target-word level. Although we do not have a clear explanation for this effect, we suggest that, in our study, visual support preceding the reading of the text (i.e., consecutive presentation) at least partially facilitated subsequent processing of textual information. Limitations This study has yielded some interesting and novel findings, but it is not exempt of some limitations. Firstly, as already reported, the sample was rather small to detect possible effects of Picture, Frequency, and Timing on some of the eye-movement measures similar to those found in our previous research work. Therefore, we cannot discard that the timing of the visual support also affects reading time or gaze duration at target-word level due to the lack of sufficient statistical power. Secondly, the participants consisted of a skilled and homogenous group of 25 college students. Further research could replicate our study in a larger sample and, more important, with a higher heterogeneity in terms of reading skills. Thirdly, regarding the effects of supporting pictures, we used coloured illustrations that provided many contextual elements and mirrored the content of the sentence. Whereas this type of images has been shown to be highly beneficial for reading (Carney & Levin, 2002) we did not explore the possible differences of other types of visual support such as visual scenes, infographics, or symbols. Thus, future studies can examine how other types of visual support and the timing of that visual support benefits language comprehension. Lastly, we did not analyse participants' eye-gaze on the pictures. This was due to several reasons (e.g., image presentation was time limited in one condition and selfpaced in the other). Therefore, we could not explore what parts of the pictures could be more beneficial for textual processing. In the consecutive condition, participants inspected the image before the text (unless they looked away from the screen). In contrast, in the concurrent condition participants were free to view the picture (or not), and there were no constraints on when they viewed the image versus when they read the sentence (before, after, or intermittently while reading the text). Applied Implications and Future Directions Regarding practical implications of our findings, it seems that visual support can positively impact skilled adult readers’ textual processing. In fact, the current study has shown the strongest effects we have observed to date for visual support. One clear extension of this work is how readers deal with unfamiliar words and how visual support might help access or direct readers to the meaning of an unknown word. One example is “cupola”, which is an example of a “difficult” low-frequency word in some vocabulary texts. In Figure 3, it will be important to test what readers actually look at (if 32 McRae, K., Brown, K. S., & Elman, J. L. (2021). Prediction-Based Learning and Processing of Event Knowledge. Topics in Cognitive Science, 13(1), 206-223. https://doi.org/10.1111/tops.12482 Mol, S. E., & Bus, A. G. (2011). To read or not to read: a meta-analysis of print exposure from infancy to early adulthood. Psychological Bulletin, 137(2), 267. https://doi.org/10.1037/a0021890 Montag, J. L., Jones, M. N., & Smith, L. B. (2015). The Words Children Hear: Picture books and the statistics for language learning. Psychological Science, 26(9), 1489– 1496. https://doi.org/10.1177/0956797615594361 Moore, M., & Gordon, P. C. (2015). Reading ability and print exposure: item response theory analysis of the author recognition test. Behavior research methods, 47(4), 1095–1109. https://doi.org/10.3758/s13428-014-0534-3 Morton, J. (1969). Categories of interference: Verbal mediation and conflict in card sorting. British Journal of Psychology, 60(3), 329–346. https://doi.org/10.1111/j.2044-8295.1969.tb01204.x Qu, Q., Zhang, Q., & Damian, M. F. (2016). Tracking the time course of lexical access in orthographic production: An event-related potential study of word frequency effects in written picture naming. Brain and Language, 159, 118–126. https://doi.org/10.1016/j.bandl.2016.06.008 Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372–422. https://doi.org/10.1037/0033- 2909.124.3.372 33 Rayner, K., & Duffy, S. A. (1986). Lexical complexity and fixation times in reading: Effects of word frequency, verb complexity, and lexical ambiguity. Memory & Cognition, 14(3), 191–201. https://doi.org/10.3758/BF03197692 Rayner, K., & Fischer, M. H. (1996). Mindless reading revisited: Eye movements during reading and scanning are different. Perception & Psychophysics, 58(5), 734–747. https://doi.org/10.3758/BF03213106 R Core Team. (2020). R: A language and environment for statistical computing. https://www.R-project.org/ Rayner, K., Sereno, S. C., & Raney, G. E. (1996). Eye movement control in reading: A comparison of two types of models. Journal of Experimental Psychology: Human Perception and Performance, 22(5), 1188–1200. https://doi.org/10.1037/0096- 1523.22.5.1188 Reichle, E. D., Rayner, K., & Pollatsek, A. (2003). The E-Z reader model of eyemovement control in reading: comparisons to other models. Behavioural and Brain Sciences, 26(4), 445‐526. https://doi.org/10.1017/s0140525x03000104 Rivero-Contreras, M., Engelhardt, P. E, & Saldaña, D. (2021). An experimental eyetracking study of text adaptation for readers with dyslexia: effects of visual support and word frequency. Annals of Dyslexia, 71, 170–187. https://doi.org/10.1007/s11881-021-00217-1 Rivero-Contreras, M., Engelhardt, P. E, & Saldaña, D. (2023). Do easy-to-read adaptations really facilitate sentence processing for adults with a lower level of 34 education? An experimental eye-tracking study. Learning and Instruction, 84, 101731. https://doi.org/10.1016/j.learninstruc.2022.101731 Scheepers, C., Sturt, P., Martin, C. J., Myachykov, A., Teevan, K., & Viskupova, I. (2011). Structural priming across cognitive domains: From simple arithmetic to relative-clause attachment. Psychological Science, 22(10), 1319- 1326. https://doi.org/10.1177/0956797611416997 Staub, A., White, S. J., Drieghe, D., Hollway, E. C., & Rayner, K. (2010). Distributional effects of word frequency on eye fixation durations. Journal of Experimental Psychology: Human Perception and Performance, 36(5), 1280–1293. https://doi.org/10.1037/a0016896 Strouse, G. A., Nyhout, A., & Ganea, P. A. (2018). The role of book features in young children's transfer of information from picture books to real-world contexts. Frontiers in Psychology, 9(50), 1-14. https://doi.org/10.3389/fpsyg.2018.00050 Takacs, Z. K., & Bus, A. G. (2018). How pictures in picture storybooks support young children’s story comprehension: An eye-tracking experiment. Journal of Experimental Child Psychology, 174, 1–12. https://doi.org/10.1016/j.jecp.2018.04.013 Wasik, B. A., Hindman, A. H., and Snell, E. K. (2016). Book reading and vocabulary development: a systematic review. Early Childhood Research Quarterly, 37, 39– 57. https://doi.org/10.1016/j.ecresq.2016.04.003 35 Wassenburg, S. I., de Koning, B. B., Bos, L. T., & van der Schoot, M. (2020). Inspecting a picture before reading affects attentional processing but not comprehension. Educational Psychology, 40, 4–21. https://doi.org/10.1080/01443410.2019.1645306 Whaley, C. P. (1978). Word—nonword classification time. Journal of Verbal Learning and Verbal Behavior, 17(2), 143–154. https://doi.org/10.1016/S0022- 5371(78)90110-X 36 Tables Table 1. Summary of participants’ age and scores on PPVT-4, ART, and Inferential comprehension by Timing conditions. Concurrent timing (n = 20) Consecutive timing (n = 20) t value M (SD) Min-Max M (SD) Min-Max Age1 19.90 (1.25) 18-22 19.05 (1.05) 18-21 2.36* PPVT-42 101.45 (8.99) 79-117 99.05 (9.91) 79-120 0.80 ART 9.70 (7.50) 0-30 6.35 (3.47) 0-13 1.81· ICS (%) 84.49 (7.94) 64.82-94.57 84.25 (5.52) 72.75-94.39 0.11 Note. PPVT-4: Peabody Picture Vocabulary Test; ART: Author Recognition Test; ICS: Inferential Comprehension (true-or-false statement from experimental task). 1In years. 2Standard scores. ·p < .10, *p < .05. 37 Table 2. Results of a priori statistical power analyses for each eye-movement measure based on the dataset and effects found in Authors (2021) Factor Eye-movement measure Necessary sample size1 to reach β = .20 60 items 30 items Frequency Text reading time 14 n/a2 Text regressions out 60* 75* Target-word gaze duration 16 n/a2 Target-word reading time 4 n/a2 Target-word regressions out 33 Target-word regression path duration 25 n/a2 Picture Text reading time 13 15 Text gaze duration 10 12 Target-word reading time 24 47 Target-word gaze duration 46* 100* Target-word regression path duration 37 65* Note. 1In number of participants; 2Statistical power simulation for 30 items was not performed on this effect because the effect of Picture on this measure was prioritised to simulate power for Timing (i.e., dataset with 30 items per participant); *Insufficient statistical power for our study’s sample size. 38 Table 3. Estimates of predicting models for ART score, PPTV-4 score, and participants’ age on each eye-movement measure. Global eye-movement measures Local eye-movement measures Text gaze duration Text reading time Text regressio ns out Targetword gaze duration Targetword reading time Targetword regressio ns out Regressio n path duration Estimates for data included in the models for Picture and Frequency (kitem = 60) ART -0.14 -0.11 -0.04 -0.11* -0.11* -0.01 -0.08* PPTV-4 -0.12 -0.07 -0.07 -0.09 -0.08 -0.02 -0.05 Age 0.004 0.01 0.12* -0.01 -0.05 0.03 0.06 Estimates for data subset included in analyses of the effect of Timing (kitem = 30) ART -0.12 -0.12 0.13 -0.12* -0.13* 0.06 -0.08 PPTV-4 -0.09 -0.09 -0.01 -0.09 -0.07 0.01 -0.05 Age 0.04 0.02 0.24* -0.01 -0.01 0.11 0.08 Note. ART: Author recognition task. PPVT-4: Peabody Picture Vocabulary Test. *p < .05. 39 Table 4. Means and standard deviations of global eye-tracking measures. Low frequency M (SD) High Frequency M (SD) No picture (n = 40) Text gaze duration (ms) 2906 (860) 2802 (865) Text reading time (ms) 4221 (1738) 3933 (1671) Text regressions out (p) .09 (.08) .08 (.07) Picture (Concurrent Timing; n = 20) Text gaze duration (ms) 2563 (731) 2485 (723) Text reading time (ms) 3690 (1389) 3390 (1457) Text regressions out (p) .09 (.08) .07 (.07) Picture (Consecutive Timing; n = 20) Text gaze duration (ms) 2748 (855) 2667 (750) Text reading time (ms) 3804 (1620) 3619 (1493) Text regressions out (p) .01 (.05) .01 (.05) Note. ms = milliseconds; p = probability. 40 Table 5. Means and standard deviations of local eye-tracking measures. Low frequency M (SD) High Frequency M (SD) No picture (n = 40) Target-word gaze duration (ms) 308 (224) 249 (158) Target-word reading time (ms) 532 (394) 357 (297) Target-word regressions out (p) .16 (.37) .15 (.36) Regression path duration (ms) 544 (581) 493(548) Picture (Concurrent Timing; n = 20) Target-word gaze duration (ms) 253 (160) 226 (145) Target-word reading time (ms) 483 (338) 335 (214) Target-word regressions out (p) .22 (.42) .13 (.34) Regression path duration (ms) 489 (459) 389 (446) Picture (Consecutive Timing; n = 20) Target-word gaze duration (ms) 278 (169) 250 (149) Target-word reading time (ms) 491 (392) 406 (281) Target-word regressions out (p) .03 (.18) .03 (.16) Regression path duration (ms) 475 (441) 430 (361) Note. ms = milliseconds; p = probability. 41 Table 6. Overview of the models for Picture and Frequency with best goodness of fit for each eye-movement measure. Global eyemovement measures Fixed effects Random effects Item Participant Text gaze duration Picture + Frequency Intercept Intercept + Slope (Picture + Frequency) Text reading time Picture + Frequency Intercept Intercept + Slope (Picture + Frequency) Text regressions out Picture + Frequency + Age Intercept Intercept + Slope (Picture) Local eye-movement measures Fixed effects Random effects Item Participant Target-word gaze duration Picture + Frequency + ART score Intercept + Slope (Picture) Intercept + Slope (Picture) Target-word reading time Picture + Frequency + ART score Intercept Intercept + Slope (Frequency) Target-word regressions out Picture + Frequency Intercept Intercept + Slope (Picture + Frequency) Regression path duration Picture + Frequency + ART score Intercept Intercept + Slope (Picture) 48 Figure 3. Examples images and texts. 49 Appendix Table A1. Model for main effects of Picture and Frequency on Text gaze duration. Fixed effects Estimate SE 95% CI t p Intercept 0.20 0.13 [-0.06, 0.45] 1.50 .14 Picture -0.28 0.05 [-0.38, -0.18] -5.72 < .001 Frequency -0.11 0.03 [-0.18, -0.04] -3.22 < .01 Random effects Variance SD Correlation Item (intercept) 0.20 0.45 - Participant (intercept) 0.53 0.73 - Participant (Picture) 0.07 0.26 -.68 Participant (Frequency) 0.02 0.14 -.45; .40 Model fit Marginal R2 Conditional R2 .02 0.62 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 50 Table A2. Model for main effects of Picture and Frequency on Text reading time. Fixed effects Estimate SE 95% CI t p Intercept 0.22 0.12 [-0.01, 0.46] 1.86 .07 Picture -0.27 0.07 [-0.40, -0.14] -4.13 < .001 Frequency -0.16 0.04 [-0.25, -0.08] -3.95 < .001 Random effects Variance SD Correlation Item (intercept) 0.20 0.44 - Participant (intercept) 0.42 0.65 - Participant (Picture) 0.14 0.38 -.66 Participant (Frequency) 0.04 0.19 -0.31; .01 Model fit Marginal R2 Conditional R2 0.03 0.53 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 51 Table A3. Model for main effects of Picture and Frequency on Text regressions out. Fixed effects Estimate SE 95% CI t p Intercept 0.11 0.06 [-0.004, 0.21] 1.89 .06 Age 0.12 0.05 [0.02, 0.23] 2.28 0.03 Picture -0.50 0.10 [-0.69, -0.31] -5.17 <.001 Frequency -0.11 0.05 [-0.20, -0.01] -2.17 .04 Random effects Variance SD Correlation Item (Intercept) 0.02 0.14 - Participant (Intercept) 0.18 0.42 - Participant (Picture) 0.32 0.57 -.37 Participant (Frequency) 0.05 0.22 -.49; -.16 Model fit Marginal R2 Conditional R2 .08 .31 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 52 Table A4. Model for main effect of Picture and Frequency on Target-word gaze duration. Fixed effects Estimate SE 95% CI t p Intercept 0.17 0.08 [0.06, 0.37] 2.29 .02 ART score -0.11 0.05 [-0.20, -0.03] -2.71 .02 Picture -0.15 0.04 [-0.25, -0.04] -3.50 .01 Frequency -0.24 0.04 [-0.32, -0.16] -5.94 < .001 Random effects Variance SD Correlation Item (Intercept) 0.12 0.34 - Item (Picture) 0.03 0.17 -.88 Participant (Intercept) 0.11 0.33 - Participant (Picture) 0.04 0.19 -.68 Model fit Marginal R2 Conditional R2 .03 0.20 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 53 Table A5. Model for main effects of Picture and Frequency on Target-word reading time. Fixed effects Estimate SE 95% CI t p Intercept 0.21 0.08 [0.05, 0.37] 2.62 .01 ART score -0.11 0.05 [-0.20, -0.02] -2.42 .02 Picture -0.12 0.04 [-0.19, -0.05] -3.29 < .01 Frequency -0.36 0.06 [-0.48, -0.24] -6.07 < .001 Random effects Variance SD Correlation Item (Intercept) 0.12 0.35 - Participant (Intercept) 0.13 0.37 - Participant (Frequency) 0.09 0.29 -.73 Model fit Marginal R2 Conditional R2 .05 .27 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 54 Table A6. Model for main effects of Picture and Frequency on Target-word regressions out. Fixed effects Estimate SE 95% CI t p Intercept 0.11 0.06 [-0.004, 0.22] 1.89 .06 Picture -0.14 0.08 [-0.30, 0.01] -1.81 .08 Frequency -0.09 0.04 [-0.17, -0.01] -2.18 .03 Random effects Variance SD Correlation Item (intercept) 0.01 0.12 - Participant (intercept) 0.07 0.27 - Participant (Picture) 0.18 0.42 -.62 Model fit Marginal R2 Conditional R2 .01 .11 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 55 Table A7. Model for main effects of Picture and Frequency on Target-word regression path duration. Fixed effects Estimate SE 95% CI t p Intercept 0.14 0.08 [-0.01, 0.29] 1.85 .07 ART score -0.08 0.03 [-0.15, -0.01] -2.35 .02 Picture -0.14 0.05 [-0.23, -0.05] -3.01 < .01 Frequency -0.13 0.04 [-0.20, -0.05] -3.39 < .001 Random effects Variance SD Correlation Item (Intercept) 0.19 0.43 - Participant (Intercept) 0.07 0.26 - Participant (Picture) 0.03 0.17 -.80 Model fit Marginal R2 Conditional R2 .02 .25 Note. Reference value for Picture: No picture. Reference value for Frequency: Low. 56 Table A8. Model for main effect of Timing on Text gaze duration. Fixed effects Estimate SE 95% CI t p Intercept -0.11 0.15 [-0.41, 0.18] -0.76 .45 Timing 0.23 0.19 [-0.14, 0.61] 1.22 .23 Random effects Variance SD Correlation Item (intercept) 0.25 0.47 - Participant (intercept) 0.36 0.60 - Model fit Marginal R2 Conditional R2 .01 .58 Note. Reference value for Timing: Concurrent. Reference value for Frequency: Low 57 Table A10. Model for main effect of Timing on Text reading time. Fixed effects Estimate SE 95% CI t p Intercept -0.05 0.13 [-0.31, 0.22] -0.35 .72 Timing 0.11 0.17 [-0.21, 0.44] 0.69 .50 Random effects Variance SD Correlation Item (intercept) 0.25 0.50 - Participant (intercept) 0.26 0.51 - Model fit Marginal R2 Conditional R2 .003 .50 Note. Reference value for Timing: Concurrent. Reference value for Frequency: Low