No CrossRef data available.
Article contents
Predicting processing effort during L1 and L2 reading: The relationship between text linguistic features and eye movements
Published online by Cambridge University Press: 06 February 2023
Abstract
Researchers have taken great interest in the assessment of text readability. This study expands on this research by developing readability models that predict the processing effort involved during first language (L1) and second language (L2) text reading. Employing natural language processing tools, the study focused on assessing complex linguistic features of texts, and these features were used to explain the variance in processing effort, as evidenced by eye movement data for L1 or L2 readers of English that were extracted from an open eye-tracking corpus. Results indicated that regression models using the indices of complex linguistic features provided better performance in predicting processing effort for both L1 and L2 reading than the models using simple linguistic features (word and sentence length). Furthermore, many of the predictive variables were lexical features for both L1 and L2 reading, emphasizing the importance of decoding for fluent reading regardless of the language used.
- Type
- Research Article
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press
References
Ardoin, SP, Suldo, SM, Witt, J, Aldrich, S and McDonald, E (2005) Accuracy of readability estimates’ predictions of CBM performance. School Psychology Quarterly, 20, 1–22. https://doi.org/10.1521/scpq.20.1.1.64193CrossRefGoogle Scholar
Baayen, RH and Milin, P (2010) Analyzing reaction times. International Journal of Psychological Research 3, 12–28. https://doi.org/10.21500/20112084.807CrossRefGoogle Scholar
Bertram, B and Newman, S (1981) Why readability formulas fail (Report No. 28). Illinois University, Urbana: Center for the Study of Reading (Eric Document Service No. ED205915).Google Scholar
Blei, DM, Ng, AY and Jordan, MI (2003) Latent dirichlet allocation. Journal of Machine Learning Research 3, 993–1022.Google Scholar
BNC Consortium, (2007) The British National Corpus, version 3. BNC Consortium. Retrieved from www.natcorp.ox.ac.ukGoogle Scholar
Britton, BK and Gülgöz, S (1991) Using Kintsch's computational model to improve instructional text: Effects of repairing inference calls on recall and cognitive structures. Journal of Educational Psychology 83(3), 329–345. https://doi.org/10.1037/0022-0663.83.3.329CrossRefGoogle Scholar
Brysbaert, M and New, B (2009) Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods 41, 977–990. https://doi.org/10.3758/BRM.41.4.977CrossRefGoogle Scholar
Buchanan, L, Westbury, C and Burgess, C (2001) Characterizing semantic space: Neighborhood effects in word recognition. Psychonomic Bulletin & Review 8(3), 531–544. https://doi.org/10.3758/BF03196189CrossRefGoogle ScholarPubMed
Chaffin, R, Morris, RK and Seely, RE (2001) Learning new word meanings from context: A study of eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition 27(1), 225–235. https://doi.org/10.1037/0278-7393.27.1.225CrossRefGoogle Scholar
Clahsen, H and Felser, C (2006) How native-like is non-native language processing? Trends in Cognitive Sciences 10(12), 564–570. https://doi.org/10.1016/j.tics.2006.10.002CrossRefGoogle ScholarPubMed
Collins-Thompson, K (2014) Computational assessment of text readability: A survey of current and future research. ITL-International Journal of Applied Linguistics 165(2), 97–135. https://doi.org/10.1075/itl.165.2.01colCrossRefGoogle Scholar
Conklin, K and Schmitt, N (2008) Formulaic sequences: Are they processed more quickly than nonformulaic language by native and nonnative speakers? Applied Linguistics 29(1), 7289. https://doi.org/10.1093/applin/amm022CrossRefGoogle Scholar
Conklin, K, Pellicer-Sánchez, A and Carrol, G (2018) Eye-tracking: A guide for applied linguistics research. Cambridge University Press.CrossRefGoogle Scholar
Cop, U, Drieghe, D and Duyck, W (2015a) Eye movement patterns in natural reading: A comparison of monolingual and bilingual reading of a novel. PLOS One 10, e0134008. https://doi.org/10.1371/journal.pone.0134008CrossRefGoogle Scholar
Cop, U, Keuleers, E, Drieghe, D and Duyck, W (2015b) Frequency effects in monolingual and bilingual natural reading. Psychonomic Bulletin & Review 22(5), 1216–1234. https://doi.org/10.3758/s13423-015-0819-2CrossRefGoogle ScholarPubMed
Crossley, SA, Allen, D and McNamara, DS (2012) Text simplification and comprehensible input: A case for an intuitive approach. Language Teaching Research 16, 89–108. https://doi.org/10.1177/1362168811423456CrossRefGoogle Scholar
Crossley, SA, Allen, DB and McNamara, DS (2011) Text readability and intuitive simplification: A comparison of readability formulas. Reading in a Foreign Language 23, 84–101.Google Scholar
Crossley, SA, Allen, LK, Kyle, K and McNamara, DS (2014) Analyzing discourse processing using a simple natural language processing tool. Discourse Processes 51(5-6), 511–534. https://doi.org/10.1080/0163853X.2014.910723CrossRefGoogle Scholar
Crossley, SA, Greenfield, J and McNamara, DS (2008) Assessing text readability using cognitively based indices. TESOL Quarterly 42, 475–493. https://doi.org/10.1002/j.1545-7249.2008.tb00142.xCrossRefGoogle Scholar
Crossley, SA, Heintz, A, Choi, J, Batchelor, J, Karimi, M and Malatinszky, A (2022) A large-scaled corpus for assessing text readability. Behavioral Research Methods. https://doi.org/10.3758/s13428-022-01802-xCrossRefGoogle ScholarPubMed
Crossley, SA., Kyle, K and Dascalu, M (2019a) The tool for the automatic analysis of cohesion 2.0: Integrating semantic similarity and text overlap. Behavioral Research Methods 50, 1030–1046. https://doi.org/10.3758/s13428-018-1142-4Google Scholar
Crossley, SA, Skalicky, S and Dascalu, M (2019b) Moving beyond classic readability formulas: New methods and new models. Journal of Research in Reading 42, 541–561. https://doi.org/10.1111/1467-9817.12283CrossRefGoogle Scholar
Crossley, SA, Kyle, K and McNamara, DS (2016) The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods 48, 1227–1237. https://doi.org/10.3758/s13428-015-0651-7CrossRefGoogle ScholarPubMed
Crossley, SA, Kyle, K and McNamara, DS (2017a) Sentiment analysis and social cognition engine (SEANCE): An automatic tool for sentiment, social cognition, and social-order analysis. Behavior Research Methods 49, 803–821. https://doi.org/10.3758/s13428-016-0743-zCrossRefGoogle ScholarPubMed
Crossley, SA, Skalicky, S, Dascalu, M, McNamara, DS and Kyle, K (2017b) Predicting text comprehension, processing, and familiarity in adult readers: New approaches to readability formulas. Discourse Processes 54(5-6), 340–359. https://doi.org/10.1080/0163853X.2017.1296264CrossRefGoogle Scholar
Dale, E and Chall, JS (1949) The concept of readability. Elementary English 26, 19–26.Google Scholar
Davies, M (2009) The 385+ million word Corpus of Contemporary American English (1990–2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics 14, 159–190. https://doi.org/10.1075/ijcl.14.2.02daCrossRefGoogle Scholar
Dascalu, M, Dessus, P, Bianco, M, Trausan-Matu, S and Nardy, A (2014) Mining texts, learner productions and strategies with ReaderBench. In Peña-Ayala, A (ed.), Educational data mining: Applications and trends. Cham, Switzerland: Springer, pp. 345–377.CrossRefGoogle Scholar
De Clercq, O and Hoste, V (2016) All mixed up? Finding the optimal feature set for general readability prediction and its application to English and Dutch. Computational Linguistics 42(3), 457–490. https://doi.org/10.1162/COLI_a_00255CrossRefGoogle Scholar
Dirix, N and Duyck, W (2017) The first-and second-language age of acquisition effect in first-and second-language book reading. Journal of Memory and Language 97, 103–120. https://doi.org/10.1016/j.jml.2017.07.012CrossRefGoogle Scholar
Dirix, N, Vander Beken, H, De Bruyne, E, Brysbaert, M and Duyck, W (2019) Reading text when studying in a second language: An eye-tracking study. Reading Research Quarterly 55(3), 371–397. https://doi.org/10.1002/rrq.277CrossRefGoogle Scholar
Eguchi, M and Kyle, K (2020) Continuing to explore the multidimensional nature of lexical sophistication: The case of oral proficiency interviews. The Modern Language Journal 104(2), 381–400. https://doi.org/10.1111/modl.12637CrossRefGoogle Scholar
Feng, L, Jansche, M, Huenerfauth, M and Elhadad, N (2010) A comparison of features for automatic readability assessment. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters (pp. 276–284). USA: Association for Computational Linguistics.Google Scholar
Flesch, R (1948) A new readability yardstick. Journal of Applied Psychology 32, 221–233. https://doi.org/10.1037/h0057532CrossRefGoogle ScholarPubMed
George, D and Mallery, P (2010) SPSS for Windows step by step: A simple guide and reference 17.0 Update (10th ed.). Boston, MA: Pearson.Google Scholar
Godfroid, A (2019) Eye tracking in second language acquisition and bilingualism: A research synthesis and methodological guide. New York, NY: Routledge.CrossRefGoogle Scholar
Godfroid, A and Hui, B (2020) Five common pitfalls in eye-tracking research. Second Language Research 36(3), 277–305. https://doi.org/10.1177/0267658320921218CrossRefGoogle Scholar
Gollan, TH, Montoya, RI, Cera, C and Sandoval, TC (2008) More use almost always means a smaller frequency effect: Aging, bilingualism, and the weaker links hypothesis. Journal of Memory and Language 58(3), 787–814. https://doi.org/10.1016/j.jml.2007.07.001CrossRefGoogle Scholar
Grabe, W (2009) Reading in a second language: Moving from theory to practice. New York, NY: Cambridge University Press.Google Scholar
Graesser, AC and McNamara, DS (2011) Computational analyses of multilevel discourse comprehension. Topics in Cognitive Science 3(2), 371–398. https://doi.org/10.1111/j.1756-8765.2010.01081.xCrossRefGoogle ScholarPubMed
Greenfield, G (1999) Classic readability formulas in an EFL context: Are they valid for Japanese speakers? Unpublished doctoral dissertation, Temple University, Philadelphia, PA, United States (University Microfilms No. 99–38670).Google Scholar
Gries, ST (2021) (Generalized linear) Mixed-effects modeling: A learner corpus example. Language Learning. https://doi.org/10.1111/lang.12448CrossRefGoogle Scholar
Hamsik, MJ (1984) Reading, readability, and the ESL reader. Unpublished doctoral dissertation, The Florida State University, U.S.Google Scholar
Holmes, VM and O'Regan, JK (1981) Eye fixation patterns during the reading of relative-clause sentences. Journal of Verbal Learning and Verbal Behavior 20, 417–430. https://doi.org/10.1016/S0022-5371(81)90533-8CrossRefGoogle Scholar
Horiba, Y (1996) Comprehension processes in L2 reading: Language competence, textual coherence, and inferences. Studies in Second Language Acquisition 18, 433–473. https://doi.org/10.1017/S0272263100015370CrossRefGoogle Scholar
Horiba, Y (2000) Reader control in reading: Effects of language competence, text type, and task. Discourse Processes 29(3), 223–267. https://doi.org/10.1207/S15326950dp2903_3CrossRefGoogle Scholar
Jeon, EH and Yamashita, J (2014) L2 reading comprehension and its correlates: A meta-analysis. Language Learning 64, 160–212. https://doi.org/10.1111/lang.12034CrossRefGoogle Scholar
Juhasz, BJ and Rayner, K (2003) Investigating the effects of a set of intercorrelated variables on eye fixation durations in reading. Journal of Experimental Psychology. Learning, Memory, and Cognition 29(6), 1312–1318. https://doi.org/10.1037/0278-7393.29.6.1312CrossRefGoogle ScholarPubMed
Kate, RJ, Luo, X, Patwardhan, S, Franz, M, Florian, R, Mooney, RJ, Roukos, R and Welty, C (2010) Learning to predict readability using diverse linguistic features. In In Proceedings of the 23rd International Conference on Computational Linguistics (pp. 546–554). USA: Association for Computational Linguistics.Google Scholar
Kim, M, Crossley, SA and Skalicky, S (2018) Effects of lexical features, textual properties, and individual differences on word processing times during second language reading comprehension. Reading and Writing 31, 1155–1180. https://doi.org/10.1007/s11145-018-9833-xCrossRefGoogle Scholar
Kincaid, JP, Fishburne, RP, Rogers, RL and Chissom, BS (1975) Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report 8–75. Millington, TN: Naval Technical Training, U. S. Naval Air Station, Memphis, TN.CrossRefGoogle Scholar
Kintsch, W (1998) Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press.Google Scholar
Koda, K (2005) Insights into second language reading: A cross-linguistic approach. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Kuperman, V, Drieghe, D, Keuleers, E and Brysbaert, M (2013) How strongly do word reading times and lexical decision times correlate? Combining data from eye movement corpora and megastudies. Quarterly Journal of Experimental Psychology 66(3), 563–580. https://doi.org/10.1080/17470218.2012.658820CrossRefGoogle ScholarPubMed
Kuperman, V, Siegelman, N, Schroeder, S, Alexeeva, A, Acartürk, C, Amenta, S, … Usual, KA (2022) Text reading in English as a second language: Evidence from the multilingual eye-movements corpus (MECO). Studies in Second Language Acquisition. https://doi.org/10.1017/S0272263121000954Google Scholar
Kyle, K and Crossley, SA (2015) Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly 49, 757–786. https://doi.org/10.1002/tesq.194CrossRefGoogle Scholar
Kyle, K and Crossley, SA (2018) Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal 102, 333–349. https://doi.org/10.1111/modl.12468CrossRefGoogle Scholar
Kyle, K, Crossley, S and Berger, C (2018) The tool for the automatic analysis of lexical sophistication (TAALES): Version 2.0. Behavior Research Methods 50, 1030–1046. https://doi.org/10.3758/s13428-017-0924-4CrossRefGoogle ScholarPubMed
Landauer, TK, Foltz, PW and Laham, D (1998) An introduction to latent semantic analysis. Discourse Processes 25(2–3), 259–284. https://doi.org/10.1080/01638539809545028CrossRefGoogle Scholar
Li, H, Warrington, KL, Pagán, A, Paterson, KB and Wang, X (2021) Independent effects of collocation strength and contextual predictability on eye movements in reading. Language, Cognition and Neuroscience, 1–9. https://doi.org/10.1080/23273798.2021.1922726Google Scholar
Lu, X (2010) Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics 15(4), 474–496. https://doi.org/10.1075/ijcl.15.4.02luCrossRefGoogle Scholar
Marinis, T, Roberts, L, Felser, C and Clahsen, H (2005) Gaps in second language sentence processing. Studies in Second Language Acquisition, 27(1), 53–78. https://doi.org/10.1017/S0272263105050035CrossRefGoogle Scholar
McDonald, SA and Shillcock, RC (2001) Rethinking the word frequency effect: The neglected role of distributional information in lexical processing. Language and Speech 44(3), 295–322. https://doi.org/10.1177/00238309010440030101CrossRefGoogle ScholarPubMed
McNamara, DS, Graesser, AC, McCarthy, PM and Cai, Z (2014) Automated evaluation of text and discourse with Coh-Metrix. New York, NY: Cambridge University Press.CrossRefGoogle Scholar
Mesmer, HAE (2008) Tools for matching readers to texts: Research-based practices. New York, NY: Guilford Press.Google Scholar
Mikolov, T, Chen, K, Corrado, G and Dean, J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv, 1301–3781.Google Scholar
Mizumoto, A (2022). Calculating the relative importance of multiple regression predictor variables using dominance analysis and random forests. Language Learning (Early View). https://doi.org/10.1111/lang.12518Google Scholar
Morishima, Y (2013) Allocation of limited cognitive resources during text comprehension in a second language. Discourse Processes 50, 577–597. https://doi.org/10.1080/0163853X.2013.846964CrossRefGoogle Scholar
Nahatame, S (2018) Comprehension and processing of paired sentences in second language reading: A comparison of causal and semantic relatedness. Modern Language Journal 102, 392–415. https://doi.org/10.1111/modl.12466CrossRefGoogle Scholar
Nahatame, S (2020) Revisiting second language readers' memory for narrative texts: The role of causal and semantic text relations. Reading Psychology 41(8), 753–777. https://doi.org/10.1080/02702711.2020.1768986CrossRefGoogle Scholar
Nahatame, S (2021) Text readability and processing effort in second language reading: A computational and eye-tracking investigation. Language Learning, 71(4), 1004–1043. doi.org/10.1111/lang.12455.CrossRefGoogle Scholar
Nahatame, S (2022) Causal and semantic relations in second language text processing: An eye-tracking study. Reading in a Foreign Language, 34(1), 91–115. https://nflrc.hawaii.edu/rfl/item/546Google Scholar
Nisbet, K, Bertram, R, Erlinghagen, C, Pieczykolan, A and Kuperman, V (2021) Quantifying the difference in reading fluency between L1 and L2 readers of English. Studies in Second Language Acquisition, 1–28. https://doi:10.1017/S0272263121000279Google Scholar
Ozuru, Y, Dempsey, K and McNamara, DS (2009) Prior knowledge, reading skill, and text cohesion in the comprehension of science texts. Learning and Instruction 19(3), 228–242. https://doi.org/10.1016/j.learninstruc.2008.04.003CrossRefGoogle Scholar
Pitler, E and Nenkova, A (2008) Revisiting readability: A unified framework for predicting text quality. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 186–195). USA: Association for Computational Linguistics.Google Scholar
Rayner, K (1998) Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 124, 372–422. https://doi.org/:10.1037/0033-2909.124.3.372CrossRefGoogle ScholarPubMed
Rayner, K, Chace, K, Slattery, TJ and Ashby, J (2006) Eye movements as reflections of comprehension processes in reading. Scientific Studies of Reading 10, 241–255. https://doi.org/10.1207/s1532799xssr1003_3CrossRefGoogle Scholar
Core Team, R (2021) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available from https://www.R-project.org/Google Scholar
Reichle, ED, Rayner, K and Pollatsek, A (1999) Eye movement control in reading: Accounting for initial fixation locations and refixations within the EZ Reader model. Vision Research 39(26), 4403–4411. https://doi.org/10.1016/S0042-6989(99)00152-2CrossRefGoogle Scholar
Rets, I (2021) Linguistic accessibility of Open Educational Resources: Text Simplification as an aid to non-native readers of English. Doctoral dissertation, the Open University. http://oro.open.ac.uk/75140/1/Rets_thesis_ORO.pdfGoogle Scholar
Rets, I and Rogaten, J (2021) To simplify or not? Facilitating English L2 users' comprehension and processing of open educational resources in English using text simplification. Journal of Computer Assisted Learning 37(3), 705–717. https://doi.org/10.1111/jcal.12517CrossRefGoogle Scholar
Richards, JC and Schmidt, RW (2013) Longman dictionary of language teaching and applied linguistics. New York, NY: Routledge.CrossRefGoogle Scholar
Sandoval, TC, Gollan, TH, Ferreira, VS, and Salmon, DP (2010) What causes the bilingual disadvantage in verbal fluency? The dualtask analogy. Bilingualism: Language and Cognition 13(2), 231–252. https://doi.org/10.1017/S1366728909990514CrossRefGoogle Scholar
Siegelman, N, Schroeder, S, Acartürk, C, Ahn, HD, Alexeeva, S, Amenta, S and Kuperman, V (2022) Behavior Research Methods. Behavior Research Methods 54, 2843–2863. https://doi.org/10.3758/s13428-021-01772-6CrossRefGoogle Scholar
Siyanova-Chanturia, A, Conklin, K and Schmitt, N (2011) Adding more fuel to the fire: An eye-tracking study of idiom processing by native and non-native speakers. Second Language Research 27(2), 251–272. https://doi.org/10.1177/0267658310382068CrossRefGoogle Scholar
Sonbul, S (2015) Fatal mistake, awful mistake, or extreme mistake? Frequency effects on off-line/on-line collocational processing. Bilingualism: Language and Cognition 18(3), 419–437. https://doi.org/10.1017/S1366728914000674CrossRefGoogle Scholar
Tabachnick, BG and Fidell, LS (2014) Using multivariate statistics (6th Ed.). Boston, MA: Pearson Education Limited.Google Scholar
Tabossi, P, Fanari, R and Wolf, K (2009) Why are idioms recognized fast? Memory & Cognition 37(4), 529–540. https://doi.org/10.3758/MC.37.4.529CrossRefGoogle ScholarPubMed
Tonidandel, S and LeBreton, JM (2011) Relative importance analysis: A useful supplement to regression analysis. Journal of Business and Psychology 26(1), 1–9. https://doi.org/10.1007/s10869-010-9204-3CrossRefGoogle Scholar
Underwood, G, Schmitt, N and Galpin, A (2004) The eyes have it: An eye-movement study into the processing of formulaic sequences. In Schmitt, N (ed.), Formulaic sequences. Amsterdam, the Netherlands: John Benjamins.Google Scholar
Van Heuven, WJ, Mandera, P, Keuleers, E and Brysbaert, M (2014) SUBTLEX-UK: A new and improved word frequency database for British English. Quarterly Journal of Experimental Psychology 67(6), 1176–1190. https://doi.org/10.1080/17470218.2013.850521CrossRefGoogle ScholarPubMed
Weiss, BA (2011) Fisher's r-to-Z transformation calculator to compare two independent samples [Computer software]. Available from https://blogs.gwu.edu/weissba/teaching/calculators/fishers-z-transformation/Google Scholar
Whitford, V and Titone, D (2017) The effects of word frequency and word predictability during first- and second-language paragraph reading in bilingual older and younger adults. Psychology and Aging 32(2), 158–177. https://doi.org/10.1037/pag0000151CrossRefGoogle ScholarPubMed
Yamashita, J (2013) Word recognition subcomponents and passage level reading in a foreign language. Reading in a Foreign Language 25, 52–71.Google Scholar
Yap, MJ and Balota, DA (2009) Visual word recognition of multisyllabic words. Journal of Memory and Language 60(4), 502–529. https://doi.org/10.1016/j.jml.2009.02.001CrossRefGoogle Scholar
Yarkoni, T, Balota, D and Yap, M (2008) Moving beyond Coltheart's N: A new measure of orthographic similarity. Psychonomic Bulletin & Review 15(5), 971–979. https://doi.org/10.3758/PBR.15.5.971CrossRefGoogle Scholar
Yeari, M, van den Broek, P and Oudega, M (2015) Processing and memory of central versus peripheral information as a function of reading goals: Evidence from eye-movements. Reading and Writing 28(8), 1071–1097. https://doi.org/10.1007/s11145-015-9561-4CrossRefGoogle Scholar
Zufferey, S, Mak, W, Degand, L and Sanders, T (2015) Advanced learners’ comprehension of discourse connectives: The role of L1 transfer across on-line and off-line tasks. Second Language Research 31, 389–411. https://doi.org/10.1177/0267658315573349CrossRefGoogle Scholar
Nahatame supplementary material
Nahatame supplementary material 1
File
152.8 KB
Nahatame supplementary material
Nahatame supplementary material 2
File
14.9 KB