Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-17T18:30:10.828Z Has data issue: false hasContentIssue false
  • English
  • Français

MEASURING LONGITUDINAL WRITING DEVELOPMENT USING INDICES OF SYNTACTIC COMPLEXITY AND SOPHISTICATION

Published online by Cambridge University Press:  27 October 2020

Kristopher Kyle Open the ORCID record for Kristopher Kyle [Opens in a new window] ,
Scott Crossley Open the ORCID record for Scott Crossley [Opens in a new window]  and
Marjolijn Verspoor
Kristopher Kyle*
Affiliation:
University of Oregon and Yonsei University
Scott Crossley*
Affiliation:
Georgia State University
Marjolijn Verspoor*
Affiliation:
University of Groningen
*
*Correspondence concerning this article should be addressed to Kristopher Kyle, Straub Hall 161, 1290 University of Oregon, Eugene, Oregon 97403-1290. E-mail: kkyle2@uoregon.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Measures of syntactic complexity such as mean length of T-unit have been common measures of language proficiency in studies of second language acquisition. Despite the ubiquity and usefulness of such structure-based measures, they could be complemented with measures based on usage-based theories, which focus on the development of not just syntactic forms but also form-meaning pairs, called constructions (Ellis, 2002). Recent cross-sectional research (Kyle & Crossley, 2017) has indicated that indices related to usage-based characteristics of verb argument construction (VAC) use may be better indicators of writing proficiency than structure-based indices of syntactic complexity. However, because cross-sectional studies can only show general trends across proficiency benchmarks, it is important to test these findings in individuals over time (Lowie & Verspoor, 2019). Thus, this study investigates the developmental trajectories of second language learners of English across two academic years with regard to syntactic complexity and VAC sophistication.

Type
Research Article
Information
Studies in Second Language Acquisition , Volume 43 , Issue 4 , September 2021 , pp. 781 - 812
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We would like to thank the four anonymous reviewers and the handling editor, Andrea Revesz, for their detailed comments on previous versions of this manuscript.

References

REFERENCES

Barton, K. (2013). MuMln: Multi-model inference . R package, version, 1.9.Google Scholar
Bates, D. M. (2010). lem4: Mixed-effects Modeling with R. Unpublished Manuscript.Google Scholar
Bestgen, Y., & Granger, S. (2014). Quantifying the development of phraseological competence in L2 English writing: An automated approach. Journal of Second Language Writing, 26, 28–41. https://doi.org/10.1016/j.jslw.2014.09.004.CrossRefGoogle Scholar
Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45, 5–35.CrossRefGoogle Scholar
Biber, D., Gray, B., & Staples, S. (2014). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37, 639–668. https://doi.org/10.1093/applin/amu059.CrossRefGoogle Scholar
BNC Consortium. (2007). The British National Corpus, version 3. BNC Consortium. http://www.natcorp.ox.ac.uk/.Google Scholar
Bulté, B., & Housen, A. (2014). Conceptualizing and measuring short-term changes in L2 writing complexity. Journal of Second Language Writing, 26, 42–65. https://doi.org/10.1016/j.jslw.2014.09.005.CrossRefGoogle Scholar
Bulté, B., & Housen, A. (2018). Syntactic complexity in L2 writing: Individual pathways and emerging group trends. International Journal of Applied Linguistics, 28, 147–164. https://doi.org/10.1111/ijal.12196.CrossRefGoogle Scholar
Byrnes, H. (2009). Emergent L2 German writing ability in a curricular context: A longitudinal study of grammatical metaphor. Linguistics and Education, 20, 50–66.CrossRefGoogle Scholar
Chen, D., & Manning, C. D. (2014). A fast and accurate dependency parser using . neural networks. , 740–750. https://cs.stanford.edu/~danqi/papers/emnlp2014.pdf.Google Scholar
Crossley, S. A., Kyle, K., & Salsbury, T. (2016). A usage-based investigation of L2 lexical acquisition: The role of input and output. The Modern Language Journal, 100, 702–715.CrossRefGoogle Scholar
Crossley, S. A., & McNamara, D. S. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66–79. https://doi.org/10.1016/j.jslw.2014.09.006.CrossRefGoogle Scholar
Crossley, S. A., Salsbury, T., & McNamara, D. (2010). The development of polysemy and frequency use in English second language speakers. Language Learning, 60, 573–605. https://doi.org/10.1111/j.1467-9922.2010.00568.x.CrossRefGoogle Scholar
Crymes, R. (1971). The relation of study about language to language performance: With special reference to nominalization. TESOL Quarterly, 5, 217–230.CrossRefGoogle Scholar
Cumming, A. H., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10, 5–43.CrossRefGoogle Scholar
Davies, M. (2009). The 385+ million word Corpus of Contemporary American English (1990–2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14, 159–190. https://doi.org/10.1075/ijcl.14.2.02dav.CrossRefGoogle Scholar
de Bot, K., Lowie, V., Thorne, S., & Verspoor, M. (2013). Dynamic systems theory as a comprehensive theory of second language development. In Mayo, M. P. G., Mangado, M. J. G., & Adrián, M. M. (Eds.), Contemporary approaches to second language acquisition (pp. 199–220). John Benjamins Publishing.Google Scholar
Durrant, P., & Schmitt, N. (2009). To what extent do native and non-native writers make use of collocations? IRAL-International Review of Applied Linguistics in Language Teaching, 47, 157–177.CrossRefGoogle Scholar
Ellis, N. C. (2002). Frequency effects in language processing. Studies in Second Language Acquisition, 24, 143–188.CrossRefGoogle Scholar
Ellis, N. C., & Ferreira-Junior, F. (2009). Constructions and their acquisition: Islands and the distinctiveness of their occupancy. Annual Review of Cognitive Linguistics, 7, 188–221. https://doi.org/10.1075/arcl.7.08ell.CrossRefGoogle Scholar
Eskildsen, S. W. (2009). Constructing another language—Usage-based linguistics in second language acquisition. Applied Linguistics, 30, 335–357. https://doi.org/10.1093/applin/amn037.CrossRefGoogle Scholar
Fisher, R. A. (1934). Two new properties of mathematical likelihood. Proceedings of the Royal Society of London. Series A, Containing Papers of a Mathematical and Physical Character, 144, 285–307.Google Scholar
Garner, J., Crossley, S., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80, 176–187. https://doi.org/10.1016/j.system.2018.12.001.CrossRefGoogle Scholar
Goldberg, A. E. (2006). Constructions at work: The nature of generalization in language. Oxford University Press.Google Scholar
Gries, S. T. (2015). The most under-used statistical method in corpus linguistics: Multi-level (and mixed-effects) models. Corpora, 10, 95–125.CrossRefGoogle Scholar
Gries, S. T., Hampe, B., & Schönefeld, D. (2005). Converging evidence: Bringing together experimental and corpus data on the association of verbs and constructions. Cognitive Linguistics, 16, 635–676. https://doi.org/10.1515/cogl.2005.16.4.635.CrossRefGoogle Scholar
Grömping, U. (2006). Relative importance for linear regression in R: The package relaimpo. Journal of Statistical Software, 17, 1–27.CrossRefGoogle Scholar
Hunt, K. W. (1965). Grammatical structures written at three grade levels . NCTE Research Report No., 3. http://eric.ed.gov/?id=ED113735.Google Scholar
Kenward, M. G., & Roger, J. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983–997.CrossRefGoogle ScholarPubMed
Kim, S., & Kim, M. S. (2015). Package “ppcor.” Communications for Statistical Applications and Methods, 22, 665–674.CrossRefGoogle Scholar
Knoch, U., Rouhshad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing, 21, 1–17. https://doi.org/10.1016/j.asw.2014.01.001.CrossRefGoogle Scholar
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2015). lmerTest: Tests in linear mixed effects models. R package version 2.0–20.https://Cran.Rproject.Org/Web/Packages/LmerTestGoogle Scholar
Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication. Georgia State University. http://scholarworks.gsu.edu/alesl_diss/35/.Google Scholar
Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49, 757–786. https://doi.org/10.1002/tesq.194.CrossRefGoogle Scholar
Kyle, K., & Crossley, S. A. (2017). Assessing syntactic sophistication in L2 writing: A usage-based approach. Language Testing, 34, 513–535.CrossRefGoogle Scholar
Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. The Modern Language Journal, 102, 333–349. https://doi.org/10.1111/modl.12468.CrossRefGoogle Scholar
Larsen-Freeman, D. (1978). An ESL index of development. TESOL Quarterly, 12, 439–448. https://doi.org/10.2307/3586142.CrossRefGoogle Scholar
Larsen-Freeman, D., & Cameron, L. (2008). Research methodology on language development from a complex systems perspective. The Modern Language Journal, 92, 200–213.CrossRefGoogle Scholar
Larsen-Freeman, D., & Strom, V. (1977). The construction of a second language acquisition index of development. Language Learning, 27, 123–134.CrossRefGoogle Scholar
Laufer, B., & Nation, I. S. P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307–322. https://doi.org/10.1093/applin/16.3.307.CrossRefGoogle Scholar
Lindeman, R. H., Merenda, P., & Gold, R. Z. (1980). Introduction to bivariate and multivariate analysis (p. 119). Foresman and Company.Google Scholar
Lowie, W. M., & Verspoor, M. H. (2019). Individual differences and the ergodicity problem. Language Learning, 69, 184–206.CrossRefGoogle Scholar
Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15, 474–496. https://doi.org/10.1075/ijcl.15.4.02luCrossRefGoogle Scholar
Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45, 36–62.CrossRefGoogle Scholar
Meurers, D., & Dickinson, M. (2017). Evidence and interpretation in language learning research: Opportunities for collaboration with computational linguistics. Language Learning, 67, 66–95.CrossRefGoogle Scholar
Murakami, A., & Alexopoulou, T. (2016). Longitudinal L2 development of the English article in individual learners. In Papafragou, D., Grodner, D., Mirman, D., & Trueswell, J. (Eds.), Proceedings of the 38th Annual Meeting of the Cognitive Science Society (pp. 1050–1055). Cognitive Science Society.Google Scholar
Ninio, A. (1999). Pathbreaking verbs in syntactic development and the question of prototypical transitivity. Journal of Child Language, 26, 619–653.CrossRefGoogle ScholarPubMed
Norris, J. M., & Ortega, L. (2009). Towards an organic approach to investigating CAF in instructed SLA: The case of complexity. Applied Linguistics, 30. https://doi.org/10.1093/applin/amp044.CrossRefGoogle Scholar
O’Donnell, M. B., & Ellis, N. (2010). Towards an inventory of English verb argument constructions (pp. 9–16). http://dl.acm.org/citation.cfm?id=1866732.1866734.Google Scholar
O’Grady, W. (2008). Innateness, universal grammar, and emergentism. Lingua, 118, 620–631.CrossRefGoogle Scholar
Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24, 492–518.CrossRefGoogle Scholar
Penris, W., & Verspoor, M. (2017). Academic writing development: A complex, dynamic process. In Pfenniger, S. & Navracsics, (Eds.), Future Research Directions for Applied Linguistics (Vol. 109, pp. 215–242). Multilingual Matters Ltd.CrossRefGoogle Scholar
R Core Team. (2016). A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/.Google Scholar
Römer, U., O’Donnell, M. B., & Ellis, N. C. (2015). Using COBUILD grammar patterns for a large-scale analysis of verb-argument constructions. Groom, N., Charles, M., & Suganthi, J. (Eds.), (Vol. 73, p. 43). John Benjamins Publishing Company.Google Scholar
Römer, U., Roberson, A., O’Donnell, M. B., & Ellis, N. C. (2014). Linking learner corpus and experimental data in studying second language learners’ knowledge of verb-argument constructions. ICAME Journal, 38, 115–135.CrossRefGoogle Scholar
Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309–316.CrossRefGoogle Scholar
Thornhill, D. E. (1969). A quantitative analysis of the development of syntactical fluency of four young adult Spanish speakers learning English [Unpublished doctoral dissertation]. The Florida State University.Google Scholar
Tomasello, M. (2003). Constructing a language: A usage-based theory of language acquisition. Harvard University Press.Google Scholar
Verspoor, M. (2017). Complex Dynamic Systems Theory and L2 pedagogy. In Ortega, L. & Han, Z. (Eds.), Complexity Theory and Language Development: In celebration of Diane Larsen-Freeman (Vol. 48, pp. 143–162). John Benjamins.CrossRefGoogle Scholar
Verspoor, M., & Behrens, H. (2011). Dynamic systems theory and a usage-based approach to second language development. A dynamic approach to second language development: Methods and techniques (pp. 25–38).CrossRefGoogle Scholar
Verspoor, M., de Bot, K., & Xu, X. (2015). The effects of English bilingual education in the Netherlands. Journal of Immersion and Content-Based Language Education, 3, 4–27.CrossRefGoogle Scholar
Verspoor, M., Schmid, M. S., & Xu, X. (2012). A dynamic usage based perspective on L2 writing. Journal of Second Language Writing, 21, 239–263. https://doi.org/10.1016/j.jslw.2012.03.007.CrossRefGoogle Scholar
Verspoor, M., & Smiskova, H. (2012). Foreign language writing development from a dynamic usage-based perspective. In Manchón, R. M. (Ed.), L2 writing development: Multiple perspectives (pp. 17–46). De Gruyter.Google Scholar
Vyatkina, N. (2012). The development of second language writing complexity in groups and individuals: A longitudinal learner corpus study. The Modern Language Journal, 96, 576–598.CrossRefGoogle Scholar
Wolfe-Quintero, K., Inagaki, S., & Kim, H.-Y. (1998). Second language development in writing: Measures of fluency, accuracy & complexity. University of Hawaii Press.Google Scholar
Yates, F. (1934). Contingency tables involving small numbers and the χ2 test. Supplement to the Journal of the Royal Statistical Society, 1, 217–235. https://doi.org/10.2307/2983604.CrossRefGoogle Scholar