Readings

Linguistic Data Analysis II

Readings are assigned by day and should be completed before the corresponding day of the intensive course. Each day lists a small required reading (marked Skim or Read); optional items are provided for those who want to go deeper.

Access

PDFs for readings not freely available online are posted on Google Classroom. Where a work is open access, a DOI link is provided below.

Day 1 · Introduction & First Experience

Skim

Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. Advances in Methods and Practices in Psychological Science, 8(2). https://doi.org/10.1177/25152459251325174
- Get a feel for the questions the course tackles; don’t worry about the details yet.

Day 2 · Annotation, Gold Standards & Metrics

Read

Eguchi, M., & Kyle, K. (2024). Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial. Research Methods in Applied Linguistics, 3(3), 100153. https://doi.org/10.1016/j.rmal.2024.100153

Optional / further reading

Kyle, K., & Eguchi, M. (2024). Evaluating NLP models with written and spoken L2 samples. Research Methods in Applied Linguistics, 3(2), 100120. https://doi.org/10.1016/j.rmal.2024.100120
Mizumoto, A. (2025). Automated analysis of common errors in L2 learner production: Prototype web application development. Studies in Second Language Acquisition, 1–18. https://doi.org/10.1017/S0272263125100934
Yamashita, T. (2024). An application of many-facet Rasch measurement to evaluate automated essay scoring: A case of ChatGPT-4.0. Research Methods in Applied Linguistics, 3(3), 100133. https://doi.org/10.1016/j.rmal.2024.100133

Day 3 · Prompt Design & Iteration

Read

Huang, J., & Mizumoto, A. (2025). Prompt engineering: Enhancing AI-driven language learning and feedback. In L. McCallum & D. Tafazoli (Eds.), The Palgrave Encyclopedia of Computer-Assisted Language Learning (pp. 1–8). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-51447-0_103-1
Kim, M., & Lu, X. (2024). Exploring the potential of using ChatGPT for rhetorical move-step analysis: The impact of prompt refinement, few-shot learning, and fine-tuning. Journal of English for Academic Purposes, 71, 101422. https://doi.org/10.1016/j.jeap.2024.101422

Day 4 · Methodology & Pipeline Assembly

Read (in full) — revisit the Day 1 primer, this time closely, as the checklist behind reproducible, transparent LLM-based reporting; Ready for full discussion of the contents and implication to applied linguistics research.

Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. Advances in Methods and Practices in Psychological Science, 8(2). https://doi.org/10.1177/25152459251325174

Day 5 · Project Finalization & Presentations

No new reading — project work only.

--- title: "Readings" subtitle: "Linguistic Data Analysis II" toc: true toc-depth: 2 --- Readings are assigned by day and should be completed **before** the corresponding day of the intensive course. Each day lists a small **required** reading (marked *Skim* or *Read*); **optional** items are provided for those who want to go deeper. ::: {.callout-note} ## Access PDFs for readings not freely available online are posted on **Google Classroom**. Where a work is open access, a DOI link is provided below. ::: ## Day 1 · Introduction & First Experience **Skim** - Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. *Advances in Methods and Practices in Psychological Science, 8*(2). [https://doi.org/10.1177/25152459251325174](https://doi.org/10.1177/25152459251325174) - Get a feel for the questions the course tackles; don't worry about the details yet. ## Day 2 · Annotation, Gold Standards & Metrics **Read** - Eguchi, M., & Kyle, K. (2024). Building custom NLP tools to annotate discourse-functional features for second language writing research: A tutorial. *Research Methods in Applied Linguistics, 3*(3), 100153. [https://doi.org/10.1016/j.rmal.2024.100153](https://doi.org/10.1016/j.rmal.2024.100153) **Optional / further reading** - Kyle, K., & Eguchi, M. (2024). Evaluating NLP models with written and spoken L2 samples. *Research Methods in Applied Linguistics, 3*(2), 100120. [https://doi.org/10.1016/j.rmal.2024.100120](https://doi.org/10.1016/j.rmal.2024.100120) - Mizumoto, A. (2025). Automated analysis of common errors in L2 learner production: Prototype web application development. *Studies in Second Language Acquisition*, 1–18. [https://doi.org/10.1017/S0272263125100934](https://doi.org/10.1017/S0272263125100934) - Yamashita, T. (2024). An application of many-facet Rasch measurement to evaluate automated essay scoring: A case of ChatGPT-4.0. *Research Methods in Applied Linguistics, 3*(3), 100133. [https://doi.org/10.1016/j.rmal.2024.100133](https://doi.org/10.1016/j.rmal.2024.100133) ## Day 3 · Prompt Design & Iteration **Read** - Huang, J., & Mizumoto, A. (2025). Prompt engineering: Enhancing AI-driven language learning and feedback. In L. McCallum & D. Tafazoli (Eds.), *The Palgrave Encyclopedia of Computer-Assisted Language Learning* (pp. 1–8). Springer Nature Switzerland. [https://doi.org/10.1007/978-3-031-51447-0_103-1](https://doi.org/10.1007/978-3-031-51447-0_103-1) - Kim, M., & Lu, X. (2024). Exploring the potential of using ChatGPT for rhetorical move-step analysis: The impact of prompt refinement, few-shot learning, and fine-tuning. *Journal of English for Academic Purposes, 71*, 101422. [https://doi.org/10.1016/j.jeap.2024.101422](https://doi.org/10.1016/j.jeap.2024.101422) ## Day 4 · Methodology & Pipeline Assembly **Read (in full)** — revisit the Day 1 primer, this time closely, as the checklist behind reproducible, transparent LLM-based reporting; Ready for full discussion of the contents and implication to applied linguistics research. - Abdurahman, S., Ziabari, A. S., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. *Advances in Methods and Practices in Psychological Science, 8*(2). [https://doi.org/10.1177/25152459251325174](https://doi.org/10.1177/25152459251325174) ## Day 5 · Project Finalization & Presentations No new reading — project work only.