Course Schedule
Detailed Daily Schedule and Session Topics
Warning🚧 Being prepared
Overview
This intensive 5-day course covers LLM-based linguistic data analysis — annotation and gold-standard construction, prompt design, and evaluation (precision/recall/F1, confusion matrices) — through lectures, tutorials, and hands-on practice, culminating in a group mini-project presentation.
Day 1 · Introduction & First Experience — Aug 3 (Mon)
| # | Session | What we’ll do |
|---|---|---|
| 1 | Introduction to LLMs & NLP Tasks | What LLMs are and how they fit linguistic analysis; the NLP tasks we’ll tackle; course overview and self-introductions. |
| 2 | First Experience with LLM Classification | A quick chat-interface demo, then Google Colab onboarding — sign in, open the notebook, run a cell, edit the prompt, re-run — plus a short “reading Python” orientation. |
| 3 | Classical NLP Tasks | Text preprocessing, tokenization, and dictionary look-up using basic Python. |
📖 Reading (before Day 1) — Skim: Abdurahman et al. (2025). See Readings →
Day 2 · Annotation, Gold Standards & Metrics — Aug 4 (Tue)
| # | Session | What we’ll do |
|---|---|---|
| 4 | Annotation Principles & Inter-Annotator Agreement | Annotation principles and the NLP pipeline for applied linguistics; an introduction to gold-standard dataset construction. |
| 5 | Hands-on: Gold-Standard Annotation & Agreement | Re-annotate a sample using a prepared scheme; compute inter-annotator agreement; compare against the published gold and against LLM annotations. |
| 6 | Evaluation Metrics | Precision, recall, F1, and the confusion matrix, with hands-on practice in Colab. |
📖 Reading (before Day 2) — Read: Eguchi & Kyle (2024); optional further reading listed. See Readings →
Day 3 · Prompt Design & Iteration — Aug 5 (Wed)
| # | Session | What we’ll do |
|---|---|---|
| 7 | Prompt Design: Zero-shot vs Few-shot | Prompt-design principles and strategies for effective prompt engineering. |
| 8 | Hands-on: LLM Classification & Prompt Iteration | Run LLM-based text classification in Colab through the provided notebook, and evaluate the outputs with the prepared tools. |
| 9 | Iterative Prompt Improvement & Error Analysis | Iterate the prompt over 2–3 cycles with error analysis, plus a short under-the-hood walkthrough. |
📖 Reading (before Day 3) — Read: Huang & Mizumoto (2025); Kim & Lu (2024). See Readings →
Day 4 · Methodology & Pipeline Assembly — Aug 6 (Thu)
| # | Session | What we’ll do |
|---|---|---|
| 10 | Methodology: Reproducibility, LLM Limits & Ethics | Reproducibility, LLM limitations (hallucination, data contamination), and ethical issues in LLM-based research. |
| 11 | Plenary Pipeline Assembly | Assemble the project notebook together, step by step; then pick a mini-project track and sample a balanced gold subset from a provided pool. |
| 12 | Project Work: QC the Gold Set & Baseline | A quality-control (adjudication) pass on your sampled gold set, then run a baseline prompt. |
📖 Reading (before Day 4) — Read (in full): Abdurahman et al. (2025). See Readings →
Day 5 · Project Finalization & Presentations — Aug 7 (Fri)
| # | Session | What we’ll do |
|---|---|---|
| 13 | Project Work: Prompt Iteration & Final Evaluation | Iterate your prompt (2–3 cycles), run the final evaluation, and begin the in-class one-page report. |
| 14 | Project Work: Finalize Report & Notebook | Finalize the one-page report, prepare your presentation, and submit the completed notebook. |
| 15 | Final Presentations & Wrap-up | Group presentations with instructor Q&A and a course wrap-up discussion. |
No new reading for Day 5 — project work only.
Important Notes
- All times are Japan Standard Time (JST)
- Bring your laptop to all sessions
- Complete the readings before each day