Course Syllabus
Linguistic Data Analysis II
Course Information
| Course Title | Linguistic Data Analysis II |
| Credits | 2 |
| Format | Intensive 5-day course (15 sessions) |
| Language | English |
| Classroom | To be announced |
Instructor Information
Instructor: Masaki Eguchi, Ph.D.
Course Description
Large Language Models (LLMs) β generative AI technologies that produce human-like language β have become one of the common tools people rely on. Linguistic research is no exception: more and more researchers are exploring the potential of this technology to support various stages of their work. One promising application of LLMs is linguistic annotation, a task that can be highly resource-intensive. If the annotation process can be supported by LLMs, the research community can benefit from greater scalability. Lowering the barriers to linguistic annotation is especially important for graduate research, where individual student researchers often have limited resources. Yet, while LLMs hold promise for linguistic research, the adequacy of model outputs must be critically evaluated rather than assumed. We must therefore be equipped with approaches to critically appraise what they can and cannot offer in order to make meaningful contributions to the field. In this course, we cover the foundational concepts and methods for evaluating LLM outputs in linguistic research projects.
Learning Objectives
By the end of this course, students will be able to:
| Area | Objective |
|---|---|
| Critical appraisal | Explain the strengths and limitations of Large Language Models for linguistic analysis, and judge when an LLM-based approach is appropriate. |
| Annotation scheme design | Design an annotation scheme and coding guidelines for a linguistic construct of interest. |
| Gold-standard datasets | Construct a gold-standard dataset, including operationalizing categories and assessing inter-annotator agreement. |
| Prompt design | Design, tune, and document prompts that elicit reliable linguistic annotations from an LLM. |
| Evaluation | Evaluate model performance using precision, recall, F1, and confusion matrices, and interpret the results critically. |
| Reproducibility | Communicate methods and findings transparently and reproducibly, following responsible research practices. |
Course Components
- π Detailed Schedule
- π Readings
Required Materials
Textbook
Other required/Optional readings are provided through Google Classroom.
Softwares (Free)
Others
- Google Colaboratory: Follow the instruction here to enable the tool.
- Visual Studio Code / Positron (Optional): If you would like to learn how to run the code on your computer, please follow instruction at [page].
Assignments and Grading
Grade Distribution (subject to final review by the start of the course on 3rd, Aug)
| Component | Percent | |
|---|---|---|
| Attendance / Participation | 20% | |
| Hands-on activities (Day 1β3) + completed notebook | 40% | |
| Mini-project group presentation + Q&A | (Day 5) | 20% |
| Mini write-up (individual) | (Day 5) | 20% |
The completed project notebook β assembled from the cell library and run end-to-end, with your groupβs sampled gold subset, prompt, and evaluation outputs β is submitted as evidence that the in-class hands-on work was done, and is graded under the hands-on component. All project deliverables are produced and submitted during the course; there is no post-course write-up.
Grading Scale
We follow the grading system at Tohoku University.
| Grade | Range | Grade Point |
|---|---|---|
| AA | 100-90% | 4.0 |
| A | 89-80% | 3.0 |
| B | 79-70% | 2.0 |
| C | 69-60% | 1.0 |
| D | 59-0% | 0.0 |
Daily Structure
Each day follows this general pattern:
| Time | Activity |
|---|---|
| 10:30-12:00 | Session 1 |
| 12:00-13:00 | Lunch break |
| 13:00-14:30 | Session 2 |
| 14:30-14:40 | Break |
| 14:40-16:10 | Session 3 |
| 16:15-17:00 | Optional sessions (on Open Science) |
Attendance Policy
- Due to the intensive nature of the course, attendance and participation are crucial to your success in this course.
- However, in case of emergency, do not hesitate to reach out to the instructor for possible accomodation. I may be able to accommodate depending on the situation.
Assignment Submission
Deadlines
- All assignments are due at 10:30 AM on the specified day
- Late submissions will receive a 10% penalty per day
- Extensions may be granted for documented emergencies.
Submission Format
- Submit all assignments via the course management system (Google Classroom)
- Use the provided templates when available
- File naming convention:
LastName_Assignment#.ext - Acceptable formats:
.docx,.pdf,.ipynb(for Python notebooks)
Collaboration
- As this is a very intensive course, collaboration is encouraged to gain most out of the time we spend. I will help each of you during the class time and office hours, but I encourage you to also help each other in:
- setting up the tools
- recalling class materials
- thinking about the approaches to corpus lab
- However, you MUST write your own write-up of the assignments, meaning that you MUST make outlines, draft, and finalize the written submission by yourselves.
Plagiarism
Policy to be determined.
Technology Policy
Required Technology
- Bring a laptop to every session.
- Ensure all required software is installed.
Classroom Etiquette
- Laptops should be used for course activities only
Communication
- Course related communications will happen via Google Classroom.
Course Announcements
- Course announcements are made through Google Classroom.
Materials Sharing
- Materials (e.g., slides) are shared through this website.
Accommodations
If you have a disability or any other circumstance that may affect your participation, please contact the instructor as early as possible β ideally before the course begins β so that reasonable accommodations can be arranged. Requests are handled confidentially. Students may also consult Tohoku Universityβs Accessibility Support Room / Student Support Division for additional support.