Session 6: Gold Standards & Evaluation Metrics

Day 2 · Annotation, Gold Standards & Metrics (2-3)

🚧 Being prepared

This page is being finalized ahead of the course (Aug 3–7, 2026) and may be incomplete or change before your session. The syllabus and readings are ready now.

Learning objectives

Reading

Abdurahman, S., Salkhordeh Ziabari, A., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. Advances in Methods and Practices in Psychological Science, 8(2). https://doi.org/10.1177/25152459251325174

Slides & Colab

Tutorial notes: Day 2 — Annotation, gold standards & evaluation
Notebook (tutorial + Corpus Lab): day2_gold_standards_and_evaluation.ipynb —

---
title: "Session 6: Gold Standards & Evaluation Metrics"
subtitle: "Day 2 · Annotation, Gold Standards & Metrics (2-3)"
toc: true
---

<!-- BEING-PREPARED-BANNER -->
::: {.callout-warning appearance="simple"}
## 🚧 Being prepared
This page is being finalized ahead of the course (**Aug 3–7, 2026**) and may be incomplete or change before your session. The [syllabus](/syllabus/index.md) and [readings](/syllabus/readings.md) are ready now.
:::


## Learning objectives
<!-- to be added -->

## Reading

- Abdurahman, S., Salkhordeh Ziabari, A., Moore, A. K., Bartels, D. M., & Dehghani, M. (2025). A primer for evaluating large language models in social-science research. *Advances in Methods and Practices in Psychological Science, 8*(2). <https://doi.org/10.1177/25152459251325174>

## Slides & Colab

- Tutorial notes: [Day 2 — Annotation, gold standards & evaluation](../../resources/tutorials/tutorial-day2-annotation-eval.md)
- Notebook (tutorial + Corpus Lab): [`day2_gold_standards_and_evaluation.ipynb`](../../resources/tutorials/day2_gold_standards_and_evaluation.ipynb) —
  [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/egumasa/linguistic-data-analysis-II-2026/blob/main/sources/resources/tutorials/day2_gold_standards_and_evaluation.ipynb)
<!-- slides link: to be added -->