Session 13: Project Work — Prompt Iteration & Final Evaluation
Day 5 · Project Finalization & Presentations (5-1)
Warning🚧 Being prepared
Learning objectives
By the end of this session you will be able to:
- Change one thing per prompt round, predict its effect, and test it against your gold set.
- Read a confusion matrix to decide what to change next, rather than guessing.
- Freeze a final run to a file, and explain why a seed alone is not enough.
- Report plain and weighted κ, and say which one your label set justifies.
Agenda
- Round 1 and 2 — few-shot, then a change of your own. For each: what did you change, and why did you expect it to help?
- Read the errors between rounds — the confusion matrix tells you which pair to attack.
- The final run — raise
N_PER_CLASS, run once,save_predictions. - Everything after that reads the frozen file, not the model.
Budget your quota: the free tier gives about 500 calls a day per key. Iterate small.
Reading
No new reading — Day 5 is project work only (see the Readings page).
Slides & Colab
Mini-project
04_develop.ipynb — the prompt rounds, on the dev half — then 05_test.ipynb for the held-out run, once.
ImportantThe final run is one run, by one person
Runtimes are per-person, and your reported numbers have to come out of a single frozen file. Decide who the driver is before you start. If their daily quota runs out, hand the role — and the runtime — to someone else; the files stay put in your shared Drive folder.