ICNALE GRA — download & preprocess

Automated writing evaluation: holistic essay score bands

What it is. Asian-learner English essays rated holistically by many trained raters. We turn the average rating into a Low/Mid/High band.

Difficulty of the labeling judgment: ★★☆ — moderate. Rubric-based, but raters disagree at the margins.

License: research use — password-gated (registration required)
Cite: Ishikawa, S. The ICNALE Global Rating Archives.

---

Every dataset in this course is reshaped into the same canonical schema so one notebook works for all of them:

[{"id": 1, "text": "...", "label": "..."}]

The raw data, though, looks different every time. That difference is the lesson — half of building a gold standard is getting messy real data into a clean, consistent shape.

Step 1 — Get access (this one is gated)

Unlike the other datasets, ICNALE GRA is not a direct download. You must:

1. Register on the ICNALE download page to receive a password.
2. Download and unzip ICNALE_GRA_2.x.zip.
3. From it, make a small spreadsheet with two columns — text (the essay) and score (its average holistic rating) — and save it as essays_scores.csv.

Then upload that CSV here:

from google.colab import files   # (Colab only)
uploaded = files.upload()        # choose your essays_scores.csv

Step 2 — Look at the raw format

Your CSV should have a text column and a numeric score column.

import csv
with open("essays_scores.csv", encoding="utf-8-sig", newline="") as f:
    reader = csv.DictReader(f)
    print("columns:", reader.fieldnames)
    for i, row in enumerate(reader):
        print(row)
        if i == 2:
            break

Step 3 — Reshape into the canonical schema

Decision: turn the continuous score into three bands. Adjust the cut-offs to match the rubric you downloaded.

def band(score):
    s = float(score)
    if s < 4:
        return "Low"
    elif s < 7:
        return "Mid"
    return "High"

rows = []
with open("essays_scores.csv", encoding="utf-8-sig", newline="") as f:
    for row in csv.DictReader(f):
        text = (row.get("text") or "").strip()
        score = (row.get("score") or "").strip()
        if text and score:
            rows.append({"text": text, "label": band(score)})

print("essays:", len(rows))

Step 4 — Inspect the labels

from collections import Counter
counts = Counter(item["label"] for item in rows)
print("total items:", len(rows))
print("label counts:", dict(counts))
rows[:3]  # peek at the first three reshaped items

Step 5 — Build a balanced gold set

# Build a small BALANCED gold set: an equal number of items per label.
# Balance matters so precision/recall/F1 and the confusion matrix are meaningful.
import random
from collections import defaultdict

PER_LABEL = 20          # how many items per label
random.seed(42)         # fixed seed = same sample every run (reproducible)

by_label = defaultdict(list)
for item in rows:
    by_label[item["label"]].append(item)

gold = []
for label in sorted(by_label):
    bucket = by_label[label]
    random.shuffle(bucket)
    gold.extend(bucket[:PER_LABEL])

random.shuffle(gold)
gold = [{"id": i + 1, "text": x["text"], "label": x["label"]} for i, x in enumerate(gold)]

from collections import Counter
print("items:", len(gold), "| per label:", dict(Counter(x["label"] for x in gold)))

Step 6 — Save it

OUT_FILE = "icnale_gra_scores.json"

import json
with open(OUT_FILE, "w", encoding="utf-8") as f:
    json.dump(gold, f, ensure_ascii=False, indent=2)
print(f"Saved {len(gold)} items to {OUT_FILE}")
gold[:3]  # preview the first three items

---

You built icnale_gra_scores.json for an automated-writing-evaluation mini-project.