CaRS-50 — download & preprocess

The Kim & Lu replication: CARS move-steps in article introductions

What it is. Sentences from 50 BioRxiv article introductions, each labeled with a Swales CARS Move (1–3) and Step (a–d) — the same scheme Kim & Lu (2024) used.

Difficulty of the labeling judgment: ★★★ — hard. The dataset’s own expert agreement is only κ ≈ 0.43.

License: CC BY 4.0
Cite: Lam & Nnamoko (2025), Mendeley Data, doi:10.17632/kwr9s5c4nk.1

---

Every dataset in this course is reshaped into the same canonical schema so one notebook works for all of them:

[{"id": 1, "text": "...", "label": "..."}]

The raw data, though, looks different every time. That difference is the lesson — half of building a gold standard is getting messy real data into a clean, consistent shape.

Step 1 — Download the raw data

This dataset is on Mendeley Data as 50 separate XML files. There is no single zip, so we ask Mendeley’s public API for the list of files and download each one. (You don’t need to understand every line — read it as: get the file list, then loop and download.)

import urllib.request, json, os, time

# Mendeley needs a normal browser User-Agent, so we add one to each request.
def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
    return urllib.request.urlopen(req, timeout=60)

os.makedirs("cars50", exist_ok=True)
meta = json.load(fetch("https://data.mendeley.com/public-api/datasets/kwr9s5c4nk"))

for f in meta["files"]:
    url = f["content_details"]["download_url"]
    dest = os.path.join("cars50", f["filename"])
    for attempt in range(3):          # retry on the occasional dropped connection
        try:
            with fetch(url) as resp, open(dest, "wb") as out:
                out.write(resp.read())
            break
        except Exception:
            time.sleep(2)

print("downloaded", len(os.listdir("cars50")), "XML files")

Step 2 — Look at the raw format

Each file is XML. Inside, every sentence is a <sentence> element holding a <text> and a <step> code like 1b (Move 1, Step b). Let’s print the start of one file.

with open("cars50/text001.xml", encoding="utf-8") as f:
    print(f.read()[:900])

Step 3 — Reshape into the canonical schema

We parse the XML and pull out each sentence’s text and code. Decision: for a first pass we use just the Move (the leading digit of the code) as the label — 3 classes instead of 11. (The full Move+Step is a stretch goal.)

import glob
import xml.etree.ElementTree as ET

rows = []
for path in glob.glob("cars50/*.xml"):
    tree = ET.parse(path)
    for sentence in tree.iter("sentence"):
        text_el = sentence.find("text")
        step_el = sentence.find("step")
        if text_el is None or step_el is None:
            continue
        text = (text_el.text or "").strip()
        code = (step_el.text or "").strip()
        if text and code and code[0].isdigit():
            rows.append({"text": text, "label": f"Move {code[0]}"})

print("sentences:", len(rows))

Step 4 — Inspect the labels

from collections import Counter
counts = Counter(item["label"] for item in rows)
print("total items:", len(rows))
print("label counts:", dict(counts))
rows[:3]  # peek at the first three reshaped items

Step 5 — Build a balanced gold set

Three moves × 20 = 60 items.

# Build a small BALANCED gold set: an equal number of items per label.
# Balance matters so precision/recall/F1 and the confusion matrix are meaningful.
import random
from collections import defaultdict

PER_LABEL = 20          # how many items per label
random.seed(42)         # fixed seed = same sample every run (reproducible)

by_label = defaultdict(list)
for item in rows:
    by_label[item["label"]].append(item)

gold = []
for label in sorted(by_label):
    bucket = by_label[label]
    random.shuffle(bucket)
    gold.extend(bucket[:PER_LABEL])

random.shuffle(gold)
gold = [{"id": i + 1, "text": x["text"], "label": x["label"]} for i, x in enumerate(gold)]

from collections import Counter
print("items:", len(gold), "| per label:", dict(Counter(x["label"] for x in gold)))

Step 6 — Save it

OUT_FILE = "cars50_moves.json"

import json
with open(OUT_FILE, "w", encoding="utf-8") as f:
    json.dump(gold, f, ensure_ascii=False, indent=2)
print(f"Saved {len(gold)} items to {OUT_FILE}")
gold[:3]  # preview the first three items

---

You built cars50_moves.json — the open stand-in for Kim & Lu’s data. Use it in the Day 3 tutorial.

Stretch: change the label to the full code (code instead of f"Move {code[0]}") for the 11-class Move+Step task and watch accuracy drop.