Day 1 · Your first LLM call & reading its data

Day 1 — Linguistic Data Analysis II

Welcome to Colab notebook

This page is a Colab notebook. Notebook is a interactive Python enviroment where you can run your own code and see the output immediately.

How to use this notebook

This is your single submission for the day.

It has two parts:

Session 2 — call a language model, then learn to read the data it hands back.
Session 3 — segment text into sentences, then write the Python you will reuse every day this week.

You only edit the cells marked ✏️ YOU EDIT. Run the 🔧 Library cells and leave them alone.

➡️ Work top to bottom. When you’re done, Runtime → Run all, then File → Download → Download .ipynb and submit that file.

Session 2 — your first LLM call, and reading its answer

We will cover Basic Python and first LLM call.

Work through eight short steps. Each one shows a worked example you run, then a 🧪 your turn cell where you change something and re-run.

Step 1 · Run a cell

A code cell runs when you press Shift+Enter (or click ▶). The first run wakes up a runtime — a temporary computer in the cloud that remembers your variables until you close the tab.

Show code

print("Hello, Colab! You just ran your first cell.")

🧪 Your turn — change the text inside the quotes to a message of your own, then press Shift+Enter again.

Step 2 · Read an error

Sooner or later a cell turns red. Python tells you what went wrong on the last line. Run this cell — it is meant to fail:

Show code

print(mesage)      # a typo for `message` — this cell is supposed to fail

The last line reads:

NameError: name 'mesage' is not defined

Three parts: the error type (NameError), the message ('mesage' is not defined), and the line it happened on. Nearly every early error is a typo or a cell you haven’t run yet.

Step 3 · Variables — store a value under a name

A variable is a name that holds a value. The = means “store this to a computer memory”. It is not “equals”. Once stored, you get the value back by writing its name.

Show code

level = "A1"        # store the text "A1" under the name `level`
print(level)        # get it back by name

🧪 Your turn — store your own name under a variable called who, then print it.

Show code

who = "..."         # ✏️ your name here
                    # print who here

NOW try to assign multiple lines to a variable

Show code

# Assign

MULTILINE = None

Step 4 · Your first LLM call

Run the setup cell first. It does one thing: from google.colab import ai brings in Colab’s built-in Gemini — free, and nothing to set up.

Then call it. ai.generate_text(...) sends your text to the model and hands back its reply. You send text; you get text back. We store the answer in a variable called reply.

Show code

#@title 📦 Setup — run me first { display-mode: "form" }
# Helper — you don't need to read this. Run it and move on.
# Generated from _notebook_lib.py — edit there, not here; changes to this cell are replaced.

# --- LLM backend: Colab's free built-in Gemini (no API key) -------------------
from google.colab import ai      # Colab's built-in Gemini — nothing to set up

print("Setup done. Colab's built-in Gemini is ready.")

✏️ YOU EDIT — change the prompt text and re-run. The prompt is just text you send; the reply is just text you get back.

Show code

# ✏️ change the text in the quotes, then press Shift+Enter to re-run.
reply = ai.generate_text("In one sentence, what is applied linguistics?")
print(reply)                                    # show what came back

One call, one answer. That reply is not correct by definition — it is the result of generation you have to check. On Day 2 you will score answers like this against labels people agreed on by hand.

Before moving on, try the model on four more prompts. Each one asks it to do something you might actually want from it, and each shows something different about what you get back.

1. Ask it to do the actual task. This is the job you will spend a few days: give it a sentence, ask for a CEFR level.

Show code

reply = ai.generate_text("What CEFR level (A1-C2) is this sentence? The cat sat on the mat.")
print(reply)

You probably got a paragraph of explanation rather than a single level. The model answered the question; it just answered at a length nobody asked for.

2. Ask for the format you want. Adding one sentence to the prompt changes the shape of the reply.

Show code

reply = ai.generate_text("What CEFR level (A1-C2) is this sentence? Answer with just the level, nothing else. The cat sat on the mat.")
print(reply)

Shorter, and much easier to put in a table. The output format is something you ask for, not something you hope for. We will cover structured output on Day 3.

3. Run the same prompt twice. Nothing in the cell changes; run it, then run it again.

Show code

print(ai.generate_text("Name one common hedge in academic writing."))

Show code

print(ai.generate_text("Name one common hedge in academic writing."))

The two answers are probably not identical. This backend gives no way to hold the output still, so the same prompt can produce different text each time.

On Day 3 you switch to a backend that can be pinned down, because a result you cannot reproduce is a result you cannot report.

On Day 4, we will discuss how to deal with the undeterminacy of the LLM output to make sure that people can replicate your results.

4. Ask it something it will get wrong. The model answers questions about language, so ask it to count.

Show code

print(ai.generate_text("How many letter r's are in the word strawberry? Answer with just the number."))

The word has three. Whatever it told you, it told you in the same confident tone as every other answer above. Fluency is not accuracy, and nothing in the reply marks which is which.

That is the reason the rest of this course exists: you will build a set of human-labelled answers (Day 2), then measure the model against it (Day 2 and Day 3) instead of trusting how the reply sounds.

🧪 Your turn — try a prompt from your own research area, and one you expect the model to get wrong. Can you tell the two replies apart without already knowing the answer?

Show code

## Construct a prompt and run a LLM call here

Summary of Steps 1-4

Step 1: Run a cell

There are code cell and text cell
Shift+Enter or clicking the play button to run the gode.

Step 2: Read an error

When you get error, do not panic. Getting error is a good thing. Computer might fail without letting you know you have errors.

Step 3: Variables — store a value under a name

Learn that a variable is a name holding a value, using the = operator for assignment.

Step 4: Your first LLM call

In colab, you get free tier of Gemini from google.colab import ai.
Call the LLM with ai.generate_text("your prompt") to send text and receive a reply.
Understand that the model’s reply is a generation and needs to be checked for accuracy.
LLM can generate different answers each time.
Note that LLMs can confidently provide incorrect information, highlighting that processing speed/plausibility does not equal accuracy.

Step 5 · Putting your data into a prompt

You know how to call an LLM. Now we will further learn useful Python syntax to help our work.

An f-string (f"...") drops a variable straight into a piece of text with {curly braces}. That’s how you build a prompt about a specific sentence, and it is how you will run one prompt over a whole dataset later.

Show code

# WITHOUT f-strings

# Prompt 1
PROMPT1 = """What CEFR level (A1-C2) is this sentence?
Answer with just the level.

Sentence: The cat sat on the mat."""

print(PROMPT1)

# You need to repeat another
PROMPT2 = """What CEFR level (A1-C2) is this sentence?
Answer with just the level.

Sentence: More research is needed."""

print(PROMPT2)

Pattern 1: f-string — build one prompt around one sentence

You can drop a variable straight into the text as you write the prompt: change sentence, re-run, and the prompt changes with it.”

Show code

sentence = "Nevertheless, the findings were inconclusive."   # ✏️ your sentence
prompt = f"What CEFR level (A1-C2) is this sentence? {sentence}"  # f = fill in {}

print("Prompt sent:", prompt)                # see what {sentence} became

Show code

# Now send that prompt
reply = ai.generate_text(prompt)             # send the finished prompt

# See the result
print("Model says:", reply)

Pattern 2: template + `.format()` — write the prompt once, reuse it for any sentence

You can store the prompt with an empty {text} slot and fill it in later, so you can send the same wording for every sentence in a dataset

Show code

### Step 1: a template — no `f`, so {text} stays an empty slot ###
TEMPLATE = """What CEFR level (A1-C2) is this sentence?
Answer with just the level.

Sentence: {text}"""

Show code

### Step 2: fill the slot, once per sentence ###
print(TEMPLATE.format(text="The cat sat on the mat."))   # .format() fills {text}
print("---")
print(TEMPLATE.format(text=sentence))        # same template, different sentence

✏️ YOU EDIT — now write one of your own. Three lines:

YOUR_TEMPLATE — a prompt with {text} where the sentence should go. Use three quotes, and no f in front, so the braces stay empty.
prompt — fill the slot with .format(text=...).
ai.generate_text(prompt) — send it, and print what comes back.

Ask the model for anything you like about the sentence: its CEFR level, whether it hedges a claim, how formal it is. Print prompt before you send it — checking what the braces became is how you catch a template that did not fill in.

Show code

### Step 1: your template — {text} is the empty slot, and there is no `f` ###
YOUR_TEMPLATE = """..."""            # ✏️ your prompt, with {text} in it somewhere

### Step 2: fill the slot ###
my_sentence = "Nevertheless, the findings were inconclusive."   # ✏️ your sentence
prompt = YOUR_TEMPLATE.format(text=my_sentence)
print("Prompt sent:", prompt)        # check what {text} became before sending

### Step 3: send it to the model ###
reply = ai.generate_text(prompt)
print("Model says:", reply)

If the reply looks like it ignored your sentence, read the printed prompt first — a template with no {text} in it sends the same prompt every time, and .format() will not warn you.

Step 6 · The three shapes your data comes in

Python (or programming languages) defines type of data it can process.

Show code

response = ai.generate_text("Generate a random sentence.")
print(response)
print(type(response)) # get a type of data

The above should return <class 'str'>. This means that LLM returns a string.

Some data type we use

str — text, in quotes. One value: a sentence, a level, a reply.
list — several values in order, in square brackets.
dict — a labelled record, in curly braces: key: value pairs.

About `len()`

len() counts the lengths of the variable. Returns something different for each data type. Run the three cells below and watch what len() reports.

String

A str is text in quotes, "..." or '...'. len() counts its characters.

Show code

### easy ###
level = "A1"
print(level, "→", len(level), "characters")

Show code

### realistic ###
sentence = "Nevertheless, the findings were inconclusive."
print(len(sentence), "characters")        # spaces and the full stop count too

NOTE: You must use """ """ or ''' ''' (Triple quotes) to assign multi-line string.

List

A list is items in square brackets, separated by commas: [a, b, c]. The order is fixed, and len() counts how many items, however long each one is.

Show code

### easy ###
levels = ["A1", "B2"]
print(levels, "→", len(levels), "items")

Show code

### realistic ###
sentences = ["The cat sat on the mat.",
             "More research is needed.",
             "Nevertheless, the findings were inconclusive."]
print(sentences)
print(len(sentences), "items")            # 3 items — not 3 + characters

Dictionary

A dict is key: value pairs in curly braces: {"key": value}. Each value is stored under a name you choose instead of a position, and len() counts how many pairs.

Show code

### easy ###
record = {"id": 1}                        # one pair: the key "id", the value 1
print(record, "→", len(record), "pair")

Show code

### realistic ###
counts = {"the": 12, "data": 3, "results": 5}   # how often each word appeared
print(len(counts), "pairs")

The three also nest: a list can hold dicts, and a value inside a dict can itself be a list. A dataset this week is a list of dicts, and you meet one in step 8.

The realistic examples are the shapes the week runs on: a sentence, a list of sentences, and a dict of counts you build on Day 2. Next, the shape that holds all of one sentence’s answers at once.

Step 7 · Put your judgment in a record

You have a sentence, and you have a level you would give it. Those are two separate values, and nothing ties them together. A dict ties them: one record, with a name on each value.

Where this is going: by the end of step 9 you will have measured how often the model agreed with you.

✏️ YOU EDIT — decide the level yourself before you look at what the model says.

Show code

# ✏️ A1 is the simplest level, C2 the most advanced. Change it if you disagree.
text = "The cat sat on the mat."
my_level = None # Assign any of the CEFR level as a string.

Show code

# Create a dictionary with three pairs of key-value.
record = {"sentence_id": 1,             # a number, so you can refer to this sentence later
          "text": text,        # the sentence itself
          "label": my_level}   # your judgment

# Print that dictionary
print(record)

Two things here are new since step 6. The values are not all the same kind — 1 is a number, the other two are text — and two of them are variables rather than something typed in place: Python puts in whatever text and my_level are holding.

{id, text, label} is the shape of the gold standard you build on Day 2, of the file you score in S6, and of the sample you draw on Day 4. Every dataset in this course is made of records like this one.

🧪 Your turn — build a record for a sentence of your own. Fill in the three values; the keys are already there. Decide the level yourself before you read on.

Show code

my_record = {"id": ...,        # ✏️ any number
             "text": ...,      # ✏️ a sentence of your own
             "label": ...}     # ✏️ your judgment: A1 to C2

print(my_record)
print("how many pairs:", len(my_record))

Step 8 · Getting the answers back out

Your record holds three values under three names. To use one, you have to get it back out.

Use square brackets to do that: record["label"] means give me “the value stored under label”.

my_record = {"id": ...,        # ✏️ any number
             "text": ...,      # ✏️ a sentence of your own
             "label": ...}     # ✏️ your judgment: A1 to C2

Show code

print("the whole record:", record)

Show code

print("just the sentence:", record["text"])     # read one value by its key

Show code

print("just my level:  ", record["label"])

Assigning a new key-value pair

The same brackets on the left of an = do the opposite: they store a value instead of reading one. Step 2 said = means “store this”, and that still holds — record["model"] = ... stores under the key model.

The key does not have to exist yet. A dict can gain a pair after it is built, which is how the model’s answer joins your judgment in the same record:

Show code

record["model"] = "B1"       # a key that was not there before
print(record)                # the same record, now with four pairs
print("how many pairs:", len(record))

That was a level typed in by hand. Now ask the model, and store what it says.

Two steps, as in step 5: get the reply into a variable, then put it in the record. .strip() removes the line break the reply arrives with — "A1 " with a trailing space is not equal to "A1", so without it the comparison below would report a disagreement where there was none.

Show code

answer = ai.generate_text(f"What CEFR level (A1-C2) is this sentence? Answer with just the level. {text}")
clean = answer.strip()               # drop the newline the reply arrives with

record["model"] = clean              # store it under its own key
print(record)

Two answers, one record. To ask whether they match, use == — two equals signs, which asks “are these the same value?” rather than storing anything.

The answer comes back as True or False. That is a third kind of value alongside the three shapes from step 6, and it is what every score this week is built from.

Show code

print("you said: ", record["label"])
print("model said:", record["model"])
print("they agree:", record["label"] == record["model"])

One record is one sentence. A dataset is a list of them.

This is the first time the shapes from step 6 sit inside one another: square brackets holding curly braces, one record per line. The punctuation nests, and reading it is a matter of finding the matching bracket.

Here are three records, already answered, so you have something to compare:

Show code

items = [
    {"id": 1, "text": "The cat sat on the mat.", "label": "A1", "model": "A1"},
    {"id": 2, "text": "More research is needed.", "label": "B1", "model": "B1"},
    {"id": 3, "text": "Nevertheless, the findings were inconclusive.", "label": "C1", "model": "B2"},
]

print("how many records:", len(items))

A list is read by position, not by key — and positions start at 0, so the third record is items[2]. Chain the two kinds of brackets to reach a value inside a record: position first, then key.

Show code

print("the third record:", items[2])
print("its sentence:", items[2]["text"])          # list by position, then dict by key
print("they agree: ", items[2]["label"] == items[2]["model"])

False — one record, one disagreement. That is the comparison the whole week rests on. In step 9 you run it over every record at once.

You can also take several records with a colon. items[:2] means “from the start, up to but not including position 2” — the first two. From Day 3 on you use this to try a prompt on the first few items of a dataset before running it on all of them.

Show code

print("the first two:", items[:2])          # a slice — records 0 and 1
print("all but the first:", items[1:])      # from position 1 to the end
print("how many in the slice:", len(items[:2]))

🧪 Your turn — answer both before you run the cell.

How many records does items[:2] hold? How many does items[1:3]?
Do items[0]["label"] and items[0]["model"] agree — and why do you already know?

Show code

print(len(items[:2]), len(items[1:3]))   # count them before you run this
print(items[0]["label"] == items[0]["model"])

items[:2] and items[1:3] both hold two records — a slice never includes its right-hand number.

Counting from 0 has one more consequence. Three records sit at positions 0, 1 and 2, so there is no position 3 — asking for one is an error rather than an empty answer:

Show code

print(items[3])          # this cell is supposed to fail — read the last line

IndexError: list index out of range. That off-by-one is the commonest mistake with lists and slices, and it is worth meeting here rather than on Day 4.

Where this goes: items[2]["label"] is how Day 2 reads a gold label out of a file, and val[:5] on Day 3 is this same slice on a real dataset.

Summary of Steps 5-8

Step 5: Putting your data into a prompt

If you have a multiple values of a variable to use for a prompt, use f-strings (e.g., f"...{variable}...").
You can also create reusable prompt templates with placeholders (e.g., {text}) and fill them using the .format() method, allowing the same prompt structure to be applied to multiple pieces of data.

Step 6: The three shapes your data comes in

Explored fundamental Python data types crucial for handling LLM outputs and linguistic data:
- str (string): For text, where len() counts characters.
- list: For ordered collections of items, where len() counts items.
- dict (dictionary): For key-value pairs (labeled records), where len() counts pairs.
Noted that these data types can be nested (e.g., a list of dictionaries).

Step 7: Put your judgment in a record

Combined str, int, and dict types to create a single record (dictionary).

Step 8: Getting the answers back out

Use dict["key"] to get the value from the key in that dict.
Discovered that the same square bracket notation on the left side of an assignment (=) can be used to add new key-value pairs to an existing dictionary.
Introduced the == operator to compare two values, returning True or False, which is the basis for scoring LLM performance.

Step 9 · How often did the model agree with you?

You often have more than ONE items in the dataset. Comparing one record took one line in step 8.

Comparing all of them takes a for loop — it repeats the same comparison for every record in the list.

In Python, you can say:

for x in some_variable:

x can be any name. If it is a word list, you can use w or word.

Show code

# Before running the next cell, remember items
print(items)

Show code

n_agree = 0                              # how many times we agreed, so far
for item in items:                       # look at each record in turn
    if item["label"] == item["model"]:   # did the two answers match?
        n_agree = n_agree + 1
        print("✅", item["text"])
    else:
        print("❌", item["text"], "— you:", item["label"], "model:", item["model"])

print("agreed", n_agree, "out of", len(items))
print("accuracy:", n_agree / len(items))

That last number is accuracy: how many times the two of you agreed, divided by how many sentences there were. It is the first thing you report on Day 2, computed there over a hundred sentences instead of three or four. Part B step 8 comes back to the division itself, and to what has to happen when the list can be empty.

So, in one session: you called a language model, stored its answers beside your own judgments in a record, and measured how often the two of you agreed. Everything else this week is that same loop over more data.

Session 3· From text to sentences, then the Python you’ll reuse

In Session 2 you called the model once. Here you’ll turn a paragraph into individual sentences (the unit you’ll annotate on Day 2), and then write the handful of Python patterns that the rest of the week is built from.

Thirteen short steps, in three blocks:

Steps	What you do
1–2	Split a paragraph into sentences, badly and then properly
3–9	Write the loops, counts and conditions you will reuse on Days 2–5
10–13	Practice on your own, with a self-check after each one

An optional extra section follows at the end, on how a model stores the meaning of a word. Nothing later in the week depends on it.

Cells marked ✏️ YOU EDIT are yours to change; run each self-check until every line prints ✅.

Step 1 · Splitting text into sentences — without a model

Text arrives as one long string. To analyse it sentence by sentence you first have to segment it. The obvious idea: split on the full stop. To do that we call a method on the string — some_text.split(".") — using a dot (.) to run a built-in action on a value.

Show code

paragraph = ("Dr. Smith reviewed the data. The results were clear, e.g. accuracy rose. Scores went from 3.14 to 9.")

naive = paragraph.split(".")     # cut the string at every "."
print("pieces:", len(naive))     # how many pieces did that give us?
for piece in naive:              # look at each piece in turn
    print(repr(piece))           # repr() shows the quotes and spaces exactly

Look at the output: "Dr" (from Dr.), "e" and "g" (from e.g.), and "3"/"14" (from 3.14) all got split in the wrong places. Sentence boundaries are not just full stops — abbreviations and decimals break the naive rule.

Step 2 · Splitting text into sentences — with a model

A proper tool knows more than “cut at every dot”. We’ll use spaCy, an NLP library. import spacy loads that toolbox; spacy.blank("en") makes a minimal English pipeline and we add a rule-based sentencizer to it (no model download needed).

Show code

### Step 1: build a sentence splitter that knows more than "cut at every dot" ###
import spacy                       # the NLP toolbox
nlp = spacy.blank("en")            # a minimal English pipeline
nlp.add_pipe("sentencizer")        # add the rule-based splitter — no download

### Step 2: run it on the paragraph and look at what came out ###
doc = nlp(paragraph)               # spaCy reads the text
sentences = list(doc.sents)        # spaCy's sentence objects, as a list
print("sentences found:", len(sentences))
print("first sentence:", sentences[0])

spaCy keeps Dr. and e.g. intact and finds the real boundaries. Why this matters: on Day 2 the unit you annotate and feed the LLM is the sentence — bad boundaries mean bad data downstream.

Steps 3–9 · The Python you’ll reuse all week

Everything from here to step 9 is a pattern you will be asked to write later this week. Each step shows it working on a small example first.

Step	Pattern	Where you’ll write it
3	`for` over a list, with `if` / `else`	every day
4	`for` over a list of records	Day 2, Day 4
5	Build a list with `.append`	Day 2 S6, the final project
6	Count with a dict	Day 2 S6
7	`if` / `elif` / `else`, and `and`	Day 2 S5 and S6
8	Divide, and guard against zero	Day 2 S6 — every metric
9	Wrap it in a function	Day 2 S6 onwards

Step 3 · Run the model over every sentence — `for` and `if`

Now that you have a list of sentences, do something to each one. A for loop repeats the same steps for every item; an if lets you react to what comes back. Below, we build a prompt for each sentence (with an f-string) and ask the model for its CEFR level.

Notice where the f-string sits: inside the loop, so the braces are filled in again for every sentence. Written once above the loop it would be filled in once, and all three calls would ask about the same sentence.

✏️ YOU EDIT — try your own sentences.

Show code

### Step 1: the sentences to ask about ###
examples = ["The findings were inconclusive.",      # ✏️ try your own sentences
            "Nevertheless, we draw some tentative conclusions.",
            "More research is needed."]

### Step 2: for each one — build a prompt, ask, and react to the reply ###
for sentence in examples:              # repeat everything below for each sentence
    prompt = f"What CEFR level (A1-C2) is this sentence? Answer with just the level. {sentence}"
    reply = ai.generate_text(prompt).strip()   # .strip() removes stray blank space
    if reply == "":                    # the model sometimes says nothing at all
        print(sentence, "→ (no answer)")
    else:                              # normal case: it answered
        print(sentence, "→", reply)

Step 4 · Loop over records, not just strings

Every dataset this week is a list of dicts — the {id, text, label} shape you built in Part A step 7. Below is that same shape without the model key, so the records hold only what a person said. Looping over the list works exactly as it did over sentences, except each item is a record, so you reach into it by key.

Show code

items = [{"id": 1, "text": "The cat sat on the mat.", "label": "A1"},
         {"id": 2, "text": "More research is needed.", "label": "B1"},
         {"id": 3, "text": "Nevertheless, the findings were inconclusive.",
          "label": "C1"}]

for item in items:                 # item is one record — a dict
    print(item["id"], "|", item["label"], "|", item["text"])

🧪 Your turn — print only the text of each item, without the id and the label.

Step 5 · Build a new list with `.append`

You met .append(...) in Part A step 9, adding one record to items. The pattern here is the same call, put to work inside a loop: start with an empty list, then add to it one item at a time.

This is the most-used pattern of the whole week: on Day 2 you build a list of verdicts this way, and in the final project you build the list of rows your coders disagreed about.

Show code

labels = []                        # start empty; the loop fills it
for item in items:
    labels.append(item["label"])   # add this item's label to the end

print(labels)                      # three labels, in the items' order
print("how many:", len(labels))

Three records in, three labels out, in the same order.

The same shape with an if in it keeps only some items:

Show code

advanced = []                      # the ones we want to keep
for item in items:
    if item["label"] == "C1":      # only C1 records get added
        advanced.append(item["text"])

print(advanced)

🧪 Your turn — build a list of the ids instead of the labels.

Step 6 · Count things with a dict

To count how many items carry each label, use a dict as a tally: the key is the label, the value is how many you have seen so far.

counts[label] = ... stores a value under a key — the same square brackets you used in Part A step 7 to put the model’s answer into record["model"], and the same ones you read a dict with.

The new piece is counts.get(label, 0), which reads the count so far and answers 0 when the label is new. Without it the very first item would fail, because counts["A1"] does not exist yet.

Show code

counts = {}                        # an empty dict: label -> how many
for item in items:
    label = item["label"]
    counts[label] = counts.get(label, 0) + 1   # count so far (0 if new), plus one

print(counts)

Each label appears once here, so every count is 1. Add another B1 record to items in step 4, re-run both cells, and watch that count become 2.

🧪 Your turn — count the items by the first letter of their label (item["label"][0] gives you "A", "B" or "C").

Step 7 · More than two branches — `elif` and `and`

An if/else splits two ways. elif (“else, if”) adds more branches: they are checked top to bottom, the first match wins, and exactly one runs.

and joins two conditions — both sides have to be true.

Show code

for item in items:
    label = item["label"]
    if label == "A1" or label == "A2":     # `or` = either side is enough
        band = "basic"
    elif label == "B1" or label == "B2":   # only checked if the first did not match
        band = "independent"
    else:                                  # anything left over
        band = "proficient"
    print(label, "→", band)

Now and. On Day 2 you compare two people’s labels for the same sentence, and a row only counts as a disagreement when both of them have actually labelled it and the two labels differ:

Show code

rows = [{"id": 1, "CoderA": "A1", "CoderB": "A1"},   # they agree
        {"id": 2, "CoderA": "B1", "CoderB": "C1"},   # they disagree
        {"id": 3, "CoderA": "B2", "CoderB": ""}]     # B has not reached this row

for row in rows:
    a = row["CoderA"]
    b = row["CoderB"]
    if a != "" and b != "" and a != b:   # both labelled it, AND they differ
        print(row["id"], "disagreement:", a, "vs", b)
    else:
        print(row["id"], "no disagreement")

Row 3 is not a disagreement: one coder simply hasn’t got there yet. That is a decision about your data, not a fact about it, and on Day 2 you will make it yourself.

Step 8 · Divide — and guard against zero

Counting leads to dividing: how many did we get right, out of how many there were? Python uses / for division, and brackets to say what to add up first.

On Day 2 every score you write has this shape, so it is worth doing once here.

Show code

right = 9        # how many the model got right
wrong = 3        # how many it got wrong

print(right / (right + wrong))            # 9 out of 12
print(round(right / (right + wrong), 3))  # round() to 3 decimal places

One thing can go wrong. If both counts are 0 there is nothing to divide by, and Python stops with ZeroDivisionError. So check before you divide — and when there is nothing to score, 0.0 is the honest answer:

Show code

right = 0
wrong = 0

if right + wrong == 0:      # nothing was scored at all
    score = 0.0             # so no credit — and no division by zero
else:
    score = right / (right + wrong)
print(score)

🧪 Your turn — set right = 7 and wrong = 1 and re-run. You should get 0.875.

Step 9 · Wrap it in a function

You have now written the same three steps twice — build a prompt, call the model, tidy the reply. Name it once with def, and the rest of the week you just call ask(sentence).

One change from step 3: print became return. print puts a value on the screen and it is gone; return hands the value back to whoever called the function, so you can store it, count it, or score it.

Show code

def ask(sentence):                     # `def` names a block of steps
    """Ask the model for the CEFR level of one sentence; return its reply."""
    prompt = f"What CEFR level (A1-C2) is this sentence? Answer with just the level. {sentence}"
    return ai.generate_text(prompt).strip()   # `return` hands the answer back

print(ask("The cat sat on the mat."))  # one line now does all three steps

Because ask hands its answer back, you can put it straight into the loop from step 9 and keep every reply:

Show code

replies = []
for sentence in examples:          # the three sentences from step 3
    replies.append(ask(sentence))  # call the function, keep what it returns

print(replies)

A list of sentences in, a list of the model’s labels out. That is the whole shape of what you will do on Days 3 and 4 — and on Day 2 you will learn how to tell whether those labels are any good.

Steps 10–13 · Your turn — Python practice

Four exercises, each rehearsing one pattern from steps 3–9. Fill in the function (replace the raise NotImplementedError(...) line), then run the self-check directly below it until it prints ✅. No grader needed — the checks are your grader.

The shared dataset for all four:

Show code

sample = [{"id": 1, "text": "Hi.", "label": "A1"},
          {"id": 2, "text": "Hello there.", "label": "A1"},
          {"id": 3, "text": "Nevertheless, the findings were inconclusive.",
           "label": "C1"}]

print(len(sample), "items")

Step 10 · Read one value out of a record

The pattern from Part A step 7: reach into a dict by key.

Show code

# ✏️ YOU EDIT — replace the NotImplementedError with your code.

def label_of(item):
    """Return the value stored under the key "label" in the dict `item`.
    Example: label_of({"id": 1, "text": "Hi", "label": "A1"}) -> "A1".
    """
    raise NotImplementedError("Return item['label'].")

Show code

#@title 🔎 Self-check — step 10 { display-mode: "form" }
ok = label_of(sample[0]) == "A1"
print(("✅" if ok else "❌"), "label_of →", label_of(sample[0]))

Step 11 · Build a list in a loop

The pattern from step 5: an empty list, a for, an if, and .append.

Show code

# ✏️ YOU EDIT — replace the NotImplementedError with your code.

def long_words(words, n):
    """Return a LIST of the words whose length is greater than n.
    Example: long_words(["a", "cat", "elephant"], 3) -> ["elephant"].
    """
    # HINT: start with an empty list; loop with `for w in words:`;
    #       keep w when len(w) > n; return the list at the end.
    raise NotImplementedError("Return the words longer than n characters.")

Show code

#@title 🔎 Self-check — step 11 { display-mode: "form" }
got = long_words(["a", "cat", "elephant"], 3)
ok = got == ["elephant"]
print(("✅" if ok else "❌"), "long_words →", got)

Step 12 · Count with a dict

The pattern from step 6: counts[label] = counts.get(label, 0) + 1.

Show code

# ✏️ YOU EDIT — replace the NotImplementedError with your code.

def count_labels(items):
    """Given a list of {id, text, label} dicts, return a dict mapping each
    label to how many times it appears.
    Example: count_labels([{"label":"A1"}, {"label":"A1"}, {"label":"B1"}])
             -> {"A1": 2, "B1": 1}.
    """
    # HINT: start with counts = {}; for each item, add 1 to counts[label]
    #       (use counts.get(label, 0) + 1 so the first time starts at 0).
    raise NotImplementedError("Count how many items carry each label.")

Show code

#@title 🔎 Self-check — step 12 { display-mode: "form" }
got = count_labels(sample)
ok = got == {"A1": 2, "C1": 1}
print(("✅" if ok else "❌"), "count_labels →", got)

Step 13 · Count, then divide — with a guard

Steps 6 and 8 together, and the shape of every score you write on Day 2: count what matched, divide by the total, and return 0.0 rather than dividing by zero.

Show code

# ✏️ YOU EDIT — replace the NotImplementedError with your code.

def accuracy(items, guesses):
    """What fraction of the guesses match the items' labels?

    `items` and `guesses` are the same length, in the same order.
    Return 0.0 when there is nothing to score.
    Example: accuracy(sample, ["A1", "A1", "B1"]) -> 0.667.
    """
    # HINT: count the matches in a loop, using a position counter like S6 does:
    #       i = 0 before the loop, guesses[i] inside it, i = i + 1 at the end.
    #       Then guard: if len(items) == 0, return 0.0 — otherwise divide.
    raise NotImplementedError("Count the matches, then divide by how many there are.")

Show code

#@title 🔎 Self-check — step 13 { display-mode: "form" }
### Step 1: two guesses right out of three, and an empty case for the guard ###
got = accuracy(sample, ["A1", "A1", "B1"])
empty = accuracy([], [])

### Step 2: compare both against the answers we already know ###
ok = round(got, 3) == 0.667 and empty == 0.0
print(("✅" if ok else "❌"), "accuracy →", round(got, 3), "| empty case:", empty)

All four together

Run this once all four print ✅ on their own.

Show code

#@title 🔎 Self-check — all four { display-mode: "form" }
### Step 1: run every function against an answer we already know ###
checks = [                             # each entry: (name, did it match?)
    ("label_of", label_of(sample[0]) == "A1"),
    ("long_words", long_words(["a", "cat", "elephant"], 3) == ["elephant"]),
    ("count_labels", count_labels(sample) == {"A1": 2, "C1": 1}),
    ("accuracy", round(accuracy(sample, ["A1", "A1", "B1"]), 3) == 0.667),
]

### Step 2: report one line per check, then an overall verdict ###
for name, ok in checks:
    print(("✅" if ok else "❌"), name)
print("All passed ✅" if all(ok for _, ok in checks)   # all() = every one of them
      else "Some checks failed — fix them and re-run.")

Optional · What a model stores about a word

This section is optional, and nothing later in the week depends on it. It is here because it makes the Session 1 slides runnable: you can look at the numbers a model keeps for each word, and see for yourself where they stop working.

If the session is running short, stop at step 13 and come back to this on your own.

This section uses an English model that carries a vector for every word. It is a large file and takes a minute or two to arrive. Start it now, and read on while it downloads.

Show code

#@title 📥 Download the word-vector model { display-mode: "form" }
# Helper — you don't need to read this. Run it and move on.
!python -m spacy download en_core_web_lg

import importlib, spacy
# The model was installed after Python started, so refresh the module list
# before loading it — otherwise the load below may not find it yet.
importlib.invalidate_caches()
nlp_vec = spacy.load("en_core_web_lg")
print("✅ word vectors ready —", f"{nlp_vec.vocab.vectors.shape[0]:,}", "words have one")

A · What a model stores about a word

In Session 1 you saw that a model turns each word into a long list of numbers, and that words with similar meanings end up with similar numbers. Those numbers are not hidden — you can look at them.

The pipeline you just used, nlp, only knows where sentences end. The one you downloaded just above, nlp_vec, also carries a vector for every word it knows.

Show code

word = nlp_vec("suggest")[0]      # read one word; [0] takes the first token
print("how many numbers:", word.vector.shape)
print("the first eight: ", word.vector[:8].round(3))   # rounded, to fit on one line

Three numbers were enough to place a colour in the Session 1 colour cube. It takes three hundred to place a word.

B · Are similar words really close?

.similarity(...) compares two vectors and gives a number between 0 and 1: the higher it is, the closer the two words sit. Compare a pair that share a meaning with a pair that do not.

✏️ YOU EDIT — swap in two words from your own research area and re-run.

Show code

print("suggest / indicate:", round(nlp_vec("suggest").similarity(nlp_vec("indicate")), 3))
print("suggest / banana:  ", round(nlp_vec("suggest").similarity(nlp_vec("banana")), 3))

Instead of guessing pairs, you can ask which words sit closest to a given one. Run the helper, then try your own word.

Show code

#@title 🔧 Helper: nearest(word) → the closest words { display-mode: "form" }
# Helper — you don't need to read this. Run it and move on.
import numpy


def nearest(word: str, n: int = 10) -> list[tuple[str, float]]:
    """The n words whose vectors sit closest to `word`.

    Args:
        word: the word to look up.
        n: how many neighbours to return.

    Returns:
        A list of (word, closeness) pairs, closest first. Empty if the model
        has never seen the word.

    Example:
        >>> nearest("hedge", n=3)
    """
    entry = nlp_vec.vocab[word]
    if not entry.has_vector:                 # nothing to compare against
        print(f"{word!r} is not in this model's vocabulary.")
        return []
    # most_similar wants a table of rows, so hand it a table with one row in it.
    one_row = entry.vector.reshape(1, -1)
    # Ask for far more candidates than we need: the table stores several spellings
    # of the same word (suggest, Suggest, SUGGEST), and we keep only one of each.
    keys, _, scores = nlp_vec.vocab.vectors.most_similar(one_row, n=n * 8)
    seen = {word.lower()}                    # never report the word itself
    neighbours = []
    for key, score in zip(keys[0], scores[0]):
        found = nlp_vec.vocab.strings[key].lower()
        if found.isalpha() and found not in seen:
            seen.add(found)
            neighbours.append((found, round(float(score), 3)))
    return neighbours[:n]


print("Helper ready. Try nearest('perhaps').")

✏️ YOU EDIT — put in a word you care about.

Show code

for neighbour in nearest("perhaps"):   # ✏️ your word here
    print(neighbour)

Two things are worth noticing before you move on.

The neighbours of perhaps are mostly other hedges — possibly, probably, certainly, might. Nothing told the model that these words hedge a claim; it placed them together because they turn up in the same positions in text.

Now try nearest("corpus"). The closest words include habeas, christi and corpora. One spelling has collected the legal sense, the religious sense and the linguistic sense into a single vector, because there is only one vector available for the form. nearest("hedge") does the same thing with garden hedges and financial hedging. Keep that in mind for the next part.

C · Where these vectors stop working

Session 1 used free in two sentences that mean different things — your account is free of charge and claim your free prize now. Take the word out of each sentence and compare the two vectors.

Show code

banking = nlp_vec("your account is free of charge")
promo   = nlp_vec("claim your free prize now")

free_1 = banking[3]      # the 4th word of the first sentence
free_2 = promo[2]        # the 3rd word of the second sentence
print("comparing:", free_1.text, "and", free_2.text)
print("similarity:", round(free_1.similarity(free_2), 3))

The score is exactly 1.0, because the two vectors are not merely close — they are the same vector. This model stores one vector per word form and looks it up the same way every time, so the surrounding words change nothing. That is what static means, and it is the limitation Session 1 said attention was built to remove.

(Both sentences use lower-case free on purpose: these vectors are looked up by exact form, so FREE and free are separate entries.)

The same lookup has a second consequence. A sentence’s vector is the average of its words, and an average does not record the order they came in:

Show code

print(round(nlp_vec("the dog chased the cat")
            .similarity(nlp_vec("the cat chased the dog")), 3))

Two sentences with opposite meanings, one score of 1.0. Word order is gone. Why this matters for the rest of the week: the categories you will annotate — a CEFR level, a rhetorical move, whether a claim is hedged — depend on word order and context. A model built on these vectors alone cannot represent that; the model you call with ai.generate_text(...) can.

D · A map of the space

Three hundred numbers per word is too many to look at, but the words can be flattened onto a page so that words with close vectors land near each other.

Show code

#@title 🔧 Helper: map_words(words) → a 2-D picture { display-mode: "form" }
# Helper — you don't need to read this. Run it and move on.
import matplotlib.pyplot as plt


def map_words(words: list[str]) -> None:
    """Plot words on a flat map, keeping words with close vectors close.

    Args:
        words: the words to place.

    Returns:
        Nothing. It draws the picture.

    Example:
        >>> map_words(["cat", "dog", "syntax"])
    """
    known = []
    for w in words:
        if nlp_vec.vocab[w].has_vector:
            known.append(w)
        else:
            print(f"skipping {w!r} — not in the model's vocabulary")
    rows = numpy.array([nlp_vec.vocab[w].vector for w in known])
    rows = rows - rows.mean(axis=0)          # centre the cloud on zero
    # 300 directions is too many to draw, so keep the two along which these
    # particular words are most spread out, and plot along those.
    _, _, directions = numpy.linalg.svd(rows, full_matrices=False)
    flat = rows @ directions[:2].T
    plt.figure(figsize=(8, 6))
    plt.scatter(flat[:, 0], flat[:, 1], s=18)
    for w, (x, y) in zip(known, flat):
        plt.annotate(w, (x, y), fontsize=11, xytext=(4, 3),
                     textcoords="offset points")
    plt.title("Words placed by their vectors")
    plt.xticks([])
    plt.yticks([])
    plt.show()


print("Helper ready. Try map_words([...]).")

✏️ YOU EDIT — change the words and re-run.

Show code

# Three groups of words: hedges, research verbs, and animals.
map_words(["perhaps", "possibly", "maybe", "likely", "presumably",
           "suggest", "indicate", "demonstrate", "argue", "conclude",
           "cat", "dog", "horse", "rabbit", "sheep"])   # ✏️ your words

The animals land well away from the other ten words. The hedges and the research verbs, though, sit mixed together rather than in two groups — they are all words of academic prose, and they appear in similar places in similar texts, which is the only thing these vectors record. The map shows you what the model separates, which is not always what you want it to separate.

Python you’ll see but won’t have to write

The pre-written 🔧 Library cells later this week occasionally use two shorthands. You never have to write them — just recognise them:

A list comprehension builds a list in one line. [row["label"] for row in rows] means “the label of every row” — the same as a for loop that .appends each label.
try / except (handle an error instead of crashing) and global (let a function update a shared variable) show up in the LLM backend and are explained on Day 3 — no need to learn them today.

Mini-project — form your group & pick a track

Before Day 2 you’ll settle into a project group and choose a dataset track. Each group needs at least one Linguistic Data Analysis I alumnus. Once formed, pick your track and note it — you’ll annotate that dataset tomorrow.

See the Final Project page for the tracks and what the project involves.

✅ Before you submit

Runtime → Run all and check every cell ran without error.
Tutorial outputs are visible (tables / charts / the model’s answers).
Every Corpus Lab self-check prints ✅ (or your TODO answers are filled in).
File → Download → Download .ipynb and upload that one file.

Welcome to Colab notebook

How to use this notebook

Session 2 — your first LLM call, and reading its answer

Step 1 · Run a cell

Step 2 · Read an error

Step 3 · Variables — store a value under a name

Step 4 · Your first LLM call

Summary of Steps 1-4

Step 5 · Putting your data into a prompt

Pattern 1: f-string — build one prompt around one sentence

Pattern 2: template + .format() — write the prompt once, reuse it for any sentence

Step 6 · The three shapes your data comes in

Some data type we use

About len()

String

List

Dictionary

Step 7 · Put your judgment in a record

Step 8 · Getting the answers back out

Assigning a new key-value pair

Summary of Steps 5-8

Step 9 · How often did the model agree with you?

Session 3· From text to sentences, then the Python you’ll reuse

Step 1 · Splitting text into sentences — without a model

Step 2 · Splitting text into sentences — with a model

Steps 3–9 · The Python you’ll reuse all week

Step 3 · Run the model over every sentence — for and if

Step 4 · Loop over records, not just strings

Step 5 · Build a new list with .append

Step 6 · Count things with a dict

Step 7 · More than two branches — elif and and

Step 8 · Divide — and guard against zero

Step 9 · Wrap it in a function

Steps 10–13 · Your turn — Python practice

Step 10 · Read one value out of a record

Step 11 · Build a list in a loop

Step 12 · Count with a dict

Step 13 · Count, then divide — with a guard

All four together

Optional · What a model stores about a word

A · What a model stores about a word

B · Are similar words really close?

C · Where these vectors stop working

D · A map of the space

Python you’ll see but won’t have to write

Mini-project — form your group & pick a track

✅ Before you submit

Pattern 2: template + `.format()` — write the prompt once, reuse it for any sentence

About `len()`

Step 3 · Run the model over every sentence — `for` and `if`

Step 5 · Build a new list with `.append`

Step 7 · More than two branches — `elif` and `and`