Skip to content

Week 7 — Evals II: judges and the CI gate

Why this week: the checker now has an agent, retrieval and a graph tool. A change in one part can quietly break another: a "better" prompt that misses more scams, a tool that starts inventing phone numbers. This week you make those breaks visible automatically, so no change reaches main without a number attached.

By Friday you have: code-based checks that catch invented phone numbers and links, an LLM judge for explanation quality with its agreement with your own labels measured, a promptfoo comparison of two prompts, a GitHub Actions workflow that runs lint, types and tests on every push, and an eval gate that fails the build when scam recall drops below a limit stored in the repo. You also publish your second write-up, "How I evaluated an anti-scam checker".

Main online course/reading: Using LLM-as-a-Judge For Evaluation: A Complete Guide by Hamel Husain, and promptfoo: Getting started. promptfoo's own examples use API keys; you run it through your own checker instead, so it needs none.

Concepts

Code-based checks. Anything you can test without asking a model: the advice field isn't empty, a "scam" verdict has at least one red flag, every phone number in the answer was already in the message. They're free, instant and never disagree with themselves. Write these first; use a model only for what code can't judge.

LLM-as-judge. A second model call that grades the first one's answer against a rubric. Here it grades explanation quality: are the red flags specific and real, and is the advice safe? Following Hamel Husain's guide, the judge gives a plain pass or fail with a short critique, not a 1-to-10 score. Pass/fail forces a clear standard; scores drift.

Agreement: true positive and true negative rates. A judge is only useful if it agrees with you, the domain expert. You label answers by hand, run the judge on the same answers, and count. True positive rate: of the answers you passed, the share the judge also passed. True negative rate: of the answers you failed, the share the judge also failed. Report both: if 90% of answers are good, a judge that passes everything is "90% accurate" and useless.

Train, dev and test labels. You'll tune the judge's prompt by reading its disagreements, which is a kind of fitting. So split your labels like any model's data: a few examples go into the prompt (train), most are used to tune it (dev), and some are kept back for one final honest number (test).

promptfoo. An open-source tool for running the same test cases against several versions of a system and showing the results side by side. You give it providers (things that answer), test cases and assertions (checks on each answer). Your provider is a small Python file that calls your own checker, so promptfoo itself never needs an API key.

Continuous integration (CI). A service that runs your checks on a fresh computer every time you push code. GitHub Actions is GitHub's version. You describe the steps in a YAML file (an indented text format for settings) in .github/workflows/; GitHub starts a clean Linux machine (a runner), copies your repo onto it, and runs the steps. A red cross on the commit means a step failed.

Secrets. Your eval gate needs Claude, and Claude needs a login. A secret is a value GitHub stores encrypted and hands to the runner only while a workflow runs. You never type it into a file, never print it, and never commit it. Your repo is public, so anything committed is public forever, even after you delete it.

Quality gate, and noise. A gate is a step that fails the build when a number is below a limit, here scam recall on your labelled messages. LLM answers vary a little from run to run, so the same code can score 0.94 one day and 0.92 the next. You measure that spread first and set the limit just below it. Otherwise the gate fails at random and you learn to ignore it.

Monday — Code-based checks, then promptfoo

Learn (75 min):

Build:

  1. The checks read your week 5 results file. Open the newest evals/results_<date>.csv and confirm it has these columns: text, is_scam, verdict, red_flags (a JSON list) and advice. If red_flags or advice are missing, add them to scripts/run_eval.py (json.dumps(result.red_flags) and result.advice) and re-run it; you need them all week.

  2. Make evals/ a package, so its files can import each other, and let pytest find it. Create an empty evals/__init__.py, then add this to the end of pyproject.toml:

    [tool.pytest.ini_options]
pythonpath = ["."]

    You'll run the files in evals/ with python -m evals.<name>. The -m runs a file as part of the project, so from evals.checks import ... works.

  3. The checks, evals/checks.py:

    """Code-based checks: anything we can test without asking a model."""

import re

from scam_checker.models import CheckResult

# Singapore numbers: 8 digits starting 3, 6, 8 or 9, or 1800 lines; spaces or dashes allowed.
PHONE = re.compile(
    r"(?:\+65[\s-]?)?\b(?:[3689]\d{3}[\s-]?\d{4}|1800[\s-]?\d{3,4}[\s-]?\d{3,4})\b"
)
DOMAIN = re.compile(
    r"\b(?:https?://)?((?:[a-z0-9-]+\.)+[a-z]{2,})(?:/\S*)?", re.IGNORECASE
)


def digits(s: str) -> str:
    return re.sub(r"\D", "", s).removeprefix("65")


def phones_in(text: str) -> set[str]:
    return {digits(m) for m in PHONE.findall(text)}


def domains_in(text: str) -> set[str]:
    return {d.lower().removeprefix("www.") for d in DOMAIN.findall(text)}


def output_text(result: CheckResult) -> str:
    return " ".join([*result.red_flags, result.advice])


def advice_present(text: str, result: CheckResult) -> bool:
    return len(result.advice.strip()) >= 20


def scam_has_red_flags(text: str, result: CheckResult) -> bool:
    return result.verdict != "scam" or len(result.red_flags) > 0


def no_invented_phone_numbers(text: str, result: CheckResult) -> bool:
    """Every number in the answer must already be in the message."""
    return phones_in(output_text(result)) <= phones_in(text)


CHECKS = [
    advice_present,
    scam_has_red_flags,
    no_invented_phone_numbers,
]


def run_checks(text: str, result: CheckResult) -> dict[str, bool]:
    return {check.__name__: check(text, result) for check in CHECKS}

    <= between two sets means "is contained in". Why care about invented numbers? A checker that tells a worried user "call DBS on 1800-..." with a number it made up is sending them to whoever owns that number.

  4. Your turn: write no_invented_links, the same idea for web addresses, using domains_in, and add it to CHECKS. A checker that says "go to dbs.com.sg instead" is guessing at a domain it can't verify.

  5. Run the checks over your results, evals/run_checks.py:

    """Run the code-based checks over a results file from the eval runner.

Run with: uv run python -m evals.run_checks evals/results_<date>.csv
"""

import json
import sys

import pandas as pd

from evals.checks import run_checks
from scam_checker.models import CheckResult

df = pd.read_csv(sys.argv[1])
rows = []
for r in df.itertuples():
    result = CheckResult(
        verdict=r.verdict, red_flags=json.loads(r.red_flags), advice=r.advice
    )
    rows.append(run_checks(r.text, result))
checks = pd.DataFrame(rows)

print("pass rate per check:")
print(checks.mean().round(2).to_string())
for name in checks.columns:
    for i in checks.index[~checks[name]]:
        print(f"\nFAIL {name}: {df.loc[i, 'text'][:70]}")
        print(f"  advice: {df.loc[i, 'advice']}")

    uv run python -m evals.run_checks evals/results_2027-01-15.csv

    Use your own file name. In testing, on 16 messages, Haiku invented a phone number in 3 answers (19%) and a web address in 4. One read: "call DBS customer service on 1800-1111-355 to verify." Write your rates in the README.

  6. Tests for the checks, tests/test_checks.py. No model involved, so they run in a second:

    from evals.checks import no_invented_links, no_invented_phone_numbers, phones_in
from scam_checker.models import CheckResult


def result(advice: str, red_flags: list[str] | None = None) -> CheckResult:
    return CheckResult(verdict="scam", red_flags=red_flags or ["x"], advice=advice)


def test_phone_formats():
    assert phones_in("call +65 9123 4567 or 1800-227-7777") == {
        "91234567",
        "18002277777",
    }


def test_invented_number_fails():
    msg = "DBS: your account is suspended."
    assert not no_invented_phone_numbers(msg, result("Call DBS on 1800 111 1111."))


def test_number_quoted_from_message_passes():
    msg = "WhatsApp me at 9123 4567"
    assert no_invented_phone_numbers(msg, result("Do not contact 91234567."))


def test_money_is_not_a_phone_number():
    assert phones_in("Pay $1,800.00 or $84.20 today") == set()


def test_invented_link_fails():
    assert not no_invented_links(
        "Pay at dbs-login.top", result("Go to dbs.com.sg instead.")
    )

    uv run pytest tests/test_checks.py -q

  7. Now compare your current prompt with a candidate that fixes the problem, using promptfoo. promptfoo runs on Node.js, a JavaScript runtime; check you have version 22.22 or later with node --version (if not, brew install node). npx promptfoo@latest downloads and runs it without a permanent install.

    First, let check_message take the system prompt as an argument, so a test can swap it. In src/scam_checker/checker.py, change the signature and the system= line:

    def check_message(
    text: str, max_attempts: int = 3, system: str = SYSTEM_PROMPT
) -> CheckResult:

                system=system,

  8. The provider, evals/promptfoo_provider.py. promptfoo calls call_api once per test case:

    """Lets promptfoo call your own checker. No API key: it goes through call_claude()."""

from typing import Any

from scam_checker.checker import SYSTEM_PROMPT, check_message


def call_api(
    prompt: str, options: dict[str, Any], context: dict[str, Any]
) -> dict[str, Any]:
    config = options.get("config", {})
    system = config.get("system_prompt", SYSTEM_PROMPT)
    result = check_message(prompt, system=system)
    return {"output": result.model_dump_json()}

  9. The assertion, evals/promptfoo_assert.py: right verdict, and all your code checks pass. promptfoo calls get_assert on every answer:

    """promptfoo assertion: right verdict, and the code-based checks pass."""

from typing import Any

from evals.checks import run_checks
from scam_checker.models import CheckResult


def get_assert(output: str, context: dict[str, Any]) -> dict[str, Any]:
    text, expected = context["vars"]["message"], context["vars"]["expected"]
    result = CheckResult.model_validate_json(output)
    flagged = result.verdict == "scam"
    parts = [
        {
            "pass": flagged == (expected == "scam"),
            "score": 1.0 if flagged == (expected == "scam") else 0.0,
            "reason": f"verdict {result.verdict}, expected {expected}",
        }
    ]
    for name, ok in run_checks(text, result).items():
        parts.append({"pass": ok, "score": float(ok), "reason": name})
    failed = [p["reason"] for p in parts if not p["pass"]]
    return {
        "pass": not failed,
        "score": sum(p["score"] for p in parts) / len(parts),
        "reason": "; ".join(failed) or "all checks passed",
        "componentResults": parts,
    }

    promptfoo also has built-in "model-graded" assertions such as llm-rubric. Those call a grading model through an API, which needs a key, so this course doesn't use them. Your judge, on Wednesday, is your own code.

  10. The candidate prompt, prompts/candidate.txt: your current SYSTEM_PROMPT with two lines added at the end:

    Never give phone numbers or web addresses that are not in the message. Tell the reader
to find the official contact themselves, such as in the organisation's own app or on their card.

  11. The test cases, evals/promptfoo_cases.csv. Start with two; grow it to 10–20 messages that must never go wrong, including the ones your checks just caught:

    message,expected
"DBS: your account has been suspended. Verify your details at dbs-secure-login.com",scam
"CPF: Your contribution for Oct 2026 has been received. Log in to the CPF website with Singpass to view details.",not_scam

  12. The config, promptfooconfig.yaml in the project root. Two providers, both your checker, one with the candidate prompt:

    # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: Scam checker, current prompt vs candidate prompt

prompts:
  - "{{message}}"

providers:
  - id: file://evals/promptfoo_provider.py
    label: current prompt
  - id: file://evals/promptfoo_provider.py
    label: candidate prompt
    config:
      system_prompt: file://prompts/candidate.txt

defaultTest:
  assert:
    - type: python
      value: file://evals/promptfoo_assert.py

tests: file://evals/promptfoo_cases.csv

    "{{message}}" means "send each test's message column as the prompt". file:// loads a file; for system_prompt, promptfoo reads the text file and passes its contents.

  13. Run it. The three settings tell promptfoo to use your project's Python, to let that Python import evals, and not to send usage statistics:

    export PROMPTFOO_PYTHON=.venv/bin/python PYTHONPATH=. PROMPTFOO_DISABLE_TELEMETRY=1
npx promptfoo@latest eval
npx promptfoo@latest view

    Each test runs once per provider, so two cases make four Claude calls. view opens a table in your browser. In testing, the current prompt failed both cases on invented contacts (no_invented_links on the DBS message; both checks on the CPF one) and the candidate passed both.

Done when: run_checks reports a pass rate for each of your four checks, the tests pass, and promptfoo shows your two prompts side by side on at least 10 cases. If the candidate wins, copy its two lines into SYSTEM_PROMPT.

Stuck? no_invented_links, and promptfoo errors 
def no_invented_links(text: str, result: CheckResult) -> bool:
    """Every web address in the answer must already be in the message."""
    return domains_in(output_text(result)) <= domains_in(text)

Add it to the CHECKS list.

  • ModuleNotFoundError: No module named 'evals' in promptfoo's error log: you didn't set PYTHONPATH=. in this terminal. Every test fails with 0 errors shown, so check the log path promptfoo prints.
  • ModuleNotFoundError: No module named 'scam_checker' or pydantic: PROMPTFOO_PYTHON isn't pointing at .venv/bin/python.
  • ModuleNotFoundError: No module named 'evals' from pytest: the [tool.pytest.ini_options] block is missing, or evals/__init__.py is.
  • promptfoo says Node is too old: brew install node (or brew upgrade node), then open a new terminal.
  • The command exits with code 100: that's promptfoo saying at least one test failed, not a crash.

Tuesday — Label answers for the judge

Learn (90 min): Using LLM-as-a-Judge For Evaluation: A Complete Guide by Hamel Husain, all of it. Note his name for the method, "Critique Shadowing", and why he insists on pass/fail with a written critique. You are the "principal domain expert" in his steps: ten years of deciding what counts as a scam is exactly the role.

Build:

  1. Decide what the judge grades. Wednesday's rubric covers explanation quality only, four things:

    • each red flag points at something actually in the message, not a generic warning
    • the red flags are real warning signs, not backwards reasoning
    • the advice says exactly what to do, and never tells the reader to click, call or reply when the message might be a scam
    • the advice matches the verdict: no alarm for a genuine message, no reassurance for a scam

    Whether the verdict is right is measured separately, by recall and precision. Invented numbers and links are caught by Monday's code. Don't make the judge do work code can do.

  2. Copy answers into a sheet, evals/make_judge_sheet.py:

    """Copy checker answers into a sheet for you to grade by hand.

Run with: uv run python -m evals.make_judge_sheet evals/results_<date>.csv
Writes: evals/judge_labels.csv (fill in the label and critique columns)
"""

import sys

import numpy as np
import pandas as pd

N = 40
COLUMNS = ["text", "verdict", "red_flags", "advice"]

df = pd.read_csv(sys.argv[1])
sheet = df.sample(min(N, len(df)), random_state=42)[COLUMNS]
# About 20% train (examples for the judge), 40% dev (to tune it), 40% test (final).
rng = np.random.default_rng(42)
sheet["split"] = rng.choice(["train", "dev", "test"], len(sheet), p=[0.2, 0.4, 0.4])
sheet["label"] = ""  # you write pass or fail
sheet["critique"] = ""  # you write one or two sentences saying why
sheet.to_csv("evals/judge_labels.csv", index=False)
print(sheet["split"].value_counts())

    uv run python -m evals.make_judge_sheet evals/results_2027-01-15.csv

    Run it once. Running it again overwrites your labels.

  3. Your turn: open evals/judge_labels.csv in Excel or Numbers and grade all 40 answers. Write pass or fail in label, and in critique one or two sentences a colleague could learn from. Be strict and consistent; you are defining the standard. Examples from testing:

    • fail: "Treats 'no direct link' as a phishing sign, which is backwards: genuine agencies avoid links." (a genuine CPF notice)
    • fail: "'Reply C' is a normal clinic convention, not a warning sign. The flags are generic." (a dental reminder)
    • pass: "Quotes the $300/day and the WhatsApp HR; advice says delete and don't contact. Specific and safe."

    Save as CSV, not as an Excel file.

  4. Count your fails per split. You need at least 3 fails in train, dev and test each, or the true negative rate means little. If the checker is too good to produce enough fails, add 20 answers from an older, weaker prompt, or from messages your week 5 error analysis found hard, and grade those too.

Done when: 40 or more rows are labelled with a critique each, with at least 3 fails in every split, and the file is committed.

Stuck?
  • You keep changing your mind on a borderline answer: that's normal; Hamel's guide calls it "criteria drift". Write the rule you settled on in the README ("a hedged red flag is fine if the advice is right") and re-grade the earlier ones.
  • The red_flags column shows ["...", "..."]: it's JSON, the format the scripts expect. Leave it as it is.
  • Excel changed the file's quotes or encoding: save as "CSV UTF-8".

Wednesday — The judge, and how far you can trust it

Learn (45 min): re-read "Step 5: Build Your LLM as A Judge, Iteratively" in Hamel's guide, and its FAQ answers "How do you validate an LLM judge against human labels?" and "What model do you use for the LLM judge?". Then skim the AI Evals FAQ you used in week 5 for anything on judges.

Build:

  1. The judge, evals/judge.py. It writes its critique before its verdict, so the verdict follows from reasons. It uses Sonnet, following Hamel's advice to use the most powerful model your budget allows for the judge; here the budget is your subscription's usage limits:

    """An LLM judge for explanation quality: pass or fail, with a critique."""

import json
from typing import Literal

import pandas as pd
from pydantic import BaseModel, Field

from scam_checker.llm import call_claude
from scam_checker.models import CheckResult


class JudgeVerdict(BaseModel):
    critique: str = Field(
        description="Two or three sentences on what is good or bad, citing it"
    )
    result: Literal["pass", "fail"]


RUBRIC = """You grade the explanation a scam checker gave for one text message.
The reader is an ordinary person in Singapore, often older and not technical.

PASS only if all four are true:
1. Each red flag points at something actually in the message. Generic flags that would
   fit any message fail.
2. The red flags are real warning signs. Backwards reasoning, such as treating a normal
   feature of genuine messages as suspicious, fails.
3. The advice says exactly what to do, and is safe: it never tells the reader to click a
   link, call a number or reply when the message might be a scam.
4. The advice matches the verdict: no alarming advice for a genuine message, and no
   reassuring advice for a scam.

Do not judge whether the verdict itself is right; that is measured separately.
Ignore phone numbers and web addresses in the answer; code checks those.
Write the critique first, then the result."""


def format_case(text: str, result: CheckResult) -> str:
    flags = "\n".join(f"- {f}" for f in result.red_flags) or "(none)"
    return (
        f"<message>\n{text}\n</message>\n<verdict>{result.verdict}</verdict>\n"
        f"<red_flags>\n{flags}\n</red_flags>\n<advice>\n{result.advice}\n</advice>"
    )


def build_system(examples: pd.DataFrame) -> str:
    """The rubric plus a few graded examples written by you."""
    shots = []
    for r in examples.itertuples():
        result = CheckResult(
            verdict=r.verdict, red_flags=json.loads(r.red_flags), advice=r.advice
        )
        shots.append(
            f"<example>\n{format_case(r.text, result)}\n"
            f"<critique>{r.critique}</critique>\n<result>{r.label}</result>\n</example>"
        )
    return RUBRIC + "\n\nGraded examples:\n\n" + "\n\n".join(shots)


def judge(
    text: str, result: CheckResult, system: str, model: str = "sonnet"
) -> JudgeVerdict:
    reply = call_claude(
        format_case(text, result),
        system=system,
        schema=JudgeVerdict.model_json_schema(),
        model=model,
    )
    return JudgeVerdict.model_validate(reply.data)

    Your train rows, with your critiques, become the graded examples inside the judge's prompt.

  2. The agreement script, evals/judge_agreement.py. It runs four judge calls at a time:

    """How often does the judge agree with your hand labels?

Run with: uv run python -m evals.judge_agreement dev    (or: test, once, at the end)
"""

import json
import sys
from concurrent.futures import ThreadPoolExecutor

import pandas as pd

from evals.judge import JudgeVerdict, build_system, judge
from scam_checker.models import CheckResult

split = sys.argv[1] if len(sys.argv) > 1 else "dev"
labels = pd.read_csv("evals/judge_labels.csv", dtype=str).fillna("")
system = build_system(labels[labels["split"] == "train"])
rows = labels[labels["split"] == split].reset_index(drop=True)


def grade(i: int) -> JudgeVerdict:
    r = rows.loc[i]
    result = CheckResult(
        verdict=r.verdict, red_flags=json.loads(r.red_flags), advice=r.advice
    )
    return judge(r.text, result, system)


with ThreadPoolExecutor(max_workers=4) as pool:
    verdicts = list(pool.map(grade, rows.index))
rows["judge"] = [v.result for v in verdicts]
rows["judge_critique"] = [v.critique for v in verdicts]

you_pass, you_fail = rows[rows.label == "pass"], rows[rows.label == "fail"]
print(f"{split}: {len(rows)} rows; you passed {len(you_pass)}, failed {len(you_fail)}")
if len(you_pass) and len(you_fail):
    # Of the answers you passed, the share the judge also passed; same for fails.
    tpr = (you_pass.judge == "pass").mean()
    tnr = (you_fail.judge == "fail").mean()
    print(f"true positive rate {tpr:.2f}   true negative rate {tnr:.2f}")
else:
    print("need both passes and fails in this split to measure agreement")
for r in rows[rows.label != rows.judge].itertuples():
    print(f"\nDISAGREE you={r.label} judge={r.judge}: {r.text[:70]}")
    print(f"  judge said: {r.judge_critique}")
rows.to_csv(f"evals/judge_{split}_results.csv", index=False)

    uv run python -m evals.judge_agreement dev

    About 16 Sonnet calls; a minute or two.

  3. Your turn: read every disagreement. For each, decide who is wrong. If the judge is, change the rubric or swap a train example, then run dev again. If you are, fix your label. Stop when both rates are around 0.8 or better, or after three rounds. Keep a log in the README: round, change, true positive rate, true negative rate.

  4. Only then, run the test split, once:

    uv run python -m evals.judge_agreement test

    This is the number you report. Don't tune on it; if you do, it stops being an honest estimate.

    In testing, on a deliberately tiny set (2 train, 9 test rows with only 2 fails), the first run gave true positive rate 0.71 true negative rate 1.00: the judge caught both bad answers and was stricter than the human on two good ones. With 2 fails, "1.00" could easily be luck; your 40 rows give a firmer number.

Done when: your README has the judge's true positive and true negative rates on the test split, how many rows they're based on, and your tuning log.

Stuck?
  • The judge fails almost everything: it's being stricter than you. Look for a rubric line it reads more harshly, often rule 1, and add a train example of a hedged but acceptable answer.
  • The judge passes almost everything: your fails are subtle. Put your two clearest fails in train, and make the rubric name the failure ("treating a normal feature of genuine messages as suspicious").
  • ValidationError on JudgeVerdict: print reply.data. Usually the schema wasn't passed.
  • It's slow or you hit usage limits: run with model="haiku" while you debug the script, then switch back for the real numbers.
  • "need both passes and fails": that split has no fails. Go back to Tuesday's step 4.

Thursday — CI: tests on every push, and the eval gate

Learn (75 min):

Build:

  1. Check that job 1 will pass before you push it. These are exactly the commands CI will run:

    uv sync --locked
uv run ruff format --check .
uv run ruff check .
uv run mypy src
uv run pytest

    uv sync --locked fails if uv.lock doesn't match pyproject.toml; that catches "works on my laptop" before CI does. If any test needs the Claude CLI or the internet, it's not mocked. Find it now: CI has neither.

  2. The first workflow, .github/workflows/ci.yml:

    name: CI

on:
  push:
  pull_request:

permissions:
  contents: read

jobs:
  checks:
    name: Lint, types, tests
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
      - name: Install the project
        run: uv sync --locked
      - name: Format
        run: uv run ruff format --check .
      - name: Lint
        run: uv run ruff check .
      - name: Types
        run: uv run mypy src
      - name: Tests (LLM mocked)
        run: uv run pytest

    Reading it top to bottom:

    • on: push runs it on every push, to any branch; pull_request also on pull requests.
    • permissions: contents: read lets the workflow read your code and nothing else. Give every workflow the least it needs.
    • runs-on: ubuntu-latest is the clean Linux machine. steps run in order; if one fails, the rest are skipped and the commit gets a red cross.
    • uses: runs an action, a reusable step someone else wrote: one copies your repo onto the machine, the other installs uv. The long string after @ is the exact version (a commit), and the comment says which release it is. Week 8 explains why that's safer than writing @v7.

  3. Push, then watch it run:

    git add .github/workflows/ci.yml
git commit -m "Run lint, types and tests in CI"
git push
gh run watch

    Or open the "Actions" tab of your repo on GitHub. A green tick means all steps passed.

  4. Now the gate. Write evals/thresholds.json; the numbers are placeholders until step 6:

    {
  "gate_set": "data/labelled100.csv",
  "limits": {
    "scam_recall": 0.90,
    "precision": 0.85
  }
}

  5. The gate, evals/gate.py. It runs the checker on the gate set, prints recall and precision, and exits with code 1 if either is below its limit. A non-zero exit code is how any program tells CI "this step failed":

    """The CI quality gate: fail the build if scam recall or precision drops.

Run with: uv run python -m evals.gate
Reads the limits from evals/thresholds.json. Exits with code 1 if any is missed.
"""

import json
import sys
from concurrent.futures import ThreadPoolExecutor
from pathlib import Path

import pandas as pd

from scam_checker.checker import check_message
from scam_checker.metrics import precision_recall

THRESHOLDS = Path("evals/thresholds.json")


def gate_failures(metrics: dict[str, float], limits: dict[str, float]) -> list[str]:
    """Every metric that is below its limit, as a readable line."""
    return [
        f"{name} {metrics[name]:.2f} is below the limit {limit:.2f}"
        for name, limit in limits.items()
        if metrics[name] < limit
    ]


def main() -> int:
    config = json.loads(THRESHOLDS.read_text())
    df = pd.read_csv(config["gate_set"])
    with ThreadPoolExecutor(max_workers=4) as pool:
        verdicts = [r.verdict for r in pool.map(check_message, df["text"])]

    df["verdict"] = verdicts
    df.to_csv("evals/gate_results.csv", index=False)
    precision, recall = precision_recall(
        df["is_scam"].tolist(), [v == "scam" for v in verdicts]
    )
    metrics = {"scam_recall": recall, "precision": precision}
    print(f"{len(df)} messages: scam recall {recall:.2f}, precision {precision:.2f}")

    failures = gate_failures(metrics, config["limits"])
    for line in failures:
        print(f"GATE FAILED: {line}")
    if not failures:
        print("Gate passed.")
    return 1 if failures else 0


if __name__ == "__main__":
    sys.exit(main())

    This gates check_message, one call per message. If your app now answers with run_agent, gate that instead: it's what users get, but it costs several calls per message. Note your choice in the README.

    Its logic gets a test with no model at all, tests/test_gate.py:

    from evals.gate import gate_failures


def test_gate_passes_at_the_limit():
    assert (
        gate_failures({"scam_recall": 0.9, "precision": 0.9}, {"scam_recall": 0.9})
        == []
    )


def test_gate_fails_below_the_limit():
    failures = gate_failures(
        {"scam_recall": 0.8, "precision": 0.95}, {"scam_recall": 0.9}
    )
    assert failures == ["scam_recall 0.80 is below the limit 0.90"]

  6. Your turn: measure the noise, then set the limits. Run the gate three times locally:

    uv run python -m evals.gate

    Each run is about 100 calls and takes a few minutes. Write the three recall and precision numbers in the README. Set each limit a little below the lowest of the three runs, so the gate fails on a real drop, not on a bad day. Remember the arithmetic: with 50 scams in the set, one extra miss moves recall by 0.02.

  7. Give CI a Claude login. On your laptop:

    claude setup-token

    It opens your browser; approve, and it prints a long token that's valid for a year and runs on your subscription. It's not saved anywhere, so go straight to the next command, which asks for the value and doesn't show what you paste:

    gh secret set CLAUDE_CODE_OAUTH_TOKEN

    Rules for this token, and every secret from now on:

    • never put it in a file in the repo, not even "temporarily": the repo is public and git keeps history
    • never echo or print it in a workflow. GitHub hides exact matches in logs, but don't rely on that
    • GitHub doesn't pass secrets to workflows started from someone else's copy (fork) of your repo, so strangers' pull requests can't use it
    • CI usage counts against your subscription limits, shared with your own use. That's why the gate runs only when it matters (next step)
    • in week 9 the deployed app moves to an API key with a spend cap, and CI switches to that key too

  8. The gate workflow, .github/workflows/eval-gate.yml. It's a separate file so that a paths: filter can limit it to pushes that change the prompt, model, checker, tools or eval files. workflow_dispatch adds a "Run workflow" button for running it by hand:

    name: Eval gate

on:
  push:
    paths:
      - "src/scam_checker/checker.py"
      - "src/scam_checker/llm.py"
      - "src/scam_checker/models.py"
      - "src/scam_checker/agent.py"
      - "src/scam_checker/tools/**"
      - "evals/**"
      - "data/labelled100.csv"
      - ".github/workflows/eval-gate.yml"
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: eval-gate
  cancel-in-progress: true

jobs:
  gate:
    name: Scam recall and precision
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: astral-sh/setup-uv@1c37ad07a6a961277cf70c0d37d6f313000f5884 # v10.3.0
      - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
        with:
          node-version: "24"
      - name: Install Claude Code
        run: npm install -g @anthropic-ai/claude-code
      - name: Install the project
        run: uv sync --locked
      - name: Run the gate
        env:
          CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
        run: uv run python -m evals.gate
      - name: Keep the results
        if: always()
        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
        with:
          name: gate-results
          path: evals/gate_results.csv

    What's new here:

    • The Claude CLI is installed with npm, Node's package installer, as the Claude Code setup page describes. The CLI reads CLAUDE_CODE_OAUTH_TOKEN from its environment, so your call_claude() works unchanged.
    • env: puts the secret into the environment of that one step only.
    • concurrency with cancel-in-progress stops an older gate run when you push again, so two runs don't both use your limits.
    • if: always() saves gate_results.csv even when the gate fails, so you can download it from the run's page and see which messages it missed.

    actionlint checks workflow files for mistakes before you push. Install it once with brew install actionlint, then run actionlint in the project folder; no output means no problems.

  9. Push both files and the thresholds. Then break it on purpose: on a branch, weaken the prompt (add "Only call something a scam if it asks for a password."), push, and watch the gate fail with GATE FAILED: scam_recall .... Screenshot it for Friday, then delete the branch.

Done when: job 1 is green on main, the gate has passed once from the "Run workflow" button, and you have a screenshot of it failing on the weakened prompt.

Stuck?
  • error: The lockfile at uv.lock needs to be updated: run uv lock locally, commit uv.lock, push again.
  • Library stubs not installed for "pandas" from mypy: uv add --dev pandas-stubs, then commit pyproject.toml and uv.lock.
  • FileNotFoundError: [Errno 2] No such file or directory: 'claude' in job 1: a test calls the real call_claude(). Mock it in that test, as in week 2.
  • A test needs Postgres (week 4's retrieval): CI has no database. Either mock search_patterns in that test, or mark it @pytest.mark.skipif(os.environ.get("CI") == "true", reason="needs Postgres"). GitHub sets CI=true on its runners.
  • The gate step says it isn't logged in or gets an authentication error: the secret name must be exactly CLAUDE_CODE_OAUTH_TOKEN. Check with gh secret list; it shows names, never values. If the token expired after a year, run claude setup-token and gh secret set again.
  • The gate didn't run after a push: you didn't change any file in paths:. Use "Run workflow" in the Actions tab, or gh workflow run eval-gate.yml.
  • ModuleNotFoundError: No module named 'evals' in CI: evals/__init__.py isn't committed.
  • The gate fails randomly: your limits are inside the noise. Re-measure and lower them a little, or grow the gate set.

Two files instead of one

You could put the gate in ci.yml as a second job. Running a job only when certain files changed then needs an extra third-party action. A separate workflow uses GitHub's built-in paths: filter, with fewer moving parts and one less outside dependency, which matters in week 8.

Friday — Demo day, and your second write-up

Learn (30 min): re-read your week 3 write-up, and look at how Hamel's guide presents its agreement numbers: what was measured, on how many examples, and what it means.

Demo day (9:00–12:00):

  1. Record a 3-minute screen video: the promptfoo table comparing two prompts, the judge's agreement numbers, a green CI run, and the gate failing on the weakened prompt.
  2. Write five lines on what you learned this week.
  3. Send both to Claude with the Friday check-in.

Build (after demo day):

  1. Your turn: write and publish "How I evaluated an anti-scam checker", in the same place as your week 3 write-up. About 800–1,200 words, with your real numbers:

    • the problem: why a missed scam and a false alarm cost different things
    • the labelled data (week 5): how many messages, how you labelled them, the failure types you found
    • code checks: what they catch, and the invented-phone-number rate before and after the prompt fix
    • the judge: what it grades, why pass/fail, and its true positive and true negative rates on held-back labels
    • the gate: what it blocks, how you set the limit from measured noise, and the screenshot of it failing
    • what you'd do next

    Every number needs its sample size next to it ("0.92 recall on 48 scams").

  2. Link the write-up from your README, and commit.

Deliverable checklist

  • evals/checks.py with four code-based checks, tested, and their pass rates in the README
  • promptfoo comparing two prompts through your own Python provider, on at least 10 cases
  • evals/judge_labels.csv: 40+ answers graded by you, with critiques and a train/dev/test split
  • evals/judge.py with its true positive and true negative rates on the test split, and a tuning log
  • .github/workflows/ci.yml green on main: format, lint, types and tests with the LLM mocked
  • .github/workflows/eval-gate.yml and evals/thresholds.json, with limits set from three measured runs
  • A screenshot of the gate failing on a weakened prompt
  • "How I evaluated an anti-scam checker" published and linked from the README
  • Demo video, five lines, and the Friday check-in sent

If you have time left: run the judge over a whole week 5 results file (not just your labels) and report the share of answers that pass, next to the judge's true positive and true negative rates. Hamel's guide explains why the raw pass rate needs those two numbers beside it before you can quote it.