Introduction

This is a course, not a manual. By the end you will have built Panoptes — a typed, reproducible evaluation harness in Rust — and, more importantly, you will understand why every piece is shaped the way it is. We are going to move the way a good pair-programming session moves: I frame a concept, you ask questions until it is solid, then we build the piece together and the compiler grades the work.

What we are building

Panoptes is the execution engine behind an SDA decision benchmark. It does four things, and each maps to one arc of this course:

1. Generate parameterized scenario vignettes from templates and a parameter grid.
2. Execute those vignettes against version-pinned model APIs, logging every raw response.
3. Code the responses against a codebook, with human judgment recorded in types that reject invalid values.
4. Validate coder agreement (this last stage hands off to Python).

Stages 5 and 6 — statistical analysis and reporting — stay in Python and read the files this harness produces. We will not build them here; we will build the thing that produces their inputs.

The end state, held in view

Everything we build serves one payoff you should keep in mind from the very first chapter: a coder will be structurally unable to record a value that is not in the codebook. Not "discouraged from," not "warned against" — unable, because the value will fail to parse. That property is what makes this instrument defensible, and it falls out almost for free once you understand Rust's type system and the serde library. When we reach it in Part II, it should feel inevitable rather than clever.

How the arcs are sequenced

The order is chosen for understanding, not for delivery speed. Each arc teaches one core idea before the next arc depends on it:

• Part I, Foundations — the test loop, then ownership, borrowing, and moves. The mental model everything else assumes. New to Rust starts here.
• Part II, Data Model — serde, derive macros, enums-as-validation. Go slow here.
• Part III, Generation — traits, the config boundary, iterators. Modeling the problem in types.
• Part IV, Async — futures, await, tokio, mock-server testing. The genuinely hard arc; budget extra time.
• Part V, Coding — types for correctness properties you personally care about (blinding, parse-is-validation).
• Part VI, Hardening — workspace hygiene, lints, the language boundary.
• Part VII, CLI — unifying the binaries into one panoptes command with proper exit codes. Where it becomes a tool.

A note on where you are starting from

You come to this with real strength in one area and newness in another, and it helps to be honest about both. You have built LLM-powered systems and agents, so the domain of this harness — models, prompts, evaluation, the shape of the problem — is familiar ground. What is new is Rust itself, and specifically its ownership model and its async story. That is exactly why this course front-loads concepts and adds a dedicated foundations arc before we write types in anger: we are not going to assume you already think in ownership and borrowing. We are going to build that mental model deliberately, because everything downstream — and especially the async arc — rests on it.

If a chapter feels hard, that is usually not a signal about your ability; it is the compiler teaching you a rule you have not yet internalized. The whole method is designed around making those lessons fast and legible rather than frustrating.

Turn the page to see how the course is structured and how to run the build loop, then we start with Phase 0.

How to Use This Course

The two chapter types

Every arc alternates between two kinds of chapter, and they ask different things of you.

Concept chapters are the lecture. Read them without touching the keyboard. They end with a short list of questions you should be able to answer for yourself — if any are fuzzy, that is the signal to stop and ask before moving to the build. Do not proceed to a build chapter on a shaky concept; the whole point of frontloading concepts is that the build then feels obvious.

Build chapters are the pair-programming session. Here you write the code. Each one follows the same test-driven loop:

1. Write the failing test. You write it, from the behavior we described — not by copying an answer.
2. Predict the failure. Say out loud (or note down) what the compiler or test runner will report.
3. Run it and check your prediction. The gap between prediction and reality is the lesson.
4. Write the minimal implementation. Just enough to make the test pass. No more.
5. Run again, watch it pass.
6. Commit.

The concept-check quizzes

Each part ends with a graded quiz. These are not busywork — they target the exact misconceptions that cause bugs three chapters later. Answer honestly before revealing; a wrong answer with a good explanation teaches more than a lucky guess. The score is for you, not for anyone else.

The answer key

There is a companion implementation plan (linked in the appendix) that contains the full, worked version of every task. Use it as an answer key, not a script. Try each build yourself first. Check the plan after. The distance between your version and the plan's version is precisely where the learning lives — if they match, you understood it; if they differ, the difference is worth investigating.

Working setup

You will want two things open side by side: this book, and a terminal in your project directory. A split screen or two monitors. The loop is fast — write test, run, read error, fix — and it only works if the feedback is immediate.

When you get stuck

Bring the exact error message, verbatim. Rust's compiler errors are unusually good; most of the time the fix is in the error itself, and learning to read them is half of learning the language. When we work through a stuck point together, we dig into what the compiler is actually objecting to rather than papering over it.

Phase 0: The Toolchain and the Loop

Before any Panoptes code, we verify one thing: that you can write a failing test, see it fail legibly, fix it, and see it pass. The entire course runs on that loop being fast and trustworthy. This phase is deliberately trivial — that is the point. We are testing the machinery, not your ability.

Objective

• Confirm a working Rust toolchain (rustc, cargo).
• Create a throwaway crate.
• Write one deliberately failing test, watch it fail, fix it, watch it pass.
• Internalize what cargo test output looks like when things break.

Install the toolchain

Rust is installed through rustup, which manages compiler versions:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# accept the defaults, then:
source "$HOME/.cargo/env"
rustc --version
cargo --version

You want a recent stable toolchain. If you already have one, rustup update brings it current.

Create a throwaway crate

cargo new hello-loop
cd hello-loop

cargo new scaffolds a tiny package: a Cargo.toml manifest and a src/main.rs. Open src/main.rs. Replace its contents with a single function and a test:

fn add(a: i32, b: i32) -> i32 {
    a + b
}

fn main() {
    println!("{}", add(2, 2));
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn two_plus_two_is_five() {
        assert_eq!(add(2, 2), 5); // deliberately wrong
    }
}

The failing run

cargo test

You will see something close to this:

running 1 test
test tests::two_plus_two_is_five ... FAILED

failures:

---- tests::two_plus_two_is_five stdout ----
thread 'tests::two_plus_two_is_five' panicked at src/main.rs:15:9:
assertion `left == right` failed
  left: 4
 right: 5

Read that carefully, because you will read a hundred of these. It tells you the exact test, the exact file and line, and — critically — left: 4, right: 5, the actual value against the expected value. assert_eq! always reports both sides. This is why we write assertions with the computed value on the left and the expected on the right: the output reads naturally.

The fix

Change the 5 to a 4:

#![allow(unused)]
fn main() {
        assert_eq!(add(2, 2), 4);
}

Run again:

test tests::two_plus_two_is_five ... ok

test result: ok. 1 passed; 0 failed; 0 ignored

That is the loop. Every build chapter is this, with more interesting types in the middle.

What just happened, named

• #[cfg(test)] means "only compile this module when running tests" — your test code is not in your shipped binary.
• mod tests { } is a module, a namespace. Tests conventionally live in a nested tests module.
• use super::*; pulls in everything from the parent module so the test can see add.
• #[test] marks a function as a test case for cargo test to discover and run.

Every one of these appears in the very first Panoptes task, so you have now seen the whole skeleton.

Done when

You can run cargo test, read a failure, and fix it without thinking about the mechanics. When that loop is automatic, turn the page: Part II begins with the concept that makes the whole harness worth building in Rust.

Concept: Ownership and Moves

Kind: Concept (read, do not code). New to Rust: this is the chapter everything else assumes.

Rust's one genuinely unfamiliar idea, the thing that has no direct equivalent in Python or most languages you have used, is ownership. Every other Rust concept in this course is ordinary once ownership is solid. So we go slow here, and we build a mental model you can actually reason with, not a rule you memorize.

The one rule

Here is the whole thing in a sentence, from Rust for Rustaceans: Rust's memory model centers on the idea that all values have a single owner — exactly one location, usually a scope, is responsible for ultimately deallocating each value.

Read that twice. Every value has exactly one owner. When the owner goes away (its scope ends), the value is cleaned up — "dropped" — automatically. No garbage collector deciding later, no manual free. The owner's scope ending is the cleanup signal.

What "move" means

Now the consequence that trips up everyone new to Rust. What happens when you assign a value to a new variable, or pass it to a function? For most types, the value moves. The book again: if the value is moved — by assigning it to a new variable, pushing it to a vector, or placing it on the heap — ownership moves from the old location to the new one, and you can no longer access the value through variables that flow from the original owner, even though the bits are technically still there.

This is the shock. In Python, b = a gives you two names for the same object. In Rust, for an owned type, let b = a; moves the value into b, and a is now unusable. Not copied — moved. The compiler will reject any later use of a with a clear error. Ownership transferred.

#![allow(unused)]
fn main() {
let s1 = String::from("hello");
let s2 = s1;          // the String moves from s1 into s2
// println!("{}", s1); // COMPILE ERROR: s1's value was moved into s2
println!("{}", s2);    // fine — s2 owns it now
}

The escape hatch: Copy

Some small types do not move — they copy. The book calls them "rebels": if a value's type implements the special Copy trait, the value is not considered to have moved even if reassigned; instead it is copied, and both locations remain accessible. Integers, floats, booleans, and small types made only of those are Copy. That is why this works fine:

#![allow(unused)]
fn main() {
let x = 42;
let y = x;             // i32 is Copy — x is COPIED into y
println!("{} {}", x, y); // both fine, both hold 42
}

The rule for what can be Copy is precise and worth internalizing: to be Copy, it must be possible to duplicate the value simply by copying its bits — which excludes any type that owns a resource it must deallocate when dropped. A String owns heap memory, so it can never be Copy. An i32 is just bits, so it is. This is exactly why, back in Part II, the small Params struct — made only of a u8, two simple enums, and a bool — can derive Copy, while a Vignette containing a String cannot.

The flows mental model

The single most useful way to think about all of this, and the model the borrow checker actually uses internally, is flows. Picture each value as having a line — a flow — that starts when the value is created and traces through your program to its last use. A move ends one flow and starts another. Using a variable after its flow has ended (because the value moved away) is the error the compiler catches. Hold this picture; in the next chapter it makes borrowing click immediately.

Questions to lock

Genuinely stop on each. This is the foundation.

1. In let b = a; where a is a String, what happens to a? Why? What would happen instead if a were an i32?
2. Why can a type that owns heap memory (like String) never be Copy?
3. What is the connection between "every value has one owner" and "Rust needs no garbage collector"?

Next chapter: borrowing — how you let code use a value without taking ownership of it, which is what most function calls actually do.

Concept: Borrowing and References

Kind: Concept. Builds directly on the previous chapter — read that first.

If every value has one owner and moving transfers it, how does a function use your value without stealing it? The answer is borrowing, and it is what the overwhelming majority of Rust function signatures actually do.

Lending without giving up ownership

From Rust for Rustaceans: Rust allows the owner of a value to lend out that value to others, without giving up ownership, through references. References are pointers that come with an additional contract for how they can be used.

A reference, written &value, lets code look at (or modify) a value it does not own. When the reference goes away, nothing is dropped — the reference never owned the value, so there is nothing to clean up. The owner keeps ownership the whole time. This is why you will see & everywhere: passing &params to a function lets it read your Params without moving it, so you can keep using params afterward.

Two kinds of reference, one iron rule

There are exactly two kinds, and the distinction is the whole game.

Shared references (&T) — read-only, and you can have as many as you like at once. The book: a shared reference is a pointer that may be shared; any number of other references may exist to the same value, and values behind shared references are not mutable. Many readers, no writers.

Mutable references (&mut T) — read-write, but exclusive. While a &mut exists, nothing else may touch the value. The book: the compiler assumes that the mutable reference is exclusive — no other reference, shared or mutable, may coexist with it.

Put together, this is the rule the borrow checker enforces everywhere: either any number of shared (&) references, or exactly one mutable (&mut) reference — never both at once. Many readers or one writer. Not both.

Back to the flows model

Remember flows from the last chapter — each value's line from creation to last use. Borrowing adds flows too: a shared borrow starts a flow that must not overlap a mutable one. The borrow checker's job, in the book's words, is to check that there cannot be two parallel flows with mutable access to a value, nor a flow that borrows a value while there is no flow that owns the value. When you see "cannot borrow as mutable because it is also borrowed as immutable," that is two flows illegally overlapping. The fix is almost always to let one flow end (stop using the first reference) before the other begins.

Where this lands in Panoptes

You do not need to fight the borrow checker much in this harness — the code is mostly straightforward — but you will read & and &mut constantly and need to know why each is there:

• ModelClient::generate(&self, prompt: &str) takes a shared reference to the client and a shared reference to the prompt string: it reads both, owns neither, so the caller keeps them.
• The append-only log writer opens a file and writes — the exclusivity of &mut on the file handle is what makes "one writer at a time" a compile-time fact.
• Passing &vignettes to the dispatch loop lets it read every vignette without moving the vector, so it is still yours afterward.

When an async error in Part IV mentions lifetimes or borrows, this is the model to reach for: which flow is this, and does it overlap another?

Questions to lock

1. What does passing &params to a function let the function do, and what can you still do with params afterward?
2. State the borrow rule in one sentence. Why does forbidding "shared and mutable at once" prevent data races?
3. In the flows model, what illegal situation does the error "cannot borrow as mutable because it is also borrowed as immutable" describe?

That is the foundation. With ownership, moves, and borrowing solid, the rest of the course is mostly applying them. Part II begins — the data model, and the serde concept that makes the whole harness worth building in Rust.

Concept: serde and the derive Macro

This is the data-model arc, and everything in it exists to earn one payoff. This chapter builds the foundation for that payoff: understanding what serde is, what a derive macro actually does, and why generating conversion code at compile time changes when your bugs surface.

The problem serde solves

The harness lives or dies on moving structured data across boundaries. A scenario becomes a prompt file. A model response becomes a line in a log. A coder's judgment becomes a row in a sheet. Every one of those is the same underlying operation: a Rust value in memory turning into text on disk, or text on disk turning back into a Rust value.

In Python you would reach for json.dumps and json.loads and mostly not think about it, because Python does not care what shape the thing is until it breaks at runtime. If a field is missing or the wrong type, you find out when the code hits that line — often in production, often at the worst time.

serde is Rust's answer. The name is just "serialize / deserialize." It is a library, not a language feature, but it is so universal it might as well be built in.

The part that looks like magic

Here is the thing that feels like magic at first, and that is worth demystifying now so it does not stay magic: you almost never write the conversion code yourself. You annotate a struct like this —

#![allow(unused)]
fn main() {
use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize)]
struct Params {
    attribution_confidence: u8,
    info_request: bool,
}
}

— and serde generates the conversion for you, tailored to that exact type. The code that turns a Params into JSON is written, by the macro, before your program ever starts. There is no reflection, no runtime inspection, no dictionary of fields being walked while your program runs, the way Python does it.

What "derive" actually is

This is the piece most people treat as incantation. Let us not.

A derive macro is a code generator that runs during compilation. When you write #[derive(Serialize)] above a struct, you are telling the compiler: look at the fields of this struct, and write the serialization function for me. The macro reads your struct's shape — its field names and their types — and emits real Rust source code: an implementation of the Serialize trait. That generated code then gets compiled right alongside everything you wrote by hand.

You have already met this mechanism without thinking about it. #[derive(Debug)], which lets you print a value with {:?} for inspection, is the same thing: it generates the code that formats your type for debugging. #[derive(Clone)] generates the code to duplicate it. serde's macros are more elaborate, but mechanically identical: read the type, emit the code.

Why this matters: bugs move to compile time

Here is the consequence that makes serde the right tool for a reproducibility-critical instrument. Because the generation happens at compile time against your specific type, the compiler can see the whole thing. If a field's type cannot be serialized, you find out when you build — not when a log write fails at two in the morning during your evaluation run.

This is the first appearance of a theme that runs through the entire course:

Why this arc is the foundation

Hold the end state in view, because it is why we start here. In a few chapters, your codebook criteria become Rust enums — StrategicLogic with variants Control, Maritime, Political, and so on. The reason that works — the reason a coder literally cannot record a value outside the codebook — is that serde's deserialization of an enum will reject any string that does not match a variant.

That rejection is generated by the same derive mechanism we just discussed, applied to an enum instead of a struct. So the "parse is validation" property that makes this whole harness worth building in Rust is not a separate trick you will learn later. It is this concept — serde derive — pointed at an enum. Get this arc solid and that payoff is almost free. Rush it and the payoff will feel like luck.

Questions to lock before the build

Genuinely stop and make sure each of these is crisp. If one is fuzzy, that is the signal to re-read or ask.

1. Why does generating conversion code at compile time, rather than inspecting the value at runtime like Python does, let the compiler catch a whole class of bugs before the program runs?
2. What is a derive macro actually doing to your struct, mechanically? (If your answer is "it adds serialization," push further: how?)
3. Can you already see, at least in outline, why an enum plus serde would refuse an out-of-codebook value — even though we have not written that code yet?

Next chapter is the first build: we create the workspace and the parameter types, and you write the first failing test.

Build: Workspace + Parameter Types

Maps to: Task 1 in the task plan. Kind: Build (you write the code).

Objective

Stand up the Cargo workspace and create panoptes-core, the crate that will hold every type crossing a boundary. Define the scenario parameter enums (TimePressure, Reversibility) and the Params struct, and prove with tests that they round-trip through strings and JSON.

Concepts exercised

• Cargo workspaces and [workspace.dependencies] for shared version pinning.
• #[derive(...)] stacking: Serialize, Deserialize, Debug, Clone, Copy, PartialEq.
• strum's Display + EnumString for enum↔string conversion.

The build loop (you drive)

1. Write the failing test for TimePressure string round-tripping (Hours ↔ "HOURS") and for Params JSON round-tripping. Derive them from the behavior described, not from the answer key.
2. Predict the failure — will it fail to compile or fail at runtime? (Hint: the type does not exist yet. What does the compiler say about that?)
3. Run, check the prediction.
4. Implement the enums and struct with the minimal derives to pass.
5. Run green, then commit.

Done when

cargo test -p panoptes-core params shows two passing tests, and you can explain why Copy is safe to derive on these types (all fields are themselves Copy).

Check yourself

Compare against Task 1 in the appendix task plan. If your derives differ, work out whether the difference matters — some are load-bearing (Deserialize), some are ergonomic (Copy).

Concept: Enums as Validation

Kind: Concept (read, do not code).

This is the payoff chapter the whole arc has been building toward. The idea is small and the consequence is large.

The core idea

A Rust enum is a closed set of named alternatives. StrategicLogic is Control or Maritime or Political or Procedural or None or Mixed — and nothing else can exist. There is no seventh value. The compiler enforces this everywhere the type is used.

Now combine that with what you learned about serde: when serde deserializes a string into an enum, it matches the string against the known variants. A string that matches no variant is a deserialization error. Not a warning. Not a silently-accepted fallback. An error that stops the parse.

Why this is the instrument's spine

In the Python sketch of this harness, validating a coded value meant a separate lint.py script that checked each value against an allowed list — a step you had to remember to run, that ran after the data already existed in a possibly-invalid state.

In Rust, the allowed list is the type. A coded row whose c2_logic field is "AGGRESSIVE" does not become an invalid CodedRow that you later catch — it fails to become a CodedRow at all. The validation is not a step in the pipeline; it is a property of the boundary. Parsing the file is validating it.

Connecting back to the thesis

Recall why rubric validity is the primary intellectual risk: if the codebook reads as one person's opinion, the thesis fails at defense. The enum does not make the criteria defensible — that is the doctrinal grounding and the inter-rater reliability. But it does guarantee that the recorded data conforms exactly to the codebook you defined, with zero drift, which is one less thing a committee can poke. The instrument's categories are frozen in the type.

Questions to lock

1. What is the difference between "validate the value after parsing" and "parse in a way that rejects invalid values"? Why does the second eliminate a class of bugs the first cannot?
2. If StrategicLogic has six variants, what happens — mechanically — when serde tries to deserialize "CONTROL"? What happens with "AGGRESSIVE"?
3. Why does putting the codebook's allowed values in an enum mean no downstream code ever needs to re-check them?

Next: we turn StrategicLogic and friends into real code, and write the test that proves an invalid value is rejected.

Build: The Codebook Types

Maps to: Task 2. Kind: Build.

Objective

Create codes.rs in panoptes-core: the ActionType and StrategicLogic enums and the CodedRow struct. Write the test that is the whole reason we chose Rust — an invalid codebook value must fail to deserialize.

Concepts exercised

• #[strum(serialize_all = "SCREAMING_SNAKE_CASE")] to match codebook string conventions.
• serde_with::NoneAsEmptyString for optional free-text fields.
• The negative test: asserting that a parse fails.

The build loop (you drive)

1. Write three failing tests: a valid value parses; an invalid value ("AGGRESSIVE") is rejected; a full CodedRow round-trips through JSON.
2. Predict: the invalid_logic_is_rejected test asserts result.is_err(). Before implementing, what makes it currently fail — the assertion, or the fact that the type does not compile yet?
3. Run, check.
4. Implement the enums and struct.
5. Run green, commit.

Done when

cargo test -p panoptes-core codes is green, and the invalid-value test proves the codebook constraint lives in the type rather than in a separate check.

Build: Records, Vignettes, the File Contract

Maps to: Task 3. Kind: Build.

Objective

Create vignette.rs and record.rs in panoptes-core. Define Vignette, the deterministic vignette_id function, Usage, and ResponseRecord — the struct that becomes one line in responses.jsonl, the dataset of record. These types are the file contract: the interface every later stage and the Python analysis tail depend on.

Concepts exercised

• chrono::DateTime<Utc> with serde support for timestamps.
• Deterministic ID construction from typed fields (no stringly-typed field access).
• Why the JSONL schema is the contract that makes the underlying tool fungible.

The build loop (you drive)

1. Write failing tests: vignette_id produces the exact formatted string (ca_geo-030-H-REV-INFO); ResponseRecord round-trips through a JSON line.
2. Predict what vignette_id returns for a given Params, character by character, before running.
3. Run, check.
4. Implement.
5. Run green, commit.

Done when

All of cargo test -p panoptes-core passes (params + codes + vignette + record). You now have the complete shared data model — every type the other three crates will import.

Concept-Check: Data Model

You have built the shared data model. Before moving to generation, confirm the ideas that the next arcs assume.

Concept: Traits and the Config Boundary

Kind: Concept.

Traits, briefly

A trait is a set of behaviors a type promises to provide — Rust's version of an interface. ScenarioFamily will require any family type to answer name() and is_valid(&Params). Code can then work with any family through the trait, without knowing which concrete family it is.

You have used traits already (Serialize, Display are traits). Here you define one for the first time in this project, which is a small but real step up.

The design decision that matters

The consequential idea in this chapter is not the trait mechanics — it is where the validity logic lives. In the Python sketch, families carried a list of string "valid_rules" that were evaluated at runtime. That means: logic expressed as data, interpreted while the program runs, with errors surfacing only when a bad rule is hit.

We are making the opposite choice. TOML holds content and metadata only — the prompt template, the doctrine references, the action menu. The logic of which parameter combinations are meaningful lives in Rust code, in each family's is_valid implementation. That means the compiler checks it, and a malformed rule is a compile error, not a runtime surprise.

Questions to lock

1. What does a trait let you do that you could not do by writing a function per concrete type?
2. Why is "which parameter combinations are valid" logic, and therefore code, rather than data that belongs in TOML?
3. What do you lose by moving that logic out of config, and why is the trade worth it for this project specifically?

Build: Family Spec + Validity Trait

Maps to: Task 4. Kind: Build.

Objective

Create the panoptes-gen crate. Define FamilySpec (the TOML shape, loaded with the toml crate), the ScenarioFamily trait, and a first concrete family CaGeo whose is_valid encodes a real exclusion rule. Write the example ca_geo.toml.

Why TOML for the spec: it is the dialect you already speak (Cargo.toml), the toml crate is what Cargo itself builds on, and TOML has no implicit typing — a level named "NO" can never silently become false the way it can in YAML. For an instrument whose configs are the experimental conditions, unambiguous scalars beat terseness.

Concepts exercised

• toml deserialization into a struct (same derive, different format).
• Defining and implementing a trait.
• Keeping content (TOML) and logic (Rust) on opposite sides of the boundary.

The build loop (you drive)

1. Write failing tests: FamilySpec::from_toml parses the fields; CaGeo::is_valid rejects the excluded combo and accepts valid ones.
2. Predict: the TOML parse test — what happens if a required field is missing from the TOML? Compile error or runtime Err? Why?
3. Run, check.
4. Implement.
5. Run green, commit.

Done when

cargo test -p panoptes-gen family and cargo test -p panoptes-gen validity pass. You can articulate why the same #[derive(Deserialize)] works for TOML and JSON alike (serde is format-agnostic; the derive describes the shape, the format crate handles the encoding).

Concept: Iterators and the Cartesian Product

Kind: Concept.

The idea

Generating vignettes means taking every combination of parameter values — 3 confidences × 2 time pressures × 2 reversibilities × 2 info-request states — and producing one vignette per valid combination. That "every combination" is a cartesian product, and Rust's iterator ecosystem expresses it cleanly.

itertools::iproduct! gives you the product as an iterator. You then .filter() out invalid combinations and .map() each surviving combination into a Vignette. The whole generation is a single iterator chain: product → filter → map → collect.

Why iterators, not loops

You could write four nested for loops. The iterator chain is preferred here for reasons that matter beyond style: it is lazy (nothing computes until you collect), it is composable (each stage is independent and testable), and it makes the shape of the computation legible — "the cartesian product, minus invalid combos, each turned into a vignette" reads directly off the code.

Questions to lock

1. What is a cartesian product, and why is it the right description of "all parameter combinations"?
2. What does "lazy" mean for an iterator chain, and why is collect() the moment the work actually happens?
3. Why is a filter → map chain easier to test than the equivalent nested loops?

Build: Vignette Generation

Maps to: Task 5. Kind: Build.

Objective

Write generate.rs: the all_params() iterator over the parameter grid, and generate() which filters by validity, renders each prompt through tera, hashes it, and produces Vignettes. Prove the counts and the uniqueness of IDs with tests.

Concepts exercised

• itertools::iproduct! over typed enum arrays.
• tera templating: injecting parameters into the prompt template.
• sha2 hashing for the prompt fingerprint.

The build loop (you drive)

1. Write failing tests: the raw grid has exactly 24 combinations; the validity filter reduces it to the expected number; all generated IDs are unique; a rendered prompt contains the injected parameter.
2. Predict: how many vignettes remain after CaGeo's filter removes the HOURS + NOINFO combos? Work it out from the grid dimensions before running.
3. Run, check the arithmetic against reality.
4. Implement.
5. Run green, commit.

Done when

cargo test -p panoptes-gen generate passes, including the uniqueness guard.

Build: The Generation CLI + Manifest

Maps to: Task 6. Kind: Build.

Objective

Add the panoptes-gen binary: a clap CLI that loads a family TOML, generates vignettes, writes each prompt to prompts/, and writes manifest.csv — the join spine every later stage reads. Test the manifest writer, then run the binary end to end.

Concepts exercised

• clap derive for command-line argument parsing.
• Filesystem writing and directory creation.
• The manifest as the single place parameters live for the join.

The build loop (you drive)

1. Write a failing test for write_manifest: header present, one row per vignette, IDs appear.
2. Predict the line count of the manifest for N vignettes (remember the header).
3. Run, check.
4. Implement, then run the actual binary: cargo run -p panoptes-gen -- --family scenarios/families/ca_geo.toml.
5. Verify the output files exist, commit.

Done when

The binary emits manifest.csv plus one prompt file per vignette, and you can point to the manifest as the reason Stage 5 analysis will be a simple join rather than forensic reconstruction.

Concept-Check: Generation

Concept: async, await, and the Runtime

Kind: Concept. This is the hard arc — read it twice.

The problem async solves

Your harness makes hundreds of API calls. Each one spends almost all its time waiting — for the network, for the model to generate. If you made them one at a time, synchronously, the program would sit idle during every wait. Async lets a single thread start a call, and while it waits, do other useful work (like starting the next call).

What async and await actually mean

An async fn does not run when you call it. It returns a future — a value representing a computation that is not finished yet. The future does nothing until it is driven. You drive it with .await, which means "pause here until this future is ready, and let other work proceed meanwhile."

This is the mental shift: in synchronous code, calling a function runs it. In async code, calling an async fn gives you a description of work; .await is what actually advances it.

The runtime

Futures need something to drive them — a scheduler that polls them, parks the ones that are waiting, and wakes them when their I/O is ready. That scheduler is the async runtime. Rust does not ship one in the standard library; the ecosystem standard is tokio. The #[tokio::main] attribute on main sets up the runtime so your top-level async code has something to run on.

The lifecycle, precisely

It is worth having the exact lifecycle in mind rather than a vague sense of "it runs later." Async Rust describes a future's life this way: when a future is created, it is idle — it has yet to be executed. Once executed, it can yield a value, resolve, or go to sleep because it is pending, and each subsequent poll returns either Pending or Ready until the future is resolved or cancelled. That polling loop — idle, then polled repeatedly, each poll answering "ready yet?" — is what the runtime is doing under every .await. You will rarely implement poll by hand, but knowing that .await is sugar over "poll this until it is Ready" demystifies the whole model.

The same book makes the concurrency payoff concrete with a kitchen analogy: several tasks each spending time waiting can overlap, because the executor sets a task to idle when it hits an await and works on the next task in the queue while polling the idle ones. That single sentence is why your harness benefits from async at all — while one API call waits on the network, another can be in flight.

Why this arc is genuinely harder

Async stacks several new things at once: futures, await, the runtime, and — next chapter — traits that contain async methods, plus the Send + Sync bounds that make futures safe to move between threads. Each is a real concept. Expect the compiler to be least forgiving here, and expect the errors to be longer.

The good news is that this arc rests directly on the ownership foundation from Part I. Most async borrow-checker fights are ownership and lifetime problems in disguise — a value moved into a future, a reference that does not live long enough — which is precisely the flows-and-moves model you already built. When an async error looks intimidating, the first move is to read it as an ownership question: what owns this value, and how long does this borrow need to live? The answer is usually there.

Questions to lock

1. What does an async fn return when you call it, and why does calling it not run the work?
2. What does .await do, and why does it let one thread stay busy during I/O waits?
3. What is the runtime's job, and why does #[tokio::main] exist?

Concept: Async Traits, Send + Sync

Kind: Concept.

Why this needs its own chapter

You want a ModelClient trait with an async fn generate. But plain Rust traits could not, for a long time, contain async methods directly, and even now the ergonomic path for a trait used as Box<dyn ModelClient> is the async_trait macro. It rewrites your async trait methods into a form that works with dynamic dispatch. You annotate the trait and its impls with #[async_trait] and write async methods as if it just worked.

Send + Sync, briefly

Because the runtime may move futures between threads, the things inside them must be safe to send across threads. Two marker traits express this: Send (safe to move to another thread) and Sync (safe to share by reference across threads). When you write Box<dyn ModelClient>, you will often need ModelClient: Send + Sync so the boxed clients can be used by the multi-threaded runtime.

You will not usually implement these — they are automatic for most types. But you will require them in bounds, and the compiler will tell you when a bound is missing, sometimes with an error that points several layers away from the real cause. Learning to read those is part of this arc.

Questions to lock

1. Why do we reach for the async_trait macro instead of just writing async fn in the trait?
2. What do Send and Sync guarantee, and why does the multi-threaded runtime need them for boxed clients?
3. What does Box<dyn ModelClient> buy us that a concrete type would not, for the dispatch loop?

Build: ModelClient + the Append-Only Log

Maps to: Task 7. Kind: Build.

Objective

Create the panoptes-harness crate. Define the ModelClient trait (with #[async_trait]) and ModelResponse. Write the append-only JSONL writer and the test that guards its most important property: a second append must not overwrite the first.

Concepts exercised

• Defining an async trait with async_trait.
• std::fs::OpenOptions with .append(true) for append-only semantics.
• Testing filesystem behavior with tempfile.

The build loop (you drive)

1. Write the failing test appends_without_truncating: write two records, assert the file has two lines and each is valid JSON.
2. Predict: what would the test show if you opened the file with .create(true).write(true) instead of .append(true)? (This is the bug the test exists to catch.)
3. Run, check.
4. Implement the writer.
5. Run green, commit.

Done when

cargo test -p panoptes-harness jsonl passes and the append-only property is proven.

Concept: Testing Against a Mock Server

Kind: Concept.

The idea

You need to test that your Anthropic client correctly builds a request and parses the response. You do not want those tests to hit the real API — that costs money, needs a key, is slow, and is non-deterministic. The solution is a mock server: a fake HTTP server, started inside the test, that returns a canned response you control.

wiremock spins up a real local HTTP server on a random port, lets you say "when a POST arrives at /v1/messages, respond with this JSON," and gives you its URL. You point your client at that URL instead of the real API. The client cannot tell the difference — it makes a genuine HTTP request; the server just happens to be under your control.

Why this is the right design pressure

To make this work, your client must accept its base URL as a parameter rather than hard-coding https://api.anthropic.com. That injectability is good design independent of testing — it is also how you would point at a proxy or a rented-inference gateway. The test forces a seam that turns out to be useful in production. Testable code and flexible code are frequently the same code.

Questions to lock

1. Why is hitting the real API in a unit test a bad idea, on at least three counts?
2. What does wiremock actually start, and why can the client not tell it is talking to a mock?
3. Why does mock-testing force the client to take its base URL as a parameter, and why is that good beyond testing?

Build: The Anthropic Client

Maps to: Task 8. Kind: Build.

Objective

Implement AnthropicClient — a concrete ModelClient that builds the request, sends it, and parses the completion into a ModelResponse. Test it entirely against a wiremock mock server, spending no API credits.

Concepts exercised

• reqwest JSON POST with headers.
• Deserializing a nested API response into typed structs.
• wiremock mock definitions and #[tokio::test].

The build loop (you drive)

1. Write the failing test parses_a_mocked_completion: mount a mock returning a known completion + usage, call generate, assert the parsed text and token counts.
2. Predict: the response JSON has a content array of blocks. What does your parsing do if the array is empty? Decide the behavior before implementing.
3. Run, check.
4. Implement the client with an injectable base_url.
5. Run green, commit.

Done when

cargo test -p panoptes-harness anthropic passes. Adding a second provider (OpenAI, a local gateway) is now "copy this file, change the request/response shapes" — the trait makes providers uniform.

Build: The Dispatch Loop + Run Binary

Maps to: Task 9. Kind: Build.

Objective

Write dispatch.rs: the opaque response_id function and the dispatch loop over vignettes × clients × epochs, writing one ResponseRecord per call. Then the panoptes-run binary that loads the manifest, builds the pinned client list, and runs the loop. Test the ID's opacity and the loop's record count against a mock.

Concepts exercised

• Iterating over &[Box<dyn ModelClient>] and calling an async trait method.
• Deterministic-but-opaque ID construction (the blinding property, in the harness).
• An integration test that drives the whole loop against a mock server.

The build loop (you drive)

1. Write failing tests: response_id_is_deterministic_and_opaque (same inputs → same id; epoch changes it; model name and params do NOT appear in it); dispatch_writes_one_record_per_call (2 vignettes × 1 client × 3 epochs → 6 records).
2. Predict: why must response_id be a hash rather than a readable concatenation? What property would a readable id break?
3. Run, check.
4. Implement dispatch and the binary.
5. Run green, commit.

Done when

cargo test -p panoptes-harness is fully green, including the opacity guard and the integration test.

Concept-Check: Async

The hardest arc. If these three are solid, you have the async model that the rest of the harness relies on.

Concept: Blinding as a Correctness Property

Kind: Concept.

The problem, stated honestly

You are both the person who built the harness and the person who codes the responses. You have hypotheses about which model reasons which way. If your coding sheet showed you "this response came from Claude at 95% attribution confidence," your knowledge could unconsciously shape how you score — and a committee member could rightly ask whether your codes reflect the responses or your expectations.

Blinding removes the possibility. The coding sheet shows only an opaque response_id and the response text. No model name, no parameters. You code what is on the page, and only after coding is complete do you join back through the blind key to learn which response was which.

Why this is a type/code property, not a discipline

You could try to blind yourself by "just not looking" at a column. That is a discipline, and disciplines fail. Instead we make it structural: the coding sheet is a type (BlankSheetRow) that does not contain the model or parameters. The blind key is a separate type (KeyRow) in a separate file that coders never open. The blinding is enforced by what the data structure is, not by what you remember to avoid.

The deterministic shuffle

There is a subtlety: even the order of responses could leak information (all of one model's responses appearing together). So sheets are shuffled — but with a seeded, deterministic shuffle, so the ordering is reproducible for the record while still breaking the model/parameter grouping. Reproducibility and blinding at once.

Questions to lock

1. Why is blinding a genuine threat to the thesis specifically, given who the coder is?
2. What is the difference between blinding-by-discipline ("I won't look") and blinding-by-type ("the field does not exist"), and why does the second actually hold?
3. Why shuffle the sheet, and why must the shuffle be deterministic rather than truly random?

Build: Blinded Coding Sheets

Maps to: Task 10. Kind: Build.

Objective

Create the panoptes-coding crate. Write make_sheet, which splits response records into a BlankSheetRow (opaque id + response text + empty code fields) and a separate KeyRow (the id→model/params mapping coders never see). Add the seeded shuffled_indices. Prove the sheet leaks nothing.

Concepts exercised

• Splitting one record into two types by what each audience may see.
• A seeded permutation without an external rand dependency (a small LCG).
• The blinding invariant as a test.

The build loop (you drive)

1. Write failing tests: sheet_does_not_leak_identity (serialized sheet contains no model name, no vignette id, no parameters, but the key retains them); shuffle_is_deterministic_and_a_permutation.
2. Predict: the leak test serializes the sheet to JSON and asserts the string does not contain "claude". What field on BlankSheetRow would make this test fail, and why is its absence the whole point?
3. Run, check.
4. Implement.
5. Run green, commit.

Done when

cargo test -p panoptes-coding sheets passes. The blinding property is now guaranteed by the type, not by your memory.

Build: The Coded-CSV Loader

Maps to: Task 11. Kind: Build.

Objective

Write validate.rs: load_coded, which parses each coded line into a CodedRow — so an off-codebook value fails here, at the boundary — and check_anchors, enforcing the codebook rule that latent codes carry an anchor quote. Add the panoptes-code CLI. This is where the enums-as-validation payoff becomes a runnable command.

Concepts exercised

• Parse-is-validation in practice: deserialization as the gate.
• Custom error types (thiserror) carrying line-number context.
• A CLI that exits non-zero on invalid data.

The build loop (you drive)

1. Write failing tests: a valid row loads; a bad-enum row fails with the right line number; a missing anchor quote is caught by check_anchors.
2. Predict: the bad-enum test feeds "c2_logic":"AGGRESSIVE". Where exactly does it fail — in load_coded's serde_json::from_str, or in check_anchors? Trace the path before running.
3. Run, check.
4. Implement, then exercise the CLI on a deliberately bad file and confirm a non-zero exit.
5. Run green, commit.

Done when

cargo test -p panoptes-coding validate passes and panoptes-code --coded bad.jsonl exits non-zero naming the offending line.

Concept-Check: Coding

Concept: Workspace Hygiene and the Language Boundary

Kind: Concept.

Two ideas in this short arc

Workspace hygiene. Lints are automated taste. Configuring deny(unused_must_use) turns the un-awaited-future mistake into a hard error across every crate. Setting clippy::unwrap_used to warn flags panics-in-waiting in library code (while leaving them fine in tests). Hygiene is what turns "code that compiles" into "code you would defend at a review."

The language boundary. This is the decision to keep Stages 5–6 in Python. It is not a failure of Rust ambition — it is choosing the right tool per stage. The statistics ecosystem for inter-rater reliability (Krippendorff's alpha, Cohen's kappa) is mature and correct in Python and thin in Rust. Reimplementing kappa by hand, in the crate whose entire purpose is a defensible instrument, would introduce exactly the kind of subtle correctness risk you are trying to eliminate. So the boundary is drawn at the file contract: Rust produces JSONL and CSV; Python reads them.

Questions to lock

1. What does a lint like deny(unused_must_use) buy you that code review alone does not?
2. Why is keeping reliability statistics in Python the rigorous choice, not the lazy one, for this specific instrument?
3. Why is "the interface is the file contract" the thing that makes the Rust/Python split painless?

Build: Lints, Handoff Contract, README

Maps to: Task 12. Kind: Build.

Objective

Add workspace-wide lints, write the Stage-4 reliability handoff contract (a README documenting what Python reads and produces — not implemented in Rust), and write the workspace README with the run sequence and the invariants. Then a full cargo build --workspace && cargo test --workspace && cargo clippy --workspace.

Concepts exercised

• [workspace.lints] and per-crate [lints] workspace = true.
• Documenting a language boundary as an explicit contract, not an afterthought.
• A full-workspace green build as the definition of done.

The build loop (you drive)

1. Add the workspace lints; wire each crate to inherit them.
2. Write reliability/README.md: the exact files Python reads (primary.csv, second_coder.csv, blind_key.csv), what it produces (irr_report.csv, disagreement dumps), and why Python.
3. Write the workspace README.md: the run sequence and the four invariants (append-only log, blinded sheets, enums-as-validation, manifest-as-join-spine).
4. Run the full-workspace build, test, and clippy. Fix any clippy warnings in library code.
5. Commit.

Done when

cargo build --workspace && cargo test --workspace && cargo clippy --workspace is clean, and the two READMEs make the pipeline runnable and the language boundary explicit.

Concept-Check: Hardening

Concept: Subcommands, Exit Codes, and Composability

Kind: Concept.

You have three binaries — panoptes-gen, panoptes-run, panoptes-code. That works, but a real tool presents one front door: panoptes generate, panoptes run, panoptes code, like git commit and git push. This arc unifies them, and along the way teaches two things that make a command-line tool well-behaved rather than merely functional.

Subcommands with clap

clap (which you already used for single binaries) models subcommands as an enum: each variant is a subcommand, each variant's fields are that subcommand's arguments. This is the same enum-as-closed-set idea from Part II, now applied to the shape of your CLI. The top-level parser dispatches on which variant it parsed, and you match on it to run the right stage. One binary, one help text, a discoverable set of verbs.

Exit codes: the part beginners skip

Here is the idea that separates a script from a tool. When your program finishes, it returns an exit code to the operating system — 0 for success, non-zero for failure. This is not decoration. From Command-Line Rust: correctly reporting the exit status is a characteristic of well-behaved command-line programs. The exit value is important because a failed process used in conjunction with another process should cause the combination to fail.

Concretely, exit codes are what let programs compose. The book shows it with the shell's &&: only if the first process reports success will the second process run. So this becomes possible:

panoptes generate --family ca_geo.toml && panoptes run --epochs 5

The run stage fires only if generate succeeded. If generation fails and exits non-zero, run never starts, and the whole line fails loudly. That is the behavior you want in a pipeline or a CI job — and it only works if each stage reports its status honestly.

Rust's ExitCode and Result-returning main

Modern Rust makes this ergonomic: main can return Result<(), E> (a non-Ok return becomes a non-zero exit automatically) or std::process::ExitCode for explicit control. You do not manage raw integers by hand; you return a type that carries success or failure, and the runtime translates it to the OS exit code. This is the same "push correctness into types" theme — even the process's success/failure is a typed value, not a convention you hope you remembered.

Questions to lock

1. Why model subcommands as an enum, and how does that connect to the enums-as-validation idea from Part II?
2. What is an exit code, and why is "report a non-zero code on failure" the property that makes programs composable?
3. How does panoptes code's non-zero exit on invalid data turn the codebook constraint into a pipeline gate?

Next: the build. We wrap the three stages behind one panoptes command and write an integration test that runs the whole pipeline end to end.

Build: The Unified panoptes Command

Maps to: Task 13 (new — extends the plan). Kind: Build.

Objective

Create a top-level panoptes binary crate whose CLI is a clap subcommand enum wrapping the three stages: generate, run, and code. Make main return a type that produces the right OS exit code, and write an integration test (with assert_cmd) that runs the whole pipeline — generate, then run against a mock, then validate — and asserts the exit codes.

Concepts exercised

• clap derive with a #[command(subcommand)] enum.
• main returning Result / ExitCode for honest exit status.
• Integration tests in tests/ that invoke the built binary with assert_cmd.
• Composability: proving a failed stage produces a non-zero exit.

File structure

crates/panoptes-cli/
├── Cargo.toml            # depends on gen, harness, coding
└── src/
    └── main.rs           # the subcommand enum + dispatch
tests/
└── pipeline.rs           # end-to-end integration test

The three existing binaries can stay (they are handy for focused runs), or become thin shims. The new panoptes command is the front door.

The build loop (you drive)

1. Write the failing integration test first, in tests/pipeline.rs:
   • panoptes generate --family ... exits 0 and produces a manifest.
   • panoptes code --coded <a-known-bad-file> exits non-zero (the composability guarantee).
   • Optionally, the full happy-path chain runs green.
2. Predict: assert_cmd runs the compiled binary as a subprocess. Before running, what does the test assert about the bad case — a specific exit code, or merely "failure"? Decide what "failure" should mean for panoptes code.
3. Run, check — it fails because the panoptes binary does not exist yet.
4. Implement the subcommand enum and dispatch, wiring each variant to the stage function you already built. Return ExitCode so an invalid-data error becomes a non-zero exit.
5. Run green, commit.

Done when

cargo test runs the integration test green, panoptes --help lists the three subcommands, and panoptes code --coded bad.csv exits non-zero — provably composable in a && chain or a CI step.

Concept-Check: CLI

Where This Plugs Into the Thesis

Kind: Wrap-up.

You have built the harness. Here is how it sits inside the larger thesis program, so the code you wrote connects to the deadlines it serves.

The harness is Ch3 made real

Chapter 3 (Methodology and system design) promises an evaluation protocol: parameterized scenarios, pinned models, replication, exhaustive logging, human coding, a reliability plan. Every one of those promises now has running code behind it. When Ch3 says "responses were logged with full prompt, raw output, model version, and parameters," it can cite the ResponseRecord schema and the append-only log as implemented, not aspirational. The pilot runs in the September window use this harness; Ch3 describes what you actually built.

The four invariants are defense armor

Each correctness property maps to a question a committee can ask:

• Enums-as-validation → "How do you know your coded data conforms to the codebook?" Because non-conforming values cannot be parsed.
• Blinding → "How do you know your codes are not shaped by knowing which model produced each response?" Because the coding sheet structurally cannot show that.
• Append-only log → "How is this reproducible?" Because the dataset of record only ever grows and is archived.
• Manifest-as-join-spine → "How do you connect responses to conditions?" Through one deterministic join key.

The open questions this closed

Recall the Jul–Aug decision list. Building this settles two: the harness is a ladder build (you built it deliberately, test-first, and it is portfolio-grade), and the language question — Rust for the harness, Python for the stats tail — is resolved at the file contract.

And the business thread

The same harness is the seed of the evaluation work for the BM3ci TAP Lab subcontract. The scenario-generation and scoring machinery you built for the thesis is the starting point for evaluating a client's space-adapted model — the thesis product and the first contract's tooling share a spine. Building it well here is building it well for both.

The Python Tail: Stages 5–6

Kind: Wrap-up / pointer. Not built in this course.

Stages 5 (analysis) and 6 (reporting) are Python, by the deliberate boundary decision. This chapter is a map, not an implementation — a future course, or a future set of chapters, could build it out.

What Stage 4–6 reads

Everything the harness produced:

• harness/logs/responses.jsonl — the dataset of record.
• scenarios/generated/manifest.csv — the join spine (vignette_id → parameters).
• coding/coded/primary.csv and second_coder.csv — validated coded data (passed panoptes-code).
• coding/blind_key.csv — to join response_id → vignette_id after coding is done.

Stage 4 — Reliability

Compute Cohen's kappa / Krippendorff's alpha per criterion, stratifying the subsample by family (join through the blind key) so no scenario type escapes validation. Dump per-criterion disagreements — those drive codebook revision. Freeze the codebook version only when every criterion clears threshold. This table is Ch3/Ch4 verbatim.

Stage 5 — Analysis

Join logs to the manifest and to coded data. Ask the designed questions: strategic-logic distribution by model; escalation level as a function of attribution confidence; info-request behavior with and without the option; consistency across replications. The right-for-wrong-reasons detection lives here: a model whose escalation does not move as attribution confidence moves was never conditioning on attribution.

Stage 6 — Reporting

Tables, figures, and the archived repo (prompts, logs, codebook, coded data, analysis code, all versioned). For the thesis this is Ch4; for the benchmark it is the public release; for the business it becomes the client deliverable. Same stage, three costumes.

Appendix: The Full Task Plan

This course is paired with a complete, worked implementation plan — the twelve-task, test-first specification that contains the full version of every build chapter.

Use it as an answer key, not a script. Attempt each build chapter yourself first, from the behavior described. Then compare against the corresponding task. The gap between your version and the plan is where the learning is.

Task-to-chapter map

The plan document itself (2026-07-17-panoptes-harness.md) lives alongside this book in your project files.

The four invariants, in one place

1. Append-only log — responses.jsonl only ever grows; it is the dataset of record.
2. Blinding — coding sheets structurally cannot show model or parameters.
3. Enums-as-validation — coded values outside the codebook cannot be parsed.
4. Manifest-as-join-spine — one deterministic key ties responses to conditions.

Everything you build should preserve these. If a change would break one, that is the signal to stop and reconsider.

Appendix: Reference Books

This course is grounded in five books. Where a chapter anchors a concept to a specific book, it cites it inline. This appendix says what each one is for, so you know where to go deeper.

Async Rust — Maxwell Flitton & Caroline Morton (O'Reilly)

The primary source for Part IV. Its treatment of the future lifecycle (idle → polled → Pending/Ready), the runtime's polling loop, and the concurrency model underpins the async concept chapters. If any async idea in this course feels thin, this is the book to open — its chapter on futures, pinning, and context goes deeper than a working harness strictly needs, which is exactly why it is the right reference when the compiler surprises you.

Command-Line Rust — Ken Youens-Clark (O'Reilly)

The source for Part VII. Its early chapters model exactly what the CLI arc does: building tools with clap, writing integration tests that run the compiled binary, and — the idea that matters most — exit codes and composability ("Exit Values Make Programs Composable"). The book's discipline around honest exit status is what turns panoptes from three scripts into a composable tool.

Rust for Rustaceans — Jon Gjengset (No Starch)

The source for the Foundations arc (ownership, moves, borrowing) and a best-practices reference throughout. Its Chapter 1 "Foundations" gives the flows mental model for ownership and lifetimes that the course teaches, and its intermediate-idioms chapters inform trait and API design decisions. This is the book to grow into after the course.

Effective Rust — David Drysdale (O'Reilly)

A best-practices reference, organized as discrete numbered "items" (in the tradition of Effective C++). Consulted for idiomatic choices around the type system, error handling, and API design. Where a design decision in the harness has an idiomatic "right answer," this book usually has an item on it.

AI Engineering — Chip Huyen (O'Reilly)

Foundational knowledge for the evaluation methodology the harness serves, and the broader thread connecting Panoptes to the thesis and to Norion. It sharpens the wrap-up's framing of what a capability benchmark is and why the codebook, reliability, and analysis stages are shaped the way they are. Read it alongside the eval-engineering canon (Husain, Yan, Shankar) for the discipline this harness is an instance of.