Files
threshing-atlas/docs/PROJECT_CONTEXT.md
kathleenandClaude Opus 5.5 c7978b4538
CI / test (pull_request) Successful in 3m36s
The Atlas now lives in the DragonKind organization
forge.yaml: repo DragonKind/threshing-atlas, and the links generated from it (data/public/links.json,
data/croissant.json); the project notes too. Old kathleen/threshing-atlas links redirect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 02:42:19 +00:00

333 lines
27 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Threshing Atlas: project context
A briefing for picking this project up in a new conversation. Written 2026-11-12 from the working state on
Kathleen's Mac (`~/PycharmProjects/threshing-atlas`, latest commit `4837332`).
---
## 1. What this is
**Dragonkind** is a browser game set in the world of *Fourth Wing*. Its centrepiece, **the threshing**, is a short
choose-your-path run: a cadet walks into a valley, makes a series of choices, the game rolls dice at certain points,
and the run ends with the cadet either **bonded to a dragon** (Red, Orange, Brown, Green, Blue or Black) or
**incinerated**. There's a cooldown of about four hours between attempts, Blue and Black are rare, and players want
to know which choices give the best odds of the dragon they want.
The **Threshing Atlas** is a hobby, non-commercial, open-data project by **Kathleen Schiano (GitHub: ekcSchiano)**
that:
1. **collects** every logged run it can find (a community Google Form and spreadsheet, Reddit, a bot, direct reports),
2. **codes** them into one clean table, one run per row (`data/public/runs_clean.csv`),
3. **analyses** them (factor tables, models, route rankings) for a public page,
4. **plans** routes: a Bayesian model of the game's dice, solved as a decision tree, that recommends the best
choice at every step for each target colour, and
5. **feeds a bot**: Ken's DragonKind bot plays the game, uses the Atlas's route model to choose, and sends its runs
back, so the model keeps learning.
Kathleen is a PhD researcher (informatics); she prefers **Python**, **declarative specs (YAML, data dictionaries,
Croissant metadata)**, and JavaScript only where it must run in a browser or in Ken's Node bot.
## 2. People, places and accounts
| What | Where | Notes |
|---|---|---|
| The Atlas repo (canonical copy) | `~/PycharmProjects/threshing-atlas` on Kathleen's Mac | Python venv at `.venv` (Python 3.9); PyCharm. Has `scipy`; **Node is not installed** (so `graph.json` can't be rebuilt there; `brew install node` fixes it). |
| GitHub repo | `github.com/ekcSchiano/threshing-atlas` (public) | **Account suspended** (flagged by GitHub's abuse detection, probably bursts of automated API activity from an AI assistant). Reinstatement request open with GitHub Support; they asked how she plans to use GitHub (answered: pet projects). GitHub Pages site paused. The Mac copy's `origin` still points here. |
| Gitea | `git.kenschiano.net` (Ken's server) | The Atlas is prepared to move there (under Kathleen's own account, visibility "limited"), but **Ken's Gitea has a problem; skipped for now**. |
| Ken | Gitea `kschiano`; GitHub issues author | Collaborator. Runs the **DragonKind bot** (private repo, now on his Gitea at `kschiano/DragonKind`). Kathleen has a local clone at `~/Desktop/DragonKind`: **read-only for us, never modify it** without asking. |
| The page | Claude artifact "Threshing Route Atlas": https://claude.ai/artifact/Amr4D5Av7sBHkWvjJuXM8A | Last republished 2026-11-04 (older data). Kathleen will view new builds locally / as an artifact while GitHub Pages is paused. |
| Community sources | u/ClumsyBadger's "Threshing Routes" Google Form (https://forms.gle/DRpj9tG8gb5E4Hcu7) and its responses spreadsheet; the "Unofficial Dragon Register"; the community route PDF; r/fourthwing Dragonkind megathreads (Part 3 `1wvwfp9`, Part 4 `1wz5cnj`) | Linked from the megathread post. u/Old_Regular3018 computes the PDF's percentages from the same spreadsheet; u/Smart_Trifle_2099 posted "precise Blue pathways" and Black theories. |
| Backups | `threshing-atlas.bundle` sent in chat (git bundle of all branches) | `git clone threshing-atlas.bundle threshing-atlas` |
## 3. The game, as the Atlas models it
The game is written down declaratively in **`core/spec/game.yaml`** (the single source of truth for the route
model, the advisor and the bot). In order:
1. **Company**: enter alone, with 1 or with 2 squadmates. **Leave** your squadmates or stay together.
2. **First direction**: East, West or Center (the forest).
3. **First dragon** (roll): Red, Brown, Green or Orange. **React**: hold still, respect/face it, run, hide, weapon,
meet its gaze, protect squadmate (which reactions exist depends on the dragon). **Survive the first dragon** (roll).
4. **The scream**: investigate it or keep walking.
- **Investigate → the clearing** (your squadmate attacked by two bully cadets): help, or walk away.
Help → **the fight**: pause to assess (then the dagger: engage the smaller cadet or charge the bigger one
anyway), charge the smaller cadet ("even the numbers"), or charge the bigger one ("put yourself in the middle").
**Survive the fight** (roll). Then **the bullies' fate**: kill them yourself, squadmate's revenge, or leave them
to the dragons. Then a **fork**: Forest, West or East.
- **Keep walking → the wounded cadet**: tourniquet, leave them to die, or take their weapons.
5. **Survive the hour** (roll). **After an hour**: climb the tallest tree, or go to the valley centre.
6. **Tree**: the **shadow** may appear (roll; **gated**, see below) → the shadow's colour (Black or Blue) → bonding
encounter. If no shadow: **climb down quickly** (the Red below the tree: jump on it, or let it pass → the two
Greens) or **slowly** (the two Greens).
7. **Valley**: **face the Red**, or pass it into the **meadow**, where a dragon waits (roll: Orange, Brown, or a Blue).
8. **Bonding encounters** (`encounters` in game.yaml): valley Red, tree Red, the two Greens (an Orange or Brown can
swoop in and rescue/bond you), meadow Orange/Brown/Blue, the Blue at the redwood (shadow), the Black (shadow).
A rider can attempt the bond or **die on purpose** ("seek Malek") rather than bond the wrong dragon.
Key modelling facts now in game.yaml:
- **The shadow is gated**: it can only appear if you **investigated the scream and helped your squadmate** (the route
PDF's "Black Dragon Gates"; data agree: 78 shadows in 4,199 gated tree runs vs 2 in 646 others, both likely
mis-entered). A closed gate fixes the roll at "no" and those runs aren't counted for it.
- **Possible bonds per encounter**: each encounter gives only its own colour (the two Greens also Orange/Brown).
Anything else in the data is treated as a data-entry error (a form row once claimed a Green at the shadow's Blue,
which broke the Green recommendation).
## 4. The four prongs + core (repo layout)
```
core/ spec/form.yaml (how each form answer is coded), spec/game.yaml (the game)
atlas/ (Python): coding.py (form → runs), model.py (counts, back-off, Params), solver.py (decision
tree, posterior draws, policy export), bias.py (reporting bias), runs.py (read runs_clean.csv), spec.py
solver.js (the small JS advisor), graph.js (route graph helpers + the bot crosswalk, generated block)
collection/ pipeline.yaml (every source, declaratively), run.py (`python -m collection ...`),
etl/ (extract, transform/*, validate, load, runs.py coders, metadata.py (Croissant), codegen.py,
report_form.py, handoff.py (zips, local refresh), forge.py (Gitea/GitHub API)),
specs/bot.yaml (bot labels + crosswalk), specs/report_form.yaml, manual/reddit_results.yaml
analysis/ stats/measures.yaml (what the page measures), site_stats.py, stats_core.py;
network/build-graph.js (graph.json); reports/POLICY.md (written by policy_report.py)
planner/ policy.yaml (which runs the model learns from, low-bias sample, reporting-bias settings),
build_policy.py, policy_report.py
site/ atlas/ (index.html, app.js, ui.js, explorer.js (valley map), valley.js, dragons.js, register.js),
build.js (→ dist/), qa/ (crawl.js, mapcheck.js: Playwright checks)
data/ public/ (everything published: runs_clean.csv, form_responses.json, register.json, reports.json,
stats.json, policy.json, graph.json, links.json), dictionary.yaml, croissant.json, manifest.json,
bot/ (scenes.json, trajectories-*.jsonl: anonymised bot runs), derived/pdf_fingerprint.json,
raw/ (git-ignored: current + previous raw file per source; never published)
tests/ core/ (coding rules, game, solver, stats, bias + JS advisor/graph tests), collection/ (pipeline),
fixtures/, parity/
docs/ BOT_INTEGRATION.md, BOT_DATA_PROTOCOL.md, PRIVACY.md, REPO_SETTINGS.md, ROADMAP.md, COLOURS.md
forge.yaml where the repo is hosted (gitea, git.kenschiano.net, repo: DragonKind/threshing-atlas)
.gitea/workflows/ CI + daily update for Gitea .github/workflows/ CI, daily update, Pages for GitHub (paused)
```
### 4a. Data collection: ETL (`collection/`)
`collection/pipeline.yaml` declares every source: how to **extract** it, which **transform**, its **privacy** rules,
where it **loads**, and which **coder** turns it into runs.
- **extract** only fetches into `data/raw/<source>/current.<ext>` (the replaced one becomes `previous.<ext>`;
nothing older is kept). `data/manifest.json` records the current raw checksum and change date.
- **transform** is pure (no network, no clock) → the public source file.
- **validate** checks runs against `data/dictionary.yaml` (types, every allowed value, unique ids). A new answer the
rules don't know **stops the load** and is reported (add it to `core/spec/form.yaml`).
- **load** is the only stage that writes; byte-stable.
- **build** codes every public source file into `data/public/runs_clean.csv` and writes generated files: the
"Report a run" issue form, the crosswalk block in `core/graph.js`, `data/public/links.json`, `data/croissant.json`.
| `source` in runs_clean | Pipeline source | Runs now | Notes |
|---|---|---|---|
| reddit spreadsheet | `reddit_spreadsheet` (the Threshing Routes form + sheet) | 8,132 | coded by `core/spec/form.yaml`; `entry` column keeps the sheet's own label ("Form" = form submission; others = bulk-pasted logs) |
| bot | `bot` (Ken's bot, imported from a folder or zip) | 421 | per-account folders; account names salted-hashed (salt in `data/raw/bot/account-salt.txt` on the Mac, never committed); `bot_mode` column |
| reddit thread | `reddit_thread` | 12 | dragonkind.com result pages read by hand (`collection/manual/reddit_results.yaml`); Reddit API extractor ready but needs `REDDIT_CLIENT_ID/SECRET` |
| reported | `reported` ("Report a run" issue form) | 0 | GitHub issues; switched to Gitea via forge.yaml (currently 404s: GitHub suspended, Gitea not set up) |
| reddit pdf | `reddit_pdf` | 0 | the route PDF is a flowchart, not runs: only its **fingerprint** is kept (never redistributed); changes are reported line by line |
| (none) | `register` (Unofficial Dragon Register) | 0 | bonded dragons without routes; names and photos dropped |
Every column is described in `data/dictionary.yaml`; `data/croissant.json` is Croissant 1.0 metadata built from it.
### 4b. Data analysis (`analysis/`)
`analysis/stats/site_stats.py` reads runs_clean (sources in `measures.yaml`: **spreadsheet only** for now) and writes
`data/public/stats.json`: every number the page shows, for six fixed filter presets (all, form only, bulk only, not
chasing Blue/Black, chasing, unflagged). Factor tables (Fisher/Mantel–Haenszel, BH q-values), ridge-logistic models,
route rankings at thresholds 5/10/20/30, per-stage route map counts. The page computes nothing itself.
`analysis/network/build-graph.js` writes `graph.json` (route graph per source + the bot's scene graph).
### 4c. Route planner (`planner/` + `core/atlas/`)
- **Model**: each roll (first dragon, survive sighting/fight/hour, shadow, shadow colour, meadow dragon) is a
categorical with a **hierarchical Dirichlet back-off** over its context (earlier choices); the shrinkage strength
(alpha) is fitted by empirical Bayes. Bond outcomes per encounter likewise. Malek/deliberate burns are not counted
as failed bonds.
- **Learns from** (`planner/policy.yaml`): reddit spreadsheet + reported + **bot** runs.
- **Reporting-bias correction** (`core/atlas/bias.py`): form submitters over-report rare sightings. For each rare roll
and each group, rho = odds(rare | group) / odds(rare | complete logs), where complete logs = **bot + bulk logs**.
Rare outcomes from biased groups count 1/rho. Current estimates (meadow Blue): **form, chasing Blue/Black rho 3.85
[2.98–5.14]; form, other 3.17 [2.33–4.41]**. For the shadow, complete logs have **0 shadows in 579 gated tree runs**,
so the bias can't be estimated and the shadow is learned from complete logs only. A roll seen only one way is pooled
fully (alpha max). Caveats: assumes the bias is the same on every route; intervals don't include rho's uncertainty.
- **Solver** (`core/atlas/solver.py`): walks game.yaml as a decision tree; two goals per colour: "this attempt"
(maximise P(colour)) and "eventually" (Dinkelbach ratio: P(X) / (P(X)+P(other bond)), i.e. dying on purpose
otherwise). 200 posterior draws give intervals and **stillBest** (share of draws where the recommended choice is best).
Two samples: **all** and **lowBias** (bot + bulk + riders not chasing rare dragons).
- **Output**: `data/public/policy.json` (game, count tables, alphas, reportingBias, support, trees per colour).
- **JS advisor** (`core/solver.js`): reads policy.json, re-solves, `advisor(sample, colour, objective, opts)`,
`recommendOption()` maps on-screen text to options (game.yaml `bot_text`). **Thompson sampling**:
`{ sampling: 'thompson', seed }` follows one posterior draw per attempt and returns a **propensity**.
Current odds per attempt (sample "all", latest build `538bc1e3d694`):
| Colour | Per attempt (90% range) | Low-bias sample | stillBest of first choice |
|---|---|---|---|
| Red | 93.1% (86.9–97.4) | 94.1% | 0.30 |
| Green | 25.2% (21.7–28.8) | 24.2% | 0.42 |
| Orange | 13.5% (11.3–15.7) | 16.7% | 0.55 |
| Brown | 9.3% (7.7–11.2) | 10.7% | 0.52 |
| Blue | 2.5% (2.1–3.0) | 1.9% | 0.45 |
| Black | 0.05% (0.00–0.25) | 0.04% | 0.31 |
(Before the bot data and bias correction, the model said Blue 7.3% and Black 2.1%: inflated by reporting bias.)
### 4d. Data visualisation (`site/`)
A single static page (tabs: Valley map, Odds, Dragons, Deep dive, Data & Methods, **Report a run**). The **valley
map** (`explorer.js`) uses progressive disclosure: key moments only, a point's options appear when selected, picking
a dragon shows its best path and choices; later choices keep the path you picked. The page shows precomputed
`stats.json` and runs the JS advisor on `policy.json`. Links come from `data/public/links.json` (filled in by
`site/build.js`). Build: `node site/build.js` → `dist/`. QA: `site/qa/crawl.js` (needs a local server on :8765),
`site/qa/mapcheck.js`. Published so far as the Claude artifact above.
## 5. The bot loop (Ken)
- **Atlas → bot**: `python3 -m collection export-bot-kit` → `exports/atlas-for-bot-<date>-<policy id>.zip`
(policy.json, solver.js, graph.js, README with checksums). Ken unzips into the bot's `vendor/threshing-atlas/`.
Latest: `atlas-for-bot-2026-11-12-538bc1e3d694.zip`.
- **Bot → atlas**: Ken zips the bot's `data/` and `logs/` folders; save in `inbox/` and run
`python3 -m collection refresh --bot inbox/<zip>`. The importer reads `data/knowledge.json` and every
`data/accounts/<acct>/runs.jsonl` + `logs/accounts/<acct>/encounters.jsonl` (top-level files mirror the first
account and are skipped). Bot **schema-2** run records carry steps, what was seen (shadow, dragons), propensity,
decisionMode, policyId. No game prose or account names are published (`check_public` refuses them).
- **How the bot should choose** (docs/BOT_INTEGRATION.md): goal accounts follow the best route
(`advisor(policy.samples.all, target, 'eventually')`); **research accounts use Thompson sampling** with a new seed
per attempt and log `propensity` with `decisionMode: "atlas_thompson"`.
- Ken's data so far: 13 accounts, 415 attempts (to 2026-11-10); research accounts burn on purpose when no shadow at
the tree / no Blue in the meadow (`TREE_NO_SHADOW_AUTO_BURN`, `MEADOW_NO_BLUE_AUTO_BURN`).
## 6. Findings so far
- **Reporting bias is large.** Meadow Blue: complete logs ~10% (bot 16/126, bulk 30/328) vs form ~33%.
- **Black is very rare.** 0 shadows in 579 gated tree runs from complete logs; form data show ~2% (inflated). Model:
~0.05% per attempt. Fits Smart_Trifle_2099's guess of 1/500–1/1000 tree runs for a Black sighting.
- **Route doesn't seem to move the rare rolls.** Shadow rate doesn't differ across routes (chi² 44 on 40 df); the
"precise Blue pathways" (Smart_Trifle_2099 and the megathread post's ~47%/~63% routes) did no better than other
meadow routes (23–30% vs 30% in the spreadsheet). Reaching the meadow seems to be the "unlock".
- **The gate is real**: the shadow needs investigate + help.
- **Tree Red is the hardest bond**: 1.7% (38/2,207), matching the post's "2%".
- **First attempts** see the meadow Blue less often (16% vs 31%), surviving adjustment in the site's model.
- Weekly swings in rates (and "the game was patched" claims) are confounded by who reports when.
## 7. How to run things (on the Mac, PyCharm terminal, venv active)
```
python3 -m collection refresh # the daily update locally (~3 min): sources → runs → graph* → model → stats → build
python3 -m collection refresh --bot inbox/x.zip # same, importing a bot zip
python3 -m collection update --source NAME # one source; --file NAME=PATH to use a local file
python3 -m collection build [--check] # rebuild runs_clean + generated files (CI uses --check)
python3 -m collection export-bot-kit # zip for Ken
python3 planner/build_policy.py # route model (~30–75 s); python3 planner/policy_report.py
python3 analysis/stats/site_stats.py # page statistics (needs scipy)
node analysis/network/build-graph.js # graph.json (*needs Node)
node site/build.js # the page → dist/
npm test # = build --check + Python unit tests + JS tests
```
Reports land in `exports/last-report.md`. If a refresh stops with "values not in the dictionary", add the new form
answer to `core/spec/form.yaml` (or the value to `data/dictionary.yaml`) and run `python3 -m collection build`.
## 8. Conventions and constraints (keep these)
- **Python first; declarative specs** (YAML for the form, game, pipeline, measures, planner, bot crosswalk, report
form; dictionary + Croissant). JS only for the browser page and Ken's Node bot (`solver.js`, `graph.js`).
- **Generated files are generated**: don't hand-edit `runs_clean.csv`, `croissant.json`, `links.json`, the
crosswalk block in `core/graph.js`, `.github/ISSUE_TEMPLATE/report-a-run.yml`. Edit the spec, then build.
- **Privacy**: no game prose, no account names/ids, no emails (redacted), no rider names (register), no Reddit or
GitHub usernames; the route PDF is never redistributed (fingerprint only). Never commit `data/raw/`, `.env`,
`.browser-profile/`, salts or tokens; tokens only in repository secrets. Don't scrape Discord with a personal token;
Reddit only via the official API.
- **Don't write to Ken's bot repo** (`~/Desktop/DragonKind`) without asking.
- **Commits** as `ekcSchiano` / Kathleen's email, ending with `Co-Authored-By: Claude …` and `Claude-Session: …`
lines. On GitHub, main was protected (PR + required `test` check, auto-merge). **Go easy on automated API
activity** (that's the likely cause of the GitHub suspension).
- Kathleen likes: progressive disclosure on the map, honest uncertainty, explaining what changed and why.
## 9. Status and open threads
- **Uncommitted / pending**: nothing important; Mac repo is clean at `4837332` (plus `.idea/`, `exports/`, `inbox/`
ignored). The artifact page is older than the data; rebuild with `node site/build.js` (needs Node) and republish.
- **GitHub**: suspended; reinstatement in progress. When back: `git push` to a branch + PR (main is protected);
GitHub Pages resumes from `.github/workflows/pages.yml`; set `forge.yaml` `kind: github` (or keep Gitea + mirror).
- **Gitea**: ready but on hold (Ken's server has a problem). Steps in `docs/REPO_SETTINGS.md`; set your username in
`forge.yaml`; the `.gitea/workflows/` are untested (runner label assumed `ubuntu-latest`).
- **"Report a run"** button points at a Gitea URL that doesn't exist yet; the Google Form is the working intake.
- **Not done yet / ideas**:
- Include bot and reported runs in the page's statistics (currently spreadsheet only; bot choices follow a strategy).
- Propagate the bias estimate's uncertainty (rho) into the route model's intervals; consider a per-source
reporting-bias model inside the posterior draws.
- Inverse-propensity weighting of bot runs using logged propensities.
- Port `build-graph.js` to Python; Reddit extractor live once API secrets exist; "Community claims, checked"
section (claims YAML recomputed against data); a "random run?" question on the report form.
- Reviews still pending from the original plan: analysis, planner (experiments.json, encounter answers), site.
- **Environment gotchas** (for Claude sessions): the cloud workspace and the Mac's Linux helper can't reach Google,
Gitea or PyPI; Kathleen's own Terminal can. The helper lacks `scipy`. Git in the helper needs delete permission
for its lock files.
## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle
Branch `first-attempt-cycle` (pushed to Gitea; not merged into main yet).
- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first
attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the
tree and the bot has none, so "0 shadows in 579" was about later attempts.
- **Model.** `core/spec/game.yaml` has `givens` (first or later attempt) and the shadow roll depends on it first,
unpooled (`distinct: 1`). `policy.json` has `targets.<colour>.firstAttempt`. Black: about 1.4% on a first attempt
(0.7–2.2%; 0.8% low-bias), 0.07% later.
- **An assumption to revisit.** The shadow's reporting bias cannot be measured, so `planner/policy.yaml` has
`borrow: { shadow: meadow }` (form reports of a shadow count about 1/3.9). Remove the line to go back to complete
logs only. The bot's first-attempt runs will settle it.
- **Plan.** `planner/experiments.yaml` (design) → `planner/build_experiments.py` → `data/public/experiments.json`
(progress and status per experiment). The bot reads it with `assign()` in `core/solver.js`; the advisor takes
`attempt`, `fix` and `sampling: "uniform"`.
- **Account lifecycle** (agreed with Ken): a new account's first attempt is the Black attempt; if it fails, the
account does research for three days, then retires (one in ten keeps running).
- **Cycle.** `python -m collection cycle --bot DIR --deliver DIR`: import, refit, re-plan, deliver; no-op without
new runs. `.gitea/workflows/bot-cycle.yml` runs it on a runner on the bot's machine (untested).
- **For Ken:** `docs/BOT_CYCLE_FOR_KEN.md`. The clone at `~/Desktop/DragonKind` is from 31 October and older than
the bot that is running, so the bot's side was written as a specification, not against its code.
- **Not done:** the page does not show first-attempt odds yet; Gitea still needs Kathleen's account, the
repository and the push (`docs/REPO_SETTINGS.md`); git.kenschiano.net answers (Gitea 1.27.0).
## 11. Update, 2026-11-16: the developer's statements, and looking for the Black's rule
- **Source.** The developer's answers in a community Discord channel (135 forwarded messages, exported
2026-11-16). The export is private raw data; `planner/knowledge.yaml` holds the statements paraphrased, dated and
graded (said / hint / declined), with what the Atlas does with each and the open questions.
- **What it changes.** The Black is not dice: the seed plays no part, it shows when the rider's choices meet its
conditions, and knowledge gives certainty. The bullies encounter and the tree are gates; there is no single route;
about five conditions. The Blue is dice from the seed. No event is rarer than 1 in 20. Runs are never forgotten
and never hurt.
- **What the runs add** (`analysis/black/evidence.py` -> `analysis/reports/BLACK.md`, rebuilt every cycle): on the
same recorded route, 21 of 79 first attempts show the shadow against 1 of 80 later ones (across routes: odds ratio
21.5, z = 9.8); on that route, runs with the same wing, section and squad still disagree; shadows are on 44
different recorded routes; after the shadow comes a Blue, then the Black; holding the Black's gaze bonded 27 of
39, lowering eyes 0 of 7.
- **Model.** The shadow is marked `guard: true` in `game.yaml`: the model's number is the share of runs that met
unknown conditions, not a chance. `shadowColour` is "past the Blue to the Black".
- **Plan (version 2).** Attempt 1 on every fresh account is one exact path (`fresh-replicate`, 60 accounts), then
one choice changed at a time (`fresh-ablate`, starts by itself). `shadow-again`, `second-attempt-same-path`,
`veteran-repeat` and `veteran-variety` test fresh against veteran. `assign()` takes account facts; the plan carries
what to record and the sequence to follow once the shadow shows.
- **Not known.** What differs between fresh accounts that see the shadow and those that do not on the same route;
what "twenty" counts; why holding the Black's gaze fails three times in ten.
- **To tidy.** One public `bond_note` holds a rider's handle ("Run 116 - ..."), in a row marked as a first attempt.
## 12. Update, 2026-11-16 (later): the master list, the screen tree, and a fix to the bot's rows
- **Master list.** `runs_clean.csv` now has 8,853 rows: spreadsheet 8,132 (every row of "Form Responses 1"; the
workbook's other tabs are views of it), bot 421 (every trajectory), register 288 (new: bonded dragons, outcome
only, each flagged because the same run may be in the spreadsheet), Reddit result pages 12. Not in it because
nothing has been collected: Discord run reports (the only Discord export is the developer's channel, which holds
no runs), "Report a run" (0 reports), megathread comments (needs the Reddit API keys; the 12 were read by hand on
19 October). Stale: the spreadsheet was last fetched 12 November and the bot's data 10 November.
- **Possible duplicates, left in.** 17 spreadsheet rows are identical to another including the timestamp; 204 more
are identical except for the timestamp (the same route and death repeated, or a double submission).
- **Fix.** The bot logs a pick it then overrode as a step. 37 bot runs were coded with choices they never made
(direction, fork, dagger, reaction); they are now coded from the played steps (`collection/etl/runs.py` `played`,
`core/graph.js` `playedSteps`).
- **Screen tree.** `analysis/tree/build_tree.py` -> `data/public/tree.json` and `analysis/reports/TREE.md`, from
the bot's logs: 45 screens, 116 options (69 played), which options are dice and their rates, the best known picks
per colour (chosen cautiously), where any set of picks ends, and the untested options. Blue: a Blue waits on 11.4%
of meadow visits and the bonding chain has survived 18.7%, so 2.1% per run on the answers tried so far.
`screenPick()` in `core/solver.js` picks from the tree (least-tried, or Thompson). Rebuilt every cycle; in the kit.
- **Limit.** The tree treats a screen's odds as the same whatever came before. That is right for the seeded draws
and wrong for the shadow. It holds only what the bot has played: no first attempts, no shadow, one Green bond.