CI / test (pull_request) Canceled after 0s
- analysis/tree/build_tree.py -> data/public/tree.json, analysis/reports/TREE.md: every screen the bot has seen, every option, where each led (fixed or dice, with ranges), best known picks per colour chosen cautiously, where any picks end, and the untested options. solver.js screenPick(): least-tried and Thompson picks from the tree. - The bot logs a pick it then overrode as a step: 37 runs were coded with choices they never made. Importer and graph.js now use the played steps only. - runs_clean.csv: the Unofficial Dragon Register's 288 bonded dragons are in the master list as outcome-only rows, flagged (no analysis uses them). - Cycle, refresh and the bot kit rebuild and carry the tree; tests check every generated report is current. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
333 lines
27 KiB
Markdown
333 lines
27 KiB
Markdown
# Threshing Atlas: project context
|
||
|
||
A briefing for picking this project up in a new conversation. Written 2026-11-12 from the working state on
|
||
Kathleen's Mac (`~/PycharmProjects/threshing-atlas`, latest commit `4837332`).
|
||
|
||
---
|
||
|
||
## 1. What this is
|
||
|
||
**Dragonkind** is a browser game set in the world of *Fourth Wing*. Its centrepiece, **the threshing**, is a short
|
||
choose-your-path run: a cadet walks into a valley, makes a series of choices, the game rolls dice at certain points,
|
||
and the run ends with the cadet either **bonded to a dragon** (Red, Orange, Brown, Green, Blue or Black) or
|
||
**incinerated**. There's a cooldown of about four hours between attempts, Blue and Black are rare, and players want
|
||
to know which choices give the best odds of the dragon they want.
|
||
|
||
The **Threshing Atlas** is a hobby, non-commercial, open-data project by **Kathleen Schiano (GitHub: ekcSchiano)**
|
||
that:
|
||
|
||
1. **collects** every logged run it can find (a community Google Form and spreadsheet, Reddit, a bot, direct reports),
|
||
2. **codes** them into one clean table, one run per row (`data/public/runs_clean.csv`),
|
||
3. **analyses** them (factor tables, models, route rankings) for a public page,
|
||
4. **plans** routes: a Bayesian model of the game's dice, solved as a decision tree, that recommends the best
|
||
choice at every step for each target colour, and
|
||
5. **feeds a bot**: Ken's DragonKind bot plays the game, uses the Atlas's route model to choose, and sends its runs
|
||
back, so the model keeps learning.
|
||
|
||
Kathleen is a PhD researcher (informatics); she prefers **Python**, **declarative specs (YAML, data dictionaries,
|
||
Croissant metadata)**, and JavaScript only where it must run in a browser or in Ken's Node bot.
|
||
|
||
## 2. People, places and accounts
|
||
|
||
| What | Where | Notes |
|
||
|---|---|---|
|
||
| The Atlas repo (canonical copy) | `~/PycharmProjects/threshing-atlas` on Kathleen's Mac | Python venv at `.venv` (Python 3.9); PyCharm. Has `scipy`; **Node is not installed** (so `graph.json` can't be rebuilt there; `brew install node` fixes it). |
|
||
| GitHub repo | `github.com/ekcSchiano/threshing-atlas` (public) | **Account suspended** (flagged by GitHub's abuse detection, probably bursts of automated API activity from an AI assistant). Reinstatement request open with GitHub Support; they asked how she plans to use GitHub (answered: pet projects). GitHub Pages site paused. The Mac copy's `origin` still points here. |
|
||
| Gitea | `git.kenschiano.net` (Ken's server) | The Atlas is prepared to move there (under Kathleen's own account, visibility "limited"), but **Ken's Gitea has a problem; skipped for now**. |
|
||
| Ken | Gitea `kschiano`; GitHub issues author | Collaborator. Runs the **DragonKind bot** (private repo, now on his Gitea at `kschiano/DragonKind`). Kathleen has a local clone at `~/Desktop/DragonKind`: **read-only for us, never modify it** without asking. |
|
||
| The page | Claude artifact "Threshing Route Atlas": https://claude.ai/artifact/Amr4D5Av7sBHkWvjJuXM8A | Last republished 2026-11-04 (older data). Kathleen will view new builds locally / as an artifact while GitHub Pages is paused. |
|
||
| Community sources | u/ClumsyBadger's "Threshing Routes" Google Form (https://forms.gle/DRpj9tG8gb5E4Hcu7) and its responses spreadsheet; the "Unofficial Dragon Register"; the community route PDF; r/fourthwing Dragonkind megathreads (Part 3 `1wvwfp9`, Part 4 `1wz5cnj`) | Linked from the megathread post. u/Old_Regular3018 computes the PDF's percentages from the same spreadsheet; u/Smart_Trifle_2099 posted "precise Blue pathways" and Black theories. |
|
||
| Backups | `threshing-atlas.bundle` sent in chat (git bundle of all branches) | `git clone threshing-atlas.bundle threshing-atlas` |
|
||
|
||
## 3. The game, as the Atlas models it
|
||
|
||
The game is written down declaratively in **`core/spec/game.yaml`** (the single source of truth for the route
|
||
model, the advisor and the bot). In order:
|
||
|
||
1. **Company**: enter alone, with 1 or with 2 squadmates. **Leave** your squadmates or stay together.
|
||
2. **First direction**: East, West or Center (the forest).
|
||
3. **First dragon** (roll): Red, Brown, Green or Orange. **React**: hold still, respect/face it, run, hide, weapon,
|
||
meet its gaze, protect squadmate (which reactions exist depends on the dragon). **Survive the first dragon** (roll).
|
||
4. **The scream**: investigate it or keep walking.
|
||
- **Investigate → the clearing** (your squadmate attacked by two bully cadets): help, or walk away.
|
||
Help → **the fight**: pause to assess (then the dagger: engage the smaller cadet or charge the bigger one
|
||
anyway), charge the smaller cadet ("even the numbers"), or charge the bigger one ("put yourself in the middle").
|
||
**Survive the fight** (roll). Then **the bullies' fate**: kill them yourself, squadmate's revenge, or leave them
|
||
to the dragons. Then a **fork**: Forest, West or East.
|
||
- **Keep walking → the wounded cadet**: tourniquet, leave them to die, or take their weapons.
|
||
5. **Survive the hour** (roll). **After an hour**: climb the tallest tree, or go to the valley centre.
|
||
6. **Tree**: the **shadow** may appear (roll; **gated**, see below) → the shadow's colour (Black or Blue) → bonding
|
||
encounter. If no shadow: **climb down quickly** (the Red below the tree: jump on it, or let it pass → the two
|
||
Greens) or **slowly** (the two Greens).
|
||
7. **Valley**: **face the Red**, or pass it into the **meadow**, where a dragon waits (roll: Orange, Brown, or a Blue).
|
||
8. **Bonding encounters** (`encounters` in game.yaml): valley Red, tree Red, the two Greens (an Orange or Brown can
|
||
swoop in and rescue/bond you), meadow Orange/Brown/Blue, the Blue at the redwood (shadow), the Black (shadow).
|
||
A rider can attempt the bond or **die on purpose** ("seek Malek") rather than bond the wrong dragon.
|
||
|
||
Key modelling facts now in game.yaml:
|
||
- **The shadow is gated**: it can only appear if you **investigated the scream and helped your squadmate** (the route
|
||
PDF's "Black Dragon Gates"; data agree: 78 shadows in 4,199 gated tree runs vs 2 in 646 others, both likely
|
||
mis-entered). A closed gate fixes the roll at "no" and those runs aren't counted for it.
|
||
- **Possible bonds per encounter**: each encounter gives only its own colour (the two Greens also Orange/Brown).
|
||
Anything else in the data is treated as a data-entry error (a form row once claimed a Green at the shadow's Blue,
|
||
which broke the Green recommendation).
|
||
|
||
## 4. The four prongs + core (repo layout)
|
||
|
||
```
|
||
core/ spec/form.yaml (how each form answer is coded), spec/game.yaml (the game)
|
||
atlas/ (Python): coding.py (form → runs), model.py (counts, back-off, Params), solver.py (decision
|
||
tree, posterior draws, policy export), bias.py (reporting bias), runs.py (read runs_clean.csv), spec.py
|
||
solver.js (the small JS advisor), graph.js (route graph helpers + the bot crosswalk, generated block)
|
||
collection/ pipeline.yaml (every source, declaratively), run.py (`python -m collection ...`),
|
||
etl/ (extract, transform/*, validate, load, runs.py coders, metadata.py (Croissant), codegen.py,
|
||
report_form.py, handoff.py (zips, local refresh), forge.py (Gitea/GitHub API)),
|
||
specs/bot.yaml (bot labels + crosswalk), specs/report_form.yaml, manual/reddit_results.yaml
|
||
analysis/ stats/measures.yaml (what the page measures), site_stats.py, stats_core.py;
|
||
network/build-graph.js (graph.json); reports/POLICY.md (written by policy_report.py)
|
||
planner/ policy.yaml (which runs the model learns from, low-bias sample, reporting-bias settings),
|
||
build_policy.py, policy_report.py
|
||
site/ atlas/ (index.html, app.js, ui.js, explorer.js (valley map), valley.js, dragons.js, register.js),
|
||
build.js (→ dist/), qa/ (crawl.js, mapcheck.js: Playwright checks)
|
||
data/ public/ (everything published: runs_clean.csv, form_responses.json, register.json, reports.json,
|
||
stats.json, policy.json, graph.json, links.json), dictionary.yaml, croissant.json, manifest.json,
|
||
bot/ (scenes.json, trajectories-*.jsonl: anonymised bot runs), derived/pdf_fingerprint.json,
|
||
raw/ (git-ignored: current + previous raw file per source; never published)
|
||
tests/ core/ (coding rules, game, solver, stats, bias + JS advisor/graph tests), collection/ (pipeline),
|
||
fixtures/, parity/
|
||
docs/ BOT_INTEGRATION.md, BOT_DATA_PROTOCOL.md, PRIVACY.md, REPO_SETTINGS.md, ROADMAP.md, COLOURS.md
|
||
forge.yaml where the repo is hosted (gitea, git.kenschiano.net, repo: kathleen/threshing-atlas)
|
||
.gitea/workflows/ CI + daily update for Gitea .github/workflows/ CI, daily update, Pages for GitHub (paused)
|
||
```
|
||
|
||
### 4a. Data collection: ETL (`collection/`)
|
||
|
||
`collection/pipeline.yaml` declares every source: how to **extract** it, which **transform**, its **privacy** rules,
|
||
where it **loads**, and which **coder** turns it into runs.
|
||
|
||
- **extract** only fetches into `data/raw/<source>/current.<ext>` (the replaced one becomes `previous.<ext>`;
|
||
nothing older is kept). `data/manifest.json` records the current raw checksum and change date.
|
||
- **transform** is pure (no network, no clock) → the public source file.
|
||
- **validate** checks runs against `data/dictionary.yaml` (types, every allowed value, unique ids). A new answer the
|
||
rules don't know **stops the load** and is reported (add it to `core/spec/form.yaml`).
|
||
- **load** is the only stage that writes; byte-stable.
|
||
- **build** codes every public source file into `data/public/runs_clean.csv` and writes generated files: the
|
||
"Report a run" issue form, the crosswalk block in `core/graph.js`, `data/public/links.json`, `data/croissant.json`.
|
||
|
||
| `source` in runs_clean | Pipeline source | Runs now | Notes |
|
||
|---|---|---|---|
|
||
| reddit spreadsheet | `reddit_spreadsheet` (the Threshing Routes form + sheet) | 8,132 | coded by `core/spec/form.yaml`; `entry` column keeps the sheet's own label ("Form" = form submission; others = bulk-pasted logs) |
|
||
| bot | `bot` (Ken's bot, imported from a folder or zip) | 421 | per-account folders; account names salted-hashed (salt in `data/raw/bot/account-salt.txt` on the Mac, never committed); `bot_mode` column |
|
||
| reddit thread | `reddit_thread` | 12 | dragonkind.com result pages read by hand (`collection/manual/reddit_results.yaml`); Reddit API extractor ready but needs `REDDIT_CLIENT_ID/SECRET` |
|
||
| reported | `reported` ("Report a run" issue form) | 0 | GitHub issues; switched to Gitea via forge.yaml (currently 404s: GitHub suspended, Gitea not set up) |
|
||
| reddit pdf | `reddit_pdf` | 0 | the route PDF is a flowchart, not runs: only its **fingerprint** is kept (never redistributed); changes are reported line by line |
|
||
| (none) | `register` (Unofficial Dragon Register) | 0 | bonded dragons without routes; names and photos dropped |
|
||
|
||
Every column is described in `data/dictionary.yaml`; `data/croissant.json` is Croissant 1.0 metadata built from it.
|
||
|
||
### 4b. Data analysis (`analysis/`)
|
||
|
||
`analysis/stats/site_stats.py` reads runs_clean (sources in `measures.yaml`: **spreadsheet only** for now) and writes
|
||
`data/public/stats.json`: every number the page shows, for six fixed filter presets (all, form only, bulk only, not
|
||
chasing Blue/Black, chasing, unflagged). Factor tables (Fisher/Mantel–Haenszel, BH q-values), ridge-logistic models,
|
||
route rankings at thresholds 5/10/20/30, per-stage route map counts. The page computes nothing itself.
|
||
`analysis/network/build-graph.js` writes `graph.json` (route graph per source + the bot's scene graph).
|
||
|
||
### 4c. Route planner (`planner/` + `core/atlas/`)
|
||
|
||
- **Model**: each roll (first dragon, survive sighting/fight/hour, shadow, shadow colour, meadow dragon) is a
|
||
categorical with a **hierarchical Dirichlet back-off** over its context (earlier choices); the shrinkage strength
|
||
(alpha) is fitted by empirical Bayes. Bond outcomes per encounter likewise. Malek/deliberate burns are not counted
|
||
as failed bonds.
|
||
- **Learns from** (`planner/policy.yaml`): reddit spreadsheet + reported + **bot** runs.
|
||
- **Reporting-bias correction** (`core/atlas/bias.py`): form submitters over-report rare sightings. For each rare roll
|
||
and each group, rho = odds(rare | group) / odds(rare | complete logs), where complete logs = **bot + bulk logs**.
|
||
Rare outcomes from biased groups count 1/rho. Current estimates (meadow Blue): **form, chasing Blue/Black rho 3.85
|
||
[2.98–5.14]; form, other 3.17 [2.33–4.41]**. For the shadow, complete logs have **0 shadows in 579 gated tree runs**,
|
||
so the bias can't be estimated and the shadow is learned from complete logs only. A roll seen only one way is pooled
|
||
fully (alpha max). Caveats: assumes the bias is the same on every route; intervals don't include rho's uncertainty.
|
||
- **Solver** (`core/atlas/solver.py`): walks game.yaml as a decision tree; two goals per colour: "this attempt"
|
||
(maximise P(colour)) and "eventually" (Dinkelbach ratio: P(X) / (P(X)+P(other bond)), i.e. dying on purpose
|
||
otherwise). 200 posterior draws give intervals and **stillBest** (share of draws where the recommended choice is best).
|
||
Two samples: **all** and **lowBias** (bot + bulk + riders not chasing rare dragons).
|
||
- **Output**: `data/public/policy.json` (game, count tables, alphas, reportingBias, support, trees per colour).
|
||
- **JS advisor** (`core/solver.js`): reads policy.json, re-solves, `advisor(sample, colour, objective, opts)`,
|
||
`recommendOption()` maps on-screen text to options (game.yaml `bot_text`). **Thompson sampling**:
|
||
`{ sampling: 'thompson', seed }` follows one posterior draw per attempt and returns a **propensity**.
|
||
|
||
Current odds per attempt (sample "all", latest build `538bc1e3d694`):
|
||
|
||
| Colour | Per attempt (90% range) | Low-bias sample | stillBest of first choice |
|
||
|---|---|---|---|
|
||
| Red | 93.1% (86.9–97.4) | 94.1% | 0.30 |
|
||
| Green | 25.2% (21.7–28.8) | 24.2% | 0.42 |
|
||
| Orange | 13.5% (11.3–15.7) | 16.7% | 0.55 |
|
||
| Brown | 9.3% (7.7–11.2) | 10.7% | 0.52 |
|
||
| Blue | 2.5% (2.1–3.0) | 1.9% | 0.45 |
|
||
| Black | 0.05% (0.00–0.25) | 0.04% | 0.31 |
|
||
|
||
(Before the bot data and bias correction, the model said Blue 7.3% and Black 2.1%: inflated by reporting bias.)
|
||
|
||
### 4d. Data visualisation (`site/`)
|
||
|
||
A single static page (tabs: Valley map, Odds, Dragons, Deep dive, Data & Methods, **Report a run**). The **valley
|
||
map** (`explorer.js`) uses progressive disclosure: key moments only, a point's options appear when selected, picking
|
||
a dragon shows its best path and choices; later choices keep the path you picked. The page shows precomputed
|
||
`stats.json` and runs the JS advisor on `policy.json`. Links come from `data/public/links.json` (filled in by
|
||
`site/build.js`). Build: `node site/build.js` → `dist/`. QA: `site/qa/crawl.js` (needs a local server on :8765),
|
||
`site/qa/mapcheck.js`. Published so far as the Claude artifact above.
|
||
|
||
## 5. The bot loop (Ken)
|
||
|
||
- **Atlas → bot**: `python3 -m collection export-bot-kit` → `exports/atlas-for-bot-<date>-<policy id>.zip`
|
||
(policy.json, solver.js, graph.js, README with checksums). Ken unzips into the bot's `vendor/threshing-atlas/`.
|
||
Latest: `atlas-for-bot-2026-11-12-538bc1e3d694.zip`.
|
||
- **Bot → atlas**: Ken zips the bot's `data/` and `logs/` folders; save in `inbox/` and run
|
||
`python3 -m collection refresh --bot inbox/<zip>`. The importer reads `data/knowledge.json` and every
|
||
`data/accounts/<acct>/runs.jsonl` + `logs/accounts/<acct>/encounters.jsonl` (top-level files mirror the first
|
||
account and are skipped). Bot **schema-2** run records carry steps, what was seen (shadow, dragons), propensity,
|
||
decisionMode, policyId. No game prose or account names are published (`check_public` refuses them).
|
||
- **How the bot should choose** (docs/BOT_INTEGRATION.md): goal accounts follow the best route
|
||
(`advisor(policy.samples.all, target, 'eventually')`); **research accounts use Thompson sampling** with a new seed
|
||
per attempt and log `propensity` with `decisionMode: "atlas_thompson"`.
|
||
- Ken's data so far: 13 accounts, 415 attempts (to 2026-11-10); research accounts burn on purpose when no shadow at
|
||
the tree / no Blue in the meadow (`TREE_NO_SHADOW_AUTO_BURN`, `MEADOW_NO_BLUE_AUTO_BURN`).
|
||
|
||
## 6. Findings so far
|
||
|
||
- **Reporting bias is large.** Meadow Blue: complete logs ~10% (bot 16/126, bulk 30/328) vs form ~33%.
|
||
- **Black is very rare.** 0 shadows in 579 gated tree runs from complete logs; form data show ~2% (inflated). Model:
|
||
~0.05% per attempt. Fits Smart_Trifle_2099's guess of 1/500–1/1000 tree runs for a Black sighting.
|
||
- **Route doesn't seem to move the rare rolls.** Shadow rate doesn't differ across routes (chi² 44 on 40 df); the
|
||
"precise Blue pathways" (Smart_Trifle_2099 and the megathread post's ~47%/~63% routes) did no better than other
|
||
meadow routes (23–30% vs 30% in the spreadsheet). Reaching the meadow seems to be the "unlock".
|
||
- **The gate is real**: the shadow needs investigate + help.
|
||
- **Tree Red is the hardest bond**: 1.7% (38/2,207), matching the post's "2%".
|
||
- **First attempts** see the meadow Blue less often (16% vs 31%), surviving adjustment in the site's model.
|
||
- Weekly swings in rates (and "the game was patched" claims) are confounded by who reports when.
|
||
|
||
## 7. How to run things (on the Mac, PyCharm terminal, venv active)
|
||
|
||
```
|
||
python3 -m collection refresh # the daily update locally (~3 min): sources → runs → graph* → model → stats → build
|
||
python3 -m collection refresh --bot inbox/x.zip # same, importing a bot zip
|
||
python3 -m collection update --source NAME # one source; --file NAME=PATH to use a local file
|
||
python3 -m collection build [--check] # rebuild runs_clean + generated files (CI uses --check)
|
||
python3 -m collection export-bot-kit # zip for Ken
|
||
python3 planner/build_policy.py # route model (~30–75 s); python3 planner/policy_report.py
|
||
python3 analysis/stats/site_stats.py # page statistics (needs scipy)
|
||
node analysis/network/build-graph.js # graph.json (*needs Node)
|
||
node site/build.js # the page → dist/
|
||
npm test # = build --check + Python unit tests + JS tests
|
||
```
|
||
|
||
Reports land in `exports/last-report.md`. If a refresh stops with "values not in the dictionary", add the new form
|
||
answer to `core/spec/form.yaml` (or the value to `data/dictionary.yaml`) and run `python3 -m collection build`.
|
||
|
||
## 8. Conventions and constraints (keep these)
|
||
|
||
- **Python first; declarative specs** (YAML for the form, game, pipeline, measures, planner, bot crosswalk, report
|
||
form; dictionary + Croissant). JS only for the browser page and Ken's Node bot (`solver.js`, `graph.js`).
|
||
- **Generated files are generated**: don't hand-edit `runs_clean.csv`, `croissant.json`, `links.json`, the
|
||
crosswalk block in `core/graph.js`, `.github/ISSUE_TEMPLATE/report-a-run.yml`. Edit the spec, then build.
|
||
- **Privacy**: no game prose, no account names/ids, no emails (redacted), no rider names (register), no Reddit or
|
||
GitHub usernames; the route PDF is never redistributed (fingerprint only). Never commit `data/raw/`, `.env`,
|
||
`.browser-profile/`, salts or tokens; tokens only in repository secrets. Don't scrape Discord with a personal token;
|
||
Reddit only via the official API.
|
||
- **Don't write to Ken's bot repo** (`~/Desktop/DragonKind`) without asking.
|
||
- **Commits** as `ekcSchiano` / Kathleen's email, ending with `Co-Authored-By: Claude …` and `Claude-Session: …`
|
||
lines. On GitHub, main was protected (PR + required `test` check, auto-merge). **Go easy on automated API
|
||
activity** (that's the likely cause of the GitHub suspension).
|
||
- Kathleen likes: progressive disclosure on the map, honest uncertainty, explaining what changed and why.
|
||
|
||
## 9. Status and open threads
|
||
|
||
- **Uncommitted / pending**: nothing important; Mac repo is clean at `4837332` (plus `.idea/`, `exports/`, `inbox/`
|
||
ignored). The artifact page is older than the data; rebuild with `node site/build.js` (needs Node) and republish.
|
||
- **GitHub**: suspended; reinstatement in progress. When back: `git push` to a branch + PR (main is protected);
|
||
GitHub Pages resumes from `.github/workflows/pages.yml`; set `forge.yaml` `kind: github` (or keep Gitea + mirror).
|
||
- **Gitea**: ready but on hold (Ken's server has a problem). Steps in `docs/REPO_SETTINGS.md`; set your username in
|
||
`forge.yaml`; the `.gitea/workflows/` are untested (runner label assumed `ubuntu-latest`).
|
||
- **"Report a run"** button points at a Gitea URL that doesn't exist yet; the Google Form is the working intake.
|
||
- **Not done yet / ideas**:
|
||
- Include bot and reported runs in the page's statistics (currently spreadsheet only; bot choices follow a strategy).
|
||
- Propagate the bias estimate's uncertainty (rho) into the route model's intervals; consider a per-source
|
||
reporting-bias model inside the posterior draws.
|
||
- Inverse-propensity weighting of bot runs using logged propensities.
|
||
- Port `build-graph.js` to Python; Reddit extractor live once API secrets exist; "Community claims, checked"
|
||
section (claims YAML recomputed against data); a "random run?" question on the report form.
|
||
- Reviews still pending from the original plan: analysis, planner (experiments.json, encounter answers), site.
|
||
- **Environment gotchas** (for Claude sessions): the cloud workspace and the Mac's Linux helper can't reach Google,
|
||
Gitea or PyPI; Kathleen's own Terminal can. The helper lacks `scipy`. Git in the helper needs delete permission
|
||
for its lock files.
|
||
|
||
## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle
|
||
|
||
Branch `first-attempt-cycle` (not merged, not pushed).
|
||
|
||
- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first
|
||
attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the
|
||
tree and the bot has none, so "0 shadows in 579" was about later attempts.
|
||
- **Model.** `core/spec/game.yaml` has `givens` (first or later attempt) and the shadow roll depends on it first,
|
||
unpooled (`distinct: 1`). `policy.json` has `targets.<colour>.firstAttempt`. Black: about 1.4% on a first attempt
|
||
(0.7–2.2%; 0.8% low-bias), 0.07% later.
|
||
- **An assumption to revisit.** The shadow's reporting bias cannot be measured, so `planner/policy.yaml` has
|
||
`borrow: { shadow: meadow }` (form reports of a shadow count about 1/3.9). Remove the line to go back to complete
|
||
logs only. The bot's first-attempt runs will settle it.
|
||
- **Plan.** `planner/experiments.yaml` (design) → `planner/build_experiments.py` → `data/public/experiments.json`
|
||
(progress and status per experiment). The bot reads it with `assign()` in `core/solver.js`; the advisor takes
|
||
`attempt`, `fix` and `sampling: "uniform"`.
|
||
- **Account lifecycle** (agreed with Ken): a new account's first attempt is the Black attempt; if it fails, the
|
||
account does research for three days, then retires (one in ten keeps running).
|
||
- **Cycle.** `python -m collection cycle --bot DIR --deliver DIR`: import, refit, re-plan, deliver; no-op without
|
||
new runs. `.gitea/workflows/bot-cycle.yml` runs it on a runner on the bot's machine (untested).
|
||
- **For Ken:** `docs/BOT_CYCLE_FOR_KEN.md`. The clone at `~/Desktop/DragonKind` is from 31 October and older than
|
||
the bot that is running, so the bot's side was written as a specification, not against its code.
|
||
- **Not done:** the page does not show first-attempt odds yet; Gitea still needs Kathleen's account, the
|
||
repository and the push (`docs/REPO_SETTINGS.md`); git.kenschiano.net answers (Gitea 1.27.0).
|
||
|
||
## 11. Update, 2026-11-16: the developer's statements, and looking for the Black's rule
|
||
|
||
- **Source.** The developer's answers in a community Discord channel (135 forwarded messages, exported
|
||
2026-11-16). The export is private raw data; `planner/knowledge.yaml` holds the statements paraphrased, dated and
|
||
graded (said / hint / declined), with what the Atlas does with each and the open questions.
|
||
- **What it changes.** The Black is not dice: the seed plays no part, it shows when the rider's choices meet its
|
||
conditions, and knowledge gives certainty. The bullies encounter and the tree are gates; there is no single route;
|
||
about five conditions. The Blue is dice from the seed. No event is rarer than 1 in 20. Runs are never forgotten
|
||
and never hurt.
|
||
- **What the runs add** (`analysis/black/evidence.py` -> `analysis/reports/BLACK.md`, rebuilt every cycle): on the
|
||
same recorded route, 21 of 79 first attempts show the shadow against 1 of 80 later ones (across routes: odds ratio
|
||
21.5, z = 9.8); on that route, runs with the same wing, section and squad still disagree; shadows are on 44
|
||
different recorded routes; after the shadow comes a Blue, then the Black; holding the Black's gaze bonded 27 of
|
||
39, lowering eyes 0 of 7.
|
||
- **Model.** The shadow is marked `guard: true` in `game.yaml`: the model's number is the share of runs that met
|
||
unknown conditions, not a chance. `shadowColour` is "past the Blue to the Black".
|
||
- **Plan (version 2).** Attempt 1 on every fresh account is one exact path (`fresh-replicate`, 60 accounts), then
|
||
one choice changed at a time (`fresh-ablate`, starts by itself). `shadow-again`, `second-attempt-same-path`,
|
||
`veteran-repeat` and `veteran-variety` test fresh against veteran. `assign()` takes account facts; the plan carries
|
||
what to record and the sequence to follow once the shadow shows.
|
||
- **Not known.** What differs between fresh accounts that see the shadow and those that do not on the same route;
|
||
what "twenty" counts; why holding the Black's gaze fails three times in ten.
|
||
- **To tidy.** One public `bond_note` holds a rider's handle ("Run 116 - ..."), in a row marked as a first attempt.
|
||
|
||
## 12. Update, 2026-11-16 (later): the master list, the screen tree, and a fix to the bot's rows
|
||
|
||
- **Master list.** `runs_clean.csv` now has 8,853 rows: spreadsheet 8,132 (every row of "Form Responses 1"; the
|
||
workbook's other tabs are views of it), bot 421 (every trajectory), register 288 (new: bonded dragons, outcome
|
||
only, each flagged because the same run may be in the spreadsheet), Reddit result pages 12. Not in it because
|
||
nothing has been collected: Discord run reports (the only Discord export is the developer's channel, which holds
|
||
no runs), "Report a run" (0 reports), megathread comments (needs the Reddit API keys; the 12 were read by hand on
|
||
19 October). Stale: the spreadsheet was last fetched 12 November and the bot's data 10 November.
|
||
- **Possible duplicates, left in.** 17 spreadsheet rows are identical to another including the timestamp; 204 more
|
||
are identical except for the timestamp (the same route and death repeated, or a double submission).
|
||
- **Fix.** The bot logs a pick it then overrode as a step. 37 bot runs were coded with choices they never made
|
||
(direction, fork, dagger, reaction); they are now coded from the played steps (`collection/etl/runs.py` `played`,
|
||
`core/graph.js` `playedSteps`).
|
||
- **Screen tree.** `analysis/tree/build_tree.py` -> `data/public/tree.json` and `analysis/reports/TREE.md`, from
|
||
the bot's logs: 45 screens, 116 options (69 played), which options are dice and their rates, the best known picks
|
||
per colour (chosen cautiously), where any set of picks ends, and the untested options. Blue: a Blue waits on 11.4%
|
||
of meadow visits and the bonding chain has survived 18.7%, so 2.1% per run on the answers tried so far.
|
||
`screenPick()` in `core/solver.js` picks from the tree (least-tried, or Thompson). Rebuilt every cycle; in the kit.
|
||
- **Limit.** The tree treats a screen's odds as the same whatever came before. That is right for the seeded draws
|
||
and wrong for the shadow. It holds only what the bot has played: no first attempts, no shadow, one Green bond.
|