cc86343668 Core: the route model in Python, built from a declarative game.yaml (#10)
Step 2 of the Python-first, declarative core.

- core/spec/game.yaml: the game as data: every decision and its options, every roll and the
  earlier steps its odds may depend on, every bonding encounter, and the flow of a run from the
  entrance to the dragons (set / keep / drop / go / end / encounter). Also how the form's coded
  answers map onto the game.
- core/atlas/model.py: what each coded run reveals about each step, count tables, empirical-Bayes
  shrinkage, and the posterior draws (same random numbers as before, reproducible from the seed).
- core/atlas/solver.py: one interpreter of game.yaml used three ways (solve, evaluate a fixed
  policy, export the published tree) instead of the tree written out five times. Vectorised
  over the 200 posterior draws: the build takes about 2 minutes instead of 4, with no memory flag.
- planner/build_policy.py and policy_report.py replace the JS build and report.

Checked against the JavaScript model: every number in policy.json agrees to within 4e-12
(last-digit differences in log/cos). Three "still best" shares differ by one draw in 200, at
near-ties under the "eventually" objective. POLICY.md is otherwise identical.
tests/parity/compare_solver.py keeps checking JS against Python in CI until the site's JS advisor
replaces solver.js; tests/core/test_game.py checks game.yaml is complete and consistent.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:04:21 +00:00
2026-11-01 16:17:21 -05:00

Threshing Atlas

Route probabilities and path finding for the Dragonkind threshing: which choices lead to which dragon, how often, how sure we can be, and (in progress) the best route to each colour.

Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publishers.

Layout

The project has four parts, built on one shared core. Each folder has its own README.

core/           the shared model every part uses. spec/ holds the declarative rules (form.yaml: every question
                and answer of the community form); atlas/ is the Python package that reads them (coding.py).
                engine.js, graph.js and solver.js are the JavaScript the site and Ken's bot still use; they move
                to Python + spec files step by step (docs/ROADMAP.md).
collection/     getting data in: sources/ (the form sheet, the Dragon Register, the route PDF check, the daily
                update job) and bot/ (the DragonKind bot importer). Reddit is not collected yet.
analysis/       making sense of it: stats/ (Python statistics, mirrored in core/engine.js), network/ (builds
                data/public/graph.json), reports/ (generated reports such as POLICY.md)
planner/        the route planner: builds data/public/policy.json and its report; the bot uses it for advice
                and experiments (docs/BOT_INTEGRATION.md)
site/           the Atlas web page (site/atlas/), its build script (site/build.js) and QA checks (site/qa/)
data/           raw/ (source exports, not committed), public/ (what the page and the bot load), derived/
                (coded runs, PDF fingerprint), bot/ (anonymised bot runs), sources.json (download links)
docs/           guides that cross sections: privacy and licences, repo settings, bot data protocol, bot
                integration, colours, roadmap
tests/          core/ (unit tests), parity/ (the JS and Python implementations must agree), fixtures/

Run it locally

Requirements: Node 18+, Python 3.9+ (pip install -r requirements.txt).

npm run dev      # builds dist/ and serves it at http://localhost:8000
npm test         # JS engine + Python reference + parity checks

New data from the community sources

A GitHub Action (.github/workflows/update-sources.yml) runs every day. It downloads the Threshing Routes responses, the Dragon Register and the route PDF, and if anything changed it rebuilds the data, runs the tests and opens a pull request that says what changed: new responses, new or renamed questions, and for the PDF the pages and lines that are new. The PDF itself is never committed, only a fingerprint (data/derived/pdf_fingerprint.json). If a link stops working, the Action opens an issue. You can also run it by hand from the Actions tab, or locally with npm run check-sources.

Best routes (the solver)

npm run policy (Python: planner/build_policy.py, from the game in core/spec/game.yaml) rebuilds data/public/policy.json and the readable analysis/reports/POLICY.md (about 2 minutes: 200 posterior draws for two data samples). For every colour it gives the best decision at every point under two objectives: the chance of bonding that colour on this attempt, and the chance the account eventually ends on that colour when you die on purpose whenever a run goes wrong. Chance events (which dragon appears, whether the shadow shows) may depend on earlier choices; how much is learned from the data. See the top of core/solver.js for the model and docs/BOT_INTEGRATION.md for how the bot uses it.

Bot data

The private DragonKind bot's files are converted locally, and only the anonymised result is committed:

DK_ACCOUNT_SALT=... npm run import-bot -- --bot /path/to/DragonKind --tag 2026-11-01
npm run graph

This writes data/bot/scenes.json and data/bot/trajectories-<tag>.jsonl: scene fingerprints, short labels, choice text and outcomes, with no game prose and no account names. The importer refuses to write anything that looks like prose. See docs/BOT_DATA_PROTOCOL.md for what the bot should log.

Updating with new form data by hand

In the browser: open the page and use Load a newer export with the responses spreadsheet (Google Sheets → File → Download → Microsoft Excel). Everything recomputes. Questions are matched by wording; a renamed question is listed as "not found", and then COLUMN_KEYS in core/spec/form.yaml (and, until the site stops coding rows itself, core/engine.js) need updating.

To update the bundled snapshot, put the new export in data/raw/form/, then:

python collection/sources/export_raw.py data/raw/form/NEW_EXPORT.xlsx data/public/form_responses.json
PYTHONPATH=core python -m atlas.coding data/public/form_responses.json data/derived/runs_clean.csv
npm test

Changing a coding rule

The tables (how each question is found, what each answer means, which answers count as missing) are in core/spec/form.yaml; the logic that combines them is code_row() in core/atlas/coding.py. Add a case to tests/core/test_coding.py for every rule you change, then run npm test. Until the site switches to precomputed statistics, codeRow() in core/engine.js must be changed the same way (the parity tests check it).

Statistics in one paragraph

Rates use 95% Wilson score intervals. Each answer is compared with the other answers to the same question by odds ratio (Woolf CI, +0.5 Haldane correction for zero cells) and Fisher's exact test, with Benjamini–Hochberg q-values. Because Blue- and Black-chasers report rare sightings far more often than players who log every run, key outcomes are also modelled with ridge-penalised logistic regression (λ = 1) that includes data source and chase goal, and route rankings use a Mantel–Haenszel odds ratio stratified by the same factors. The page's Methods section has the details.

Privacy and licences

Code: MIT (LICENSE). Data produced by this project: CC BY 4.0 (DATA_LICENSE.md), which also lists third-party sources that keep their own terms. The rules every part follows (what may be published, what stays local, how bot accounts are anonymised) are in docs/PRIVACY.md.

Release checks

npm test                                     # parity and unit tests
python site/qa/check_colours.py              # dragon colours stay distinguishable (docs/COLOURS.md)
npm run build && python3 -m http.server 8765 -d dist &
node site/qa/crawl.js /tmp/qa                # every tab on desktop and phone: text, links, buttons, errors, overflow
node site/qa/mapcheck.js /tmp/qa             # every dragon's routes: off-map labels, overlaps, too-faint dragons
S
Description
No description provided
Readme MIT
118 MiB
Languages
Python 49.4%
JavaScript 39%
HTML 11.4%
Dockerfile 0.2%