Files
threshing-atlas/README.md
T
11b4a0398e Core: a small JS advisor reads the game from policy.json; engine.js is gone (#13)
Step 4b of the Python-first core.

- core/solver.js is now a 250-line advisor (was 900 lines): it reads the game (core/spec/game.yaml,
  published inside policy.json as model.game) and the count tables, and re-solves from any point in
  a run with the same interpreter as the Python model. Same API for the site and Ken's bot
  (advisor, recommendOption, solverHistory). The bot's on-screen text for each option moves to
  game.yaml (bot_text); the deadly rolls carry their risk labels there too.
- tests/core/advisor.test.js: the advisor gives exactly the Python model's values and choices for
  every colour and goal (to 1e-12, every decision on every published route), walks a run in the
  game's order, and maps advice onto the bot's screen.
- Removed core/engine.js and everything that only compared it with Python (tests/parity/run_engine,
  compare_runs, compare_results, compare_routes, compare_solver, compare_stats, analysis/stats/
  analyze.py). In their place: tests/core/test_stats.py checks the statistics against known answers,
  test_solver.py checks the route model on a synthetic world (tests/fixtures/synthetic_policy.py).
- analysis/network/build-graph.js reads the Python-coded runs; graph.json is byte-identical.
- docs/BOT_INTEGRATION.md: what changed for the bot (history keys, new choices, re-sync all three files).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 22:01:32 +00:00

116 lines
6.3 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Threshing Atlas
Route probabilities and path finding for the Dragonkind threshing: which choices lead to which
dragon, how often, how sure we can be, and (in progress) the best route to each colour.
Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publishers.
## Layout
The project has four parts, built on one shared core. Each folder has its own README.
```
core/ the shared model every part uses. spec/ holds the declarative rules (form.yaml: the community
form; game.yaml: the game); atlas/ is the Python package that reads them (coding, route model).
solver.js (the route advisor) and graph.js (the bot's text) are the only JavaScript, because
they run in the browser and in Ken's bot.
collection/ getting data in: sources/ (the form sheet, the Dragon Register, the route PDF check, the daily
update job) and bot/ (the DragonKind bot importer). Reddit is not collected yet.
analysis/ making sense of it: stats/ (measures.yaml and site_stats.py: every number on the page), network/ (builds
data/public/graph.json), reports/ (generated reports such as POLICY.md)
planner/ the route planner: builds data/public/policy.json and its report; the bot uses it for advice
and experiments (docs/BOT_INTEGRATION.md)
site/ the Atlas web page (site/atlas/), its build script (site/build.js) and QA checks (site/qa/)
data/ raw/ (source exports, not committed), public/ (what the page and the bot load), derived/
(coded runs, PDF fingerprint), bot/ (anonymised bot runs), sources.json (download links)
docs/ guides that cross sections: privacy and licences, repo settings, bot data protocol, bot
integration, colours, roadmap
tests/ core/ (rule, statistics, route model and advisor tests), parity/ (tools to compare two policies), fixtures/
```
## Run it locally
Requirements: Node 18+, Python 3.9+ (`pip install -r requirements.txt`).
```
npm run dev # builds dist/ and serves it at http://localhost:8000
npm test # coding rules, statistics, route model, game spec, advisor and bot-text tests
```
## New data from the community sources
A GitHub Action (`.github/workflows/update-sources.yml`) runs every day. It downloads the Threshing Routes
responses, the Dragon Register and the route PDF, and if anything changed it rebuilds the data, runs the
tests and opens a pull request that says what changed: new responses, new or renamed questions, and for
the PDF the pages and lines that are new. The PDF itself is never committed, only a fingerprint
(`data/derived/pdf_fingerprint.json`). If a link stops working, the Action opens an issue. You can also
run it by hand from the Actions tab, or locally with `npm run check-sources`.
## Best routes (the solver)
`npm run policy` (Python: `planner/build_policy.py`, from the game in `core/spec/game.yaml`) rebuilds `data/public/policy.json` and the readable `analysis/reports/POLICY.md` (about 2 minutes:
200 posterior draws for two data samples). For every colour it gives the best decision at every point under two
objectives: the chance of bonding that colour on this attempt, and the chance the account eventually ends on that
colour when you die on purpose whenever a run goes wrong. Chance events (which dragon appears, whether the shadow
shows) may depend on earlier choices; how much is learned from the data. See the top of
`core/solver.js` for the model and `docs/BOT_INTEGRATION.md` for how the bot uses it.
## Bot data
The private DragonKind bot's files are converted locally, and only the anonymised result is committed:
```
DK_ACCOUNT_SALT=... npm run import-bot -- --bot /path/to/DragonKind --tag 2026-11-01
npm run graph
```
This writes `data/bot/scenes.json` and `data/bot/trajectories-<tag>.jsonl`: scene fingerprints, short labels,
choice text and outcomes, with no game prose and no account names. The importer refuses to write anything that
looks like prose. See `docs/BOT_DATA_PROTOCOL.md` for what the bot should log.
## Updating with new form data by hand
In the browser: open the page and use **Load a newer export** with the responses spreadsheet
(Google Sheets → File → Download → Microsoft Excel). Everything recomputes. Questions are matched by
wording; a renamed question is listed as "not found", and then `COLUMN_KEYS` in
`core/spec/form.yaml` needs updating.
To update the bundled snapshot, put the new export in `data/raw/form/`, then:
```
python collection/sources/export_raw.py data/raw/form/NEW_EXPORT.xlsx data/public/form_responses.json
PYTHONPATH=core python -m atlas.coding data/public/form_responses.json data/derived/runs_clean.csv
npm test
```
## Changing a coding rule
The tables (how each question is found, what each answer means, which answers count as missing) are in
`core/spec/form.yaml`; the logic that combines them is `code_row()` in `core/atlas/coding.py`. Add a case to
`tests/core/test_coding.py` for every rule you change, then run `npm test`.
## Statistics in one paragraph
Rates use 95% Wilson score intervals. Each answer is compared with the other answers to the same
question by odds ratio (Woolf CI, +0.5 Haldane correction for zero cells) and Fisher's exact test, with
Benjamini–Hochberg q-values. Because Blue- and Black-chasers report rare sightings far more often than
players who log every run, key outcomes are also modelled with ridge-penalised logistic regression
(λ = 1) that includes data source and chase goal, and route rankings use a Mantel–Haenszel odds ratio
stratified by the same factors. The page's Methods section has the details.
## Privacy and licences
Code: MIT (`LICENSE`). Data produced by this project: CC BY 4.0 (`DATA_LICENSE.md`), which also lists
third-party sources that keep their own terms. The rules every part follows (what may be published, what stays
local, how bot accounts are anonymised) are in `docs/PRIVACY.md`.
## Release checks
```
npm test # all tests
python site/qa/check_colours.py # dragon colours stay distinguishable (docs/COLOURS.md)
npm run build && python3 -m http.server 8765 -d dist &
node site/qa/crawl.js /tmp/qa # every tab on desktop and phone: text, links, buttons, errors, overflow
node site/qa/mapcheck.js /tmp/qa # every dragon's routes: off-map labels, overlaps, too-faint dragons
```