kathleenandClaude Opus 5.5 b51a6c7039
CI / test (pull_request) Successful in 40s
The Black on the page: a first-attempt range marked low evidence; the Black panels load again
- The odds card for the Black: "Low evidence", "about 1 in N" first attempts, the 95% range written as
  "between 1 in X and 1 in Y", later attempts effectively never, why it is uncertain (every logged shadow is a
  rider who chose to post), and the steps (shadow -> reached the Black -> bonded). Dashed border.
- Valley map: the Black's routes are solved for a first attempt (they showed ~0% per attempt), labelled "on a first
  attempt", and "Why this route?" gives the same range and caveat.
- "The Black is not dice" panel: why the odds are uncertain, and what replaces the assumption (fresh-account bot
  runs to the tree).
- analysis/black/evidence.py wrote "odds_ratio": Infinity (no later-attempt shadow at all), which is not JSON:
  the page's two Black panels showed "could not be loaded". Now null, and the report says "unbounded".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 18:40:01 +00:00
2026-11-01 16:17:21 -05:00

Threshing Atlas

Route probabilities and path finding for the Dragonkind threshing: which choices lead to which dragon, how often, how sure we can be, and (in progress) the best route to each colour.

Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publishers.

Layout

The project has four parts, built on one shared core. Each folder has its own README.

core/           the shared model every part uses. spec/ holds the declarative rules (form.yaml: the community
                form; game.yaml: the game); atlas/ is the Python package that reads them (coding, route model).
                solver.js (the route advisor) and graph.js (the bot's text) are the only JavaScript, because
                they run in the browser and in Ken's bot.
collection/     getting data in, as extract -> transform -> load: pipeline.yaml declares every source (the
                community spreadsheet, Reddit threads, "Report a run", the bot, the route PDF, the register);
                `python -m collection` runs it and builds data/public/runs_clean.csv from all of them.
analysis/       making sense of it: stats/ (measures.yaml and site_stats.py: every number on the page), network/ (builds
                data/public/graph.json), reports/ (generated reports such as POLICY.md)
planner/        the route planner: builds data/public/policy.json and its report; the bot uses it for advice
                and experiments (docs/BOT_INTEGRATION.md)
site/           the Atlas web page (site/atlas/), its build script (site/build.js) and QA checks (site/qa/)
data/           public/ (everything published, including runs_clean.csv), dictionary.yaml (every column),
                croissant.json (dataset metadata), manifest.json (current raw versions), bot/ (anonymised bot
                runs), derived/ (PDF fingerprint), raw/ (current + previous raw files; never committed)
docs/           guides that cross sections: privacy and licences, repo settings, bot data protocol, bot
                integration, colours, roadmap
tests/          core/ (rule, statistics, route model and advisor tests), collection/ (the pipeline), parity/
                (tools to compare two policies), fixtures/

Run it locally

Requirements: Node 18+, Python 3.9+ (pip install -r requirements.txt).

npm run dev      # builds dist/ and serves it at http://localhost:8000
npm test         # pipeline, coding rules, statistics, route model, game spec, advisor and bot-text tests

New data from the community sources

A scheduled workflow (.gitea/workflows/update-sources.yml on the Atlas's Gitea; .github/workflows/ holds the GitHub version) runs the collection pipeline every day (python -m collection update, see collection/README.md). It checks every source in collection/pipeline.yaml, and if anything changed it rebuilds the runs and the analysis, runs the tests and opens a pull request that says what changed: new responses, new or renamed questions, new run reports, new result links on Reddit, and for the PDF the pages and lines that are new. The PDF itself is never committed, only a fingerprint. If a source cannot be reached, or new answers do not match data/dictionary.yaml, the Action opens an issue instead. Locally: npm run collect.

Best routes (the solver)

npm run policy (Python: planner/build_policy.py, from the game in core/spec/game.yaml) rebuilds data/public/policy.json and the readable analysis/reports/POLICY.md (about 2 minutes: 200 posterior draws for two data samples). For every colour it gives the best decision at every point under two objectives: the chance of bonding that colour on this attempt, and the chance the account eventually ends on that colour when you die on purpose whenever a run goes wrong. Chance events (which dragon appears, whether the shadow shows) may depend on earlier choices; how much is learned from the data. See the top of core/solver.js for the model and docs/BOT_INTEGRATION.md for how the bot uses it.

Bot data

The private DragonKind bot's files are converted locally, and only the anonymised result is committed:

DRAGONKIND_BOT_DIR=/path/to/DragonKind DK_ACCOUNT_SALT=... python -m collection update --source bot
npm run graph

This writes data/bot/scenes.json and data/bot/trajectories-<date>.jsonl: scene fingerprints, short labels (collection/specs/bot.yaml), choice text and outcomes, with no game prose and no account names, and adds the bot's attempts to runs_clean.csv (source bot). The importer refuses to write anything that looks like prose. See docs/BOT_DATA_PROTOCOL.md for what the bot should log.

Running everything locally

python -m collection refresh does what the daily workflow does (python -m collection rebuild alone redoes everything built from the runs): checks every source, rebuilds the runs, the graph, the route model and the statistics. Add --bot <folder or zip> to import bot runs, and python -m collection export-bot-kit zips the files the bot uses (see docs/BOT_INTEGRATION.md).

Updating with new form data by hand

Download the responses spreadsheet (Google Sheets → File → Download → Microsoft Excel), then:

python -m collection update --source reddit_spreadsheet --file reddit_spreadsheet=path/to/export.xlsx
npm test

Questions are matched by wording; a renamed question is listed in the report, and then core/spec/form.yaml needs updating.

Changing a coding rule

The tables (how each question is found, what each answer means, which answers count as missing) are in core/spec/form.yaml; the logic that combines them is code_row() in core/atlas/coding.py. Add a case to tests/core/test_coding.py for every rule you change, then run npm test.

Statistics in one paragraph

Rates use 95% Wilson score intervals. Each answer is compared with the other answers to the same question by odds ratio (Woolf CI, +0.5 Haldane correction for zero cells) and Fisher's exact test, with Benjamini–Hochberg q-values. Because Blue- and Black-chasers report rare sightings far more often than players who log every run, key outcomes are also modelled with ridge-penalised logistic regression (λ = 1) that includes data source and chase goal, and route rankings use a Mantel–Haenszel odds ratio stratified by the same factors. The page's Methods section has the details.

Privacy and licences

Code: MIT (LICENSE). Data produced by this project: CC BY 4.0 (DATA_LICENSE.md), which also lists third-party sources that keep their own terms. The rules every part follows (what may be published, what stays local, how bot accounts are anonymised) are in docs/PRIVACY.md.

Release checks

npm test                                     # all tests
python site/qa/check_colours.py              # dragon colours stay distinguishable (docs/COLOURS.md)
npm run build && python3 -m http.server 8765 -d dist &
node site/qa/crawl.js /tmp/qa                # every tab on desktop and phone: text, links, buttons, errors, overflow
node site/qa/mapcheck.js /tmp/qa             # every dragon's routes: off-map labels, overlaps, too-faint dragons
S
Description
No description provided
Readme MIT
118 MiB
Languages
Python 49.4%
JavaScript 39%
HTML 11.4%
Dockerfile 0.2%