Refactor: lay the repo out by section (core, collection, analysis, planner, site) (#8)

Moves only; no behaviour changes. Every file now lives under the part of the project it
belongs to, each section has a README, and docs/PRIVACY.md collects the rules all parts follow.

  core/        engine.js, graph.js, solver.js, python/clean.py
  collection/  sources/ (form, register, PDF check, daily update), bot/ (importer)
  analysis/    stats/ (Python statistics), network/ (build-graph.js), reports/ (POLICY.md)
  planner/     build-policy.js, policy-report.js
  site/        atlas/ (the page), build.js, qa/
  tests/       core/ (unit), parity/ (JS vs Python), fixtures/

Checked against the previous layout: tests pass; coded runs, test outputs, graph.json and
policy.json are byte-identical; the built site differs only in comments that name file paths.
The published file names Ken's bot downloads (solver.js, graph.js, data/policy.json) are unchanged.

Also removes the undefined/ folder of QA screenshots committed by mistake (the QA scripts now
default to a temp folder) and test.txt, and drops the unused npm workspaces.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ekcSchiano
2026-11-03 19:46:33 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent dcd5117373
commit 432e04c2c7
66 changed files with 201 additions and 2695 deletions
+2 -2
View File
@@ -14,8 +14,8 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r research/python/requirements.txt
- run: pip install -r requirements.txt
- run: npm test
- name: graph.json is up to date
run: node scripts/build-graph.js && git diff --exit-code data/public/graph.json
run: node analysis/network/build-graph.js && git diff --exit-code data/public/graph.json
- run: npm run build
+3 -3
View File
@@ -21,13 +21,13 @@ jobs:
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- run: pip install -r research/python/requirements.txt
- run: pip install -r requirements.txt
- id: check
run: python research/python/update_sources.py --report "$RUNNER_TEMP/report.md"
run: python collection/sources/update_sources.py --report "$RUNNER_TEMP/report.md"
- name: Rebuild graph and run tests
if: steps.check.outputs.changed == 'true'
run: |
node scripts/build-graph.js
node analysis/network/build-graph.js
npm run policy
npm test
# The PR is opened with ATLAS_BOT_TOKEN (a fine-grained token, see docs/REPO_SETTINGS.md) when it is set:
+3
View File
@@ -232,3 +232,6 @@ data/raw/register/
# Third-party guide PDF: link to it, do not redistribute
*.pdf
data/raw/form/*.xlsx
# QA screenshots
undefined/
+31 -35
View File
@@ -7,32 +7,29 @@ Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publis
## Layout
The project has four parts, built on one shared core. Each folder has its own README.
```
apps/atlas/ the web page. The Valley (explorer.js: the map, the dragons, routes on demand; dragons.js:
original silhouettes), The Odds, The Dragons, Deep Dive, Data & Methods
packages/core/ @threshing-atlas/core, no dependencies, runs in Node and the browser:
src/engine.js coding rules and statistics for the community form
src/graph.js the shared route graph: scene graph, decision-variable graph,
bot-to-form crosswalk, Beta credible intervals
src/solver.js best decisions per dragon colour (history-dependent decision tree,
empirical-Bayes back-off, two objectives) + the bot advisor
packages/ingest/ @threshing-atlas/ingest: DragonKind bot importer (fingerprints and choice text only)
research/python/ independent Python implementation of the coding rules and statistics
tests/ JS/Python parity tests (they fail loudly if the two disagree)
scripts/build-site.js assembles the static site into dist/
scripts/build-graph.js combines every source into data/public/graph.json
scripts/build-policy.js solves for every colour -> data/public/policy.json; policy-report.js -> docs/POLICY.md
data/raw/ source exports (form .xlsx; the register export stays local, git-ignored)
data/public/ what the page loads: form_responses.json, register.json, graph.json
data/sources.json where the community spreadsheets and the PDF are downloaded from
data/derived/ coded runs (runs_clean.csv)
data/bot/ anonymised DragonKind bot runs, added by pull request
docs/ ROADMAP.md, BOT_DATA_PROTOCOL.md, REDDIT_POST.md
core/ the shared model every part uses: the run format and coding rules (engine.js, with the
Python mirror in core/python/clean.py), the route graph and bot crosswalk (graph.js), and the
decision model and bot advisor (solver.js). No dependencies; runs in Node and the browser.
collection/ getting data in: sources/ (the form sheet, the Dragon Register, the route PDF check, the daily
update job) and bot/ (the DragonKind bot importer). Reddit is not collected yet.
analysis/ making sense of it: stats/ (Python statistics, mirrored in core/engine.js), network/ (builds
data/public/graph.json), reports/ (generated reports such as POLICY.md)
planner/ the route planner: builds data/public/policy.json and its report; the bot uses it for advice
and experiments (docs/BOT_INTEGRATION.md)
site/ the Atlas web page (site/atlas/), its build script (site/build.js) and QA checks (site/qa/)
data/ raw/ (source exports, not committed), public/ (what the page and the bot load), derived/
(coded runs, PDF fingerprint), bot/ (anonymised bot runs), sources.json (download links)
docs/ guides that cross sections: privacy and licences, repo settings, bot data protocol, bot
integration, colours, roadmap
tests/ core/ (unit tests), parity/ (the JS and Python implementations must agree), fixtures/
```
## Run it locally
Requirements: Node 18+, Python 3.9+ (`pip install -r research/python/requirements.txt`).
Requirements: Node 18+, Python 3.9+ (`pip install -r requirements.txt`).
```
npm run dev # builds dist/ and serves it at http://localhost:8000
@@ -50,12 +47,12 @@ run it by hand from the Actions tab, or locally with `npm run check-sources`.
## Best routes (the solver)
`npm run policy` rebuilds `data/public/policy.json` and the readable `docs/POLICY.md` (about 2.5 minutes:
`npm run policy` rebuilds `data/public/policy.json` and the readable `analysis/reports/POLICY.md` (about 4 minutes:
200 posterior draws for two data samples). For every colour it gives the best decision at every point under two
objectives: the chance of bonding that colour on this attempt, and the chance the account eventually ends on that
colour when you die on purpose whenever a run goes wrong. Chance events (which dragon appears, whether the shadow
shows) may depend on earlier choices; how much is learned from the data. See the top of
`packages/core/src/solver.js` for the model and `docs/BOT_INTEGRATION.md` for how the bot uses it.
`core/solver.js` for the model and `docs/BOT_INTEGRATION.md` for how the bot uses it.
## Bot data
@@ -75,22 +72,20 @@ looks like prose. See `docs/BOT_DATA_PROTOCOL.md` for what the bot should log.
In the browser: open the page and use **Load a newer export** with the responses spreadsheet
(Google Sheets → File → Download → Microsoft Excel). Everything recomputes. Questions are matched by
wording; a renamed question is listed as "not found", and then `COLUMN_KEYS` in
`packages/core/src/engine.js` and `research/python/clean.py` need updating.
`core/engine.js` and `core/python/clean.py` need updating.
To update the bundled snapshot, put the new export in `data/raw/form/`, then:
```
cd research/python
python export_raw.py ../../data/raw/form/NEW_EXPORT.xlsx ../../data/public/form_responses.json
python clean.py ../../data/raw/form/NEW_EXPORT.xlsx
cd ../..
python collection/sources/export_raw.py data/raw/form/NEW_EXPORT.xlsx data/public/form_responses.json
python core/python/clean.py data/public/form_responses.json data/derived/runs_clean.csv
npm test
```
## Changing a coding rule
Every rule lives in `codeRow()` in `packages/core/src/engine.js` and `code_row()` in
`research/python/clean.py`. Change both and run `npm test`.
Every rule lives in `codeRow()` in `core/engine.js` and `code_row()` in
`core/python/clean.py`. Change both and run `npm test`.
## Statistics in one paragraph
@@ -101,17 +96,18 @@ players who log every run, key outcomes are also modelled with ridge-penalised l
(λ = 1) that includes data source and chase goal, and route rankings use a Mantel–Haenszel odds ratio
stratified by the same factors. The page's Methods section has the details.
## Licences
## Privacy and licences
Code: MIT (`LICENSE`). Data produced by this project: CC BY 4.0 (`DATA_LICENSE.md`), which also lists
third-party sources that keep their own terms.
third-party sources that keep their own terms. The rules every part follows (what may be published, what stays
local, how bot accounts are anonymised) are in `docs/PRIVACY.md`.
## Release checks
```
npm test # parity and unit tests
python research/python/check_colours.py # dragon colours stay distinguishable (docs/COLOURS.md)
python site/qa/check_colours.py # dragon colours stay distinguishable (docs/COLOURS.md)
npm run build && python3 -m http.server 8765 -d dist &
node scripts/qa/crawl.js /tmp/qa # every tab on desktop and phone: text, links, buttons, errors, overflow
node scripts/qa/mapcheck.js /tmp/qa # every dragon's routes: off-map labels, overlaps, too-faint dragons
node site/qa/crawl.js /tmp/qa # every tab on desktop and phone: text, links, buttons, errors, overflow
node site/qa/mapcheck.js /tmp/qa # every dragon's routes: off-map labels, overlaps, too-faint dragons
```
+9
View File
@@ -0,0 +1,9 @@
# analysis
Making sense of the data.
| Folder | What it does |
|---|---|
| `stats/` | General statistics in Python (`analyze.py`, `stats_core.py`): rates with Wilson intervals, odds ratios with Fisher's test and Benjamini–Hochberg, ridge logistic models that adjust for data source and chase goal. `core/engine.js` computes the same numbers for the site, and `tests/parity/` checks they match. |
| `network/` | `build-graph.js` combines every source into `data/public/graph.json` (the decision-variable graph and the bot's scene graph). CI fails if the committed graph is out of date. |
| `reports/` | Generated reports. `POLICY.md` is written by `planner/policy-report.js`; do not edit it by hand. |
@@ -1,10 +1,10 @@
// Build data/public/graph.json from every source: community form runs + DragonKind bot trajectories.
// node scripts/build-graph.js
// node analysis/network/build-graph.js
const fs = require("fs");
const path = require("path");
const E = require("../packages/core/src/engine.js");
const G = require("../packages/core/src/graph.js");
const root = path.resolve(__dirname, "..");
const E = require("../../core/engine.js");
const G = require("../../core/graph.js");
const root = path.resolve(__dirname, "../..");
const rel = (p) => path.join(root, p);
const raw = JSON.parse(fs.readFileSync(rel("data/public/form_responses.json"), "utf8"));
@@ -1,6 +1,6 @@
# Best routes by dragon colour
Generated from 5988 community runs by `scripts/build-policy.js`. Machine-readable version: `data/public/policy.json`.
Generated from 5988 community runs by `planner/build-policy.js`. Machine-readable version: `data/public/policy.json`.
**How to read this.** "This attempt" maximises the chance of bonding the colour on one run. "Eventually" assumes you will die on purpose and try again whenever the run goes wrong, so it avoids any choice that can end the account on a different dragon. Percentages are posterior means with 95% credible intervals. "Held-out" is the same policy chosen on half of the runs and scored on the other half: when it is much lower than the in-sample number, the route's advantage is mostly noise.
@@ -3,7 +3,7 @@ analyze.py — Python reference implementation of the statistics shown on the ro
python analyze.py ../../data/derived/runs_clean.csv ../../tests/out/results_py.json
It mirrors analyse() in packages/core/src/engine.js (same denominators, same tests, same model); tests/compare_results.py
It mirrors analyse() in core/engine.js (same denominators, same tests, same model); tests/parity/compare_results.py
checks that both produce the same numbers. Edit either, then re-run the comparison.
"""
import json
@@ -1,4 +1,4 @@
"""Small, dependency-light statistics used by the analysis (mirrored in packages/core/src/engine.js)."""
"""Small, dependency-light statistics used by the analysis (mirrored in core/engine.js)."""
import math
import numpy as np
+19
View File
@@ -0,0 +1,19 @@
# collection
Getting data in. Everything here turns a source into the shared run format (see `core/`) or into public files
under `data/`.
| Source | Code | Output |
|---|---|---|
| Community "Threshing Routes" form (Google Sheet) | `sources/export_raw.py`, then `core/python/clean.py` | `data/public/form_responses.json`, `data/derived/runs_clean.csv` |
| Unofficial Dragon Register (Google Sheet) | `sources/export_register.py` | `data/public/register.json` (rider names stay local) |
| Community route PDF | `sources/update_sources.py` | `data/derived/pdf_fingerprint.json` only; the PDF is never committed |
| DragonKind bot (Ken) | `bot/import-bot.js`, `bot/bot.js` | `data/bot/scenes.json`, `data/bot/trajectories-<tag>.jsonl` |
| Reddit megathreads | not collected yet | — |
`sources/update_sources.py` runs every day from `.github/workflows/update-sources.yml`. It downloads the sheets
and the PDF (links in `data/sources.json`), rebuilds the data if anything changed, and opens a pull request that
merges itself when CI passes. A broken link opens an issue.
The bot importer only ever writes fingerprints, short labels, choice text and outcomes. It refuses anything that
looks like game prose or an account identifier. What the bot should log is in `docs/BOT_DATA_PROTOCOL.md`.
@@ -3,7 +3,7 @@
/**
* Convert a local copy of the DragonKind bot's data into the public, anonymised format.
*
* node packages/ingest/bin/import-bot.js --bot /path/to/DragonKind [--out data/bot] [--tag 2026-11-01]
* node collection/bot/import-bot.js --bot /path/to/DragonKind [--out data/bot] [--tag 2026-11-01]
*
* Reads <bot>/data/knowledge.json, <bot>/data/runs.jsonl and <bot>/logs/encounters.jsonl.
* Writes <out>/scenes.json (merged with any existing catalogue) and <out>/trajectories-<tag>.jsonl.
@@ -11,7 +11,7 @@
*/
const fs = require('node:fs');
const path = require('node:path');
const { importBot, assertPublicSafe } = require('../src/bot');
const { importBot, assertPublicSafe } = require('./bot');
const args = Object.fromEntries(process.argv.slice(2).reduce((acc, a, i, all) => {
if (a.startsWith('--')) acc.push([a.slice(2), all[i + 1]]);
@@ -51,4 +51,4 @@ fs.writeFileSync(trajFile, trajectories.map((t) => JSON.stringify(t)).join('\n')
const outcomes = trajectories.reduce((m, t) => ((m[t.outcome.type] = (m[t.outcome.type] || 0) + 1), m), {});
console.log(`${Object.keys(scenes).length} scenes, ${trajectories.length} attempts ${JSON.stringify(outcomes)}`);
console.log(`wrote ${path.relative(process.cwd(), sceneFile)} and ${path.relative(process.cwd(), trajFile)}`);
if (unlabeled.length) console.log(`scenes without a short label (add them to LABELS in packages/ingest/src/bot.js):\n ${unlabeled.join('\n ')}`);
if (unlabeled.length) console.log(`scenes without a short label (add them to LABELS in collection/bot/bot.js):\n ${unlabeled.join('\n ')}`);
@@ -1,6 +1,6 @@
"""update_sources.py: check the community spreadsheets and the route PDF for new information.
python research/python/update_sources.py [--report report.md] [--form FILE] [--register FILE] [--pdf FILE]
python collection/sources/update_sources.py [--report report.md] [--form FILE] [--register FILE] [--pdf FILE]
Run from the repo root. Without file arguments it downloads the current versions from the URLs in
data/sources.json (the GitHub Action does this every day). With file arguments it uses local copies,
@@ -73,8 +73,8 @@ def update_form(src, report):
if headers == current["headers"] and not added and not removed:
report.append(f"- **Threshing Routes form:** no change ({len(rows)} responses).")
return False
subprocess.run([sys.executable, P("research", "python", "export_raw.py"), src, out], check=True, cwd=HERE)
subprocess.run([sys.executable, P("research", "python", "clean.py"), out, P("data", "derived", "runs_clean.csv")], check=True, cwd=HERE)
subprocess.run([sys.executable, P("collection", "sources", "export_raw.py"), src, out], check=True, cwd=HERE)
subprocess.run([sys.executable, P("core", "python", "clean.py"), out, P("data", "derived", "runs_clean.csv")], check=True, cwd=HERE)
raw_copy = P("data", "raw", "form", "Threshing_Routes_Responses.xlsx")
if os.path.abspath(src) != raw_copy: # local copy only; data/raw/form/*.xlsx is git-ignored
os.makedirs(os.path.dirname(raw_copy), exist_ok=True)
@@ -98,7 +98,7 @@ def update_register(src, report):
if headers == current["headers"] and not added and not removed:
report.append(f"- **Dragon Register:** no change ({len(rows)} dragons).")
return False
subprocess.run([sys.executable, P("research", "python", "export_register.py"), src, out], check=True, cwd=HERE)
subprocess.run([sys.executable, P("collection", "sources", "export_register.py"), src, out], check=True, cwd=HERE)
report.append(f"- **Dragon Register:** {len(current['rows'])} → {len(rows)} entries ({added} new or edited, {removed} removed or edited).")
return True
@@ -107,7 +107,7 @@ def pdf_lines(path):
try:
from pypdf import PdfReader
except ImportError:
sys.exit("pypdf is required: pip install -r research/python/requirements.txt")
sys.exit("pypdf is required: pip install -r requirements.txt")
pages = []
for page in PdfReader(path).pages:
text = page.extract_text() or ""
+16
View File
@@ -0,0 +1,16 @@
# core
The shared model. Every other part uses it, so a change here touches everything: run `npm test` after any edit.
| File | What it does |
|---|---|
| `engine.js` | Turns a row of the community form (any version of it) into one coded run: the run format, every coding rule, missing answers, email redaction. Also the statistics the site shows. |
| `python/clean.py` | The same coding rules in Python. `tests/parity/` fails if the two ever disagree. |
| `graph.js` | The route graph: the bot's scene graph, the decision-variable graph shared by all sources, the crosswalk from the bot's on-screen text to form answers, Beta credible intervals. |
| `solver.js` | The decision model: every choice and roll in a run, chance estimates with empirical-Bayes back-off, backward induction for each dragon, and the advisor the site and the bot use to get advice from any point in a run. |
All three JS files have no dependencies and run unchanged in Node and the browser. The site and Ken's bot load
the published copies (`https://ekcschiano.github.io/threshing-atlas/solver.js` etc.), so their file names on the
site must not change.
To change a coding rule, edit `codeRow()` in `engine.js` **and** `code_row()` in `python/clean.py`.
@@ -3,7 +3,7 @@
*
* One file, used two ways:
* - in the browser (the route map page loads it and recomputes everything from the raw form export)
* - in Node (`node tests/run_engine.js data.json`), so results can be reproduced and checked against
* - in Node (`node tests/parity/run_engine.js data.json`), so results can be reproduced and checked against
* the Python reference implementation in python/.
*
* Pipeline: raw form rows -> codeRow() (one coded run per row) -> analyse() -> results object
@@ -2,8 +2,8 @@
clean.py — turn the raw "Threshing Routes Responses" Google Form export into one
coded row per threshing run.
Reference implementation in Python. The web page (packages/core/src/engine.js) implements the
same rules in JavaScript; tests/compare.py checks that both give identical output.
Reference implementation in Python. The web page (core/engine.js) implements the
same rules in JavaScript; tests/parity/compare_runs.py checks that both give identical output.
Usage:
python clean.py ../../data/raw/form/Threshing_Routes_Responses.xlsx -> writes ../../data/derived/runs_clean.csv
+2 -2
View File
@@ -2,9 +2,9 @@
| Source | Where it lives here | How to update |
|---|---|---|
| Community "Threshing Routes" Google Form (u/ClumsyBadger) | `raw/form/*.xlsx` → `public/form_responses.json` | Download the sheet as .xlsx, then `python research/python/export_raw.py` |
| Community "Threshing Routes" Google Form (u/ClumsyBadger) | `raw/form/*.xlsx` → `public/form_responses.json` | Download the sheet as .xlsx, then `python collection/sources/export_raw.py` |
| Community route PDF (u/ClumsyBadger) | not redistributed; linked from the Reddit megathreads | A scene crosswalk will be kept in `derived/` |
| Unofficial Dragon Register | `raw/register/` (git-ignored, has rider names) → `public/register.json` | `python research/python/export_register.py` |
| Unofficial Dragon Register | `raw/register/` (git-ignored, has rider names) → `public/register.json` | `python collection/sources/export_register.py` |
| Reddit megathreads (r/fourthwing, Dragonkind bonding Parts 3 and 4) | qualitative; routes cited in the valley map | manual |
| DragonKind bot exports | `bot/runs-YYYY-MM-DD.jsonl` | by pull request, see `docs/BOT_DATA_PROTOCOL.md` |
| Discord | not yet | planned, with admin consent and anonymised usernames |
+1 -1
View File
@@ -107,7 +107,7 @@ and it is exactly the randomised data the atlas needs to tell real history effec
- `samples.lowBias`: bulk logs and riders not chasing a rare dragon. Use it for absolute expectations, such as how
many attempts a Blue should take. Rare sightings are over-reported in form posts, so `all` is optimistic for Blue.
Current headline (from `docs/POLICY.md`): Black about 1.7% per attempt (about 60 attempts), Blue 3.7–7% per attempt
Current headline (from `analysis/reports/POLICY.md`): Black about 1.7% per attempt (about 60 attempts), Blue 3.7–7% per attempt
(14–28 attempts), depending on the sample.
## 2. From the bot to the atlas
+1 -1
View File
@@ -4,7 +4,7 @@ Two palettes, one per background. Both were chosen so that every pair of dragon
readers with the common colour-vision deficiencies, not only with normal vision. Check them with:
```
python research/python/check_colours.py
python site/qa/check_colours.py
```
It reports CIEDE2000 colour differences for every pair under normal vision and simulated protanopia, deuteranopia and
+34
View File
@@ -0,0 +1,34 @@
# Privacy and licences
Rules every part of the project follows. If a change would break one, stop and ask first.
## What may be published
- **Code** is MIT (`LICENSE`). **Data this project produces** is CC BY 4.0 (`DATA_LICENSE.md`). Third-party
sources keep their own terms.
- **Community form responses:** published as exported, with email addresses in free-text answers redacted by
the coding rules (`core/engine.js`, `core/python/clean.py`).
- **Dragon Register:** only `data/public/register.json`. The raw export has rider names and stays local
(`data/raw/register/` is git-ignored).
- **The community route PDF:** never committed or redistributed. Only a fingerprint
(`data/derived/pdf_fingerprint.json`) so we can tell when it changes, and a link to the original.
- **Bot data:** only scene fingerprints, short labels, choice text and outcomes. Never game prose, never account
names. The importer (`collection/bot/bot.js`) refuses to write anything that looks like either.
## The bot
- Ken's bot repository stays private. Only the anonymised import comes here.
- Account ids are salted and hashed. The salt (`DK_ACCOUNT_SALT`) lives in the environment and is never
committed.
- The bot's browser profiles, `.env` and credential files are never committed or shared.
## Sources we do not scrape
- **Discord:** not without the server admins' consent; never with a personal account token.
- **Reddit:** not collected yet. When it is, it should go through Reddit's official API with an app key, not page
scraping.
## Secrets
- `ATLAS_BOT_TOKEN` (repository secret) lets the daily data update open and auto-merge its pull request
(`docs/REPO_SETTINGS.md`). Tokens go in secrets only: never in a file, issue, chat or commit.
+4 -4
View File
@@ -4,16 +4,16 @@ The aim is to move from summary statistics to path finding: for each dragon colo
policy that maximises the chance of bonding it, with honest uncertainty.
1. **Repo and packaging** (done): monorepo, CI, GitHub Pages, JS/Python parity tests.
2. **Graph format** (`packages/core`, first version done): typed scene, chance and terminal nodes; history-context keys;
2. **Graph format** (`core/`, first version done): typed scene, chance and terminal nodes; history-context keys;
per-source counts (bot, form, PDF, Discord); fingerprints and short labels only, no game prose.
Importers in `packages/ingest` for the bot's `knowledge.json`, `runs.jsonl` and `encounters.jsonl`
Importers in `collection/bot` for the bot's `knowledge.json`, `runs.jsonl` and `encounters.jsonl`
(done), the form (done, via the crosswalk in `graph.js`), and a PDF scene crosswalk (to do).
Daily source checks for the form, the register and the PDF (done).
3. **Context learning** (`research/`): tests of history dependence (G-tests, BH); variable-order
3. **Context learning** (`analysis/`): tests of history dependence (G-tests, BH); variable-order
Markov / context tree with BIC; temporal-tiered DAG structure learning (BDeu/BIC); latent
"character" model vs specific triggers vs memoryless baseline, compared on held-out likelihood;
hierarchical Dirichlet back-off pooling.
4. **Solver** (first version done: `packages/core/src/solver.js`, `docs/POLICY.md`; still to do: richer history models from step 3): backward induction on the augmented state (scene, history summary), i.e. an influence
4. **Solver** (first version done: `core/solver.js`, `analysis/reports/POLICY.md`; still to do: richer history models from step 3): backward induction on the augmented state (scene, history summary), i.e. an influence
diagram with perfect recall. Two objectives side by side per colour:
- P(target colour on this attempt);
- eventual success when deliberate deaths are free: maximise p_X / (p_X + p_other bond),
+11 -12
View File
@@ -3,19 +3,18 @@
"private": true,
"description": "Route probabilities and path-finding for the Dragonkind threshing",
"license": "MIT",
"workspaces": [
"packages/*"
],
"scripts": {
"build": "node scripts/build-site.js",
"dev": "node scripts/build-site.js && python3 -m http.server 8000 -d dist",
"test:js": "node tests/run_engine.js data/public/form_responses.json tests/out",
"test:py": "cd research/python && python3 clean.py && python3 analyze.py",
"test": "npm run test:js && npm run test:py && python3 tests/compare_runs.py && python3 tests/compare_results.py && python3 tests/compare_routes.py && node --test tests/graph.test.js tests/solver.test.js",
"graph": "node scripts/build-graph.js",
"import-bot": "node packages/ingest/bin/import-bot.js",
"check-sources": "python3 research/python/update_sources.py",
"policy": "node --max-old-space-size=6000 scripts/build-policy.js && node scripts/policy-report.js"
"build": "node site/build.js",
"dev": "node site/build.js && python3 -m http.server 8000 -d dist",
"test:js": "node tests/parity/run_engine.js data/public/form_responses.json tests/out",
"test:py": "cd core/python && python3 clean.py && cd ../../analysis/stats && python3 analyze.py",
"test:parity": "python3 tests/parity/compare_runs.py && python3 tests/parity/compare_results.py && python3 tests/parity/compare_routes.py",
"test:unit": "node --test tests/core/graph.test.js tests/core/solver.test.js",
"test": "npm run test:js && npm run test:py && npm run test:parity && npm run test:unit",
"graph": "node analysis/network/build-graph.js",
"import-bot": "node collection/bot/import-bot.js",
"check-sources": "python3 collection/sources/update_sources.py",
"policy": "node --max-old-space-size=6000 planner/build-policy.js && node planner/policy-report.js"
},
"engines": {
"node": ">=18"
-8
View File
@@ -1,8 +0,0 @@
{
"name": "@threshing-atlas/core",
"version": "0.1.0",
"description": "Coding rules, statistics and (soon) the route graph and solver for the Dragonkind threshing. Runs in Node and the browser, no dependencies.",
"main": "src/engine.js",
"files": ["src"],
"license": "MIT"
}
-8
View File
@@ -1,8 +0,0 @@
{
"name": "@threshing-atlas/ingest",
"version": "0.1.0",
"description": "Importers that turn DragonKind bot data and community sources into the shared graph format (no game prose).",
"main": "src/bot.js",
"bin": { "threshing-import-bot": "bin/import-bot.js" },
"license": "MIT"
}
+15
View File
@@ -0,0 +1,15 @@
# planner
The route planner: the best route to every dragon, and what goes back to Ken's bot.
| File | What it does |
|---|---|
| `build-policy.js` | Solves the decision model (`core/solver.js`) for every colour and both goals ("this attempt" and "eventually"), with 200 posterior draws and a cross-fit check. Writes `data/public/policy.json`, which has a content id so each bot run can record which policy it followed. About 4 minutes. |
| `policy-report.js` | Writes the readable version, `analysis/reports/POLICY.md`. |
Run both with `npm run policy`. The site's valley map and The Odds read `policy.json`; the bot downloads it with
`solver.js` and `graph.js` (see `docs/BOT_INTEGRATION.md`).
**The loop with the bot.** The planner's routes go out to the bot; the bot's runs come back through
`collection/bot/`. Experiments the bot should run to fill gaps the community form cannot (for example the
answers at each bonding encounter) belong here; `experiments.json` is on the roadmap.
@@ -1,9 +1,9 @@
// Build data/public/policy.json: the best decisions for each dragon colour, under both objectives.
// node scripts/build-policy.js [--draws 200]
// node planner/build-policy.js [--draws 200]
const fs = require("fs");
const path = require("path");
const E = require("../packages/core/src/engine.js");
const S = require("../packages/core/src/solver.js");
const E = require("../core/engine.js");
const S = require("../core/solver.js");
const root = path.resolve(__dirname, "..");
const arg = (k, d) => { const i = process.argv.indexOf("--" + k); return i > 0 ? process.argv[i + 1] : d; };
const draws = +arg("draws", 200);
@@ -1,5 +1,5 @@
// Write docs/POLICY.md: a readable summary of data/public/policy.json.
// node scripts/policy-report.js
// Write analysis/reports/POLICY.md: a readable summary of data/public/policy.json.
// node planner/policy-report.js
const fs = require("fs");
const path = require("path");
const root = path.resolve(__dirname, "..");
@@ -38,7 +38,7 @@ function walk(n, out, indent = "") {
const s = P.samples.all, lb = P.samples.lowBias;
const L = [];
L.push("# Best routes by dragon colour", "");
L.push(`Generated from ${s.data.runs} community runs by \`scripts/build-policy.js\`. Machine-readable version: \`data/public/policy.json\`.`, "");
L.push(`Generated from ${s.data.runs} community runs by \`planner/build-policy.js\`. Machine-readable version: \`data/public/policy.json\`.`, "");
L.push("**How to read this.** \"This attempt\" maximises the chance of bonding the colour on one run. \"Eventually\" assumes you will die on purpose and try again whenever the run goes wrong, so it avoids any choice that can end the account on a different dragon. Percentages are posterior means with 95% credible intervals. \"Held-out\" is the same policy chosen on half of the runs and scored on the other half: when it is much lower than the in-sample number, the route's advantage is mostly noise.", "");
L.push("**Caveats.** The form records choices only at the level of company, direction, first dragon, cadet encounter, fork, tree or valley, and which dragon you met. Choices inside each encounter are not recorded, so the bond chance at an encounter is the community's average with whatever answers people used. Riders chasing a rare dragon report sightings more often than riders who log every run; the \"low-bias\" columns use only bulk logs and non-chasers.", "");
L.push("| Colour | This attempt | Low-bias sample | Held-out | Eventually: chance the account ends on this colour | Attempts needed (about) |");
@@ -58,5 +58,5 @@ for (const X of Object.keys(s.targets)) {
L.push("", `## ${X}: ${title}`, "", ...out);
}
}
fs.writeFileSync(path.join(root, "docs/POLICY.md"), L.join("\n") + "\n");
console.log("wrote docs/POLICY.md", L.length, "lines");
fs.writeFileSync(path.join(root, "analysis/reports/POLICY.md"), L.join("\n") + "\n");
console.log("wrote analysis/reports/POLICY.md", L.length, "lines");
+7 -6
View File
@@ -1,5 +1,5 @@
// Assemble the static Atlas site into dist/ (no dependencies).
// node scripts/build-site.js [outDir]
// node site/build.js [outDir]
const fs = require("fs");
const path = require("path");
const root = path.resolve(__dirname, "..");
@@ -11,7 +11,7 @@ const copy = (from, to) => {
};
fs.rmSync(out, { recursive: true, force: true });
for (const f of fs.readdirSync(path.join(root, "apps/atlas"))) copy(`apps/atlas/${f}`, f);
for (const f of fs.readdirSync(path.join(root, "site/atlas"))) copy(`site/atlas/${f}`, f);
// The page is written as an Artifact body (the artifact host adds the document skeleton). For GitHub Pages and
// local serving, give it its own doctype, charset and viewport.
{
@@ -20,12 +20,13 @@ for (const f of fs.readdirSync(path.join(root, "apps/atlas"))) copy(`apps/atlas/
if (!/^\s*<!doctype/i.test(html))
fs.writeFileSync(p, `<!doctype html>\n<meta charset="utf-8">\n<meta name="viewport" content="width=device-width, initial-scale=1">\n${html}`);
}
copy("packages/core/src/engine.js", "engine.js");
copy("packages/core/src/graph.js", "graph.js");
copy("packages/core/src/solver.js", "solver.js");
copy("core/engine.js", "engine.js");
copy("core/graph.js", "graph.js");
copy("core/solver.js", "solver.js");
for (const f of ["form_responses.json", "register.json", "graph.json", "policy.json"]) copy(`data/public/${f}`, `data/${f}`);
// Source files shown in the page's "view the code" panel
for (const f of ["analyze.py", "clean.py", "stats_core.py"]) copy(`research/python/${f}`, `python/${f}`);
copy("core/python/clean.py", "python/clean.py");
for (const f of ["analyze.py", "stats_core.py"]) copy(`analysis/stats/${f}`, `python/${f}`);
copy("README.md", "README.md");
fs.writeFileSync(path.join(out, ".nojekyll"), "");
console.log(`Site built in ${path.relative(root, out) || "."}`);
@@ -1,6 +1,6 @@
"""check_colours.py: are the dragon colours (valley.js INK, index.html --dr-*) still distinguishable?
python research/python/check_colours.py
python site/qa/check_colours.py
"""
"""Pairwise CIEDE2000 between dragon colours, normal vision and simulated colour-vision deficiencies (Machado 2009, severity 1),
plus contrast against the backgrounds they sit on."""
+3 -1
View File
@@ -1,7 +1,9 @@
// QA crawl: every tab, text dump, links, buttons, console errors. node crawl.js <outdir>
const { chromium } = require("playwright");
const fs = require("fs");
const OUT = process.argv[2];
// screenshots and logs go to the folder given, or a temp folder (never a folder named "undefined" in the repo)
const OUT = process.argv[2] || require("path").join(require("os").tmpdir(), "atlas-qa");
require("fs").mkdirSync(OUT, { recursive: true });
const BASE = "http://localhost:8765/";
(async () => {
const b = await chromium.launch();
@@ -1,6 +1,8 @@
// Valley map QA: for idle and every dragon/route, find text or labels that overlap, fall outside the map, or are too faint.
const { chromium } = require("playwright");
const OUT = process.argv[2];
// screenshots and logs go to the folder given, or a temp folder (never a folder named "undefined" in the repo)
const OUT = process.argv[2] || require("path").join(require("os").tmpdir(), "atlas-qa");
require("fs").mkdirSync(OUT, { recursive: true });
(async () => {
const b = await chromium.launch();
const p = await b.newPage({ viewport: { width: 1440, height: 1000 } });
-1
View File
@@ -1 +0,0 @@
needed
@@ -2,10 +2,10 @@
const test = require("node:test");
const assert = require("node:assert");
const path = require("path");
const { importBot, assertPublicSafe } = require("../packages/ingest/src/bot");
const G = require("../packages/core/src/graph");
const { importBot, assertPublicSafe } = require("../../collection/bot/bot");
const G = require("../../core/graph");
const fx = path.join(__dirname, "fixtures/bot");
const fx = path.join(__dirname, "../fixtures/bot");
const { scenes, trajectories } = importBot({ dataDir: path.join(fx, "data"), logsDir: path.join(fx, "logs"), salt: "test-salt" });
test("one trajectory per attempt, duplicates and repeated burns collapsed", () => {
@@ -1,7 +1,7 @@
// node --test tests/solver.test.js
// node --test tests/core/solver.test.js
const test = require("node:test");
const assert = require("node:assert");
const S = require("../packages/core/src/solver");
const S = require("../../core/solver");
// A synthetic world: the meadow shows Blue 30% of the time, the shadow appears 2% of the time,
// facing the valley Red always bonds it, the Greens burn half the time and bond Brown (rescue) a quarter of the time.
@@ -1,11 +1,11 @@
"""Recompute route rankings in Python (crude Fisher and stratified Mantel-Haenszel) and compare with engine.js."""
import json, subprocess, sys
sys.path.insert(0, "research/python")
sys.path.insert(0, "analysis/stats")
import pandas as pd
from stats_core import odds_ratio, mantel_haenszel, bh
d = pd.DataFrame(json.load(open("tests/out/runs_js.json")))
js = json.loads(subprocess.check_output(["node", "-e", """
const E=require('./packages/core/src/engine.js'),fs=require('fs');const raw=JSON.parse(fs.readFileSync('data/public/form_responses.json'));
const E=require('./core/engine.js'),fs=require('fs');const raw=JSON.parse(fs.readFileSync('data/public/form_responses.json'));
const {runs}=E.codeRows(raw.rows,raw.headers);const o={};for(const t of ['see_Blue','bond_Blue','see_shadow','bond_Green']){o[t]=E.rankRoutes(runs,t,10)}
console.log(JSON.stringify(o));"""]))
MISSING = {"Unknown", "Unsure", "Unsure/unknown", "Multiple", "(not recorded)", "(not asked)", "(not asked/blank)", "Unknown/lost count", "Blank/not asked"}
@@ -1,4 +1,4 @@
"""Check that the JS engine (packages/core/src/engine.js) and the Python reference (python/clean.py) code every run identically."""
"""Check that the JS engine (core/engine.js) and the Python reference (python/clean.py) code every run identically."""
import json, sys
import pandas as pd
js = pd.DataFrame(json.load(open(sys.argv[1] if len(sys.argv) > 1 else "tests/out/runs_js.json")))
@@ -1,8 +1,8 @@
// node tests/run_engine.js data/public/form_responses.json out_dir
// node tests/parity/run_engine.js data/public/form_responses.json out_dir
// Runs the browser engine in Node and writes coded runs + results, for comparison with Python.
const fs = require("fs");
const path = require("path");
const E = require("../packages/core/src/engine.js");
const E = require("../../core/engine.js");
const [, , src = "data/public/form_responses.json", outDir = "tests/out"] = process.argv;
const raw = JSON.parse(fs.readFileSync(src, "utf8"));
const { runs, missing } = E.codeRows(raw.rows, raw.headers);
Binary file not shown.

Before

Width:  |  Height:  |  Size: 738 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.6 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.1 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 837 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.3 MiB

-606
View File
File diff suppressed because one or more lines are too long
Binary file not shown.

Before

Width:  |  Height:  |  Size: 605 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 3.5 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 697 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 542 KiB

-64
View File
@@ -1,64 +0,0 @@
WHAT 5,651 RUNS SAY
Three things worth knowing
Lucky runs get shared
Riders hunting a rare dragon logged a Blue in 35% of their meadow visits. Riders who logged every run saw it in 9%. The game can't know what you're chasing, so the difference is who chose to write in. The Odds page leans on the riders who logged every run.
Earlier choices barely change who waits
Which dragon waits in the meadow shows no link to anything you chose before. The shadow at the tree depends on them only weakly. The model lets every roll depend on your history, and measured how much it does. Choices the form doesn't record (what you answered inside an encounter) might still matter.
The right answer can still burn
Even with the community's best answers, bond chances at the encounter run from 23% (the two Greens) to 94% (the valley Red). One death with the "right" answer doesn't mean the answer was wrong.
WHERE THE NUMBERS COME FROM
Data
5,651 runs
Form 4,787 · Data Import 348 · BulkUpload3 237 · Manual Upload 196 · Historical import 83
Form dates 2026-10-04 → 2026-11-02
bundled snapshot (Threshing_Routes_Responses.xlsx, exported 2026-11-01)
LOAD A NEWER ROUTE EXPORT…
OPEN BY DESIGN
Share & reproduce
Everything on this page is recomputed in your browser from the raw form export. To update it, download the latest responses sheet (File → Download → Microsoft Excel) and load it with "Load a newer route export" above. Column order does not matter, because questions are matched by their wording.
Copy the outputs
COPY REDDIT POST (MARKDOWN)
COPY CODED RUNS (CSV)
COPY ALL RESULTS (JSON)
The Reddit post is written from the current numbers and filters.
Read the code
ENGINE.JS (ALL RULES AND STATISTICS)
APP.JS (THIS PAGE)
CLEAN.PY
ANALYZE.PY
STATS_CORE.PY
SOLVER.JS (BEST ROUTES)
EXPLORER.JS (THE VALLEY MAP)
REGISTER.JS
README
The Python files are an independent second implementation. The test scripts confirm that both produce identical coded runs and identical odds ratios, p-values and model estimates.
HOW EVERY NUMBER IS MADE
Methods
1. RUNS
2. RATES
3. COMPARISONS
4. REPORTING BIAS
5. LIMITS
6. BEST ROUTES
MISSING ANSWERS
1. From form answers to runs
Each row of the "Threshing Routes" form (and each row pasted in from players' own logs) is one threshing run. The page reads the raw export and codes every row with the same fixed rules, which are visible in engine.js:
Source. Rows with a real timestamp are form submissions. Rows labelled "Data Import", "Manual Upload" or "Historical import" were bulk-pasted from players' own run logs and are called bulk logs here.
Cadet encounter. Split into the scream choice (help, investigate and leave, or ignore) and its resolution (killed the bullies, squadmate's revenge, left them to the dragons, tourniquet, left the cadet, took their weapons). The fork in the road only exists after helping, so other branches are coded "(no fork)".
Encounter branch. A tree run with a Blue or Black recorded took the shadow. A tree run with Greens recorded descended slowly, and one with only a Red descended quickly. A valley run with an Orange, Brown or Blue recorded (or a Red marked "Changed my mind") passed the Red and reached the meadow.
Outcome. Bonded to the one dragon marked "Bonded!". Otherwise incinerated by the last dragon marked "Incinerated" in encounter order, or died before the hour. "Saved by another" on the Greens plus a bonded Orange or Brown counts as a rescue bond.
Flags. Rows are flagged, not deleted, when answers contradict each other: several directions or wings, Greens in the valley, two bonds, or a first sighting outside the documented pool. You can drop flagged rows with the checkbox on Deep Dive's route map.
-576
View File
@@ -1,576 +0,0 @@
INTERACTIVE
Route map
Pick an outcome. Click any box to keep only the runs that passed through it, and click more boxes to narrow further. Hover or tap a box or band for its numbers.
BOND A DRAGON
BOND RED
BOND ORANGE
BOND BROWN
BOND GREEN
BOND BLUE
BOND BLACK
BOND ANY DRAGON
RARE SIGHTINGS
MEET A BLUE (MEADOW)
GET THE TREE SHADOW
BAND WIDTH SHOWS
HOW MANY RUNS
CHANCE OF OUTCOME
Grey out steps with fewer runs than
Data source
All rows
Form submissions only
Bulk logs only (imports)
Players chasing
Any goal
Not chasing Blue/Black
Chasing Blue/Black
Drop rows with data-quality flags
Showing all runs. Click a box to follow only the runs through it.
SHOW TOP ROUTES FOR THIS OUTCOME
CLEAR SELECTION
COMPANY
DIRECTION
FIRST SIGHTING
GAME ROLL
CADET ENCOUNTER
FORK IN THE ROAD
AFTER AN HOUR
ENCOUNTER
GAME ROLL
OUTCOME
Alone10%
1 Squadmate16%
2 Squadmates7%
East11%
Center12%
West8%
Red12%
Orange10%
Brown8%
Green7%
Help · squadmate's re…8%
Help · killed bullies10%
Help · left to dragons9%
Ignore · tourniquet17%
Ignore · left cadet8%
Investigate · walked …28%
Ignore · took weapons22%
Help · died in combat0%
Forest10%
West7%
East9%
(no fork)16%
Climb the tree7%
Valley centre17%
Tree: Red below2%
Tree: two Greens16%
Tree: shadow (Blue→Bl…64%
Valley: bond the Red82%
Meadow: Orange11%
Meadow: Brown15%
Meadow: Orange/Brown2%
Meadow: Blue25%
Meadow: none recorded0%
Bonded Red100%
Bonded Green100%
Bonded Orange100%
Bonded Brown100%
Bonded Blue100%
Bonded Black100%
Incinerated0%
Died before the hour0%
ALL RUNS · BOND ANY DRAGON
11.1% 589/5310 runs that could reach it · 95% CI 10.3%–12.0%
Band width = number of runs
0%
82%
Grey = fewer than 10 runs
Gold headings = steps the game rolls, not you
STATISTICS · ALL SIX COLOURS
Every dragon: how often the attempt ends in a bond
A run counts here when the player reached that dragon's bonding encounter and the form shows Bonded or Incinerated. Many players die on purpose to reroll, which drags the raw rate down. "Wanted it" keeps only runs where the player said they were after that colour, or any dragon.
Bond rate once the encounter starts
0%
20%
40%
60%
80%
100%
Red (valley)
82.0% · 73/89
↳ players who wanted it
95.1% · 39/41
Red (tree, descend quickly)
1.6% · 32/2011
↳ players who wanted it
22.2% · 4/18
Green (two Greens)
12.2% · 109/896
↳ players who wanted it
38.0% · 38/100
Orange (meadow)
11.4% · 86/753
↳ players who wanted it
51.0% · 26/51
Brown (meadow)
14.8% · 77/520
↳ players who wanted it
53.5% · 23/43
Blue (meadow)
25.0% · 131/523
↳ players who wanted it
26.6% · 109/410
Blue (shadow)
0.0% · 0/4
↳ players who wanted it
0.0% · 0/1
Black (shadow)
69.8% · 30/43
↳ players who wanted it
68.4% · 26/38
The two Greens
969 tree runs that descended slowly to the two Greens. A wrong answer sometimes brings an Orange or Brown swooping in to bond you instead.
Bonded a Green 109 11.2% 9.4%–13.4%
Rescued, bonded the Orange 24 2.5% 1.7%–3.7%
Rescued, bonded the Brown 18 1.9% 1.2%–2.9%
Incinerated 786 81.1% 78.5%–83.5%
No outcome recorded 32 3.3% 2.3%–4.6%
First sighting: what the game rolls
Which dragon met the cadet first, by company and direction. Pools come from the community PDF, and cells outside them are counted as data-quality flags.
COMPANY · DIRECTION RED ORANGE BROWN GREEN RUNS
1 Squadmate · Center 39 · 56% 5 · 7% 25 · 36% 1 · 1% 70 +2 unsure
1 Squadmate · East 23 · 13% 92 · 50% 5 · 3% 64 · 35% 184 +6 unsure
1 Squadmate · West 79 · 47% 2 · 1% 73 · 44% 13 · 8% 167 +2 unsure
2 Squadmates · Center 21 · 43% 3 · 6% 22 · 45% 3 · 6% 49 +1 unsure
2 Squadmates · East 35 · 43% 36 · 44% 5 · 6% 6 · 7% 82 +4 unsure
2 Squadmates · West 13 · 12% 2 · 2% 57 · 53% 35 · 33% 107 +1 unsure
Alone · Center 1167 · 51% 30 · 1% 1049 · 46% 25 · 1% 2271 +60 unsure
Alone · East 483 · 50% 422 · 44% 35 · 4% 28 · 3% 968 +18 unsure
Alone · West 59 · 4% 10 · 1% 737 · 48% 736 · 48% 1542 +31 unsure
BLUE & BLACK
The rare dragons
Two separate rolls. The meadow Blue can join the Orange and Brown after you turn down the valley Red. The tree's shadow option leads to a Blue and then a Black.
Does the Blue show up in the meadow?
Share of meadow visits where a Blue was attempted, by who logged the run. The game cannot know which dragon a player is chasing, so gaps between these bars come from how people report, not from the game.
0%
10%
20%
30%
40%
All rows
29.1% · 539/1854
Bulk logs (imports)
9.1% · 30/328
Form: other/blank goal
29.9% · 117/391
Form: chasing Blue or Black
34.5% · 392/1135
First attempt
16.0% · 33/206
Later attempt
30.8% · 501/1626
Does the shadow show up at the tree?
Share of tree climbs where the shadow option led to a Blue and/or Black.
0%
5%
10%
All rows
1.5% · 47/3045
Bulk logs (imports)
0.0% · 0/424
Form: other/blank goal
1.7% · 11/641
Form: chasing Blue or Black
1.8% · 36/1980
First attempt
8.0% · 31/386
Later attempt
0.6% · 16/2628
What changes the Blue's appearance, once reporting bias is held fixed?
Adjusted odds ratios from one logistic model: route choices plus data source and chase goal. Bars that cross 1 mean no detectable effect.
0.1
0.25
0.5
1
2
4
10
Data source: Bulk log (import) (n=325)
0.21
Chasing: Other colour/any (n=127)
0.63
Direction: West (n=376)
1.07
Direction: East (n=368)
0.98
First sighting: Brown (n=670)
0.82
First sighting: Orange (n=181)
1.00
First sighting: Green (n=138)
0.58
Cadet encounter: Tourniquet (n=340)
0.96
Cadet encounter: Squadmate's revenge (n=210)
1.00
Cadet encounter: Left injured cadet (n=177)
0.97
Cadet encounter: Took their weapons (n=73)
1.20
Cadet encounter: Left them to it (n=66)
0.99
Cadet encounter: Left bullies to the dr… (n=63)
0.88
Company: 1 Squadmate (n=106)
0.95
Company: 2 Squadmates (n=42)
1.75
Reference answers: Data source = Form submission, Chasing = Blue or Black, Direction = Center, First sighting = Red, Cadet encounter = Killed the bullies yourself, Company = Alone. n = 1776, events = 502. Gold = p < 0.05 (not corrected for the number of terms).
What changes the shadow's appearance?
Same approach for the shadow. With so few shadows, intervals are wide.
0.1
0.25
0.5
1
2
4
10
Data source: Bulk log (import) (n=423)
0.19
Chasing: Other colour/any (n=395)
0.49
Attempt: First attempt (n=385)
11.7
Direction: Center (n=1065)
0.48
Direction: East (n=691)
0.26
Reference answers: Data source = Form submission, Chasing = Blue or Black, Attempt = Later attempt, Direction = West. n = 2996, events = 46. Gold = p < 0.05 (not corrected for the number of terms).
Day by day (form submissions)
Bulk imports carry no dates, so they are left out here. Dots are sized by the number of runs, and whiskers show 95% intervals.
0%
34%
68%
Blue in meadow
0%
16%
32%
Shadow at tree
10-04
10-08
10-12
10-16
10-20
10-24
10-28
11-01
11-02
Bonding once you reach them
0%
20%
40%
60%
80%
100%
Blue (shadow)
0.0% · 0/4
↳ players who wanted it
0.0% · 0/1
Black (shadow)
69.8% · 30/43
↳ players who wanted it
68.4% · 26/38
Blue (meadow)
25.0% · 131/523
↳ players who wanted it
26.6% · 109/410
Step-by-step survival inside the encounter (from the community PDF)
As published in the community PDF (u/ClumsyBadger, u/Old_Regular3018). These come from hand-coded free-text answers. The underlying counts are not shown there, and many cells rest on fewer than 10 runs.
MEADOW BLUE, STANDARD ANSWERS SURVIVED STEP
Approach the Blue: “Absolutely…” 90%
Hold the Blue's glare and advance 80%
Do nothing (it seizes the other cadet) 82%
Hold glare and watch 84%
Straighten your spine 83%
Launch yourself right (boulder) 93%
“Bring down an empire” 81%
Move slowly forward, wait for an invitation 81%
All eight steps (product) 25%
The product, about 25%, agrees with the form's own Blue bond rate (25.0% (131/523; 95% CI 21.5%–28.9%)), which is a useful cross-check.
BLACK (AFTER THE SHADOW AND THE TREE BLUE) PDF OUTCOME
Turn to face the threat → hold its gaze 66% bonded · 33% incinerated
Turn to face the threat → lower your eyes → try to mount → meet its gaze 100% bonded (very few runs)
WHAT THE DATA SUPPORTS
Findings
These sentences are generated from the current data and filters, so they update when the data does. "Lead" means a pattern worth testing that is not yet established.
1
REPORTING BIAS
The meadow Blue appeared in 29.1% (539/1854; 95% CI 27.1%–31.2%) of logged meadow visits, but that figure is inflated. Players chasing a rare dragon report it in 34.5% of their visits. Bulk logs, which record every run, show 9.1% (30/328; 95% CI 6.5%–12.8%). The game cannot see what you are chasing, so that gap comes from which runs get reported.
A realistic expectation is nearer the bulk-log rate, about one meadow visit in 11, than the headline figure. Form players who were not chasing a Blue report 29.9%, which still carries some selective reporting.
2
BLUE
After adjusting for reporting bias, one recorded choice was linked to the Blue appearing: First sighting: Green (OR 0.58, 95% CI 0.35–0.99, p = 0.04).
Of 32 full routes with at least 10 meadow visits, 0 beat the others after adjustment and correction for multiple testing. The "precise pathways" shared on Reddit may still matter through choices the form does not record, such as the answers given inside each encounter.
3
BLUE
Reaching the Blue is not enough. Only 25.0% (131/523; 95% CI 21.5%–28.9%) of Blue attempts ended in a bond (26.6% when the player wanted a Blue). The community PDF's step-by-step survival rates multiply out to about 25% for the standard answers. Expect several Blue encounters per bond, and do not treat one death as proof that an answer is wrong.
4
BLACK
The tree shadow showed up in 1.5% (47/3045; 95% CI 1.2%–2.0%) of logged tree climbs. Form submissions: 1.8% (47/2621; 95% CI 1.4%–2.4%). Bulk logs: 0.0% (0/424; 95% CI 0.0%–0.9%). Those two cannot both be the true rate for the same game. The form figure is the one exposed to selective reporting, so the true rate is probably nearer the bulk logs, whose 95% upper bound is 0.9% per climb.
Once the Black appears it is the easiest dragon to bond: 69.8% (30/43; 95% CI 54.9%–81.4%).
5
LEAD
Shadow leads to test, not established: Direction: West had the shadow in 2.5% (31/1246; 95% CI 1.8%–3.5%) vs the rest (OR 3.00, q = 0.004); First sighting: Green had the shadow in 2.4% (16/664; 95% CI 1.5%–3.9%) vs the rest (OR 2.57, q = 0.02); Cadet encounter: Squadmate's revenge had the shadow in 2.2% (27/1233; 95% CI 1.5%–3.2%) vs the rest (OR 2.22, q = 0.03); Fork in the road: West had the shadow in 2.6% (22/854; 95% CI 1.7%–3.9%) vs the rest (OR 2.37, q = 0.02); Attempt: First attempt had the shadow in 8.0% (31/386; 95% CI 5.7%–11.2%) vs the rest (OR 14.3, q = <0.0001).
The first-attempt pattern survives adjustment for data source, chase goal and direction (OR 11.7, 95% CI 6.55–21.0). That matches a theory on Reddit that a fresh account rolls the shadow more often, but players who strike lucky on a first run may also be more likely to submit the form.
The cheapest test is a bulk log that records every tree climb, including the dull ones, with these choices noted.
6
COMMON
For the common colours, intent matters more than route. Among players who wanted that colour: Red (tree, descend quickly) 22% (4/18), Green (two Greens) 38% (38/100), Red (valley) 95% (39/41), Orange (meadow) 51% (26/51), Brown (meadow) 53% (23/43). The tree Red is the hardest bond in the game.
RANKED
Route table
A route here means every choice up to the hour. Routes are ranked by the lower end of their 95% interval, so a route needs both a high rate and enough runs to rank high. Each route is compared with every other route that could reach the same outcome. "Adjusted" holds data source and chase goal fixed (Mantel–Haenszel). q-values correct for testing many routes at once.
Outcome
Bond Blue
Bond Black
Meet a Blue (meadow)
Get the tree shadow
Bond Red
Bond Green
Bond Orange
Bond Brown
Bond any dragon
Minimum runs per route
32 routes tested · comparison group 29.1% (539/1854; 95% CI 27.1%–31.2%) · 0 clearly better after adjustment
# ROUTE (CHOICES UP TO THE HOUR) RUNS HITS RATE 95% CI CRUDE OR Q ADJ. OR ADJ. Q
1 Alone › Center › Red › Ignore the scream → Took their weapons › (no fork) › Valley centre 18 9 50.0% 29.0%–71.0%
2.46 0.70 2.53 1.00
2 Alone › Center › Red › Help → Killed the bullies yourself › Forest › Valley centre 236 78 33.1% 27.4%–39.3%
1.24 0.88 1.07 1.00
3 Alone › Center › Brown › Investigate & leave → Left them to it › (no fork) › Valley centre 17 8 47.1% 26.2%–69.0%
2.19 0.88 1.96 1.00
4 Alone › East › Red › Help → Killed the bullies yourself › East › Valley centre 59 22 37.3% 26.1%–50.0%
1.47 0.88 1.23 1.00
5 Alone › Center › Brown › Help → Squadmate's revenge › Forest › Valley centre 29 11 37.9% 22.7%–56.0%
1.50 1.00 1.42 1.00
6 Alone › East › Orange › Help → Killed the bullies yourself › East › Valley centre 37 13 35.1% 21.8%–51.2%
1.33 1.00 1.17 1.00
7 Alone › Center › Red › Ignore the scream → Tourniquet › (no fork) › Valley centre 128 37 28.9% 21.8%–37.3%
0.99 1.00 0.91 1.00
8 Alone › West › Brown › Ignore the scream → Tourniquet › (no fork) › Valley centre 14 6 42.9% 21.4%–67.4%
1.84 1.00 2.11 1.00
9 Alone › Center › Brown › Help → Killed the bullies yourself › Forest › Valley centre 185 50 27.0% 21.1%–33.8%
0.89 1.00 0.70 1.00
10 Alone › Center › Brown › Ignore the scream → Left injured cadet › (no fork) › Valley centre 113 31 27.4% 20.1%–36.3%
0.92 1.00 1.00 1.00
11 Alone › West › Brown › Help → Killed the bullies yourself › Forest › Valley centre 15 6 40.0% 19.8%–64.3%
1.63 1.00 1.65 1.00
12 Alone › Center › Red › Help → Squadmate's revenge › Forest › Valley centre 69 20 29.0% 19.6%–40.6%
1.00 1.00 0.87 1.00
13 Alone › East › Orange › Help → Squadmate's revenge › East › Valley centre 12 5 41.7% 19.3%–68.0%
1.75 1.00 1.63 1.00
14 Alone › East › Orange › Ignore the scream → Tourniquet › (no fork) › Valley centre 38 12 31.6% 19.1%–47.5%
1.13 1.00 0.95 1.00
15 Alone › East › Red › Ignore the scream → Tourniquet › (no fork) › Valley centre 51 15 29.4% 18.7%–43.0%
1.02 1.00 0.86 1.00
16 Alone › Center › Brown › Ignore the scream → Tourniquet › (no fork) › Valley centre 59 16 27.1% 17.4%–39.6%
0.90 1.00 1.01 1.00
17 Alone › West › Green › Help → Killed the bullies yourself › Forest › Valley centre 14 5 35.7% 16.3%–61.2%
1.36 1.00 1.20 1.00
18 Alone › Center › Red › Help → Left bullies to the dragons › Forest › Valley centre 18 6 33.3% 16.3%–56.3%
1.22 1.00 1.05 1.00
19 Alone › East › Red › Help → Killed the bullies yourself › Forest › Valley centre 19 6 31.6% 15.4%–54.0%
1.13 1.00 1.18 1.00
20 Alone › Center › Brown › Ignore the scream → Took their weapons › (no fork) › Valley centre 15 5 33.3% 15.2%–58.3%
1.22 1.00 1.17 1.00
21 Alone › Center › Red › Investigate & leave → Left them to it › (no fork) › Valley centre 22 6 27.3% 13.2%–48.2%
0.91 1.00 0.90 1.00
22 Alone › East › Orange › Help → Killed the bullies yourself › Forest › Valley centre 19 5 26.3% 11.8%–48.8%
0.87 1.00 0.79 1.00
23 Alone › West › Green › Ignore the scream → Tourniquet › (no fork) › Valley centre 14 4 28.6% 11.7%–54.6%
0.98 1.00 0.91 1.00
24 Alone › West › Brown › Help → Squadmate's revenge › Forest › Valley centre 10 3 30.0% 10.8%–60.3%
1.05 1.00 0.97 1.00
25 Alone › West › Green › Help → Killed the bullies yourself › East › Valley centre 18 4 22.2% 9.0%–45.2%
0.69 1.00 1.02 1.00
26 Alone › West › Brown › Help → Killed the bullies yourself › East › Valley centre 19 4 21.1% 8.5%–43.3%
0.65 1.00 1.43 1.00
27 Alone › Center › Red › Ignore the scream → Left injured cadet › (no fork) › Valley centre 21 4 19.0% 7.7%–40.0%
0.57 1.00 0.74 1.00
28 Alone › West › Green › Help → Killed the bullies yourself › West › Valley centre 37 5 13.5% 5.9%–28.0%
0.38 0.68 0.47 1.00
29 Alone › East › Orange › Ignore the scream → Took their weapons › (no fork) › Valley centre 11 2 18.2% 5.1%–47.7%
0.54 1.00 0.58 1.00
30 Alone › Center › Brown › Help → Killed the bullies yourself › East › Valley centre 12 2 16.7% 4.7%–44.8%
0.49 1.00 0.70 1.00
31 Alone › West › Brown › Help → Killed the bullies yourself › West › Valley centre 83 7 8.4% 4.1%–16.4%
0.21 1.7e-4 0.53 1.00
32 Alone › West › Green › Ignore the scream → Left injured cadet › (no fork) › Valley centre 11 1 9.1% 1.6%–37.7%
0.24 0.88 0.32 1.00
ONE CHOICE AT A TIME
Factor explorer
Each row compares runs with that answer against runs with any other answer to the same question, using Fisher's exact test. These comparisons are unadjusted, so check the adjusted models above before trusting a difference. Shaded rows have q < 0.05 after correcting for every comparison in this table.
Outcome
Blue appears in the meadow
Shadow option appears at the tree
Bond, given you approached the meadow Blue
Survive to the hour (tree/valley choice)
Run ends bonded to Red
Run ends bonded to Green
Run ends bonded to Orange
Run ends bonded to Brown
Run ends bonded to Blue
Run ends bonded to Black
Question
Company
Direction
First sighting
Scream choice
Cadet encounter
First combat move
Fork in the road
After an hour
Wing
Squad
Attempt
Data source
Chasing
Reaction to the first dragon
Second combat move
Planned to seek Malek
CADET ENCOUNTER HITS/RUNS RATE 95% CI OR VS REST P Q
All 539/1854 29.1% 27.1%–31.2%
Killed the bullies yourself 247/870 28.4% 25.5%–31.5% 0.95 (0.78–1.17) 0.64 0.86
Left bullies to the dragons 21/74 28.4% 19.4%–39.5% 0.97 (0.58–1.63) 1.00 1.00
Left injured cadet 43/181 23.8% 18.1%–30.5% 0.75 (0.52–1.07) 0.12 0.28
Left them to it 21/71 29.6% 20.2%–41.0% 1.03 (0.61–1.74) 0.89 0.99
Squadmate's revenge 69/220 31.4% 25.6%–37.8% 1.14 (0.84–1.55) 0.43 0.63
Took their weapons 27/76 35.5% 25.7%–46.7% 1.37 (0.85–2.22) 0.20 0.40
Tourniquet 105/351 29.9% 25.4%–34.9% 1.06 (0.82–1.37) 0.65 0.86
11 runs with no usable answer to this question ("unsure", blank, or not asked on that version of the form) are left out of these comparisons rather than treated as a choice.
Odds ratios
0.1
0.25
0.5
1
2
4
10
Killed the bullies yourself (n=870)
0.95
Left bullies to the dragons (n=74)
0.97
Left injured cadet (n=181)
0.75
Left them to it (n=71)
1.03
Squadmate's revenge (n=220)
1.14
Took their weapons (n=76)
1.37
Tourniquet (n=351)
1.06
READ THIS BEFORE QUOTING A NUMBER
Data quality and bias
Who logged the runs
SOURCE RUNS BLUE IN MEADOW SHADOW AT TREE
Form 4787 33.4% 509/1526 1.8% 47/2621
Data Import 348 8.4% 8/95 0.0% 0/231
BulkUpload3 237 4.6% 6/130 0.0% 0/58
Manual Upload 196 22.6% 12/53 0.0% 0/122
Historical import 83 8.0% 4/50 0.0% 0/13
Bulk logs come from players who recorded every run, so they are the least selective rows. Their rare-dragon rates are several times lower than the form's, which is the strongest sign of reporting bias in this data.
Rows flagged for contradictions
454 of 5651 rows (8.0%). Flagged rows stay in unless you drop them above.
first sighting outside documented pool 160
multiple wings 108
direction taken from the older sighting column 92
direction missing 62
more than one dragon marked Incinerated 29
Green recorded on a Valley run (no Greens in the valley) 21
tree run with Orange/Brown but no Green recorded 15
more than one dragon marked Bonded 5
How runs ended
Incinerated Red 2005 35.5%
Incinerated Green 791 14.0%
Incinerated Orange 661 11.7%
Incinerated Brown 448 7.9%
No outcome recorded 431 7.6%
Incinerated Blue 395 7.0%
Died before the hour 308 5.5%
Bonded Blue 131 2.3%
Bonded Orange 113 2.0%
Bonded Green 113 2.0%
Bonded Red 110 1.9%
Bonded Brown 97 1.7%
Bonded Black 30 0.5%
Incinerated Black 13 0.2%
Ambiguous 5 0.09%
"No outcome recorded" rows reached the hour but the form shows no Bonded or Incinerated. They count toward reaching a branch but not toward bond rates.
Independence
The form has no player ID. One player's 50 runs count as 50 independent runs, so intervals are somewhat narrower than they should be. 15% of rows arrived as bulk imports, each probably one player's or one group's log.
What the form does not ask
The redesigned form asks how you reacted to the first dragon (answered in 1475 rows; most riders answer "Unsure") and how the bully fight went (3246 rows). It still does not record most choices inside the bonding encounters. Theories that rely on those choices cannot be confirmed or refuted here.
File diff suppressed because it is too large Load Diff
-66
View File
@@ -1,66 +0,0 @@
22%
become a rider
594 of 2,714 cadets bonded a dragon. 89% made it to the tree-or-valley choice alive.
2,120
riders sent to Malek
78% of cadets were burned or killed. Deliberate deaths (2,501 runs where the rider chose to die and try again) are not counted.
94%
of runs end bonded to a Red, on the best route
94% per attempt (95% 87.6%–97.6%), from riders who logged every run. At the valley Red, 72 of 77 genuine attempts bonded.
SEE THE ROUTE →
1 in 5
runs end bonded to a Green, on the best route
22% per attempt (95% 18.2%–25.7%), from riders who logged every run. At the two Greens, 107 of 464 genuine attempts bonded.
SEE THE ROUTE →
1 in 7
runs end bonded to an Orange, on the best route
15% per attempt (95% 11.9%–18.6%), from riders who logged every run. At the meadow Orange, 81 of 207 genuine attempts bonded.
SEE THE ROUTE →
1 in 11
runs end bonded to a Brown, on the best route
9.1% per attempt (95% 7.3%–11.0%), from riders who logged every run. At the meadow Brown, 74 of 272 genuine attempts bonded.
SEE THE ROUTE →
1 in 26
runs end bonded to a Blue, on the best route
3.8% per attempt (95% 2.5%–5.4%), from riders who logged every run; all form posts suggest 7.3%. At the meadow Blue, 129 of 518 genuine attempts bonded.
SEE THE ROUTE →
1 in 45
runs end bonded to a Black, on the best route
2.2% per attempt (95% 0.8%–4.3%), from riders who logged every run. At the Black after the shadow, 30 of 43 genuine attempts bonded.
SEE THE ROUTE →
SEVEN STEPS, TWO OF THEM PURE LUCK
How a run unfolds
I
Choose company
Go in alone, or with one or two squadmates.
Alone, 96% reached the hour. With two squadmates, 74% did.
II
Pick a direction
East toward the roar, west toward the scent, or the forest centre.
Direction decides which first dragon can find you.
III
The first dragon
A dragon finds you. Which colour is the game's roll, not yours.
Keep your eyes down. This one is about surviving, not bonding.
IV
The scream
Help the squadmate fighting two cadets, walk away, or ignore it and find a wounded cadet.
162 logged runs died in the fight.
V
Fork in the road
After the fight: west, deeper into the forest, or east.
Only the bully path has a fork.
VI
Tree or valley
Climb the tallest tree, or walk to the valley centre.
The tree holds a Red, two Greens and the rare shadow. The valley holds a Red, then the meadow's Orange, Brown and sometimes a Blue.
VII
The bonding
Who is waiting is partly luck. Then every answer is a test.
At the two Greens, 4% were rescued by an Orange or Brown.
In the books, first-years who survive to Threshing walk into a valley of unbonded dragons, and the dragons do the choosing. The Dragonkind site recreates that: every choice can end with you incinerated, and the author has said there is a 10% chance of that no matter what you pick.
WALK THE VALLEY MAP
DIG INTO THE STATISTICS
MEET THE BONDED DRAGONS
-27
View File
@@ -1,27 +0,0 @@
West, toward the scent
Forest centre
East, toward the roar
The scream
The bully fight
The wounded cadet
An hour's walk
The tallest tree
Valley centre
The meadow
The shadow
ENTER THE VALLEY
PICK A DRAGON, OR A KEY MOMENT
UNOFFICIAL ATLAS · BUILT FROM 5,651 LOGGED RUNS
The valley
Every dragon is drawn where riders meet it. The fainter it is, the fewer riders reach it. Choose one to see its best route and the choices on it. Or click a key moment on the map, including the entrance, to see the choices you make there and your best chance at each dragon from it.
RED
ORANGE
BROWN
GREEN
BLUE
BLACK
Show where riders actually went
Prefer the numbers? The Odds has the statistics behind the map.