Threshing Atlas: monorepo, route graph, bot importer, daily source checks

- apps/atlas: the web page (valley map, route flow, Dragon Register)
- packages/core: coding rules and statistics (engine.js) and the shared route
  graph (graph.js): scene graph, decision-variable graph, bot-to-form crosswalk,
  Beta credible intervals
- packages/ingest: DragonKind bot importer to anonymised trajectories (no prose,
  no account identifiers); data/bot holds Ken's sample
- research/python: independent reference implementation + update_sources.py
- tests: JS/Python parity checks and graph/importer unit tests
- CI, GitHub Pages deploy, daily check of the community sheets and route PDF
- Email addresses in free-text form answers are redacted; the raw spreadsheet
  is not committed
- MIT code licence, CC BY 4.0 data licence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-01 21:54:54 +00:00
co-authored by Claude Opus 5.5
parent 0052d1a5fd
commit c3d067b88c
48 changed files with 8334 additions and 0 deletions
+152
View File
@@ -0,0 +1,152 @@
# Bot data protocol
How the DragonKind bot should run and log its threshing attempts so that the Threshing Atlas can
estimate route probabilities, including history effects, and find the best route to each dragon colour.
Goal: bond a Blue and bond a Black, and know the best route for every colour.
## 1. Why the bot matters
Community data are observational. People copy routes from guides, so most choices are correlated
with each other and with what people chose to report. That correlation is copied behaviour, not game
mechanics. Only the bot can choose at random, and random assignment is what tells us that an earlier
choice *causes* a later dragon to appear. The bot is the experiment. The community data are the prior.
Two facts shape everything below:
- **An attempt is the scarce resource.** Each one costs a cooldown, and a bond is permanent for that
account. A run must never risk bonding the wrong colour.
- **Seeing a dragon is usually not the same as bonding it.** If the colour is visible before the step
where the player commits, the bot can record what appeared and then burn deliberately. Spawn rates
for every colour can then be learned without using up an account. Only the final "does this choice
actually bond?" step needs real bonds, and that step is close to deterministic.
## 2. What to log
`src/runhistory.js` already records one structured record per attempt. That is the right shape. These
additions turn it into data the solver can use. The most important ones come first.
| # | Field | Why |
|---|-------|-----|
| 1 | `steps[].offered`: every option on screen, as `[{text, norm, hash}]`, plus the index that was picked | Shows "every option at every point", and detects options that only appear after certain earlier choices (that is itself history dependence) |
| 2 | `steps[].observed`: what the scene showed, e.g. `{dragons: ["red"], shadow: false, cadetWounded: true}` | Chance outcomes are the core data. Today a burned run has `dragonColor: null`, and the tree check does not say whether the shadow was there |
| 3 | `steps[].propensity`: the probability with which the bot picked this option (1.0 for a fixed rule, 0.5 for a coin flip) | Lets runs from different policies be pooled without bias (inverse-propensity weighting) |
| 4 | `policyId` / `experimentId` / `arm` per run, plus `botCommit` (git short hash) | Says which policy or experiment produced the run. The learner changes over time, so this is not optional |
| 5 | `burn: {reason, stage, nodeId}` for intentional burns | A deliberate burn cuts the run short on purpose (censoring). The model must know where and why |
| 6 | `accountKey`: a salted hash of the account id, plus `accountAttempt` (that account's attempt number) | Separates account effects from route effects. Never log the account name or email |
| 7 | `startedAt` in UTC, plus the in-game hour if the scene shows one | Time-of-day and patch effects |
| 8 | `manual: true` on any step a human picked, and `decisionMode` on every step (not only the run) | `ai_pilot` and manual picks have no known propensity, so they are analysed separately |
| 9 | `ABORTED` runs kept, with `abortReason` | Dropping failures biases rates |
| 10 | `sceneFp` (the existing sha1 node id) and a short `label` | The public data carry fingerprints and labels only, never the game's prose |
A step would then look like this:
```json
{
"nodeId": "bc4ee4754b9c", "stage": "FIRST_SIGHTING", "label": "first sighting, forest centre",
"observed": { "dragons": ["brown"] },
"offered": [
{ "norm": "lower your eyes in respect", "hash": "9a1c…" },
{ "norm": "meet its gaze", "hash": "41fe…" }
],
"picked": 0, "propensity": 0.5, "decisionMode": "explore", "rule": "EXP-03:arm-A",
"outcome": "SCENE", "toKey": "84780dab01e3", "at": "2026-11-01T21:03:11Z"
}
```
Parsing `observed` from the prose can start as simple keyword rules (colour words, "shadow", "wounded")
kept in one table. The atlas will flag scenes where the rules return nothing.
## 3. How to run the accounts
### Two kinds of account
- **Goal accounts** (the ones meant to bond Blue or Black) run the current best policy for their target
colour from `policy.json`, with the existing Account Lock Guard and intentional-burn protection.
They still log everything. Their runs are data too, with propensity 1.0.
- **Research accounts** run experiments. They also never accept a non-target bond. They observe and
then burn. If one reaches a genuine Blue or Black bonding choice, that is a real result: let it bond
and log it.
### Freeze the policy during an experiment
`auto_learn` rewrites its own rules from its own results. While an experiment is running, that mixes up
the effect of the experiment with the bot's learning, and makes the data non-stationary. For research
runs, use `explore` with a fixed experiment file, and record `policyId`. Update the policy between
batches, not during them.
### Randomize one decision point against a fixed baseline
- The **baseline** is the community guide route (the PDF route) for the target colour.
- In an experiment, exactly one decision point (or two, in a 2×2 design) is chosen by a coin flip with
a known probability. Everything else follows the baseline.
- Assign arms in **balanced blocks per account** (for example A, B, B, A, …), so every account
contributes equally to both arms.
- Use **pre-set run counts**. Do not stop because the results look good: stopping early on a good run
inflates the effect.
- Do not use `ai_pilot` for research runs. If it is used, the runs are tagged and kept out of the
causal estimates.
### Phases
1. **Mapping (first week or two).** At every decision point, try each option a few times (about 5 to 10)
so that every branch has at least been seen. This finds hidden options, dead ends and death scenes.
It uses few runs, because the bot can burn as soon as a branch has been recorded.
2. **Spawn rates (ongoing, nearly free).** Every run records which dragon appeared at the first
sighting, the meadow and the valley, and whether the shadow was at the tree. This builds the
colour-by-location table for all six colours as a by-product.
3. **Targeted experiments.** The atlas produces an experiment list (`experiments.json`), ranked by how
much each result could change the best route for each colour. That ranking is a value-of-information
calculation. The bot runs the top items.
### Blue and Black need different strategies
**Blue (meadow, roughly 10% of meadow visits).** The question is whether anything earlier changes the
Blue rate at the meadow: company, direction, how you met the first dragon, what happened with the
cadets. Each research run goes to the meadow, records the colour, and burns unless it is Blue. Telling a
10% rate from a 20% rate needs about **200 runs per arm**. That is achievable across a few accounts over
a few weeks.
**Black (tree shadow, roughly 1%).** Too rare for full factorial experiments. Instead:
- Take the **shortest route to the tree check**, and burn immediately when there is no shadow. This is
what the bot already does, and it maximizes tree visits per day.
- At the same time, randomize **one** earlier factor 50/50 on every run, so each tree visit also counts
towards a test. Run that one test until it reaches its planned count, then move on to the next factor.
- Log the tree scene's fingerprint and options carefully even when there is no shadow. A change in the
prose or options may be an early sign.
- Telling 1% from 2% needs thousands of runs per arm. This is where pooling with community logs, and
later Discord, will matter most.
### Other colours
Red, Brown, Orange and Green come mostly from the spawn-rate logging in phase 2. Green also needs the
rescue branch (the wounded cadet) to be tried in both directions in phase 1.
## 4. Getting data to the atlas
- The atlas repo already has the exporter: `npm run import-bot -- --bot /path/to/DragonKind`. It reads
`data/knowledge.json`, `data/runs.jsonl` and `logs/encounters.jsonl` from the bot folder and writes
`data/bot/scenes.json` and `data/bot/trajectories-<date>.jsonl`, with:
- prose removed (scene fingerprints, short labels, choice text and outcomes only). The importer
refuses to write any field or string that looks like prose,
- account ids replaced by salted hashes. Set `DK_ACCOUNT_SALT` in your environment; never commit it,
- nothing read from `.browser-profile*/`, `data/accounts.json`, `data/integrations.json` or
`data/state.json`.
- Its scene fingerprints are computed exactly as the bot's `computeSceneFingerprint()` does, so they
match the bot's node ids. If the bot's `normalize()` or fingerprint changes, the importer must change too.
- Commit the result to `threshing-atlas` by pull request. CI rebuilds the graph and runs the tests.
- In return, the atlas publishes `policy.json` (the best decision tree per colour, under both goals)
and `experiments.json`. The bot reads them as a strategy source. The bot's own guards still have the
final say.
## 5. Checklist for Ken
- [ ] Add `offered`, `picked`, `observed`, `propensity` and per-step `decisionMode` to `noteChoice()`.
- [ ] Add `policyId`, `experimentId`, `arm`, `botCommit`, `accountKey` and `accountAttempt` to `beginRun()`.
- [ ] Add `burn.reason` and `abortReason` to `endRun()`.
- [ ] Write the keyword table that fills `observed` (dragon colours, shadow, wounded cadet).
- [ ] Add an `explore` mode that reads an experiment file, assigns arms in balanced blocks per account, and freezes learning.
- [ ] Add the export script, with the account salt in `.env`.
- [ ] Split the accounts into goal accounts and research accounts.
- [ ] Run phase 1 (mapping). Then keep phase 2 logging on for every run, goal accounts included.
+19
View File
@@ -0,0 +1,19 @@
# I ran the numbers on 2,573 logged threshing runs. Here's what holds up (and what doesn't)
*Per the megathread rules, anything that names a specific route is behind spoiler tags.*
Data: the community Threshing Routes form/spreadsheet (thanks u/ClumsyBadger, u/Old_Regular3018, u/Smart_Trifle_2099 and everyone who logged runs). 1946 form submissions, 83 Historical import, 348 Data Import, 196 Manual Upload; form dates 2026-10-04 to 2026-10-19. Every rate below has a 95% confidence interval.
**TL;DR**
* **The meadow Blue shows up a lot less than the spreadsheet suggests.** Runs logged by Blue-chasers show it 50.1% of the time, but bulk logs (people who recorded *every* run) show 11.0% (21/191). The game can't know what you're chasing, so that gap is reporting bias. Expect something closer to the bulk-log number.
* **No recorded route choice changed the Blue's odds** once you account for that bias (direction, first dragon, how you handled the cadet, company). 0 of 25 full routes beat the rest after correcting for multiple testing. The "precise pathways" might still matter through choices the form doesn't record (like how you react to the first dragon), but the form can't confirm them.
* **Getting the Blue to show up is only half of it.** Only 23.1% (92/398) of Blue attempts bonded, which matches the PDF's step-by-step numbers (~25%). Dying with the "right" answer is normal.
* **The tree shadow (→ Black) is rarer than the spreadsheet suggests.** All rows: 2.2% (24/1089), but bulk logs show 0/364 (upper 95% bound 1.0%). Plan on roughly 1 in 100 climbs at best. Once you get the Black, it bonds 86.7% (13/15) of the time.
* **Leads worth testing (not proven):** >!Fork in the road = Forest: shadow in 3.7% (19/514) vs the rest; Attempt = First attempt: shadow in 7.0% (8/114) vs the rest!<. If you bulk-log your tree climbs *including the boring ones*, we can settle this.
**How to help:** log every run in the form, including the ones where nothing rare happened. Those dull runs are what make the rates trustworthy.
Interactive route map, full methods, and code to reproduce every number: [Threshing Route Atlas](LINK)
*Caveats: self-reported data, no player IDs (so runs from one person aren't independent), and the form doesn't record every in-game choice. The author has said there's a ~10% chaos roll, so nothing is guaranteed.*
+25
View File
@@ -0,0 +1,25 @@
# Roadmap
The aim is to move from summary statistics to path finding: for each dragon colour, the decision
policy that maximises the chance of bonding it, with honest uncertainty.
1. **Repo and packaging** (done): monorepo, CI, GitHub Pages, JS/Python parity tests.
2. **Graph format** (`packages/core`, first version done): typed scene, chance and terminal nodes; history-context keys;
per-source counts (bot, form, PDF, Discord); fingerprints and short labels only, no game prose.
Importers in `packages/ingest` for the bot's `knowledge.json`, `runs.jsonl` and `encounters.jsonl`
(done), the form (done, via the crosswalk in `graph.js`), and a PDF scene crosswalk (to do).
Daily source checks for the form, the register and the PDF (done).
3. **Context learning** (`research/`): tests of history dependence (G-tests, BH); variable-order
Markov / context tree with BIC; temporal-tiered DAG structure learning (BDeu/BIC); latent
"character" model vs specific triggers vs memoryless baseline, compared on held-out likelihood;
hierarchical Dirichlet back-off pooling.
4. **Solver**: backward induction on the augmented state (scene, history summary), i.e. an influence
diagram with perfect recall. Two objectives side by side per colour:
- P(target colour on this attempt);
- eventual success when deliberate deaths are free: maximise p_X / (p_X + p_other bond),
solved by Dinkelbach iteration.
Monte Carlo credible intervals, top-k routes, required steps and bottlenecks.
5. **Valley graph UI**: every option at every point, with P(survive) and P(bond) per colour, the
policy overlay for a chosen colour, and untested branches marked.
6. **Bot loop**: the atlas publishes `policy.json` and `experiments.json` (ranked by value of
information); the private DragonKind bot consumes them and returns anonymised runs.