Threshing Atlas: monorepo, route graph, bot importer, daily source checks
- apps/atlas: the web page (valley map, route flow, Dragon Register) - packages/core: coding rules and statistics (engine.js) and the shared route graph (graph.js): scene graph, decision-variable graph, bot-to-form crosswalk, Beta credible intervals - packages/ingest: DragonKind bot importer to anonymised trajectories (no prose, no account identifiers); data/bot holds Ken's sample - research/python: independent reference implementation + update_sources.py - tests: JS/Python parity checks and graph/importer unit tests - CI, GitHub Pages deploy, daily check of the community sheets and route PDF - Email addresses in free-text form answers are redacted; the raw spreadsheet is not committed - MIT code licence, CC BY 4.0 data licence Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
0052d1a5fd
commit
c3d067b88c
@@ -0,0 +1,152 @@
|
||||
# Bot data protocol
|
||||
|
||||
How the DragonKind bot should run and log its threshing attempts so that the Threshing Atlas can
|
||||
estimate route probabilities, including history effects, and find the best route to each dragon colour.
|
||||
|
||||
Goal: bond a Blue and bond a Black, and know the best route for every colour.
|
||||
|
||||
## 1. Why the bot matters
|
||||
|
||||
Community data are observational. People copy routes from guides, so most choices are correlated
|
||||
with each other and with what people chose to report. That correlation is copied behaviour, not game
|
||||
mechanics. Only the bot can choose at random, and random assignment is what tells us that an earlier
|
||||
choice *causes* a later dragon to appear. The bot is the experiment. The community data are the prior.
|
||||
|
||||
Two facts shape everything below:
|
||||
|
||||
- **An attempt is the scarce resource.** Each one costs a cooldown, and a bond is permanent for that
|
||||
account. A run must never risk bonding the wrong colour.
|
||||
- **Seeing a dragon is usually not the same as bonding it.** If the colour is visible before the step
|
||||
where the player commits, the bot can record what appeared and then burn deliberately. Spawn rates
|
||||
for every colour can then be learned without using up an account. Only the final "does this choice
|
||||
actually bond?" step needs real bonds, and that step is close to deterministic.
|
||||
|
||||
## 2. What to log
|
||||
|
||||
`src/runhistory.js` already records one structured record per attempt. That is the right shape. These
|
||||
additions turn it into data the solver can use. The most important ones come first.
|
||||
|
||||
| # | Field | Why |
|
||||
|---|-------|-----|
|
||||
| 1 | `steps[].offered`: every option on screen, as `[{text, norm, hash}]`, plus the index that was picked | Shows "every option at every point", and detects options that only appear after certain earlier choices (that is itself history dependence) |
|
||||
| 2 | `steps[].observed`: what the scene showed, e.g. `{dragons: ["red"], shadow: false, cadetWounded: true}` | Chance outcomes are the core data. Today a burned run has `dragonColor: null`, and the tree check does not say whether the shadow was there |
|
||||
| 3 | `steps[].propensity`: the probability with which the bot picked this option (1.0 for a fixed rule, 0.5 for a coin flip) | Lets runs from different policies be pooled without bias (inverse-propensity weighting) |
|
||||
| 4 | `policyId` / `experimentId` / `arm` per run, plus `botCommit` (git short hash) | Says which policy or experiment produced the run. The learner changes over time, so this is not optional |
|
||||
| 5 | `burn: {reason, stage, nodeId}` for intentional burns | A deliberate burn cuts the run short on purpose (censoring). The model must know where and why |
|
||||
| 6 | `accountKey`: a salted hash of the account id, plus `accountAttempt` (that account's attempt number) | Separates account effects from route effects. Never log the account name or email |
|
||||
| 7 | `startedAt` in UTC, plus the in-game hour if the scene shows one | Time-of-day and patch effects |
|
||||
| 8 | `manual: true` on any step a human picked, and `decisionMode` on every step (not only the run) | `ai_pilot` and manual picks have no known propensity, so they are analysed separately |
|
||||
| 9 | `ABORTED` runs kept, with `abortReason` | Dropping failures biases rates |
|
||||
| 10 | `sceneFp` (the existing sha1 node id) and a short `label` | The public data carry fingerprints and labels only, never the game's prose |
|
||||
|
||||
A step would then look like this:
|
||||
|
||||
```json
|
||||
{
|
||||
"nodeId": "bc4ee4754b9c", "stage": "FIRST_SIGHTING", "label": "first sighting, forest centre",
|
||||
"observed": { "dragons": ["brown"] },
|
||||
"offered": [
|
||||
{ "norm": "lower your eyes in respect", "hash": "9a1c…" },
|
||||
{ "norm": "meet its gaze", "hash": "41fe…" }
|
||||
],
|
||||
"picked": 0, "propensity": 0.5, "decisionMode": "explore", "rule": "EXP-03:arm-A",
|
||||
"outcome": "SCENE", "toKey": "84780dab01e3", "at": "2026-11-01T21:03:11Z"
|
||||
}
|
||||
```
|
||||
|
||||
Parsing `observed` from the prose can start as simple keyword rules (colour words, "shadow", "wounded")
|
||||
kept in one table. The atlas will flag scenes where the rules return nothing.
|
||||
|
||||
## 3. How to run the accounts
|
||||
|
||||
### Two kinds of account
|
||||
|
||||
- **Goal accounts** (the ones meant to bond Blue or Black) run the current best policy for their target
|
||||
colour from `policy.json`, with the existing Account Lock Guard and intentional-burn protection.
|
||||
They still log everything. Their runs are data too, with propensity 1.0.
|
||||
- **Research accounts** run experiments. They also never accept a non-target bond. They observe and
|
||||
then burn. If one reaches a genuine Blue or Black bonding choice, that is a real result: let it bond
|
||||
and log it.
|
||||
|
||||
### Freeze the policy during an experiment
|
||||
|
||||
`auto_learn` rewrites its own rules from its own results. While an experiment is running, that mixes up
|
||||
the effect of the experiment with the bot's learning, and makes the data non-stationary. For research
|
||||
runs, use `explore` with a fixed experiment file, and record `policyId`. Update the policy between
|
||||
batches, not during them.
|
||||
|
||||
### Randomize one decision point against a fixed baseline
|
||||
|
||||
- The **baseline** is the community guide route (the PDF route) for the target colour.
|
||||
- In an experiment, exactly one decision point (or two, in a 2×2 design) is chosen by a coin flip with
|
||||
a known probability. Everything else follows the baseline.
|
||||
- Assign arms in **balanced blocks per account** (for example A, B, B, A, …), so every account
|
||||
contributes equally to both arms.
|
||||
- Use **pre-set run counts**. Do not stop because the results look good: stopping early on a good run
|
||||
inflates the effect.
|
||||
- Do not use `ai_pilot` for research runs. If it is used, the runs are tagged and kept out of the
|
||||
causal estimates.
|
||||
|
||||
### Phases
|
||||
|
||||
1. **Mapping (first week or two).** At every decision point, try each option a few times (about 5 to 10)
|
||||
so that every branch has at least been seen. This finds hidden options, dead ends and death scenes.
|
||||
It uses few runs, because the bot can burn as soon as a branch has been recorded.
|
||||
2. **Spawn rates (ongoing, nearly free).** Every run records which dragon appeared at the first
|
||||
sighting, the meadow and the valley, and whether the shadow was at the tree. This builds the
|
||||
colour-by-location table for all six colours as a by-product.
|
||||
3. **Targeted experiments.** The atlas produces an experiment list (`experiments.json`), ranked by how
|
||||
much each result could change the best route for each colour. That ranking is a value-of-information
|
||||
calculation. The bot runs the top items.
|
||||
|
||||
### Blue and Black need different strategies
|
||||
|
||||
**Blue (meadow, roughly 10% of meadow visits).** The question is whether anything earlier changes the
|
||||
Blue rate at the meadow: company, direction, how you met the first dragon, what happened with the
|
||||
cadets. Each research run goes to the meadow, records the colour, and burns unless it is Blue. Telling a
|
||||
10% rate from a 20% rate needs about **200 runs per arm**. That is achievable across a few accounts over
|
||||
a few weeks.
|
||||
|
||||
**Black (tree shadow, roughly 1%).** Too rare for full factorial experiments. Instead:
|
||||
|
||||
- Take the **shortest route to the tree check**, and burn immediately when there is no shadow. This is
|
||||
what the bot already does, and it maximizes tree visits per day.
|
||||
- At the same time, randomize **one** earlier factor 50/50 on every run, so each tree visit also counts
|
||||
towards a test. Run that one test until it reaches its planned count, then move on to the next factor.
|
||||
- Log the tree scene's fingerprint and options carefully even when there is no shadow. A change in the
|
||||
prose or options may be an early sign.
|
||||
- Telling 1% from 2% needs thousands of runs per arm. This is where pooling with community logs, and
|
||||
later Discord, will matter most.
|
||||
|
||||
### Other colours
|
||||
|
||||
Red, Brown, Orange and Green come mostly from the spawn-rate logging in phase 2. Green also needs the
|
||||
rescue branch (the wounded cadet) to be tried in both directions in phase 1.
|
||||
|
||||
## 4. Getting data to the atlas
|
||||
|
||||
- The atlas repo already has the exporter: `npm run import-bot -- --bot /path/to/DragonKind`. It reads
|
||||
`data/knowledge.json`, `data/runs.jsonl` and `logs/encounters.jsonl` from the bot folder and writes
|
||||
`data/bot/scenes.json` and `data/bot/trajectories-<date>.jsonl`, with:
|
||||
- prose removed (scene fingerprints, short labels, choice text and outcomes only). The importer
|
||||
refuses to write any field or string that looks like prose,
|
||||
- account ids replaced by salted hashes. Set `DK_ACCOUNT_SALT` in your environment; never commit it,
|
||||
- nothing read from `.browser-profile*/`, `data/accounts.json`, `data/integrations.json` or
|
||||
`data/state.json`.
|
||||
- Its scene fingerprints are computed exactly as the bot's `computeSceneFingerprint()` does, so they
|
||||
match the bot's node ids. If the bot's `normalize()` or fingerprint changes, the importer must change too.
|
||||
- Commit the result to `threshing-atlas` by pull request. CI rebuilds the graph and runs the tests.
|
||||
- In return, the atlas publishes `policy.json` (the best decision tree per colour, under both goals)
|
||||
and `experiments.json`. The bot reads them as a strategy source. The bot's own guards still have the
|
||||
final say.
|
||||
|
||||
## 5. Checklist for Ken
|
||||
|
||||
- [ ] Add `offered`, `picked`, `observed`, `propensity` and per-step `decisionMode` to `noteChoice()`.
|
||||
- [ ] Add `policyId`, `experimentId`, `arm`, `botCommit`, `accountKey` and `accountAttempt` to `beginRun()`.
|
||||
- [ ] Add `burn.reason` and `abortReason` to `endRun()`.
|
||||
- [ ] Write the keyword table that fills `observed` (dragon colours, shadow, wounded cadet).
|
||||
- [ ] Add an `explore` mode that reads an experiment file, assigns arms in balanced blocks per account, and freezes learning.
|
||||
- [ ] Add the export script, with the account salt in `.env`.
|
||||
- [ ] Split the accounts into goal accounts and research accounts.
|
||||
- [ ] Run phase 1 (mapping). Then keep phase 2 logging on for every run, goal accounts included.
|
||||
Reference in New Issue
Block a user