- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group, the factor by which the group over-reports the rare outcome relative to complete logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count 1/rho; a roll the complete logs have (almost) never shown is learned from them only - planner/policy.yaml: groups, rolls and thresholds; estimates published in policy.json under model.reportingBias - fit_alpha: a roll seen only one way is pooled fully - docs: goal accounts follow the best route, research accounts use Thompson sampling (decisionMode atlas_thompson) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
193 lines
11 KiB
Markdown
193 lines
11 KiB
Markdown
# Connecting the DragonKind bot to the atlas
|
||
|
||
Data flows both ways:
|
||
|
||
```
|
||
community sources, reports ─────┐
|
||
├─► threshing-atlas ──► policy.json + solver.js ──► bot (advice per screen)
|
||
bot runs (anonymised) ──────────┘ ▲ │
|
||
└──────────── collection --source bot + PR ◄┘
|
||
```
|
||
|
||
## Without GitHub: swapping zips
|
||
|
||
When GitHub Pages is not available, the same files travel as zips:
|
||
|
||
- **Atlas → bot:** `python -m collection export-bot-kit` writes `exports/atlas-for-bot-<date>-<policy id>.zip`
|
||
(`policy.json`, `solver.js`, `graph.js`, a README with checksums). Unzip it into the bot's `vendor/threshing-atlas/`.
|
||
- **Bot → atlas:** zip the bot's `data/` and `logs/` folders (folder names kept; a zip of the whole repo works too;
|
||
screenshots are not needed) and send it. The importer reads `data/knowledge.json` and, for every account,
|
||
`data/accounts/<account>/runs.jsonl` and `logs/accounts/<account>/encounters.jsonl` (with one folder per
|
||
account, the top-level `data/runs.jsonl` and `logs/encounters.jsonl` mirror the first account and are skipped).
|
||
Run records that carry their own steps (the bot's schema 2) are used as they are, including what the bot saw
|
||
(shadow, meadow dragons), its decision mode and the policy id; older runs are rebuilt from the decision log.
|
||
Account names become salted hashes. Put it in the atlas's `inbox/` and run
|
||
`python -m collection refresh --bot inbox/<the zip>`: it imports the runs (anonymised: no prose, no account names),
|
||
rebuilds everything, and then `export-bot-kit` makes the new zip to send back.
|
||
|
||
## 1. From the atlas to the bot
|
||
|
||
### What is published
|
||
|
||
GitHub Pages serves these files (they update whenever new data is merged):
|
||
|
||
| File | What it is |
|
||
|---|---|
|
||
| `https://ekcschiano.github.io/threshing-atlas/data/policy.json` | The model: count tables, fitted shrinkage, allowed options, and for every colour the recommended decision tree under both objectives, with credible intervals. About 400 KB. Has an `id` (content hash). |
|
||
| `https://ekcschiano.github.io/threshing-atlas/solver.js` | The solver and the bot helper (`advisor`, `recommendOption`, `solverHistory`). No dependencies, CommonJS or browser. |
|
||
| `https://ekcschiano.github.io/threshing-atlas/graph.js` | `trajectoryVariables()`: turns the bot's step trail into the community form's variables. |
|
||
|
||
### Getting them into the bot
|
||
|
||
Keep a pinned local copy, so a bad publish can never change the bot mid-run. A small sync script in the bot
|
||
repo (for example `scripts/sync-atlas.js`):
|
||
|
||
```js
|
||
// node scripts/sync-atlas.js -> vendor/threshing-atlas/{solver.js,graph.js,policy.json}
|
||
const fs = require('node:fs');
|
||
const path = require('node:path');
|
||
const BASE = 'https://ekcschiano.github.io/threshing-atlas/';
|
||
const OUT = path.join(__dirname, '..', 'vendor', 'threshing-atlas');
|
||
(async () => {
|
||
fs.mkdirSync(OUT, { recursive: true });
|
||
for (const f of ['solver.js', 'graph.js', 'data/policy.json']) {
|
||
const res = await fetch(BASE + f);
|
||
if (!res.ok) throw new Error(`${f}: HTTP ${res.status}`);
|
||
const body = await res.text();
|
||
if (f.endsWith('.json')) JSON.parse(body); // refuse a broken file
|
||
fs.writeFileSync(path.join(OUT, path.basename(f)), body);
|
||
}
|
||
const { id } = JSON.parse(fs.readFileSync(path.join(OUT, 'policy.json'), 'utf8'));
|
||
console.log('atlas policy', id);
|
||
})();
|
||
```
|
||
|
||
Run it by hand, or once a day before the first attempt. Commit `vendor/threshing-atlas/` in the bot repo if you
|
||
want the exact policy in its history.
|
||
|
||
### What changed in November 2026
|
||
|
||
The calls above are unchanged, but `solver.js` is now a small advisor (about 250 lines) that reads the game itself
|
||
from `policy.json` (`samples.*.model.game`, from `core/spec/game.yaml`) instead of having it written into the code.
|
||
For the bot that means:
|
||
|
||
- Histories (`h`) use the game's step names: `company`, `leave`, `direction`, `fs`, `react`, `scream`, `clearing`,
|
||
`fight`, `dagger`, `fate` or `wounded`, `post`, `hour`, `shadow`, `shadowColour`, `descend`, `red`, `valley`,
|
||
`meadow`, `encounter`. `solverHistory()` already produces these; the old `cadet` and `left` keys are no longer read.
|
||
- New choices the bot will be asked about: leaving your squadmates (`leave`), how you react to the first dragon
|
||
(`react`), and, below the tree, jumping onto the Red or letting it pass (`red`). The text the bot sees for each
|
||
option is in `game.yaml` under `bot_text`, so new screens can be matched by editing that file.
|
||
- If a history skips a decision but already has a later step (an older record, say), the recommended option is
|
||
assumed for the skipped one.
|
||
- Re-sync `solver.js`, `graph.js` and `policy.json` together: the new `solver.js` needs the game inside the new
|
||
`policy.json`.
|
||
|
||
### Asking for advice on a screen
|
||
|
||
```js
|
||
const Solver = require('../vendor/threshing-atlas/solver.js');
|
||
const Graph = require('../vendor/threshing-atlas/graph.js');
|
||
const policy = require('../vendor/threshing-atlas/policy.json');
|
||
|
||
// once per attempt: target colour from settings, "eventually" because the bot burns on purpose
|
||
const advisor = Solver.advisor(policy.samples.all, 'Black', 'eventually');
|
||
|
||
// on every decision screen
|
||
function atlasAdvice(run, offeredChoiceTexts, seen) {
|
||
const { vars } = Graph.trajectoryVariables({ steps: run.steps, outcome: {} });
|
||
const h = Solver.solverHistory(vars, seen); // seen: { shadow: 'yes'|'no', shadowColour, meadow }
|
||
return Solver.recommendOption(advisor, h, offeredChoiceTexts);
|
||
}
|
||
```
|
||
|
||
`recommendOption` returns `null` when the next step is chance (nothing to decide), or
|
||
`{ index, decision, choice, margin, options, reason }`:
|
||
|
||
- `index` is the option to click, or `null` when the advice cannot be matched to this screen. Then use the bot's own rules.
|
||
- `decision === 'encounter'` comes without an index: `choice` is `"Attempt the bond"` or `"Die on purpose"`, and the
|
||
bot already knows which of its options do each.
|
||
- `margin` is the gap in P(target) between the best and second-best option. `options` has every option's value.
|
||
|
||
`seen` is what only the bot can observe: whether the shadow was at the treetop (`shadow: 'yes' | 'no'`), its colour
|
||
(`shadowColour`), and the colour waiting in the meadow (`meadow`), all from the scene tint.
|
||
|
||
The advisor re-solves the model on the bot's machine (a few milliseconds), so it gives advice from any history,
|
||
including runs that have already left the recommended route.
|
||
|
||
### Where it goes in the decision order
|
||
|
||
The atlas is advice. The bot's own safety rules stay in charge:
|
||
|
||
1. Account Lock Guard and the intentional-burn protection (never risk a non-target bond).
|
||
2. Atlas advice, when `index` is not null and the margin is clear (see ties below).
|
||
3. The bot's PDF strategy and learned rules.
|
||
|
||
Log every atlas pick with `rule: 'ATLAS_POLICY'`, `source: 'atlas'`, and the run-level `policyId: policy.id`
|
||
(see `docs/BOT_DATA_PROTOCOL.md`), so we can tell which policy produced which runs.
|
||
|
||
### Learning: goal accounts follow the best route, research accounts sample
|
||
|
||
The atlas learns from the bot's own runs: every zip of the bot's data that is imported adds its rolls (first dragon,
|
||
deaths, the shadow, the meadow dragon, bond results) to the route model, and the next `atlas-for-bot` zip carries
|
||
the updated model. Deliberate burns (Malek, "no shadow", "no Blue") never count as failed bond attempts.
|
||
|
||
How the bot should choose depends on the account's role:
|
||
|
||
- **Goal accounts** (`accountRole: "goal"`): the best route under the model, as before:
|
||
`Solver.advisor(policy.samples.all, target, 'eventually')`.
|
||
- **Research accounts** (`accountRole: "research"`): Thompson sampling. Make a new advisor for every attempt with a
|
||
new seed:
|
||
|
||
```js
|
||
const adv = Solver.advisor(policy.samples.all, target, 'eventually', { sampling: 'thompson', seed: attemptNumber });
|
||
```
|
||
|
||
Each attempt follows the best route of one plausible version of the odds, drawn from the model's uncertainty. An
|
||
option that might be the best one gets tried about as often as it is likely to be the best, so the attempts go
|
||
where the model is least sure; as the data firm up, sampling settles on the best route by itself.
|
||
`recommendOption()` then returns a `propensity` (the chance this option was picked, from 40 other draws; set
|
||
`draws` to change that). Log it on the step with `decisionMode: "atlas_thompson"`, so the runs can be re-weighted.
|
||
Making the advisor takes a few seconds per attempt (it solves the game once per draw).
|
||
|
||
The tie rule below still works for goal accounts; research accounts don't need it.
|
||
|
||
### Ties are free experiments
|
||
|
||
### Which numbers to trust
|
||
|
||
`policy.json` has two samples:
|
||
|
||
- `samples.all`: every run. Use it for the advisor (more data, decisions are about relative differences).
|
||
- `samples.lowBias`: bulk logs and riders not chasing a rare dragon. Use it for absolute expectations, such as how
|
||
many attempts a Blue should take. Rare sightings are over-reported in form posts, so `all` is optimistic for Blue.
|
||
|
||
Current headline, with the bot's runs and the reporting-bias correction (November 2026): Black about 0.03% per
|
||
attempt (90% range 0–0.15%: the bot and bulk logs have never seen the shadow in about 700 tree runs), Blue about
|
||
1.8–2.5% per attempt (40–55 attempts), depending on the sample.
|
||
|
||
Form submissions over-report rare sightings: riders report a meadow Blue about 4 times more readily than a meadow
|
||
without one. The model estimates that bias for each kind of source and corrects for it (`planner/policy.yaml`,
|
||
`reporting_bias`; the estimates are in `policy.json` under `model.reportingBias`).
|
||
|
||
## 2. From the bot to the atlas
|
||
|
||
Already in place: `DRAGONKIND_BOT_DIR=/path/to/DragonKind python -m collection update --source bot` in the atlas repo converts the bot's
|
||
`knowledge.json`, `runs.jsonl` and `logs/encounters.jsonl` into anonymised files under `data/bot/`, and refuses to
|
||
write prose or account names. Commit them by pull request; CI rebuilds the graph and checks everything.
|
||
|
||
What would make the bot's data far more useful (details in `docs/BOT_DATA_PROTOCOL.md`):
|
||
|
||
- record the options on screen, what was seen (tint, shadow), `policyId`, `propensity` and `decisionMode` on every step;
|
||
- split goal accounts (follow the advisor) from research accounts (run the tie experiments and the
|
||
experiment list);
|
||
- add `accountKey` (salted hash), and the reason for every deliberate burn.
|
||
|
||
## 3. Limits to keep in mind
|
||
|
||
- The community form does not record the answers inside each bonding encounter, so the bond chance at an encounter is
|
||
the community average. The bot's own scene-level data will refine that over time.
|
||
- The model learns how much history matters from the data. Today it finds that the meadow colour does not depend on
|
||
earlier choices at all, and the shadow only weakly. The bot's randomised runs are what can confirm or overturn that.
|
||
- The policy is only as good as the crosswalk between the bot's screen text and the form's variables. If the game
|
||
changes its wording, `OPTION_PATTERNS` in `solver.js` and the crosswalk in `collection/specs/bot.yaml` (written into `graph.js` by `python -m collection build`) need updating.
|