Route model: correct for reporting bias

- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group,
  the factor by which the group over-reports the rare outcome relative to complete
  logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count
  1/rho; a roll the complete logs have (almost) never shown is learned from them only
- planner/policy.yaml: groups, rolls and thresholds; estimates published in
  policy.json under model.reportingBias
- fit_alpha: a roll seen only one way is pooled fully
- docs: goal accounts follow the best route, research accounts use Thompson sampling
  (decisionMode atlas_thompson)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-10 02:29:09 +00:00
co-authored by Claude Opus 5.5
parent c0355616b6
commit 2271fe0976
8 changed files with 244 additions and 18 deletions
+33 -7
View File
@@ -125,12 +125,33 @@ The atlas is advice. The bot's own safety rules stay in charge:
Log every atlas pick with `rule: 'ATLAS_POLICY'`, `source: 'atlas'`, and the run-level `policyId: policy.id`
(see `docs/BOT_DATA_PROTOCOL.md`), so we can tell which policy produced which runs.
### Ties are free experiments
### Learning: goal accounts follow the best route, research accounts sample
Many choices barely matter by the data we have (for Black, the first direction differs by a tenth of a percentage
point). When `margin` is below the tie threshold (start with 0.002, i.e. 0.2 percentage points), pick at random
among the tied options and log `propensity` (1 / number of tied options). That costs nothing in expected success,
and it is exactly the randomised data the atlas needs to tell real history effects from guide-copying.
The atlas learns from the bot's own runs: every zip of the bot's data that is imported adds its rolls (first dragon,
deaths, the shadow, the meadow dragon, bond results) to the route model, and the next `atlas-for-bot` zip carries
the updated model. Deliberate burns (Malek, "no shadow", "no Blue") never count as failed bond attempts.
How the bot should choose depends on the account's role:
- **Goal accounts** (`accountRole: "goal"`): the best route under the model, as before:
`Solver.advisor(policy.samples.all, target, 'eventually')`.
- **Research accounts** (`accountRole: "research"`): Thompson sampling. Make a new advisor for every attempt with a
new seed:
```js
const adv = Solver.advisor(policy.samples.all, target, 'eventually', { sampling: 'thompson', seed: attemptNumber });
```
Each attempt follows the best route of one plausible version of the odds, drawn from the model's uncertainty. An
option that might be the best one gets tried about as often as it is likely to be the best, so the attempts go
where the model is least sure; as the data firm up, sampling settles on the best route by itself.
`recommendOption()` then returns a `propensity` (the chance this option was picked, from 40 other draws; set
`draws` to change that). Log it on the step with `decisionMode: "atlas_thompson"`, so the runs can be re-weighted.
Making the advisor takes a few seconds per attempt (it solves the game once per draw).
The tie rule below still works for goal accounts; research accounts don't need it.
### Ties are free experiments
### Which numbers to trust
@@ -140,8 +161,13 @@ and it is exactly the randomised data the atlas needs to tell real history effec
- `samples.lowBias`: bulk logs and riders not chasing a rare dragon. Use it for absolute expectations, such as how
many attempts a Blue should take. Rare sightings are over-reported in form posts, so `all` is optimistic for Blue.
Current headline (from `analysis/reports/POLICY.md`): Black about 1.7% per attempt (about 60 attempts), Blue 3.7–7% per attempt
(14–28 attempts), depending on the sample.
Current headline, with the bot's runs and the reporting-bias correction (November 2026): Black about 0.03% per
attempt (90% range 0–0.15%: the bot and bulk logs have never seen the shadow in about 700 tree runs), Blue about
1.8–2.5% per attempt (40–55 attempts), depending on the sample.
Form submissions over-report rare sightings: riders report a meadow Blue about 4 times more readily than a meadow
without one. The model estimates that bias for each kind of source and corrects for it (`planner/policy.yaml`,
`reporting_bias`; the estimates are in `policy.json` under `model.reportingBias`).
## 2. From the bot to the atlas