Route model: correct for reporting bias
- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group, the factor by which the group over-reports the rare outcome relative to complete logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count 1/rho; a roll the complete logs have (almost) never shown is learned from them only - planner/policy.yaml: groups, rolls and thresholds; estimates published in policy.json under model.reportingBias - fit_alpha: a roll seen only one way is pooled fully - docs: goal accounts follow the best route, research accounts use Thompson sampling (decisionMode atlas_thompson) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
c0355616b6
commit
2271fe0976
+33
-7
@@ -125,12 +125,33 @@ The atlas is advice. The bot's own safety rules stay in charge:
|
||||
Log every atlas pick with `rule: 'ATLAS_POLICY'`, `source: 'atlas'`, and the run-level `policyId: policy.id`
|
||||
(see `docs/BOT_DATA_PROTOCOL.md`), so we can tell which policy produced which runs.
|
||||
|
||||
### Ties are free experiments
|
||||
### Learning: goal accounts follow the best route, research accounts sample
|
||||
|
||||
Many choices barely matter by the data we have (for Black, the first direction differs by a tenth of a percentage
|
||||
point). When `margin` is below the tie threshold (start with 0.002, i.e. 0.2 percentage points), pick at random
|
||||
among the tied options and log `propensity` (1 / number of tied options). That costs nothing in expected success,
|
||||
and it is exactly the randomised data the atlas needs to tell real history effects from guide-copying.
|
||||
The atlas learns from the bot's own runs: every zip of the bot's data that is imported adds its rolls (first dragon,
|
||||
deaths, the shadow, the meadow dragon, bond results) to the route model, and the next `atlas-for-bot` zip carries
|
||||
the updated model. Deliberate burns (Malek, "no shadow", "no Blue") never count as failed bond attempts.
|
||||
|
||||
How the bot should choose depends on the account's role:
|
||||
|
||||
- **Goal accounts** (`accountRole: "goal"`): the best route under the model, as before:
|
||||
`Solver.advisor(policy.samples.all, target, 'eventually')`.
|
||||
- **Research accounts** (`accountRole: "research"`): Thompson sampling. Make a new advisor for every attempt with a
|
||||
new seed:
|
||||
|
||||
```js
|
||||
const adv = Solver.advisor(policy.samples.all, target, 'eventually', { sampling: 'thompson', seed: attemptNumber });
|
||||
```
|
||||
|
||||
Each attempt follows the best route of one plausible version of the odds, drawn from the model's uncertainty. An
|
||||
option that might be the best one gets tried about as often as it is likely to be the best, so the attempts go
|
||||
where the model is least sure; as the data firm up, sampling settles on the best route by itself.
|
||||
`recommendOption()` then returns a `propensity` (the chance this option was picked, from 40 other draws; set
|
||||
`draws` to change that). Log it on the step with `decisionMode: "atlas_thompson"`, so the runs can be re-weighted.
|
||||
Making the advisor takes a few seconds per attempt (it solves the game once per draw).
|
||||
|
||||
The tie rule below still works for goal accounts; research accounts don't need it.
|
||||
|
||||
### Ties are free experiments
|
||||
|
||||
### Which numbers to trust
|
||||
|
||||
@@ -140,8 +161,13 @@ and it is exactly the randomised data the atlas needs to tell real history effec
|
||||
- `samples.lowBias`: bulk logs and riders not chasing a rare dragon. Use it for absolute expectations, such as how
|
||||
many attempts a Blue should take. Rare sightings are over-reported in form posts, so `all` is optimistic for Blue.
|
||||
|
||||
Current headline (from `analysis/reports/POLICY.md`): Black about 1.7% per attempt (about 60 attempts), Blue 3.7–7% per attempt
|
||||
(14–28 attempts), depending on the sample.
|
||||
Current headline, with the bot's runs and the reporting-bias correction (November 2026): Black about 0.03% per
|
||||
attempt (90% range 0–0.15%: the bot and bulk logs have never seen the shadow in about 700 tree runs), Blue about
|
||||
1.8–2.5% per attempt (40–55 attempts), depending on the sample.
|
||||
|
||||
Form submissions over-report rare sightings: riders report a meadow Blue about 4 times more readily than a meadow
|
||||
without one. The model estimates that bias for each kind of source and corrects for it (`planner/policy.yaml`,
|
||||
`reporting_bias`; the estimates are in `policy.json` under `model.reportingBias`).
|
||||
|
||||
## 2. From the bot to the atlas
|
||||
|
||||
|
||||
Reference in New Issue
Block a user