Route model: first attempts are their own shadow roll; experiment plan and automated bot cycle

- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
  depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
  so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
  0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
  experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
This commit is contained in:
ekcSchiano
2026-11-12 03:31:33 +00:00
co-authored by Claude Fable 5.2
parent ee07ee143e
commit ab3664f369
25 changed files with 1938 additions and 573 deletions
+58 -5
View File
@@ -9,6 +9,56 @@ Data flows both ways:
└──────────── collection --source bot + PR ◄┘
```
## The automated cycle
One command closes the loop, and it is meant to run on a timer on the machine that has the bot's folder:
```
python -m collection cycle --bot /path/to/DragonKind --deliver /path/to/DragonKind/vendor/threshing-atlas
```
1. **Ingest.** It imports the bot's new runs (anonymised: no prose, no account names; set `DK_ACCOUNT_SALT`, the
same value every time, so an account keeps its hash).
2. **Refit.** When there are new runs it rebuilds the coded runs, the route model (`policy.json`) and
3. **Re-plan:** the experiment plan (`experiments.json`): progress per arm, and which experiments are still
running (`planner/experiments.yaml` is the design).
4. **Deliver.** It copies `policy.json`, `experiments.json`, `solver.js` and `graph.js` into the `--deliver` folder,
each file whole, then writes `atlas-kit.json` (`policyId`, `planId`, checksums) last. The bot should load the
kit at the start of an attempt, never in the middle of one, and log both ids on the run.
With no new runs it only delivers, so running it every hour costs nothing. `exports/last-cycle.md` says what
changed (ids, headline odds, experiment progress). `--community` also checks the community sources first.
On Gitea, `.gitea/workflows/bot-cycle.yml` runs the same command on a runner that can see the bot's folder and
opens the pull request with the new data.
### What each attempt is for: the plan
```js
const plan = require('../vendor/threshing-atlas/experiments.json');
// accountSeq: the account's number in the order accounts were made (0, 1, 2, ...), fixed for its life
// attempt: the account's attempt number (1 = its first threshing)
const a = Solver.assign(plan, { accountSeq, attempt });
// -> { experimentId, arm, fix, target, objective, sampling, seed, propensity, burn, planId } (null: no experiment)
const advisor = Solver.advisor(policy.samples.all, a.target, a.objective,
{ attempt, fix: a.fix, sampling: a.sampling, seed: a.seed });
```
- **`attempt` matters.** The shadow is close to a first-attempt event, so `advisor(..., { attempt: 1 })` uses the
first attempt's odds. Without `attempt` a later attempt is assumed. **Attempt 1 must be played by the bot and
logged** (today the bot's records start at attempt 2).
- **`fix`** holds some decisions to the arm's options; the advisor plays its best around them.
- **`sampling: "uniform"`** (the mapping experiment): `recommendOption()` picks one of the options on the screen
at random and returns its propensity.
- Log on the run: `experimentId`, `arm`, `planId: plan.id`, `policyId: policy.id`, `accountAttempt`, and the
propensity. On the steps an arm fixes, the propensity is `a.propensity`.
- `burn` names the deliberate-burn rule the experiment expects (`TREE_NO_SHADOW_AUTO_BURN`,
`MEADOW_NO_BLUE_AUTO_BURN`). A genuine Black or Blue bond is always accepted.
- The account's life (`plan.lifecycle`): attempt 1 is the Black attempt; then about three days of research
attempts; then the account retires, except one in ten, which keeps going so that late attempts are observed.
An experiment that reaches its planned count becomes `complete`, one that meets its stopping rule becomes `paused`;
either way `assign()` stops giving it out and its share goes to the others. Nothing in the bot changes.
## From the Atlas's Gitea repository
Both repositories live on git.kenschiano.net. The bot can read the atlas files straight from the repository with a
@@ -20,7 +70,8 @@ https://git.kenschiano.net/<owner>/threshing-atlas/raw/branch/main/core/solver.j
https://git.kenschiano.net/<owner>/threshing-atlas/raw/branch/main/core/graph.js
```
(header `Authorization: token <token>`). Keep the three files together: solver.js reads the game from policy.json.
(header `Authorization: token <token>`), plus `data/public/experiments.json`. Keep the files together: solver.js
reads the game from policy.json.
## Without a shared repository: swapping zips
@@ -174,9 +225,10 @@ The tie rule below still works for goal accounts; research accounts don't need i
- `samples.lowBias`: bulk logs and riders not chasing a rare dragon. Use it for absolute expectations, such as how
many attempts a Blue should take. Rare sightings are over-reported in form posts, so `all` is optimistic for Blue.
Current headline, with the bot's runs and the reporting-bias correction (November 2026): Black about 0.03% per
attempt (90% range 0–0.15%: the bot and bulk logs have never seen the shadow in about 700 tree runs), Blue about
1.8–2.5% per attempt (40–55 attempts), depending on the sample.
Current headline (November 2026): Black about 1.4% on an account's first attempt (95% range 0.7–2.2%; 0.8% in the
low-bias sample) and about 0.07% on any later attempt; Blue about 1.9–2.5% per attempt (40–55 attempts), depending
on the sample. The first-attempt figure rests on form submissions and an assumed reporting bias (see below): the
bot and bulk logs have only 28 first attempts at the tree, and no shadow in about 570 tree runs in all.
The shadow is gated: it can only appear on runs that investigated the scream and helped the squadmate (the
"Black Dragon Gates" of the community route PDF, and the runs agree). The model knows this
@@ -184,7 +236,8 @@ The shadow is gated: it can only appear on runs that investigated the scream and
Form submissions over-report rare sightings: riders report a meadow Blue about 4 times more readily than a meadow
without one. The model estimates that bias for each kind of source and corrects for it (`planner/policy.yaml`,
`reporting_bias`; the estimates are in `policy.json` under `model.reportingBias`).
`reporting_bias`; the estimates are in `policy.json` under `model.reportingBias`). For the shadow the bias cannot be
measured (complete logs have never seen one), so it is assumed equal to the meadow Blue's (`borrow`).
## 2. From the bot to the atlas