diff --git a/docs/BOT_CYCLE_FOR_KEN.md b/docs/BOT_CYCLE_FOR_KEN.md new file mode 100644 index 0000000..a15daa2 --- /dev/null +++ b/docs/BOT_CYCLE_FOR_KEN.md @@ -0,0 +1,53 @@ +# For Ken: what the bot needs for the automated cycle + +The Atlas side is done (branch `first-attempt-cycle`). This is the bot's side. Details and code are in +`docs/BOT_INTEGRATION.md`, "The automated cycle". Nothing here was changed in the bot's repository. + +## Why + +In the community form, the shadow shows on 9% of first attempts at the tree and 0.3% of later ones (69 in 748 +against 7 in 1,995, the same in every week). The bot has 236 tree runs and no shadow, all on attempt 2 or later; +it has never logged a first attempt. So: every new account's first attempt is the Black attempt, and it is also +the experiment that measures the first-attempt rate without reporting bias. + +## The five changes + +1. **Play attempt 1 with the bot and log it** (`accountAttempt: 1`). Today the records start at attempt 2. +2. **Give every account a number** in the order the accounts are made (`accountSeq`: 0, 1, 2, ...), fixed for + its life. The plan uses it to balance the arms across accounts. +3. **At the start of every attempt**, load the kit from `vendor/threshing-atlas/` (when `atlas-kit.json` changed), + then: + + ```js + const a = Solver.assign(plan, { accountSeq, attempt }); // what this attempt is for + const adv = Solver.advisor(policy.samples.all, a.target, a.objective, + { attempt, fix: a.fix, sampling: a.sampling, seed: a.seed }); + ``` + + and log `experimentId`, `arm`, `planId`, `policyId`, `accountAttempt` and the propensity on the run. Never + reload the kit in the middle of an attempt. +4. **The account's life** (`plan.lifecycle`): attempt 1 is the Black attempt; then three days of research + attempts; then retire the account, except when `accountSeq % 10 === 0` (those keep running). +5. **Run the cycle on a timer** on the bot's machine, from a clone of the Atlas: + + ``` + # every hour; does nothing but deliver when there are no new runs + 17 * * * * cd ~/threshing-atlas && git pull -q && DK_ACCOUNT_SALT=... python -m collection cycle \ + --bot ~/DragonKind --deliver ~/DragonKind/vendor/threshing-atlas >> ~/atlas-cycle.log 2>&1 + ``` + + or as the Gitea workflow `.gitea/workflows/bot-cycle.yml` on a runner on that machine, which also opens the + pull request with the new public data. Use Kathleen's salt (sent privately), so the accounts keep the hashes + they already have in the data. + +## What the plan asks for now (`planner/experiments.yaml`) + +| Attempts | Experiment | What the bot does | +|---|---|---| +| 1 | `first-attempt-shadow` | Investigate, help, tree. Direction and fork are West or East by arm (4 arms, 60 accounts each). Burn when there is no shadow; bond a Black. | +| 2 (half the accounts) | `attempt-2-shadow` | The same tree run once more, to check the shadow is gone. | +| 2 onward, 60% | `meadow-blue-route` | Go to the meadow; investigate or keep walking, and the first direction, by arm (6 arms, 70 each). Burn unless a Blue waits. | +| 2 onward, 25% | `mapping` | Pick at random among the options on each screen (`recommendOption` does it). | +| 2 onward, 15% | `sentinel` | One fixed route, to notice if the game's odds change. | + +The plan updates itself from the runs: a finished experiment stops being assigned and its share goes to the others. diff --git a/docs/PROJECT_CONTEXT.md b/docs/PROJECT_CONTEXT.md index f0f0b23..612293f 100644 --- a/docs/PROJECT_CONTEXT.md +++ b/docs/PROJECT_CONTEXT.md @@ -260,3 +260,28 @@ answer to `core/spec/form.yaml` (or the value to `data/dictionary.yaml`) and run - **Environment gotchas** (for Claude sessions): the cloud workspace and the Mac's Linux helper can't reach Google, Gitea or PyPI; Kathleen's own Terminal can. The helper lacks `scipy`. Git in the helper needs delete permission for its lock files. + +## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle + +Branch `first-attempt-cycle` (not merged, not pushed). + +- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first + attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the + tree and the bot has none, so "0 shadows in 579" was about later attempts. +- **Model.** `core/spec/game.yaml` has `givens` (first or later attempt) and the shadow roll depends on it first, + unpooled (`distinct: 1`). `policy.json` has `targets..firstAttempt`. Black: about 1.4% on a first attempt + (0.7–2.2%; 0.8% low-bias), 0.07% later. +- **An assumption to revisit.** The shadow's reporting bias cannot be measured, so `planner/policy.yaml` has + `borrow: { shadow: meadow }` (form reports of a shadow count about 1/3.9). Remove the line to go back to complete + logs only. The bot's first-attempt runs will settle it. +- **Plan.** `planner/experiments.yaml` (design) → `planner/build_experiments.py` → `data/public/experiments.json` + (progress and status per experiment). The bot reads it with `assign()` in `core/solver.js`; the advisor takes + `attempt`, `fix` and `sampling: "uniform"`. +- **Account lifecycle** (agreed with Ken): a new account's first attempt is the Black attempt; if it fails, the + account does research for three days, then retires (one in ten keeps running). +- **Cycle.** `python -m collection cycle --bot DIR --deliver DIR`: import, refit, re-plan, deliver; no-op without + new runs. `.gitea/workflows/bot-cycle.yml` runs it on a runner on the bot's machine (untested). +- **For Ken:** `docs/BOT_CYCLE_FOR_KEN.md`. The clone at `~/Desktop/DragonKind` is from 31 October and older than + the bot that is running, so the bot's side was written as a specification, not against its code. +- **Not done:** the page does not show first-attempt odds yet; Gitea still needs Kathleen's account, the + repository and the push (`docs/REPO_SETTINGS.md`); git.kenschiano.net answers (Gitea 1.27.0).