First attempt cycle #1
@@ -0,0 +1,53 @@
|
||||
# For Ken: what the bot needs for the automated cycle
|
||||
|
||||
The Atlas side is done (branch `first-attempt-cycle`). This is the bot's side. Details and code are in
|
||||
`docs/BOT_INTEGRATION.md`, "The automated cycle". Nothing here was changed in the bot's repository.
|
||||
|
||||
## Why
|
||||
|
||||
In the community form, the shadow shows on 9% of first attempts at the tree and 0.3% of later ones (69 in 748
|
||||
against 7 in 1,995, the same in every week). The bot has 236 tree runs and no shadow, all on attempt 2 or later;
|
||||
it has never logged a first attempt. So: every new account's first attempt is the Black attempt, and it is also
|
||||
the experiment that measures the first-attempt rate without reporting bias.
|
||||
|
||||
## The five changes
|
||||
|
||||
1. **Play attempt 1 with the bot and log it** (`accountAttempt: 1`). Today the records start at attempt 2.
|
||||
2. **Give every account a number** in the order the accounts are made (`accountSeq`: 0, 1, 2, ...), fixed for
|
||||
its life. The plan uses it to balance the arms across accounts.
|
||||
3. **At the start of every attempt**, load the kit from `vendor/threshing-atlas/` (when `atlas-kit.json` changed),
|
||||
then:
|
||||
|
||||
```js
|
||||
const a = Solver.assign(plan, { accountSeq, attempt }); // what this attempt is for
|
||||
const adv = Solver.advisor(policy.samples.all, a.target, a.objective,
|
||||
{ attempt, fix: a.fix, sampling: a.sampling, seed: a.seed });
|
||||
```
|
||||
|
||||
and log `experimentId`, `arm`, `planId`, `policyId`, `accountAttempt` and the propensity on the run. Never
|
||||
reload the kit in the middle of an attempt.
|
||||
4. **The account's life** (`plan.lifecycle`): attempt 1 is the Black attempt; then three days of research
|
||||
attempts; then retire the account, except when `accountSeq % 10 === 0` (those keep running).
|
||||
5. **Run the cycle on a timer** on the bot's machine, from a clone of the Atlas:
|
||||
|
||||
```
|
||||
# every hour; does nothing but deliver when there are no new runs
|
||||
17 * * * * cd ~/threshing-atlas && git pull -q && DK_ACCOUNT_SALT=... python -m collection cycle \
|
||||
--bot ~/DragonKind --deliver ~/DragonKind/vendor/threshing-atlas >> ~/atlas-cycle.log 2>&1
|
||||
```
|
||||
|
||||
or as the Gitea workflow `.gitea/workflows/bot-cycle.yml` on a runner on that machine, which also opens the
|
||||
pull request with the new public data. Use Kathleen's salt (sent privately), so the accounts keep the hashes
|
||||
they already have in the data.
|
||||
|
||||
## What the plan asks for now (`planner/experiments.yaml`)
|
||||
|
||||
| Attempts | Experiment | What the bot does |
|
||||
|---|---|---|
|
||||
| 1 | `first-attempt-shadow` | Investigate, help, tree. Direction and fork are West or East by arm (4 arms, 60 accounts each). Burn when there is no shadow; bond a Black. |
|
||||
| 2 (half the accounts) | `attempt-2-shadow` | The same tree run once more, to check the shadow is gone. |
|
||||
| 2 onward, 60% | `meadow-blue-route` | Go to the meadow; investigate or keep walking, and the first direction, by arm (6 arms, 70 each). Burn unless a Blue waits. |
|
||||
| 2 onward, 25% | `mapping` | Pick at random among the options on each screen (`recommendOption` does it). |
|
||||
| 2 onward, 15% | `sentinel` | One fixed route, to notice if the game's odds change. |
|
||||
|
||||
The plan updates itself from the runs: a finished experiment stops being assigned and its share goes to the others.
|
||||
@@ -260,3 +260,28 @@ answer to `core/spec/form.yaml` (or the value to `data/dictionary.yaml`) and run
|
||||
- **Environment gotchas** (for Claude sessions): the cloud workspace and the Mac's Linux helper can't reach Google,
|
||||
Gitea or PyPI; Kathleen's own Terminal can. The helper lacks `scipy`. Git in the helper needs delete permission
|
||||
for its lock files.
|
||||
|
||||
## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle
|
||||
|
||||
Branch `first-attempt-cycle` (not merged, not pushed).
|
||||
|
||||
- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first
|
||||
attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the
|
||||
tree and the bot has none, so "0 shadows in 579" was about later attempts.
|
||||
- **Model.** `core/spec/game.yaml` has `givens` (first or later attempt) and the shadow roll depends on it first,
|
||||
unpooled (`distinct: 1`). `policy.json` has `targets.<colour>.firstAttempt`. Black: about 1.4% on a first attempt
|
||||
(0.7–2.2%; 0.8% low-bias), 0.07% later.
|
||||
- **An assumption to revisit.** The shadow's reporting bias cannot be measured, so `planner/policy.yaml` has
|
||||
`borrow: { shadow: meadow }` (form reports of a shadow count about 1/3.9). Remove the line to go back to complete
|
||||
logs only. The bot's first-attempt runs will settle it.
|
||||
- **Plan.** `planner/experiments.yaml` (design) → `planner/build_experiments.py` → `data/public/experiments.json`
|
||||
(progress and status per experiment). The bot reads it with `assign()` in `core/solver.js`; the advisor takes
|
||||
`attempt`, `fix` and `sampling: "uniform"`.
|
||||
- **Account lifecycle** (agreed with Ken): a new account's first attempt is the Black attempt; if it fails, the
|
||||
account does research for three days, then retires (one in ten keeps running).
|
||||
- **Cycle.** `python -m collection cycle --bot DIR --deliver DIR`: import, refit, re-plan, deliver; no-op without
|
||||
new runs. `.gitea/workflows/bot-cycle.yml` runs it on a runner on the bot's machine (untested).
|
||||
- **For Ken:** `docs/BOT_CYCLE_FOR_KEN.md`. The clone at `~/Desktop/DragonKind` is from 31 October and older than
|
||||
the bot that is running, so the bot's side was written as a specification, not against its code.
|
||||
- **Not done:** the page does not show first-attempt odds yet; Gitea still needs Kathleen's account, the
|
||||
repository and the push (`docs/REPO_SETTINGS.md`); git.kenschiano.net answers (Gitea 1.27.0).
|
||||
|
||||
Reference in New Issue
Block a user