- planner/knowledge.yaml: what the game's developer has said (paraphrased, dated, graded), what the Atlas does with each statement, and the open questions with the experiment that tests each. - analysis/black/evidence.py -> analysis/reports/BLACK.md: counts from the runs. On the same recorded route the shadow shows on 21 of 79 first attempts and 1 of 80 later ones; no single route; Blue then Black after the shadow. - game.yaml: the shadow is a guard (not a roll); what follows it is the Blue, then the Black; on-screen text for the squad's direction screen. - planner/experiments.yaml v2: one exact path on every fresh account (fresh-replicate), then one choice changed at a time (fresh-ablate); shadow-again, second-attempt-same-path, veteran-repeat, veteran-variety; what the bot records about the account; the sequence to follow once the shadow shows. - solver.js: assign() takes account facts, account offsets and cycled arms; recommendOption() answers the squad screen (stay + direction) in one pick. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
235 lines
13 KiB
YAML
235 lines
13 KiB
YAML
# The experiment plan for Ken's bot: what each attempt of each account is for.
|
|
#
|
|
# This file is the design (written by hand). planner/build_experiments.py checks it against the game
|
|
# (core/spec/game.yaml), counts what the bot has run so far, decides which experiments are running, and writes
|
|
# data/public/experiments.json. The bot reads that file with assign() in core/solver.js. The cycle
|
|
# (python -m collection cycle) rebuilds it after every import of bot runs, so the plan follows the data.
|
|
#
|
|
# Version 2 (2026-11-16). The developer has said the Black is not dice: it shows when a rider's choices meet its
|
|
# conditions (planner/knowledge.yaml), and riders report it almost only on a first attempt, on many different
|
|
# routes (analysis/reports/BLACK.md). So the Black experiments no longer estimate a rate. They look for the rule:
|
|
# hold one exact path fixed on fresh accounts, see whether the answer is always the same, then change one choice
|
|
# at a time. Because the answer is not dice, a few accounts per question are enough.
|
|
#
|
|
# Rules that keep the results honest:
|
|
# - counts are planned in advance; an experiment does not stop because its results look good;
|
|
# - every started attempt is logged, including aborted ones, and every screen with its options and the pick;
|
|
# - the bot never accepts a bond that is not the experiment's target colour (it burns), and it does accept a
|
|
# genuine Black or Blue bond.
|
|
version: 2
|
|
seed: 20261116 # changes every arm assignment: leave it alone once accounts are running
|
|
|
|
# ---------------------------------------------------------------- an account's life
|
|
lifecycle:
|
|
# attempt 1 of every new account is the Black attempt, on one exact path (fresh-replicate, then fresh-ablate)
|
|
research_days: 3 # then the account collects route data for this long (about 18 attempts at 4 hours each)
|
|
retire: true # and stops, except the long-lived accounts
|
|
long_lived: { every: 10, take: 2 } # two accounts in ten keep running (veteran-repeat and veteran-variety)
|
|
|
|
# What the bot records about the account on every run, beyond the steps: the Black's conditions rest on choices and
|
|
# on what the site knows about the account, so anything that could be such a fact is kept.
|
|
record:
|
|
- accountSeq # the account's number, in the order the accounts were made
|
|
- accountAttempt # 1 = the account's first threshing
|
|
- accountCreatedAt # when the account was made (UTC)
|
|
- wing # the cadet's assignment, as the site shows it
|
|
- section
|
|
- squad
|
|
- sawShadowBefore # whether any earlier run of the account showed the shadow
|
|
|
|
# ---------------------------------------------------------------- named paths
|
|
routes:
|
|
# The recorded route with the most first-attempt shadows in the community form: 21 in 79 first attempts, 1 in 80
|
|
# later ones (BLACK.md section 2).
|
|
best-reported:
|
|
company: Alone
|
|
direction: West
|
|
react: Hold still
|
|
scream: Investigate the scream
|
|
clearing: Help your squadmate
|
|
fight: Pause to assess
|
|
dagger: Engage the smaller cadet
|
|
fate: Squadmate's revenge
|
|
post: West
|
|
hour: Tree
|
|
|
|
# ---------------------------------------------------------------- what to do once the shadow shows
|
|
# The sequence in the notes of riders who bonded the Black (BLACK.md section 5), as keywords of the option to
|
|
# take on each screen, in order. The bot has never seen these screens, and a shadow must not be wasted: where a
|
|
# screen matches nothing here it should stop and notify a person, not guess.
|
|
scripts:
|
|
shadow:
|
|
- { step: the tree top, take: "shadow", never: "green|descend" }
|
|
- { step: asked to turn back, take: "stay the course|certain you saw", never: "second thought|green" }
|
|
- { step: asked again, take: "intriguing|even if this kills", never: "second thought|green" }
|
|
- { step: the Blue lands, take: "freeze and lower", never: "approach|run" }
|
|
- { step: another cadet arrives, take: "hold (completely )?still", never: "warn|turn while|approach" }
|
|
- { step: the Blue burns the cadet, take: "hold (your )?position", never: "interven|approach|warn" }
|
|
- { step: the Blue judges, take: "head lowered|await.*judg", never: "approach|sure" }
|
|
- { step: the Blue has left, take: "help the .*cadet", never: "" }
|
|
- { step: the cadet's burns, take: "wrap|cut .*sleeve|dagger to cut", never: "" }
|
|
- { step: moving on, take: "head south", never: "" }
|
|
- { step: a branch snaps, take: "turn to face", never: "ignore" }
|
|
- { step: the Black, take: "hold (its|it's|the) gaze|lift your chin", never: "lower your eyes" }
|
|
|
|
# ---------------------------------------------------------------- the experiments
|
|
# attempts: which of an account's attempts it covers (from / to, inclusive; no `to` = from then on).
|
|
# unit: what is randomised. account = one arm for the account's life; attempt = balanced blocks within an account.
|
|
# exclusive: takes every attempt it covers; the shared experiments (share) split the rest.
|
|
# accounts: { every: N, take: K, offset: J } limits it to K accounts in every N, starting at the J-th.
|
|
# when: facts about the account that must hold (from `record`).
|
|
# starts_after: waits until that experiment is complete.
|
|
# route: decisions held on every arm (a name from `routes`, or a mapping). factors: every combination is an arm.
|
|
# arms: explicit arms, each a `change` to the route (one choice at a time). cycle: a different combination of
|
|
# these decisions on every attempt, never repeating until all are used.
|
|
# measures: the roll or guard and the outcome that are counted. planned_per_arm: observed outcomes per arm.
|
|
experiments:
|
|
- id: fresh-replicate
|
|
question: On a fresh account, does one exact path always show the shadow, sometimes, or never?
|
|
why: >
|
|
If the Black is decided by this run's choices alone, identical choices on fresh accounts must agree: all
|
|
shadow or none. If they disagree, fresh accounts differ in something the site knows and we do not record
|
|
yet, and comparing the accounts that saw it with those that did not is how to find it.
|
|
attempts: { from: 1, to: 1 }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: best-reported
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 60 # if riders' 27% is three to four times too high, about 4 to 16 shadows
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: fresh-ablate
|
|
question: Which choices on that path does the shadow need?
|
|
why: Change one choice, keep the rest. The developer hints at about five conditions, two of them the known gates.
|
|
starts_after: fresh-replicate
|
|
attempts: { from: 1, to: 1 }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: best-reported
|
|
arms:
|
|
- { id: with one squadmate, change: { company: 1 Squadmate, leave: Stay together } }
|
|
- { id: with two squadmates, change: { company: 2 Squadmates, leave: Stay together } }
|
|
- { id: leave the squad, change: { company: 2 Squadmates, leave: Leave them } }
|
|
- { id: east first, change: { direction: East } }
|
|
- { id: centre first, change: { direction: Center } }
|
|
- { id: face the first dragon, change: { react: Respect / face it } }
|
|
- { id: charge the smaller cadet, change: { fight: Charge the smaller cadet } }
|
|
- { id: charge the bigger cadet, change: { dagger: Charge the bigger cadet anyway } }
|
|
- { id: kill the bullies, change: { fate: Killed the bullies yourself } }
|
|
- { id: leave the bullies, change: { fate: Left bullies to the dragons } }
|
|
- { id: fork to the forest, change: { post: Forest } }
|
|
- { id: fork east, change: { post: East } }
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 15 # set from fresh-replicate: enough that "none" is not bad luck at the rate found there
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: shadow-again
|
|
question: Does an account that saw the shadow, and died, see it again on the same path?
|
|
why: The cleanest test of fresh against veteran. About three in ten riders who reach the Black die there.
|
|
attempts: { from: 2, to: 4 }
|
|
when: { sawShadowBefore: true }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: best-reported
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 12
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: second-attempt-same-path
|
|
question: Does an account that missed on its first attempt miss again on the same path?
|
|
why: The form shows 21 in 79 on first attempts and 1 in 80 later on this route; the bot can check it on identical paths.
|
|
attempts: { from: 2, to: 2 }
|
|
when: { sawShadowBefore: false }
|
|
accounts: { every: 2, take: 1, offset: 0 }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: best-reported
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 60
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: veteran-repeat
|
|
question: Does an older account open up by repeating one path?
|
|
why: >
|
|
The developer says a run never hurts and names twenty as a count that matters. No bot account has repeated
|
|
one route more than 11 times in a row.
|
|
attempts: { from: 3 }
|
|
when: { sawShadowBefore: false }
|
|
accounts: { every: 10, take: 1, offset: 0 }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: best-reported
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 300
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: veteran-variety
|
|
question: Does an older account open up by varying its choices?
|
|
why: The opposite of veteran-repeat, on the next account. A different combination every attempt, both gates kept.
|
|
attempts: { from: 3 }
|
|
when: { sawShadowBefore: false }
|
|
accounts: { every: 10, take: 1, offset: 1 }
|
|
unit: account
|
|
exclusive: true
|
|
share: 1
|
|
target: Black
|
|
route: { scream: Investigate the scream, clearing: Help your squadmate, hour: Tree }
|
|
cycle:
|
|
company: [Alone, 1 Squadmate, 2 Squadmates]
|
|
direction: [West, East, Center]
|
|
fight: [Pause to assess, Charge the smaller cadet, Charge between the bigger cadet and your squadmate]
|
|
dagger: [Engage the smaller cadet, Charge the bigger cadet anyway]
|
|
fate: [Killed the bullies yourself, Squadmate's revenge, Left bullies to the dragons]
|
|
post: [Forest, West, East]
|
|
measures: { roll: shadow, rare: "yes" }
|
|
planned_per_arm: 300
|
|
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
|
|
|
|
- id: meadow-blue-route
|
|
question: Does anything before the meadow change how often the Blue waits there?
|
|
why: >
|
|
The developer says the Blue is dice from the seed. Complete logs show a Blue on about 10% of meadow visits
|
|
whatever the route; random assignment confirms it or finds the scene where the odds differ.
|
|
attempts: { from: 2 }
|
|
unit: attempt
|
|
share: 0.5
|
|
target: Blue
|
|
factors: { scream: [Investigate the scream, Keep walking], direction: [West, East, Center] }
|
|
route: { hour: Valley, valley: Pass the Red (meadow) }
|
|
measures: { roll: meadow, rare: Blue }
|
|
planned_per_arm: 70 # 210 per scream choice, 140 per direction: enough to tell 10% from 20%
|
|
burn: { reason: MEADOW_NO_BLUE_AUTO_BURN, when: no Blue in the meadow }
|
|
|
|
- id: mapping
|
|
question: What does every option on every screen lead to?
|
|
why: Probabilities for every path need runs on the paths nobody chooses. The game has about 24 million paths, so this maps screens, not paths.
|
|
attempts: { from: 2 }
|
|
unit: attempt
|
|
share: 0.35
|
|
target: Blue
|
|
sampling: uniform
|
|
planned_per_arm: 1000
|
|
burn: { reason: NOT_TARGET_AUTO_BURN, when: the encounter is not a Blue or a Black }
|
|
|
|
- id: sentinel
|
|
question: Have the game's odds changed?
|
|
why: One fixed route, run all the time by accounts that log everything. It is the most-logged meadow route.
|
|
attempts: { from: 2 }
|
|
unit: attempt
|
|
share: 0.15
|
|
target: Blue
|
|
route: { company: Alone, direction: Center, scream: Keep walking, wounded: Tourniquet, hour: Valley, valley: Pass the Red (meadow) }
|
|
measures: { roll: meadow, rare: Blue }
|
|
ongoing: true
|
|
burn: { reason: MEADOW_NO_BLUE_AUTO_BURN, when: no Blue in the meadow }
|