The Black is not dice: developer statements, evidence report, and a plan that looks for the rule

- planner/knowledge.yaml: what the game's developer has said (paraphrased, dated, graded), what the Atlas does
  with each statement, and the open questions with the experiment that tests each.
- analysis/black/evidence.py -> analysis/reports/BLACK.md: counts from the runs. On the same recorded route the
  shadow shows on 21 of 79 first attempts and 1 of 80 later ones; no single route; Blue then Black after the shadow.
- game.yaml: the shadow is a guard (not a roll); what follows it is the Blue, then the Black; on-screen text for
  the squad's direction screen.
- planner/experiments.yaml v2: one exact path on every fresh account (fresh-replicate), then one choice changed
  at a time (fresh-ablate); shadow-again, second-attempt-same-path, veteran-repeat, veteran-variety; what the bot
  records about the account; the sequence to follow once the shadow shows.
- solver.js: assign() takes account facts, account offsets and cycled arms; recommendOption() answers the
  squad screen (stay + direction) in one pick.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
This commit is contained in:
ekcSchiano
2026-11-16 15:24:03 +00:00
co-authored by Claude Fable 5.2
parent 51842e1aa9
commit d01ec46fcc
19 changed files with 1588 additions and 231 deletions
+173 -53
View File
@@ -1,85 +1,209 @@
# The experiment plan for Ken's bot: what each attempt of each account is for.
#
# This file is the design (written by hand). planner/build_experiments.py checks it against the game
# (core/spec/game.yaml), counts what the bot has run so far, decides which experiments are still running, and
# writes data/public/experiments.json. The bot reads that file with assign() in core/solver.js. The cycle
# (python -m collection cycle) rebuilds it after every import of bot runs, so the plan follows the data:
# an experiment that reaches its planned count, or its stopping rule, stops being assigned, and its share goes
# to the others.
# (core/spec/game.yaml), counts what the bot has run so far, decides which experiments are running, and writes
# data/public/experiments.json. The bot reads that file with assign() in core/solver.js. The cycle
# (python -m collection cycle) rebuilds it after every import of bot runs, so the plan follows the data.
#
# Version 2 (2026-11-16). The developer has said the Black is not dice: it shows when a rider's choices meet its
# conditions (planner/knowledge.yaml), and riders report it almost only on a first attempt, on many different
# routes (analysis/reports/BLACK.md). So the Black experiments no longer estimate a rate. They look for the rule:
# hold one exact path fixed on fresh accounts, see whether the answer is always the same, then change one choice
# at a time. Because the answer is not dice, a few accounts per question are enough.
#
# Rules that keep the results honest:
# - counts are planned in advance; an experiment does not stop because its results look good;
# - every started attempt is logged, including aborted ones;
# - every started attempt is logged, including aborted ones, and every screen with its options and the pick;
# - the bot never accepts a bond that is not the experiment's target colour (it burns), and it does accept a
# genuine Black or Blue bond.
version: 1
seed: 20261112 # changes every arm assignment: leave it alone once accounts are running
version: 2
seed: 20261116 # changes every arm assignment: leave it alone once accounts are running
# ---------------------------------------------------------------- an account's life
lifecycle:
# attempt 1 of every new account is the Black attempt (the shadow is close to a first-attempt event), and is
# itself an experiment: first-attempt-shadow below
# attempt 1 of every new account is the Black attempt, on one exact path (fresh-replicate, then fresh-ablate)
research_days: 3 # then the account collects route data for this long (about 18 attempts at 4 hours each)
retire: true # and stops, except the long-lived accounts
long_lived: { every: 10, take: 1 } # one account in ten keeps running, so attempts past 18 are still observed
long_lived: { every: 10, take: 2 } # two accounts in ten keep running (veteran-repeat and veteran-variety)
# What the bot records about the account on every run, beyond the steps: the Black's conditions rest on choices and
# on what the site knows about the account, so anything that could be such a fact is kept.
record:
- accountSeq # the account's number, in the order the accounts were made
- accountAttempt # 1 = the account's first threshing
- accountCreatedAt # when the account was made (UTC)
- wing # the cadet's assignment, as the site shows it
- section
- squad
- sawShadowBefore # whether any earlier run of the account showed the shadow
# ---------------------------------------------------------------- named paths
routes:
# The recorded route with the most first-attempt shadows in the community form: 21 in 79 first attempts, 1 in 80
# later ones (BLACK.md section 2).
best-reported:
company: Alone
direction: West
react: Hold still
scream: Investigate the scream
clearing: Help your squadmate
fight: Pause to assess
dagger: Engage the smaller cadet
fate: Squadmate's revenge
post: West
hour: Tree
# ---------------------------------------------------------------- what to do once the shadow shows
# The sequence in the notes of riders who bonded the Black (BLACK.md section 5), as keywords of the option to
# take on each screen, in order. The bot has never seen these screens, and a shadow must not be wasted: where a
# screen matches nothing here it should stop and notify a person, not guess.
scripts:
shadow:
- { step: the tree top, take: "shadow", never: "green|descend" }
- { step: asked to turn back, take: "stay the course|certain you saw", never: "second thought|green" }
- { step: asked again, take: "intriguing|even if this kills", never: "second thought|green" }
- { step: the Blue lands, take: "freeze and lower", never: "approach|run" }
- { step: another cadet arrives, take: "hold (completely )?still", never: "warn|turn while|approach" }
- { step: the Blue burns the cadet, take: "hold (your )?position", never: "interven|approach|warn" }
- { step: the Blue judges, take: "head lowered|await.*judg", never: "approach|sure" }
- { step: the Blue has left, take: "help the .*cadet", never: "" }
- { step: the cadet's burns, take: "wrap|cut .*sleeve|dagger to cut", never: "" }
- { step: moving on, take: "head south", never: "" }
- { step: a branch snaps, take: "turn to face", never: "ignore" }
- { step: the Black, take: "hold (its|it's|the) gaze|lift your chin", never: "lower your eyes" }
# ---------------------------------------------------------------- the experiments
# attempts: which of an account's attempts it covers (from / to, inclusive; no `to` = from then on).
# unit: what is randomised. account = one arm for the account's life (needed when only attempt 1 counts);
# attempt = arms in balanced blocks within each account.
# share: when several experiments cover an attempt, how often each gets it.
# exclusive: takes every attempt it covers; the shared experiments get the rest.
# accounts: { every: N, take: K } limits it to K accounts in every N (by the order accounts were made).
# factors: decisions that are randomised; every combination is an arm. route: decisions held on every arm.
# Everything else is the advisor's best play for the target.
# sampling: best (default), thompson, or uniform (every option on the screen equally likely: mapping).
# measures: the roll and the outcome that are counted. planned_per_arm: how many observed rolls per arm.
# stop_early: pause when, after `after` observed rolls in total, at most `rare_at_most` came out rare.
# unit: what is randomised. account = one arm for the account's life; attempt = balanced blocks within an account.
# exclusive: takes every attempt it covers; the shared experiments (share) split the rest.
# accounts: { every: N, take: K, offset: J } limits it to K accounts in every N, starting at the J-th.
# when: facts about the account that must hold (from `record`).
# starts_after: waits until that experiment is complete.
# route: decisions held on every arm (a name from `routes`, or a mapping). factors: every combination is an arm.
# arms: explicit arms, each a `change` to the route (one choice at a time). cycle: a different combination of
# these decisions on every attempt, never repeating until all are used.
# measures: the roll or guard and the outcome that are counted. planned_per_arm: observed outcomes per arm.
experiments:
- id: first-attempt-shadow
question: How often does the shadow appear on an account's first attempt, and does heading West (at the start, at the fork) raise it?
- id: fresh-replicate
question: On a fresh account, does one exact path always show the shadow, sometimes, or never?
why: >
Form submissions show the shadow on 9% of first attempts and 0.3% of later ones, but form submissions
over-report it, complete logs have only 28 first attempts, and the bot has none. Among first attempts the
form shows more shadows after West (14%) than East (4%), which may be copied guide routes.
If the Black is decided by this run's choices alone, identical choices on fresh accounts must agree: all
shadow or none. If they disagree, fresh accounts differ in something the site knows and we do not record
yet, and comparing the accounts that saw it with those that did not is how to find it.
attempts: { from: 1, to: 1 }
unit: account
exclusive: true
share: 1
target: Black
objective: eventually
factors: { direction: [West, East], post: [West, East] }
route: { scream: Investigate the scream, clearing: Help your squadmate, hour: Tree }
route: best-reported
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 60 # 240 accounts: 120 per level of each factor
stop_early: { after: 120, rare_at_most: 0 } # no shadow in 120 first attempts rules out 2.5% or more
planned_per_arm: 60 # if riders' 27% is three to four times too high, about 4 to 16 shadows
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: attempt-2-shadow
question: Is the shadow really gone after the first attempt?
why: The form shows 3 shadows in 295 second attempts (1%), then almost none.
attempts: { from: 2, to: 2 }
- id: fresh-ablate
question: Which choices on that path does the shadow need?
why: Change one choice, keep the rest. The developer hints at about five conditions, two of them the known gates.
starts_after: fresh-replicate
attempts: { from: 1, to: 1 }
unit: account
accounts: { every: 2, take: 1 }
exclusive: true
share: 1
target: Black
objective: eventually
route: { scream: Investigate the scream, clearing: Help your squadmate, hour: Tree }
route: best-reported
arms:
- { id: with one squadmate, change: { company: 1 Squadmate, leave: Stay together } }
- { id: with two squadmates, change: { company: 2 Squadmates, leave: Stay together } }
- { id: leave the squad, change: { company: 2 Squadmates, leave: Leave them } }
- { id: east first, change: { direction: East } }
- { id: centre first, change: { direction: Center } }
- { id: face the first dragon, change: { react: Respect / face it } }
- { id: charge the smaller cadet, change: { fight: Charge the smaller cadet } }
- { id: charge the bigger cadet, change: { dagger: Charge the bigger cadet anyway } }
- { id: kill the bullies, change: { fate: Killed the bullies yourself } }
- { id: leave the bullies, change: { fate: Left bullies to the dragons } }
- { id: fork to the forest, change: { post: Forest } }
- { id: fork east, change: { post: East } }
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 120
planned_per_arm: 15 # set from fresh-replicate: enough that "none" is not bad luck at the rate found there
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: shadow-again
question: Does an account that saw the shadow, and died, see it again on the same path?
why: The cleanest test of fresh against veteran. About three in ten riders who reach the Black die there.
attempts: { from: 2, to: 4 }
when: { sawShadowBefore: true }
unit: account
exclusive: true
share: 1
target: Black
route: best-reported
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 12
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: second-attempt-same-path
question: Does an account that missed on its first attempt miss again on the same path?
why: The form shows 21 in 79 on first attempts and 1 in 80 later on this route; the bot can check it on identical paths.
attempts: { from: 2, to: 2 }
when: { sawShadowBefore: false }
accounts: { every: 2, take: 1, offset: 0 }
unit: account
exclusive: true
share: 1
target: Black
route: best-reported
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 60
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: veteran-repeat
question: Does an older account open up by repeating one path?
why: >
The developer says a run never hurts and names twenty as a count that matters. No bot account has repeated
one route more than 11 times in a row.
attempts: { from: 3 }
when: { sawShadowBefore: false }
accounts: { every: 10, take: 1, offset: 0 }
unit: account
exclusive: true
share: 1
target: Black
route: best-reported
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 300
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: veteran-variety
question: Does an older account open up by varying its choices?
why: The opposite of veteran-repeat, on the next account. A different combination every attempt, both gates kept.
attempts: { from: 3 }
when: { sawShadowBefore: false }
accounts: { every: 10, take: 1, offset: 1 }
unit: account
exclusive: true
share: 1
target: Black
route: { scream: Investigate the scream, clearing: Help your squadmate, hour: Tree }
cycle:
company: [Alone, 1 Squadmate, 2 Squadmates]
direction: [West, East, Center]
fight: [Pause to assess, Charge the smaller cadet, Charge between the bigger cadet and your squadmate]
dagger: [Engage the smaller cadet, Charge the bigger cadet anyway]
fate: [Killed the bullies yourself, Squadmate's revenge, Left bullies to the dragons]
post: [Forest, West, East]
measures: { roll: shadow, rare: "yes" }
planned_per_arm: 300
burn: { reason: TREE_NO_SHADOW_AUTO_BURN, when: no shadow at the tree }
- id: meadow-blue-route
question: Does anything before the meadow change how often the Blue waits there?
why: >
Complete logs show a Blue on about 10% of meadow visits whatever the route, from routes the bot's rules chose.
Random assignment is what can confirm that the route does not matter (or find the route where it does).
The developer says the Blue is dice from the seed. Complete logs show a Blue on about 10% of meadow visits
whatever the route; random assignment confirms it or finds the scene where the odds differ.
attempts: { from: 2 }
unit: attempt
share: 0.6
share: 0.5
target: Blue
objective: eventually
factors: { scream: [Investigate the scream, Keep walking], direction: [West, East, Center] }
route: { hour: Valley, valley: Pass the Red (meadow) }
measures: { roll: meadow, rare: Blue }
@@ -87,28 +211,24 @@ experiments:
burn: { reason: MEADOW_NO_BLUE_AUTO_BURN, when: no Blue in the meadow }
- id: mapping
question: What does every branch of the valley lead to?
why: Probabilities for every path to every dragon need runs on the paths nobody chooses.
question: What does every option on every screen lead to?
why: Probabilities for every path need runs on the paths nobody chooses. The game has about 24 million paths, so this maps screens, not paths.
attempts: { from: 2 }
unit: attempt
share: 0.25
share: 0.35
target: Blue
objective: eventually
sampling: uniform
planned_per_arm: 600
planned_per_arm: 1000
burn: { reason: NOT_TARGET_AUTO_BURN, when: the encounter is not a Blue or a Black }
- id: sentinel
question: Have the game's odds changed?
why: >
One fixed route, run all the time, by accounts that log everything: a change in its rates is a change in the
game, not in who reports. It is the most-logged meadow route.
why: One fixed route, run all the time by accounts that log everything. It is the most-logged meadow route.
attempts: { from: 2 }
unit: attempt
share: 0.15
target: Blue
objective: eventually
route: { company: Alone, direction: Center, scream: Keep walking, wounded: Tourniquet, hour: Valley, valley: Pass the Red (meadow) }
measures: { roll: meadow, rare: Blue }
ongoing: true # never completes
ongoing: true
burn: { reason: MEADOW_NO_BLUE_AUTO_BURN, when: no Blue in the meadow }