Advisor: Black research runs always keep the gates and climb the tree #56

Merged
kathleen merged 1 commits from solver/never-rule-out-target into main 2026-11-22 02:20:54 +00:00
Owner

The bug. A research account seeking the Black went to the valley center on attempts 90, 91 and 102. In all three, the bot logged the pick as ATLAS_POLICY_TIE:hour with propensity 0.5. In 91 and 102 it also walked away from the squadmate (ATLAS_POLICY_TIE:clearing), which closes the Black's gate. Every run up to attempt 89 climbed the tree.

Why. Since the route model treats the shadow's attempt as its own level (#45), a later attempt puts the Black at about 0 on every route:

  • first attempts: 21 shadows in 79 form runs;
  • later attempts at the tree: 1 shadow in about 600 runs across form, bot and bulk logs;
  • the model's later-attempt estimate is a point value of about 1e-8.

With every option at 0%, everything is within the bot's 0.2-point tie margin, so AtlasPolicy.advise() picks among all of them at random.

The fix (core/solver.js). The advisor also solves the game with no data. Every roll is possible except through a closed gate (game.yaml requires), and an encounter bonds its own colour or any colour seen there. An option after which the target can't happen at all is marked ruledOut:

  • the advisor never chooses it while another option keeps the target possible;
  • recommendOption() leaves it out of the options it hands the bot, so the tie rule can't pick it.

For the Black, that means always investigate the scream, help the squadmate and climb the tree, matching the developer's statement that the Black can't be bonded without climbing the tree. Real near-ties between options that keep the target possible are still randomized as before. No values change, and the published route tree still matches option for option (advisor.test.js).

Test. A new case in bot_contract.test.js copies the bot's tie rule on the exact screens of run 91. It checks the tie set never contains Keep walking, Walk away or the Valley. It fails on the old solver and passes on this one.

Not in this PR. The "first attempt only" assumption itself, meaning the near-zero later-attempt estimate, isn't proven. Loosening it changes the published Black numbers, so it gets its own discussion.

🤖 Generated with Claude Code

**The bug.** A research account seeking the Black went to the valley center on attempts 90, 91 and 102. In all three, the bot logged the pick as `ATLAS_POLICY_TIE:hour` with propensity 0.5. In 91 and 102 it also walked away from the squadmate (`ATLAS_POLICY_TIE:clearing`), which closes the Black's gate. Every run up to attempt 89 climbed the tree. **Why.** Since the route model treats the shadow's attempt as its own level (#45), a later attempt puts the Black at about 0 on every route: - first attempts: 21 shadows in 79 form runs; - later attempts at the tree: 1 shadow in about 600 runs across form, bot and bulk logs; - the model's later-attempt estimate is a point value of about 1e-8. With every option at 0%, everything is within the bot's 0.2-point tie margin, so `AtlasPolicy.advise()` picks among all of them at random. **The fix (`core/solver.js`).** The advisor also solves the game with no data. Every roll is possible except through a closed gate (game.yaml `requires`), and an encounter bonds its own colour or any colour seen there. An option after which the target can't happen at all is marked `ruledOut`: - the advisor never chooses it while another option keeps the target possible; - `recommendOption()` leaves it out of the options it hands the bot, so the tie rule can't pick it. For the Black, that means always investigate the scream, help the squadmate and climb the tree, matching the developer's statement that the Black can't be bonded without climbing the tree. Real near-ties between options that keep the target possible are still randomized as before. No values change, and the published route tree still matches option for option (advisor.test.js). **Test.** A new case in `bot_contract.test.js` copies the bot's tie rule on the exact screens of run 91. It checks the tie set never contains Keep walking, Walk away or the Valley. It fails on the old solver and passes on this one. **Not in this PR.** The "first attempt only" assumption itself, meaning the near-zero later-attempt estimate, isn't proven. Loosening it changes the published Black numbers, so it gets its own discussion. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
kathleen added 1 commit 2026-11-22 02:16:14 +00:00
On a later attempt the data put the Black near zero on every route, so every option
tied and the bot's tie rule (random among options within 0.2 points) walked away from
the squadmate or went to the valley (DragonKind research runs 90, 91 and 102). The
advisor now also solves the game with no data (every roll possible except through a
closed gate; an encounter bonds its own colour) and marks options after which the
target cannot happen at all as `ruledOut`. The choice is never one of them while
another option keeps the target possible, and recommendOption leaves them out of the
options it gives the bot. For the Black: always investigate, help and climb the tree.

Values are unchanged; the published route tree still matches option for option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
kathleen scheduled this pull request to auto merge when all checks succeed 2026-11-22 02:17:57 +00:00
kathleen merged commit 2ec176a864 into main 2026-11-22 02:20:54 +00:00
kathleen deleted branch solver/never-rule-out-target 2026-11-22 02:20:55 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DragonKind/threshing-atlas#56