Commit Graph
35 Commits
Author SHA1 Message Date
kathleenandClaude Opus 5.5 6510bfbd06 Tests for the route model's arithmetic on hand-made data (#26)
CI / test (pull_request) Successful in 1m57s
The hierarchical back-off, the empirical-Bayes concentration and its one-outcome
guard, the shadow's gate (counting and solving), which bonds an encounter allows,
the first-attempt given with its token prior, and the reproducible random numbers.
Expected values are worked out by hand in the comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 21:37:28 +00:00
ekcSchianoandClaude Opus 5.5 1dfa8fb4c3 Stable run ids (fingerprints) for the form and the register; tests for the site statistics (#25)
CI / test (pull_request) Successful in 43s
- collection/etl/runs.py fingerprints(): a form or register run's id is now <prefix>-<hash of the row's non-empty
  answers, timestamp included> instead of its row number, so deleting or re-sorting rows in the sheet moves no
  other run's id, and a column added later changes none. The 32 rows that are exact copies of another row get
  -2, -3 (identical, so which is which does not matter). A row edited in the sheet gets a new id. Every id in
  runs_clean.csv and overlap.json changes once; nothing else does (no statistic or model reads the ids).
- tests/core/test_site_stats.py (9 tests): measures.yaml conditions, source groups and presets, bond
  denominators without deliberate deaths (cards, encounters, route map), the riders' paths, and a JSON-safe
  stats.json, on a fixed sample of the published runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 20:16:16 +00:00
ekcSchianoandClaude Opus 5.5 98be27fc1f Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s
- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
  on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
  runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
  be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
  to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
  form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
  match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
  page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
  stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:44:52 +00:00
ekcSchianoandClaude Opus 5.5 46c1f00150 Impossible runs: one definition, left out of the route model and the site statistics (#21, #22)
CI / test (pull_request) Successful in 41s
- core/spec/form.yaml impossible_flags: the flags that mean a run cannot have happened as recorded (a dragon an
  encounter does not have, a Green or the Black in the valley, two bonds, two burns, a cadet path that does not
  follow, an early death that still reached the hour...). atlas.runs.impossible() reads it.
- The route model and the site statistics (and the riders' paths) leave those runs out, from every source; every
  other run is used, however partial. The runs stay in runs_clean.csv, flagged. 80 runs: 75 form, 5 Reddit.
  Route-model odds move by at most 0.2 points.
- collection/etl/runs.py: the community form gets the same game-rule checks as "Report a run" and the Reddit
  comments (10 form rows newly flagged); new rule: the Black in the valley.
- The first-sighting pool stays a flag only: it is what riders have seen, not a rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:03:45 +00:00
kathleen fdb603f5a4 Merge pull request 'Game-rule checks for runs answered through "Report a run"' (#37) from data/report-rule-checks into main
CI / test (push) Successful in 41s
2026-11-19 14:10:08 +00:00
kschianoandClaude Opus 5.5 0f4430f6f5 Game-rule checks for runs answered through "Report a run"
CI / test (pull_request) Successful in 2m0s
Reported and Reddit-comment runs now get the checks the community form
gets in core/atlas/coding.py, as data_flags (the run is kept):
- first dragon outside the documented pool for the company/direction
- a bond or burn by a dragon the encounter does not have (new
  encounter_colours table in core/spec/form.yaml; Orange/Brown after the
  two Greens allowed, as a Green's rescue)
- a Green on a valley run
- a cadet outcome off its scream path, fight answers or a post-fight path
  on a run that did not help, a second fight move without assessing
- died before the hour, yet reached the tree/valley or an encounter

planner/policy.yaml drop_flagged: flagged runs of sources read from free
text (reddit comment) stay out of the route model; provenance notes do not
count as flags. No effect today: the source is not in `sources` yet.

On the first 82 scraped runs: 6 flagged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 08:46:57 -05:00
ekcSchianoandClaude Opus 5.5 e10312ab02 Known answers for the bot: the guide's encounter answers and the bot's own results, by choice text (#35)
CI / test (pull_request) Successful in 2m3s
analysis/known/build_known.py writes data/public/known_answers.json from pdf_encounters.json and tree.json;
solver.js knownAnswer(known, offered, { target, sceneColour }) matches a screen's choices against it (the guide's
paraphrases by their words, the bot's texts exactly) and returns a pick when one choice is clearly on the guide's
recommended path to the target, plus the evidence for every choice to hand to Clef. Rebuilt daily, checked in CI,
and sent to the bot with the route model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 13:32:09 +00:00
kschianoandClaude Opus 5.5 197c16dac0 Receive runs from the Reddit megathread comments (source "reddit comment")
CI / test (pull_request) Successful in 46s
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs
riders describe in the r/fourthwing megathreads and delivers them here by
pull request as data/public/reddit_comments.json: each run is the "Report a
run" answers (option labels of collection/specs/report_form.yaml), with the
comment id and day only; no usernames, no comment text.

- pipeline.yaml: a reddit_comments source of extract kind "delivered" (the
  atlas fetches nothing; `update` skips it with a note)
- runs.py: the coder, sharing one answers_row() with the reported coder;
  every row is flagged as read from a Reddit comment (and "not checked by
  hand" until corrected); a result page a comment already carries is not
  counted twice as a "reddit thread" row
- collection/manual/reddit_comments.yaml: reject / fix / add, applied at
  build time
- dictionary, SOURCES, README, croissant: the new source

The route model and statistics do not use the source until it is named in
planner/policy.yaml / analysis/stats/measures.yaml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 22:06:14 -05:00
ekcSchianoandClaude Opus 5.5 3b5c655d51 Refresh to 8,748 spreadsheet runs; squad answers ignore case; rebuild checks the runs first
CI / test (pull_request) Successful in 1m59s
- spreadsheet 8,132 -> 8,748 rows (about 600 new; the sheet's keeper fixed 'Sqaudmate' in 72 answers,
  corrected 15 attempt numbers to 'First Attempt' and re-pasted the daily bulk logs); register 288 -> 292
- form.yaml squad_any_case: a typed 'second' is Second (it stopped the build)
- collection rebuild: the runs are built and checked first, so a failed check stops it before the model,
  plan, reports and statistics are rebuilt from stale runs
- route model, plan, Black evidence, screen tree, statistics, graph and metadata rebuilt (9,473 runs)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-16 17:13:23 +00:00
ekcSchianoandClaude Fable 5.2 f8e4f575f4 Screen tree, register rows in the master list, and bot runs coded from the picks that were played
CI / test (pull_request) Canceled after 0s
- analysis/tree/build_tree.py -> data/public/tree.json, analysis/reports/TREE.md: every screen the bot has seen,
  every option, where each led (fixed or dice, with ranges), best known picks per colour chosen cautiously, where
  any picks end, and the untested options. solver.js screenPick(): least-tried and Thompson picks from the tree.
- The bot logs a pick it then overrode as a step: 37 runs were coded with choices they never made. Importer and
  graph.js now use the played steps only.
- runs_clean.csv: the Unofficial Dragon Register's 288 bonded dragons are in the master list as outcome-only
  rows, flagged (no analysis uses them).
- Cycle, refresh and the bot kit rebuild and carry the tree; tests check every generated report is current.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-16 15:50:12 +00:00
ekcSchianoandClaude Fable 5.2 d01ec46fcc The Black is not dice: developer statements, evidence report, and a plan that looks for the rule
- planner/knowledge.yaml: what the game's developer has said (paraphrased, dated, graded), what the Atlas does
  with each statement, and the open questions with the experiment that tests each.
- analysis/black/evidence.py -> analysis/reports/BLACK.md: counts from the runs. On the same recorded route the
  shadow shows on 21 of 79 first attempts and 1 of 80 later ones; no single route; Blue then Black after the shadow.
- game.yaml: the shadow is a guard (not a roll); what follows it is the Blue, then the Black; on-screen text for
  the squad's direction screen.
- planner/experiments.yaml v2: one exact path on every fresh account (fresh-replicate), then one choice changed
  at a time (fresh-ablate); shadow-again, second-attempt-same-path, veteran-repeat, veteran-variety; what the bot
  records about the account; the sequence to follow once the shadow shows.
- solver.js: assign() takes account facts, account offsets and cycled arms; recommendOption() answers the
  squad screen (stay + direction) in one pick.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-16 15:24:03 +00:00
ekcSchianoandClaude Fable 5.2 ab3664f369 Route model: first attempts are their own shadow roll; experiment plan and automated bot cycle
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
  depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
  so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
  0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
  experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-12 03:31:33 +00:00
ekcSchianoandClaude Opus 5.5 614bfcda7c Host on Gitea: forge.yaml, Gitea workflows, host-neutral links
- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
  the "Report a run" collector, the page's links (data/public/links.json, filled in by
  site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
  for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
  merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-12 02:46:17 +00:00
ekcSchianoandClaude Opus 5.5 8f1f0383fe Game: which dragons each encounter can give; impossible bonds are not counted
A form row recording a Green bonded at the shadow Blue let the "eventually" objective
find a route whose only possible bond was that Green (ratio 1, near-zero chance).
game.yaml now lists the possible bonds per encounter (own colour; the two Greens also
Orange and Brown); anything else is a data-entry error and is not counted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:36:49 +00:00
ekcSchianoandClaude Opus 5.5 64fd3af235 Game: the shadow is gated on investigating the scream and helping
- game.yaml: chance events can have `requires` (a gate) and `otherwise`; the shadow
  requires "Investigate the scream" and "Help your squadmate" (the route PDF's Black
  Dragon Gates; spreadsheet: 78 shadows in 4199 gated tree runs, 2 in 646 others)
- model and advisor: a closed gate fixes the roll; runs off the gate are not counted
- form.yaml: "Protect / Help Squadmate" (new answer) -> Protect squadmate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:33:07 +00:00
ekcSchianoandClaude Opus 5.5 2271fe0976 Route model: correct for reporting bias
- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group,
  the factor by which the group over-reports the rare outcome relative to complete
  logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count
  1/rho; a roll the complete logs have (almost) never shown is learned from them only
- planner/policy.yaml: groups, rolls and thresholds; estimates published in
  policy.json under model.reportingBias
- fit_alpha: a roll seen only one way is pooled fully
- docs: goal accounts follow the best route, research accounts use Thompson sampling
  (decisionMode atlas_thompson)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:29:09 +00:00
ekcSchianoandClaude Opus 5.5 c0355616b6 Route model learns from bot runs; Thompson sampling in the advisor (work in progress)
- planner/policy.yaml: which sources the route model learns from (spreadsheet,
  reported, bot) and the low-bias sample
- bot runs: deaths coded by stage, Malek intent for runs with a target
- model: survival of the first dragon is not counted when nothing after it is
  known; the meadow dragon falls back to what the bot saw
- solver.js: advisor(..., {sampling: "thompson", seed, draws}) follows one
  posterior draw per attempt and reports each pick's propensity

Not shipped yet: mixing sources lowers the fitted shrinkage for the shadow roll
(alpha 500 -> 50 in the low-bias sample), see the conversation of 2026-11-09.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:29:09 +00:00
ekcSchianoandClaude Opus 5.5 43066ad67f Bot import: one folder per account, run records with steps
- the importer reads data/accounts/*/runs.jsonl and logs/accounts/*/encounters.jsonl;
  with account folders, the top-level files (a mirror of the first account) are skipped
- run records that carry their steps (schema 2) become trajectories directly, keeping
  what the bot saw (shadow, meadow dragons), propensities, decision mode and policy id;
  older runs are still rebuilt from the decision log; bonded runs without steps are kept
- account names become salted hashes (DK_ACCOUNT_SALT, or a private salt in data/raw)
- runs_clean.csv: new bot_mode column; saw_shadow and saw_blue_meadow from what the bot
  saw; deliberate burns say why
- crosswalk: the valley Red, leaving squadmates, squad direction, hold your ground,
  hide/approach reactions, the wounded cadet's other two choices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:08:57 +00:00
ekcSchianoandClaude Opus 5.5 5c49591dae Hand-offs without GitHub: bot kit zip, bot zip import, local refresh
- python -m collection export-bot-kit: policy.json, solver.js, graph.js and a
  README with checksums, zipped for the bot
- the bot import takes a folder or a zip (files may sit under any top folder)
- python -m collection refresh: the daily workflow, run locally
- trajectories files are named by date and raw checksum, so two imports on
  one day never overwrite each other

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 01:50:12 +00:00
ekcSchianoandClaude Opus 5.5 3b24a11f5a Collection as an ETL pipeline; runs_clean.csv from every source; Report a run
- collection/pipeline.yaml declares every source (extract kind, transform,
  privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
  data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
  before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
  `source` column (reddit spreadsheet, reddit thread, reported, bot,
  reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
  labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
  result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
  collection/specs/report_form.yaml, collected as source "reported", plus a
  site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
  measures.yaml); stats.json and policy.json unchanged apart from the
  factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
  the Actions cache, encrypted with RAW_ARCHIVE_KEY

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 02:46:48 +00:00
ekcSchianoandClaude Opus 5.5 63b5d40e54 Step 5: data dictionary and Croissant metadata
- data/dictionary.yaml: every published file, and every column of runs_clean.csv
  with its type, meaning and allowed values
- collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from
  it, with a sha256 per file; `npm run metadata`, run by the daily update
- tests/core/test_dictionary.py: every column and value described, types hold,
  croissant.json up to date and checksums match
- attempt is now written as a whole number (was 1.0, 2.0); no statistic changes
- the site ships both files and the Methods text links them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 01:37:10 +00:00
11b4a0398e Core: a small JS advisor reads the game from policy.json; engine.js is gone (#13)
Step 4b of the Python-first core.

- core/solver.js is now a 250-line advisor (was 900 lines): it reads the game (core/spec/game.yaml,
  published inside policy.json as model.game) and the count tables, and re-solves from any point in
  a run with the same interpreter as the Python model. Same API for the site and Ken's bot
  (advisor, recommendOption, solverHistory). The bot's on-screen text for each option moves to
  game.yaml (bot_text); the deadly rolls carry their risk labels there too.
- tests/core/advisor.test.js: the advisor gives exactly the Python model's values and choices for
  every colour and goal (to 1e-12, every decision on every published route), walks a run in the
  game's order, and maps advice onto the bot's screen.
- Removed core/engine.js and everything that only compared it with Python (tests/parity/run_engine,
  compare_runs, compare_results, compare_routes, compare_solver, compare_stats, analysis/stats/
  analyze.py). In their place: tests/core/test_stats.py checks the statistics against known answers,
  test_solver.py checks the route model on a synthetic world (tests/fixtures/synthetic_policy.py).
- analysis/network/build-graph.js reads the Python-coded runs; graph.json is byte-identical.
- docs/BOT_INTEGRATION.md: what changed for the bot (history keys, new choices, re-sync all three files).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 22:01:32 +00:00
d04b88e7b4 Site: show precomputed statistics; drop the upload and the click-to-select route map (#12)
Step 4a of the Python-first core.

- analysis/stats/measures.yaml (declarative): outcomes and denominators, route targets and
  comparison groups, encounter contexts, the route map's stages, six filter presets and the
  route thresholds (5, 10, 20, 30).
- analysis/stats/site_stats.py computes every number the page shows, for every preset, and
  writes data/public/stats.json: factor tables, models, chance events, encounters, the route map
  for every outcome, route rankings with q-values at every threshold, the valley map's place and
  step counts, and The Odds' summary cards.
- The page now only displays: app.js, ui.js and explorer.js read stats.json; engine.js is no
  longer loaded. The three filter controls become one "Runs" menu; minimum runs become fixed
  thresholds. Removed: "Load a newer route export", click-to-select and "Follow" on the Deep
  Dive route map, and the unused old valley renderer. The coded runs are a CSV download.
- tests/parity/compare_stats.py: every number matches what engine.js computed, for all six
  presets (factor tables, models, encounters, route rankings at every threshold, route map).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:37:10 +00:00
963b08c522 Core: fix three coding bugs from the core review, with rule tests (#11)
1. Letting the Red pass. Only climbing down quickly puts you above the Red, so a tree run that
   passed on the Red and then met the Greens let it pass. Those 16 runs were coded as slow
   descents; they are now "Descend quickly: let the Red pass" and count for the Greens'
   encounter, so the route model's "let it pass" option rests on its own runs. A tree run with
   both the Red and the Greens attempted (impossible in one run) is now flagged.
2. Leaving your squadmates. The form only offered "> Left them" from 2026-10-27 (0 of 504 squad
   runs before chose it, 40 of 144 after). Earlier and undated squad runs are now unknown rather
   than "stayed together" (form.yaml: left_squadmates_asked_from).
3. Bot text. The wounded cadet's other two choices map onto the form; a test checks every choice
   the bot has logged maps onto a form variable (encounter answers excepted, not modelled yet).

Each rule has a test in tests/core/test_coding.py. Data rebuilt (runs, graph, policy, report);
the headline odds move by at most 0.02 points. The Methods tab describes both rules.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:12:34 +00:00
cc86343668 Core: the route model in Python, built from a declarative game.yaml (#10)
Step 2 of the Python-first, declarative core.

- core/spec/game.yaml: the game as data: every decision and its options, every roll and the
  earlier steps its odds may depend on, every bonding encounter, and the flow of a run from the
  entrance to the dragons (set / keep / drop / go / end / encounter). Also how the form's coded
  answers map onto the game.
- core/atlas/model.py: what each coded run reveals about each step, count tables, empirical-Bayes
  shrinkage, and the posterior draws (same random numbers as before, reproducible from the seed).
- core/atlas/solver.py: one interpreter of game.yaml used three ways (solve, evaluate a fixed
  policy, export the published tree) instead of the tree written out five times. Vectorised
  over the 200 posterior draws: the build takes about 2 minutes instead of 4, with no memory flag.
- planner/build_policy.py and policy_report.py replace the JS build and report.

Checked against the JavaScript model: every number in policy.json agrees to within 4e-12
(last-digit differences in log/cos). Three "still best" shares differ by one draw in 200, at
near-ties under the "eventually" objective. POLICY.md is otherwise identical.
tests/parity/compare_solver.py keeps checking JS against Python in CI until the site's JS advisor
replaces solver.js; tests/core/test_game.py checks game.yaml is complete and consistent.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:04:21 +00:00
3536007ade Core: coding rules as declarative form.yaml + Python package (atlas.coding) (#9)
Step 1 of moving the repo to Python with declarative rules.

- core/spec/form.yaml: every question (how it is found in an export), every answer and what it
  is coded as, missing answers, email redaction, burn order, first-dragon pools.
- core/atlas/coding.py reads it and replaces core/python/clean.py (python -m atlas.coding).
- tests/core/test_coding.py: 16 rule tests, one hand-written row per rule, checking the rules
  are right rather than only that JS and Python agree.
- analysis/stats/analyze.py takes the colours and missing answers from the spec instead of its
  own copy; the daily update job codes rows through the package.
- pyproject.toml for the package; pyyaml and pytest added to requirements.

The coded runs are byte-identical to before (5,988 runs); all parity checks still match.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 15:27:13 -05:00
432e04c2c7 Refactor: lay the repo out by section (core, collection, analysis, planner, site) (#8)
Moves only; no behaviour changes. Every file now lives under the part of the project it
belongs to, each section has a README, and docs/PRIVACY.md collects the rules all parts follow.

  core/        engine.js, graph.js, solver.js, python/clean.py
  collection/  sources/ (form, register, PDF check, daily update), bot/ (importer)
  analysis/    stats/ (Python statistics), network/ (build-graph.js), reports/ (POLICY.md)
  planner/     build-policy.js, policy-report.js
  site/        atlas/ (the page), build.js, qa/
  tests/       core/ (unit), parity/ (JS vs Python), fixtures/

Checked against the previous layout: tests pass; coded runs, test outputs, graph.json and
policy.json are byte-identical; the built site differs only in comments that name file paths.
The published file names Ken's bot downloads (solver.js, graph.js, data/policy.json) are unchanged.

Also removes the undefined/ folder of QA screenshots committed by mistake (the QA scripts now
default to a temp folder) and test.txt, and drops the unused npm workspaces.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 19:46:33 +00:00
ekcSchianoandClaude Opus 5.5 83bc19521e Route model: every logged choice; map keeps your choices on every route
Solver: adds leaving your squadmates, your reaction to the first dragon (options offered per
dragon, from the form's categories) and, below the tree, jumping onto the Red or letting it pass
to the two Greens. Bot crosswalk and option patterns map the game's text for each.

Map: a choice you make now holds for every route to that dragon, together with the earlier steps
of the path you were looking at, so a later choice never sends the route back through another
place. Routes your choices rule out are hidden; choosing an option closes its card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 20:50:59 +00:00
ekcSchianoandClaude Opus 5.5 5f66b9647a Bully fight: add the first move and the dagger choice to the solver and the map
After helping your squadmate the game asks your first move (pause to assess, charge the
smaller cadet, or get between your squadmate and the bigger one) and, after pausing, what to
do about the smaller cadet's dagger. Both are now decisions in the solver, and the chance of
dying in the fight depends on them (about 3 in 10 riders who charged in died; almost none who
paused). With that split, helping now beats walking away.

On the map, several choices made at one place share one diamond, which opens a card listing
each choice in the game's order. Policy rebuilt; the policy build gets a larger heap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 20:09:13 +00:00
ekcSchianoandClaude Opus 5.5 e8c177fc15 Cadet encounter in the game's real order
The solver treated the cadet encounter as one seven-way choice made at the
first dragon, so the map put a diamond there with no matching choices. It is
now a sequence, as in the game: at the scream, investigate or keep walking;
in the clearing, help your squadmate or walk away; then the bullies' fate
(after surviving the fight) or what to do for the wounded cadet. Each
choice point sits at its own place on the map and opens only its own
options; an option that stays in the same place (helping, in the clearing)
appears as a clickable label there. Bot histories that only know the final
resolution still resolve correctly; the bot advisor maps each step onto the
screen text. Policy rebuilt; tests added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 19:07:39 +00:00
ekcSchianoandClaude Opus 5.5 1b0ef9a2c1 Treat unsure and blank answers as missing information, not choices
- engine.js / analyze.py: one MISSING set ("Unknown", "Unsure",
  "Unsure/unknown", "Multiple", "(not recorded)", "(not asked)", ...).
  Factor tables compare only runs that answered each question and report how
  many were left out; models are complete-case for every choice variable;
  route rankings group only fully known routes; the route flow skips
  missing steps.
- "I've lost count" now counts as a later attempt.
- The chase goal keeps its "not asked" stratum as an adjustment only; it is
  no longer displayed as a finding.
- Page: no unknown levels in charts, tables or filters; factor explorer notes
  how many runs were left out; Methods explains the rule.
JS/Python parity still exact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 16:14:08 +00:00
ekcSchianoandClaude Opus 5.5 de80e99c74 The Valley as the landing page: dragons first, routes on demand
- New tab order: The Valley, The Odds, The Dragons, Deep Dive, Data & Methods
  (Route Flow and Blue & Black fold into Deep Dive; methods into Data &
  Methods; old #links redirect).
- explorer.js replaces the trails map and the route planner, with progressive
  disclosure: the painted valley shows landmarks and the dragons where riders
  meet them (faint = rarely met), no routes and no deaths; hover for a note;
  choose a dragon to draw every route to every place it can be bonded, with
  width = chance of still being on the route, and solid / dashed / dotted and
  opacity for how a route compares with the best; click a choice point to
  branch off and re-plan; "Why this route?" holds intervals and risks; a
  toggle shows the community's actual traffic; #valley/blue/meadow-Blue links.
- dragons.js: original dragon silhouettes in six poses, built from simple
  parts.
- Solver: route to one specific encounter (only: [...]); ENCOUNTERS per
  colour, including the Greens rescue for Orange and Brown.
- The Odds rewritten for all six dragons, ranked easiest to hardest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 14:24:47 +00:00
ekcSchianoandClaude Opus 5.5 d2ffb84b5e Solver: best decisions per dragon colour, and the bot advisor
- packages/core/src/solver.js: decision tree with perfect recall over the
  community form's choices; chance events (first dragon, deaths, shadow,
  meadow dragon, encounter outcome) depend on earlier choices through
  hierarchical Dirichlet back-off with empirical-Bayes concentration per
  event; backward induction for P(colour this attempt) and Dinkelbach for
  P(colour) / (P(colour) + P(other colour)); 200 posterior draws for
  credible intervals and "still best" rates; cross-fitted held-out check.
- Only genuine attempts inform encounter outcomes (goal / Malek intent).
- data/public/policy.json (all runs and a low-bias sample) and the readable
  docs/POLICY.md; published on Pages with solver.js for the bot.
- Bot advisor: re-solves from any history, maps advice onto screen text,
  reports margins so ties can be randomised. docs/BOT_INTEGRATION.md.
- Tests on a synthetic world with known answers; daily source update now
  rebuilds the policy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-01 22:34:38 +00:00
ekcSchianoandClaude Opus 5.5 46ee08a56b Coding rules for the redesigned community form; import the new data
The Threshing Routes form was redesigned (5,651 responses). Both implementations
(engine.js and clean.py) now read either version:
- one direction question + one colour question, with the older per-direction
  columns used only to fill gaps and cross-check
- cadet encounter as an action path (scream, first and second combat move)
  plus a separate outcome
- new fields: sighting_action, combat_second, rare_reported, malek_intent,
  left_squadmates, tail; "Why are you doing this run?" mapped onto the goal
  ("Blue or Black" for rare chasers)
- the rare-dragon question feeds saw_blue_meadow / saw_shadow and the branch
The old export still codes identically. JS/Python parity: all match.

Data: form 2,573 -> 5,651 rows, Dragon Register 216 -> 273, new route PDF
fingerprint. Supersedes the automated pull request #2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-01 22:10:51 +00:00
ekcSchianoandClaude Opus 5.5 c3d067b88c Threshing Atlas: monorepo, route graph, bot importer, daily source checks
- apps/atlas: the web page (valley map, route flow, Dragon Register)
- packages/core: coding rules and statistics (engine.js) and the shared route
  graph (graph.js): scene graph, decision-variable graph, bot-to-form crosswalk,
  Beta credible intervals
- packages/ingest: DragonKind bot importer to anonymised trajectories (no prose,
  no account identifiers); data/bot holds Ken's sample
- research/python: independent reference implementation + update_sources.py
- tests: JS/Python parity checks and graph/importer unit tests
- CI, GitHub Pages deploy, daily check of the community sheets and route PDF
- Email addresses in free-text form answers are redacted; the raw spreadsheet
  is not committed
- MIT code licence, CC BY 4.0 data licence

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-01 21:54:54 +00:00