Commit Graph
56 Commits
Author SHA1 Message Date
kathleen 6b4b1344be New data from the community sources (#52)
Publish the site (pages branch) / publish (push) Successful in 6s
CI / test (push) Successful in 46s
Send the route model to the bot / send (push) Successful in 1m14s
2026-11-20 10:29:09 +00:00
kathleenandClaude Opus 5.5 ee222dd775 Black report: Reddit comments (and reported runs) in the attempt table (#25)
CI / test (pull_request) Successful in 42s
- analysis/black/evidence.py: Reddit comments and "Report a run" are their own groups. The attempt table gets a
  column for each once it has a gated tree run (Reddit: 4 runs, 3 with a shadow; reported: none yet). A note says
  Reddit describes 17 shadows but only 3 say both gates and 3 the attempt, and that, like form posts, they show
  what shadow runs look like, not how often the shadow comes. black.json: by_attempt["Reddit comments"],
  reddit_shadows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 20:47:00 +00:00
ekcSchianoandClaude Opus 5.5 1dfa8fb4c3 Stable run ids (fingerprints) for the form and the register; tests for the site statistics (#25)
CI / test (pull_request) Successful in 43s
- collection/etl/runs.py fingerprints(): a form or register run's id is now <prefix>-<hash of the row's non-empty
  answers, timestamp included> instead of its row number, so deleting or re-sorting rows in the sheet moves no
  other run's id, and a column added later changes none. The 32 rows that are exact copies of another row get
  -2, -3 (identical, so which is which does not matter). A row edited in the sheet gets a new id. Every id in
  runs_clean.csv and overlap.json changes once; nothing else does (no statistic or model reads the ids).
- tests/core/test_site_stats.py (9 tests): measures.yaml conditions, source groups and presets, bond
  denominators without deliberate deaths (cards, encounters, route map), the riders' paths, and a JSON-safe
  stats.json, on a fixed sample of the published runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 20:16:16 +00:00
kathleenandClaude Opus 5.5 b51a6c7039 The Black on the page: a first-attempt range marked low evidence; the Black panels load again
CI / test (pull_request) Successful in 40s
- The odds card for the Black: "Low evidence", "about 1 in N" first attempts, the 95% range written as
  "between 1 in X and 1 in Y", later attempts effectively never, why it is uncertain (every logged shadow is a
  rider who chose to post), and the steps (shadow -> reached the Black -> bonded). Dashed border.
- Valley map: the Black's routes are solved for a first attempt (they showed ~0% per attempt), labelled "on a first
  attempt", and "Why this route?" gives the same range and caveat.
- "The Black is not dice" panel: why the odds are uncertain, and what replaces the assumption (fresh-account bot
  runs to the tree).
- analysis/black/evidence.py wrote "odds_ratio": Infinity (no later-attempt shadow at all), which is not JSON:
  the page's two Black panels showed "could not be loaded". Now null, and the report says "unbounded".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 18:40:01 +00:00
ekcSchianoandClaude Opus 5.5 98be27fc1f Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s
- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
  on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
  runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
  be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
  to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
  form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
  match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
  page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
  stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:44:52 +00:00
ekcSchianoandClaude Opus 5.5 46c1f00150 Impossible runs: one definition, left out of the route model and the site statistics (#21, #22)
CI / test (pull_request) Successful in 41s
- core/spec/form.yaml impossible_flags: the flags that mean a run cannot have happened as recorded (a dragon an
  encounter does not have, a Green or the Black in the valley, two bonds, two burns, a cadet path that does not
  follow, an early death that still reached the hour...). atlas.runs.impossible() reads it.
- The route model and the site statistics (and the riders' paths) leave those runs out, from every source; every
  other run is used, however partial. The runs stay in runs_clean.csv, flagged. 80 runs: 75 form, 5 Reddit.
  Route-model odds move by at most 0.2 points.
- collection/etl/runs.py: the community form gets the same game-rule checks as "Report a run" and the Reddit
  comments (10 form rows newly flagged); new rule: the Black in the valley.
- The first-sighting pool stays a flag only: it is what riders have seen, not a rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:03:45 +00:00
ekcSchianoandClaude Opus 5.5 036c86a1d1 Site statistics: the bot's runs, every source labelled as itself, chosen deaths never counted as failed bonds
CI / test (pull_request) Successful in 39s
- analysis/stats/measures.yaml: the bot is a statistics source. A deliberate death (`_deliberate`, the route
  model's rule: burned at an encounter the rider was not attempting) is left out of every bond rate, encounter
  rate and route ranking, for every source, as the model and the summary cards already did. Biggest moves (all
  runs): two Greens 8.9% -> 29.1%, meadow Orange 9.2% -> 25.2%, meadow Brown 14.3% -> 28.3%, valley Red 79.2% ->
  93.3%; mostly from form riders who burned on purpose, not from the bot.
- analysis/stats/site_stats.py: runs that are not from the spreadsheet are grouped as their own source
  ("Reddit comments", "DragonKind bot", "Reported on the Atlas"), not as bulk logs; two new presets (Reddit
  comments only, DragonKind bot only); the route map's bond denominators skip deliberate deaths.
- site/atlas/app.js: the bulk-import share counts only spreadsheet imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 14:58:17 +00:00
ekcSchianoandClaude Opus 5.5 d606ed1651 Reddit comment runs in the route model and the odds; Black report gate wording; cite both megathreads
- planner/policy.yaml: `reddit comment` is a model source. Only runs that break a game rule (the rule_flags
  checks) are left out; partial runs are kept. Its own reporting-bias group, and out of the low-bias sample
  (riders posting a run pick the notable ones). 77 of the 82 runs are used.
- analysis/stats/measures.yaml: the site statistics use every source answered in the form's terms (form,
  "Report a run", Reddit comments).
- analysis/black/evidence.py: a shadow run is "off the gates" only when an answer contradicts a gate (3); runs
  that do not say are counted separately (14).
- croissant isBasedOn lists both megathreads (pipeline.yaml `cite`).
- Rebuilt: policy.json, stats.json, BLACK.md, black.json, croissant.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 14:50:08 +00:00
ekcSchiano 75f11378e2 data: keep main's route model (Reddit comment runs are not a model source yet)
CI / test (pull_request) Successful in 2m13s
2026-11-19 09:45:17 -05:00
kathleenandClaude Opus 5.5 b82ef16a9f data: croissant checksum for the rebuilt runs_clean.csv
CI / test (pull_request) Successful in 2m2s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 14:14:50 +00:00
kathleenandClaude Opus 5.5 e30825ec84 data: rebuild the Reddit comment runs with the game-rule checks from #37
CI / test (pull_request) Failing after 9s
6 of the 82 runs now carry rule flags (Brown/Red at the shadow, Green after
jumping on the Red, Red in the meadow, Orange first for Alone/Center, fight
answers on an ignored scream). Only their data_flags change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 14:14:16 +00:00
kathleen 93dab0b963 Merge branch 'main' into data/reddit-comments
CI / test (pull_request) Failing after 1m22s
2026-11-19 14:10:13 +00:00
Atlas Reddit reader 7b0a01efb0 data: runs from the Reddit megathread comments
CI / test (pull_request) Successful in 39s
2026-11-19 08:57:59 -05:00
ekcSchianoandClaude Opus 5.5 e10312ab02 Known answers for the bot: the guide's encounter answers and the bot's own results, by choice text (#35)
CI / test (pull_request) Successful in 2m3s
analysis/known/build_known.py writes data/public/known_answers.json from pdf_encounters.json and tree.json;
solver.js knownAnswer(known, offered, { target, sceneColour }) matches a screen's choices against it (the guide's
paraphrases by their words, the bot's texts exactly) and returns a pick when one choice is clearly on the guide's
recommended path to the target, plus the evidence for every choice to hand to Clef. Rebuilt daily, checked in CI,
and sent to the bot with the route model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 13:32:09 +00:00
ekcSchianoandClaude Opus 5.5 cd79dc8c5a Valley map: the answers inside each encounter, from the community guide and the bot
CI / test (pull_request) Successful in 39s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 13:17:24 +00:00
kathleen c943f5a289 New data from the community sources (#33)
Send the route model to the bot / send (push) Successful in 7s
CI / test (push) Successful in 1m59s
CI / test (pull_request) Successful in 1m58s
2026-11-19 10:31:00 +00:00
kathleen 986cdd9c0a New data from the community sources (#30)
CI / test (push) Successful in 43s
Send the route model to the bot / send (push) Successful in 1m18s
2026-11-19 03:36:53 +00:00
kschianoandClaude Opus 5.5 197c16dac0 Receive runs from the Reddit megathread comments (source "reddit comment")
CI / test (pull_request) Successful in 46s
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs
riders describe in the r/fourthwing megathreads and delivers them here by
pull request as data/public/reddit_comments.json: each run is the "Report a
run" answers (option labels of collection/specs/report_form.yaml), with the
comment id and day only; no usernames, no comment text.

- pipeline.yaml: a reddit_comments source of extract kind "delivered" (the
  atlas fetches nothing; `update` skips it with a note)
- runs.py: the coder, sharing one answers_row() with the reported coder;
  every row is flagged as read from a Reddit comment (and "not checked by
  hand" until corrected); a result page a comment already carries is not
  counted twice as a "reddit thread" row
- collection/manual/reddit_comments.yaml: reject / fix / add, applied at
  build time
- dictionary, SOURCES, README, croissant: the new source

The route model and statistics do not use the source until it is named in
planner/policy.yaml / analysis/stats/measures.yaml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-18 22:06:14 -05:00
kathleen f3bbde24b7 New data from the community sources (#17)
Send the route model to the bot / send (push) Successful in 7s
CI / test (push) Successful in 39s
2026-11-19 02:55:25 +00:00
kathleenandClaude Opus 5.5 c7978b4538 The Atlas now lives in the DragonKind organization
CI / test (pull_request) Successful in 3m36s
forge.yaml: repo DragonKind/threshing-atlas, and the links generated from it (data/public/links.json,
data/croissant.json); the project notes too. Old kathleen/threshing-atlas links redirect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 02:42:19 +00:00
kathleen 574fa50b2a New data from the community sources (#13)
Send the route model to the bot / send (push) Failing after 29s
CI / test (push) Successful in 41s
2026-11-19 02:31:20 +00:00
kathleen f1027212b6 New data from the community sources (#11)
CI / test (push) Successful in 39s
2026-11-18 10:28:08 +00:00
kathleen f41bd22439 New data from the community sources (#10)
CI / test (push) Successful in 2m15s
2026-11-18 00:00:18 +00:00
kathleenandClaude Opus 5.5 d301e71d79 croissant.json: the new entry value (follow-up to #8)
CI / test (pull_request) Successful in 2m6s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-17 23:50:54 +00:00
kathleenandClaude Opus 5.5 47da9f1f82 Code the sheet's new answers; a failed build opens an issue instead of crashing the rebuild
The evening source check stopped (correctly) on two answers the coding rules did not know:
- first sighting "2 Greens": a new form option. Coded as Green, with a data flag that keeps the
  original answer (first_sighting_alias in core/spec/form.yaml).
- row label "Unknown" on a bulk-pasted row: added to the entry values in data/dictionary.yaml.

Workflow: when `collection update` reports the build failed, skip the rebuild (it would only fail
again) and always open the "Source check failed" issue, so new answers are reported, not lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-17 23:48:47 +00:00
kathleen 3e21fd8992 New data from the community sources (#5)
CI / test (push) Successful in 2m28s
2026-11-17 03:23:52 +00:00
ekcSchianoandClaude Opus 5.5 73a6f5d355 Site repairs: first-attempt Black, shadow evidence, author's statements
CI / test (pull_request) Successful in 2m12s
- analysis/black/evidence.py also writes data/public/black.json (--check covers both)
- Odds: the Black card shows first-attempt odds; new "The Black is not dice" panel from black.json
- "Earlier choices barely change who waits" card replaced by the first-attempt finding
- Journey eyebrow and bonding step no longer call the rolls pure luck
- Footer, lore line and Reddit post: the author's statements (no roll worse than 1 in 20,
  the Black is not luck) replace the "10% chaos" claim
- Report tab: the community Google Form first (no account needed), the Gitea form second;
  register listed as a source
- register.js: snapshot label reads raw.source (was undefined)
- build.js copies black.json, tree.json and experiments.json

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-16 18:22:21 +00:00
ekcSchianoandClaude Opus 5.5 3b5c655d51 Refresh to 8,748 spreadsheet runs; squad answers ignore case; rebuild checks the runs first
CI / test (pull_request) Successful in 1m59s
- spreadsheet 8,132 -> 8,748 rows (about 600 new; the sheet's keeper fixed 'Sqaudmate' in 72 answers,
  corrected 15 attempt numbers to 'First Attempt' and re-pasted the daily bulk logs); register 288 -> 292
- form.yaml squad_any_case: a typed 'second' is Second (it stopped the build)
- collection rebuild: the runs are built and checked first, so a failed check stops it before the model,
  plan, reports and statistics are rebuilt from stale runs
- route model, plan, Black evidence, screen tree, statistics, graph and metadata rebuilt (9,473 runs)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-16 17:13:23 +00:00
ekcSchianoandClaude Opus 5.5 93bdfcd4da One rebuild list for the refresh, the bot cycle and the daily workflows; graph.json rebuilt
CI / test (pull_request) Canceled after 0s
- python -m collection rebuild: graph, route model, experiment plan, the Black evidence, the screen
  tree, statistics, build. refresh, cycle and both update-sources workflows use it, so the morning
  update also rebuilds the reports the tests check (it rebuilt only graph, model and stats before)
- graph.json had not been rebuilt since the played-steps fix (no Node on the Mac)
- docs: the branch is pushed; README mentions rebuild

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-16 16:27:20 +00:00
ekcSchianoandClaude Fable 5.2 f8e4f575f4 Screen tree, register rows in the master list, and bot runs coded from the picks that were played
CI / test (pull_request) Canceled after 0s
- analysis/tree/build_tree.py -> data/public/tree.json, analysis/reports/TREE.md: every screen the bot has seen,
  every option, where each led (fixed or dice, with ranges), best known picks per colour chosen cautiously, where
  any picks end, and the untested options. solver.js screenPick(): least-tried and Thompson picks from the tree.
- The bot logs a pick it then overrode as a step: 37 runs were coded with choices they never made. Importer and
  graph.js now use the played steps only.
- runs_clean.csv: the Unofficial Dragon Register's 288 bonded dragons are in the master list as outcome-only
  rows, flagged (no analysis uses them).
- Cycle, refresh and the bot kit rebuild and carry the tree; tests check every generated report is current.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-16 15:50:12 +00:00
ekcSchianoandClaude Fable 5.2 d01ec46fcc The Black is not dice: developer statements, evidence report, and a plan that looks for the rule
- planner/knowledge.yaml: what the game's developer has said (paraphrased, dated, graded), what the Atlas does
  with each statement, and the open questions with the experiment that tests each.
- analysis/black/evidence.py -> analysis/reports/BLACK.md: counts from the runs. On the same recorded route the
  shadow shows on 21 of 79 first attempts and 1 of 80 later ones; no single route; Blue then Black after the shadow.
- game.yaml: the shadow is a guard (not a roll); what follows it is the Blue, then the Black; on-screen text for
  the squad's direction screen.
- planner/experiments.yaml v2: one exact path on every fresh account (fresh-replicate), then one choice changed
  at a time (fresh-ablate); shadow-again, second-attempt-same-path, veteran-repeat, veteran-variety; what the bot
  records about the account; the sequence to follow once the shadow shows.
- solver.js: assign() takes account facts, account offsets and cycled arms; recommendOption() answers the
  squad screen (stay + direction) in one pick.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-16 15:24:03 +00:00
ekcSchianoandClaude Fable 5.2 51842e1aa9 Host: the repository is kathleen/threshing-atlas on git.kenschiano.net
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-12 03:35:03 +00:00
ekcSchianoandClaude Fable 5.2 ab3664f369 Route model: first attempts are their own shadow roll; experiment plan and automated bot cycle
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
  depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
  so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
  0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
  experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-12 03:31:33 +00:00
ekcSchianoandClaude Opus 5.5 4837332a7d Refresh to 8,132 spreadsheet runs; rebuild generated files for the Gitea links
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-12 02:46:31 +00:00
ekcSchianoandClaude Opus 5.5 614bfcda7c Host on Gitea: forge.yaml, Gitea workflows, host-neutral links
- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
  the "Report a run" collector, the page's links (data/public/links.json, filled in by
  site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
  for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
  merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-12 02:46:17 +00:00
ekcSchianoandClaude Opus 5.5 8713ccff1a Refresh to 7,775 spreadsheet runs; route model learns from the bot and corrects for reporting bias
- spreadsheet 5,988 -> 7,775 runs; new route PDF (Black Dragon Gates); register unchanged
- form.yaml: 'Protect / Help Squadmate' -> Protect squadmate
- policy, statistics, graph, metadata rebuilt

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-12 01:31:11 +00:00
ekcSchianoandClaude Opus 5.5 2271fe0976 Route model: correct for reporting bias
- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group,
  the factor by which the group over-reports the rare outcome relative to complete
  logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count
  1/rho; a roll the complete logs have (almost) never shown is learned from them only
- planner/policy.yaml: groups, rolls and thresholds; estimates published in
  policy.json under model.reportingBias
- fit_alpha: a roll seen only one way is pooled fully
- docs: goal accounts follow the best route, research accounts use Thompson sampling
  (decisionMode atlas_thompson)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:29:09 +00:00
ekcSchianoandClaude Opus 5.5 d37eaa3378 Import Ken's bot runs (13 accounts, 415 attempts, to 2026-11-10)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:09:26 +00:00
ekcSchianoandClaude Opus 5.5 43066ad67f Bot import: one folder per account, run records with steps
- the importer reads data/accounts/*/runs.jsonl and logs/accounts/*/encounters.jsonl;
  with account folders, the top-level files (a mirror of the first account) are skipped
- run records that carry their steps (schema 2) become trajectories directly, keeping
  what the bot saw (shadow, meadow dragons), propensities, decision mode and policy id;
  older runs are still rebuilt from the decision log; bonded runs without steps are kept
- account names become salted hashes (DK_ACCOUNT_SALT, or a private salt in data/raw)
- runs_clean.csv: new bot_mode column; saw_shadow and saw_blue_meadow from what the bot
  saw; deliberate burns say why
- crosswalk: the valley Red, leaving squadmates, squad direction, hold your ground,
  hide/approach reactions, the wounded cadet's other two choices

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 02:08:57 +00:00
ekcSchianoandClaude Opus 5.5 dd25dfaa32 Record the local bot data as imported (same 6 runs as the sample)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-10 01:52:46 +00:00
ekcSchianoandClaude Opus 5.5 3b24a11f5a Collection as an ETL pipeline; runs_clean.csv from every source; Report a run
- collection/pipeline.yaml declares every source (extract kind, transform,
  privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
  data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
  before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
  `source` column (reddit spreadsheet, reddit thread, reported, bot,
  reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
  labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
  result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
  collection/specs/report_form.yaml, collected as source "reported", plus a
  site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
  measures.yaml); stats.json and policy.json unchanged apart from the
  factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
  the Actions cache, encrypted with RAW_ARCHIVE_KEY

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 02:46:48 +00:00
ekcSchianoandClaude Opus 5.5 63b5d40e54 Step 5: data dictionary and Croissant metadata
- data/dictionary.yaml: every published file, and every column of runs_clean.csv
  with its type, meaning and allowed values
- collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from
  it, with a sha256 per file; `npm run metadata`, run by the daily update
- tests/core/test_dictionary.py: every column and value described, types hold,
  croissant.json up to date and checksums match
- attempt is now written as a whole number (was 1.0, 2.0); no statistic changes
- the site ships both files and the Methods text links them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 01:37:10 +00:00
11b4a0398e Core: a small JS advisor reads the game from policy.json; engine.js is gone (#13)
Step 4b of the Python-first core.

- core/solver.js is now a 250-line advisor (was 900 lines): it reads the game (core/spec/game.yaml,
  published inside policy.json as model.game) and the count tables, and re-solves from any point in
  a run with the same interpreter as the Python model. Same API for the site and Ken's bot
  (advisor, recommendOption, solverHistory). The bot's on-screen text for each option moves to
  game.yaml (bot_text); the deadly rolls carry their risk labels there too.
- tests/core/advisor.test.js: the advisor gives exactly the Python model's values and choices for
  every colour and goal (to 1e-12, every decision on every published route), walks a run in the
  game's order, and maps advice onto the bot's screen.
- Removed core/engine.js and everything that only compared it with Python (tests/parity/run_engine,
  compare_runs, compare_results, compare_routes, compare_solver, compare_stats, analysis/stats/
  analyze.py). In their place: tests/core/test_stats.py checks the statistics against known answers,
  test_solver.py checks the route model on a synthetic world (tests/fixtures/synthetic_policy.py).
- analysis/network/build-graph.js reads the Python-coded runs; graph.json is byte-identical.
- docs/BOT_INTEGRATION.md: what changed for the bot (history keys, new choices, re-sync all three files).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 22:01:32 +00:00
d04b88e7b4 Site: show precomputed statistics; drop the upload and the click-to-select route map (#12)
Step 4a of the Python-first core.

- analysis/stats/measures.yaml (declarative): outcomes and denominators, route targets and
  comparison groups, encounter contexts, the route map's stages, six filter presets and the
  route thresholds (5, 10, 20, 30).
- analysis/stats/site_stats.py computes every number the page shows, for every preset, and
  writes data/public/stats.json: factor tables, models, chance events, encounters, the route map
  for every outcome, route rankings with q-values at every threshold, the valley map's place and
  step counts, and The Odds' summary cards.
- The page now only displays: app.js, ui.js and explorer.js read stats.json; engine.js is no
  longer loaded. The three filter controls become one "Runs" menu; minimum runs become fixed
  thresholds. Removed: "Load a newer route export", click-to-select and "Follow" on the Deep
  Dive route map, and the unused old valley renderer. The coded runs are a CSV download.
- tests/parity/compare_stats.py: every number matches what engine.js computed, for all six
  presets (factor tables, models, encounters, route rankings at every threshold, route map).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:37:10 +00:00
963b08c522 Core: fix three coding bugs from the core review, with rule tests (#11)
1. Letting the Red pass. Only climbing down quickly puts you above the Red, so a tree run that
   passed on the Red and then met the Greens let it pass. Those 16 runs were coded as slow
   descents; they are now "Descend quickly: let the Red pass" and count for the Greens'
   encounter, so the route model's "let it pass" option rests on its own runs. A tree run with
   both the Red and the Greens attempted (impossible in one run) is now flagged.
2. Leaving your squadmates. The form only offered "> Left them" from 2026-10-27 (0 of 504 squad
   runs before chose it, 40 of 144 after). Earlier and undated squad runs are now unknown rather
   than "stayed together" (form.yaml: left_squadmates_asked_from).
3. Bot text. The wounded cadet's other two choices map onto the form; a test checks every choice
   the bot has logged maps onto a form variable (encounter answers excepted, not modelled yet).

Each rule has a test in tests/core/test_coding.py. Data rebuilt (runs, graph, policy, report);
the headline odds move by at most 0.02 points. The Methods tab describes both rules.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:12:34 +00:00
cc86343668 Core: the route model in Python, built from a declarative game.yaml (#10)
Step 2 of the Python-first, declarative core.

- core/spec/game.yaml: the game as data: every decision and its options, every roll and the
  earlier steps its odds may depend on, every bonding encounter, and the flow of a run from the
  entrance to the dragons (set / keep / drop / go / end / encounter). Also how the form's coded
  answers map onto the game.
- core/atlas/model.py: what each coded run reveals about each step, count tables, empirical-Bayes
  shrinkage, and the posterior draws (same random numbers as before, reproducible from the seed).
- core/atlas/solver.py: one interpreter of game.yaml used three ways (solve, evaluate a fixed
  policy, export the published tree) instead of the tree written out five times. Vectorised
  over the 200 posterior draws: the build takes about 2 minutes instead of 4, with no memory flag.
- planner/build_policy.py and policy_report.py replace the JS build and report.

Checked against the JavaScript model: every number in policy.json agrees to within 4e-12
(last-digit differences in log/cos). Three "still best" shares differ by one draw in 200, at
near-ties under the "eventually" objective. POLICY.md is otherwise identical.
tests/parity/compare_solver.py keeps checking JS against Python in CI until the site's JS advisor
replaces solver.js; tests/core/test_game.py checks game.yaml is complete and consistent.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:04:21 +00:00
432e04c2c7 Refactor: lay the repo out by section (core, collection, analysis, planner, site) (#8)
Moves only; no behaviour changes. Every file now lives under the part of the project it
belongs to, each section has a README, and docs/PRIVACY.md collects the rules all parts follow.

  core/        engine.js, graph.js, solver.js, python/clean.py
  collection/  sources/ (form, register, PDF check, daily update), bot/ (importer)
  analysis/    stats/ (Python statistics), network/ (build-graph.js), reports/ (POLICY.md)
  planner/     build-policy.js, policy-report.js
  site/        atlas/ (the page), build.js, qa/
  tests/       core/ (unit), parity/ (JS vs Python), fixtures/

Checked against the previous layout: tests pass; coded runs, test outputs, graph.json and
policy.json are byte-identical; the built site differs only in comments that name file paths.
The published file names Ken's bot downloads (solver.js, graph.js, data/policy.json) are unchanged.

Also removes the undefined/ folder of QA screenshots committed by mistake (the QA scripts now
default to a temp folder) and test.txt, and drops the unused npm workspaces.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 19:46:33 +00:00
ekcSchianoandgithub-actions[bot] df103ae1fa data: update from community sources 2026-11-03 11:21:28 +00:00
ekcSchianoandClaude Opus 5.5 83bc19521e Route model: every logged choice; map keeps your choices on every route
Solver: adds leaving your squadmates, your reaction to the first dragon (options offered per
dragon, from the form's categories) and, below the tree, jumping onto the Red or letting it pass
to the two Greens. Bot crosswalk and option patterns map the game's text for each.

Map: a choice you make now holds for every route to that dragon, together with the earlier steps
of the path you were looking at, so a later choice never sends the route back through another
place. Routes your choices rule out are hidden; choosing an option closes its card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 20:50:59 +00:00
ekcSchianoandClaude Opus 5.5 5f66b9647a Bully fight: add the first move and the dagger choice to the solver and the map
After helping your squadmate the game asks your first move (pause to assess, charge the
smaller cadet, or get between your squadmate and the bigger one) and, after pausing, what to
do about the smaller cadet's dagger. Both are now decisions in the solver, and the chance of
dying in the fight depends on them (about 3 in 10 riders who charged in died; almost none who
paused). With that split, helping now beats walking away.

On the map, several choices made at one place share one diamond, which opens a card listing
each choice in the game's order. Policy rebuilt; the policy build gets a larger heap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-02 20:09:13 +00:00