Files
threshing-atlas/core
ekcSchianoandClaude Opus 5.5 3b24a11f5a Collection as an ETL pipeline; runs_clean.csv from every source; Report a run
- collection/pipeline.yaml declares every source (extract kind, transform,
  privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
  data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
  before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
  `source` column (reddit spreadsheet, reddit thread, reported, bot,
  reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
  labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
  result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
  collection/specs/report_form.yaml, collected as source "reported", plus a
  site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
  measures.yaml); stats.json and policy.json unchanged apart from the
  factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
  the Actions cache, encrypted with RAW_ARCHIVE_KEY

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 02:46:48 +00:00
..

core

The shared model. Every other part uses it, so run npm test after any change.

The rules and the game are declarative files in spec/; the Python package atlas/ reads them. JavaScript is used only where it has to run in a browser or in Ken's bot.

File What it does
spec/form.yaml Every question of the community form (how to find it in an export) and every answer (what it is coded as), the answers that count as missing, email redaction, when "left your squadmates" was first asked.
spec/game.yaml The game: every decision and its options in order, every roll and what its odds may depend on, every bonding encounter, the flow of a run from the entrance to the dragons, how form answers map onto it, and the text the bot sees for each option.
atlas/coding.py Codes every form row into a run (python -m atlas.coding). Rule tests: tests/core/test_coding.py.
atlas/model.py, atlas/solver.py The route model: what each coded run reveals about each step, the count tables and shrinkage, the posterior draws, and one interpreter of game.yaml used to solve, evaluate and export the best routes (planner/build_policy.py). Tests: tests/core/test_solver.py, test_game.py.
solver.js JavaScript, for the site and the bot. The route advisor: reads the game and the count tables from data/policy.json and re-solves from any point in a run. tests/core/advisor.test.js checks it gives exactly the Python model's values and choices.
graph.js JavaScript, for the bot. Turns the bot's on-screen choices into form answers, and builds the route graph (analysis/network/build-graph.js). Moves to Python with the collection review.

The site and Ken's bot load the published copies (https://ekcschiano.github.io/threshing-atlas/solver.js, graph.js, data/policy.json), so those names on the site must not change.

To support a new form answer, add it to spec/form.yaml. To change a rule, edit atlas/coding.py and add a test case. To add a choice to the game, add it to spec/game.yaml (and, if the form records it, to atlas/model.py).