- collection/pipeline.yaml declares every source (extract kind, transform, privacy rules, load target, run source); `python -m collection update|build` - extract keeps data/raw/<source>/current + previous only (git-ignored); data/manifest.json records the current raw checksum and change date - transforms are pure; validate checks runs against data/dictionary.yaml before load; load writes byte-stable files only when they change - data/public/runs_clean.csv: every run from every source with a new `source` column (reddit spreadsheet, reddit thread, reported, bot, reddit pdf); the sheet's own row label moves to `entry`; run_id is text - bot importer ported to Python (same output as bot.js on the fixture); labels and crosswalk in collection/specs/bot.yaml, written into graph.js - Reddit through the official API (result links to add by hand); the 12 result pages move to collection/manual/reddit_results.yaml - "Report a run": a GitHub issue form generated from collection/specs/report_form.yaml, collected as source "reported", plus a site tab pointing to it, the community spreadsheet and the megathread - analysis and the route model read runs_clean.csv (sources in measures.yaml); stats.json and policy.json unchanged apart from the factor's new name - daily workflow runs the pipeline; the raw store is kept between runs in the Actions cache, encrypted with RAW_ARCHIVE_KEY Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
core
The shared model. Every other part uses it, so run npm test after any change.
The rules and the game are declarative files in spec/; the Python package atlas/ reads them. JavaScript is
used only where it has to run in a browser or in Ken's bot.
| File | What it does |
|---|---|
spec/form.yaml |
Every question of the community form (how to find it in an export) and every answer (what it is coded as), the answers that count as missing, email redaction, when "left your squadmates" was first asked. |
spec/game.yaml |
The game: every decision and its options in order, every roll and what its odds may depend on, every bonding encounter, the flow of a run from the entrance to the dragons, how form answers map onto it, and the text the bot sees for each option. |
atlas/coding.py |
Codes every form row into a run (python -m atlas.coding). Rule tests: tests/core/test_coding.py. |
atlas/model.py, atlas/solver.py |
The route model: what each coded run reveals about each step, the count tables and shrinkage, the posterior draws, and one interpreter of game.yaml used to solve, evaluate and export the best routes (planner/build_policy.py). Tests: tests/core/test_solver.py, test_game.py. |
solver.js |
JavaScript, for the site and the bot. The route advisor: reads the game and the count tables from data/policy.json and re-solves from any point in a run. tests/core/advisor.test.js checks it gives exactly the Python model's values and choices. |
graph.js |
JavaScript, for the bot. Turns the bot's on-screen choices into form answers, and builds the route graph (analysis/network/build-graph.js). Moves to Python with the collection review. |
The site and Ken's bot load the published copies (https://ekcschiano.github.io/threshing-atlas/solver.js,
graph.js, data/policy.json), so those names on the site must not change.
To support a new form answer, add it to spec/form.yaml. To change a rule, edit atlas/coding.py and add a test case.
To add a choice to the game, add it to spec/game.yaml (and, if the form records it, to atlas/model.py).