- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- core/spec/form.yaml impossible_flags: the flags that mean a run cannot have happened as recorded (a dragon an
encounter does not have, a Green or the Black in the valley, two bonds, two burns, a cadet path that does not
follow, an early death that still reached the hour...). atlas.runs.impossible() reads it.
- The route model and the site statistics (and the riders' paths) leave those runs out, from every source; every
other run is used, however partial. The runs stay in runs_clean.csv, flagged. 80 runs: 75 form, 5 Reddit.
Route-model odds move by at most 0.2 points.
- collection/etl/runs.py: the community form gets the same game-rule checks as "Report a run" and the Reddit
comments (10 form rows newly flagged); new rule: the Black in the valley.
- The first-sighting pool stays a flag only: it is what riders have seen, not a rule.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- analysis/stats/measures.yaml: the bot is a statistics source. A deliberate death (`_deliberate`, the route
model's rule: burned at an encounter the rider was not attempting) is left out of every bond rate, encounter
rate and route ranking, for every source, as the model and the summary cards already did. Biggest moves (all
runs): two Greens 8.9% -> 29.1%, meadow Orange 9.2% -> 25.2%, meadow Brown 14.3% -> 28.3%, valley Red 79.2% ->
93.3%; mostly from form riders who burned on purpose, not from the bot.
- analysis/stats/site_stats.py: runs that are not from the spreadsheet are grouped as their own source
("Reddit comments", "DragonKind bot", "Reported on the Atlas"), not as bulk logs; two new presets (Reddit
comments only, DragonKind bot only); the route map's bond denominators skip deliberate deaths.
- site/atlas/app.js: the bulk-import share counts only spreadsheet imports.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- planner/policy.yaml: `reddit comment` is a model source. Only runs that break a game rule (the rule_flags
checks) are left out; partial runs are kept. Its own reporting-bias group, and out of the low-bias sample
(riders posting a run pick the notable ones). 77 of the 82 runs are used.
- analysis/stats/measures.yaml: the site statistics use every source answered in the form's terms (form,
"Report a run", Reddit comments).
- analysis/black/evidence.py: a shadow run is "off the gates" only when an answer contradicts a gate (3); runs
that do not say are counted separately (14).
- croissant isBasedOn lists both megathreads (pipeline.yaml `cite`).
- Rebuilt: policy.json, stats.json, BLACK.md, black.json, croissant.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs
riders describe in the r/fourthwing megathreads and delivers them here by
pull request as data/public/reddit_comments.json: each run is the "Report a
run" answers (option labels of collection/specs/report_form.yaml), with the
comment id and day only; no usernames, no comment text.
- pipeline.yaml: a reddit_comments source of extract kind "delivered" (the
atlas fetches nothing; `update` skips it with a note)
- runs.py: the coder, sharing one answers_row() with the reported coder;
every row is flagged as read from a Reddit comment (and "not checked by
hand" until corrected); a result page a comment already carries is not
counted twice as a "reddit thread" row
- collection/manual/reddit_comments.yaml: reject / fix / add, applied at
build time
- dictionary, SOURCES, README, croissant: the new source
The route model and statistics do not use the source until it is named in
planner/policy.yaml / analysis/stats/measures.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
forge.yaml: repo DragonKind/threshing-atlas, and the links generated from it (data/public/links.json,
data/croissant.json); the project notes too. Old kathleen/threshing-atlas links redirect.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- spreadsheet 8,132 -> 8,748 rows (about 600 new; the sheet's keeper fixed 'Sqaudmate' in 72 answers,
corrected 15 attempt numbers to 'First Attempt' and re-pasted the daily bulk logs); register 288 -> 292
- form.yaml squad_any_case: a typed 'second' is Second (it stopped the build)
- collection rebuild: the runs are built and checked first, so a failed check stops it before the model,
plan, reports and statistics are rebuilt from stale runs
- route model, plan, Black evidence, screen tree, statistics, graph and metadata rebuilt (9,473 runs)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- python -m collection rebuild: graph, route model, experiment plan, the Black evidence, the screen
tree, statistics, build. refresh, cycle and both update-sources workflows use it, so the morning
update also rebuilds the reports the tests check (it rebuilt only graph, model and stats before)
- graph.json had not been rebuilt since the played-steps fix (no Node on the Mac)
- docs: the branch is pushed; README mentions rebuild
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- analysis/tree/build_tree.py -> data/public/tree.json, analysis/reports/TREE.md: every screen the bot has seen,
every option, where each led (fixed or dice, with ranges), best known picks per colour chosen cautiously, where
any picks end, and the untested options. solver.js screenPick(): least-tried and Thompson picks from the tree.
- The bot logs a pick it then overrode as a step: 37 runs were coded with choices they never made. Importer and
graph.js now use the played steps only.
- runs_clean.csv: the Unofficial Dragon Register's 288 bonded dragons are in the master list as outcome-only
rows, flagged (no analysis uses them).
- Cycle, refresh and the bot kit rebuild and carry the tree; tests check every generated report is current.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
- planner/knowledge.yaml: what the game's developer has said (paraphrased, dated, graded), what the Atlas does
with each statement, and the open questions with the experiment that tests each.
- analysis/black/evidence.py -> analysis/reports/BLACK.md: counts from the runs. On the same recorded route the
shadow shows on 21 of 79 first attempts and 1 of 80 later ones; no single route; Blue then Black after the shadow.
- game.yaml: the shadow is a guard (not a roll); what follows it is the Blue, then the Black; on-screen text for
the squad's direction screen.
- planner/experiments.yaml v2: one exact path on every fresh account (fresh-replicate), then one choice changed
at a time (fresh-ablate); shadow-again, second-attempt-same-path, veteran-repeat, veteran-variety; what the bot
records about the account; the sequence to follow once the shadow shows.
- solver.js: assign() takes account facts, account offsets and cycled arms; recommendOption() answers the
squad screen (stay + direction) in one pick.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
- the importer reads data/accounts/*/runs.jsonl and logs/accounts/*/encounters.jsonl;
with account folders, the top-level files (a mirror of the first account) are skipped
- run records that carry their steps (schema 2) become trajectories directly, keeping
what the bot saw (shadow, meadow dragons), propensities, decision mode and policy id;
older runs are still rebuilt from the decision log; bonded runs without steps are kept
- account names become salted hashes (DK_ACCOUNT_SALT, or a private salt in data/raw)
- runs_clean.csv: new bot_mode column; saw_shadow and saw_blue_meadow from what the bot
saw; deliberate burns say why
- crosswalk: the valley Red, leaving squadmates, squad direction, hold your ground,
hide/approach reactions, the wounded cadet's other two choices
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- collection/pipeline.yaml declares every source (extract kind, transform,
privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
`source` column (reddit spreadsheet, reddit thread, reported, bot,
reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
collection/specs/report_form.yaml, collected as source "reported", plus a
site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
measures.yaml); stats.json and policy.json unchanged apart from the
factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
the Actions cache, encrypted with RAW_ARCHIVE_KEY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- data/dictionary.yaml: every published file, and every column of runs_clean.csv
with its type, meaning and allowed values
- collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from
it, with a sha256 per file; `npm run metadata`, run by the daily update
- tests/core/test_dictionary.py: every column and value described, types hold,
croissant.json up to date and checksums match
- attempt is now written as a whole number (was 1.0, 2.0); no statistic changes
- the site ships both files and the Methods text links them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ