- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs
riders describe in the r/fourthwing megathreads and delivers them here by
pull request as data/public/reddit_comments.json: each run is the "Report a
run" answers (option labels of collection/specs/report_form.yaml), with the
comment id and day only; no usernames, no comment text.
- pipeline.yaml: a reddit_comments source of extract kind "delivered" (the
atlas fetches nothing; `update` skips it with a note)
- runs.py: the coder, sharing one answers_row() with the reported coder;
every row is flagged as read from a Reddit comment (and "not checked by
hand" until corrected); a result page a comment already carries is not
counted twice as a "reddit thread" row
- collection/manual/reddit_comments.yaml: reject / fix / add, applied at
build time
- dictionary, SOURCES, README, croissant: the new source
The route model and statistics do not use the source until it is named in
planner/policy.yaml / analysis/stats/measures.yaml.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The evening source check stopped (correctly) on two answers the coding rules did not know:
- first sighting "2 Greens": a new form option. Coded as Green, with a data flag that keeps the
original answer (first_sighting_alias in core/spec/form.yaml).
- row label "Unknown" on a bulk-pasted row: added to the entry values in data/dictionary.yaml.
Workflow: when `collection update` reports the build failed, skip the rebuild (it would only fail
again) and always open the "Source check failed" issue, so new answers are reported, not lost.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- analysis/tree/build_tree.py -> data/public/tree.json, analysis/reports/TREE.md: every screen the bot has seen,
every option, where each led (fixed or dice, with ranges), best known picks per colour chosen cautiously, where
any picks end, and the untested options. solver.js screenPick(): least-tried and Thompson picks from the tree.
- The bot logs a pick it then overrode as a step: 37 runs were coded with choices they never made. Importer and
graph.js now use the played steps only.
- runs_clean.csv: the Unofficial Dragon Register's 288 bonded dragons are in the master list as outcome-only
rows, flagged (no analysis uses them).
- Cycle, refresh and the bot kit rebuild and carry the tree; tests check every generated report is current.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
the "Report a run" collector, the page's links (data/public/links.json, filled in by
site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- core/atlas/bias.py: for each rare roll (shadow, meadow Blue) and each source group,
the factor by which the group over-reports the rare outcome relative to complete
logging (bot, bulk logs), with a 90% range; rare outcomes from biased groups count
1/rho; a roll the complete logs have (almost) never shown is learned from them only
- planner/policy.yaml: groups, rolls and thresholds; estimates published in
policy.json under model.reportingBias
- fit_alpha: a roll seen only one way is pooled fully
- docs: goal accounts follow the best route, research accounts use Thompson sampling
(decisionMode atlas_thompson)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- the importer reads data/accounts/*/runs.jsonl and logs/accounts/*/encounters.jsonl;
with account folders, the top-level files (a mirror of the first account) are skipped
- run records that carry their steps (schema 2) become trajectories directly, keeping
what the bot saw (shadow, meadow dragons), propensities, decision mode and policy id;
older runs are still rebuilt from the decision log; bonded runs without steps are kept
- account names become salted hashes (DK_ACCOUNT_SALT, or a private salt in data/raw)
- runs_clean.csv: new bot_mode column; saw_shadow and saw_blue_meadow from what the bot
saw; deliberate burns say why
- crosswalk: the valley Red, leaving squadmates, squad direction, hold your ground,
hide/approach reactions, the wounded cadet's other two choices
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- collection/pipeline.yaml declares every source (extract kind, transform,
privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
`source` column (reddit spreadsheet, reddit thread, reported, bot,
reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
collection/specs/report_form.yaml, collected as source "reported", plus a
site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
measures.yaml); stats.json and policy.json unchanged apart from the
factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
the Actions cache, encrypted with RAW_ARCHIVE_KEY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- data/dictionary.yaml: every published file, and every column of runs_clean.csv
with its type, meaning and allowed values
- collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from
it, with a sha256 per file; `npm run metadata`, run by the daily update
- tests/core/test_dictionary.py: every column and value described, types hold,
croissant.json up to date and checksums match
- attempt is now written as a whole number (was 1.0, 2.0); no statistic changes
- the site ships both files and the Methods text links them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ