- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
The daily check keeps the community route PDF only in its private raw store (never published here). When that
copy differs from "Threshing Options.pdf" in the bot's private repo, `python -m collection send-pdf` opens (or
updates) a pull request there with the new PDF and the lines its text gained and lost, so Ken can update the
hand-written routes in src/pdfRoutes.js (the bot does not read the PDF itself).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- update-sources: download DragonKind/DragonKind (read with ATLAS_BOT_TOKEN, or DRAGONKIND_TOKEN if set) and
import it as the bot source. Needs DK_ACCOUNT_SALT (the same salt as earlier imports, so accounts keep their keys);
without it the bot source is skipped as before.
- The bot load keeps one file, data/bot/trajectories-bot.jsonl: the bot's repo holds every account's full history,
so each import is merged by attempt id (attempts already published stay; older per-import files are folded in).
One copy of each attempt also keeps build-graph.js from counting an attempt twice.
- New workflow push-policy: when policy.json, solver.js or graph.js change on main, open (or update) a pull request
on DragonKind/DragonKind that replaces vendor/threshing-atlas/ with them. Ken merges it.
- forge open-pr: --repo/--base for another repository, and reuse the open pull request for the same branch.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- python -m collection rebuild: graph, route model, experiment plan, the Black evidence, the screen
tree, statistics, build. refresh, cycle and both update-sources workflows use it, so the morning
update also rebuilds the reports the tests check (it rebuilt only graph, model and stats before)
- graph.json had not been rebuilt since the played-steps fix (no Node on the Mac)
- docs: the branch is pushed; README mentions rebuild
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
the "Report a run" collector, the page's links (data/public/links.json, filled in by
site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- python -m collection export-bot-kit: policy.json, solver.js, graph.js and a
README with checksums, zipped for the bot
- the bot import takes a folder or a zip (files may sit under any top folder)
- python -m collection refresh: the daily workflow, run locally
- trajectories files are named by date and raw checksum, so two imports on
one day never overwrite each other
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- collection/pipeline.yaml declares every source (extract kind, transform,
privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
`source` column (reddit spreadsheet, reddit thread, reported, bot,
reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
collection/specs/report_form.yaml, collected as source "reported", plus a
site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
measures.yaml); stats.json and policy.json unchanged apart from the
factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
the Actions cache, encrypted with RAW_ARCHIVE_KEY
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ