Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s

- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
  on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
  runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
  be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
  to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
  form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
  match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
  page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
  stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-19 16:44:52 +00:00
co-authored by Claude Opus 5.5
parent 0df6b8d473
commit 98be27fc1f
15 changed files with 497 additions and 6 deletions
+19
View File
@@ -41,6 +41,25 @@ The route PDF is a flowchart of the choices, not a log of runs, so it adds no ro
fingerprint tells us when the routes it documents change. The register lists bonded dragons without the route that
led there, so it adds no rows either.
## Reviewing Reddit runs
Ken's scraper opens (or updates) a pull request from `data/reddit-comments` that changes
`data/public/reddit_comments.json`. The **Review Reddit runs** workflow (`.gitea/workflows/reddit-review.yml`) posts a
summary on it, refreshed on every push (`python -m collection review-reddit --base <main's file>` locally):
- runs added, changed and gone since main;
- **runs that break a game rule** (`core/spec/form.yaml` `impossible_flags`): left out of the route model and the
statistics until corrected. Open the comment and decide;
- other flags on the new runs (kept);
- runs that may also be in the community form: the same answers within a day, with how many chance alone gives
(`analysis/overlap/overlap.py`, `data/public/overlap.json`). Measured only, never removed;
- runs whose result page is already a `reddit thread` row (counted once automatically).
Corrections go in `manual/reddit_comments.yaml`, applied at build time, so they never wait for the threads:
`reject: [id]` (not a run), `fix: {id: {question: answer}}` (misread; `null` removes an answer) and
`add: [{id, thread, posted, answers}]` (missed). A corrected run loses the "not checked by hand" note. No usernames:
the comment id is enough to find the comment again. Then merge, once CI passes.
## Raw files and privacy
`data/raw/` is git-ignored and never published. It holds the current raw file of each source and the one before