Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s

- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
  on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
  runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
  be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
  to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
  form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
  match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
  page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
  stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-19 16:44:52 +00:00
co-authored by Claude Opus 5.5
parent 0df6b8d473
commit 98be27fc1f
15 changed files with 497 additions and 6 deletions
+12 -2
View File
@@ -126,6 +126,16 @@
"sha256": "b169d157e731db23c99f2f8317a9718d31801b7d64336b3d414511ac13f32493",
"contentSize": "23174 B"
},
{
"@type": "cr:FileObject",
"@id": "overlap",
"name": "overlap",
"description": "How many Reddit comment runs may also be in the community form (the same answers within a day), against what chance alone gives (analysis/overlap/overlap.py). Measured only: no run is removed.",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/overlap.json",
"encodingFormat": "application/json",
"sha256": "04280a52f028334451c505ddc35004d98cc1f4595b2294c3746a09e34454e8b0",
"contentSize": "1976 B"
},
{
"@type": "cr:FileObject",
"@id": "reports",
@@ -153,8 +163,8 @@
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json",
"encodingFormat": "application/json",
"sha256": "89a6246da2495685733e20f2de9897299bd19d469aaeadbd8883972856787ce5",
"contentSize": "5151910 B"
"sha256": "7431173d9b667d4914d17bbc299d994d14c3104e3e89a1eb75d690c1b21ade6a",
"contentSize": "5110214 B"
},
{
"@type": "cr:FileObject",