Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s
CI / test (pull_request) Successful in 40s
- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml: on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it to one comment. collection/README.md: "Reviewing Reddit runs". - analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`. - Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page) stays the one #18 applies. - data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
0df6b8d473
commit
98be27fc1f
@@ -132,6 +132,10 @@ def main(argv=None):
|
||||
k.add_argument("--out", default="exports")
|
||||
sub.add_parser("rebuild", help="rebuild everything made from the runs: graph, route model, plan, reports, statistics, build")
|
||||
sub.add_parser("send-pdf", help="open a pull request on the bot's repo when the route PDF changed (private copy only)")
|
||||
rv = sub.add_parser("review-reddit", help="a review summary for a Reddit scraper pull request (run `build` first)")
|
||||
rv.add_argument("--base", help="data/public/reddit_comments.json as it is on main")
|
||||
rv.add_argument("--out", help="write the summary here (default: print it)")
|
||||
rv.add_argument("--pr", help="also post it on this pull request, replacing the previous summary")
|
||||
r = sub.add_parser("refresh", help="update every source, then rebuild everything (the daily workflow, locally)")
|
||||
r.add_argument("--bot", action="append", default=[], help="a bot folder or a .zip of its data (repeatable)")
|
||||
c = sub.add_parser("cycle", help="the loop with the bot: import its runs, refit, re-plan, deliver (safe on a timer)")
|
||||
@@ -155,6 +159,9 @@ def main(argv=None):
|
||||
if a.cmd == "rebuild":
|
||||
from .etl.handoff import rebuild
|
||||
return rebuild()
|
||||
if a.cmd == "review-reddit":
|
||||
from .etl.review_reddit import main as review
|
||||
return review(a.base, a.out, a.pr)
|
||||
if a.cmd == "send-pdf":
|
||||
from .etl.send_pdf import send
|
||||
return send()
|
||||
|
||||
Reference in New Issue
Block a user