Review routine for the Reddit scraper's pull requests #23

Closed
opened 2026-11-19 03:29:26 +00:00 by kathleen · 0 comments
Owner

Ken's scraper (DragonKind/threshing-reddit) reads the megathread comments every 6 hours. It opens or updates one pull request from data/reddit-comments that changes data/public/reddit_comments.json. Each scraper PR needs a quick human review before merging.

Make the review easy

CI posts a summary comment on each scraper PR (or adds it to the PR body) with:

  • runs added, changed and removed since main;
  • runs that fail a game-rule check, with the comment id and which rule;
  • likely duplicates of spreadsheet rows. People often post the same run in the form and in a comment: same day, same answers;
  • result-page ids already in reddit thread, which are deduplicated automatically. This is for information.

The reviewer's options

Corrections go in collection/manual/reddit_comments.yaml, which is applied at build time:

  • reject: [comment id]: not a run, or a joke;
  • fix: {comment id: {question: answer}}: misread answers;
  • add: a run the scraper missed.

Then merge. Rows touched by a correction lose the "automatically, not checked by hand" flag.

Docs

Add a short "Reviewing Reddit runs" section to collection/README.md.

Related: plausibility checks; "Use every possible run…"

Ken's scraper (`DragonKind/threshing-reddit`) reads the megathread comments every 6 hours. It opens or updates one pull request from `data/reddit-comments` that changes `data/public/reddit_comments.json`. Each scraper PR needs a quick human review before merging. ## Make the review easy CI posts a summary comment on each scraper PR (or adds it to the PR body) with: - runs added, changed and removed since main; - runs that fail a game-rule check, with the comment id and which rule; - likely duplicates of spreadsheet rows. People often post the same run in the form and in a comment: same day, same answers; - result-page ids already in `reddit thread`, which are deduplicated automatically. This is for information. ## The reviewer's options Corrections go in `collection/manual/reddit_comments.yaml`, which is applied at build time: - `reject: [comment id]`: not a run, or a joke; - `fix: {comment id: {question: answer}}`: misread answers; - `add`: a run the scraper missed. Then merge. Rows touched by a correction lose the "automatically, not checked by hand" flag. ## Docs Add a short "Reviewing Reddit runs" section to `collection/README.md`. Related: plausibility checks; "Use every possible run…"
kathleen added the redditdata quality labels 2026-11-19 03:29:26 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DragonKind/threshing-atlas#23