Receive runs from the Reddit megathread comments (source "reddit comment")
CI / test (pull_request) Successful in 46s
CI / test (pull_request) Successful in 46s
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs riders describe in the r/fourthwing megathreads and delivers them here by pull request as data/public/reddit_comments.json: each run is the "Report a run" answers (option labels of collection/specs/report_form.yaml), with the comment id and day only; no usernames, no comment text. - pipeline.yaml: a reddit_comments source of extract kind "delivered" (the atlas fetches nothing; `update` skips it with a note) - runs.py: the coder, sharing one answers_row() with the reported coder; every row is flagged as read from a Reddit comment (and "not checked by hand" until corrected); a result page a comment already carries is not counted twice as a "reddit thread" row - collection/manual/reddit_comments.yaml: reject / fix / add, applied at build time - dictionary, SOURCES, README, croissant: the new source The route model and statistics do not use the source until it is named in planner/policy.yaml / analysis/stats/measures.yaml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -64,6 +64,11 @@ def reddit(spec, env):
|
||||
return json.dumps(threads, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||
|
||||
|
||||
def delivered(spec, env):
|
||||
"""A source another repository delivers by pull request (it changes the public file itself): nothing to fetch."""
|
||||
raise Skip(f"delivered by pull request from {spec.get('by') or 'another repository'}")
|
||||
|
||||
|
||||
def forge_issues(spec, env):
|
||||
"""Every issue with the label (open and closed), from the repository's host (forge.yaml: Gitea or GitHub)."""
|
||||
from . import forge
|
||||
@@ -107,7 +112,7 @@ def pack_local(spec, src):
|
||||
return json.dumps(files, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||
|
||||
|
||||
EXTRACTORS = {"http": http, "reddit": reddit, "forge_issues": forge_issues, "local_dir": local_dir}
|
||||
EXTRACTORS = {"http": http, "reddit": reddit, "forge_issues": forge_issues, "local_dir": local_dir, "delivered": delivered}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- the raw store
|
||||
|
||||
Reference in New Issue
Block a user