Receive runs from the Reddit megathread comments (source "reddit comment")
CI / test (pull_request) Successful in 46s
CI / test (pull_request) Successful in 46s
The threshing-reddit scraper (DragonKind/threshing-reddit) reads the runs riders describe in the r/fourthwing megathreads and delivers them here by pull request as data/public/reddit_comments.json: each run is the "Report a run" answers (option labels of collection/specs/report_form.yaml), with the comment id and day only; no usernames, no comment text. - pipeline.yaml: a reddit_comments source of extract kind "delivered" (the atlas fetches nothing; `update` skips it with a note) - runs.py: the coder, sharing one answers_row() with the reported coder; every row is flagged as read from a Reddit comment (and "not checked by hand" until corrected); a result page a comment already carries is not counted twice as a "reddit thread" row - collection/manual/reddit_comments.yaml: reject / fix / add, applied at build time - dictionary, SOURCES, README, croissant: the new source The route model and statistics do not use the source until it is named in planner/policy.yaml / analysis/stats/measures.yaml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -18,7 +18,7 @@ manifest: data/manifest.json # for each source: the sha256 of the current
|
||||
runs:
|
||||
path: data/public/runs_clean.csv
|
||||
# sources in this order; within a source, the order the coder returns
|
||||
order: [reddit_spreadsheet, reddit_thread, register, reported, bot, reddit_pdf]
|
||||
order: [reddit_spreadsheet, reddit_thread, reddit_comments, register, reported, bot, reddit_pdf]
|
||||
|
||||
sources:
|
||||
reddit_spreadsheet:
|
||||
@@ -69,6 +69,24 @@ sources:
|
||||
load: null # the public source file is the hand-checked collection/manual/reddit_results.yaml
|
||||
runs: reddit_results
|
||||
|
||||
reddit_comments:
|
||||
label: Runs riders describe in the r/fourthwing Dragonkind megathread comments
|
||||
run_source: reddit comment
|
||||
links:
|
||||
part3: https://www.reddit.com/r/fourthwing/comments/1wvwfp9/
|
||||
part4: https://www.reddit.com/r/fourthwing/comments/1wz5cnj/
|
||||
scraper: https://git.kenschiano.net/DragonKind/threshing-reddit
|
||||
# delivered, not extracted: the threshing-reddit scraper (its own repository) reads the threads every 6 hours and
|
||||
# opens a pull request here that changes data/public/reddit_comments.json. Each comment is read into the
|
||||
# "Report a run" answers (option labels of specs/report_form.yaml); only the comment id, the day and the
|
||||
# answers are kept: no usernames, no comment text. `update` leaves this source alone.
|
||||
extract: { kind: delivered, by: threshing-reddit }
|
||||
transform: null
|
||||
load: data/public/reddit_comments.json
|
||||
# hand corrections (reject / fix / add a run), applied when the runs are built
|
||||
manual: collection/manual/reddit_comments.yaml
|
||||
runs: reddit_comments
|
||||
|
||||
reported:
|
||||
label: Runs reported on the Atlas ("Report a run", an issue form on the Atlas's repository)
|
||||
run_source: reported
|
||||
|
||||
Reference in New Issue
Block a user