Collection as an ETL pipeline; runs_clean.csv from every source; Report a run
- collection/pipeline.yaml declares every source (extract kind, transform, privacy rules, load target, run source); `python -m collection update|build` - extract keeps data/raw/<source>/current + previous only (git-ignored); data/manifest.json records the current raw checksum and change date - transforms are pure; validate checks runs against data/dictionary.yaml before load; load writes byte-stable files only when they change - data/public/runs_clean.csv: every run from every source with a new `source` column (reddit spreadsheet, reddit thread, reported, bot, reddit pdf); the sheet's own row label moves to `entry`; run_id is text - bot importer ported to Python (same output as bot.js on the fixture); labels and crosswalk in collection/specs/bot.yaml, written into graph.js - Reddit through the official API (result links to add by hand); the 12 result pages move to collection/manual/reddit_results.yaml - "Report a run": a GitHub issue form generated from collection/specs/report_form.yaml, collected as source "reported", plus a site tab pointing to it, the community spreadsheet and the megathread - analysis and the route model read runs_clean.csv (sources in measures.yaml); stats.json and policy.json unchanged apart from the factor's new name - daily workflow runs the pipeline; the raw store is kept between runs in the Actions cache, encrypted with RAW_ARCHIVE_KEY Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
61bc50583c
commit
3b24a11f5a
@@ -0,0 +1,19 @@
|
||||
# Dragons from dragonkind.com result pages that riders linked in the r/fourthwing megathreads, read by hand.
|
||||
# Each is one run in runs_clean.csv (source "reddit thread": bonded, colour and tail known, route not recorded) and
|
||||
# appears under "extra" in data/public/register.json. The reddit_thread source lists result links found in the
|
||||
# threads that are not here yet; add them after reading the result page. No usernames are kept.
|
||||
version: 1
|
||||
read: "2026-10-19"
|
||||
results:
|
||||
- { id: THRESH-VFRQVM5K3B0CP, name: Thiduach, colour: Blue, tail: Swordtail, sex: Female, age: 51, lineage: Gormfaileas }
|
||||
- { id: THRESH-3M4P9MNFQGBS8, name: Caenuis, colour: Blue, tail: Swordtail, sex: Female, age: 38, lineage: Gormfaileas }
|
||||
- { id: THRESH-5JQDJ21G1BYKR, name: Oavuagh, colour: Green, tail: Morningstartail, sex: Female, age: 43, lineage: Uaineloidsig }
|
||||
- { id: THRESH-5KRAAVFB3D1MG, name: Geagios, colour: Blue, tail: Clubtail, sex: Male, age: 99, lineage: Gormfaileas }
|
||||
- { id: THRESH-B5HSM00J4A2Z4, name: Fiedheil, colour: Black, tail: Clubtail, sex: Male, age: 32, lineage: Moerlaighen }
|
||||
- { id: THRESH-BMPM2GAPK58JT, name: Teirfua, colour: Green, tail: Morningstartail, sex: Female, age: 68, lineage: Eifyrielle }
|
||||
- { id: THRESH-MCQ8D6GAGF4K8, name: Caollo, colour: Blue, tail: Morningstartail, sex: Female, age: 93, lineage: Nyghmuiron }
|
||||
- { id: THRESH-Q0XDBFMNBEB86, name: Aenminn, colour: Red, tail: Swordtail, sex: Male, age: 59, lineage: Berradhyr }
|
||||
- { id: THRESH-W99DB29MG16QA, name: Meinso, colour: Green, tail: Clubtail, sex: Female, age: 63, lineage: Eifyrielle }
|
||||
- { id: THRESH-XCJCY7HW7QSSP, name: Einmial, colour: Blue, tail: Clubtail, sex: Male, age: 58, lineage: Gormfaileas }
|
||||
- { id: THRESH-XR6G68VPGT48M, name: Recuagh, colour: Brown, tail: Morningstartail, sex: Female, age: 55, lineage: Vormalgh }
|
||||
- { id: THRESH-YQC97TK34214E, name: Dannidh, colour: Orange, tail: Clubtail, sex: Male, age: 61, lineage: Cariaslog }
|
||||
Reference in New Issue
Block a user