Reddit comment runs in the route model and the odds; Black report gate wording; cite both megathreads

- planner/policy.yaml: `reddit comment` is a model source. Only runs that break a game rule (the rule_flags
  checks) are left out; partial runs are kept. Its own reporting-bias group, and out of the low-bias sample
  (riders posting a run pick the notable ones). 77 of the 82 runs are used.
- analysis/stats/measures.yaml: the site statistics use every source answered in the form's terms (form,
  "Report a run", Reddit comments).
- analysis/black/evidence.py: a shadow run is "off the gates" only when an answer contradicts a gate (3); runs
  that do not say are counted separately (14).
- croissant isBasedOn lists both megathreads (pipeline.yaml `cite`).
- Rebuilt: policy.json, stats.json, BLACK.md, black.json, croissant.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-19 14:50:08 +00:00
co-authored by Claude Opus 5.5
parent 75f11378e2
commit d606ed1651
11 changed files with 74 additions and 35 deletions
+14 -3
View File
@@ -186,9 +186,20 @@ def build():
("combat_second", "The dagger"), ("resolution", "The bullies' fate"), ("post_path", "The fork")): ("combat_second", "The dagger"), ("resolution", "The bullies' fate"), ("post_path", "The fork")):
t = collections.Counter(r[col] for r in S1) t = collections.Counter(r[col] for r in S1)
L.append(f"- **{label}:** " + ", ".join(f"{k} {v}" for k, v in t.most_common())) L.append(f"- **{label}:** " + ", ".join(f"{k} {v}" for k, v in t.most_common()))
outside = [r for r in runs if r["saw_shadow"] and not (r["scream"] == "Investigate & help" and r["hour"] == "Tree")] # a gate counts as missed only when the run answers it otherwise; a run that does not say (a short Reddit
facts["shadow_routes"] = {"runs": len(S1), "routes": len({tuple(r[k] for k in ROUTE) for r in S1}), "off_gate": len(outside)} # comment, a blank form answer) is not a contradiction
L += ["", f"Shadows reported without both gates: {len(outside)} (likely mis-entered).", ""] def stated(v):
return bool(v) and v != "Unknown" and not str(v).startswith("(")
def contradicts(r):
return (stated(r["scream"]) and r["scream"] != "Investigate & help") or (stated(r["hour"]) and r["hour"] != "Tree")
outside = [r for r in runs if r["saw_shadow"] and contradicts(r)]
unstated = [r for r in runs if r["saw_shadow"] and not contradicts(r)
and not (r["scream"] == "Investigate & help" and r["hour"] == "Tree")]
facts["shadow_routes"] = {"runs": len(S1), "routes": len({tuple(r[k] for k in ROUTE) for r in S1}),
"off_gate": len(outside), "gates_unstated": len(unstated)}
L += ["", f"Shadows reported with an answer that contradicts a gate: {len(outside)} (likely mis-entered or misread). "
f"Another {len(unstated)} do not say whether they passed both gates.", ""]
# 5. after the shadow # 5. after the shadow
SH = [r for r in runs if r["saw_shadow"]] SH = [r for r in runs if r["saw_shadow"]]
+1 -1
View File
@@ -56,7 +56,7 @@ The 97 form runs with a shadow (all attempts) took 47 different recorded routes:
- **The bullies' fate:** Squadmate's revenge 61, Left bullies to the dragons 25, Killed the bullies yourself 9, Unknown 2 - **The bullies' fate:** Squadmate's revenge 61, Left bullies to the dragons 25, Killed the bullies yourself 9, Unknown 2
- **The fork:** West 46, Forest 36, East 14, Unknown 1 - **The fork:** West 46, Forest 36, East 14, Unknown 1
Shadows reported without both gates: 17 (likely mis-entered). Shadows reported with an answer that contradicts a gate: 3 (likely mis-entered or misread). Another 14 do not say whether they passed both gates.
## 5. After the shadow: the Blue, then the Black ## 5. After the shadow: the Blue, then the Black
+4 -4
View File
@@ -6,10 +6,10 @@
# (not blank), {any: [condition, ...]} (at least one). Several fields in one condition must all hold. # (not blank), {any: [condition, ...]} (at least one). Several fields in one condition must all hold.
version: 1 version: 1
# Which runs of data/public/runs_clean.csv the statistics use (its `source` column). Runs from the other sources # Which runs of data/public/runs_clean.csv the statistics use (its `source` column): every source answered in the
# (reported, bot, Reddit result pages) are in the published file but not in these statistics yet: they are coded # form's terms (the community form, "Report a run", the Reddit megathread comments). The bot follows a strategy and
# from different questions, and the bot follows a strategy, so pooling them needs its own review. # the Reddit result pages and the register record only the end, so those stay out of these statistics for now.
sources: [reddit spreadsheet] sources: [reddit spreadsheet, reported, reddit comment]
colours: [Red, Green, Orange, Brown, Blue, Black] colours: [Red, Green, Orange, Brown, Blue, Black]
+4 -1
View File
@@ -112,10 +112,13 @@ def build(dictionary=None):
"citeAs": ds["citation"].format(repository=forge.repo_url(fg)), "citeAs": ds["citation"].format(repository=forge.repo_url(fg)),
"isLiveDataset": True, "isLiveDataset": True,
"isBasedOn": [ "isBasedOn": [
{"@type": "sc:CreativeWork", "name": s["label"], "url": next(iter(s["links"].values()))} {"@type": "sc:CreativeWork", "url": s["links"][k],
"name": s["label"] + (f" ({k.replace('part', 'part ')})" if len(s.get("cite") or ()) > 1 else "")}
for s in sources.values() for s in sources.values()
# the route PDF is only fingerprinted, never redistributed; the bot's repo is private # the route PDF is only fingerprinted, never redistributed; the bot's repo is private
if (s.get("privacy") or {}).get("redistribute", True) and s.get("links") if (s.get("privacy") or {}).get("redistribute", True) and s.get("links")
# `cite` (collection/pipeline.yaml) names the links the source is based on; default: the first link
for k in (s.get("cite") or [next(iter(s["links"]))])
], ],
"distribution": distribution, "distribution": distribution,
"recordSet": [ "recordSet": [
+2
View File
@@ -57,6 +57,7 @@ sources:
runs: null # a flowchart of the routes, not a log of runs (see collection/README.md) runs: null # a flowchart of the routes, not a log of runs (see collection/README.md)
reddit_thread: reddit_thread:
cite: [part3, part4] # both megathreads, in the dataset's isBasedOn
label: Runs posted in the r/fourthwing Dragonkind megathreads label: Runs posted in the r/fourthwing Dragonkind megathreads
run_source: reddit thread run_source: reddit thread
links: links:
@@ -70,6 +71,7 @@ sources:
runs: reddit_results runs: reddit_results
reddit_comments: reddit_comments:
cite: [part3, part4] # both megathreads, in the dataset's isBasedOn
label: Runs riders describe in the r/fourthwing Dragonkind megathread comments label: Runs riders describe in the r/fourthwing Dragonkind megathread comments
run_source: reddit comment run_source: reddit comment
links: links:
+22 -12
View File
@@ -66,23 +66,33 @@
"isBasedOn": [ "isBasedOn": [
{ {
"@type": "sc:CreativeWork", "@type": "sc:CreativeWork",
"name": "Threshing Routes form and spreadsheet (u/ClumsyBadger, linked from the r/fourthwing megathreads)", "url": "https://docs.google.com/spreadsheets/d/1gnw1AdRg1XT4JsOfRM6QTA0jSaS3n9ZF-rPWnYewzv0",
"url": "https://docs.google.com/spreadsheets/d/1gnw1AdRg1XT4JsOfRM6QTA0jSaS3n9ZF-rPWnYewzv0" "name": "Threshing Routes form and spreadsheet (u/ClumsyBadger, linked from the r/fourthwing megathreads)"
}, },
{ {
"@type": "sc:CreativeWork", "@type": "sc:CreativeWork",
"name": "Unofficial Dragon Register (which dragon each rider bonded)", "url": "https://docs.google.com/spreadsheets/d/1KMiE9sec0l597Pzue1OhGYf07NF4howtQ_lPeqx7jXQ",
"url": "https://docs.google.com/spreadsheets/d/1KMiE9sec0l597Pzue1OhGYf07NF4howtQ_lPeqx7jXQ" "name": "Unofficial Dragon Register (which dragon each rider bonded)"
}, },
{ {
"@type": "sc:CreativeWork", "@type": "sc:CreativeWork",
"name": "Runs posted in the r/fourthwing Dragonkind megathreads", "url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/",
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/" "name": "Runs posted in the r/fourthwing Dragonkind megathreads (part 3)"
}, },
{ {
"@type": "sc:CreativeWork", "@type": "sc:CreativeWork",
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments", "url": "https://www.reddit.com/r/fourthwing/comments/1wz5cnj/",
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/" "name": "Runs posted in the r/fourthwing Dragonkind megathreads (part 4)"
},
{
"@type": "sc:CreativeWork",
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/",
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments (part 3)"
},
{
"@type": "sc:CreativeWork",
"url": "https://www.reddit.com/r/fourthwing/comments/1wz5cnj/",
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments (part 4)"
} }
], ],
"distribution": [ "distribution": [
@@ -143,8 +153,8 @@
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).", "description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json", "contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json",
"encodingFormat": "application/json", "encodingFormat": "application/json",
"sha256": "bb401225065ae17311bbd02fd63acd4010f4e325d7eab74e044115b98d04bec4", "sha256": "6037558db5eaed34b625b9f482db9a107e9262b9c13451bcf14855b4addc46d3",
"contentSize": "5498476 B" "contentSize": "5628300 B"
}, },
{ {
"@type": "cr:FileObject", "@type": "cr:FileObject",
@@ -153,8 +163,8 @@
"description": "The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).", "description": "The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/policy.json", "contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/policy.json",
"encodingFormat": "application/json", "encodingFormat": "application/json",
"sha256": "f65aaebb23cb3f56dad0387a94113857eb44c4ba077cfe5fc6215c202546588c", "sha256": "d464fb6c90b1c3b56e8ac1a90aa86e23d0cfde617b7aeb1d601233ed8a1872cc",
"contentSize": "987411 B" "contentSize": "987551 B"
}, },
{ {
"@type": "cr:FileObject", "@type": "cr:FileObject",
+2 -1
View File
@@ -116,7 +116,8 @@
"shadow_routes": { "shadow_routes": {
"runs": 97, "runs": 97,
"routes": 47, "routes": 47,
"off_gate": 17 "off_gate": 3,
"gates_unstated": 14
}, },
"after_shadow": { "after_shadow": {
"reached_black": 89, "reached_black": 89,
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+8 -5
View File
@@ -31,18 +31,21 @@ def low_bias(r, lb=CONFIG["low_bias"]):
"""Kept in the low-bias sample: every run of an always-kept source, bulk logs, and riders not chasing a rare dragon.""" """Kept in the low-bias sample: every run of an always-kept source, bulk logs, and riders not chasing a rare dragon."""
if r["source"] in lb["always"]: if r["source"] in lb["always"]:
return True return True
if r["source"] in (lb.get("never") or ()):
return False
if r["source"] == "reddit spreadsheet" and "bulk" in lb["entries_kept"] and r["entry"] not in (None, "Form"): if r["source"] == "reddit spreadsheet" and "bulk" in lb["entries_kept"] and r["entry"] not in (None, "Form"):
return True return True
return r["goal"] not in lb["rare_goals"] return r["goal"] not in lb["rare_goals"]
def drop_flagged(runs, cfg=CONFIG.get("drop_flagged") or {}): def drop_flagged(runs, cfg=CONFIG.get("drop_flagged") or {}):
"""Without the flagged runs of the sources read automatically from free text (planner/policy.yaml drop_flagged).""" """Without the runs of the sources read automatically from free text that break a game rule
sources, keep = set(cfg.get("sources") or ()), tuple(cfg.get("except") or ()) (planner/policy.yaml drop_flagged): every other run is kept, however partial."""
sources, rules = set(cfg.get("sources") or ()), tuple(cfg.get("rules") or ())
def problems(r): def impossible(r):
return [f for f in (r.get("data_flags") or "").split("; ") if f and not f.startswith(keep)] if keep else [f for f in (r.get("data_flags") or "").split("; ") if f] return any(rule in f for f in (r.get("data_flags") or "").split("; ") for rule in rules)
return [r for r in runs if r["source"] not in sources or not problems(r)] return [r for r in runs if r["source"] not in sources or not impossible(r)]
def clean(o): def clean(o):
+15 -6
View File
@@ -6,21 +6,29 @@
version: 1 version: 1
# the `source` values of data/public/runs_clean.csv the model learns from # the `source` values of data/public/runs_clean.csv the model learns from
sources: [reddit spreadsheet, reported, bot] sources: [reddit spreadsheet, reported, bot, reddit comment]
# Sources read automatically from free text: a run with any data flag (a broken game rule, an unrecognised answer) # Sources read automatically from free text: only a run that breaks one of the game's rules (the checks of
# is left out of the model, since the flag may mean the text was misread or misremembered. Flags that only say where # collection/etl/runs.py rule_flags, matched as text inside its flags) is left out of the model, since it cannot have
# the run came from (`except`, matched at the start of a flag) do not count. A hand fix that clears the flags # happened as read. Every other run is kept, however partial. A hand fix that clears the flag brings the run back.
# brings the run back. Applies once the source is also listed in `sources` above. # The first-sighting pool is not here: form runs outside it are kept too, so these are kept for the same reason.
drop_flagged: drop_flagged:
sources: [reddit comment] sources: [reddit comment]
except: ["read from a Reddit comment"] rules:
- "encounter, which has only"
- "Green recorded on a Valley run"
- "does not follow"
- "fight answers on a run that did not help"
- "a path after the fight on a run without a fight"
- "a second fight move without pausing to assess"
- "died before the hour, yet reached"
# Two samples. "all" is every run above. "lowBias" leaves out runs where a rare sighting is more likely to have # Two samples. "all" is every run above. "lowBias" leaves out runs where a rare sighting is more likely to have
# been reported than a miss: form submissions and reports from riders chasing Blue or Black. The bot logs every # been reported than a miss: form submissions and reports from riders chasing Blue or Black. The bot logs every
# run, and bulk-pasted logs record every run of the players who kept them, so both stay in. # run, and bulk-pasted logs record every run of the players who kept them, so both stay in.
low_bias: low_bias:
always: [bot] # sources whose every run is kept always: [bot] # sources whose every run is kept
never: [reddit comment] # riders posting a run in a thread pick the notable ones, like rare-chasers
entries_kept: [bulk] # spreadsheet rows whose `entry` is not "Form" (bulk-pasted logs) entries_kept: [bulk] # spreadsheet rows whose `entry` is not "Form" (bulk-pasted logs)
rare_goals: [Blue, Black, Blue or Black] rare_goals: [Blue, Black, Blue or Black]
@@ -46,3 +54,4 @@ reporting_bias:
- { name: "form, chasing Blue/Black", when: { source: reddit spreadsheet, goal: rare } } - { name: "form, chasing Blue/Black", when: { source: reddit spreadsheet, goal: rare } }
- { name: "form, other", when: { source: reddit spreadsheet } } - { name: "form, other", when: { source: reddit spreadsheet } }
- { name: reported, when: { source: reported }, same_as: "form, other" } - { name: reported, when: { source: reported }, same_as: "form, other" }
- { name: reddit comments, when: { source: reddit comment } } # self-selected: its own reporting rate