Reddit comments and the bot in the route model and the site odds; chosen deaths never count as failed bonds #40

Merged
kathleen merged 2 commits from atlas/reddit-in-model into main 2026-11-19 15:00:39 +00:00
5 changed files with 53 additions and 25 deletions
Showing only changes of commit 036c86a1d1 - Show all commits
+17 -13
View File
@@ -6,10 +6,12 @@
# (not blank), {any: [condition, ...]} (at least one). Several fields in one condition must all hold. # (not blank), {any: [condition, ...]} (at least one). Several fields in one condition must all hold.
version: 1 version: 1
# Which runs of data/public/runs_clean.csv the statistics use (its `source` column): every source answered in the # Which runs of data/public/runs_clean.csv the statistics use (its `source` column): every source that records the
# form's terms (the community form, "Report a run", the Reddit megathread comments). The bot follows a strategy and # route (the community form, "Report a run", the Reddit megathread comments, the DragonKind bot). The Reddit result
# the Reddit result pages and the register record only the end, so those stay out of these statistics for now. # pages and the register record only the end, so they stay out. The bot plays to a strategy and burns on purpose at
sources: [reddit spreadsheet, reported, reddit comment] # encounters it is not chasing: like every rider's deliberate death (`_deliberate`, the route model's rule), those
# burns are left out of the bond rates, never counted as failed bonds.
sources: [reddit spreadsheet, reported, reddit comment, bot]
colours: [Red, Green, Orange, Brown, Blue, Black] colours: [Red, Green, Orange, Brown, Blue, Black]
@@ -25,7 +27,7 @@ outcomes:
event: { saw_shadow: true } event: { saw_shadow: true }
bond_blue_meadow: bond_blue_meadow:
label: Bond, given you approached the meadow Blue label: Bond, given you approached the meadow Blue
denom: { branch: "Passed the Red: meadow", enc_Blue: { in: ["Bonded!", "Incinerated"] } } denom: { branch: "Passed the Red: meadow", enc_Blue: { in: ["Bonded!", "Incinerated"] }, _deliberate: { not: true } }
event: { enc_Blue: "Bonded!" } event: { enc_Blue: "Bonded!" }
reach_hour: reach_hour:
label: Survive to the hour (tree/valley choice) label: Survive to the hour (tree/valley choice)
@@ -34,7 +36,7 @@ outcomes:
# one more outcome per colour: "Run ends bonded to <colour>", among runs that reached the hour # one more outcome per colour: "Run ends bonded to <colour>", among runs that reached the hour
bonded_outcome: bonded_outcome:
label: Run ends bonded to {colour} label: Run ends bonded to {colour}
denom: { hour: { in: [Tree, Valley] } } denom: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } }
event: { outcome: Bonded, outcome_color: "{colour}" } event: { outcome: Bonded, outcome_color: "{colour}" }
# Factors compared in the factor tables (each answer against the other answers to the same question) # Factors compared in the factor tables (each answer against the other answers to the same question)
@@ -43,15 +45,15 @@ factors: [company, direction, first_sighting, scream, resolution, combat_first,
# Route targets: the outcome (event) and which runs could have reached it at all (pool) # Route targets: the outcome (event) and which runs could have reached it at all (pool)
targets: targets:
bond_Blue: { label: Bond Blue, colour: Blue, event: { outcome: Bonded, outcome_color: Blue }, pool: { hour: Valley } } bond_Blue: { label: Bond Blue, colour: Blue, event: { outcome: Bonded, outcome_color: Blue }, pool: { hour: Valley, _deliberate: { not: true } } }
bond_Black: { label: Bond Black, colour: Black, event: { outcome: Bonded, outcome_color: Black }, pool: { hour: Tree } } bond_Black: { label: Bond Black, colour: Black, event: { outcome: Bonded, outcome_color: Black }, pool: { hour: Tree, _deliberate: { not: true } } }
see_Blue: { label: Meet a Blue (meadow), colour: Blue, event: { saw_blue_meadow: true }, pool: { branch: "Passed the Red: meadow" } } see_Blue: { label: Meet a Blue (meadow), colour: Blue, event: { saw_blue_meadow: true }, pool: { branch: "Passed the Red: meadow" } }
see_shadow: { label: Get the tree shadow, colour: Black, event: { saw_shadow: true }, pool: { hour: Tree, branch: { exists: true } } } see_shadow: { label: Get the tree shadow, colour: Black, event: { saw_shadow: true }, pool: { hour: Tree, branch: { exists: true } } }
bond_Red: { label: Bond Red, colour: Red, event: { outcome: Bonded, outcome_color: Red }, pool: { hour: { in: [Tree, Valley] } } } bond_Red: { label: Bond Red, colour: Red, event: { outcome: Bonded, outcome_color: Red }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
bond_Green: { label: Bond Green, colour: Green, event: { outcome: Bonded, outcome_color: Green }, pool: { hour: Tree } } bond_Green: { label: Bond Green, colour: Green, event: { outcome: Bonded, outcome_color: Green }, pool: { hour: Tree, _deliberate: { not: true } } }
bond_Orange: { label: Bond Orange, colour: Orange, event: { outcome: Bonded, outcome_color: Orange }, pool: { hour: { in: [Tree, Valley] } } } bond_Orange: { label: Bond Orange, colour: Orange, event: { outcome: Bonded, outcome_color: Orange }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
bond_Brown: { label: Bond Brown, colour: Brown, event: { outcome: Bonded, outcome_color: Brown }, pool: { hour: { in: [Tree, Valley] } } } bond_Brown: { label: Bond Brown, colour: Brown, event: { outcome: Bonded, outcome_color: Brown }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
bond_any: { label: Bond any dragon, colour: null, event: { outcome: Bonded }, pool: { hour: { in: [Tree, Valley] } } } bond_any: { label: Bond any dragon, colour: null, event: { outcome: Bonded }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
# Bonding encounters: which recorded attempt (column) belongs to which in-game encounter (where) # Bonding encounters: which recorded attempt (column) belongs to which in-game encounter (where)
encounters: encounters:
@@ -80,6 +82,8 @@ presets:
all: { label: All runs, filters: { source: all, goal: all, excludeFlagged: false } } all: { label: All runs, filters: { source: all, goal: all, excludeFlagged: false } }
form: { label: Form submissions only, filters: { source: form, goal: all, excludeFlagged: false } } form: { label: Form submissions only, filters: { source: form, goal: all, excludeFlagged: false } }
bulk: { label: Bulk logs only (imports), filters: { source: bulk, goal: all, excludeFlagged: false } } bulk: { label: Bulk logs only (imports), filters: { source: bulk, goal: all, excludeFlagged: false } }
reddit: { label: Reddit comments only, filters: { source: reddit, goal: all, excludeFlagged: false } }
bot: { label: DragonKind bot only, filters: { source: bot, goal: all, excludeFlagged: false } }
not_chasing: { label: Not chasing Blue/Black, filters: { source: all, goal: exclude_rare, excludeFlagged: false } } not_chasing: { label: Not chasing Blue/Black, filters: { source: all, goal: exclude_rare, excludeFlagged: false } }
chasing: { label: Chasing Blue/Black, filters: { source: all, goal: only_rare, excludeFlagged: false } } chasing: { label: Chasing Blue/Black, filters: { source: all, goal: only_rare, excludeFlagged: false } }
unflagged: { label: Rows without data-quality flags, filters: { source: all, goal: all, excludeFlagged: true } } unflagged: { label: Rows without data-quality flags, filters: { source: all, goal: all, excludeFlagged: true } }
+32 -8
View File
@@ -95,7 +95,18 @@ def is_rare(goal):
return goal in M["rare_goals"] return goal in M["rare_goals"]
# sources other than the community spreadsheet: their own group wherever runs are split by where they came from
SOURCE_LABEL = {"reddit comment": "Reddit comments", "bot": "DragonKind bot", "reported": "Reported on the Atlas"}
def entry_group(r):
"""Where a run came from: the spreadsheet's entry type (Form, Data Import, ...) or the source's own label."""
return SOURCE_LABEL.get(r["source"]) or r["entry"]
def stratum(r): def stratum(r):
if r["source"] in SOURCE_LABEL:
return SOURCE_LABEL[r["source"]]
if r["entry"] != "Form": if r["entry"] != "Form":
return "Bulk logs (imports)" return "Bulk logs (imports)"
g = r["goal"] g = r["goal"]
@@ -108,9 +119,14 @@ def goal2(r):
def apply_filters(runs, f): def apply_filters(runs, f):
def ok(r): def ok(r):
if f.get("source") == "form" and r["entry"] != "Form": sheet = r["source"] == "reddit spreadsheet"
if f.get("source") == "form" and not (sheet and r["entry"] == "Form"):
return False return False
if f.get("source") == "bulk" and r["entry"] == "Form": if f.get("source") == "bulk" and not (sheet and r["entry"] != "Form"):
return False
if f.get("source") == "reddit" and r["source"] != "reddit comment":
return False
if f.get("source") == "bot" and r["source"] != "bot":
return False return False
if f.get("goal") == "exclude_rare" and is_rare(r["goal"]): if f.get("goal") == "exclude_rare" and is_rare(r["goal"]):
return False return False
@@ -197,7 +213,7 @@ def analyse(all_runs, filters):
dates = sorted(r["date"] for r in runs if r.get("date")) dates = sorted(r["date"] for r in runs if r.get("date"))
outs = group_by(runs, lambda r: r["outcome"] + (" " + r["outcome_color"] if r.get("outcome_color") else "")) outs = group_by(runs, lambda r: r["outcome"] + (" " + r["outcome_color"] if r.get("outcome_color") else ""))
res["meta"] = { res["meta"] = {
"sources": {k: len(v) for k, v in group_by(runs, "entry").items()}, "sources": {k: len(v) for k, v in group_by(runs, entry_group).items()},
"date_min": dates[0] if dates else None, "date_max": dates[-1] if dates else None, "date_min": dates[0] if dates else None, "date_max": dates[-1] if dates else None,
"n_flagged": sum(1 for r in runs if r["data_flags"]), "flag_counts": flag_counts, "n_flagged": sum(1 for r in runs if r["data_flags"]), "flag_counts": flag_counts,
"outcomes": {k: len(v) for k, v in outs.items()}, "outcomes": {k: len(v) for k, v in outs.items()},
@@ -220,13 +236,14 @@ def analyse(all_runs, filters):
res["encounters"] = [] res["encounters"] = []
for e in M["encounters"]: for e in M["encounters"]:
col, colour = e["column"], e["colour"] col, colour = e["column"], e["colour"]
A = [r for r in runs if match(r, e["where"]) and r.get(col) in ("Bonded!", "Incinerated")] # a rider who chose to burn here (chasing another colour) did not fail this encounter
A = [r for r in runs if match(r, e["where"]) and r.get(col) in ("Bonded!", "Incinerated") and not r.get("_deliberate")]
win = lambda r, col=col: r.get(col) == "Bonded!" # noqa: E731 win = lambda r, col=col: r.get(col) == "Bonded!" # noqa: E731
intent = [r for r in A if r["goal"] == colour or r["goal"].startswith("Any") or (r["goal"] == "Blue or Black" and colour in ("Blue", "Black"))] intent = [r for r in A if r["goal"] == colour or r["goal"].startswith("Any") or (r["goal"] == "Blue or Black" and colour in ("Blue", "Black"))]
res["encounters"].append({"context": e["context"], "color": colour, "overall": rate(A, win), "intent_matched": rate(intent, win), res["encounters"].append({"context": e["context"], "color": colour, "overall": rate(A, win), "intent_matched": rate(intent, win),
"by_stratum": {s: rate(g, win) for s, g in sorted(group_by(A, stratum).items())}}) "by_stratum": {s: rate(g, win) for s, g in sorted(group_by(A, stratum).items())}})
greens_where = next(e["where"] for e in M["encounters"] if e["colour"] == "Green") greens_where = next(e["where"] for e in M["encounters"] if e["colour"] == "Green")
greens = [r for r in runs if match(r, greens_where) and r.get("enc_Green")] greens = [r for r in runs if match(r, greens_where) and r.get("enc_Green") and not r.get("_deliberate")]
g = {"green_encounters": len(greens), g = {"green_encounters": len(greens),
"bonded_green": sum(1 for r in greens if r["enc_Green"] == "Bonded!"), "bonded_green": sum(1 for r in greens if r["enc_Green"] == "Bonded!"),
"rescued_orange": sum(1 for r in greens if r["rescued"] and r.get("outcome_color") == "Orange"), "rescued_orange": sum(1 for r in greens if r["rescued"] and r.get("outcome_color") == "Orange"),
@@ -240,7 +257,7 @@ def analyse(all_runs, filters):
first2 = lambda r: r["first_sighting"] if r["first_sighting"] in ("Red", "Brown", "Orange", "Green") else None # noqa: E731 first2 = lambda r: r["first_sighting"] if r["first_sighting"] in ("Red", "Brown", "Orange", "Green") else None # noqa: E731
known = lambda f: (lambda r: None if is_missing(r.get(f)) else r[f]) # noqa: E731 known = lambda f: (lambda r: None if is_missing(r.get(f)) else r[f]) # noqa: E731
src2 = lambda r: "Form submission" if r["entry"] == "Form" else "Bulk log (import)" # noqa: E731 src2 = lambda r: SOURCE_LABEL.get(r["source"]) or ("Form submission" if r["entry"] == "Form" else "Bulk log (import)") # noqa: E731
res["models"] = { res["models"] = {
"blue_meadow": fit_model(runs, "blue_meadow", {"source": src2, "goal": goal2, "direction": known("direction"), "blue_meadow": fit_model(runs, "blue_meadow", {"source": src2, "goal": goal2, "direction": known("direction"),
"first_sighting": first2, "resolution": known("resolution"), "company": known("company")}), "first_sighting": first2, "resolution": known("resolution"), "company": known("company")}),
@@ -257,7 +274,7 @@ def analyse(all_runs, filters):
"blue": W(bk, len(m)), "shadow": W(sk, len(t))}) "blue": W(bk, len(m)), "shadow": W(sk, len(t))})
res["source_rates"] = {key: {s: rate([r for r in g if match(r, OUTCOMES[key]["denom"])], lambda r, k=key: match(r, OUTCOMES[k]["event"])) res["source_rates"] = {key: {s: rate([r for r in g if match(r, OUTCOMES[key]["denom"])], lambda r, k=key: match(r, OUTCOMES[k]["event"]))
for s, g in group_by(runs, "entry").items()} for key in ("blue_meadow", "shadow")} for s, g in group_by(runs, entry_group).items()} for key in ("blue_meadow", "shadow")}
return res return res
@@ -302,9 +319,11 @@ def route_map(runs):
"""The route map: nodes and links with run counts, and for every target the hits and their Wilson interval.""" """The route map: nodes and links with run counts, and for every target the hits and their Wilson interval."""
nodes, links = {}, {} nodes, links = {}, {}
hits = {t: {} for t in TARGETS} hits = {t: {} for t in TARGETS}
tries = {t: {} for t in TARGETS} # runs that count for the target (no deliberate deaths for a bond)
for r in runs: for r in runs:
prev = None prev = None
hit = {t: match(r, T["event"]) for t, T in TARGETS.items()} hit = {t: match(r, T["event"]) for t, T in TARGETS.items()}
counts = {t: not (r.get("_deliberate") and t.startswith("bond_")) for t in TARGETS}
for si, s in enumerate(STAGES): for si, s in enumerate(STAGES):
v = stage_value(r, s["key"]) v = stage_value(r, s["key"])
if v is None: if v is None:
@@ -314,15 +333,18 @@ def route_map(runs):
nd["n"] += 1 nd["n"] += 1
for t in TARGETS: for t in TARGETS:
hits[t][nid] = hits[t].get(nid, 0) + hit[t] hits[t][nid] = hits[t].get(nid, 0) + hit[t]
tries[t][nid] = tries[t].get(nid, 0) + counts[t]
if prev: if prev:
lid = prev + "→" + nid lid = prev + "→" + nid
ln = links.setdefault(lid, {"id": lid, "source": prev, "target": nid, "n": 0, "chance": bool(s.get("chance"))}) ln = links.setdefault(lid, {"id": lid, "source": prev, "target": nid, "n": 0, "chance": bool(s.get("chance"))})
ln["n"] += 1 ln["n"] += 1
for t in TARGETS: for t in TARGETS:
hits[t][lid] = hits[t].get(lid, 0) + hit[t] hits[t][lid] = hits[t].get(lid, 0) + hit[t]
tries[t][lid] = tries[t].get(lid, 0) + counts[t]
prev = nid prev = nid
nl, ll = list(nodes.values()), list(links.values()) nl, ll = list(nodes.values()), list(links.values())
by_target = {t: {"nodes": [W(hits[t][n["id"]], n["n"]) for n in nl], "links": [W(hits[t][l["id"]], l["n"]) for l in ll]} for t in TARGETS} by_target = {t: {"nodes": [W(hits[t][n["id"]], tries[t][n["id"]]) for n in nl],
"links": [W(hits[t][l["id"]], tries[t][l["id"]]) for l in ll]} for t in TARGETS}
return {"nodes": nl, "links": ll, "by_target": by_target} return {"nodes": nl, "links": ll, "by_target": by_target}
@@ -551,6 +573,8 @@ def main(argv=None):
ap.add_argument("--out", default=str(ROOT / "data/public/stats.json")) ap.add_argument("--out", default=str(ROOT / "data/public/stats.json"))
a = ap.parse_args(argv) a = ap.parse_args(argv)
runs = runs_io.read(a.data, sources=M["sources"]) runs = runs_io.read(a.data, sources=M["sources"])
for r in runs:
r["_deliberate"] = deliberate(r)
out = build(runs, len(runs), exported(), every_run=runs_io.read(a.data)) out = build(runs, len(runs), exported(), every_run=runs_io.read(a.data))
Path(a.out).write_text(json.dumps(out, separators=(",", ":"), ensure_ascii=False), encoding="utf-8") Path(a.out).write_text(json.dumps(out, separators=(",", ":"), ensure_ascii=False), encoding="utf-8")
print(f"{a.out}: {len(runs)} runs, {len(out['presets'])} presets, {Path(a.out).stat().st_size / 1e6:.1f} MB") print(f"{a.out}: {len(runs)} runs, {len(out['presets'])} presets, {Path(a.out).stat().st_size / 1e6:.1f} MB")
+2 -2
View File
@@ -153,8 +153,8 @@
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).", "description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json", "contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json",
"encodingFormat": "application/json", "encodingFormat": "application/json",
"sha256": "6037558db5eaed34b625b9f482db9a107e9262b9c13451bcf14855b4addc46d3", "sha256": "89a6246da2495685733e20f2de9897299bd19d469aaeadbd8883972856787ce5",
"contentSize": "5628300 B" "contentSize": "5151910 B"
}, },
{ {
"@type": "cr:FileObject", "@type": "cr:FileObject",
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -554,7 +554,7 @@
</tbody></table></div><p class="small">Bulk logs come from players who recorded every run, so they are the least selective rows. Their rare-dragon rates are several times lower than the form's, which is the strongest sign of reporting bias in this data.</p></div> </tbody></table></div><p class="small">Bulk logs come from players who recorded every run, so they are the least selective rows. Their rare-dragon rates are several times lower than the form's, which is the strongest sign of reporting bias in this data.</p></div>
<div class="panel"><h3>Rows flagged for contradictions</h3><p class="small muted">${m.n_flagged} of ${r.n} rows (${pct(m.n_flagged / r.n)}). Flagged rows stay in unless you pick "Rows without data-quality flags" above.</p><div class="tbl-wrap"><table><tbody>${flags.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td></tr>`).join("") || `<tr><td>None</td></tr>`}</tbody></table></div></div> <div class="panel"><h3>Rows flagged for contradictions</h3><p class="small muted">${m.n_flagged} of ${r.n} rows (${pct(m.n_flagged / r.n)}). Flagged rows stay in unless you pick "Rows without data-quality flags" above.</p><div class="tbl-wrap"><table><tbody>${flags.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td></tr>`).join("") || `<tr><td>None</td></tr>`}</tbody></table></div></div>
<div class="panel"><h3>How runs ended</h3><div class="tbl-wrap"><table><tbody>${outs.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td><td class="num">${pct(n / r.n)}</td></tr>`).join("")}</tbody></table></div><p class="small muted">"No outcome recorded" rows reached the hour but the form shows no Bonded or Incinerated. They count toward reaching a branch but not toward bond rates.</p></div> <div class="panel"><h3>How runs ended</h3><div class="tbl-wrap"><table><tbody>${outs.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td><td class="num">${pct(n / r.n)}</td></tr>`).join("")}</tbody></table></div><p class="small muted">"No outcome recorded" rows reached the hour but the form shows no Bonded or Incinerated. They count toward reaching a branch but not toward bond rates.</p></div>
<div class="panel"><h3>Independence</h3><p class="small">The form has no player ID. One player's 50 runs count as 50 independent runs, so intervals are somewhat narrower than they should be. ${pct((r.n - (m.sources["Form"] || 0)) / r.n, 0)} of rows arrived as bulk imports, each probably one player's or one group's log.</p></div> <div class="panel"><h3>Independence</h3><p class="small">The form has no player ID. One player's 50 runs count as 50 independent runs, so intervals are somewhat narrower than they should be. ${pct(Object.entries(m.sources).filter(([k]) => !["Form", "Reddit comments", "DragonKind bot", "Reported on the Atlas"].includes(k)).reduce((s, [, n]) => s + n, 0) / r.n, 0)} of rows arrived as bulk imports, each probably one player's or one group's log.</p></div>
<div class="panel"><h3>What the form does not ask</h3><p class="small">The redesigned form asks how you reacted to the first dragon (answered in ${r.factors.reach_hour.rows.filter((x) => x.factor === "sighting_action" && !["Unknown", "(not asked)"].includes(x.level)).reduce((a, x) => a + x.n, 0)} rows; most riders answer "Unsure") and how the bully fight went (${r.factors.reach_hour.rows.filter((x) => x.factor === "combat_first" && x.level !== "(not recorded)").reduce((a, x) => a + x.n, 0)} rows). It still does not record most choices inside the bonding encounters. Theories that rely on those choices cannot be confirmed or refuted here.</p></div>`; <div class="panel"><h3>What the form does not ask</h3><p class="small">The redesigned form asks how you reacted to the first dragon (answered in ${r.factors.reach_hour.rows.filter((x) => x.factor === "sighting_action" && !["Unknown", "(not asked)"].includes(x.level)).reduce((a, x) => a + x.n, 0)} rows; most riders answer "Unsure") and how the bully fight went (${r.factors.reach_hour.rows.filter((x) => x.factor === "combat_first" && x.level !== "(not recorded)").reduce((a, x) => a + x.n, 0)} rows). It still does not record most choices inside the bonding encounters. Theories that rely on those choices cannot be confirmed or refuted here.</p></div>`;
} }
const sourceRate = (outKey, source) => state.res.source_rates[outKey][source]; const sourceRate = (outKey, source) => state.res.source_rates[outKey][source];