Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
cd618d37d3 | ||
|
|
0ae29b5423 | ||
|
|
0df6b8d473 | ||
|
|
46c1f00150 | ||
|
|
ae370329e3 | ||
|
|
5f080e0a82 | ||
|
|
d0141f2d9c | ||
|
|
036c86a1d1 | ||
|
|
99563a59ea | ||
|
|
d606ed1651 |
@@ -10,9 +10,11 @@
|
||||
# DK_ACCOUNT_SALT the salt account names are hashed with (the same one every time)
|
||||
# ATLAS_BOT_TOKEN as in update-sources.yml
|
||||
name: Bot cycle
|
||||
# Not scheduled: no runner has the `bot-host` label yet (issue #27), and the bot's data already arrives through the
|
||||
# daily source check. Put the schedule back once Ken adds a runner on the bot's machine:
|
||||
# schedule:
|
||||
# - cron: "41 */4 * * *"
|
||||
on:
|
||||
schedule:
|
||||
- cron: "41 */4 * * *"
|
||||
workflow_dispatch:
|
||||
jobs:
|
||||
cycle:
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
# The published Atlas page. Every push to main builds the static site (npm run build -> dist/) and force-pushes it
|
||||
# as the single commit of the `pages` branch. Ken's server serves that branch as plain files (a web server whose
|
||||
# folder is kept in step with the branch: a git-sync sidecar, or `git pull` on a timer). The repository is public,
|
||||
# so his server needs no token. Only the build output is on `pages`, never the raw data.
|
||||
#
|
||||
# Secrets: ATLAS_BOT_TOKEN (as in update-sources.yml), to push the branch.
|
||||
name: Publish the site (pages branch)
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
concurrency:
|
||||
group: pages
|
||||
cancel-in-progress: true
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 20
|
||||
- run: npm run build
|
||||
- name: Push dist/ as the pages branch
|
||||
env:
|
||||
ATLAS_BOT_TOKEN: ${{ secrets.ATLAS_BOT_TOKEN }}
|
||||
run: |
|
||||
cd dist
|
||||
git init -q -b pages
|
||||
git config user.name "Atlas data bot"
|
||||
git config user.email "atlas-bot@noreply.localhost"
|
||||
git add -A
|
||||
git commit -q -m "site built from ${GITHUB_SHA::8}"
|
||||
GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0="http.https://git.kenschiano.net/.extraheader" \
|
||||
GIT_CONFIG_VALUE_0="AUTHORIZATION: token $ATLAS_BOT_TOKEN" \
|
||||
git push -q -f "https://git.kenschiano.net/${GITHUB_REPOSITORY}.git" pages
|
||||
echo "pages branch updated from ${GITHUB_SHA::8}"
|
||||
@@ -16,6 +16,10 @@ on:
|
||||
schedule:
|
||||
- cron: "23 10 * * *" # daily
|
||||
workflow_dispatch:
|
||||
# one check at a time: a second dispatch waits instead of building and pushing alongside the first
|
||||
concurrency:
|
||||
group: update-sources
|
||||
cancel-in-progress: false
|
||||
jobs:
|
||||
check:
|
||||
runs-on: ubuntu-latest # a runner label on the server (act_runner)
|
||||
|
||||
+1
-2
@@ -166,8 +166,7 @@ venv.bak/
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs/Zensical documentation
|
||||
/site
|
||||
# (mkdocs' /site is not ignored: site/ here is the Atlas page)
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
|
||||
@@ -186,9 +186,20 @@ def build():
|
||||
("combat_second", "The dagger"), ("resolution", "The bullies' fate"), ("post_path", "The fork")):
|
||||
t = collections.Counter(r[col] for r in S1)
|
||||
L.append(f"- **{label}:** " + ", ".join(f"{k} {v}" for k, v in t.most_common()))
|
||||
outside = [r for r in runs if r["saw_shadow"] and not (r["scream"] == "Investigate & help" and r["hour"] == "Tree")]
|
||||
facts["shadow_routes"] = {"runs": len(S1), "routes": len({tuple(r[k] for k in ROUTE) for r in S1}), "off_gate": len(outside)}
|
||||
L += ["", f"Shadows reported without both gates: {len(outside)} (likely mis-entered).", ""]
|
||||
# a gate counts as missed only when the run answers it otherwise; a run that does not say (a short Reddit
|
||||
# comment, a blank form answer) is not a contradiction
|
||||
def stated(v):
|
||||
return bool(v) and v != "Unknown" and not str(v).startswith("(")
|
||||
|
||||
def contradicts(r):
|
||||
return (stated(r["scream"]) and r["scream"] != "Investigate & help") or (stated(r["hour"]) and r["hour"] != "Tree")
|
||||
outside = [r for r in runs if r["saw_shadow"] and contradicts(r)]
|
||||
unstated = [r for r in runs if r["saw_shadow"] and not contradicts(r)
|
||||
and not (r["scream"] == "Investigate & help" and r["hour"] == "Tree")]
|
||||
facts["shadow_routes"] = {"runs": len(S1), "routes": len({tuple(r[k] for k in ROUTE) for r in S1}),
|
||||
"off_gate": len(outside), "gates_unstated": len(unstated)}
|
||||
L += ["", f"Shadows reported with an answer that contradicts a gate: {len(outside)} (likely mis-entered or misread). "
|
||||
f"Another {len(unstated)} do not say whether they passed both gates.", ""]
|
||||
|
||||
# 5. after the shadow
|
||||
SH = [r for r in runs if r["saw_shadow"]]
|
||||
|
||||
@@ -56,7 +56,7 @@ The 97 form runs with a shadow (all attempts) took 47 different recorded routes:
|
||||
- **The bullies' fate:** Squadmate's revenge 61, Left bullies to the dragons 25, Killed the bullies yourself 9, Unknown 2
|
||||
- **The fork:** West 46, Forest 36, East 14, Unknown 1
|
||||
|
||||
Shadows reported without both gates: 17 (likely mis-entered).
|
||||
Shadows reported with an answer that contradicts a gate: 3 (likely mis-entered or misread). Another 14 do not say whether they passed both gates.
|
||||
|
||||
## 5. After the shadow: the Blue, then the Black
|
||||
|
||||
|
||||
@@ -6,10 +6,12 @@
|
||||
# (not blank), {any: [condition, ...]} (at least one). Several fields in one condition must all hold.
|
||||
version: 1
|
||||
|
||||
# Which runs of data/public/runs_clean.csv the statistics use (its `source` column). Runs from the other sources
|
||||
# (reported, bot, Reddit result pages) are in the published file but not in these statistics yet: they are coded
|
||||
# from different questions, and the bot follows a strategy, so pooling them needs its own review.
|
||||
sources: [reddit spreadsheet]
|
||||
# Which runs of data/public/runs_clean.csv the statistics use (its `source` column): every source that records the
|
||||
# route (the community form, "Report a run", the Reddit megathread comments, the DragonKind bot). The Reddit result
|
||||
# pages and the register record only the end, so they stay out. The bot plays to a strategy and burns on purpose at
|
||||
# encounters it is not chasing: like every rider's deliberate death (`_deliberate`, the route model's rule), those
|
||||
# burns are left out of the bond rates, never counted as failed bonds.
|
||||
sources: [reddit spreadsheet, reported, reddit comment, bot]
|
||||
|
||||
colours: [Red, Green, Orange, Brown, Blue, Black]
|
||||
|
||||
@@ -25,7 +27,7 @@ outcomes:
|
||||
event: { saw_shadow: true }
|
||||
bond_blue_meadow:
|
||||
label: Bond, given you approached the meadow Blue
|
||||
denom: { branch: "Passed the Red: meadow", enc_Blue: { in: ["Bonded!", "Incinerated"] } }
|
||||
denom: { branch: "Passed the Red: meadow", enc_Blue: { in: ["Bonded!", "Incinerated"] }, _deliberate: { not: true } }
|
||||
event: { enc_Blue: "Bonded!" }
|
||||
reach_hour:
|
||||
label: Survive to the hour (tree/valley choice)
|
||||
@@ -34,7 +36,7 @@ outcomes:
|
||||
# one more outcome per colour: "Run ends bonded to <colour>", among runs that reached the hour
|
||||
bonded_outcome:
|
||||
label: Run ends bonded to {colour}
|
||||
denom: { hour: { in: [Tree, Valley] } }
|
||||
denom: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } }
|
||||
event: { outcome: Bonded, outcome_color: "{colour}" }
|
||||
|
||||
# Factors compared in the factor tables (each answer against the other answers to the same question)
|
||||
@@ -43,15 +45,15 @@ factors: [company, direction, first_sighting, scream, resolution, combat_first,
|
||||
|
||||
# Route targets: the outcome (event) and which runs could have reached it at all (pool)
|
||||
targets:
|
||||
bond_Blue: { label: Bond Blue, colour: Blue, event: { outcome: Bonded, outcome_color: Blue }, pool: { hour: Valley } }
|
||||
bond_Black: { label: Bond Black, colour: Black, event: { outcome: Bonded, outcome_color: Black }, pool: { hour: Tree } }
|
||||
bond_Blue: { label: Bond Blue, colour: Blue, event: { outcome: Bonded, outcome_color: Blue }, pool: { hour: Valley, _deliberate: { not: true } } }
|
||||
bond_Black: { label: Bond Black, colour: Black, event: { outcome: Bonded, outcome_color: Black }, pool: { hour: Tree, _deliberate: { not: true } } }
|
||||
see_Blue: { label: Meet a Blue (meadow), colour: Blue, event: { saw_blue_meadow: true }, pool: { branch: "Passed the Red: meadow" } }
|
||||
see_shadow: { label: Get the tree shadow, colour: Black, event: { saw_shadow: true }, pool: { hour: Tree, branch: { exists: true } } }
|
||||
bond_Red: { label: Bond Red, colour: Red, event: { outcome: Bonded, outcome_color: Red }, pool: { hour: { in: [Tree, Valley] } } }
|
||||
bond_Green: { label: Bond Green, colour: Green, event: { outcome: Bonded, outcome_color: Green }, pool: { hour: Tree } }
|
||||
bond_Orange: { label: Bond Orange, colour: Orange, event: { outcome: Bonded, outcome_color: Orange }, pool: { hour: { in: [Tree, Valley] } } }
|
||||
bond_Brown: { label: Bond Brown, colour: Brown, event: { outcome: Bonded, outcome_color: Brown }, pool: { hour: { in: [Tree, Valley] } } }
|
||||
bond_any: { label: Bond any dragon, colour: null, event: { outcome: Bonded }, pool: { hour: { in: [Tree, Valley] } } }
|
||||
bond_Red: { label: Bond Red, colour: Red, event: { outcome: Bonded, outcome_color: Red }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
|
||||
bond_Green: { label: Bond Green, colour: Green, event: { outcome: Bonded, outcome_color: Green }, pool: { hour: Tree, _deliberate: { not: true } } }
|
||||
bond_Orange: { label: Bond Orange, colour: Orange, event: { outcome: Bonded, outcome_color: Orange }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
|
||||
bond_Brown: { label: Bond Brown, colour: Brown, event: { outcome: Bonded, outcome_color: Brown }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
|
||||
bond_any: { label: Bond any dragon, colour: null, event: { outcome: Bonded }, pool: { hour: { in: [Tree, Valley] }, _deliberate: { not: true } } }
|
||||
|
||||
# Bonding encounters: which recorded attempt (column) belongs to which in-game encounter (where)
|
||||
encounters:
|
||||
@@ -80,6 +82,8 @@ presets:
|
||||
all: { label: All runs, filters: { source: all, goal: all, excludeFlagged: false } }
|
||||
form: { label: Form submissions only, filters: { source: form, goal: all, excludeFlagged: false } }
|
||||
bulk: { label: Bulk logs only (imports), filters: { source: bulk, goal: all, excludeFlagged: false } }
|
||||
reddit: { label: Reddit comments only, filters: { source: reddit, goal: all, excludeFlagged: false } }
|
||||
bot: { label: DragonKind bot only, filters: { source: bot, goal: all, excludeFlagged: false } }
|
||||
not_chasing: { label: Not chasing Blue/Black, filters: { source: all, goal: exclude_rare, excludeFlagged: false } }
|
||||
chasing: { label: Chasing Blue/Black, filters: { source: all, goal: only_rare, excludeFlagged: false } }
|
||||
unflagged: { label: Rows without data-quality flags, filters: { source: all, goal: all, excludeFlagged: true } }
|
||||
|
||||
@@ -95,7 +95,18 @@ def is_rare(goal):
|
||||
return goal in M["rare_goals"]
|
||||
|
||||
|
||||
# sources other than the community spreadsheet: their own group wherever runs are split by where they came from
|
||||
SOURCE_LABEL = {"reddit comment": "Reddit comments", "bot": "DragonKind bot", "reported": "Reported on the Atlas"}
|
||||
|
||||
|
||||
def entry_group(r):
|
||||
"""Where a run came from: the spreadsheet's entry type (Form, Data Import, ...) or the source's own label."""
|
||||
return SOURCE_LABEL.get(r["source"]) or r["entry"]
|
||||
|
||||
|
||||
def stratum(r):
|
||||
if r["source"] in SOURCE_LABEL:
|
||||
return SOURCE_LABEL[r["source"]]
|
||||
if r["entry"] != "Form":
|
||||
return "Bulk logs (imports)"
|
||||
g = r["goal"]
|
||||
@@ -108,9 +119,14 @@ def goal2(r):
|
||||
|
||||
def apply_filters(runs, f):
|
||||
def ok(r):
|
||||
if f.get("source") == "form" and r["entry"] != "Form":
|
||||
sheet = r["source"] == "reddit spreadsheet"
|
||||
if f.get("source") == "form" and not (sheet and r["entry"] == "Form"):
|
||||
return False
|
||||
if f.get("source") == "bulk" and r["entry"] == "Form":
|
||||
if f.get("source") == "bulk" and not (sheet and r["entry"] != "Form"):
|
||||
return False
|
||||
if f.get("source") == "reddit" and r["source"] != "reddit comment":
|
||||
return False
|
||||
if f.get("source") == "bot" and r["source"] != "bot":
|
||||
return False
|
||||
if f.get("goal") == "exclude_rare" and is_rare(r["goal"]):
|
||||
return False
|
||||
@@ -197,7 +213,7 @@ def analyse(all_runs, filters):
|
||||
dates = sorted(r["date"] for r in runs if r.get("date"))
|
||||
outs = group_by(runs, lambda r: r["outcome"] + (" " + r["outcome_color"] if r.get("outcome_color") else ""))
|
||||
res["meta"] = {
|
||||
"sources": {k: len(v) for k, v in group_by(runs, "entry").items()},
|
||||
"sources": {k: len(v) for k, v in group_by(runs, entry_group).items()},
|
||||
"date_min": dates[0] if dates else None, "date_max": dates[-1] if dates else None,
|
||||
"n_flagged": sum(1 for r in runs if r["data_flags"]), "flag_counts": flag_counts,
|
||||
"outcomes": {k: len(v) for k, v in outs.items()},
|
||||
@@ -220,13 +236,14 @@ def analyse(all_runs, filters):
|
||||
res["encounters"] = []
|
||||
for e in M["encounters"]:
|
||||
col, colour = e["column"], e["colour"]
|
||||
A = [r for r in runs if match(r, e["where"]) and r.get(col) in ("Bonded!", "Incinerated")]
|
||||
# a rider who chose to burn here (chasing another colour) did not fail this encounter
|
||||
A = [r for r in runs if match(r, e["where"]) and r.get(col) in ("Bonded!", "Incinerated") and not r.get("_deliberate")]
|
||||
win = lambda r, col=col: r.get(col) == "Bonded!" # noqa: E731
|
||||
intent = [r for r in A if r["goal"] == colour or r["goal"].startswith("Any") or (r["goal"] == "Blue or Black" and colour in ("Blue", "Black"))]
|
||||
res["encounters"].append({"context": e["context"], "color": colour, "overall": rate(A, win), "intent_matched": rate(intent, win),
|
||||
"by_stratum": {s: rate(g, win) for s, g in sorted(group_by(A, stratum).items())}})
|
||||
greens_where = next(e["where"] for e in M["encounters"] if e["colour"] == "Green")
|
||||
greens = [r for r in runs if match(r, greens_where) and r.get("enc_Green")]
|
||||
greens = [r for r in runs if match(r, greens_where) and r.get("enc_Green") and not r.get("_deliberate")]
|
||||
g = {"green_encounters": len(greens),
|
||||
"bonded_green": sum(1 for r in greens if r["enc_Green"] == "Bonded!"),
|
||||
"rescued_orange": sum(1 for r in greens if r["rescued"] and r.get("outcome_color") == "Orange"),
|
||||
@@ -240,7 +257,7 @@ def analyse(all_runs, filters):
|
||||
|
||||
first2 = lambda r: r["first_sighting"] if r["first_sighting"] in ("Red", "Brown", "Orange", "Green") else None # noqa: E731
|
||||
known = lambda f: (lambda r: None if is_missing(r.get(f)) else r[f]) # noqa: E731
|
||||
src2 = lambda r: "Form submission" if r["entry"] == "Form" else "Bulk log (import)" # noqa: E731
|
||||
src2 = lambda r: SOURCE_LABEL.get(r["source"]) or ("Form submission" if r["entry"] == "Form" else "Bulk log (import)") # noqa: E731
|
||||
res["models"] = {
|
||||
"blue_meadow": fit_model(runs, "blue_meadow", {"source": src2, "goal": goal2, "direction": known("direction"),
|
||||
"first_sighting": first2, "resolution": known("resolution"), "company": known("company")}),
|
||||
@@ -257,7 +274,7 @@ def analyse(all_runs, filters):
|
||||
"blue": W(bk, len(m)), "shadow": W(sk, len(t))})
|
||||
|
||||
res["source_rates"] = {key: {s: rate([r for r in g if match(r, OUTCOMES[key]["denom"])], lambda r, k=key: match(r, OUTCOMES[k]["event"]))
|
||||
for s, g in group_by(runs, "entry").items()} for key in ("blue_meadow", "shadow")}
|
||||
for s, g in group_by(runs, entry_group).items()} for key in ("blue_meadow", "shadow")}
|
||||
return res
|
||||
|
||||
|
||||
@@ -302,9 +319,11 @@ def route_map(runs):
|
||||
"""The route map: nodes and links with run counts, and for every target the hits and their Wilson interval."""
|
||||
nodes, links = {}, {}
|
||||
hits = {t: {} for t in TARGETS}
|
||||
tries = {t: {} for t in TARGETS} # runs that count for the target (no deliberate deaths for a bond)
|
||||
for r in runs:
|
||||
prev = None
|
||||
hit = {t: match(r, T["event"]) for t, T in TARGETS.items()}
|
||||
counts = {t: not (r.get("_deliberate") and t.startswith("bond_")) for t in TARGETS}
|
||||
for si, s in enumerate(STAGES):
|
||||
v = stage_value(r, s["key"])
|
||||
if v is None:
|
||||
@@ -314,15 +333,18 @@ def route_map(runs):
|
||||
nd["n"] += 1
|
||||
for t in TARGETS:
|
||||
hits[t][nid] = hits[t].get(nid, 0) + hit[t]
|
||||
tries[t][nid] = tries[t].get(nid, 0) + counts[t]
|
||||
if prev:
|
||||
lid = prev + "→" + nid
|
||||
ln = links.setdefault(lid, {"id": lid, "source": prev, "target": nid, "n": 0, "chance": bool(s.get("chance"))})
|
||||
ln["n"] += 1
|
||||
for t in TARGETS:
|
||||
hits[t][lid] = hits[t].get(lid, 0) + hit[t]
|
||||
tries[t][lid] = tries[t].get(lid, 0) + counts[t]
|
||||
prev = nid
|
||||
nl, ll = list(nodes.values()), list(links.values())
|
||||
by_target = {t: {"nodes": [W(hits[t][n["id"]], n["n"]) for n in nl], "links": [W(hits[t][l["id"]], l["n"]) for l in ll]} for t in TARGETS}
|
||||
by_target = {t: {"nodes": [W(hits[t][n["id"]], tries[t][n["id"]]) for n in nl],
|
||||
"links": [W(hits[t][l["id"]], tries[t][l["id"]]) for l in ll]} for t in TARGETS}
|
||||
return {"nodes": nl, "links": ll, "by_target": by_target}
|
||||
|
||||
|
||||
@@ -550,8 +572,12 @@ def main(argv=None):
|
||||
ap.add_argument("--data", default=str(runs_io.PATH))
|
||||
ap.add_argument("--out", default=str(ROOT / "data/public/stats.json"))
|
||||
a = ap.parse_args(argv)
|
||||
runs = runs_io.read(a.data, sources=M["sources"])
|
||||
out = build(runs, len(runs), exported(), every_run=runs_io.read(a.data))
|
||||
# a run that cannot have happened as recorded (core/spec/form.yaml impossible_flags) is left out, as in the model
|
||||
runs = [r for r in runs_io.read(a.data, sources=M["sources"]) if not runs_io.impossible(r)]
|
||||
for r in runs:
|
||||
r["_deliberate"] = deliberate(r)
|
||||
every = [r for r in runs_io.read(a.data) if not runs_io.impossible(r)]
|
||||
out = build(runs, len(runs), exported(), every_run=every)
|
||||
Path(a.out).write_text(json.dumps(out, separators=(",", ":"), ensure_ascii=False), encoding="utf-8")
|
||||
print(f"{a.out}: {len(runs)} runs, {len(out['presets'])} presets, {Path(a.out).stat().st_size / 1e6:.1f} MB")
|
||||
|
||||
|
||||
@@ -112,10 +112,13 @@ def build(dictionary=None):
|
||||
"citeAs": ds["citation"].format(repository=forge.repo_url(fg)),
|
||||
"isLiveDataset": True,
|
||||
"isBasedOn": [
|
||||
{"@type": "sc:CreativeWork", "name": s["label"], "url": next(iter(s["links"].values()))}
|
||||
{"@type": "sc:CreativeWork", "url": s["links"][k],
|
||||
"name": s["label"] + (f" ({k.replace('part', 'part ')})" if len(s.get("cite") or ()) > 1 else "")}
|
||||
for s in sources.values()
|
||||
# the route PDF is only fingerprinted, never redistributed; the bot's repo is private
|
||||
if (s.get("privacy") or {}).get("redistribute", True) and s.get("links")
|
||||
# `cite` (collection/pipeline.yaml) names the links the source is based on; default: the first link
|
||||
for k in (s.get("cite") or [next(iter(s["links"]))])
|
||||
],
|
||||
"distribution": distribution,
|
||||
"recordSet": [
|
||||
|
||||
@@ -52,7 +52,8 @@ def encounter_columns(row, intentional=False):
|
||||
# ---------------------------------------------------------------- coders, by pipeline.yaml `runs:`
|
||||
|
||||
def spreadsheet(name, spec):
|
||||
"""The community form: core/spec/form.yaml + core/atlas/coding.py, unchanged."""
|
||||
"""The community form: core/spec/form.yaml + core/atlas/coding.py, plus the game-rule checks every source that
|
||||
answers the form's questions gets (rule_flags; a flag the coder already set is not repeated)."""
|
||||
from atlas import coding
|
||||
df, missing = coding.code_frame(coding.read_export(path(spec["load"])))
|
||||
if missing:
|
||||
@@ -62,7 +63,10 @@ def spreadsheet(name, spec):
|
||||
for rec in df.astype(object).where(df.notna(), None).to_dict("records"):
|
||||
n = rec.pop("run_id")
|
||||
entry = rec.pop("source")
|
||||
rows.append({**{c: None for c in COLUMNS}, **rec, "run_id": f"sheet-{n}", "source": spec["run_source"], "entry": entry})
|
||||
row = {**{c: None for c in COLUMNS}, **rec, "run_id": f"sheet-{n}", "source": spec["run_source"], "entry": entry}
|
||||
have = [f for f in (row["data_flags"] or "").split("; ") if f]
|
||||
row["data_flags"] = "; ".join(have + [f for f in rule_flags(row) if f not in have])
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
@@ -231,6 +235,8 @@ def rule_flags(row):
|
||||
flags.append(f"{colour} at the {branch} encounter, which has only {'/'.join(sorted(enc_colours[branch]))}")
|
||||
if row["hour"] == "Valley" and colour == "Green":
|
||||
flags.append("Green recorded on a Valley run (no Greens in the valley)")
|
||||
if row["hour"] == "Valley" and colour == "Black":
|
||||
flags.append("Black recorded on a Valley run (the Black is only reached from the tree)")
|
||||
scream, res = row["scream"], row["resolution"]
|
||||
if scream in ("Investigate & help", "Ignore the scream", "Investigate & leave"):
|
||||
if res in CADET_PATH and CADET_PATH[res] != scream:
|
||||
|
||||
@@ -57,6 +57,7 @@ sources:
|
||||
runs: null # a flowchart of the routes, not a log of runs (see collection/README.md)
|
||||
|
||||
reddit_thread:
|
||||
cite: [part3, part4] # both megathreads, in the dataset's isBasedOn
|
||||
label: Runs posted in the r/fourthwing Dragonkind megathreads
|
||||
run_source: reddit thread
|
||||
links:
|
||||
@@ -70,6 +71,7 @@ sources:
|
||||
runs: reddit_results
|
||||
|
||||
reddit_comments:
|
||||
cite: [part3, part4] # both megathreads, in the dataset's isBasedOn
|
||||
label: Runs riders describe in the r/fourthwing Dragonkind megathread comments
|
||||
run_source: reddit comment
|
||||
links:
|
||||
|
||||
@@ -25,6 +25,13 @@ def _parse(col, v):
|
||||
return v
|
||||
|
||||
|
||||
def impossible(r):
|
||||
"""A run that cannot have happened as recorded: one of its flags is in core/spec/form.yaml impossible_flags."""
|
||||
from . import spec
|
||||
rules = spec.form()["impossible_flags"]
|
||||
return any(rule in f for f in (r.get("data_flags") or "").split("; ") for rule in rules)
|
||||
|
||||
|
||||
def read(path=PATH, sources=None, drop=("run_id",)):
|
||||
"""Typed records, optionally only from some sources (the `source` column)."""
|
||||
with open(path, encoding="utf-8", newline="") as f:
|
||||
|
||||
@@ -156,6 +156,26 @@ sighting_pool:
|
||||
# The dragons each bonding encounter can end with (bond or burn), from the community PDF. Only used to FLAG runs
|
||||
# answered through "Report a run" (reported, reddit comment) that end with a dragon the encounter does not have.
|
||||
# Orange and Brown after the two Greens: a Green's rescue, then the Orange or Brown.
|
||||
# Flags (data_flags, matched as text inside a flag) that mean a run cannot have happened as recorded: the game's own
|
||||
# rules, from the form coder (core/atlas/coding.py) and the checks of collection/etl/runs.py rule_flags. A run with one
|
||||
# is kept in runs_clean.csv, flagged, and left out of the route model and the site statistics (atlas.runs.impossible).
|
||||
# Everything else, however partial, is used. Not here on purpose: a first sighting outside the documented pool (the
|
||||
# pool is what riders have seen, not a rule), a missing direction, several wings, a missing Green before a rescue.
|
||||
impossible_flags:
|
||||
- "Green recorded on a Valley run"
|
||||
- "Black recorded on a Valley run"
|
||||
- "encounter, which has only"
|
||||
- "more than one dragon marked Bonded"
|
||||
- "more than one dragon marked Incinerated"
|
||||
- "tree run with both the Red and the two Greens attempted"
|
||||
- "died in combat on a path without combat"
|
||||
- "rare-dragon answer does not match the path"
|
||||
- "does not follow"
|
||||
- "fight answers on a run that did not help"
|
||||
- "a path after the fight on a run without a fight"
|
||||
- "a second fight move without pausing to assess"
|
||||
- "died before the hour, yet reached"
|
||||
|
||||
encounter_colours:
|
||||
"Shadow (rare)": [Blue, Black]
|
||||
"Descend quickly: Red": [Red]
|
||||
|
||||
+24
-14
@@ -66,23 +66,33 @@
|
||||
"isBasedOn": [
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"name": "Threshing Routes form and spreadsheet (u/ClumsyBadger, linked from the r/fourthwing megathreads)",
|
||||
"url": "https://docs.google.com/spreadsheets/d/1gnw1AdRg1XT4JsOfRM6QTA0jSaS3n9ZF-rPWnYewzv0"
|
||||
"url": "https://docs.google.com/spreadsheets/d/1gnw1AdRg1XT4JsOfRM6QTA0jSaS3n9ZF-rPWnYewzv0",
|
||||
"name": "Threshing Routes form and spreadsheet (u/ClumsyBadger, linked from the r/fourthwing megathreads)"
|
||||
},
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"name": "Unofficial Dragon Register (which dragon each rider bonded)",
|
||||
"url": "https://docs.google.com/spreadsheets/d/1KMiE9sec0l597Pzue1OhGYf07NF4howtQ_lPeqx7jXQ"
|
||||
"url": "https://docs.google.com/spreadsheets/d/1KMiE9sec0l597Pzue1OhGYf07NF4howtQ_lPeqx7jXQ",
|
||||
"name": "Unofficial Dragon Register (which dragon each rider bonded)"
|
||||
},
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"name": "Runs posted in the r/fourthwing Dragonkind megathreads",
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/"
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/",
|
||||
"name": "Runs posted in the r/fourthwing Dragonkind megathreads (part 3)"
|
||||
},
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments",
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/"
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wz5cnj/",
|
||||
"name": "Runs posted in the r/fourthwing Dragonkind megathreads (part 4)"
|
||||
},
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wvwfp9/",
|
||||
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments (part 3)"
|
||||
},
|
||||
{
|
||||
"@type": "sc:CreativeWork",
|
||||
"url": "https://www.reddit.com/r/fourthwing/comments/1wz5cnj/",
|
||||
"name": "Runs riders describe in the r/fourthwing Dragonkind megathread comments (part 4)"
|
||||
}
|
||||
],
|
||||
"distribution": [
|
||||
@@ -93,8 +103,8 @@
|
||||
"description": "Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's rules are core/spec/form.yaml). The `source` column says where each run came from.",
|
||||
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/runs_clean.csv",
|
||||
"encodingFormat": "text/csv",
|
||||
"sha256": "657da20e614160d2a44d8854937270c7867fb9ae470509de1e12516f07d2af23",
|
||||
"contentSize": "3748021 B"
|
||||
"sha256": "b5e121cf5deab5f8bc1420b34a5bc434d6f933c7452e200eed4891c35de45122",
|
||||
"contentSize": "3748768 B"
|
||||
},
|
||||
{
|
||||
"@type": "cr:FileObject",
|
||||
@@ -143,8 +153,8 @@
|
||||
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
|
||||
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/stats.json",
|
||||
"encodingFormat": "application/json",
|
||||
"sha256": "bb401225065ae17311bbd02fd63acd4010f4e325d7eab74e044115b98d04bec4",
|
||||
"contentSize": "5498476 B"
|
||||
"sha256": "89a6246da2495685733e20f2de9897299bd19d469aaeadbd8883972856787ce5",
|
||||
"contentSize": "5151910 B"
|
||||
},
|
||||
{
|
||||
"@type": "cr:FileObject",
|
||||
@@ -153,8 +163,8 @@
|
||||
"description": "The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).",
|
||||
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/policy.json",
|
||||
"encodingFormat": "application/json",
|
||||
"sha256": "f65aaebb23cb3f56dad0387a94113857eb44c4ba077cfe5fc6215c202546588c",
|
||||
"contentSize": "987411 B"
|
||||
"sha256": "eec67140c63f028d31553be4f2310d92c9378293488aecd73a41ba1700b9e646",
|
||||
"contentSize": "987412 B"
|
||||
},
|
||||
{
|
||||
"@type": "cr:FileObject",
|
||||
|
||||
@@ -116,7 +116,8 @@
|
||||
"shadow_routes": {
|
||||
"runs": 97,
|
||||
"routes": 47,
|
||||
"off_gate": 17
|
||||
"off_gate": 3,
|
||||
"gates_unstated": 14
|
||||
},
|
||||
"after_shadow": {
|
||||
"reached_black": 89,
|
||||
|
||||
File diff suppressed because one or more lines are too long
+10
-10
@@ -1181,7 +1181,7 @@ sheet-1135,reddit spreadsheet,Form,,2026-10-09,First,Tail,Third,4,Later attempt,
|
||||
sheet-1136,reddit spreadsheet,Form,,2026-10-09,Multiple,,Second,,Later attempt,Blue or Black,Alone,False,East,Orange,Unknown,Investigate & help,Left bullies to the dragons,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,multiple wings,jump on its back,
|
||||
sheet-1137,reddit spreadsheet,Form,,2026-10-09,Third,Claw,First,1,First attempt,Any (for fun),Alone,False,Center,Green,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Tree,Descend slowly: two Greens,,Incinerated,,,,,Incinerated,Green,False,False,False,No,Unknown,,first sighting Green not in documented pool for Alone/Center,looking down and letting them choose,
|
||||
sheet-1138,reddit spreadsheet,Form,,2026-10-09,Third,Flame,Third,,Unknown,Blue or Black,Alone,False,East,Orange,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,jump on its back,
|
||||
sheet-1139,reddit spreadsheet,Form,,2026-10-09,Second,Claw,Third,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Ignore the scream,Took their weapons,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Unknown,,Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated,Ran,
|
||||
sheet-1139,reddit spreadsheet,Form,,2026-10-09,Second,Claw,Third,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Ignore the scream,Took their weapons,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Unknown,,"Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated; Green at the Passed the Red: meadow encounter, which has only Blue/Brown/Orange",Ran,
|
||||
sheet-1140,reddit spreadsheet,Form,,2026-10-09,First,Tail,Second,,Later attempt,Blue or Black,Alone,False,Center,Red,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,Descended quickly > jumped on its back > lowered gaze,
|
||||
sheet-1141,reddit spreadsheet,Form,,2026-10-09,Second,Tail,Third,,Later attempt,Blue or Black,Alone,False,Center,Red,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,Run away - Trying to bond a blue here,
|
||||
sheet-1142,reddit spreadsheet,Form,,2026-10-09,Second,Tail,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,Jump now onto the red - Trying to bond a blue here,
|
||||
@@ -1668,7 +1668,7 @@ sheet-1586,reddit spreadsheet,Form,,2026-10-14,First,Tail,Third,,Later attempt,A
|
||||
sheet-1587,reddit spreadsheet,Form,,2026-10-14,First,Tail,Third,1,First attempt,Any (for fun),Alone,False,Center,Red,Unknown,Investigate & help,Left bullies to the dragons,Even the numbers (smaller cadet),(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,Roar,
|
||||
sheet-1588,reddit spreadsheet,Form,,2026-10-14,Second,Claw,Second,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,,,Incinerated,,Incinerated,Blue,False,True,False,Blue (valley),No,,,Hold glare and watch,
|
||||
sheet-1589,reddit spreadsheet,Form,,2026-10-14,Fourth,Claw,First,,Later attempt,Blue or Black,Alone,False,Center,Red,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,try brown instead,
|
||||
sheet-1590,reddit spreadsheet,Form,,2026-10-14,First,Tail,Third,,Later attempt,Blue or Black,Alone,False,East,Red,Unknown,Ignore the scream,Took their weapons,(not recorded),(not recorded),(no fork),Tree,Descend slowly: two Greens,Incinerated,Changed my mind,,,,,Incinerated,Red,False,False,False,No,Unknown,,tree run with both the Red and the two Greens attempted,Jumping beside Red's tail,
|
||||
sheet-1590,reddit spreadsheet,Form,,2026-10-14,First,Tail,Third,,Later attempt,Blue or Black,Alone,False,East,Red,Unknown,Ignore the scream,Took their weapons,(not recorded),(not recorded),(no fork),Tree,Descend slowly: two Greens,Incinerated,Changed my mind,,,,,Incinerated,Red,False,False,False,No,Unknown,,"tree run with both the Red and the two Greens attempted; Red at the Descend slowly: two Greens encounter, which has only Brown/Green/Orange",Jumping beside Red's tail,
|
||||
sheet-1591,reddit spreadsheet,Form,,2026-10-14,Fourth,Claw,Second,31,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,,,,Incinerated,,Incinerated,Blue,False,True,False,Blue (valley),No,,,"Died doing nothing when blue grabbed the cadet, followed right path and still died (again)",
|
||||
sheet-1592,reddit spreadsheet,Form,,2026-10-14,First,Claw,First,,Later attempt,Blue or Black,Alone,False,West,Brown,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,jumped on its back,
|
||||
sheet-1593,reddit spreadsheet,Form,,2026-10-14,Third,Flame,First,,Later attempt,Blue or Black,Alone,False,West,Brown,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,"successfully landed on its back, but then lowered eyes and was still scorched",
|
||||
@@ -1707,7 +1707,7 @@ sheet-1625,reddit spreadsheet,Form,,2026-10-14,Third,Claw,Second,,Later attempt,
|
||||
sheet-1626,reddit spreadsheet,Form,,2026-10-14,Third,Claw,Second,,Later attempt,Blue or Black,Alone,False,East,Red,Unknown,Unknown,Unknown,(not recorded),(not recorded),,Valley,Passed the Red: meadow,,,,,Bonded!,,Bonded,Blue,False,True,False,Blue (valley),Unknown,,,,Absolutely- hold glare and advance- do nothing- hold glare and watch- straighten- launch yourself- bring down- move slowly
|
||||
sheet-1627,reddit spreadsheet,Form,,2026-10-14,Second,Claw,Don't know,1,First attempt,Any (for fun),Alone,False,Center,Red,Unknown,Investigate & help,DIED in combat,Put yourself in the middle (bigger cadet),(not recorded),,,,,,,,,,Died before the hour,,False,False,False,No,Unknown,,,,
|
||||
sheet-1628,reddit spreadsheet,Form,,2026-10-14,Second,Claw,Don't know,2,Later attempt,Any (for fun),Alone,False,Center,Red,Unknown,Investigate & help,Left bullies to the dragons,Assess the situation,(not recorded),Forest,Valley,Valley Red,Bonded!,,,,,,Bonded,Red,False,False,False,No,No,,,,
|
||||
sheet-1629,reddit spreadsheet,Form,,2026-10-14,First,Flame,Second,,Later attempt,Any (for fun),Alone,False,Center,Unsure,Unknown,Ignore the scream,Left injured cadet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Unknown,,direction taken from the older sighting column; Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated,Running away,
|
||||
sheet-1629,reddit spreadsheet,Form,,2026-10-14,First,Flame,Second,,Later attempt,Any (for fun),Alone,False,Center,Unsure,Unknown,Ignore the scream,Left injured cadet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Unknown,,"direction taken from the older sighting column; Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated; Green at the Passed the Red: meadow encounter, which has only Blue/Brown/Orange",Running away,
|
||||
sheet-1630,reddit spreadsheet,Form,,2026-10-14,Fourth,Tail,First,2,Later attempt,Blue or Black,Alone,False,East,Red,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Tree,Descend slowly: two Greens,,Incinerated,,,,,Incinerated,Green,False,False,False,No,Unknown,,,I turned left,
|
||||
sheet-1631,reddit spreadsheet,Form,,2026-10-14,First,Tail,Second,,Later attempt,Any (for fun),Alone,False,Center,Orange,Unknown,Ignore the scream,Left injured cadet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,,,Incinerated,,,Incinerated,Brown,False,False,False,No,Unknown,,first sighting Orange not in documented pool for Alone/Center,"Dragon flicked mud at me, I chose to flick it back",
|
||||
sheet-1632,reddit spreadsheet,Form,,2026-10-14,Third,Tail,Third,,Later attempt,Blue or Black,Unknown,False,East,Orange,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),East,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,ran. no blue,
|
||||
@@ -1995,7 +1995,7 @@ sheet-1892,reddit spreadsheet,Form,,2026-10-18,Third,Flame,First,,Later attempt,
|
||||
sheet-1893,reddit spreadsheet,Form,,2026-10-18,Fourth,Claw,Third,5,Later attempt,Any (for fun),Alone,False,East,Red,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),East,Valley,Passed the Red: meadow,,,Bonded!,,,,Bonded,Orange,False,False,False,No,No,,,,"Do not approach the red dragon who incinerated someone else. When you get to choose Brown vs Orange, choose Orange. He will roar at you. Roar back! (Really!) He will then spread his talons. Take it as an invitation to mount and you will bond :)"
|
||||
sheet-1894,reddit spreadsheet,Form,,2026-10-18,Second,Claw,First,,Later attempt,Blue or Black,Alone,False,Center,Red,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,,,Incinerated,,Incinerated,Blue,False,True,False,Blue (valley),No,,,"Hold glare and advance, do nothing, hold glare and watch, straighten spine, instantly incinerated",
|
||||
sheet-1895,reddit spreadsheet,Form,,2026-10-18,First,Don't know,Don't know,,Later attempt,Blue or Black,Alone,False,Center,Red,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,,,Saved by another,,,,,No outcome recorded,,False,False,False,No,Unknown,,Green recorded on a Valley run (no Greens in the valley),,"Told the two greens (took the left one) that I'm always right even if I'm not, green tried to burn me, orange swept in"
|
||||
sheet-1896,reddit spreadsheet,Form,,2026-10-18,Second,Claw,Third,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),East,Valley,Valley Red,Incinerated,Incinerated,,,,,Incinerated,Green,False,False,False,No,Unknown,,Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated,lowered eyes,
|
||||
sheet-1896,reddit spreadsheet,Form,,2026-10-18,Second,Claw,Third,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),East,Valley,Valley Red,Incinerated,Incinerated,,,,,Incinerated,Green,False,False,False,No,Unknown,,"Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated; Green at the Valley Red encounter, which has only Red",lowered eyes,
|
||||
sheet-1897,reddit spreadsheet,Form,,2026-10-18,Fourth,Flame,First,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Investigate & help,Killed the bullies yourself,Even the numbers (smaller cadet),(not recorded),East,Valley,Passed the Red: meadow,,,,,Bonded!,,Bonded,Blue,False,True,False,Blue (valley),Unknown,,,,Followed the Google PDF for the Blue
|
||||
sheet-1898,reddit spreadsheet,Form,,2026-10-18,Fourth,Claw,First,,Later attempt,Blue or Black,Alone,False,East,Orange,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),East,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,nvm go for brown,
|
||||
sheet-1899,reddit spreadsheet,Form,,2026-10-18,First,Claw,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Ignore the scream,Left injured cadet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,,,,Bonded!,,Bonded,Blue,False,True,False,Blue (valley),Unknown,,,,Followed the pdf dark shaded route exactly (absolutely>hold glare>do nothing>hold glare and watch>straighten>launch right>bring down an empire>move slowly forward> bonded!!!!)
|
||||
@@ -2031,7 +2031,7 @@ sheet-1928,reddit spreadsheet,Form,,2026-10-19,First,Flame,Third,,Later attempt,
|
||||
sheet-1929,reddit spreadsheet,Form,,2026-10-19,Fourth,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
sheet-1930,reddit spreadsheet,Form,,2026-10-19,First,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),East,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
sheet-1931,reddit spreadsheet,Form,,2026-10-19,Fourth,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
sheet-1932,reddit spreadsheet,Form,,2026-10-19,Fourth,Claw,First,5,Later attempt,Any (for fun),Alone,False,East,Orange,Unknown,Investigate & leave,Left them to it,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Bonded!,,Changed my mind,,,Bonded,Green,False,False,False,No,No,,Green recorded on a Valley run (no Greens in the valley),,"Mount on foreleg, hit the ground and hoped its claw misses"
|
||||
sheet-1932,reddit spreadsheet,Form,,2026-10-19,Fourth,Claw,First,5,Later attempt,Any (for fun),Alone,False,East,Orange,Unknown,Investigate & leave,Left them to it,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Bonded!,,Changed my mind,,,Bonded,Green,False,False,False,No,No,,"Green recorded on a Valley run (no Greens in the valley); Green at the Passed the Red: meadow encounter, which has only Blue/Brown/Orange",,"Mount on foreleg, hit the ground and hoped its claw misses"
|
||||
sheet-1933,reddit spreadsheet,Form,,2026-10-19,First,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
sheet-1934,reddit spreadsheet,Form,,2026-10-19,Fourth,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
sheet-1935,reddit spreadsheet,Form,,2026-10-19,Fourth,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Unknown,,,run! intentional death,
|
||||
@@ -2350,7 +2350,7 @@ sheet-2238,reddit spreadsheet,Form,,2026-10-21,First,Flame,Second,2,Later attemp
|
||||
sheet-2239,reddit spreadsheet,Form,,2026-10-21,Fourth,Claw,First,,Later attempt,Blue or Black,Alone,False,East,Orange,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Yes,,,a nice brown,
|
||||
sheet-2240,reddit spreadsheet,Form,,2026-10-21,First,Tail,Second,,Later attempt,Blue or Black,1 Squadmate,,East,Orange,Unknown,Investigate & help,Left bullies to the dragons,Assess the situation,(not recorded),West,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,No,,,"jump on tail, dead",
|
||||
sheet-2241,reddit spreadsheet,Form,,2026-10-21,First,Tail,Third,,Later attempt,Any (for fun),Alone,False,West,Green,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),West,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,No,,,There were two dragons a brown one and an orange one. First I tried to bond with the brown but it went away and when I tried to bond the orange I got incinerated,
|
||||
sheet-2242,reddit spreadsheet,Form,,2026-10-21,Second,Flame,Second,2,Later attempt,Any (for fun),Alone,False,West,Brown,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,Bonded!,,,Changed my mind,,,Bonded,Red,False,False,False,No,No,,,,"I approached red dragon, then got a whole lot of: are you sure? Are you sure? Are you sure. With the option to run."
|
||||
sheet-2242,reddit spreadsheet,Form,,2026-10-21,Second,Flame,Second,2,Later attempt,Any (for fun),Alone,False,West,Brown,Unknown,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,Bonded!,,,Changed my mind,,,Bonded,Red,False,False,False,No,No,,"Red at the Passed the Red: meadow encounter, which has only Blue/Brown/Orange",,"I approached red dragon, then got a whole lot of: are you sure? Are you sure? Are you sure. With the option to run."
|
||||
sheet-2243,reddit spreadsheet,Form,,2026-10-21,Third,Flame,Second,,Later attempt,Blue or Black,Alone,False,West,Green,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped on its back,
|
||||
sheet-2244,reddit spreadsheet,Form,,2026-10-21,First,Tail,Third,,Later attempt,Blue or Black,Alone,False,West,Brown,Unknown,Investigate & help,Killed the bullies yourself,Assess the situation,(not recorded),Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped on its back,
|
||||
sheet-2245,reddit spreadsheet,Form,,2026-10-21,Third,Tail,Second,1,First attempt,Any (for fun),1 Squadmate,,West,Red,Unknown,Investigate & help,Squadmate's revenge,Assess the situation,(not recorded),Forest,Valley,Passed the Red: meadow,,,Bonded!,,,,Bonded,Orange,False,False,False,No,No,,,,"Roared right back at it, charged with intent to mount, bonded"
|
||||
@@ -4691,7 +4691,7 @@ sheet-4568,reddit spreadsheet,Form,,2026-11-03,Second,Flame,Third,2,Later attemp
|
||||
sheet-4569,reddit spreadsheet,Form,,2026-11-03,Third,Tail,Third,1,First attempt,Blue or Black,Alone,False,Center,Brown,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Shadow (rare),,,,,,Incinerated,Incinerated,Black,False,False,True,Shadow,Yes,,,Lower eyes > Dead,
|
||||
sheet-4570,reddit spreadsheet,Form,,2026-11-03,Third,Tail,Third,2,Later attempt,Blue or Black,Alone,False,Center,Red,Hide,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Tail > Gaze > Dead,
|
||||
sheet-4571,reddit spreadsheet,Form,,2026-11-03,Second,Claw,Second,1,First attempt,Blue or Black,Alone,False,West,Brown,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Shadow (rare),,,,,Incinerated,,Incinerated,Blue,False,False,True,Shadow,Yes,,,Follow Shadow > Approach Blue > Sure > Dead,
|
||||
sheet-4572,reddit spreadsheet,Form,,2026-11-03,First,Claw,Third,1,First attempt,Blue or Black,Alone,False,Center,Red,Hide,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Shadow (rare),,,Changed my mind,Incinerated,Changed my mind,,Incinerated,Brown,False,False,True,Shadow,Yes,,,Follow shadow > Last Chance to escape Blue > Valley > Orange > Brown > Change > Dead,
|
||||
sheet-4572,reddit spreadsheet,Form,,2026-11-03,First,Claw,Third,1,First attempt,Blue or Black,Alone,False,Center,Red,Hide,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Shadow (rare),,,Changed my mind,Incinerated,Changed my mind,,Incinerated,Brown,False,False,True,Shadow,Yes,,"Brown at the Shadow (rare) encounter, which has only Black/Blue",Follow shadow > Last Chance to escape Blue > Valley > Orange > Brown > Change > Dead,
|
||||
sheet-4573,reddit spreadsheet,Form,,2026-11-03,Third,Flame,First,,Later attempt,Blue or Black,Alone,False,East,Red,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Valley,Passed the Red: meadow,,,,,Bonded!,,Bonded,Blue,False,True,False,Blue (valley),Yes,Clubtail,,,
|
||||
sheet-4574,reddit spreadsheet,Form,,2026-11-03,Third,Claw,Third,,Later attempt,Daily run data,Alone,False,Center,Brown,Respect / face it,Investigate & help,Left bullies to the dragons,Assess the situation,Daggers or not (protect squadmate),East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,,
|
||||
sheet-4575,reddit spreadsheet,Form,,2026-11-03,Second,Tail,Third,,Later attempt,Daily run data,Alone,False,East,Red,Respect / face it,Investigate & help,Left bullies to the dragons,Assess the situation,Engage the smaller cadet,West,Tree,Descend quickly: Red,Bonded!,,,,,,Bonded,Red,False,False,False,No,No,Swordtail,,,"Tail, gaze, stomach, crawl"
|
||||
@@ -5129,7 +5129,7 @@ sheet-5006,reddit spreadsheet,Form,,2026-11-05,First,Claw,First,1,First attempt,
|
||||
sheet-5007,reddit spreadsheet,Form,,2026-11-05,Fourth,Tail,First,32,Later attempt,Blue or Black,Alone,False,Center,Brown,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped on the red's back - Meet gaze - Drop (died),
|
||||
sheet-5008,reddit spreadsheet,Form,,2026-11-05,Third,Tail,Second,,Later attempt,Daily run data,Alone,False,East,Orange,Retreat / run,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,No,,,,
|
||||
sheet-5009,reddit spreadsheet,Form,,2026-11-05,Fourth,Tail,First,34,Later attempt,Blue or Black,Alone,False,Center,Brown,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped on the red's back,
|
||||
sheet-5010,reddit spreadsheet,Form,,2026-11-05,Second,Claw,First,1,First attempt,Blue or Black,Alone,False,Center,Brown,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Shadow (rare),Incinerated,,,,,,Incinerated,Red,False,False,True,Shadow,Yes,,,Jumped on the reds back - Meet gaze (died),
|
||||
sheet-5010,reddit spreadsheet,Form,,2026-11-05,Second,Claw,First,1,First attempt,Blue or Black,Alone,False,Center,Brown,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Shadow (rare),Incinerated,,,,,,Incinerated,Red,False,False,True,Shadow,Yes,,"Red at the Shadow (rare) encounter, which has only Black/Blue",Jumped on the reds back - Meet gaze (died),
|
||||
sheet-5011,reddit spreadsheet,Form,,2026-11-05,Second,Claw,First,,Later attempt,Blue or Black,Alone,False,East,Red,Respect / face it,Investigate & help,Killed the bullies yourself,Assess the situation,Engage the smaller cadet,East,Valley,Passed the Red: meadow,,,,Incinerated,,,Incinerated,Brown,False,False,False,No,Yes,,,,
|
||||
sheet-5012,reddit spreadsheet,Form,,2026-11-05,Fourth,Claw,Second,50,Later attempt,Blue or Black,Alone,False,Center,Brown,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,No,,,Jumping on risky tree reds back,
|
||||
sheet-5013,reddit spreadsheet,Form,,2026-11-05,Fourth,Tail,Third,1,First attempt,Daily run data,1 Squadmate,True,East,Red,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,,
|
||||
@@ -5391,7 +5391,7 @@ sheet-5268,reddit spreadsheet,Form,,2026-11-06,Third,Claw,Third,19,Later attempt
|
||||
sheet-5269,reddit spreadsheet,Form,,2026-11-06,Second,Flame,Third,,Later attempt,Blue or Black,Alone,False,Center,Red,Hide,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Descend slowly: two Greens,,Incinerated,,,,,Incinerated,Green,False,False,False,No,Yes,,,,
|
||||
sheet-5270,reddit spreadsheet,Form,,2026-11-06,Second,Flame,Third,,Unknown,Blue or Black,Alone,False,Center,Brown,Respect / face it,Investigate & help,Killed the bullies yourself,Assess the situation,Daggers or not (protect squadmate),Forest,Valley,Passed the Red: meadow,,,Incinerated,,,,Incinerated,Orange,False,False,False,No,Yes,,,Ran,
|
||||
sheet-5271,reddit spreadsheet,Form,,2026-11-06,Fourth,Flame,Second,147,Later attempt,Blue or Black,1 Squadmate,False,East,Green,Hold still,Investigate & help,Left bullies to the dragons,Assess the situation,Engage the smaller cadet,East,Tree,Descend slowly: two Greens,,Incinerated,,,,,Incinerated,Green,False,False,False,No,Yes,,,Neutral on purpose,
|
||||
sheet-5272,reddit spreadsheet,Form,,2026-11-06,Second,Claw,First,67,Later attempt,Blue or Black,Alone,False,Center,Red,Respect / face it,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Yes,,Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated,Approached orange -> said I was sure -> decided to run away (in order to die and try again) -> incinerated,
|
||||
sheet-5272,reddit spreadsheet,Form,,2026-11-06,Second,Claw,First,67,Later attempt,Blue or Black,Alone,False,Center,Red,Respect / face it,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Valley,Passed the Red: meadow,,Incinerated,Incinerated,,,,Incinerated,Green,False,False,False,No,Yes,,"Green recorded on a Valley run (no Greens in the valley); more than one dragon marked Incinerated; Green at the Passed the Red: meadow encounter, which has only Blue/Brown/Orange",Approached orange -> said I was sure -> decided to run away (in order to die and try again) -> incinerated,
|
||||
sheet-5273,reddit spreadsheet,Form,,2026-11-06,Fourth,Claw,Third,1,First attempt,Daily run data,1 Squadmate,True,East,Orange,Retreat / run,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,East,Tree,Shadow (rare),,,,,,Bonded!,Bonded,Black,False,False,True,Shadow,Yes,Scorpiontail,,,Followed the spreadsheet options for the Shadow choice EXACTLY.
|
||||
sheet-5274,reddit spreadsheet,Form,,2026-11-06,First,Flame,First,87,Later attempt,Blue or Black,1 Squadmate,True,Center,Brown,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Daggers or not (protect squadmate),Forest,Valley,Passed the Red: meadow,,,,Bonded!,,,Bonded,Brown,False,False,False,No,No,Scorpiontail,,,"Followed, tail mount immediately grabbed, told them they are already talkin to me"
|
||||
sheet-5275,reddit spreadsheet,Form,,2026-11-06,Third,Flame,First,2,Later attempt,Daily run data,1 Squadmate,False,West,Red,Hold still,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,,
|
||||
@@ -6067,7 +6067,7 @@ sheet-5940,reddit spreadsheet,Form,,2026-11-10,First,Flame,Third,,Later attempt,
|
||||
sheet-5941,reddit spreadsheet,Form,,2026-11-10,Second,Claw,Second,,Later attempt,Blue or Black,Alone,False,East,Orange,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped on its back,
|
||||
sheet-5942,reddit spreadsheet,Form,,2026-11-10,First,Claw,First,,Later attempt,Blue or Black,Alone,False,East,Red,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Lowered eyes,
|
||||
sheet-5943,reddit spreadsheet,Form,,2026-11-10,Second,Flame,Second,,Later attempt,Blue or Black,Alone,False,East,Orange,Respect / face it,Investigate & help,Left bullies to the dragons,Assess the situation,Engage the smaller cadet,East,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Lowered eyes,
|
||||
sheet-5944,reddit spreadsheet,Form,,2026-11-10,First,Claw,Third,1,First attempt,Blue or Black,1 Squadmate,True,East,Green,Hold still,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Tree,Shadow (rare),,Bonded!,,,,,Bonded,Green,False,False,True,Shadow,No,Morningstartail,,,
|
||||
sheet-5944,reddit spreadsheet,Form,,2026-11-10,First,Claw,Third,1,First attempt,Blue or Black,1 Squadmate,True,East,Green,Hold still,Ignore the scream,Tourniquet,(not recorded),(not recorded),(no fork),Tree,Shadow (rare),,Bonded!,,,,,Bonded,Green,False,False,True,Shadow,No,Morningstartail,"Green at the Shadow (rare) encounter, which has only Black/Blue",,
|
||||
sheet-5945,reddit spreadsheet,Form,,2026-11-10,Second,Tail,First,,Later attempt,Blue or Black,Alone,False,East,Orange,Retreat / run,Investigate & help,Killed the bullies yourself,Assess the situation,Engage the smaller cadet,Forest,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Yes,,,Jumped onto reds back from tree and fell off,
|
||||
sheet-5946,reddit spreadsheet,Form,,2026-11-10,First,Claw,Third,,Later attempt,Blue or Black,Alone,False,Center,Brown,Respect / face it,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,Forest,Valley,Passed the Red: meadow,,,,,Bonded!,,Bonded,Blue,False,True,False,Blue (valley),Unknown,Clubtail,,,Followed the PDF on blue decisions and make my choices quickly - no hesitations.
|
||||
sheet-5947,reddit spreadsheet,Form,,2026-11-10,Third,Tail,First,47,Later attempt,Daily run data,2 Squadmates,False,East,Orange,Retreat / run,Investigate & help,Squadmate's revenge,Assess the situation,Engage the smaller cadet,West,Tree,Descend quickly: Red,Incinerated,,,,,,Incinerated,Red,False,False,False,No,Unknown,,,"Leapt onto back, survived for once, got roasted when held gaze",
|
||||
|
||||
|
Can't render this file because it is too large.
|
File diff suppressed because one or more lines are too long
+2
-2
@@ -13,8 +13,8 @@ branch: main
|
||||
# public, limited (only people signed in to this server) or private: who can open the links below
|
||||
visibility: limited
|
||||
|
||||
# the published page, when it is hosted somewhere (GitHub Pages is paused while the GitHub account is suspended;
|
||||
# the page is viewed locally or as a Claude artifact meanwhile)
|
||||
# the published page: Ken's server serves the `pages` branch (.gitea/workflows/pages.yml). Set this to its address
|
||||
# once it is up (then `python -m collection build`): the page's links and the dataset metadata use it.
|
||||
site: null
|
||||
|
||||
# other copies of the repository (informational)
|
||||
|
||||
@@ -31,18 +31,17 @@ def low_bias(r, lb=CONFIG["low_bias"]):
|
||||
"""Kept in the low-bias sample: every run of an always-kept source, bulk logs, and riders not chasing a rare dragon."""
|
||||
if r["source"] in lb["always"]:
|
||||
return True
|
||||
if r["source"] in (lb.get("never") or ()):
|
||||
return False
|
||||
if r["source"] == "reddit spreadsheet" and "bulk" in lb["entries_kept"] and r["entry"] not in (None, "Form"):
|
||||
return True
|
||||
return r["goal"] not in lb["rare_goals"]
|
||||
|
||||
|
||||
def drop_flagged(runs, cfg=CONFIG.get("drop_flagged") or {}):
|
||||
"""Without the flagged runs of the sources read automatically from free text (planner/policy.yaml drop_flagged)."""
|
||||
sources, keep = set(cfg.get("sources") or ()), tuple(cfg.get("except") or ())
|
||||
|
||||
def problems(r):
|
||||
return [f for f in (r.get("data_flags") or "").split("; ") if f and not f.startswith(keep)] if keep else [f for f in (r.get("data_flags") or "").split("; ") if f]
|
||||
return [r for r in runs if r["source"] not in sources or not problems(r)]
|
||||
def drop_flagged(runs, on=CONFIG.get("drop_impossible", True)):
|
||||
"""Without the runs that cannot have happened as recorded (atlas.runs.impossible; planner/policy.yaml
|
||||
drop_impossible): every other run is kept, however partial."""
|
||||
return [r for r in runs if not (on and runs_io.impossible(r))]
|
||||
|
||||
|
||||
def clean(o):
|
||||
|
||||
+7
-8
@@ -6,21 +6,19 @@
|
||||
version: 1
|
||||
|
||||
# the `source` values of data/public/runs_clean.csv the model learns from
|
||||
sources: [reddit spreadsheet, reported, bot]
|
||||
sources: [reddit spreadsheet, reported, bot, reddit comment]
|
||||
|
||||
# Sources read automatically from free text: a run with any data flag (a broken game rule, an unrecognised answer)
|
||||
# is left out of the model, since the flag may mean the text was misread or misremembered. Flags that only say where
|
||||
# the run came from (`except`, matched at the start of a flag) do not count. A hand fix that clears the flags
|
||||
# brings the run back. Applies once the source is also listed in `sources` above.
|
||||
drop_flagged:
|
||||
sources: [reddit comment]
|
||||
except: ["read from a Reddit comment"]
|
||||
# Every run of the sources above is used, however partial, except a run that cannot have happened as recorded
|
||||
# (core/spec/form.yaml impossible_flags: it breaks one of the game's rules). Such runs stay in runs_clean.csv, flagged;
|
||||
# a hand fix that clears the flag brings a Reddit or reported run back. The site statistics use the same rule.
|
||||
drop_impossible: true
|
||||
|
||||
# Two samples. "all" is every run above. "lowBias" leaves out runs where a rare sighting is more likely to have
|
||||
# been reported than a miss: form submissions and reports from riders chasing Blue or Black. The bot logs every
|
||||
# run, and bulk-pasted logs record every run of the players who kept them, so both stay in.
|
||||
low_bias:
|
||||
always: [bot] # sources whose every run is kept
|
||||
never: [reddit comment] # riders posting a run in a thread pick the notable ones, like rare-chasers
|
||||
entries_kept: [bulk] # spreadsheet rows whose `entry` is not "Form" (bulk-pasted logs)
|
||||
rare_goals: [Blue, Black, Blue or Black]
|
||||
|
||||
@@ -46,3 +44,4 @@ reporting_bias:
|
||||
- { name: "form, chasing Blue/Black", when: { source: reddit spreadsheet, goal: rare } }
|
||||
- { name: "form, other", when: { source: reddit spreadsheet } }
|
||||
- { name: reported, when: { source: reported }, same_as: "form, other" }
|
||||
- { name: reddit comments, when: { source: reddit comment } } # self-selected: its own reporting rate
|
||||
|
||||
+1
-1
@@ -554,7 +554,7 @@
|
||||
</tbody></table></div><p class="small">Bulk logs come from players who recorded every run, so they are the least selective rows. Their rare-dragon rates are several times lower than the form's, which is the strongest sign of reporting bias in this data.</p></div>
|
||||
<div class="panel"><h3>Rows flagged for contradictions</h3><p class="small muted">${m.n_flagged} of ${r.n} rows (${pct(m.n_flagged / r.n)}). Flagged rows stay in unless you pick "Rows without data-quality flags" above.</p><div class="tbl-wrap"><table><tbody>${flags.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td></tr>`).join("") || `<tr><td>None</td></tr>`}</tbody></table></div></div>
|
||||
<div class="panel"><h3>How runs ended</h3><div class="tbl-wrap"><table><tbody>${outs.map(([f, n]) => `<tr><td>${esc(f)}</td><td class="num">${n}</td><td class="num">${pct(n / r.n)}</td></tr>`).join("")}</tbody></table></div><p class="small muted">"No outcome recorded" rows reached the hour but the form shows no Bonded or Incinerated. They count toward reaching a branch but not toward bond rates.</p></div>
|
||||
<div class="panel"><h3>Independence</h3><p class="small">The form has no player ID. One player's 50 runs count as 50 independent runs, so intervals are somewhat narrower than they should be. ${pct((r.n - (m.sources["Form"] || 0)) / r.n, 0)} of rows arrived as bulk imports, each probably one player's or one group's log.</p></div>
|
||||
<div class="panel"><h3>Independence</h3><p class="small">The form has no player ID. One player's 50 runs count as 50 independent runs, so intervals are somewhat narrower than they should be. ${pct(Object.entries(m.sources).filter(([k]) => !["Form", "Reddit comments", "DragonKind bot", "Reported on the Atlas"].includes(k)).reduce((s, [, n]) => s + n, 0) / r.n, 0)} of rows arrived as bulk imports, each probably one player's or one group's log.</p></div>
|
||||
<div class="panel"><h3>What the form does not ask</h3><p class="small">The redesigned form asks how you reacted to the first dragon (answered in ${r.factors.reach_hour.rows.filter((x) => x.factor === "sighting_action" && !["Unknown", "(not asked)"].includes(x.level)).reduce((a, x) => a + x.n, 0)} rows; most riders answer "Unsure") and how the bully fight went (${r.factors.reach_hour.rows.filter((x) => x.factor === "combat_first" && x.level !== "(not recorded)").reduce((a, x) => a + x.n, 0)} rows). It still does not record most choices inside the bonding encounters. Theories that rely on those choices cannot be confirmed or refuted here.</p></div>`;
|
||||
}
|
||||
const sourceRate = (outKey, source) => state.res.source_rates[outKey][source];
|
||||
|
||||
@@ -308,6 +308,8 @@ class RedditComments(unittest.TestCase):
|
||||
dict(branch="Valley: passed the Red, the meadow", outcome="Incinerated at a bonding encounter", outcome_color="Red"),
|
||||
"Green recorded on a Valley run":
|
||||
dict(hour="Went to the valley centre", outcome="Bonded", outcome_color="Green"),
|
||||
"Black recorded on a Valley run":
|
||||
dict(hour="Went to the valley centre", outcome="Bonded", outcome_color="Black"),
|
||||
"does not follow 'Ignore the scream'":
|
||||
dict(scream="Kept walking (ignored it)", resolution="Bullies: my squadmate's revenge", outcome="Died before the hour"),
|
||||
"fight answers on a run that did not help":
|
||||
@@ -331,7 +333,7 @@ class RedditComments(unittest.TestCase):
|
||||
{"source": "reddit spreadsheet", "data_flags": "multiple wings"},
|
||||
{"source": "reported", "data_flags": "Green recorded on a Valley run (no Greens in the valley)"}]
|
||||
kept = build_policy.drop_flagged(rows)
|
||||
self.assertEqual([r["source"] for r in kept], ["reddit comment", "reddit spreadsheet", "reported"])
|
||||
self.assertEqual([r["source"] for r in kept], ["reddit comment", "reddit spreadsheet"])
|
||||
|
||||
def test_delivered_not_extracted(self):
|
||||
spec = read_yaml("collection/pipeline.yaml")["sources"]["reddit_comments"]
|
||||
|
||||
Reference in New Issue
Block a user