Core: a small JS advisor reads the game from policy.json; engine.js is gone (#13)

Step 4b of the Python-first core.

- core/solver.js is now a 250-line advisor (was 900 lines): it reads the game (core/spec/game.yaml,
  published inside policy.json as model.game) and the count tables, and re-solves from any point in
  a run with the same interpreter as the Python model. Same API for the site and Ken's bot
  (advisor, recommendOption, solverHistory). The bot's on-screen text for each option moves to
  game.yaml (bot_text); the deadly rolls carry their risk labels there too.
- tests/core/advisor.test.js: the advisor gives exactly the Python model's values and choices for
  every colour and goal (to 1e-12, every decision on every published route), walks a run in the
  game's order, and maps advice onto the bot's screen.
- Removed core/engine.js and everything that only compared it with Python (tests/parity/run_engine,
  compare_runs, compare_results, compare_routes, compare_solver, compare_stats, analysis/stats/
  analyze.py). In their place: tests/core/test_stats.py checks the statistics against known answers,
  test_solver.py checks the route model on a synthetic world (tests/fixtures/synthetic_policy.py).
- analysis/network/build-graph.js reads the Python-coded runs; graph.json is byte-identical.
- docs/BOT_INTEGRATION.md: what changed for the bot (history keys, new choices, re-sync all three files).


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
ekcSchiano
2026-11-03 22:01:32 +00:00
committed by GitHub
co-authored by Claude Opus 5.5
parent d04b88e7b4
commit 11b4a0398e
27 changed files with 578 additions and 2260 deletions
+10 -11
View File
@@ -10,13 +10,13 @@ Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publis
The project has four parts, built on one shared core. Each folder has its own README.
```
core/ the shared model every part uses. spec/ holds the declarative rules (form.yaml: every question
and answer of the community form); atlas/ is the Python package that reads them (coding.py).
engine.js, graph.js and solver.js are the JavaScript the site and Ken's bot still use; they move
to Python + spec files step by step (docs/ROADMAP.md).
core/ the shared model every part uses. spec/ holds the declarative rules (form.yaml: the community
form; game.yaml: the game); atlas/ is the Python package that reads them (coding, route model).
solver.js (the route advisor) and graph.js (the bot's text) are the only JavaScript, because
they run in the browser and in Ken's bot.
collection/ getting data in: sources/ (the form sheet, the Dragon Register, the route PDF check, the daily
update job) and bot/ (the DragonKind bot importer). Reddit is not collected yet.
analysis/ making sense of it: stats/ (Python statistics, mirrored in core/engine.js), network/ (builds
analysis/ making sense of it: stats/ (measures.yaml and site_stats.py: every number on the page), network/ (builds
data/public/graph.json), reports/ (generated reports such as POLICY.md)
planner/ the route planner: builds data/public/policy.json and its report; the bot uses it for advice
and experiments (docs/BOT_INTEGRATION.md)
@@ -25,7 +25,7 @@ data/ raw/ (source exports, not committed), public/ (what the page and
(coded runs, PDF fingerprint), bot/ (anonymised bot runs), sources.json (download links)
docs/ guides that cross sections: privacy and licences, repo settings, bot data protocol, bot
integration, colours, roadmap
tests/ core/ (unit tests), parity/ (the JS and Python implementations must agree), fixtures/
tests/ core/ (rule, statistics, route model and advisor tests), parity/ (tools to compare two policies), fixtures/
```
## Run it locally
@@ -34,7 +34,7 @@ Requirements: Node 18+, Python 3.9+ (`pip install -r requirements.txt`).
```
npm run dev # builds dist/ and serves it at http://localhost:8000
npm test # JS engine + Python reference + parity checks
npm test # coding rules, statistics, route model, game spec, advisor and bot-text tests
```
## New data from the community sources
@@ -73,7 +73,7 @@ looks like prose. See `docs/BOT_DATA_PROTOCOL.md` for what the bot should log.
In the browser: open the page and use **Load a newer export** with the responses spreadsheet
(Google Sheets → File → Download → Microsoft Excel). Everything recomputes. Questions are matched by
wording; a renamed question is listed as "not found", and then `COLUMN_KEYS` in
`core/spec/form.yaml` (and, until the site stops coding rows itself, `core/engine.js`) need updating.
`core/spec/form.yaml` needs updating.
To update the bundled snapshot, put the new export in `data/raw/form/`, then:
@@ -87,8 +87,7 @@ npm test
The tables (how each question is found, what each answer means, which answers count as missing) are in
`core/spec/form.yaml`; the logic that combines them is `code_row()` in `core/atlas/coding.py`. Add a case to
`tests/core/test_coding.py` for every rule you change, then run `npm test`. Until the site switches to
precomputed statistics, `codeRow()` in `core/engine.js` must be changed the same way (the parity tests check it).
`tests/core/test_coding.py` for every rule you change, then run `npm test`.
## Statistics in one paragraph
@@ -108,7 +107,7 @@ local, how bot accounts are anonymised) are in `docs/PRIVACY.md`.
## Release checks
```
npm test # parity and unit tests
npm test # all tests
python site/qa/check_colours.py # dragon colours stay distinguishable (docs/COLOURS.md)
npm run build && python3 -m http.server 8765 -d dist &
node site/qa/crawl.js /tmp/qa # every tab on desktop and phone: text, links, buttons, errors, overflow
+2 -2
View File
@@ -5,7 +5,7 @@ Making sense of the data.
| Folder | What it does |
|---|---|
| `stats/measures.yaml` | **Declarative.** What the site measures: outcomes and their denominators, route targets and comparison groups, the bonding-encounter contexts, the route map's stages, the filter presets and the route thresholds. |
| `stats/site_stats.py` | Computes every number the Atlas page shows for every preset and writes `data/public/stats.json`; the page computes nothing itself. `tests/parity/compare_stats.py` checks it against the old in-browser engine until that is removed. |
| `stats/` | General statistics in Python (`analyze.py`, `stats_core.py`): rates with Wilson intervals, odds ratios with Fisher's test and Benjamini–Hochberg, ridge logistic models that adjust for data source and chase goal. `core/engine.js` computes the same numbers for the site, and `tests/parity/` checks they match. |
| `stats/site_stats.py` | Computes every number the Atlas page shows for every preset and writes `data/public/stats.json`; the page computes nothing itself. |
| `stats/stats_core.py` | The statistics themselves: rates with Wilson intervals, odds ratios with Fisher's test and Benjamini–Hochberg, Mantel–Haenszel, ridge logistic models that adjust for data source and chase goal. Tested against known answers (`tests/core/test_stats.py`). |
| `network/` | `build-graph.js` combines every source into `data/public/graph.json` (the decision-variable graph and the bot's scene graph). CI fails if the committed graph is out of date. |
| `reports/` | Generated reports. `POLICY.md` is written by `planner/policy_report.py`; do not edit it by hand. |
+20 -3
View File
@@ -2,13 +2,30 @@
// node analysis/network/build-graph.js
const fs = require("fs");
const path = require("path");
const E = require("../../core/engine.js");
const G = require("../../core/graph.js");
const root = path.resolve(__dirname, "../..");
const rel = (p) => path.join(root, p);
const raw = JSON.parse(fs.readFileSync(rel("data/public/form_responses.json"), "utf8"));
const { runs } = E.codeRows(raw.rows, raw.headers);
/** The coded runs written by the Python coding rules (python -m atlas.coding -> data/derived/runs_clean.csv). */
function readCSV(file) {
const text = fs.readFileSync(file, "utf8"), rows = [];
let row = [], cell = "", q = false;
for (let i = 0; i < text.length; i++) {
const ch = text[i];
if (q) {
if (ch === '"' && text[i + 1] === '"') { cell += '"'; i++; }
else if (ch === '"') q = false;
else cell += ch;
} else if (ch === '"') q = true;
else if (ch === ",") { row.push(cell); cell = ""; }
else if (ch === "\n") { row.push(cell); rows.push(row); row = []; cell = ""; }
else if (ch !== "\r") cell += ch;
}
if (cell || row.length) { row.push(cell); rows.push(row); }
const [head, ...body] = rows;
return body.map((r) => Object.fromEntries(head.map((h, i) => [h, r[i] === "" || r[i] === undefined ? null : r[i]])));
}
const runs = readCSV(rel("data/derived/runs_clean.csv"));
const records = runs.map((r) => ({ source: r.source === "Form" ? "form" : "form-bulk", vars: G.formRunVariables(r) }));
const botDir = rel("data/bot");
-139
View File
@@ -1,139 +0,0 @@
"""
analyze.py — Python reference implementation of the statistics shown on the route-map page.
python analyze.py ../../data/derived/runs_clean.csv ../../tests/out/results_py.json
It mirrors analyse() in core/engine.js (same denominators, same tests, same model); tests/parity/compare_results.py
checks that both produce the same numbers. Edit either, then re-run the comparison.
"""
import json
import sys
import numpy as np
import pandas as pd
from stats_core import Z95, bh, logistic_ridge, odds_ratio, p_from_z, wilson
# colours and the answers that count as missing come from core/spec/form.yaml, like everything else
from atlas.coding import COLORS, MISSING, is_missing # noqa: E402
FACTORS = ["company", "direction", "first_sighting", "scream", "resolution", "combat_first",
"post_path", "hour", "wing", "squad", "attempt_group", "source", "goal", "sighting_action", "combat_second", "malek_intent"]
def W(k, n):
p, lo, hi = wilson(k, n)
return {"k": int(k), "n": int(n), "p": p, "lo": lo, "hi": hi}
def outcomes():
o = {
"blue_meadow": (lambda d: d.branch == "Passed the Red: meadow", lambda d: d.saw_blue_meadow),
"shadow": (lambda d: (d.hour == "Tree") & d.branch.notna(), lambda d: d.saw_shadow),
"bond_blue_meadow": (lambda d: (d.branch == "Passed the Red: meadow") & d.enc_Blue.isin(["Bonded!", "Incinerated"]),
lambda d: d.enc_Blue == "Bonded!"),
"reach_hour": (lambda d: pd.Series(True, index=d.index), lambda d: d.hour.isin(["Tree", "Valley"])),
}
for c in COLORS:
o["bonded_" + c] = (lambda d: d.hour.isin(["Tree", "Valley"]),
(lambda c: lambda d: (d.outcome == "Bonded") & (d.outcome_color == c))(c))
return o
OUT = outcomes()
def factor_table(d, key, min_n=5):
den, ev = OUT[key]
D = d[den(d)].copy()
D["_e"] = ev(D).astype(bool)
rows = []
for f in FACTORS:
sub = D[D[f].notna() & ~D[f].isin(MISSING)]
nf, ef = len(sub), int(sub._e.sum())
for lv in sorted(sub[f].astype(str).unique()):
g = sub[sub[f].astype(str) == lv]
k, n = int(g._e.sum()), len(g)
row = {"factor": f, "level": lv, **W(k, n)}
rk, rn = ef - k, nf - n
if n >= min_n and rn >= min_n and 0 < ef < nf:
or_, lo, hi, p, corr = odds_ratio(k, n - k, rk, rn - rk)
row.update({"or": or_, "or_lo": lo, "or_hi": hi, "fisher_p": p, "corrected": corr})
rows.append(row)
tested = [r for r in rows if "fisher_p" in r]
for r, q in zip(tested, bh([r["fisher_p"] for r in tested])):
r["q"] = q
return {"outcome": key, "overall": W(int(D._e.sum()), len(D)), "rows": rows}
def model(d, key, factors, lam=1.0):
den, ev = OUT[key]
D = d[den(d)].copy()
y = ev(D).astype(int).values
F = pd.DataFrame({name: D.apply(fn, axis=1) for name, fn in factors.items()}, index=D.index)
keep = F.notna().all(axis=1).values
F, y = F[keep], y[keep]
cols, names = [np.ones(len(F))], []
refs = {}
for f in factors:
counts = F[f].value_counts()
levels = sorted(counts.index, key=lambda lv: (-counts[lv], lv))
refs[f] = levels[0]
for lv in levels[1:]:
cols.append((F[f] == lv).astype(float).values)
names.append((f, lv, int(counts[lv])))
X = np.column_stack(cols)
b, se = logistic_ridge(X, y, lam)
terms = [{"term": f"{f} = {lv}", "factor": f, "level": lv, "n": n, "or": float(np.exp(b[i + 1])),
"lo": float(np.exp(b[i + 1] - Z95 * se[i + 1])), "hi": float(np.exp(b[i + 1] + Z95 * se[i + 1])),
"p": float(p_from_z(b[i + 1] / se[i + 1]))} for i, (f, lv, n) in enumerate(names)]
return {"outcome": key, "n": int(len(y)), "events": int(y.sum()), "refs": refs, "lambda": lam, "terms": terms}
def goal2(r):
if r.goal in ("Blue", "Black", "Blue or Black"):
return r.goal
return "Blank/not asked" if r.goal == "(not asked/blank)" else "Other colour/any"
def src2(r):
return "Form submission" if r.source == "Form" else "Bulk log (import)"
def first2(r):
return r.first_sighting if r.first_sighting in ("Red", "Brown", "Orange", "Green") else None
def known(f):
return lambda r: None if is_missing(r[f]) else r[f]
def period(r):
if not isinstance(r.date, str):
return "No date (import)"
return "Oct 4–10" if r.date < "2026-10-11" else "Oct 11 onward"
def analyse(d):
res = {"n": len(d), "factors": {k: factor_table(d, k) for k in OUT}}
res["models"] = {
"blue_meadow": model(d, "blue_meadow", {
"source": src2, "goal": goal2, "direction": known("direction"), "first_sighting": first2,
"resolution": known("resolution"), "company": known("company")}),
"shadow": model(d, "shadow", {
"source": src2, "goal": goal2, "attempt": known("attempt_group"),
"direction": known("direction")}),
"bond_blue_meadow": model(d, "bond_blue_meadow", {
"source": src2, "goal": goal2, "direction": known("direction"), "resolution": known("resolution")}),
}
return res
if __name__ == "__main__":
src = sys.argv[1] if len(sys.argv) > 1 else "../../data/derived/runs_clean.csv"
out = sys.argv[2] if len(sys.argv) > 2 else "../../tests/out/results_py.json"
d = pd.read_csv(src, keep_default_na=False, na_values=[""])
for c in ("saw_blue_meadow", "saw_shadow", "rescued"):
d[c] = d[c].astype(str) == "True"
res = analyse(d)
json.dump(res, open(out, "w"), indent=1, default=float)
print("wrote", out)
+3 -2
View File
@@ -1,4 +1,5 @@
"""Small, dependency-light statistics used by the analysis (mirrored in core/engine.js)."""
"""The statistics behind every number on the page: Wilson intervals, odds ratios with Fisher's test, Benjamini-Hochberg
q-values, Mantel-Haenszel, ridge logistic regression. Tested against known answers in tests/core/test_stats.py."""
import math
import numpy as np
@@ -79,7 +80,7 @@ def p_from_z(z):
def mantel_haenszel(tables):
"""Mantel-Haenszel common OR over 2x2 tables [a, b, c, d], Robins-Breslow-Greenland 95% CI,
and the CMH chi-square p-value (1 df, continuity-corrected). Mirrors mantelHaenszel() in engine.js."""
and the CMH chi-square p-value (1 df, continuity-corrected)."""
R = S = PR = PSQR = QS = sA = sE = sV = 0.0
for a, b, c, d in tables:
n = a + b + c + d
+14 -14
View File
@@ -1,21 +1,21 @@
# core
The shared model. Every other part uses it, so a change here touches everything: run `npm test` after any edit.
The shared model. Every other part uses it, so run `npm test` after any change.
The rules and the game are **declarative** files in `spec/`; the Python package `atlas/` reads them. JavaScript is
used only where it has to run in a browser or in Ken's bot.
| File | What it does |
|---|---|
| `engine.js` | Turns a row of the community form (any version of it) into one coded run: the run format, every coding rule, missing answers, email redaction. Also the statistics the site shows. |
| `spec/form.yaml` | **Declarative.** Every question of the community form (how to find it in an export) and every answer (what it is coded as), the answers that count as missing, email redaction. |
| `spec/game.yaml` | **Declarative.** The game: every decision and its options in order, every roll and what its odds may depend on, every bonding encounter, and the flow of a run from the entrance to the dragons. |
| `atlas/model.py`, `atlas/solver.py` | The route model in Python: what each coded run reveals about each step, the count tables and shrinkage, and one interpreter of `game.yaml` used to solve, evaluate and export the best routes. `tests/parity/compare_solver.py` checks it against `solver.js`, which the site and the bot use until the JS advisor replaces it. |
| `atlas/coding.py` | The Python coding rules: reads `form.yaml`, codes every row (`python -m atlas.coding`). Rule tests: `tests/core/test_coding.py`. |
| `graph.js` | The route graph: the bot's scene graph, the decision-variable graph shared by all sources, the crosswalk from the bot's on-screen text to form answers, Beta credible intervals. |
| `solver.js` | The decision model: every choice and roll in a run, chance estimates with empirical-Bayes back-off, backward induction for each dragon, and the advisor the site and the bot use to get advice from any point in a run. |
| `spec/form.yaml` | Every question of the community form (how to find it in an export) and every answer (what it is coded as), the answers that count as missing, email redaction, when "left your squadmates" was first asked. |
| `spec/game.yaml` | The game: every decision and its options in order, every roll and what its odds may depend on, every bonding encounter, the flow of a run from the entrance to the dragons, how form answers map onto it, and the text the bot sees for each option. |
| `atlas/coding.py` | Codes every form row into a run (`python -m atlas.coding`). Rule tests: `tests/core/test_coding.py`. |
| `atlas/model.py`, `atlas/solver.py` | The route model: what each coded run reveals about each step, the count tables and shrinkage, the posterior draws, and one interpreter of `game.yaml` used to solve, evaluate and export the best routes (`planner/build_policy.py`). Tests: `tests/core/test_solver.py`, `test_game.py`. |
| `solver.js` | **JavaScript, for the site and the bot.** The route advisor: reads the game and the count tables from `data/policy.json` and re-solves from any point in a run. `tests/core/advisor.test.js` checks it gives exactly the Python model's values and choices. |
| `graph.js` | **JavaScript, for the bot.** Turns the bot's on-screen choices into form answers, and builds the route graph (`analysis/network/build-graph.js`). Moves to Python with the collection review. |
All three JS files have no dependencies and run unchanged in Node and the browser. The site and Ken's bot load
the published copies (`https://ekcschiano.github.io/threshing-atlas/solver.js` etc.), so their file names on the
site must not change.
The site and Ken's bot load the published copies (`https://ekcschiano.github.io/threshing-atlas/solver.js`,
`graph.js`, `data/policy.json`), so those names on the site must not change.
To support a new answer, add it to `spec/form.yaml`. To change a rule, edit `atlas/coding.py`, add a test case, and (until the site stops coding rows in the browser) mirror it in `codeRow()` in `engine.js`; `tests/parity/` fails if they disagree.
The project is moving to Python plus declarative spec files; JavaScript stays only where it must run in the browser or in Ken's bot (see `docs/ROADMAP.md`).
To support a new form answer, add it to `spec/form.yaml`. To change a rule, edit `atlas/coding.py` and add a test case.
To add a choice to the game, add it to `spec/game.yaml` (and, if the form records it, to `atlas/model.py`).
+3
View File
@@ -366,6 +366,9 @@ def build_policy(runs, draws=200, seed=20261101, min_support=None, alpha=None, c
"chanceContext": {k: s["context"] for k, s in model.CHANCE.items()},
"decisions": model.DECISIONS, "support": support, "supportCounts": counts,
"tables": T,
# the game itself (core/spec/game.yaml), so the advisor in core/solver.js can re-solve from any point
"game": {k: G[k] for k in ("version", "colours", "decisions", "chance", "encounters", "encounter_outcomes",
"by_colour", "start", "steps", "from_form", "bot_text")},
},
"data": {"runs": len(runs)},
"targets": targets,
-854
View File
@@ -1,854 +0,0 @@
/*
* engine.js — Threshing route analysis engine.
*
* One file, used two ways:
* - in the browser (the route map page loads it and recomputes everything from the raw form export)
* - in Node (`node tests/parity/run_engine.js data.json`), so results can be reproduced and checked against
* the Python reference implementation in python/.
*
* Pipeline: raw form rows -> codeRow() (one coded run per row) -> analyse() -> results object
* coded runs + a target outcome -> buildGraph() -> nodes/links for the route map
*
* Every rule here is described in plain language in the Methods section of the page.
*/
(function (root, factory) {
if (typeof module === "object" && module.exports) module.exports = factory();
else root.ThreshEngine = factory();
})(typeof self !== "undefined" ? self : this, function () {
"use strict";
// Free-text answers sometimes contain a respondent's email address; never display or export it.
const EMAIL_RE = /[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/g;
const redact = (v) => (v == null ? v : String(v).replace(EMAIL_RE, "[email removed]"));
const COLORS = ["Red", "Green", "Orange", "Brown", "Blue", "Black"];
// Answers that carry no information about the choice ("Unsure", blank, a question the form did not ask yet,
// conflicting answers). They are kept in the coded data for transparency, but every analysis treats them as
// missing: they are never shown or compared as if they were a choice.
const MISSING = new Set(["Unknown", "Unsure", "Unsure/unknown", "Multiple", "(not recorded)", "(not asked)",
"(not asked/blank)", "Unknown/lost count", "Blank/not asked"]);
const isMissing = (v) => v === null || v === undefined || v === "" || MISSING.has(v);
// ---------------------------------------------------------------- column detection
const COLUMN_KEYS = {
timestamp: ["timestamp"],
wing_First: ["wing and section", "[first]"],
wing_Second: ["wing and section", "[second]"],
wing_Third: ["wing and section", "[third]"],
wing_Fourth: ["wing and section", "[fourth]"],
squad: ["what squad"],
attempt: ["attempt number"],
company: ["who accompanied"],
dir_East: ["initial direction", "[east"],
dir_West: ["initial direction", "[west"],
dir_Center: ["initial direction", "[forest center"],
cadet: ["outcome of your cadet encounter"],
post: ["path did you take after the bully"],
hour: ["after walking an hour"],
enc_Red: ["attempt to bond", "[red"],
enc_Green: ["attempt to bond", "[green"],
enc_Orange: ["attempt to bond", "[orange"],
enc_Brown: ["attempt to bond", "[brown"],
enc_Blue: ["attempt to bond", "[blue"],
enc_Black: ["attempt to bond", "[black"],
death_note: ["those who died"],
bond_note: ["those who bonded"],
goal: ["colour dragon are you trying"],
combat: ["combat choice"],
// Questions added in the redesigned form (late October 2026). "!x" means the header must NOT contain x.
first_dir: ["what was your first direction"],
first_colour: ["colour was your initial dragon sighting", "!["],
leg_West: ["colour was your initial dragon sighting", "[west"],
leg_Center: ["colour was your initial dragon sighting", "[forest center"],
sighting_action: ["action did you take during the first dragon sighting"],
cadet_actions: ["during your cadet encounter"],
goal_v2: ["why are you doing this run"],
malek: ["seek malek"],
rare: ["encounter a rare dragon"],
tail: ["what tail does your dragon have"],
};
// A key is only reported missing when neither it nor its replacement in the other form version was found.
const ALTERNATIVE = {
dir_East: "first_dir", dir_West: "first_dir", dir_Center: "first_dir", first_dir: "dir_Center", first_colour: "dir_Center",
goal: "goal_v2", goal_v2: "goal", combat: "cadet_actions", cadet_actions: "combat",
};
const OPTIONAL = new Set(["leg_West", "leg_Center", "sighting_action", "malek", "rare", "tail"]);
function findColumns(headers) {
const C = {};
for (const [key, words] of Object.entries(COLUMN_KEYS)) {
const hit = headers.find((h) => words.every((w) => w[0] === "!" ? !String(h).toLowerCase().includes(w.slice(1)) : String(h).toLowerCase().includes(w)));
if (hit !== undefined) C[key] = hit;
}
const missing = Object.keys(COLUMN_KEYS).filter((k) => !(k in C) && !OPTIONAL.has(k) && !(ALTERNATIVE[k] in C));
return { C, missing };
}
// ---------------------------------------------------------------- coding rules
const CADET_MAP = {
"Bullies > You dealt with them": ["Investigate & help", "Killed the bullies yourself"],
"Bullies > Squadmate's revenge": ["Investigate & help", "Squadmate's revenge"],
"Bullies > Left to the Dragons": ["Investigate & help", "Left bullies to the dragons"],
"Bullies > Assessed the situation": ["Investigate & help", "Unknown"],
"Injured Cadet > Tourniquet": ["Ignore the scream", "Tourniquet"],
"Injured Cadet > Left them there": ["Ignore the scream", "Left injured cadet"],
"Injured Cadet > Took their weapons": ["Ignore the scream", "Took their weapons"],
"Avoided combat (Investigated scream > Left them to it)": ["Investigate & leave", "Left them to it"],
"You died in combat": ["Investigate & help", "DIED in combat"],
"N/A - You died with initial dragon sighting": ["DIED at first sighting", "DIED at first sighting"],
};
const COMBAT_MAP = {
"Bullies > Assessed the situation": "Assess the situation",
"Bullies > Even the numbers (Rushed the smaller cadet)": "Even the numbers (smaller cadet)",
"Bullies > Put yourself in the middle (Rushed the bigger cadet attacking your squadmate)":
"Put yourself in the middle (bigger cadet)",
};
// Redesigned form: the cadet encounter is asked as a path of actions plus a separate outcome.
// value: [scream, first combat move, second combat move, resolution (when the action decides it)]
const CADET_ACTIONS = {
"Bullies > Assess > Unknown": ["Investigate & help", "Assess the situation", null, null],
"Bullies > Assess > Smaller Cadet": ["Investigate & help", "Assess the situation", "Engage the smaller cadet", null],
"Bullies > Assess > Daggers or not": ["Investigate & help", "Assess the situation", "Daggers or not (protect squadmate)", null],
"Bullies > Unknown": ["Investigate & help", null, null, null],
"Bullies > Smaller Cadet (Did not assess)": ["Investigate & help", "Even the numbers (smaller cadet)", null, null],
"Bullies > Bigger Cadet (Did not assess)": ["Investigate & help", "Put yourself in the middle (bigger cadet)", null, null],
"Injured Cadet > Tourniquet": ["Ignore the scream", null, null, "Tourniquet"],
"Injured Cadet > Left them there": ["Ignore the scream", null, null, "Left injured cadet"],
"Injured Cadet > Took their weapons": ["Ignore the scream", null, null, "Took their weapons"],
"Avoided combat (Investigated scream > Left them to it)": ["Investigate & leave", null, null, "Left them to it"],
"N/A - You died with initial dragon sighting": ["DIED at first sighting", null, null, "DIED at first sighting"],
"Unsure": ["Unknown", null, null, "Unknown"],
};
const CADET_OUTCOME = {
"Squadmate's revenge": "Squadmate's revenge",
"You ended the bullies yourself": "Killed the bullies yourself",
"Left to the Dragons": "Left bullies to the dragons",
"You died in combat": "DIED in combat",
};
const DIRECTION_MAP = { "Forest Center": "Center", "West / Scent": "West", "East / Roar": "East" };
const GOAL_V2 = {
"Hoping for a Rare Dragon (Blue or Black)": "Blue or Black",
"Trying to bond the Red dragon under the tree": "Red (tree)",
"Daily Run Assignment data": "Daily run data",
"Just here for fun, and maybe a dragon": "Any (for fun)",
};
const SIGHTING_ACTION = {
"Hold Ground / Stay Still": "Hold still", "Respect / Face it": "Respect / face it",
"Retreat / Run / Back Away": "Retreat / run", "Hide / Duck": "Hide", "Draw / Drop Weapon": "Weapon",
"Meet Gaze": "Meet its gaze", "Protect / Help Sqaudmate": "Protect squadmate", "Approach": "Approach",
"Stand on your own": "Stand alone", "Unsure": "Unknown", "Unsure - You don't remember": "Unknown",
};
const RARE_MAP = { "Yes - A blue dragon in the valley": "Blue (valley)", "Yes - Shadow!!!": "Shadow", "No": "No" };
const RARE_GOALS = ["Blue", "Black", "Blue or Black"];
const isRareGoal = (goal) => RARE_GOALS.includes(goal);
const HOUR_MAP = {
"Continued to Valley Centre": "Valley",
"Climbed the tallest tree": "Tree",
"N/A - You're dead": "DIED before the hour",
};
// First-sighting pools per (company, direction) from the community PDF. Only used to FLAG rows.
const SIGHTING_POOL = {
"Alone|East": ["Red", "Orange"],
"Alone|West": ["Red", "Brown", "Green"],
"Alone|Center": ["Red", "Brown"],
"1 Squadmate|East": ["Green", "Orange", "Red"],
"1 Squadmate|West": ["Red", "Brown", "Green"],
"1 Squadmate|Center": ["Red", "Brown"],
"2 Squadmates|East": ["Red", "Orange"],
"2 Squadmates|West": ["Green", "Brown", "Red"],
"2 Squadmates|Center": ["Red", "Brown"],
};
const str = (v) => {
if (v === null || v === undefined) return null;
if (typeof v === "number" && isNaN(v)) return null;
const t = String(v).trim();
return t === "" || t === "nan" || t === "NaN" ? null : t;
};
// Form submissions carry a real timestamp; rows bulk-pasted into the sheet carry a label
// ("Data Import", "Manual Upload", "Historical import") instead.
function parseTimestamp(v) {
if (v instanceof Date && !isNaN(v)) return v;
if (typeof v === "number" && v > 40000 && v < 80000) return new Date(Date.UTC(1899, 11, 30) + Math.floor(v) * 86400000); // Excel serial date
const t = str(v);
if (!t) return null;
let m = t.match(/^(\d{4})-(\d{2})-(\d{2})/);
if (m) return new Date(Date.UTC(+m[1], +m[2] - 1, +m[3]));
m = t.match(/^(\d{1,2})\/(\d{1,2})\/(\d{4})/); // Google Sheets CSV: M/D/YYYY
if (m) return new Date(Date.UTC(+m[3], +m[1] - 1, +m[2]));
return null;
}
const isoDate = (d) => d.toISOString().slice(0, 10);
function parseAttempt(v) {
const t0 = str(v);
if (!t0) return null;
const t = t0.toLowerCase();
if (t.includes("lost count")) return null;
const words = { first: 1, second: 2, third: 3, fourth: 4, fifth: 5 };
for (const [w, n] of Object.entries(words)) if (t.startsWith(w)) return n;
const m = t.match(/\d+/);
return m ? parseInt(m[0], 10) : null;
}
function codeRow(row, C) {
const g = (k) => (k in C ? str(row[C[k]]) : null);
const flags = [];
const tsRaw = "timestamp" in C ? row[C.timestamp] : null;
const ts = parseTimestamp(tsRaw);
const source = ts ? "Form" : str(tsRaw) || "Unknown import";
const date = ts ? isoDate(ts) : null;
const wings = ["First", "Second", "Third", "Fourth"].filter((w) => g("wing_" + w));
const wing = wings.length === 1 ? wings[0] : wings.length ? "Multiple" : "Unknown";
const section = wings.length === 1 ? g("wing_" + wings[0]) : null;
if (wings.length > 1) flags.push("multiple wings");
let direction, first;
if ("first_dir" in C) {
// Redesigned form: one direction question and one colour question. The older per-direction colour
// columns are still in the sheet for some rows and are used only to fill gaps and to cross-check.
direction = DIRECTION_MAP[g("first_dir")] || null;
first = g("first_colour");
const legacy = ["West", "Center"].filter((d) => g("leg_" + d));
if (!direction && legacy.length === 1) {
direction = legacy[0];
flags.push("direction taken from the older sighting column");
}
const leg = direction ? g("leg_" + direction) : null;
if (!first && leg) first = leg;
else if (first && leg && first !== leg && first !== "Unsure" && leg !== "Unsure")
flags.push("first sighting differs between the old and new columns");
if (!direction) {
direction = "Unknown";
flags.push("direction missing");
}
} else {
const dirs = ["East", "West", "Center"].filter((d) => g("dir_" + d));
if (dirs.length === 1) {
direction = dirs[0];
first = g("dir_" + dirs[0]);
} else {
direction = dirs.length ? "Multiple" : "Unknown";
first = null;
flags.push("direction " + (dirs.length ? "multiple" : "missing"));
}
}
const companyRaw = g("company") || "Unknown";
const leftSquad = / > Left them$/.test(companyRaw);
const company = leftSquad ? companyRaw.replace(/ > Left them$/, "") : companyRaw;
// "left them" was only offered from 2026-10-27 (core/spec/form.yaml): earlier and undated squad runs are unknown
const leftKnown = leftSquad || !["1 Squadmate", "2 Squadmates"].includes(company) || (date && date >= "2026-10-27");
// Leaving your squadmates sends you into the solo pools, so either pool is plausible.
const pool = (SIGHTING_POOL[company + "|" + direction] || []).concat(leftSquad ? SIGHTING_POOL["Alone|" + direction] || [] : []);
if (first && first !== "Unsure" && pool.length && !pool.includes(first))
flags.push(`first sighting ${first} not in documented pool for ${company}/${direction}`);
const cadetRaw = g("cadet");
let scream, resolution, combat = null, combat2 = null;
if ("cadet_actions" in C) {
const act = g("cadet_actions");
const a = CADET_ACTIONS[act] || ["Unknown", null, null, "Unknown"];
if (act && !CADET_ACTIONS[act]) flags.push("unrecognised cadet answer");
[scream, combat, combat2] = a;
const out = CADET_OUTCOME[cadetRaw] || null;
if (a[3] !== null) {
resolution = a[3];
if (out === "DIED in combat" && !resolution.startsWith("DIED")) flags.push("died in combat on a path without combat");
} else resolution = out || "Unknown";
} else {
[scream, resolution] = CADET_MAP[cadetRaw] || ["Unknown", "Unknown"];
if (cadetRaw && !CADET_MAP[cadetRaw]) flags.push("unrecognised cadet answer");
combat = g("combat") ? COMBAT_MAP[g("combat")] || null : null;
}
const postRaw = g("post");
let post = null;
if (scream === "Investigate & help" && !resolution.startsWith("DIED"))
post = ["Forest", "West", "East"].includes(postRaw) ? postRaw : "Unknown";
else if (scream === "Ignore the scream" || scream === "Investigate & leave") post = "(no fork)";
const diedEarly = scream.startsWith("DIED") || resolution.startsWith("DIED");
let hour = HOUR_MAP[g("hour")] || null;
if (diedEarly) hour = null;
else if (hour === null) hour = "Unknown";
const enc = {};
for (const c of COLORS) enc[c] = g("enc_" + c);
const rare = RARE_MAP[g("rare")] || null;
const bonded = COLORS.filter((c) => enc[c] === "Bonded!");
const burned = COLORS.filter((c) => enc[c] === "Incinerated");
let branch = null;
if (hour === "Tree") {
if (enc.Blue || enc.Black || rare === "Shadow") branch = "Shadow (rare)";
else if (enc.Red === "Changed my mind" && enc.Green) branch = "Descend quickly: let the Red pass";
else if (enc.Green) {
branch = "Descend slowly: two Greens";
if (enc.Red) flags.push("tree run with both the Red and the two Greens attempted");
}
else if (enc.Red) branch = "Descend quickly: Red";
else if (enc.Orange || enc.Brown) {
branch = "Descend slowly: two Greens";
flags.push("tree run with Orange/Brown but no Green recorded");
}
} else if (hour === "Valley") {
if (enc.Green) flags.push("Green recorded on a Valley run (no Greens in the valley)");
const meadow = ["Orange", "Brown", "Blue"].some((c) => enc[c]) || rare === "Blue (valley)";
if (meadow || enc.Red === "Changed my mind") branch = "Passed the Red: meadow";
else if (enc.Red) branch = "Valley Red";
}
let outcome, outcomeColor = null;
if (diedEarly) outcome = "Died before the hour";
else if (bonded.length === 1) { outcome = "Bonded"; outcomeColor = bonded[0]; }
else if (bonded.length > 1) { outcome = "Ambiguous"; flags.push("more than one dragon marked Bonded"); }
else if (burned.length) {
const order = hour !== "Valley"
? ["Red", "Green", "Orange", "Brown", "Blue", "Black"]
: ["Red", "Orange", "Brown", "Blue", "Green", "Black"];
const inOrder = order.filter((c) => burned.includes(c));
outcome = "Incinerated";
outcomeColor = inOrder[inOrder.length - 1];
if (burned.length > 1) flags.push("more than one dragon marked Incinerated");
} else if (hour === "DIED before the hour") outcome = "Died before the hour";
else outcome = "No outcome recorded";
const rescued = enc.Green === "Saved by another" && outcome === "Bonded" && ["Orange", "Brown"].includes(outcomeColor);
const attempt = parseAttempt(g("attempt"));
if ((rare === "Blue (valley)" && hour === "Tree") || (rare === "Shadow" && hour === "Valley"))
flags.push("rare-dragon answer does not match the path");
const goalRaw = g("goal_v2");
const goal = g("goal") || (goalRaw && goalRaw !== "Unknown" ? GOAL_V2[goalRaw] || goalRaw : null) || "(not asked/blank)";
return {
source, date, wing, section, squad: g("squad") || "Unknown", attempt,
// "I've lost count" means many attempts, so it is a later attempt, not missing
attempt_group: attempt === 1 ? "First attempt" : attempt > 1 || /lost count/i.test(g("attempt") || "") ? "Later attempt" : "Unknown",
goal, company, left_squadmates: leftKnown ? leftSquad : null, direction, first_sighting: first || "Unknown",
sighting_action: "sighting_action" in C ? SIGHTING_ACTION[g("sighting_action")] || g("sighting_action") || "Unknown" : "(not asked)",
scream, resolution, combat_first: combat || "(not recorded)", combat_second: combat2 || "(not recorded)",
post_path: post, hour, branch,
enc_Red: enc.Red, enc_Green: enc.Green, enc_Orange: enc.Orange, enc_Brown: enc.Brown,
enc_Blue: enc.Blue, enc_Black: enc.Black,
outcome, outcome_color: outcomeColor, rescued,
saw_blue_meadow: !!(hour === "Valley" && (enc.Blue || rare === "Blue (valley)")),
saw_shadow: !!(hour === "Tree" && (enc.Blue || enc.Black || rare === "Shadow")),
rare_reported: "rare" in C ? rare || "Unknown" : "(not asked)",
malek_intent: "malek" in C ? g("malek") || "Unknown" : "(not asked)",
tail: g("tail"),
data_flags: flags.join("; "),
death_note: redact(g("death_note")), bond_note: redact(g("bond_note")),
};
}
function codeRows(rows, headers) {
if (!headers) {
const seen = new Set();
rows.forEach((r) => Object.keys(r).forEach((k) => seen.add(k)));
headers = [...seen];
}
const { C, missing } = findColumns(headers);
const runs = rows.map((r, i) => Object.assign({ run_id: i + 1 }, codeRow(r, C)));
return { runs, missing };
}
// ---------------------------------------------------------------- statistics
const Z95 = 1.959963984540054;
function wilson(k, n) {
if (!n) return { k, n, p: null, lo: null, hi: null };
const z = Z95, p = k / n, d = 1 + (z * z) / n;
const c = (p + (z * z) / (2 * n)) / d;
const h = (z * Math.sqrt((p * (1 - p)) / n + (z * z) / (4 * n * n))) / d;
return { k, n, p, lo: Math.max(0, c - h), hi: Math.min(1, c + h) };
}
const LOGF = [0];
function logFact(n) {
for (let i = LOGF.length; i <= n; i++) LOGF[i] = LOGF[i - 1] + Math.log(i);
return LOGF[n];
}
// Two-sided Fisher exact test: sum of probabilities of all tables (same margins) no more likely
// than the observed one (same rule as scipy.stats.fisher_exact, with a 1e-7 relative tolerance).
function fisher(a, b, c, d) {
const r1 = a + b, r2 = c + d, c1 = a + c, n = r1 + r2;
const lp = (x) => logFact(r1) + logFact(r2) + logFact(c1) + logFact(n - c1) -
logFact(n) - logFact(x) - logFact(r1 - x) - logFact(c1 - x) - logFact(r2 - c1 + x);
const lo = Math.max(0, c1 - r2), hi = Math.min(r1, c1);
const pObs = lp(a);
let p = 0;
for (let x = lo; x <= hi; x++) {
const q = lp(x);
if (q <= pObs + 1e-7 * Math.abs(pObs) + 1e-12) p += Math.exp(q);
}
return Math.min(1, p);
}
function oddsRatio(a, b, c, d) {
const corrected = a === 0 || b === 0 || c === 0 || d === 0;
const k = corrected ? 0.5 : 0;
const A = a + k, B = b + k, Cc = c + k, D = d + k;
const or = (A * D) / (B * Cc);
const se = Math.sqrt(1 / A + 1 / B + 1 / Cc + 1 / D);
return {
or, or_lo: Math.exp(Math.log(or) - Z95 * se), or_hi: Math.exp(Math.log(or) + Z95 * se),
fisher_p: fisher(a, b, c, d), corrected,
};
}
// Mantel-Haenszel common odds ratio across strata (tables [a,b,c,d]), Robins-Breslow-Greenland 95% CI,
// and the Cochran-Mantel-Haenszel chi-square test (1 df, with continuity correction).
function mantelHaenszel(tables) {
let R = 0, S = 0, PR = 0, PSQR = 0, QS = 0, sumA = 0, sumE = 0, sumV = 0;
for (const [a, b, c, d] of tables) {
const n = a + b + c + d;
if (n < 2) continue;
const r = (a * d) / n, s_ = (b * c) / n, P = (a + d) / n, Q = (b + c) / n;
R += r; S += s_; PR += P * r; PSQR += P * s_ + Q * r; QS += Q * s_;
const m1 = a + b, m0 = c + d, n1 = a + c, n0 = b + d;
sumA += a; sumE += (m1 * n1) / n;
sumV += (m1 * m0 * n1 * n0) / (n * n * (n - 1));
}
if (!R || !S) return { mh_or: null, mh_lo: null, mh_hi: null, mh_p: null };
const or = R / S;
const varLog = PR / (2 * R * R) + PSQR / (2 * R * S) + QS / (2 * S * S);
const se = Math.sqrt(varLog);
const chi = sumV > 0 ? Math.pow(Math.max(0, Math.abs(sumA - sumE) - 0.5), 2) / sumV : 0;
return { mh_or: or, mh_lo: Math.exp(Math.log(or) - Z95 * se), mh_hi: Math.exp(Math.log(or) + Z95 * se), mh_p: normSf2(Math.sqrt(chi)) };
}
function bh(ps) {
const m = ps.length, order = ps.map((p, i) => [p, i]).sort((x, y) => x[0] - y[0]);
const q = new Array(m);
let prev = 1;
for (let r = m; r >= 1; r--) {
const [p, i] = order[r - 1];
prev = Math.min(prev, (p * m) / r);
q[i] = prev;
}
return q;
}
// Normal CDF (Abramowitz-Stegun 7.1.26 is too coarse for small p; use erfc via continued fraction)
function normSf2(z) { // two-sided p for |z|
z = Math.abs(z);
const t = 1 / (1 + 0.5 * z / Math.SQRT2);
const x = z / Math.SQRT2;
const tau = t * Math.exp(-x * x - 1.26551223 + t * (1.00002368 + t * (0.37409196 + t * (0.09678418 +
t * (-0.18628806 + t * (0.27886807 + t * (-1.13520398 + t * (1.48851587 +
t * (-0.82215223 + t * 0.17087277)))))))));
return Math.min(1, tau); // erfc(x) = 2*P(Z>z)
}
function solve(A, b) { // Gaussian elimination with partial pivoting
const n = b.length, M = A.map((r, i) => r.concat([b[i]]));
for (let i = 0; i < n; i++) {
let p = i;
for (let r = i + 1; r < n; r++) if (Math.abs(M[r][i]) > Math.abs(M[p][i])) p = r;
[M[i], M[p]] = [M[p], M[i]];
for (let r = i + 1; r < n; r++) {
const f = M[r][i] / M[i][i];
for (let c = i; c <= n; c++) M[r][c] -= f * M[i][c];
}
}
const x = new Array(n).fill(0);
for (let i = n - 1; i >= 0; i--) {
let s = M[i][n];
for (let c = i + 1; c < n; c++) s -= M[i][c] * x[c];
x[i] = s / M[i][i];
}
return x;
}
function invert(A) {
const n = A.length;
return A.map((_, j) => solve(A, A.map((__, i) => (i === j ? 1 : 0)))).reduce((acc, col, j) => {
col.forEach((v, i) => (acc[i][j] = v));
return acc;
}, A.map(() => new Array(n).fill(0)));
}
// Logistic regression, Newton-Raphson with an L2 penalty (lambda) on all slopes (not the intercept).
function logisticRidge(X, y, lam = 1, iters = 100) {
const n = X.length, k = X[0].length;
let beta = new Array(k).fill(0);
const info = (b) => {
const H = Array.from({ length: k }, () => new Array(k).fill(0));
const grad = new Array(k).fill(0);
for (let i = 0; i < n; i++) {
let eta = 0;
for (let j = 0; j < k; j++) eta += X[i][j] * b[j];
const mu = 1 / (1 + Math.exp(-eta)), w = mu * (1 - mu);
for (let j = 0; j < k; j++) {
grad[j] += X[i][j] * (y[i] - mu);
if (!X[i][j]) continue;
for (let l = 0; l < k; l++) H[j][l] += X[i][j] * X[i][l] * w;
}
}
for (let j = 1; j < k; j++) { H[j][j] += lam; grad[j] -= lam * b[j]; }
return { H, grad };
};
for (let it = 0; it < iters; it++) {
const { H, grad } = info(beta);
const step = solve(H, grad);
beta = beta.map((v, j) => v + step[j]);
if (Math.max(...step.map(Math.abs)) < 1e-10) break;
}
const { H } = info(beta);
const Hi = invert(H);
const se = Hi.map((r, j) => Math.sqrt(r[j]));
return { beta, se };
}
// ---------------------------------------------------------------- analysis helpers
const rate = (runs, pred) => wilson(runs.filter(pred).length, runs.length);
const groupBy = (arr, key) => arr.reduce((m, r) => {
const k = typeof key === "function" ? key(r) : r[key];
(m[k] = m[k] || []).push(r);
return m;
}, {});
const sortedKeys = (o) => Object.keys(o).sort();
const stratum = (r) => r.source !== "Form" ? "Bulk logs (imports)" :
r.goal === "Blue" ? "Form: chasing Blue" : r.goal === "Black" ? "Form: chasing Black" :
r.goal === "Blue or Black" ? "Form: chasing Blue or Black" : "Form: other/blank goal";
const goal2 = (r) => isRareGoal(r.goal) ? r.goal : r.goal === "(not asked/blank)" ? "Blank/not asked" : "Other colour/any";
// Denominators for each outcome ("who could have had this outcome?")
const OUTCOMES = {
blue_meadow: {
label: "Blue appears in the meadow", denom: (r) => r.branch === "Passed the Red: meadow", event: (r) => r.saw_blue_meadow,
},
shadow: {
label: "Shadow option appears at the tree", denom: (r) => r.hour === "Tree" && r.branch !== null, event: (r) => r.saw_shadow,
},
bond_blue_meadow: {
label: "Bond, given you approached the meadow Blue", denom: (r) => r.branch === "Passed the Red: meadow" && ["Bonded!", "Incinerated"].includes(r.enc_Blue),
event: (r) => r.enc_Blue === "Bonded!",
},
reach_hour: {
label: "Survive to the hour (tree/valley choice)", denom: () => true, event: (r) => r.hour === "Tree" || r.hour === "Valley",
},
};
for (const c of COLORS) {
OUTCOMES["bonded_" + c] = {
label: `Run ends bonded to ${c}`, denom: (r) => r.hour === "Tree" || r.hour === "Valley",
event: (r) => r.outcome === "Bonded" && r.outcome_color === c,
};
}
const FACTORS = ["company", "direction", "first_sighting", "scream", "resolution", "combat_first",
"post_path", "hour", "wing", "squad", "attempt_group", "source", "goal", "sighting_action", "combat_second", "malek_intent"];
// the two Greens are met by climbing down slowly, or by climbing down quickly and letting the Red pass
const GREENS = (r) => r.branch === "Descend slowly: two Greens" || r.branch === "Descend quickly: let the Red pass";
// Encounter contexts: which recorded dragon attempt belongs to which in-game encounter.
const CONTEXTS = [
["Red (tree, descend quickly)", (r) => r.hour === "Tree" && r.branch === "Descend quickly: Red", "enc_Red", "Red"],
["Green (two Greens)", (r) => r.hour === "Tree" && GREENS(r), "enc_Green", "Green"],
["Blue (shadow)", (r) => r.hour === "Tree" && r.branch === "Shadow (rare)", "enc_Blue", "Blue"],
["Black (shadow)", (r) => r.hour === "Tree" && r.branch === "Shadow (rare)", "enc_Black", "Black"],
["Red (valley)", (r) => r.hour === "Valley" && r.branch === "Valley Red", "enc_Red", "Red"],
["Orange (meadow)", (r) => r.hour === "Valley" && r.branch === "Passed the Red: meadow", "enc_Orange", "Orange"],
["Brown (meadow)", (r) => r.hour === "Valley" && r.branch === "Passed the Red: meadow", "enc_Brown", "Brown"],
["Blue (meadow)", (r) => r.hour === "Valley" && r.branch === "Passed the Red: meadow", "enc_Blue", "Blue"],
];
function applyFilters(runs, f = {}) {
return runs.filter((r) => {
if (f.source === "form" && r.source !== "Form") return false;
if (f.source === "bulk" && r.source === "Form") return false;
if (f.goal === "exclude_rare" && isRareGoal(r.goal)) return false;
if (f.goal === "only_rare" && !isRareGoal(r.goal)) return false;
if (f.excludeFlagged && r.data_flags) return false;
return true;
});
}
function factorTable(runs, outKey, minN = 5) {
const O = OUTCOMES[outKey];
const D = runs.filter(O.denom);
const total = D.length, events = D.filter(O.event).length;
const rows = [], missing = {};
for (const f of FACTORS) {
const groups = groupBy(D.filter((r) => !isMissing(r[f])), f);
missing[f] = D.filter((r) => isMissing(r[f])).length;
const nf = Object.values(groups).reduce((s, g) => s + g.length, 0);
const ef = Object.values(groups).reduce((s, g) => s + g.filter(O.event).length, 0);
for (const lv of sortedKeys(groups)) {
const g = groups[lv], k = g.filter(O.event).length, n = g.length;
const row = Object.assign({ factor: f, level: lv }, wilson(k, n));
const rk = ef - k, rn = nf - n; // rest of the same factor
if (n >= minN && rn >= minN && ef > 0 && ef < nf) Object.assign(row, oddsRatio(k, n - k, rk, rn - rk));
rows.push(row);
}
}
const tested = rows.filter((r) => r.fisher_p !== undefined);
bh(tested.map((r) => r.fisher_p)).forEach((q, i) => (tested[i].q = q));
return { outcome: outKey, label: O.label, overall: wilson(events, total), rows, missing };
}
// Multivariable model: one row per run in the outcome's denominator; categorical factors dummy-coded
// against their most common level.
function model(runs, outKey, factors, lam = 1) {
const O = OUTCOMES[outKey];
const D = runs.filter(O.denom).map((r) => {
const o = { y: O.event(r) ? 1 : 0 };
for (const [name, fn] of Object.entries(factors)) o[name] = fn(r);
return o;
}).filter((o) => Object.keys(factors).every((f) => o[f] !== null && o[f] !== undefined));
if (D.length < 20) return null;
const cols = [["(intercept)", () => 1]], refs = {};
for (const f of Object.keys(factors)) {
const counts = groupBy(D, f);
const levels = sortedKeys(counts).sort((a, b) => counts[b].length - counts[a].length || (a < b ? -1 : 1));
refs[f] = levels[0];
for (const lv of levels.slice(1)) cols.push([`${f} = ${lv}`, (o) => (o[f] === lv ? 1 : 0), f, lv, counts[lv].length]);
}
const X = D.map((o) => cols.map((c) => c[1](o)));
const y = D.map((o) => o.y);
const { beta, se } = logisticRidge(X, y, lam);
const terms = cols.slice(1).map((c, i) => {
const b = beta[i + 1], s = se[i + 1];
return { term: c[0], factor: c[2], level: c[3], n: c[4], or: Math.exp(b), lo: Math.exp(b - Z95 * s), hi: Math.exp(b + Z95 * s), p: normSf2(b / s) };
});
return { outcome: outKey, label: O.label, n: D.length, events: y.reduce((a, b) => a + b, 0), refs, lambda: lam, terms };
}
const routeKey = (r) => [r.company, r.direction, r.first_sighting, r.scream, r.resolution, r.post_path || "—", r.hour || "—"];
const ROUTE_FIELDS = ["company", "direction", "first_sighting", "scream", "resolution", "post_path", "hour"];
function analyse(allRuns, filters = {}) {
const runs = applyFilters(allRuns, filters);
const res = { filters, n: runs.length };
// meta / data quality
const flagCounts = {};
runs.forEach((r) => r.data_flags && r.data_flags.split("; ").forEach((f) => {
const key = f.startsWith("first sighting") ? "first sighting outside documented pool" : f;
flagCounts[key] = (flagCounts[key] || 0) + 1;
}));
const dates = runs.map((r) => r.date).filter(Boolean).sort();
res.meta = {
sources: Object.fromEntries(Object.entries(groupBy(runs, "source")).map(([k, v]) => [k, v.length])),
date_min: dates[0] || null, date_max: dates[dates.length - 1] || null,
n_flagged: runs.filter((r) => r.data_flags).length, flag_counts: flagCounts,
outcomes: Object.fromEntries(Object.entries(groupBy(runs, (r) => r.outcome + (r.outcome_color ? " " + r.outcome_color : ""))).map(([k, v]) => [k, v.length])),
};
// chance events, overall and by stratum
res.chance = {};
for (const key of ["blue_meadow", "shadow"]) {
const O = OUTCOMES[key], D = runs.filter(O.denom);
res.chance[key] = {
label: O.label, overall: rate(D, O.event),
by_stratum: Object.fromEntries(sortedKeys(groupBy(D, stratum)).map((s) => [s, rate(groupBy(D, stratum)[s], O.event)])),
by_goal: Object.fromEntries(sortedKeys(groupBy(D, "goal")).map((s) => [s, rate(groupBy(D, "goal")[s], O.event)])),
by_attempt: Object.fromEntries(sortedKeys(groupBy(D, "attempt_group")).map((s) => [s, rate(groupBy(D, "attempt_group")[s], O.event)])),
};
}
// first sighting pools
const fs = groupBy(runs.filter((r) => !isMissing(r.direction) && !isMissing(r.company)), (r) => r.company + "|" + r.direction);
res.first_sighting = sortedKeys(fs).map((k) => {
const g = fs[k], cnt = groupBy(g, "first_sighting");
return { key: k, n: g.length, colors: Object.fromEntries(sortedKeys(cnt).map((c) => [c, cnt[c].length])) };
});
// bonding encounters
res.encounters = CONTEXTS.map(([name, where, col, color]) => {
const A = runs.filter((r) => where(r) && ["Bonded!", "Incinerated"].includes(r[col]));
const win = (r) => r[col] === "Bonded!";
const intent = A.filter((r) => r.goal === color || r.goal.startsWith("Any") || (r.goal === "Blue or Black" && (color === "Blue" || color === "Black")));
const byS = groupBy(A, stratum);
return {
context: name, color, overall: rate(A, win), intent_matched: rate(intent, win),
by_stratum: Object.fromEntries(sortedKeys(byS).map((s) => [s, rate(byS[s], win)])),
};
});
const greens = runs.filter((r) => r.hour === "Tree" && GREENS(r) && r.enc_Green);
res.green_rescue = {
green_encounters: greens.length,
bonded_green: greens.filter((r) => r.enc_Green === "Bonded!").length,
rescued_orange: greens.filter((r) => r.rescued && r.outcome_color === "Orange").length,
rescued_brown: greens.filter((r) => r.rescued && r.outcome_color === "Brown").length,
incinerated: greens.filter((r) => r.outcome === "Incinerated").length,
};
// factor tables
res.factors = {};
for (const key of Object.keys(OUTCOMES)) res.factors[key] = factorTable(runs, key);
// adjusted models
const first2 = (r) => (["Red", "Brown", "Orange", "Green"].includes(r.first_sighting) ? r.first_sighting : null);
const known = (f) => (r) => (isMissing(r[f]) ? null : r[f]);
const src2 = (r) => (r.source === "Form" ? "Form submission" : "Bulk log (import)");
res.models = {
blue_meadow: model(runs, "blue_meadow", {
source: src2, goal: goal2, direction: known("direction"), first_sighting: first2,
resolution: known("resolution"), company: known("company"),
}),
shadow: model(runs, "shadow", {
source: src2, goal: goal2, attempt: known("attempt_group"), direction: known("direction"),
}),
bond_blue_meadow: model(runs, "bond_blue_meadow", {
source: src2, goal: goal2, direction: known("direction"), resolution: known("resolution"),
}),
};
// daily trend (form submissions only, since imports have no date)
const byDate = groupBy(runs.filter((r) => r.date), "date");
res.daily = sortedKeys(byDate).map((d) => {
const g = byDate[d];
const m = g.filter(OUTCOMES.blue_meadow.denom), t = g.filter(OUTCOMES.shadow.denom);
return { date: d, runs: g.length, meadow_n: m.length, blue_k: m.filter((r) => r.saw_blue_meadow).length, tree_n: t.length, shadow_k: t.filter((r) => r.saw_shadow).length };
});
// routes (choices up to the hour) with outcome rates
const R = groupBy(runs.filter((r) => r.hour === "Tree" || r.hour === "Valley"), (r) => routeKey(r).join(" › "));
res.routes = sortedKeys(R).map((k) => {
const g = R[k], m = g.filter(OUTCOMES.blue_meadow.denom), t = g.filter(OUTCOMES.shadow.denom);
const o = { route: k, parts: routeKey(g[0]), n: g.length, blue_meadow: rate(m, (r) => r.saw_blue_meadow), shadow: rate(t, (r) => r.saw_shadow) };
for (const c of COLORS) o["bonded_" + c] = rate(g, (r) => r.outcome === "Bonded" && r.outcome_color === c);
o.bonded_any = rate(g, (r) => r.outcome === "Bonded");
return o;
});
return res;
}
// ---------------------------------------------------------------- route-map graph
const STAGES = [
{ key: "company", label: "Company" },
{ key: "direction", label: "Direction" },
{ key: "first_sighting", label: "First sighting" },
{ key: "cadet", label: "Cadet encounter" },
{ key: "post_path", label: "Fork in the road" },
{ key: "hour", label: "After an hour" },
{ key: "encounter", label: "Encounter" },
{ key: "end", label: "Outcome" },
];
const CHANCE_STAGES = new Set(["first_sighting", "encounter"]); // decided (partly) by the game, not the player
function stageValue(r, key) {
switch (key) {
case "company": return isMissing(r.company) ? null : r.company;
case "direction": return isMissing(r.direction) ? null : r.direction;
case "first_sighting": return r.scream === "DIED at first sighting" || isMissing(r.first_sighting) ? null : r.first_sighting;
case "cadet":
if (r.scream.startsWith("DIED") || isMissing(r.scream) || isMissing(r.resolution)) return null;
return r.scream === "Investigate & help" ? "Help → " + r.resolution : r.scream + " → " + r.resolution;
case "post_path": return isMissing(r.post_path) ? null : r.post_path;
case "hour": return r.hour === "Tree" ? "Climb the tree" : r.hour === "Valley" ? "Valley centre" : null;
case "encounter": {
if (!r.branch) return null;
if (r.hour === "Tree") {
if (r.branch === "Shadow (rare)") return "Tree: shadow → Blue/Black";
if (r.branch === "Descend quickly: Red") return "Tree: Red below";
return "Tree: two Greens";
}
if (r.branch === "Valley Red") return "Valley: bond the Red";
if (r.enc_Blue) return "Meadow: Blue";
if (r.enc_Orange && r.enc_Brown) return "Meadow: Orange/Brown";
if (r.enc_Orange) return "Meadow: Orange";
if (r.enc_Brown) return "Meadow: Brown";
return "Meadow (no attempt recorded)";
}
case "end":
if (r.outcome === "Bonded") return "Bonded " + r.outcome_color;
if (r.outcome === "Incinerated") return "Incinerated";
if (r.outcome === "Died before the hour") return "Died before the hour";
return null;
}
return null;
}
// pool = which runs could have reached this outcome at all (used by rankRoutes as the comparison group)
const BOTH = (r) => r.hour === "Tree" || r.hour === "Valley";
const TREE = (r) => r.hour === "Tree";
const VALLEY = (r) => r.hour === "Valley";
const TARGETS = {
bond_Blue: { label: "Bond Blue", color: "Blue", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Blue", pool: VALLEY },
bond_Black: { label: "Bond Black", color: "Black", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Black", pool: TREE },
see_Blue: { label: "Meet a Blue (meadow)", color: "Blue", fn: (r) => r.saw_blue_meadow, pool: (r) => r.branch === "Passed the Red: meadow" },
see_shadow: { label: "Get the tree shadow", color: "Black", fn: (r) => r.saw_shadow, pool: (r) => r.hour === "Tree" && r.branch !== null },
bond_Red: { label: "Bond Red", color: "Red", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Red", pool: BOTH },
bond_Green: { label: "Bond Green", color: "Green", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Green", pool: TREE },
bond_Orange: { label: "Bond Orange", color: "Orange", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Orange", pool: BOTH },
bond_Brown: { label: "Bond Brown", color: "Brown", fn: (r) => r.outcome === "Bonded" && r.outcome_color === "Brown", pool: BOTH },
bond_any: { label: "Bond any dragon", color: null, fn: (r) => r.outcome === "Bonded", pool: BOTH },
};
// conditions: { stageKey: Set(values) } — a run passes if, for every stage with a condition, its value is in the set.
function buildGraph(runs, targetKey, conditions = {}) {
const T = TARGETS[targetKey].fn;
const sel = runs.filter((r) => Object.entries(conditions).every(([k, set]) => !set.size || set.has(stageValue(r, k))));
const nodes = new Map(), links = new Map();
const node = (si, v) => {
const id = si + "|" + v;
if (!nodes.has(id)) nodes.set(id, { id, stage: si, stageKey: STAGES[si].key, value: v, n: 0, k: 0 });
return nodes.get(id);
};
for (const r of sel) {
const hit = T(r) ? 1 : 0;
let prev = null;
STAGES.forEach((s, si) => {
const v = stageValue(r, s.key);
if (v === null || v === undefined) return;
const nd = node(si, v);
nd.n++; nd.k += hit;
if (prev) {
const id = prev.id + "→" + nd.id;
if (!links.has(id)) links.set(id, { id, source: prev.id, target: nd.id, n: 0, k: 0, chance: CHANCE_STAGES.has(s.key) });
const l = links.get(id); l.n++; l.k += hit;
}
prev = nd;
});
}
const out = (o) => Object.assign(o, wilson(o.k, o.n));
return {
stages: STAGES, selected: sel.length, target: targetKey,
nodes: [...nodes.values()].map(out), links: [...links.values()].map(out),
};
}
// Rank full routes (every choice up to the hour) for a target outcome.
// Each route with at least minN runs is compared with all other runs by Fisher's exact test;
// q-values use Benjamini-Hochberg across all ranked routes. Ranking uses the lower 95% Wilson
// bound, so a route needs both a high rate and enough runs to rank high.
function rankRoutes(runs, targetKey, minN = 10, denomStage = null) {
const T = TARGETS[targetKey].fn;
const keyStages = STAGES.slice(0, 6).map((s) => s.key);
const pool = runs.filter(TARGETS[targetKey].pool);
const totK = pool.filter(T).length, totN = pool.length;
// a route is only ranked from runs whose every step is known; runs with a gap still count in the comparison group
const groups = groupBy(pool.filter((r) => keyStages.every((k) => stageValue(r, k) || (k === "post_path" && r.post_path === null))),
(r) => keyStages.map((k) => stageValue(r, k) || "—").join(" › "));
const rows = Object.entries(groups).filter(([, g]) => g.length >= minN).map(([key, g]) => {
const k = g.filter(T).length, n = g.length;
const o = Object.assign({ route: key, values: keyStages.map((s) => stageValue(g[0], s)) }, wilson(k, n));
Object.assign(o, oddsRatio(k, n - k, totK - k, totN - n - (totK - k)));
// same comparison, adjusted for data source / chase-goal stratum (reporting bias)
const inRoute = new Set(g);
const tabs = Object.values(groupBy(pool, stratum)).map((sg) => {
const a = sg.filter((r) => inRoute.has(r) && T(r)).length, nin = sg.filter((r) => inRoute.has(r)).length;
const c = sg.filter((r) => !inRoute.has(r) && T(r)).length, nout = sg.length - nin;
return [a, nin - a, c, nout - c];
});
Object.assign(o, mantelHaenszel(tabs));
return o;
});
bh(rows.map((r) => r.fisher_p)).forEach((q, i) => (rows[i].q = q));
const mhIdx = rows.map((r, i) => [r.mh_p, i]).filter(([p]) => p !== null);
bh(mhIdx.map(([p]) => p)).forEach((q, j) => (rows[mhIdx[j][1]].mh_q = q));
rows.sort((a, b) => b.lo - a.lo || b.n - a.n);
return { target: targetKey, overall: wilson(totK, totN), minN, n_routes_tested: rows.length, rows };
}
return {
COLORS, MISSING, isMissing, COLUMN_KEYS, STAGES, TARGETS, OUTCOMES, FACTORS, ROUTE_FIELDS, rankRoutes,
findColumns, codeRow, codeRows, parseTimestamp,
wilson, fisher, oddsRatio, bh, logisticRidge, mantelHaenszel,
applyFilters, analyse, factorTable, model, buildGraph, stageValue, stratum,
};
});
+199 -800
View File
File diff suppressed because it is too large Load Diff
+22 -3
View File
@@ -51,9 +51,10 @@ min_support: 20
# strength learned from the data).
chance:
fs: { label: First dragon, outcomes: [Red, Brown, Green, Orange], context: [direction, company, left] }
dieSight: { label: Survive the first dragon, outcomes: [die, live], context: [fs, direction, company, left, react] }
dieCombat: { label: Survive the fight, outcomes: [die, live], context: [fight, dagger, fs, direction] }
dieHour: { label: Survive the hour, outcomes: [die, live], context: [cadet, post] }
# rolls that can kill you: risk_label names the danger when the advisor reports it
dieSight: { label: Survive the first dragon, outcomes: [die, live], context: [fs, direction, company, left, react], risk_label: Killed at the first sighting }
dieCombat: { label: Survive the fight, outcomes: [die, live], context: [fight, dagger, fs, direction], risk_label: Killed in the fight }
dieHour: { label: Survive the hour, outcomes: [die, live], context: [cadet, post], risk_label: Killed before the hour }
shadow: { label: Shadow at the tree, outcomes: ["yes", "no"], context: [cadet, post, fs, direction, company] }
shadowColour: { label: Shadow dragon, outcomes: [Black, Blue], context: [] }
meadow: { label: Meadow dragon, outcomes: [Orange, Brown, Blue], context: [cadet, post, fs, direction, company] }
@@ -192,3 +193,21 @@ from_form:
ignored: [Tourniquet, Left injured cadet, Took their weapons]
# a rider's stated goal that means "chasing a rare dragon"
rare_goals: [Blue, Black, Blue or Black]
# ---------------------------------------------------------------- the bot: which on-screen text is which option
# Regular expressions over the option text, lower-cased with punctuation removed (see core/graph.js norm()).
# recommendOption() in core/solver.js uses these to point the bot at the recommended option on its screen.
bot_text:
company: { Alone: "^enter the valley alone", "1 Squadmate": "^enter the valley with one", "2 Squadmates": "^enter the valley with two" }
direction: { East: "^head east", West: "^head west", Center: "^head toward the center of the forest" }
react: { Hold still: "hold still", Respect / face it: "in respect|turn to face it", Retreat / run: "^run and hope", Hide: "^hide", Meet its gaze: "meeting its gaze" }
scream: { Investigate the scream: "^investigate the scream", Keep walking: "^keep walking" }
clearing: { Help your squadmate: "^help your squadmate", Walk away: "^play it safe and leave them" }
fight: { Pause to assess: "^pause to assess", Charge the smaller cadet: "^charge the other cadet", Charge between the bigger cadet and your squadmate: "^unsheathe your weapon" }
dagger: { Engage the smaller cadet: "^engage the smaller cadet", Charge the bigger cadet anyway: "^daggers or not" }
fate: { Killed the bullies yourself: "^end the lives of the bullies", Squadmate's revenge: "^allow your squadmate to exact", Left bullies to the dragons: "^leave the bullies alive" }
wounded: { Tourniquet: "tourniquet", Left injured cadet: "leave (them|him|her) to die", Took their weapons: "take (their|his|her) weapons?" }
post: { East: "^walk toward the east", West: "^walk toward the west", Forest: "^walk farther into the forest" }
hour: { Valley: "^continue toward the center of the valley", Tree: "^find the tallest tree" }
descend: { Quickly (Red): "^descend quickly", Slowly (two Greens): "^climb down slowly" }
red: { Jump onto the Red: "^jump", Let it pass (two Greens): "^reds are too hot tempered" }
File diff suppressed because one or more lines are too long
+17
View File
@@ -49,6 +49,23 @@ const OUT = path.join(__dirname, '..', 'vendor', 'threshing-atlas');
Run it by hand, or once a day before the first attempt. Commit `vendor/threshing-atlas/` in the bot repo if you
want the exact policy in its history.
### What changed in November 2026
The calls above are unchanged, but `solver.js` is now a small advisor (about 250 lines) that reads the game itself
from `policy.json` (`samples.*.model.game`, from `core/spec/game.yaml`) instead of having it written into the code.
For the bot that means:
- Histories (`h`) use the game's step names: `company`, `leave`, `direction`, `fs`, `react`, `scream`, `clearing`,
`fight`, `dagger`, `fate` or `wounded`, `post`, `hour`, `shadow`, `shadowColour`, `descend`, `red`, `valley`,
`meadow`, `encounter`. `solverHistory()` already produces these; the old `cadet` and `left` keys are no longer read.
- New choices the bot will be asked about: leaving your squadmates (`leave`), how you react to the first dragon
(`react`), and, below the tree, jumping onto the Red or letting it pass (`red`). The text the bot sees for each
option is in `game.yaml` under `bot_text`, so new screens can be matched by editing that file.
- If a history skips a decision but already has a later step (an older record, say), the recommended option is
assumed for the skipped one.
- Re-sync `solver.js`, `graph.js` and `policy.json` together: the new `solver.js` needs the game inside the new
`policy.json`.
### Asking for advice on a screen
```js
+4 -3
View File
@@ -34,7 +34,8 @@ share.
2. Route model in Python, built from `core/spec/game.yaml`. **Done** (the bot's text, `core/spec/crosswalk.yaml`, moves with the bot importer and the JS advisor in step 4).
3. **Done.** Fix the coding bugs found in the core review (letting the Red pass, "left your squadmates" before the
redesigned form, missing bot text), each with a rule test.
4. The site shows precomputed statistics with fixed filter presets (**done**: `analysis/stats/site_stats.py`,
`measures.yaml`, `data/public/stats.json`; the upload button and the click-to-select route map are gone); next,
a small JS advisor that reads the Python-built model replaces `solver.js`, and `engine.js` goes.
4. **Done.** The site shows precomputed statistics with fixed filter presets (`analysis/stats/site_stats.py`,
`measures.yaml`, `data/public/stats.json`); the upload button and the click-to-select route map are gone; a
small JS advisor (`core/solver.js`, about 250 lines) reads the game and the Python-built model from `policy.json`
and replaces the 900-line JS solver; `engine.js` is gone.
5. `data/dictionary.yaml` for every published column, and Croissant metadata for the datasets.
+3 -5
View File
@@ -6,11 +6,9 @@
"scripts": {
"build": "node site/build.js",
"dev": "node site/build.js && python3 -m http.server 8000 -d dist",
"test:js": "node tests/parity/run_engine.js data/public/form_responses.json tests/out",
"test:py": "PYTHONPATH=core python3 -m atlas.coding && cd analysis/stats && PYTHONPATH=../../core python3 analyze.py && cd ../.. && PYTHONPATH=core python3 -m unittest discover -s tests/core -p 'test_*.py'",
"test:parity": "python3 tests/parity/compare_runs.py && python3 tests/parity/compare_results.py && python3 tests/parity/compare_routes.py && python3 tests/parity/compare_solver.py && python3 tests/parity/compare_stats.py",
"test:unit": "node --test tests/core/graph.test.js tests/core/solver.test.js",
"test": "npm run test:js && npm run test:py && npm run test:parity && npm run test:unit",
"test:py": "PYTHONPATH=core python3 -m atlas.coding && PYTHONPATH=core python3 -m unittest discover -s tests/core -p 'test_*.py'",
"test:unit": "node --test tests/core/graph.test.js tests/core/advisor.test.js",
"test": "npm run test:py && npm run test:unit",
"graph": "node analysis/network/build-graph.js",
"import-bot": "node collection/bot/import-bot.js",
"check-sources": "python3 collection/sources/update_sources.py",
+3 -1
View File
@@ -785,7 +785,8 @@
return;
}
const c = parts[1] ? COLOURS.find((x) => x.toLowerCase() === parts[1].toLowerCase()) : null;
const enc = parts[2] && c && S.ENCOUNTERS[c].includes(parts[2]) ? parts[2] : null;
// an encounter that is not one of this colour's is ignored once the route model has loaded (see render)
const enc = parts[2] && c ? parts[2] : null;
// plain "#valley" (e.g. the tab itself) keeps whatever the reader was looking at
if (!c) { writeHash(); return; }
if (!st.policy) { st.pendingHash = { c, enc }; return; }
@@ -856,6 +857,7 @@
render();
fetch("data/policy.json").then((r) => r.json()).then((p) => {
st.policy = p;
S.useGame(p.samples.all); // the game (and where each colour can be bonded) comes with the route model
if (st.pendingHash) { st.colour = st.pendingHash.c; st.focus = st.pendingHash.enc; st.pendingHash = null; }
render();
}).catch(() => { $("#ex-panel").innerHTML = '<p class="small muted">The route model (data/policy.json) could not be loaded.</p>'; });
-1
View File
@@ -2,7 +2,6 @@
* Reads the same coded runs and results as the rest of the page (window.ThreshAtlas.state). */
(function () {
"use strict";
const E = window.ThreshEngine;
const $ = (q, el = document) => el.querySelector(q);
const $$ = (q, el = document) => [...el.querySelectorAll(q)];
const NS = "http://www.w3.org/2000/svg";
+112
View File
@@ -0,0 +1,112 @@
// The route advisor (core/solver.js): it must give exactly the Python route model's answers, and walk a run in the
// game's order. node --test tests/core/advisor.test.js
const test = require("node:test");
const assert = require("node:assert");
const path = require("node:path");
const fs = require("node:fs");
const os = require("node:os");
const { execFileSync } = require("node:child_process");
const S = require("../../core/solver");
const ROOT = path.join(__dirname, "../..");
const published = JSON.parse(fs.readFileSync(path.join(ROOT, "data/public/policy.json"), "utf8"));
// a synthetic world with known answers, built by the Python model (tests/fixtures/synthetic_policy.py)
const tmp = path.join(fs.mkdtempSync(path.join(os.tmpdir(), "atlas-")), "synthetic.json");
execFileSync("python3", [path.join(ROOT, "tests/fixtures/synthetic_policy.py"), tmp], { stdio: "inherit" });
const policy = JSON.parse(fs.readFileSync(tmp, "utf8"));
const choice = (X, h, obj = "thisAttempt") => S.advisor({ model: policy.model }, X, obj).next(h).choice;
test("the advisor gives exactly the Python model's values and choices, for every colour and goal", () => {
let checked = 0;
for (const smp of Object.values(published.samples)) {
for (const [X, t] of Object.entries(smp.targets)) {
for (const obj of ["thisAttempt", "eventually"]) {
const adv = S.advisor(smp, X, obj);
assert.ok(Math.abs(adv.value.x - t[obj].pX.point) < 1e-12, `${X} ${obj}: ${adv.value.x} vs ${t[obj].pX.point}`);
// follow the published tree: every decision on the route must match, option by option
let n = t[obj].tree, h = {};
while (n && n.type !== "end") {
if (n.type === "decision") {
const s = adv.step(h);
assert.strictEqual(s.var, n.var); assert.strictEqual(s.choice, n.choice, `${X} ${obj} ${n.var}`);
for (const [o, v] of Object.entries(n.options)) assert.ok(Math.abs(s.options[o].pX - v.pX) < 1e-12, `${X} ${obj} ${n.var}=${o}`);
checked++;
if (n.var === "encounter") break;
h[n.var] = n.choice; n = n.next;
} else {
const [k, b] = Object.entries(n.branches).filter(([k]) => k !== "die").sort((a, c) => c[1].p - a[1].p)[0];
if (!n.var.startsWith("die")) h[n.var] = k;
n = b.next;
}
}
}
}
}
assert.ok(checked > 100, checked);
});
test("synthetic world: Blue goes to the valley, passes the Red, attempts only the Blue", () => {
const h = { company: "Alone", direction: "Center", fs: "Red", react: "Hold still", fate: "Killed the bullies yourself", post: "Forest" };
assert.strictEqual(choice("Blue", h), "Valley");
assert.strictEqual(choice("Blue", { ...h, hour: "Valley" }), "Pass the Red (meadow)");
const meadow = { ...h, hour: "Valley", valley: "Pass the Red (meadow)" };
assert.strictEqual(choice("Blue", { ...meadow, meadow: "Blue" }), "Attempt the bond");
assert.strictEqual(choice("Blue", { ...meadow, meadow: "Orange" }), "Die on purpose");
assert.strictEqual(choice("Red", { ...h, hour: "Valley" }), "Face the Red");
assert.strictEqual(choice("Black", h), "Tree");
});
test("routing to one encounter: Blue only at the meadow equals the Blue policy; only at the tree Greens gives 0", () => {
const smp = { model: policy.model };
const meadow = S.advisor(smp, "Blue", "thisAttempt", { only: ["meadow-Blue"] });
const all = S.advisor(smp, "Blue", "thisAttempt");
assert.ok(Math.abs(meadow.value.x - all.value.x) < 1e-9);
assert.ok(S.advisor(smp, "Blue", "thisAttempt", { only: ["tree-Greens"] }).value.x < 1e-9);
assert.ok(S.advisor(smp, "Brown", "thisAttempt", { only: ["tree-Greens"] }).value.x > 0.05); // the Greens rescue
assert.ok(meadow.attempts("meadow-Blue") > 100);
});
test("a run is walked in the game's order", () => {
const adv = S.advisor(published.samples.all, "Blue", "thisAttempt");
assert.strictEqual(adv.step({}).var, "company");
assert.strictEqual(adv.step({ company: "1 Squadmate" }).var, "leave");
assert.strictEqual(adv.step({ company: "1 Squadmate", leave: "Stay together" }).var, "direction");
assert.strictEqual(adv.step({ company: "Alone", direction: "Center" }).kind, "chance"); // the first dragon is a roll
const seen = { company: "Alone", direction: "Center", fs: "Red" };
assert.strictEqual(adv.step(seen).var, "react");
const base = { ...seen, react: "Hold still" };
const scream = adv.step(base);
assert.strictEqual(scream.var, "scream");
assert.strictEqual(scream.risk.label, "Killed at the first sighting");
const help = { ...base, scream: "Investigate the scream", clearing: "Help your squadmate" };
assert.strictEqual(adv.step({ ...base, scream: "Investigate the scream" }).var, "clearing");
assert.strictEqual(adv.step(help).var, "fight");
assert.strictEqual(adv.step({ ...help, fight: "Pause to assess" }).var, "dagger");
assert.strictEqual(adv.step({ ...help, fight: "Charge the smaller cadet" }).var, "fate"); // charging skips the dagger
const fate = adv.step({ ...help, fight: "Pause to assess", dagger: "Engage the smaller cadet" });
assert.strictEqual(fate.var, "fate");
assert.strictEqual(fate.risk.label, "Killed in the fight");
assert.strictEqual(adv.step({ ...base, scream: "Keep walking" }).var, "wounded");
assert.strictEqual(adv.step({ ...base, scream: "Investigate the scream", clearing: "Walk away" }).var, "hour");
// an older record that only knows the bullies' fate still lands in the right place (the skipped steps are
// assumed to be the recommended ones)
assert.strictEqual(adv.next({ ...base, fate: "Killed the bullies yourself" }).decision, "post");
});
test("recommendOption points the bot at the recommended option on its screen", () => {
const adv = S.advisor(published.samples.all, "Blue", "thisAttempt");
const h = { company: "Alone", direction: "Center", fs: "Red", react: "Hold still", scream: "Keep walking", wounded: "Tourniquet" };
const rec = S.recommendOption(adv, h, ["Continue toward the center of the valley", "Find the tallest tree and climb for a better view"]);
assert.strictEqual(rec.decision, "hour");
assert.strictEqual(rec.index, rec.choice === "Valley" ? 0 : 1);
assert.strictEqual(S.recommendOption(adv, {}, ["Something we have never seen"]).index, null);
});
test("solverHistory turns form variables into a history keyed by the game's steps", () => {
S.useGame(published.samples.all);
const h = S.solverHistory({ company: "Alone", direction: "Center", first_sighting: "Red", scream: "Ignore the scream", resolution: "Tourniquet", hour: "Tree", branch: "Descend slowly: two Greens" });
assert.deepStrictEqual(h, { company: "Alone", direction: "Center", fs: "Red", scream: "Keep walking", wounded: "Tourniquet", hour: "Tree", descend: "Slowly (two Greens)", shadow: "no" });
const l = S.solverHistory({ company: "1 Squadmate", left_squadmates: true, branch: "Descend quickly: let the Red pass", hour: "Tree" });
assert.strictEqual(l.leave, "Leave them");
assert.strictEqual(l.red, "Let it pass (two Greens)");
});
-131
View File
@@ -1,131 +0,0 @@
// node --test tests/core/solver.test.js
const test = require("node:test");
const assert = require("node:assert");
const S = require("../../core/solver");
// A synthetic world: the meadow shows Blue 30% of the time, the shadow appears 2% of the time,
// facing the valley Red always bonds it, the Greens burn half the time and bond Brown (rescue) a quarter of the time.
function world(n = 4000, seed = 7) {
let s = seed;
const r = () => ((s = (s * 1103515245 + 12345) % 2147483648) / 2147483648);
const runs = [];
for (let i = 0; i < n; i++) {
const run = { company: "Alone", direction: ["East", "West", "Center"][i % 3], first_sighting: ["Red", "Brown"][i % 2],
scream: "Investigate & help", resolution: ["Killed the bullies yourself", "Squadmate's revenge"][i % 2], post_path: "Forest",
goal: "Any (for fun)", malek_intent: "No", enc_Red: null, enc_Green: null, enc_Orange: null, enc_Brown: null, enc_Blue: null, enc_Black: null };
run.hour = r() < 0.5 ? "Tree" : "Valley";
if (run.hour === "Tree") {
if (r() < 0.02) { run.branch = "Shadow (rare)"; run.enc_Black = "Bonded!"; run.outcome = "Bonded"; run.outcome_color = "Black"; }
else if (r() < 0.5) { run.branch = "Descend quickly: Red"; run.enc_Red = "Incinerated"; run.outcome = "Incinerated"; run.outcome_color = "Red"; }
else {
run.branch = "Descend slowly: two Greens";
const u = r();
if (u < 0.5) { run.enc_Green = "Incinerated"; run.outcome = "Incinerated"; run.outcome_color = "Green"; }
else if (u < 0.75) { run.enc_Green = "Bonded!"; run.outcome = "Bonded"; run.outcome_color = "Green"; }
else { run.enc_Green = "Saved by another"; run.enc_Brown = "Bonded!"; run.outcome = "Bonded"; run.outcome_color = "Brown"; }
}
} else if (r() < 0.2) { run.branch = "Valley Red"; run.enc_Red = "Bonded!"; run.outcome = "Bonded"; run.outcome_color = "Red"; }
else {
run.branch = "Passed the Red: meadow";
const u = r(), c = u < 0.3 ? "Blue" : u < 0.65 ? "Orange" : "Brown";
const bond = r() < 0.5;
run["enc_" + c] = bond ? "Bonded!" : "Incinerated"; run.outcome = bond ? "Bonded" : "Incinerated"; run.outcome_color = c;
}
runs.push(run);
}
return runs;
}
const policy = S.buildPolicy(world(), { draws: 10, colours: ["Blue", "Black", "Red", "Brown"] });
const choice = (X, h, obj = "thisAttempt") => S.advisor(policy, X, obj).next(h).choice;
test("Blue: go to the valley, pass the Red, attempt only the Blue", () => {
const t = policy.targets.Blue.thisAttempt;
const h = { company: "Alone", direction: "Center", fs: "Red", cadet: "Killed the bullies yourself", post: "Forest" };
assert.strictEqual(choice("Blue", h), "Valley");
assert.strictEqual(choice("Blue", { ...h, hour: "Valley" }), "Pass the Red (meadow)");
const meadow = { ...h, hour: "Valley", valley: "Pass the Red (meadow)" };
assert.strictEqual(choice("Blue", { ...meadow, meadow: "Blue" }), "Attempt the bond");
assert.strictEqual(choice("Blue", { ...meadow, meadow: "Orange" }), "Die on purpose");
assert.strictEqual(S.locate(t.tree, {}).var, "company");
// 30% Blue in the meadow x 50% bond
assert.ok(Math.abs(t.pX.point - 0.15) < 0.03, t.pX.point);
});
test("Black: climb the tree; the value is about the shadow rate x bond rate", () => {
const t = policy.targets.Black.thisAttempt;
const h = { company: "Alone", direction: "East", fs: "Red", cadet: "Squadmate's revenge", post: "Forest" };
assert.strictEqual(choice("Black", h), "Tree");
assert.ok(t.pX.point > 0.01 && t.pX.point < 0.035, t.pX.point);
});
test("Red: face the valley Red", () => {
const h = { company: "Alone", direction: "West", fs: "Brown", cadet: "Killed the bullies yourself", post: "Forest", hour: "Valley" };
assert.strictEqual(choice("Red", h), "Face the Red");
});
test("Brown 'eventually' never risks a non-Brown bond that 'this attempt' accepts", () => {
const a = policy.targets.Brown.thisAttempt, e = policy.targets.Brown.eventually;
assert.ok(e.pOther.point <= a.pOther.point + 1e-12);
assert.ok(e.ratio.mean >= a.ratio.mean - 1e-9);
});
test("no history dependence in the data -> large fitted concentration", () => {
assert.ok(policy.model.alphas.meadow >= 500, JSON.stringify(policy.model.alphas));
});
test("recommendOption maps the policy onto the bot's screen text", () => {
const adv = S.advisor(policy, "Blue", "thisAttempt");
const rec = S.recommendOption(adv, { company: "Alone", direction: "Center", fs: "Red", cadet: "Killed the bullies yourself", post: "Forest" },
["Continue toward the center of the valley", "Find the tallest tree and climb for a better view"]);
assert.strictEqual(rec.index, 0);
const none = S.recommendOption(adv, {}, ["Something we have never seen"]);
assert.strictEqual(none.index, null);
});
test("advisor matches the solver's own decisions on the recommended route", () => {
const adv = S.advisor(policy, "Blue", "eventually");
assert.strictEqual(adv.next({}).choice, policy.targets.Blue.eventually.tree.choice);
});
test("routing to one encounter: Blue only at the meadow equals the Blue policy; only at the tree Greens gives 0", () => {
const meadow = S.advisor(policy, "Blue", "thisAttempt", { only: ["meadow-Blue"] });
const all = S.advisor(policy, "Blue", "thisAttempt");
assert.ok(Math.abs(meadow.value.x - all.value.x) < 1e-6);
const greens = S.advisor(policy, "Blue", "thisAttempt", { only: ["tree-Greens"] });
assert.ok(greens.value.x < 1e-9);
// Brown via the Greens rescue is possible in this world
const rescue = S.advisor(policy, "Brown", "thisAttempt", { only: ["tree-Greens"] });
assert.ok(rescue.value.x > 0.05, rescue.value.x);
assert.ok(meadow.attempts("meadow-Blue") > 100);
});
test("the cadet encounter is asked in the game's order: scream, clearing or wounded cadet, the fight's moves, then the fate", () => {
const adv = S.advisor(policy, "Blue", "thisAttempt");
// the first dragon: you react to it before the scream
assert.strictEqual(adv.step({ company: "Alone", direction: "Center", fs: "Red" }).var, "react");
// entering with squadmates asks whether you stay with them
assert.strictEqual(adv.step({ company: "1 Squadmate" }).var, "leave");
assert.strictEqual(adv.step({ company: "1 Squadmate", left: "Stay together" }).var, "direction");
const base = { company: "Alone", direction: "Center", fs: "Red", react: "Hold still" };
assert.strictEqual(adv.step(base).var, "scream");
// the synthetic world has no one who kept walking, so that option is (correctly) not offered
assert.deepStrictEqual(Object.keys(adv.step(base).options), ["Investigate the scream"]);
assert.strictEqual(adv.step({ ...base, scream: "Investigate the scream" }).var, "clearing");
const help = { ...base, scream: "Investigate the scream", clearing: "Help your squadmate" };
assert.strictEqual(adv.step(help).var, "fight");
assert.strictEqual(adv.step({ ...help, fight: "Pause to assess" }).var, "dagger");
// charging straight in skips the dagger choice
assert.strictEqual(adv.step({ ...help, fight: "Charge the smaller cadet" }).var, "fate");
const fate = adv.step({ ...help, fight: "Pause to assess", dagger: "Engage the smaller cadet" });
assert.strictEqual(fate.var, "fate");
assert.ok(fate.risk && fate.risk.label === "Killed in the fight");
assert.strictEqual(adv.step({ ...base, scream: "Investigate the scream", clearing: "Walk away" }).var, "hour");
// a bot history that only knows the final resolution still lands in the right place
assert.strictEqual(adv.next({ ...base, cadet: "Killed the bullies yourself" }).decision, "post");
});
test("solverHistory converts form variables", () => {
const h = S.solverHistory({ company: "Alone", direction: "Center", first_sighting: "Red", resolution: "Tourniquet", hour: "Tree", branch: "Descend slowly: two Greens" });
assert.deepStrictEqual(h, { company: "Alone", direction: "Center", fs: "Red", cadet: "Tourniquet", wounded: "Tourniquet", post: "(no fork)", hour: "Tree", descend: "Slowly (two Greens)", shadow: "no" });
});
+55
View File
@@ -0,0 +1,55 @@
"""The route model (core/atlas/solver.py) on a synthetic world whose answers are known."""
import sys
import unittest
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "fixtures"))
import synthetic_policy # noqa: E402
P = synthetic_policy.policy()
T = P["targets"]
def first(tree, var):
"""The first decision on a variable along the recommended route (following the likeliest branch)."""
n = tree
while n and n["type"] != "end":
if n["type"] == "decision":
if n["var"] == var:
return n
n = n["next"]
else:
n = max(n["branches"].values(), key=lambda b: b["p"])["next"]
return None
class Synthetic(unittest.TestCase):
def test_blue_goes_to_the_valley_and_passes_the_red(self):
t = T["Blue"]["thisAttempt"]
self.assertEqual(first(t["tree"], "hour")["choice"], "Valley")
self.assertEqual(first(t["tree"], "valley")["choice"], "Pass the Red (meadow)")
# the chance is (share of meadow visits with a Blue) x (Blue bond rate), as logged in the world
m = [r for r in synthetic_policy.world() if r.get("branch") == "Passed the Red: meadow"]
blue = [r for r in m if r["enc_Blue"]]
expect = len(blue) / len(m) * sum(r["enc_Blue"] == "Bonded!" for r in blue) / len(blue)
self.assertLess(abs(t["pX"]["point"] - expect), 0.01)
def test_black_climbs_the_tree(self):
t = T["Black"]["thisAttempt"]
self.assertEqual(first(t["tree"], "hour")["choice"], "Tree")
self.assertTrue(0.01 < t["pX"]["point"] < 0.035, t["pX"]["point"])
def test_red_faces_the_valley_red(self):
self.assertEqual(first(T["Red"]["thisAttempt"]["tree"], "valley")["choice"], "Face the Red")
def test_eventually_never_risks_another_bond(self):
a, e = T["Brown"]["thisAttempt"], T["Brown"]["eventually"]
self.assertLessEqual(e["pOther"]["point"], a["pOther"]["point"] + 1e-12)
self.assertGreaterEqual(e["ratio"]["mean"], a["ratio"]["mean"] - 1e-9)
def test_no_history_dependence_gives_a_large_concentration(self):
self.assertGreaterEqual(P["model"]["alphas"]["meadow"], 500)
if __name__ == "__main__":
unittest.main()
+49
View File
@@ -0,0 +1,49 @@
"""The statistics behind every number on the page (analysis/stats/stats_core.py), against known answers."""
import sys
import unittest
from pathlib import Path
import numpy as np
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "analysis/stats"))
import stats_core as S # noqa: E402
class Stats(unittest.TestCase):
def test_wilson(self):
p, lo, hi = S.wilson(5, 10) # textbook: 0.2366 to 0.7634
self.assertAlmostEqual(lo, 0.2366, places=4)
self.assertAlmostEqual(hi, 0.7634, places=4)
self.assertEqual(S.wilson(0, 0), (None, None, None))
self.assertEqual(S.wilson(0, 20)[1], 0.0)
def test_odds_ratio_and_fisher(self):
or_, lo, hi, p, corr = S.odds_ratio(3, 1, 1, 3) # Fisher's lady tasting tea: two-sided p = 34/70
self.assertAlmostEqual(or_, 9.0)
self.assertLess(lo, or_)
self.assertGreater(hi, or_)
self.assertAlmostEqual(p, 34 / 70)
self.assertFalse(corr)
self.assertTrue(S.odds_ratio(3, 0, 1, 4)[4]) # a zero cell gets the +0.5 correction
def test_benjamini_hochberg(self):
q = S.bh([0.01, 0.04, 0.03, 0.5])
self.assertEqual([round(x, 4) for x in q], [0.04, 0.0533, 0.0533, 0.5])
def test_mantel_haenszel_equals_the_odds_ratio_when_strata_agree(self):
or_, lo, hi, p = S.mantel_haenszel([[10, 20, 5, 40], [20, 40, 10, 80]])
self.assertAlmostEqual(or_, 4.0)
self.assertLess(p, 0.001)
self.assertIsNone(S.mantel_haenszel([[0, 5, 0, 5]]))
def test_logistic_ridge_recovers_a_known_effect(self):
rng = np.random.default_rng(1)
x = rng.integers(0, 2, 20000)
y = (rng.random(20000) < 1 / (1 + np.exp(-(-1 + 1.2 * x)))).astype(float)
b, se = S.logistic_ridge(np.column_stack([np.ones_like(x), x]), y, lam=1.0)
self.assertAlmostEqual(b[1], 1.2, delta=0.1)
self.assertTrue(np.all(se > 0))
if __name__ == "__main__":
unittest.main()
+61
View File
@@ -0,0 +1,61 @@
"""A small synthetic world for testing the route model, and its policy.
The meadow shows Blue 30% of the time, the shadow appears 2% of the time, facing the valley Red always bonds it,
the Greens burn half the time and bond Brown (rescue) a quarter of the time.
python tests/fixtures/synthetic_policy.py OUT.json (used by tests/core/advisor.test.js)
"""
import json
import random
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "core"))
from atlas import solver # noqa: E402
def world(n=4000, seed=7):
r = random.Random(seed).random
runs = []
for i in range(n):
run = {"company": "Alone", "direction": ["East", "West", "Center"][i % 3], "first_sighting": ["Red", "Brown"][i % 2],
"scream": "Investigate & help", "resolution": ["Killed the bullies yourself", "Squadmate's revenge"][i % 2],
"post_path": "Forest", "goal": "Any (for fun)", "malek_intent": "No", "left_squadmates": False,
"sighting_action": "Unknown", "combat_first": "(not recorded)", "combat_second": "(not recorded)", "rare_reported": "No",
**{f"enc_{c}": None for c in ("Red", "Green", "Orange", "Brown", "Blue", "Black")}}
run["hour"] = "Tree" if r() < 0.5 else "Valley"
if run["hour"] == "Tree":
if r() < 0.02:
run.update(branch="Shadow (rare)", enc_Black="Bonded!", outcome="Bonded", outcome_color="Black")
elif r() < 0.5:
run.update(branch="Descend quickly: Red", enc_Red="Incinerated", outcome="Incinerated", outcome_color="Red")
else:
run["branch"] = "Descend slowly: two Greens"
u = r()
if u < 0.5:
run.update(enc_Green="Incinerated", outcome="Incinerated", outcome_color="Green")
elif u < 0.75:
run.update(enc_Green="Bonded!", outcome="Bonded", outcome_color="Green")
else:
run.update(enc_Green="Saved by another", enc_Brown="Bonded!", outcome="Bonded", outcome_color="Brown")
elif r() < 0.2:
run.update(branch="Valley Red", enc_Red="Bonded!", outcome="Bonded", outcome_color="Red")
else:
run["branch"] = "Passed the Red: meadow"
u = r()
c = "Blue" if u < 0.3 else "Orange" if u < 0.65 else "Brown"
bond = r() < 0.5
run.update({"enc_" + c: "Bonded!" if bond else "Incinerated", "outcome": "Bonded" if bond else "Incinerated", "outcome_color": c})
runs.append(run)
return runs
def policy(draws=10, colours=("Blue", "Black", "Red", "Brown")):
return solver.build_policy(world(), draws=draws, colours=list(colours))
if __name__ == "__main__":
sys.path.insert(0, str(ROOT / "planner"))
from build_policy import clean
Path(sys.argv[1]).write_text(json.dumps(clean(policy())))
-37
View File
@@ -1,37 +0,0 @@
"""Compare factor tables and models between the JS engine and the Python reference."""
import json, sys
js = json.load(open("tests/out/results_js.json"))
py = json.load(open("tests/out/results_py.json"))
TOL = 1e-6
fails = 0
def close(a, b, rel=TOL):
if a is None or b is None:
return a is None and b is None
return abs(a - b) <= rel * max(1, abs(a), abs(b))
for key, ft in py["factors"].items():
jrows = {(r["factor"], r["level"]): r for r in js["factors"][key]["rows"]}
for r in ft["rows"]:
j = jrows.get((r["factor"], r["level"]))
if j is None:
print("missing in JS:", key, r["factor"], r["level"]); fails += 1; continue
for f in ("k", "n", "p", "lo", "hi", "or", "or_lo", "or_hi", "fisher_p", "q"):
if (f in r) != (f in j) or (f in r and not close(r[f], j[f], 1e-5)):
print("diff", key, r["factor"], r["level"], f, r.get(f), j.get(f)); fails += 1
for key, m in py["models"].items():
jm = js["models"][key]
if m["n"] != jm["n"] or m["events"] != jm["events"]:
print("model size diff", key); fails += 1
jt = {t["term"]: t for t in jm["terms"]}
for t in m["terms"]:
j = jt.get(t["term"])
if not j:
print("term missing", key, t["term"]); fails += 1; continue
for f in ("or", "lo", "hi"):
if not close(t[f], j[f], 1e-4):
print("model diff", key, t["term"], f, t[f], j[f]); fails += 1
if abs(t["p"] - j["p"]) > 1e-6:
print("model p diff", key, t["term"], t["p"], j["p"]); fails += 1
n_rows = sum(len(f["rows"]) for f in py["factors"].values())
n_terms = sum(len(m["terms"]) for m in py["models"].values())
print(f"checked {n_rows} factor rows and {n_terms} model terms: {'ALL MATCH' if not fails else str(fails) + ' differences'}")
sys.exit(1 if fails else 0)
-60
View File
@@ -1,60 +0,0 @@
"""Recompute route rankings in Python (crude Fisher and stratified Mantel-Haenszel) and compare with engine.js."""
import json, subprocess, sys
sys.path.insert(0, "analysis/stats")
import pandas as pd
from stats_core import odds_ratio, mantel_haenszel, bh
d = pd.DataFrame(json.load(open("tests/out/runs_js.json")))
js = json.loads(subprocess.check_output(["node", "-e", """
const E=require('./core/engine.js'),fs=require('fs');const raw=JSON.parse(fs.readFileSync('data/public/form_responses.json'));
const {runs}=E.codeRows(raw.rows,raw.headers);const o={};for(const t of ['see_Blue','bond_Blue','see_shadow','bond_Green']){o[t]=E.rankRoutes(runs,t,10)}
console.log(JSON.stringify(o));"""]))
MISSING = {"Unknown", "Unsure", "Unsure/unknown", "Multiple", "(not recorded)", "(not asked)", "(not asked/blank)", "Unknown/lost count", "Blank/not asked"}
miss = lambda v: not isinstance(v, str) or v == "" or v in MISSING # noqa: E731
def stage(r):
# a route is only ranked from runs whose every step is known (mirrors rankRoutes in engine.js)
if r.scream.startswith("DIED") or miss(r.scream) or miss(r.resolution): cad = None # noqa
else: cad = ("Help → " + r.resolution) if r.scream == "Investigate & help" else (r.scream + " → " + r.resolution)
hour = {"Tree": "Climb the tree", "Valley": "Valley centre"}.get(r.hour)
fs = None if r.scream == "DIED at first sighting" or miss(r.first_sighting) else r.first_sighting
post_null = not isinstance(r.post_path, str)
post = None if post_null or miss(r.post_path) else r.post_path
vals = [None if miss(r.company) else r.company, None if miss(r.direction) else r.direction, fs, cad, post, hour]
if any(v is None for i, v in enumerate(vals) if not (i == 4 and post_null)): return None
return " › ".join(v if v else "—" for v in vals)
def stratum(r):
if r.source != "Form": return "bulk"
return {"Blue": "blue", "Black": "black", "Blue or Black": "blue-or-black"}.get(r.goal, "other")
d["route"] = d.apply(stage, axis=1); d["stratum"] = d.apply(stratum, axis=1)
d["branch"] = d.branch.where(d.branch.notna(), None)
T = {"see_Blue": (lambda x: x.branch == "Passed the Red: meadow", lambda x: x.saw_blue_meadow),
"bond_Blue": (lambda x: x.hour == "Valley", lambda x: (x.outcome == "Bonded") & (x.outcome_color == "Blue")),
"see_shadow": (lambda x: (x.hour == "Tree") & x.branch.notna(), lambda x: x.saw_shadow),
"bond_Green": (lambda x: x.hour == "Tree", lambda x: (x.outcome == "Bonded") & (x.outcome_color == "Green"))}
fails = 0
for t, (pool, ev) in T.items():
P = d[pool(d)].copy(); P["e"] = ev(P).astype(bool)
K, N = int(P.e.sum()), len(P)
rows = {}
for route, g in P[P.route.notna()].groupby("route"):
if len(g) < 10: continue
k, n = int(g.e.sum()), len(g)
orr = odds_ratio(k, n - k, K - k, N - n - (K - k))
tabs = []
for s, sg in P.groupby("stratum"):
inr = sg.route == route
a = int((inr & sg.e).sum()); nin = int(inr.sum()); c = int((~inr & sg.e).sum()); nout = len(sg) - nin
tabs.append([a, nin - a, c, nout - c])
rows[route] = (k, n, orr, mantel_haenszel(tabs))
jr = {r["route"]: r for r in js[t]["rows"]}
if set(jr) != set(rows): print(t, "route sets differ", set(jr) ^ set(rows)); fails += 1
for route, (k, n, orr, mh) in rows.items():
j = jr.get(route)
if not j: continue
if (k, n) != (j["k"], j["n"]) or abs(orr[0] - j["or"]) > 1e-6 * max(1, orr[0]) or abs(orr[3] - j["fisher_p"]) > 1e-6:
print("crude diff", t, route); fails += 1
if mh is None:
if j["mh_or"] is not None: print("mh none diff", t, route); fails += 1
elif j["mh_or"] is None or abs(mh[0] - j["mh_or"]) > 1e-6 * max(1, mh[0]) or abs(mh[3] - j["mh_p"]) > 1e-5:
print("mh diff", t, route, mh, j["mh_or"], j["mh_p"]); fails += 1
print(f"route rankings for {len(T)} outcomes: {'ALL MATCH' if not fails else str(fails) + ' differences'}")
sys.exit(1 if fails else 0)
-21
View File
@@ -1,21 +0,0 @@
"""Check that the JS engine (core/engine.js) and the Python coding rules (core/atlas/coding.py) code every run identically."""
import json, sys
import pandas as pd
js = pd.DataFrame(json.load(open(sys.argv[1] if len(sys.argv) > 1 else "tests/out/runs_js.json")))
py = pd.read_csv(sys.argv[2] if len(sys.argv) > 2 else "data/derived/runs_clean.csv", keep_default_na=False, na_values=[""])
def norm(v):
if v is None or (not isinstance(v, str) and pd.isna(v)) or v == "":
return None
if isinstance(v, float) and v.is_integer():
return str(int(v))
if isinstance(v, bool):
return str(v)
return str(v)
bad = {}
for c in py.columns:
a, b = [norm(v) for v in py[c]], [norm(v) for v in js[c]]
d = [(i, x, y) for i, (x, y) in enumerate(zip(a, b)) if x != y]
if d:
bad[c] = (len(d), d[:3])
print("runs:", len(py), len(js), "| mismatching columns:", bad or "NONE")
sys.exit(1 if bad else 0)
-37
View File
@@ -1,37 +0,0 @@
"""The Python route model (core/atlas/solver.py, built from core/spec/game.yaml) must give the same policy as the
JavaScript one the site and the bot still use (core/solver.js): same choices everywhere, same numbers.
A quick version for CI: one colour, both samples' coding, 3 posterior draws.
python tests/parity/compare_solver.py
"""
import json
import subprocess
import sys
import tempfile
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "core"))
sys.path.insert(0, str(ROOT / "planner"))
sys.path.insert(0, str(Path(__file__).parent))
import build_policy # noqa: E402
from atlas import solver # noqa: E402
from compare_policy import compare # noqa: E402
COLOURS, DRAWS = ["Blue"], 3
runs = build_policy.coded_runs(ROOT / "data/public/form_responses.json")
py = build_policy.clean(solver.build_policy(runs, draws=DRAWS, colours=COLOURS))
js = json.loads(subprocess.check_output(["node", "-e", f"""
const S=require('./core/solver.js'),E=require('./core/engine.js'),fs=require('fs');
const raw=JSON.parse(fs.readFileSync('data/public/form_responses.json'));const {{runs}}=E.codeRows(raw.rows,raw.headers);
console.log(JSON.stringify(S.buildPolicy(runs,{{draws:{DRAWS},colours:{json.dumps(COLOURS)}}})));"""], cwd=ROOT))
with tempfile.TemporaryDirectory() as d:
a, b = Path(d, "js.json"), Path(d, "py.json")
a.write_text(json.dumps(js))
b.write_text(json.dumps(py))
o = compare(a, b, 1e-9)
bad = o["keys"] + o["other"] + o["numbers"]
print(f"route model ({', '.join(COLOURS)}, {DRAWS} draws): " + ("ALL MATCH" if not bad else f"{len(bad)} DIFFER"), f"(largest relative difference {o['max']:.2g})")
for x in bad[:10]:
print(" ", x)
sys.exit(1 if bad else 0)
-122
View File
@@ -1,122 +0,0 @@
"""The precomputed statistics (analysis/stats/site_stats.py -> stats.json) must match what core/engine.js computed
in the browser before, for every preset: factor tables, models, encounters, chance events, route rankings at every
threshold, and the route map.
python tests/parity/compare_stats.py [stats.json]
"""
import json
import subprocess
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
stats = json.load(open(sys.argv[1] if len(sys.argv) > 1 else ROOT / "data/public/stats.json"))
presets = {k: v["analyse"]["filters"] for k, v in stats["presets"].items()}
targets = list(stats["meta"]["targets"])
th = stats["meta"]["route_thresholds"]
js = json.loads(subprocess.run(["node", "-e", f"""
const E=require('./core/engine.js'),fs=require('fs');const raw=JSON.parse(fs.readFileSync('data/public/form_responses.json'));
const {{runs}}=E.codeRows(raw.rows,raw.headers);const out={{}};
for (const [k,f] of Object.entries({json.dumps(presets)})) {{
const sub=E.applyFilters(runs,f); const o={{analyse:E.analyse(runs,f), routes:{{}}, map:{{}}}};
for (const t of {json.dumps(targets)}) {{ o.routes[t]={{}}; for (const m of {json.dumps(th)}) o.routes[t][m]=E.rankRoutes(sub,t,m); o.map[t]=E.buildGraph(sub,t,{{}}); }}
out[k]=o; }}
process.stdout.write(JSON.stringify(out));"""], cwd=ROOT, capture_output=True, text=True, check=True).stdout)
bad = []
def close(a, b, tol):
if a is None or b is None:
return a is None and b is None
if isinstance(a, bool) or isinstance(b, bool) or isinstance(a, str) or isinstance(b, str):
return a == b
return abs(a - b) <= tol * max(1, abs(a), abs(b))
def same(path, a, b, fields, tol=1e-9):
for f in fields:
if not close(a.get(f), b.get(f), tol): # a field left out counts as empty
bad.append(f"{path}.{f}: python {a.get(f)!r} vs js {b.get(f)!r}")
W = ("k", "n", "p", "lo", "hi")
for pid, P in stats["presets"].items():
py, J = P["analyse"], js[pid]["analyse"]
if py["n"] != J["n"]:
bad.append(f"{pid}: n {py['n']} vs {J['n']}")
same(f"{pid}.meta", py["meta"], J["meta"], ("date_min", "date_max", "n_flagged"))
for k in ("sources", "flag_counts", "outcomes"):
if py["meta"][k] != J["meta"][k]:
bad.append(f"{pid}.meta.{k} differ")
for key, c in py["chance"].items():
same(f"{pid}.chance.{key}", c["overall"], J["chance"][key]["overall"], W)
for grp in ("by_stratum", "by_goal", "by_attempt"):
if set(c[grp]) != set(J["chance"][key][grp]):
bad.append(f"{pid}.chance.{key}.{grp} keys differ")
for s in c[grp]:
same(f"{pid}.chance.{key}.{grp}.{s}", c[grp][s], J["chance"][key][grp].get(s, {}), W)
if py["first_sighting"] != J["first_sighting"]:
bad.append(f"{pid}.first_sighting differ")
for a, b in zip(py["encounters"], J["encounters"]):
same(f"{pid}.enc.{a['context']}.overall", a["overall"], b["overall"], W)
same(f"{pid}.enc.{a['context']}.intent", a["intent_matched"], b["intent_matched"], W)
for k in ("green_encounters", "bonded_green", "rescued_orange", "rescued_brown", "incinerated"):
if py["green_rescue"][k] != J["green_rescue"][k]:
bad.append(f"{pid}.green_rescue.{k}")
for key, ft in py["factors"].items():
jr = {(r["factor"], r["level"]): r for r in J["factors"][key]["rows"]}
if len(jr) != len(ft["rows"]):
bad.append(f"{pid}.factors.{key}: {len(ft['rows'])} rows vs {len(jr)}")
for r in ft["rows"]:
j = jr.get((r["factor"], r["level"]))
if j is None:
bad.append(f"{pid}.factors.{key}: {r['factor']}={r['level']} missing in JS")
continue
same(f"{pid}.factors.{key}.{r['factor']}={r['level']}", r, j, W + ("or", "or_lo", "or_hi"))
same(f"{pid}.factors.{key}.{r['factor']}={r['level']}", r, j, ("fisher_p", "q"), 1e-5)
for key, m in py["models"].items():
jm = J["models"][key]
if (m is None) != (jm is None):
bad.append(f"{pid}.models.{key} missing on one side")
continue
if m is None:
continue
same(f"{pid}.models.{key}", m, jm, ("n", "events"))
jt = {t["term"]: t for t in jm["terms"]}
for t in m["terms"]:
same(f"{pid}.models.{key}.{t['term']}", t, jt.get(t["term"], {}), ("or", "lo", "hi"), 1e-4)
same(f"{pid}.models.{key}.{t['term']}", t, jt.get(t["term"], {}), ("p",), 1e-6)
for a, b in zip(py["daily"], J["daily"]):
same(f"{pid}.daily.{a['date']}", a, b, ("runs", "meadow_n", "blue_k", "tree_n", "shadow_k"))
for t, R in P["routes"].items():
for m in th:
jr = js[pid]["routes"][t][str(m)]
bt = R["by_threshold"][str(m)]
if bt["n_routes_tested"] != jr["n_routes_tested"]:
bad.append(f"{pid}.routes.{t}.{m}: {bt['n_routes_tested']} routes vs {jr['n_routes_tested']}")
jrows = {r["route"]: r for r in jr["rows"]}
for i, r in enumerate(R["rows"]):
if r["n"] < m:
continue
j = jrows.get(r["route"])
if j is None:
bad.append(f"{pid}.routes.{t}.{m}: {r['route']} missing in JS")
continue
same(f"{pid}.routes.{t}.{m}.{r['route']}", r, j, W + ("or", "or_lo", "or_hi", "mh_or", "mh_lo", "mh_hi"))
same(f"{pid}.routes.{t}.{m}.{r['route']}", r, j, ("fisher_p", "mh_p"), 1e-5)
same(f"{pid}.routes.{t}.{m}.{r['route']}", {"q": bt["q"].get(str(i)), "mh_q": bt["mh_q"].get(str(i))}, j, ("q", "mh_q"), 1e-5)
for t in targets:
jn = {n["id"]: n for n in js[pid]["map"][t]["nodes"]}
jl = {l["id"]: l for l in js[pid]["map"][t]["links"]}
for n, w in zip(P["map"]["nodes"], P["map"]["by_target"][t]["nodes"]):
same(f"{pid}.map.{t}.{n['id']}", {**n, **w}, jn.get(n["id"], {}), W)
for l, w in zip(P["map"]["links"], P["map"]["by_target"][t]["links"]):
same(f"{pid}.map.{t}.{l['id']}", {**l, **w}, jl.get(l["id"], {}), W + ("chance",))
if len(jn) != len(P["map"]["nodes"]) or len(jl) != len(P["map"]["links"]):
bad.append(f"{pid}.map.{t}: size differs")
print(f"precomputed statistics, {len(stats['presets'])} presets: " + ("ALL MATCH" if not bad else f"{len(bad)} DIFFER"))
for b in bad[:25]:
print(" ", b)
sys.exit(1 if bad else 0)
-13
View File
@@ -1,13 +0,0 @@
// node tests/parity/run_engine.js data/public/form_responses.json out_dir
// Runs the browser engine in Node and writes coded runs + results, for comparison with Python.
const fs = require("fs");
const path = require("path");
const E = require("../../core/engine.js");
const [, , src = "data/public/form_responses.json", outDir = "tests/out"] = process.argv;
const raw = JSON.parse(fs.readFileSync(src, "utf8"));
const { runs, missing } = E.codeRows(raw.rows, raw.headers);
fs.mkdirSync(outDir, { recursive: true });
fs.writeFileSync(path.join(outDir, "runs_js.json"), JSON.stringify(runs));
const res = E.analyse(runs, {});
fs.writeFileSync(path.join(outDir, "results_js.json"), JSON.stringify(res, null, 1));
console.log(`${runs.length} runs coded; missing columns: ${missing.join(", ") || "none"}`);