The Black on the page: a first-attempt range marked low evidence; the Black panels load again
CI / test (pull_request) Successful in 40s

- The odds card for the Black: "Low evidence", "about 1 in N" first attempts, the 95% range written as
  "between 1 in X and 1 in Y", later attempts effectively never, why it is uncertain (every logged shadow is a
  rider who chose to post), and the steps (shadow -> reached the Black -> bonded). Dashed border.
- Valley map: the Black's routes are solved for a first attempt (they showed ~0% per attempt), labelled "on a first
  attempt", and "Why this route?" gives the same range and caveat.
- "The Black is not dice" panel: why the odds are uncertain, and what replaces the assumption (fresh-account bot
  runs to the tree).
- analysis/black/evidence.py wrote "odds_ratio": Infinity (no later-attempt shadow at all), which is not JSON:
  the page's two Black panels showed "could not be loaded". Now null, and the report says "unbounded".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
2026-11-19 18:40:01 +00:00
co-authored by Claude Opus 5.5
parent d7befc8e18
commit b51a6c7039
6 changed files with 25 additions and 11 deletions
+4 -3
View File
@@ -141,7 +141,8 @@ def build():
c[i + 1] += 1 c[i + 1] += 1
orr, obs, exp, z = mantel_haenszel(by.values()) orr, obs, exp, z = mantel_haenszel(by.values())
facts["same_route"] = {"routes": len(by), "first_shadows": round(obs), "expected": round(exp, 1), facts["same_route"] = {"routes": len(by), "first_shadows": round(obs), "expected": round(exp, 1),
"odds_ratio": round(orr, 1), "z": round(z, 1)} "odds_ratio": round(orr, 1) if math.isfinite(orr) else None, # JSON has no Infinity
"z": round(z, 1)}
facts["form_first"] = [sum(c[0] for c in by.values()), sum(c[1] for c in by.values())] facts["form_first"] = [sum(c[0] for c in by.values()), sum(c[1] for c in by.values())]
facts["form_later"] = [sum(c[2] for c in by.values()), sum(c[3] for c in by.values())] facts["form_later"] = [sum(c[2] for c in by.values()), sum(c[3] for c in by.values())]
L += ["## 2. The same route on a first attempt and on a later one", "", L += ["## 2. The same route on a first attempt and on a later one", "",
@@ -153,7 +154,7 @@ def build():
L.append(f"| {' / '.join(key)} | {frac(y1, n1)} | {frac(y0, n0)} |") L.append(f"| {' / '.join(key)} | {frac(y1, n1)} | {frac(y0, n0)} |")
L += ["", f"Across all {len(by)} recorded routes, comparing first and later attempts on the same route: " L += ["", f"Across all {len(by)} recorded routes, comparing first and later attempts on the same route: "
f"{obs:.0f} first-attempt shadows where {exp:.1f} would be expected if the attempt did not matter " f"{obs:.0f} first-attempt shadows where {exp:.1f} would be expected if the attempt did not matter "
f"(odds ratio {orr:.1f}, z = {z:.1f}). First and later attempts choose almost the same routes, so the " f"(odds ratio {f'{orr:.1f}' if math.isfinite(orr) else 'unbounded: no later-attempt shadow at all'}, z = {z:.1f}). First and later attempts choose almost the same routes, so the "
"difference is not in the recorded choices: it is something about the account, or a choice the form does " "difference is not in the recorded choices: it is something about the account, or a choice the form does "
"not record.", ""] "not record.", ""]
@@ -266,7 +267,7 @@ def main(argv=None):
ap.add_argument("--check", action="store_true") ap.add_argument("--check", action="store_true")
a = ap.parse_args(argv) a = ap.parse_args(argv)
text, facts = build() text, facts = build()
data = json.dumps(facts, ensure_ascii=False, indent=1) + "\n" data = json.dumps(facts, ensure_ascii=False, indent=1, allow_nan=False) + "\n"
if a.check: if a.check:
for path, want in ((OUT, text), (OUT_JSON, data)): for path, want in ((OUT, text), (OUT_JSON, data)):
if not path.exists() or path.read_text(encoding="utf-8") != want: if not path.exists() or path.read_text(encoding="utf-8") != want:
+1 -1
View File
@@ -31,7 +31,7 @@ If the shadow depended only on the choices the form records (company, first dire
| Alone / Center / Assess the situation / Daggers or not (protect squadmate) / Left bullies to the dragons / Forest | 1 / 28 (3.6%) | 0 / 64 (0.0%) | | Alone / Center / Assess the situation / Daggers or not (protect squadmate) / Left bullies to the dragons / Forest | 1 / 28 (3.6%) | 0 / 64 (0.0%) |
| Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 2 / 9 (22.2%) | 0 / 81 (0.0%) | | Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 2 / 9 (22.2%) | 0 / 81 (0.0%) |
Across all 326 recorded routes, comparing first and later attempts on the same route: 93 first-attempt shadows where 38.8 would be expected if the attempt did not matter (odds ratio inf, z = 12.5). First and later attempts choose almost the same routes, so the difference is not in the recorded choices: it is something about the account, or a choice the form does not record. Across all 326 recorded routes, comparing first and later attempts on the same route: 93 first-attempt shadows where 38.8 would be expected if the attempt did not matter (odds ratio unbounded: no later-attempt shadow at all, z = 12.5). First and later attempts choose almost the same routes, so the difference is not in the recorded choices: it is something about the account, or a choice the form does not record.
## 3. On one route, is the shadow decided by the cadet's wing, section and squad? ## 3. On one route, is the shadow decided by the cadet's wing, section and squad?
+1 -1
View File
@@ -84,7 +84,7 @@
"routes": 326, "routes": 326,
"first_shadows": 93, "first_shadows": 93,
"expected": 38.8, "expected": 38.8,
"odds_ratio": Infinity, "odds_ratio": null,
"z": 12.5 "z": 12.5
}, },
"form_first": [ "form_first": [
+7 -6
View File
@@ -92,7 +92,8 @@
const model = () => st.policy.samples[st.sample]; const model = () => st.policy.samples[st.sample];
function advisorFor(enc, colour = st.colour) { function advisorFor(enc, colour = st.colour) {
const k = `${st.sample}|${colour}|${st.objective}|${enc}`; const k = `${st.sample}|${colour}|${st.objective}|${enc}`;
if (!st.cache.has(k)) st.cache.set(k, S.advisor(model(), colour, st.objective, { only: [enc] })); // the Black needs the shadow, which (almost) only shows on a first attempt: its routes are solved for that attempt
if (!st.cache.has(k)) st.cache.set(k, S.advisor(model(), colour, st.objective, { only: [enc], attempt: colour === "Black" ? "First attempt" : null }));
return st.cache.get(k); return st.cache.get(k);
} }
let encParams = null; let encParams = null;
@@ -530,7 +531,7 @@
}); });
// destination label // destination label
const end = pts[pts.length - 1]; const end = pts[pts.length - 1];
pill(gLab, end.P.x, end.P.y + 30, `${pct(r.value.x)} per attempt`, { edge: tc, strong: rel === 1 }); pill(gLab, end.P.x, end.P.y + 30, `${pct(r.value.x)} ${st.colour === "Black" ? "on a first attempt" : "per attempt"}`, { edge: tc, strong: rel === 1 });
// choice points on the focused route: one diamond per place, even when several choices are made there // choice points on the focused route: one diamond per place, even when several choices are made there
// (at the bully fight: help or walk away, your first move, the dagger, then the bullies' fate) // (at the bully fight: help or walk away, your first move, the dagger, then the bullies' fate)
if (focus) { if (focus) {
@@ -841,7 +842,7 @@
function panelRoutes(routes) { function panelRoutes(routes) {
const best = Math.max(1e-12, ...routes.map((r) => r.value.x)); const best = Math.max(1e-12, ...routes.map((r) => r.value.x));
const T = model().targets[st.colour], b = T[st.objective]; const T = model().targets[st.colour], first = st.colour === "Black" && T.firstAttempt, b = first ? T.firstAttempt.thisAttempt : T[st.objective];
const focus = routes.find((r) => r.enc === st.focus) || routes[0]; const focus = routes.find((r) => r.enc === st.focus) || routes[0];
const card = (r) => { const card = (r) => {
const rel = r.value.x / best, o = encOutcome(r.enc), c = encColour(r.enc); const rel = r.value.x / best, o = encOutcome(r.enc), c = encColour(r.enc);
@@ -851,7 +852,7 @@
return `<button type="button" class="ex-route-card${r.enc === focus.enc ? " on" : ""}" data-enc="${r.enc}" aria-pressed="${r.enc === focus.enc}"> return `<button type="button" class="ex-route-card${r.enc === focus.enc ? " on" : ""}" data-enc="${r.enc}" aria-pressed="${r.enc === focus.enc}">
<i class="ex-swatch ${style}" style="--c: var(${DV[st.colour]}); opacity:${(0.3 + 0.7 * rel).toFixed(2)}"></i> <i class="ex-swatch ${style}" style="--c: var(${DV[st.colour]}); opacity:${(0.3 + 0.7 * rel).toFixed(2)}"></i>
<span class="nm">${esc(ENC_NAME[r.enc])}${RESCUE[st.colour] === r.enc && st.colour !== "Green" ? " <em>(rescue)</em>" : ""}</span> <span class="nm">${esc(ENC_NAME[r.enc])}${RESCUE[st.colour] === r.enc && st.colour !== "Green" ? " <em>(rescue)</em>" : ""}</span>
<b>${pct(r.value.x)}</b><small>per attempt · ${bond}${n < 30 ? ` · only ${n} attempts logged` : ""}${r.value.o > 0.005 ? ` · ends on another dragon ${pct(r.value.o)}` : ""}</small></button>`; <b>${pct(r.value.x)}</b><small>${st.colour === "Black" ? "on a first attempt" : "per attempt"} · ${bond}${n < 30 ? ` · only ${n} attempts logged` : ""}${r.value.o > 0.005 ? ` · ends on another dragon ${pct(r.value.o)}` : ""}</small></button>`;
}; };
const steps = focus.steps.filter((s) => s.kind === "decision" || s.kind === "chance").map((s) => { const steps = focus.steps.filter((s) => s.kind === "decision" || s.kind === "chance").map((s) => {
if (s.kind === "chance") return `<li class="chance"><span>${esc(s.label)}</span><b>${esc(String(s.taken))}</b><small>roll ${pct(s.probs[s.taken])}</small></li>`; if (s.kind === "chance") return `<li class="chance"><span>${esc(s.label)}</span><b>${esc(String(s.taken))}</b><small>roll ${pct(s.probs[s.taken])}</small></li>`;
@@ -862,7 +863,7 @@
return `<li class="dec${open ? " open" : ""}"><button type="button" class="ex-step" data-var="${s.var}" aria-expanded="${open}"><span>${esc(s.label)}</span><b>${esc(String(s.taken))}</b><small>${s.userSet ? "your choice" : "best"} · ${s.var === "encounter" ? pct(s.options[s.taken].pX) : pct(s.options[s.taken].pX)}</small></button>${opts}${inside}</li>`; return `<li class="dec${open ? " open" : ""}"><button type="button" class="ex-step" data-var="${s.var}" aria-expanded="${open}"><span>${esc(s.label)}</span><b>${esc(String(s.taken))}</b><small>${s.userSet ? "your choice" : "best"} · ${s.var === "encounter" ? pct(s.options[s.taken].pX) : pct(s.options[s.taken].pX)}</small></button>${opts}${inside}</li>`;
}).join(""); }).join("");
const risks = focus.steps.filter((s) => s.risk && s.risk.p > 0.0005).map((s) => `${s.risk.label.toLowerCase()} ${pct(s.risk.p)}`); const risks = focus.steps.filter((s) => s.risk && s.risk.p > 0.0005).map((s) => `${s.risk.label.toLowerCase()} ${pct(s.risk.p)}`);
const lb = st.policy.samples.lowBias.targets[st.colour].thisAttempt; const lbT = st.policy.samples.lowBias.targets[st.colour], lb = first && lbT.firstAttempt ? lbT.firstAttempt.thisAttempt : lbT.thisAttempt;
return `<div class="ex-head" style="--c: var(${DV[st.colour]})">${mini(st.colour, POSE_OF[st.colour])}<div><span class="eyebrow">Routes to a</span><h2 id="h-valley">${st.colour} dragon</h2></div> return `<div class="ex-head" style="--c: var(${DV[st.colour]})">${mini(st.colour, POSE_OF[st.colour])}<div><span class="eyebrow">Routes to a</span><h2 id="h-valley">${st.colour} dragon</h2></div>
<button type="button" class="ex-close" id="ex-clear" aria-label="Back to the whole valley">×</button></div> <button type="button" class="ex-close" id="ex-clear" aria-label="Back to the whole valley">×</button></div>
${chooser()} ${chooser()}
@@ -874,7 +875,7 @@
<ol class="ex-steps">${steps}</ol> <ol class="ex-steps">${steps}</ol>
<p class="small muted">Diamonds on the map are choices: click one to see where the other options lead. ${st.blocked ? "" : `Your choices apply to every route to this dragon. Changed your mind? <button type="button" class="linklike" id="ex-reset-route">Reset your choices</button>`}</p> <p class="small muted">Diamonds on the map are choices: click one to see where the other options lead. ${st.blocked ? "" : `Your choices apply to every route to this dragon. Changed your mind? <button type="button" class="linklike" id="ex-reset-route">Reset your choices</button>`}</p>
<details class="ex-why"><summary>Why this route?</summary> <details class="ex-why"><summary>Why this route?</summary>
<p class="small">Best route for any ${st.colour}: <b>${pct(b.pX.mean)}</b> per attempt (95% interval ${pct(b.pX.lo)}–${pct(b.pX.hi)}), about ${isFinite(b.expectedAttempts.mean) ? Math.round(b.expectedAttempts.mean) : "–"} attempts. Riders who logged every run suggest ${pct(lb.pX.mean)}.</p> ${first ? `<p class="small"><span class="pill warn">Low evidence</span> Best route for the Black, on a <b>first attempt</b>: about <b>1 in ${Math.round(1 / Math.max(b.pX.mean, 1e-9))}</b>, somewhere between 1 in ${Math.round(1 / Math.max(b.pX.hi, 1e-9))} and 1 in ${Math.round(1 / Math.max(b.pX.lo, 1e-9))} (95%). Later attempts almost never see the shadow. The low-bias sample suggests about 1 in ${Math.round(1 / Math.max(lb.pX.mean, 1e-9))}. Every logged shadow comes from riders who chose to post their run, so the shadow's odds rest on an assumption until complete first-attempt logs exist.</p>` : `<p class="small">Best route for any ${st.colour}: <b>${pct(b.pX.mean)}</b> per attempt (95% interval ${pct(b.pX.lo)}–${pct(b.pX.hi)}), about ${isFinite(b.expectedAttempts.mean) ? Math.round(b.expectedAttempts.mean) : "–"} attempts. Riders who logged every run suggest ${pct(lb.pX.mean)}.</p>`}
<p class="small">Each roll's odds may depend on your earlier choices, as far as the data support it. Choices marked "no clear winner" in <a href="#deep">Deep Dive</a> are within the noise.${risks.length ? ` Risks on this route: ${risks.join(", ")}.` : ""}</p> <p class="small">Each roll's odds may depend on your earlier choices, as far as the data support it. Choices marked "no clear winner" in <a href="#deep">Deep Dive</a> are within the noise.${risks.length ? ` Risks on this route: ${risks.join(", ")}.` : ""}</p>
<p class="small">The bond chance at each encounter is the community average: the form does not record the answers riders gave there.</p> <p class="small">The bond chance at each encounter is the community average: the form does not record the answers riders gave there.</p>
<div class="seg" role="group" aria-label="Data"><button type="button" data-s="all" aria-pressed="${st.sample === "all"}">All runs</button><button type="button" data-s="lowBias" aria-pressed="${st.sample === "lowBias"}">Low-bias</button></div> <div class="seg" role="group" aria-label="Data"><button type="button" data-s="all" aria-pressed="${st.sample === "all"}">All runs</button><button type="button" data-s="lowBias" aria-pressed="${st.sample === "lowBias"}">Low-bias</button></div>
+2
View File
@@ -182,6 +182,8 @@ section h2 { font-family: var(--font-crest); letter-spacing: 0.03em; }
.odd .big { font-family: var(--font-crest); font-weight: 700; font-size: 2.1rem; line-height: 1; color: var(--c, var(--ink)); } .odd .big { font-family: var(--font-crest); font-weight: 700; font-size: 2.1rem; line-height: 1; color: var(--c, var(--ink)); }
.odd .what { font-weight: 600; } .odd .what { font-weight: 600; }
.odd .how { font-size: var(--step--1); color: var(--muted); } .odd .how { font-size: var(--step--1); color: var(--muted); }
.odd-low { border-style: dashed; }
.odd-low .big { opacity: 0.85; }
.odd svg { display: block; width: 100%; max-width: 220px; height: auto; } .odd svg { display: block; width: 100%; max-width: 220px; height: auto; }
.odd::before { content: ""; position: absolute; inset: 0 auto 0 0; width: 4px; background: var(--c, var(--line)); } .odd::before { content: ""; position: absolute; inset: 0 auto 0 0; width: 4px; background: var(--c, var(--line)); }
.combo { display: grid; grid-template-columns: repeat(auto-fit, minmax(min(100%, 320px), 1fr)); gap: 16px; } .combo { display: grid; grid-template-columns: repeat(auto-fit, minmax(min(100%, 320px), 1fr)); gap: 16px; }
+10
View File
@@ -159,6 +159,8 @@
if (!b) { if (t2) t2.innerHTML = '<h3>Your first attempt is different</h3><p class="small muted">The shadow evidence could not be loaded.</p>'; return; } if (!b) { if (t2) t2.innerHTML = '<h3>Your first attempt is different</h3><p class="small muted">The shadow evidence could not be loaded.</p>'; return; }
const ff = b.form_first, fl = b.form_later, sr = b.same_route, as = b.after_shadow, gaze = as.answers["held its gaze"]; const ff = b.form_first, fl = b.form_later, sr = b.same_route, as = b.after_shadow, gaze = as.answers["held its gaze"];
if (t2) t2.innerHTML = `<h3>Your first attempt is different</h3><p class="small">Among riders who helped their squadmate and climbed the tree, the shadow came in <b>${frac(ff)}</b> first attempts (${pct(ff[0] / ff[1], 1)}) and <b>${frac(fl)}</b> later ones. On the very same routes, first attempts saw it about ${Math.round(sr.first_shadows / Math.max(sr.expected, 1) * 10) / 10}× as often as chance would give. The shadow is not a dice roll you can repeat.</p>`; if (t2) t2.innerHTML = `<h3>Your first attempt is different</h3><p class="small">Among riders who helped their squadmate and climbed the tree, the shadow came in <b>${frac(ff)}</b> first attempts (${pct(ff[0] / ff[1], 1)}) and <b>${frac(fl)}</b> later ones. On the very same routes, first attempts saw it about ${Math.round(sr.first_shadows / Math.max(sr.expected, 1) * 10) / 10}× as often as chance would give. The shadow is not a dice roll you can repeat.</p>`;
const steps = $("#black-steps");
if (steps) steps.innerHTML = `How it breaks down: the shadow came in ${frac(ff)} reported first attempts (over-reported); after it, ${as.reached_black} riders reached the Black and <b>${as.bonded_black}</b> bonded (${pct(as.bonded_black / Math.max(as.reached_black, 1))}).`;
if (!box) return; if (!box) return;
const br = b.best_route; const br = b.best_route;
box.hidden = false; box.hidden = false;
@@ -170,6 +172,7 @@
${gaze ? `<li>At the Black, riders who held its gaze bonded in <b>${frac(gaze)}</b> logged answers.</li>` : ""} ${gaze ? `<li>At the Black, riders who held its gaze bonded in <b>${frac(gaze)}</b> logged answers.</li>` : ""}
<li>The bot has made ${b.bot.attempts.toLocaleString()} attempts on ${b.bot.accounts} accounts, ${b.bot.gated_tree} on the gated path, and seen the shadow ${b.bot.shadows} times: none of them was a first attempt${b.bot.earliest_attempt ? ` (its earliest was attempt ${b.bot.earliest_attempt})` : ""}, which fits.</li> <li>The bot has made ${b.bot.attempts.toLocaleString()} attempts on ${b.bot.accounts} accounts, ${b.bot.gated_tree} on the gated path, and seen the shadow ${b.bot.shadows} times: none of them was a first attempt${b.bot.earliest_attempt ? ` (its earliest was attempt ${b.bot.earliest_attempt})` : ""}, which fits.</li>
</ul> </ul>
<p class="small"><b>Why the Black's odds are uncertain.</b> The Black comes only after the shadow, and every logged shadow comes from riders who chose to post their run. Riders are much more likely to post a run with a shadow than one without, and no complete log (the bot, bulk logs) has a first attempt with a shadow yet, so the estimate assumes riders over-report the shadow as much as the meadow Blue. Fresh accounts the bot plays to the tree on their first attempt will replace that assumption; until then the odds are a range, not a figure.</p>
<p class="small muted">Method and every table: <code>analysis/reports/BLACK.md</code>; the numbers: <a href="data/black.json">data/black.json</a>.</p>`; <p class="small muted">Method and every table: <code>analysis/reports/BLACK.md</code>; the numbers: <a href="data/black.json">data/black.json</a>.</p>`;
} }
@@ -189,6 +192,13 @@
const art = (c) => window.DragonArt ? `<svg class="odd-art" viewBox="0 0 200 160" aria-hidden="true">${window.DragonArt.svg(POSE[c], { fill: `var(${DV[c]})`, eyeFill: "var(--bg)", dark: c === "Black" })}</svg>` : ""; const art = (c) => window.DragonArt ? `<svg class="odd-art" viewBox="0 0 200 160" aria-hidden="true">${window.DragonArt.svg(POSE[c], { fill: `var(${DV[c]})`, eyeFill: "var(--bg)", dark: c === "Black" })}</svg>` : "";
$("#odds-colours").innerHTML = cols.map(({ c, first, p, pAll, lo, hi }) => { $("#odds-colours").innerHTML = cols.map(({ c, first, p, pAll, lo, hi }) => {
const e = enc(c); const e = enc(c);
// The Black rests on one roll no complete log has measured (the shadow on a first attempt): a range, not a figure
if (first) return `<a class="odd odd-colour odd-low" href="#valley/${c.toLowerCase()}" style="--c: var(${DV[c]})">${art(c)}
<span class="pill warn">Low evidence</span>
<span class="big">about 1 in ${oneIn(p)}</span><span class="what">first attempts end bonded to a Black, on the best route</span>
${dots(Math.round(p * 1000), 1000, cssVar(DV[c]))}
<span class="how">Somewhere between <b>1 in ${oneIn(hi)}</b> and <b>1 in ${oneIn(lo)}</b> (95%). Later attempts: no shadow has ever been logged, so effectively never.${Math.abs(pAll - p) / p > 0.25 ? ` All form posts suggest about 1 in ${oneIn(pAll)}.` : ""} Every logged shadow comes from riders who chose to post their run, so this rests on an assumption until complete first-attempt logs exist. <span id="black-steps"></span></span>
<span class="go">See the route →</span></a>`;
return `<a class="odd odd-colour" href="#valley/${c.toLowerCase()}" style="--c: var(${DV[c]})">${art(c)} return `<a class="odd odd-colour" href="#valley/${c.toLowerCase()}" style="--c: var(${DV[c]})">${art(c)}
<span class="big">${p >= 0.5 ? pct(p) : `1 in ${oneIn(p)}`}</span><span class="what">${p >= 0.5 ? "of runs" : "runs"} end bonded to ${/^[AEIOU]/.test(c) ? "an" : "a"} ${c}, on the best route${first ? ", on your first attempt" : ""}</span> <span class="big">${p >= 0.5 ? pct(p) : `1 in ${oneIn(p)}`}</span><span class="what">${p >= 0.5 ? "of runs" : "runs"} end bonded to ${/^[AEIOU]/.test(c) ? "an" : "a"} ${c}, on the best route${first ? ", on your first attempt" : ""}</span>
${dots(Math.round(p * 1000), 1000, cssVar(DV[c]))} ${dots(Math.round(p * 1000), 1000, cssVar(DV[c]))}