Route model: first attempts are their own shadow roll; experiment plan and automated bot cycle
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
This commit is contained in:
co-authored by
Claude Fable 5.2
parent
ee07ee143e
commit
ab3664f369
+17
-1
@@ -62,6 +62,10 @@ def estimate(runs, prepared, cfg, rare_goals, seed=20261109, sims=4000):
|
||||
e = {"rare": y, "n": n, "completeRare": ry, "completeN": rn}
|
||||
if g.get("complete"):
|
||||
e.update(rho=1.0, lo=1.0, hi=1.0, used="complete: counted as reported")
|
||||
elif ry < cfg.get("min_complete_rare", 5) and (cfg.get("borrow") or {}).get(node):
|
||||
# complete logging has (almost) never seen it, so the bias cannot be estimated for this roll: assume
|
||||
# the group over-reports it as much as it over-reports another rare roll (filled in below)
|
||||
e.update(rho=None, borrow=cfg["borrow"][node])
|
||||
elif ry < cfg.get("min_complete_rare", 5):
|
||||
# complete logging has (almost) never seen it: the bias cannot be estimated, so this roll is learned
|
||||
# from complete logging only (its range then shows how little is known)
|
||||
@@ -83,10 +87,22 @@ def estimate(runs, prepared, cfg, rare_goals, seed=20261109, sims=4000):
|
||||
for g in groups:
|
||||
for node in rolls:
|
||||
e = out[g["name"]][node]
|
||||
if e.get("rho") is None and not e.get("exclude"):
|
||||
if e.get("rho") is None and not e.get("exclude") and not e.get("borrow"):
|
||||
src = out.get(g.get("same_as") or "", {}).get(node, {})
|
||||
e["rho"] = src.get("rho") or 1.0
|
||||
e["lo"], e["hi"] = src.get("lo", e["rho"]), src.get("hi", e["rho"])
|
||||
# a roll whose bias cannot be estimated borrows the same group's estimate for another roll (`borrow`)
|
||||
for g in groups:
|
||||
for node in rolls:
|
||||
e = out[g["name"]][node]
|
||||
if e.get("borrow"):
|
||||
src = out[g["name"]].get(e["borrow"], {})
|
||||
if src.get("rho"):
|
||||
e.update(rho=src["rho"], lo=src.get("lo"), hi=src.get("hi"),
|
||||
used=f"assumed equal to this group's bias for the {e['borrow']} roll: complete logs have only "
|
||||
f"{e['completeRare']} rare outcomes, too few to estimate it")
|
||||
else:
|
||||
e.update(lo=None, hi=None, exclude=True, used="left out: no bias estimate to borrow")
|
||||
return out, names
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user