Files
threshing-atlas/tests/parity/compare_policy.py
T
cc86343668 Core: the route model in Python, built from a declarative game.yaml (#10)
Step 2 of the Python-first, declarative core.

- core/spec/game.yaml: the game as data: every decision and its options, every roll and the
  earlier steps its odds may depend on, every bonding encounter, and the flow of a run from the
  entrance to the dragons (set / keep / drop / go / end / encounter). Also how the form's coded
  answers map onto the game.
- core/atlas/model.py: what each coded run reveals about each step, count tables, empirical-Bayes
  shrinkage, and the posterior draws (same random numbers as before, reproducible from the seed).
- core/atlas/solver.py: one interpreter of game.yaml used three ways (solve, evaluate a fixed
  policy, export the published tree) instead of the tree written out five times. Vectorised
  over the 200 posterior draws: the build takes about 2 minutes instead of 4, with no memory flag.
- planner/build_policy.py and policy_report.py replace the JS build and report.

Checked against the JavaScript model: every number in policy.json agrees to within 4e-12
(last-digit differences in log/cos). Three "still best" shares differ by one draw in 200, at
near-ties under the "eventually" objective. POLICY.md is otherwise identical.
tests/parity/compare_solver.py keeps checking JS against Python in CI until the site's JS advisor
replaces solver.js; tests/core/test_game.py checks game.yaml is complete and consistent.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 21:04:21 +00:00

47 lines
1.8 KiB
Python

"""Compare two policy.json files structurally: same keys, same choices and labels, numbers equal within a tolerance.
python tests/parity/compare_policy.py A.json B.json [--tol 1e-9]
"""
import json
import math
import sys
def walk(a, b, path, out, tol):
if isinstance(a, dict) and isinstance(b, dict):
for k in sorted(set(a) | set(b)):
if k == "id" and path == "":
continue
if k not in a or k not in b:
out["keys"].append(f"{path}/{k} only in {'B' if k not in a else 'A'}")
continue
walk(a[k], b[k], f"{path}/{k}", out, tol)
elif isinstance(a, list) and isinstance(b, list):
if len(a) != len(b):
out["other"].append(f"{path}: length {len(a)} vs {len(b)}")
for i, (x, y) in enumerate(zip(a, b)):
walk(x, y, f"{path}[{i}]", out, tol)
elif isinstance(a, (int, float)) and isinstance(b, (int, float)) and not isinstance(a, bool) and not isinstance(b, bool):
d = abs(a - b) / max(1.0, abs(a), abs(b))
out["max"] = max(out["max"], d)
if d > tol:
out["numbers"].append(f"{path}: {a} vs {b}")
elif a != b:
out["other"].append(f"{path}: {a!r} vs {b!r}")
def compare(fa, fb, tol=1e-9):
out = {"keys": [], "numbers": [], "other": [], "max": 0.0}
walk(json.load(open(fa)), json.load(open(fb)), "", out, tol)
return out
if __name__ == "__main__":
tol = float(sys.argv[sys.argv.index("--tol") + 1]) if "--tol" in sys.argv else 1e-9
o = compare(sys.argv[1], sys.argv[2], tol)
for k in ("keys", "other", "numbers"):
if o[k]:
print(f"{len(o[k])} {k} differ, e.g.:", *o[k][:8], sep="\n ")
print(f"largest relative difference: {o['max']:.3g}")
sys.exit(1 if o["keys"] or o["other"] or o["numbers"] else 0)