Files
threshing-atlas/tests/parity/compare_results.py
T
432e04c2c7 Refactor: lay the repo out by section (core, collection, analysis, planner, site) (#8)
Moves only; no behaviour changes. Every file now lives under the part of the project it
belongs to, each section has a README, and docs/PRIVACY.md collects the rules all parts follow.

  core/        engine.js, graph.js, solver.js, python/clean.py
  collection/  sources/ (form, register, PDF check, daily update), bot/ (importer)
  analysis/    stats/ (Python statistics), network/ (build-graph.js), reports/ (POLICY.md)
  planner/     build-policy.js, policy-report.js
  site/        atlas/ (the page), build.js, qa/
  tests/       core/ (unit), parity/ (JS vs Python), fixtures/

Checked against the previous layout: tests pass; coded runs, test outputs, graph.json and
policy.json are byte-identical; the built site differs only in comments that name file paths.
The published file names Ken's bot downloads (solver.js, graph.js, data/policy.json) are unchanged.

Also removes the undefined/ folder of QA screenshots committed by mistake (the QA scripts now
default to a temp folder) and test.txt, and drops the unused npm workspaces.


Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 19:46:33 +00:00

38 lines
1.8 KiB
Python

"""Compare factor tables and models between the JS engine and the Python reference."""
import json, sys
js = json.load(open("tests/out/results_js.json"))
py = json.load(open("tests/out/results_py.json"))
TOL = 1e-6
fails = 0
def close(a, b, rel=TOL):
if a is None or b is None:
return a is None and b is None
return abs(a - b) <= rel * max(1, abs(a), abs(b))
for key, ft in py["factors"].items():
jrows = {(r["factor"], r["level"]): r for r in js["factors"][key]["rows"]}
for r in ft["rows"]:
j = jrows.get((r["factor"], r["level"]))
if j is None:
print("missing in JS:", key, r["factor"], r["level"]); fails += 1; continue
for f in ("k", "n", "p", "lo", "hi", "or", "or_lo", "or_hi", "fisher_p", "q"):
if (f in r) != (f in j) or (f in r and not close(r[f], j[f], 1e-5)):
print("diff", key, r["factor"], r["level"], f, r.get(f), j.get(f)); fails += 1
for key, m in py["models"].items():
jm = js["models"][key]
if m["n"] != jm["n"] or m["events"] != jm["events"]:
print("model size diff", key); fails += 1
jt = {t["term"]: t for t in jm["terms"]}
for t in m["terms"]:
j = jt.get(t["term"])
if not j:
print("term missing", key, t["term"]); fails += 1; continue
for f in ("or", "lo", "hi"):
if not close(t[f], j[f], 1e-4):
print("model diff", key, t["term"], f, t[f], j[f]); fails += 1
if abs(t["p"] - j["p"]) > 1e-6:
print("model p diff", key, t["term"], t["p"], j["p"]); fails += 1
n_rows = sum(len(f["rows"]) for f in py["factors"].values())
n_terms = sum(len(m["terms"]) for m in py["models"].values())
print(f"checked {n_rows} factor rows and {n_terms} model terms: {'ALL MATCH' if not fails else str(fails) + ' differences'}")
sys.exit(1 if fails else 0)