Step 1 of moving the repo to Python with declarative rules. - core/spec/form.yaml: every question (how it is found in an export), every answer and what it is coded as, missing answers, email redaction, burn order, first-dragon pools. - core/atlas/coding.py reads it and replaces core/python/clean.py (python -m atlas.coding). - tests/core/test_coding.py: 16 rule tests, one hand-written row per rule, checking the rules are right rather than only that JS and Python agree. - analysis/stats/analyze.py takes the colours and missing answers from the spec instead of its own copy; the daily update job codes rows through the package. - pyproject.toml for the package; pyyaml and pytest added to requirements. The coded runs are byte-identical to before (5,988 runs); all parity checks still match. Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
22 lines
939 B
Python
22 lines
939 B
Python
"""Check that the JS engine (core/engine.js) and the Python coding rules (core/atlas/coding.py) code every run identically."""
|
|
import json, sys
|
|
import pandas as pd
|
|
js = pd.DataFrame(json.load(open(sys.argv[1] if len(sys.argv) > 1 else "tests/out/runs_js.json")))
|
|
py = pd.read_csv(sys.argv[2] if len(sys.argv) > 2 else "data/derived/runs_clean.csv", keep_default_na=False, na_values=[""])
|
|
def norm(v):
|
|
if v is None or (not isinstance(v, str) and pd.isna(v)) or v == "":
|
|
return None
|
|
if isinstance(v, float) and v.is_integer():
|
|
return str(int(v))
|
|
if isinstance(v, bool):
|
|
return str(v)
|
|
return str(v)
|
|
bad = {}
|
|
for c in py.columns:
|
|
a, b = [norm(v) for v in py[c]], [norm(v) for v in js[c]]
|
|
d = [(i, x, y) for i, (x, y) in enumerate(zip(a, b)) if x != y]
|
|
if d:
|
|
bad[c] = (len(d), d[:3])
|
|
print("runs:", len(py), len(js), "| mismatching columns:", bad or "NONE")
|
|
sys.exit(1 if bad else 0)
|