Core: coding rules as declarative form.yaml + Python package (atlas.coding) (#9)
Step 1 of moving the repo to Python with declarative rules. - core/spec/form.yaml: every question (how it is found in an export), every answer and what it is coded as, missing answers, email redaction, burn order, first-dragon pools. - core/atlas/coding.py reads it and replaces core/python/clean.py (python -m atlas.coding). - tests/core/test_coding.py: 16 rule tests, one hand-written row per rule, checking the rules are right rather than only that JS and Python agree. - analysis/stats/analyze.py takes the colours and missing answers from the spec instead of its own copy; the daily update job codes rows through the package. - pyproject.toml for the package; pyyaml and pytest added to requirements. The coded runs are byte-identical to before (5,988 runs); all parity checks still match. Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
432e04c2c7
commit
3536007ade
@@ -10,9 +10,10 @@ Unofficial fan project. Not affiliated with Dragonkind or the Fourth Wing publis
|
||||
The project has four parts, built on one shared core. Each folder has its own README.
|
||||
|
||||
```
|
||||
core/ the shared model every part uses: the run format and coding rules (engine.js, with the
|
||||
Python mirror in core/python/clean.py), the route graph and bot crosswalk (graph.js), and the
|
||||
decision model and bot advisor (solver.js). No dependencies; runs in Node and the browser.
|
||||
core/ the shared model every part uses. spec/ holds the declarative rules (form.yaml: every question
|
||||
and answer of the community form); atlas/ is the Python package that reads them (coding.py).
|
||||
engine.js, graph.js and solver.js are the JavaScript the site and Ken's bot still use; they move
|
||||
to Python + spec files step by step (docs/ROADMAP.md).
|
||||
collection/ getting data in: sources/ (the form sheet, the Dragon Register, the route PDF check, the daily
|
||||
update job) and bot/ (the DragonKind bot importer). Reddit is not collected yet.
|
||||
analysis/ making sense of it: stats/ (Python statistics, mirrored in core/engine.js), network/ (builds
|
||||
@@ -72,20 +73,22 @@ looks like prose. See `docs/BOT_DATA_PROTOCOL.md` for what the bot should log.
|
||||
In the browser: open the page and use **Load a newer export** with the responses spreadsheet
|
||||
(Google Sheets → File → Download → Microsoft Excel). Everything recomputes. Questions are matched by
|
||||
wording; a renamed question is listed as "not found", and then `COLUMN_KEYS` in
|
||||
`core/engine.js` and `core/python/clean.py` need updating.
|
||||
`core/spec/form.yaml` (and, until the site stops coding rows itself, `core/engine.js`) need updating.
|
||||
|
||||
To update the bundled snapshot, put the new export in `data/raw/form/`, then:
|
||||
|
||||
```
|
||||
python collection/sources/export_raw.py data/raw/form/NEW_EXPORT.xlsx data/public/form_responses.json
|
||||
python core/python/clean.py data/public/form_responses.json data/derived/runs_clean.csv
|
||||
PYTHONPATH=core python -m atlas.coding data/public/form_responses.json data/derived/runs_clean.csv
|
||||
npm test
|
||||
```
|
||||
|
||||
## Changing a coding rule
|
||||
|
||||
Every rule lives in `codeRow()` in `core/engine.js` and `code_row()` in
|
||||
`core/python/clean.py`. Change both and run `npm test`.
|
||||
The tables (how each question is found, what each answer means, which answers count as missing) are in
|
||||
`core/spec/form.yaml`; the logic that combines them is `code_row()` in `core/atlas/coding.py`. Add a case to
|
||||
`tests/core/test_coding.py` for every rule you change, then run `npm test`. Until the site switches to
|
||||
precomputed statistics, `codeRow()` in `core/engine.js` must be changed the same way (the parity tests check it).
|
||||
|
||||
## Statistics in one paragraph
|
||||
|
||||
|
||||
Reference in New Issue
Block a user