Stable run ids (fingerprints) for the form and the register; tests for the site statistics (#25)
CI / test (pull_request) Successful in 43s

- collection/etl/runs.py fingerprints(): a form or register run's id is now <prefix>-<hash of the row's non-empty
  answers, timestamp included> instead of its row number, so deleting or re-sorting rows in the sheet moves no
  other run's id, and a column added later changes none. The 32 rows that are exact copies of another row get
  -2, -3 (identical, so which is which does not matter). A row edited in the sheet gets a new id. Every id in
  runs_clean.csv and overlap.json changes once; nothing else does (no statistic or model reads the ids).
- tests/core/test_site_stats.py (9 tests): measures.yaml conditions, source groups and presets, bond
  denominators without deliberate deaths (cards, encounters, route map), the riders' paths, and a JSON-safe
  stats.json, on a fixed sample of the published runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-19 20:16:16 +00:00
co-authored by Claude Opus 5.5
parent ca411e8315
commit 1dfa8fb4c3
7 changed files with 9979 additions and 9818 deletions
+5 -5
View File
@@ -103,8 +103,8 @@
"description": "Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's rules are core/spec/form.yaml). The `source` column says where each run came from.",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/runs_clean.csv",
"encodingFormat": "text/csv",
"sha256": "b5e121cf5deab5f8bc1420b34a5bc434d6f933c7452e200eed4891c35de45122",
"contentSize": "3748768 B"
"sha256": "24321aa853d696b4a06226e28bac5e7843759dc3393721ef020d7f5c6beadf4e",
"contentSize": "3828261 B"
},
{
"@type": "cr:FileObject",
@@ -133,8 +133,8 @@
"description": "How many Reddit comment runs may also be in the community form (the same answers within a day), against what chance alone gives (analysis/overlap/overlap.py). Measured only: no run is removed.",
"contentUrl": "https://git.kenschiano.net/DragonKind/threshing-atlas/raw/branch/main/data/public/overlap.json",
"encodingFormat": "application/json",
"sha256": "04280a52f028334451c505ddc35004d98cc1f4595b2294c3746a09e34454e8b0",
"contentSize": "1976 B"
"sha256": "e7511cccf9e8379d8e892639cb38772269ac228756b984e9e0d4d8e9a583a9dd",
"contentSize": "2544 B"
},
{
"@type": "cr:FileObject",
@@ -201,7 +201,7 @@
"@type": "cr:Field",
"@id": "runs/run_id",
"name": "run_id",
"description": "Unique id of the run: sheet-<row of the form export>, thread-<result id>, comment-<Reddit comment id>, reported-<issue number> or bot-<hash>.",
"description": "Unique id of the run, stable when rows are deleted or re-sorted: sheet-<hash of the form row's answers and timestamp>, register-<hash of the register row>, thread-<result id>, comment-<Reddit comment id>, reported-<issue number> or bot-<hash>.",
"dataType": "sc:Text",
"source": {
"fileObject": {