Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3b5c655d51 | ||
|
|
e5bcd13db2 | ||
|
|
93bdfcd4da | ||
|
|
b3fa6c4c5b |
@@ -50,10 +50,7 @@ jobs:
|
|||||||
- name: Rebuild the analysis and run tests
|
- name: Rebuild the analysis and run tests
|
||||||
if: steps.check.outputs.changed == 'true'
|
if: steps.check.outputs.changed == 'true'
|
||||||
run: |
|
run: |
|
||||||
node analysis/network/build-graph.js
|
python -m collection rebuild
|
||||||
npm run policy
|
|
||||||
npm run stats
|
|
||||||
python -m collection build
|
|
||||||
npm test
|
npm test
|
||||||
- name: Encrypt the raw store
|
- name: Encrypt the raw store
|
||||||
id: archive
|
id: archive
|
||||||
|
|||||||
@@ -50,10 +50,7 @@ jobs:
|
|||||||
- name: Rebuild the analysis and run tests
|
- name: Rebuild the analysis and run tests
|
||||||
if: steps.check.outputs.changed == 'true'
|
if: steps.check.outputs.changed == 'true'
|
||||||
run: |
|
run: |
|
||||||
node analysis/network/build-graph.js
|
python -m collection rebuild
|
||||||
npm run policy
|
|
||||||
npm run stats
|
|
||||||
python -m collection build
|
|
||||||
npm test
|
npm test
|
||||||
- name: Encrypt the raw store
|
- name: Encrypt the raw store
|
||||||
id: archive
|
id: archive
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ See `docs/BOT_DATA_PROTOCOL.md` for what the bot should log.
|
|||||||
|
|
||||||
## Running everything locally
|
## Running everything locally
|
||||||
|
|
||||||
`python -m collection refresh` does what the daily workflow does: checks every source, rebuilds the runs,
|
`python -m collection refresh` does what the daily workflow does (`python -m collection rebuild` alone redoes everything built from the runs): checks every source, rebuilds the runs,
|
||||||
the graph, the route model and the statistics. Add `--bot <folder or zip>` to import bot runs, and
|
the graph, the route model and the statistics. Add `--bot <folder or zip>` to import bot runs, and
|
||||||
`python -m collection export-bot-kit` zips the files the bot uses (see `docs/BOT_INTEGRATION.md`).
|
`python -m collection export-bot-kit` zips the files the bot uses (see `docs/BOT_INTEGRATION.md`).
|
||||||
|
|
||||||
|
|||||||
+31
-31
@@ -6,13 +6,13 @@ Written by `analysis/black/evidence.py` from `data/public/runs_clean.csv`. Count
|
|||||||
|
|
||||||
| Attempt | Form submissions | Bulk logs | Bot |
|
| Attempt | Form submissions | Bulk logs | Bot |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| 1 | 69 / 748 (9.2%) | 0 / 28 (0.0%) | 0 / 0 |
|
| 1 | 89 / 879 (10.1%) | 0 / 28 (0.0%) | 0 / 0 |
|
||||||
| 2 | 3 / 274 (1.1%) | 0 / 21 (0.0%) | 0 / 4 (0.0%) |
|
| 2 | 0 / 288 (0.0%) | 0 / 21 (0.0%) | 0 / 4 (0.0%) |
|
||||||
| 3 | 1 / 177 (0.6%) | 0 / 14 (0.0%) | 0 / 5 (0.0%) |
|
| 3 | 0 / 184 (0.0%) | 0 / 14 (0.0%) | 0 / 5 (0.0%) |
|
||||||
| 4-10 | 1 / 485 (0.2%) | 0 / 102 (0.0%) | 0 / 36 (0.0%) |
|
| 4-10 | 0 / 506 (0.0%) | 0 / 102 (0.0%) | 0 / 36 (0.0%) |
|
||||||
| 11-30 | 1 / 524 (0.2%) | 0 / 57 (0.0%) | 0 / 137 (0.0%) |
|
| 11-30 | 0 / 573 (0.0%) | 0 / 57 (0.0%) | 0 / 137 (0.0%) |
|
||||||
| 31 or more | 1 / 535 (0.2%) | 0 / 0 | 0 / 54 (0.0%) |
|
| 31 or more | 1 / 610 (0.2%) | 0 / 0 | 0 / 54 (0.0%) |
|
||||||
| not given | 11 / 1406 (0.8%) | 0 / 112 (0.0%) | 0 / 0 |
|
| not given | 3 / 1509 (0.2%) | 0 / 112 (0.0%) | 0 / 0 |
|
||||||
|
|
||||||
Complete logs (bot and bulk) have 28 first attempts at the tree and 430 later ones, with 0 shadows. Zero in 430 later attempts rules out a rate above 0.7% for them; zero in 28 first attempts only rules out more than 10%.
|
Complete logs (bot and bulk) have 28 first attempts at the tree and 430 later ones, with 0 shadows. Zero in 430 later attempts rules out a rate above 0.7% for them; zero in 28 first attempts only rules out more than 10%.
|
||||||
|
|
||||||
@@ -22,39 +22,39 @@ If the shadow depended only on the choices the form records (company, first dire
|
|||||||
|
|
||||||
| Route | First attempt | Later attempts |
|
| Route | First attempt | Later attempts |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 21 / 79 (26.6%) | 1 / 80 (1.2%) |
|
| Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 22 / 88 (25.0%) | 0 / 89 (0.0%) |
|
||||||
| Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / Forest | 4 / 28 (14.3%) | 0 / 91 (0.0%) |
|
| Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / Forest | 5 / 29 (17.2%) | 0 / 93 (0.0%) |
|
||||||
| Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / Forest | 4 / 25 (16.0%) | 0 / 70 (0.0%) |
|
| Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / Forest | 4 / 28 (14.3%) | 0 / 76 (0.0%) |
|
||||||
| Alone / Center / Assess the situation / Daggers or not (protect squadmate) / Left bullies to the dragons / Forest | 1 / 28 (3.6%) | 0 / 62 (0.0%) |
|
| 1 Squadmate / East / Assess the situation / Engage the smaller cadet / Squadmate's revenge / East | 2 / 31 (6.5%) | 0 / 64 (0.0%) |
|
||||||
| Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 1 / 8 (12.5%) | 1 / 80 (1.2%) |
|
| Alone / Center / Assess the situation / Daggers or not (protect squadmate) / Left bullies to the dragons / Forest | 1 / 28 (3.6%) | 0 / 63 (0.0%) |
|
||||||
| 1 Squadmate / East / Assess the situation / Engage the smaller cadet / Squadmate's revenge / East | 2 / 28 (7.1%) | 0 / 60 (0.0%) |
|
| Alone / Center / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West | 1 / 8 (12.5%) | 1 / 82 (1.2%) |
|
||||||
| Alone / East / Assess the situation / Engage the smaller cadet / Squadmate's revenge / East | 1 / 33 (3.0%) | 0 / 48 (0.0%) |
|
| 2 Squadmates / East / Assess the situation / Engage the smaller cadet / Squadmate's revenge / East | 2 / 43 (4.7%) | 0 / 45 (0.0%) |
|
||||||
| Alone / West / Assess the situation / Engage the smaller cadet / Left bullies to the dragons / West | 2 / 23 (8.7%) | 0 / 53 (0.0%) |
|
| Alone / East / Assess the situation / Engage the smaller cadet / Squadmate's revenge / East | 1 / 33 (3.0%) | 0 / 52 (0.0%) |
|
||||||
|
|
||||||
Across all 296 recorded routes, comparing first and later attempts on the same route: 66 first-attempt shadows where 29.4 would be expected if the attempt did not matter (odds ratio 21.5, z = 9.8). First and later attempts choose almost the same routes, so the difference is not in the recorded choices: it is something about the account, or a choice the form does not record.
|
Across all 321 recorded routes, comparing first and later attempts on the same route: 85 first-attempt shadows where 36.4 would be expected if the attempt did not matter (odds ratio 626.3, z = 11.9). First and later attempts choose almost the same routes, so the difference is not in the recorded choices: it is something about the account, or a choice the form does not record.
|
||||||
|
|
||||||
## 3. On one route, is the shadow decided by the cadet's wing, section and squad?
|
## 3. On one route, is the shadow decided by the cadet's wing, section and squad?
|
||||||
|
|
||||||
The developer says the Black is decided by choices and by what the site knows about the account, with no dice. If the recorded route and the cadet assignment were all of it, every first attempt with the same route and assignment would agree. On the best-recorded route (Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West; 21 / 79 (26.6%) on first attempts), 20 assignments have two or more runs and 10 of them disagree (some runs with the shadow, some without). So something else decides it: a choice the form does not record, or a fact about the account other than its assignment. No wing, section or squad stands out:
|
The developer says the Black is decided by choices and by what the site knows about the account, with no dice. If the recorded route and the cadet assignment were all of it, every first attempt with the same route and assignment would agree. On the best-recorded route (Alone / West / Assess the situation / Engage the smaller cadet / Squadmate's revenge / West; 22 / 88 (25.0%) on first attempts), 22 assignments have two or more runs and 10 of them disagree (some runs with the shadow, some without). So something else decides it: a choice the form does not record, or a fact about the account other than its assignment. No wing, section or squad stands out:
|
||||||
|
|
||||||
| | shadow / runs |
|
| | shadow / runs |
|
||||||
|---|---|
|
|---|---|
|
||||||
| wing | Fourth: 5/25, Third: 6/23, First: 6/16, Second: 3/12 |
|
| wing | Fourth: 5/27, Third: 7/26, First: 6/20, Second: 3/12 |
|
||||||
| section | Claw: 8/27, Tail: 7/26, Flame: 5/23, (blank): 1/3 |
|
| section | Claw: 9/30, Flame: 5/29, Tail: 7/26, (blank): 1/3 |
|
||||||
| squad | Second: 10/28, First: 6/26, Third: 4/23 |
|
| squad | Second: 11/34, First: 6/27, Third: 4/25 |
|
||||||
| first_sighting | Brown: 10/40, Green: 9/37 |
|
| first_sighting | Brown: 11/45, Green: 9/41 |
|
||||||
| sighting_action | Hold still: 10/43, Respect / face it: 4/16, Unknown: 7/13, Weapon: 0/5 |
|
| sighting_action | Hold still: 10/49, Respect / face it: 4/18, Unknown: 8/14, Weapon: 0/5 |
|
||||||
|
|
||||||
## 4. There is no single route
|
## 4. There is no single route
|
||||||
|
|
||||||
The 87 form runs with a shadow (all attempts) took 44 different recorded routes:
|
The 93 form runs with a shadow (all attempts) took 46 different recorded routes:
|
||||||
|
|
||||||
- **Company:** Alone 70, 2 Squadmates 9, 1 Squadmate 8
|
- **Company:** Alone 70, 2 Squadmates 12, 1 Squadmate 11
|
||||||
- **First direction:** West 53, Center 20, East 14
|
- **First direction:** West 58, Center 20, East 15
|
||||||
- **First move in the fight:** Assess the situation 76, (not recorded) 9, Even the numbers (smaller cadet) 2
|
- **First move in the fight:** Assess the situation 82, (not recorded) 9, Even the numbers (smaller cadet) 2
|
||||||
- **The dagger:** Engage the smaller cadet 56, (not recorded) 22, Daggers or not (protect squadmate) 9
|
- **The dagger:** Engage the smaller cadet 60, (not recorded) 22, Daggers or not (protect squadmate) 11
|
||||||
- **The bullies' fate:** Squadmate's revenge 54, Left bullies to the dragons 22, Killed the bullies yourself 9, Unknown 2
|
- **The bullies' fate:** Squadmate's revenge 58, Left bullies to the dragons 23, Killed the bullies yourself 9, Unknown 3
|
||||||
- **The fork:** West 40, Forest 35, East 11, Unknown 1
|
- **The fork:** West 44, Forest 36, East 12, Unknown 1
|
||||||
|
|
||||||
Shadows reported without both gates: 3 (likely mis-entered).
|
Shadows reported without both gates: 3 (likely mis-entered).
|
||||||
|
|
||||||
@@ -62,9 +62,9 @@ Shadows reported without both gates: 3 (likely mis-entered).
|
|||||||
|
|
||||||
The riders' notes describe one sequence: follow the shadow, a Blue lands and burns another cadet, the Blue leaves, then the Black appears. The Blue is a trial on the way, not an alternative to the Black.
|
The riders' notes describe one sequence: follow the shadow, a Blue lands and burns another cadet, the Blue leaves, then the Black appears. The Blue is a trial on the way, not an alternative to the Black.
|
||||||
|
|
||||||
- Runs that reached the Black: 70, bonded 49.
|
- Runs that reached the Black: 76, bonded 55.
|
||||||
- Runs that died at the Blue: 15. What their notes say they did: approached the Blue 10, warned the other cadet / intervened 4.
|
- Runs that died at the Blue: 15. What their notes say they did: approached the Blue 10, warned the other cadet / intervened 4.
|
||||||
- The answer given to the Black, where the note says (bonded / runs): held its gaze 27/39; lowered eyes 0/7; ignored the sound 0/1.
|
- The answer given to the Black, where the note says (bonded / runs): held its gaze 30/42; lowered eyes 0/7; ignored the sound 0/1.
|
||||||
|
|
||||||
Holding its gaze is the only answer recorded with a bond, and it still fails about a quarter of the time. The developer says a rider who knows the system bonds with certainty, so those failures are not explained yet: the notes of the riders who died are too short to show what else they did differently.
|
Holding its gaze is the only answer recorded with a bond, and it still fails about a quarter of the time. The developer says a rider who knows the system bonds with certainty, so those failures are not explained yet: the notes of the riders who died are too short to show what else they did differently.
|
||||||
|
|
||||||
|
|||||||
+25
-22
@@ -73,6 +73,29 @@ def _step(label, cmd):
|
|||||||
subprocess.run(cmd, cwd=ROOT, check=True)
|
subprocess.run(cmd, cwd=ROOT, check=True)
|
||||||
|
|
||||||
|
|
||||||
|
def rebuild():
|
||||||
|
"""Everything built from the runs, in order. The one list the local refresh, the bot cycle and the daily
|
||||||
|
workflows all use (`python -m collection rebuild`), so a new report only needs adding here."""
|
||||||
|
py = sys.executable
|
||||||
|
# the runs first: if they do not match data/dictionary.yaml, stop here rather than model yesterday's runs
|
||||||
|
_step("runs, checked against the dictionary", [py, "-m", "collection", "build"])
|
||||||
|
if shutil.which("node"):
|
||||||
|
_step("route graph", ["node", "analysis/network/build-graph.js"])
|
||||||
|
else:
|
||||||
|
print("\n(node is not installed: data/public/graph.json is not rebuilt; everything else is)")
|
||||||
|
_step("route model (a few minutes)", [py, "planner/build_policy.py"])
|
||||||
|
_step("experiment plan", [py, "planner/build_experiments.py"])
|
||||||
|
_step("what the runs show about the Black", [py, "analysis/black/evidence.py"])
|
||||||
|
_step("screen tree", [py, "analysis/tree/build_tree.py"])
|
||||||
|
try:
|
||||||
|
import scipy # noqa: F401
|
||||||
|
_step("statistics", [py, "analysis/stats/site_stats.py"])
|
||||||
|
except ImportError:
|
||||||
|
print("\n(scipy is not installed: data/public/stats.json is not rebuilt)")
|
||||||
|
_step("runs, generated files and metadata", [py, "-m", "collection", "build"])
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def refresh(update_main, bots=()):
|
def refresh(update_main, bots=()):
|
||||||
"""update -> (graph, policy, stats, build) when something changed. Returns 0 on success."""
|
"""update -> (graph, policy, stats, build) when something changed. Returns 0 on success."""
|
||||||
py = sys.executable
|
py = sys.executable
|
||||||
@@ -81,16 +104,7 @@ def refresh(update_main, bots=()):
|
|||||||
rc = update_main(args)
|
rc = update_main(args)
|
||||||
for i, b in enumerate(bots):
|
for i, b in enumerate(bots):
|
||||||
rc |= update_main(["update", "--source", "bot", "--file", f"bot={b}", "--report", str(path(f"exports/last-report-bot{i + 1}.md"))])
|
rc |= update_main(["update", "--source", "bot", "--file", f"bot={b}", "--report", str(path(f"exports/last-report-bot{i + 1}.md"))])
|
||||||
if shutil.which("node"):
|
rebuild()
|
||||||
_step("route graph", ["node", "analysis/network/build-graph.js"])
|
|
||||||
else:
|
|
||||||
print("\n(node is not installed: data/public/graph.json is not rebuilt; everything else is)")
|
|
||||||
_step("route model (about 2 minutes)", [py, "planner/build_policy.py"])
|
|
||||||
_step("experiment plan", [py, "planner/build_experiments.py"])
|
|
||||||
_step("what the runs show about the Black", [py, "analysis/black/evidence.py"])
|
|
||||||
_step("screen tree", [py, "analysis/tree/build_tree.py"])
|
|
||||||
_step("statistics", [py, "analysis/stats/site_stats.py"])
|
|
||||||
_step("runs, generated files and metadata", [py, "-m", "collection", "build"])
|
|
||||||
print(f"\nDone. What changed: exports/last-report.md. Send the bot its files with: {py} -m collection export-bot-kit")
|
print(f"\nDone. What changed: exports/last-report.md. Send the bot its files with: {py} -m collection export-bot-kit")
|
||||||
return rc
|
return rc
|
||||||
|
|
||||||
@@ -169,18 +183,7 @@ def cycle(update_main, bots, deliver_to=None, community=False, force=False):
|
|||||||
built = all(path(src).exists() for src in KIT.values())
|
built = all(path(src).exists() for src in KIT.values())
|
||||||
changed = force or not built or _fingerprint() != before
|
changed = force or not built or _fingerprint() != before
|
||||||
if changed:
|
if changed:
|
||||||
if shutil.which("node"):
|
rebuild()
|
||||||
_step("route graph", ["node", "analysis/network/build-graph.js"])
|
|
||||||
_step("route model (a few minutes)", [py, "planner/build_policy.py"])
|
|
||||||
_step("experiment plan", [py, "planner/build_experiments.py"])
|
|
||||||
_step("what the runs show about the Black", [py, "analysis/black/evidence.py"])
|
|
||||||
_step("screen tree", [py, "analysis/tree/build_tree.py"])
|
|
||||||
try:
|
|
||||||
import scipy # noqa: F401
|
|
||||||
_step("statistics", [py, "analysis/stats/site_stats.py"])
|
|
||||||
except ImportError:
|
|
||||||
print("\n(scipy is not installed: data/public/stats.json is not rebuilt; the bot does not use it)")
|
|
||||||
_step("runs, generated files and metadata", [py, "-m", "collection", "build"])
|
|
||||||
else:
|
else:
|
||||||
print("\nNo new runs: the route model and the plan are unchanged.")
|
print("\nNo new runs: the route model and the plan are unchanged.")
|
||||||
now = _headline()
|
now = _headline()
|
||||||
|
|||||||
@@ -109,6 +109,7 @@ def main(argv=None):
|
|||||||
fg.add_argument("args", nargs=argparse.REMAINDER)
|
fg.add_argument("args", nargs=argparse.REMAINDER)
|
||||||
k = sub.add_parser("export-bot-kit", help="zip policy.json, experiments.json, solver.js and graph.js for the bot")
|
k = sub.add_parser("export-bot-kit", help="zip policy.json, experiments.json, solver.js and graph.js for the bot")
|
||||||
k.add_argument("--out", default="exports")
|
k.add_argument("--out", default="exports")
|
||||||
|
sub.add_parser("rebuild", help="rebuild everything made from the runs: graph, route model, plan, reports, statistics, build")
|
||||||
r = sub.add_parser("refresh", help="update every source, then rebuild everything (the daily workflow, locally)")
|
r = sub.add_parser("refresh", help="update every source, then rebuild everything (the daily workflow, locally)")
|
||||||
r.add_argument("--bot", action="append", default=[], help="a bot folder or a .zip of its data (repeatable)")
|
r.add_argument("--bot", action="append", default=[], help="a bot folder or a .zip of its data (repeatable)")
|
||||||
c = sub.add_parser("cycle", help="the loop with the bot: import its runs, refit, re-plan, deliver (safe on a timer)")
|
c = sub.add_parser("cycle", help="the loop with the bot: import its runs, refit, re-plan, deliver (safe on a timer)")
|
||||||
@@ -129,6 +130,9 @@ def main(argv=None):
|
|||||||
from .etl.handoff import export_bot_kit
|
from .etl.handoff import export_bot_kit
|
||||||
print(f"wrote {export_bot_kit(a.out)}")
|
print(f"wrote {export_bot_kit(a.out)}")
|
||||||
return 0
|
return 0
|
||||||
|
if a.cmd == "rebuild":
|
||||||
|
from .etl.handoff import rebuild
|
||||||
|
return rebuild()
|
||||||
if a.cmd == "refresh":
|
if a.cmd == "refresh":
|
||||||
from .etl.handoff import refresh
|
from .etl.handoff import refresh
|
||||||
return refresh(main, a.bot)
|
return refresh(main, a.bot)
|
||||||
|
|||||||
@@ -250,7 +250,7 @@ def code_row(r, C):
|
|||||||
goal = g("goal") or (A["goal_v2"].get(goal_raw, goal_raw) if goal_raw and goal_raw != "Unknown" else None) or "(not asked/blank)"
|
goal = g("goal") or (A["goal_v2"].get(goal_raw, goal_raw) if goal_raw and goal_raw != "Unknown" else None) or "(not asked/blank)"
|
||||||
return {
|
return {
|
||||||
"source": source, "date": date, "wing": wing, "section": section,
|
"source": source, "date": date, "wing": wing, "section": section,
|
||||||
"squad": g("squad") or "Unknown", "attempt": attempt,
|
"squad": A["squad_any_case"].get((g("squad") or "").strip().lower(), g("squad") or "Unknown"), "attempt": attempt,
|
||||||
# "I've lost count" means many attempts, so it is a later attempt, not missing
|
# "I've lost count" means many attempts, so it is a later attempt, not missing
|
||||||
"attempt_group": "First attempt" if attempt == 1 else ("Later attempt" if (attempt and attempt > 1) or "lost count" in (g("attempt") or "").lower() else "Unknown"),
|
"attempt_group": "First attempt" if attempt == 1 else ("Later attempt" if (attempt and attempt > 1) or "lost count" in (g("attempt") or "").lower() else "Unknown"),
|
||||||
"goal": goal, "company": company, "left_squadmates": left_squad,
|
"goal": goal, "company": company, "left_squadmates": left_squad,
|
||||||
|
|||||||
@@ -104,6 +104,12 @@ answers:
|
|||||||
"Trying to bond the Red dragon under the tree": "Red (tree)"
|
"Trying to bond the Red dragon under the tree": "Red (tree)"
|
||||||
"Daily Run Assignment data": "Daily run data"
|
"Daily Run Assignment data": "Daily run data"
|
||||||
"Just here for fun, and maybe a dragon": "Any (for fun)"
|
"Just here for fun, and maybe a dragon": "Any (for fun)"
|
||||||
|
# Squad: matched ignoring case and spaces (riders type it); an answer not listed still stops the build
|
||||||
|
squad_any_case:
|
||||||
|
"first": "First"
|
||||||
|
"second": "Second"
|
||||||
|
"third": "Third"
|
||||||
|
"don't know": "Don't know"
|
||||||
sighting_action:
|
sighting_action:
|
||||||
"Hold Ground / Stay Still": "Hold still"
|
"Hold Ground / Stay Still": "Hold still"
|
||||||
"Respect / Face it": "Respect / face it"
|
"Respect / Face it": "Respect / face it"
|
||||||
|
|||||||
+12
-12
@@ -88,8 +88,8 @@
|
|||||||
"description": "Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's rules are core/spec/form.yaml). The `source` column says where each run came from.",
|
"description": "Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's rules are core/spec/form.yaml). The `source` column says where each run came from.",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/runs_clean.csv",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/runs_clean.csv",
|
||||||
"encodingFormat": "text/csv",
|
"encodingFormat": "text/csv",
|
||||||
"sha256": "12060f4497e92abf60de728167cb3d37eaf9ed889060fd110be581df0e1d29f4",
|
"sha256": "efe9d8395e0fdd15d936a22915a9f9ee13e0ebf32f685141562bdc5681b6bf5e",
|
||||||
"contentSize": "3123252 B"
|
"contentSize": "3324468 B"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"@type": "cr:FileObject",
|
"@type": "cr:FileObject",
|
||||||
@@ -98,8 +98,8 @@
|
|||||||
"description": "The community \"Threshing Routes\" form export as published (headers and rows), with email addresses in free-text answers redacted. The source of runs_clean.",
|
"description": "The community \"Threshing Routes\" form export as published (headers and rows), with email addresses in free-text answers redacted. The source of runs_clean.",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/form_responses.json",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/form_responses.json",
|
||||||
"encodingFormat": "application/json",
|
"encodingFormat": "application/json",
|
||||||
"sha256": "7ca72f9967a9b15fd52160106b1508c8cfbde804539d6afa3f04df326d7bf2b2",
|
"sha256": "742635ebb21616020025063d2e6587e8ad06564a2f1807b94a99f38518479008",
|
||||||
"contentSize": "11019524 B"
|
"contentSize": "11878421 B"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"@type": "cr:FileObject",
|
"@type": "cr:FileObject",
|
||||||
@@ -118,8 +118,8 @@
|
|||||||
"description": "The Unofficial Dragon Register (which dragon each rider bonded), without rider names.",
|
"description": "The Unofficial Dragon Register (which dragon each rider bonded), without rider names.",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/register.json",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/register.json",
|
||||||
"encodingFormat": "application/json",
|
"encodingFormat": "application/json",
|
||||||
"sha256": "b55e62a7be1d2f8858bc3bf0ad35881bbcd87f61a634f219f2b2bea5a453ff58",
|
"sha256": "24ed1b9dfcae322df1d123d169557cc22a933426b48f7a4e5b0a0ba0d412b7a5",
|
||||||
"contentSize": "116221 B"
|
"contentSize": "117781 B"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"@type": "cr:FileObject",
|
"@type": "cr:FileObject",
|
||||||
@@ -128,8 +128,8 @@
|
|||||||
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
|
"description": "Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is analysis/stats/measures.yaml).",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/stats.json",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/stats.json",
|
||||||
"encodingFormat": "application/json",
|
"encodingFormat": "application/json",
|
||||||
"sha256": "aa3b1eb313655b75c290f538373b72b1a29b4685f389feabea487b333c0f96a6",
|
"sha256": "aa9e84d1336dcdd8d957e531ad6e5b9f95adcdc2b98363b0e856e2cc6e28dcb8",
|
||||||
"contentSize": "5063942 B"
|
"contentSize": "5308966 B"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"@type": "cr:FileObject",
|
"@type": "cr:FileObject",
|
||||||
@@ -138,8 +138,8 @@
|
|||||||
"description": "The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).",
|
"description": "The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/policy.json",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/policy.json",
|
||||||
"encodingFormat": "application/json",
|
"encodingFormat": "application/json",
|
||||||
"sha256": "695f2ca834f2164ea0bae07bb70353b6a54931b0b984e8aed6f26d1b451e018d",
|
"sha256": "966d32a52d86fde0239ae8b1821b43f888a159d94b0bce908ef8aaf18119efa0",
|
||||||
"contentSize": "1083842 B"
|
"contentSize": "1102627 B"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"@type": "cr:FileObject",
|
"@type": "cr:FileObject",
|
||||||
@@ -148,8 +148,8 @@
|
|||||||
"description": "The route graph over every run in runs_clean.csv, counted per source (analysis/network/build-graph.js).",
|
"description": "The route graph over every run in runs_clean.csv, counted per source (analysis/network/build-graph.js).",
|
||||||
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/graph.json",
|
"contentUrl": "https://git.kenschiano.net/kathleen/threshing-atlas/raw/branch/main/data/public/graph.json",
|
||||||
"encodingFormat": "application/json",
|
"encodingFormat": "application/json",
|
||||||
"sha256": "2757b6935b3dec6854303e271e606e7e5fe089b38e3770a8db2fe664bcb74b5f",
|
"sha256": "5b5dc0f215b42e9f65454fffb256172f2f70dedf703000aaf080ad42ee79d280",
|
||||||
"contentSize": "73446 B"
|
"contentSize": "74020 B"
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"recordSet": [
|
"recordSet": [
|
||||||
|
|||||||
@@ -1,9 +1,9 @@
|
|||||||
{
|
{
|
||||||
"sha256": "3778ece486c7fca9b38f1738e7065ba20d7b1fd739bcd1b845d46e7520e3a22e",
|
"sha256": "99953503376b54ee2ca2061bc277c712681a66add4d62737535e4afca11ba03e",
|
||||||
"bytes": 641363,
|
"bytes": 638678,
|
||||||
"pages": 1,
|
"pages": 1,
|
||||||
"page_hashes": [
|
"page_hashes": [
|
||||||
"e88c846a14efc549"
|
"513739c2cb809008"
|
||||||
],
|
],
|
||||||
"line_hashes": [
|
"line_hashes": [
|
||||||
"0005ef79b6b0",
|
"0005ef79b6b0",
|
||||||
@@ -27,7 +27,6 @@
|
|||||||
"09ede21a5670",
|
"09ede21a5670",
|
||||||
"0a6389a6b822",
|
"0a6389a6b822",
|
||||||
"0c3e015009e5",
|
"0c3e015009e5",
|
||||||
"0c5d7989af41",
|
|
||||||
"0c6edb02c260",
|
"0c6edb02c260",
|
||||||
"0caca604d8ec",
|
"0caca604d8ec",
|
||||||
"0d00728c456e",
|
"0d00728c456e",
|
||||||
@@ -63,7 +62,6 @@
|
|||||||
"1fa9e3998e8b",
|
"1fa9e3998e8b",
|
||||||
"1fbdd6f48a30",
|
"1fbdd6f48a30",
|
||||||
"20539cda4fc1",
|
"20539cda4fc1",
|
||||||
"208a6d1ce0de",
|
|
||||||
"21499b8293b5",
|
"21499b8293b5",
|
||||||
"21e4274038e8",
|
"21e4274038e8",
|
||||||
"2287d8613d2a",
|
"2287d8613d2a",
|
||||||
@@ -90,7 +88,6 @@
|
|||||||
"2f251aff0a77",
|
"2f251aff0a77",
|
||||||
"2f501595d6f2",
|
"2f501595d6f2",
|
||||||
"2f63258bd7ea",
|
"2f63258bd7ea",
|
||||||
"2f80077a0788",
|
|
||||||
"2f943c163229",
|
"2f943c163229",
|
||||||
"2fe14cd5feca",
|
"2fe14cd5feca",
|
||||||
"306a70a396b4",
|
"306a70a396b4",
|
||||||
@@ -244,6 +241,7 @@
|
|||||||
"860ce7852dc3",
|
"860ce7852dc3",
|
||||||
"87094fc18528",
|
"87094fc18528",
|
||||||
"877de8ebb965",
|
"877de8ebb965",
|
||||||
|
"87ad118c11e5",
|
||||||
"87c3e080dcb4",
|
"87c3e080dcb4",
|
||||||
"8838a3634b92",
|
"8838a3634b92",
|
||||||
"883caeb7ca11",
|
"883caeb7ca11",
|
||||||
@@ -263,7 +261,6 @@
|
|||||||
"919bc9b8d743",
|
"919bc9b8d743",
|
||||||
"920a5924f124",
|
"920a5924f124",
|
||||||
"935f14ac67b6",
|
"935f14ac67b6",
|
||||||
"94844cb129a8",
|
|
||||||
"94ab1f4a7216",
|
"94ab1f4a7216",
|
||||||
"954f0ca1b653",
|
"954f0ca1b653",
|
||||||
"95a14ed2a7fc",
|
"95a14ed2a7fc",
|
||||||
@@ -298,7 +295,6 @@
|
|||||||
"a3d7a6bf2036",
|
"a3d7a6bf2036",
|
||||||
"a3f146576c0e",
|
"a3f146576c0e",
|
||||||
"a5973cf86c17",
|
"a5973cf86c17",
|
||||||
"a69204c67bcd",
|
|
||||||
"a69a4139b340",
|
"a69a4139b340",
|
||||||
"a6e0a42de76e",
|
"a6e0a42de76e",
|
||||||
"a6e8ab03915b",
|
"a6e8ab03915b",
|
||||||
@@ -349,14 +345,12 @@
|
|||||||
"b7bd4220ff85",
|
"b7bd4220ff85",
|
||||||
"b8ab74d65421",
|
"b8ab74d65421",
|
||||||
"b8b0e7effd20",
|
"b8b0e7effd20",
|
||||||
"b9514000c923",
|
|
||||||
"b95206531cb4",
|
"b95206531cb4",
|
||||||
"ba4788b226aa",
|
"ba4788b226aa",
|
||||||
"bac081e175b0",
|
"bac081e175b0",
|
||||||
"bb0e505c6386",
|
"bb0e505c6386",
|
||||||
"bb36f57782ca",
|
"bb36f57782ca",
|
||||||
"bb6db2fb1f33",
|
"bb6db2fb1f33",
|
||||||
"bb8b9de8437d",
|
|
||||||
"bc807bf2a135",
|
"bc807bf2a135",
|
||||||
"be5ba8fe14a3",
|
"be5ba8fe14a3",
|
||||||
"bec9019352b4",
|
"bec9019352b4",
|
||||||
@@ -414,6 +408,7 @@
|
|||||||
"d978e0e48792",
|
"d978e0e48792",
|
||||||
"d9a847a1a79a",
|
"d9a847a1a79a",
|
||||||
"d9ae6af36f3c",
|
"d9ae6af36f3c",
|
||||||
|
"dab3df07f542",
|
||||||
"dab61aea1eb7",
|
"dab61aea1eb7",
|
||||||
"db875b25b06f",
|
"db875b25b06f",
|
||||||
"dbb3b50e59c7",
|
"dbb3b50e59c7",
|
||||||
@@ -491,5 +486,5 @@
|
|||||||
"ff1f017072cc",
|
"ff1f017072cc",
|
||||||
"ffe503ead4e4"
|
"ffe503ead4e4"
|
||||||
],
|
],
|
||||||
"checked": "2026-11-10"
|
"checked": "2026-11-16"
|
||||||
}
|
}
|
||||||
|
|||||||
+9
-9
@@ -5,19 +5,19 @@
|
|||||||
"changed": "2026-11-10"
|
"changed": "2026-11-10"
|
||||||
},
|
},
|
||||||
"reddit_pdf": {
|
"reddit_pdf": {
|
||||||
"sha256": "3778ece486c7fca9b38f1738e7065ba20d7b1fd739bcd1b845d46e7520e3a22e",
|
"sha256": "99953503376b54ee2ca2061bc277c712681a66add4d62737535e4afca11ba03e",
|
||||||
"bytes": 641363,
|
"bytes": 638678,
|
||||||
"changed": "2026-11-10"
|
"changed": "2026-11-16"
|
||||||
},
|
},
|
||||||
"reddit_spreadsheet": {
|
"reddit_spreadsheet": {
|
||||||
"sha256": "97377c3c3cfc9756a3520580441111d5154642e224441e35754f021c701a46fd",
|
"sha256": "cfc2cff4a0f69d65816a4c7ac7b6ce21a845f9e0324d796ec8ac5196916222ea",
|
||||||
"bytes": 4695125,
|
"bytes": 6943129,
|
||||||
"changed": "2026-11-12"
|
"changed": "2026-11-16"
|
||||||
},
|
},
|
||||||
"register": {
|
"register": {
|
||||||
"sha256": "1a4e98f3a140015f8849c2ff5f3c1635def12614ad60461b8213dcb3b3b420a2",
|
"sha256": "0f818a7c7e56e1346522153ddac169e726d0b663d030392e6114bffe55f6f649",
|
||||||
"bytes": 50431,
|
"bytes": 51026,
|
||||||
"changed": "2026-11-12"
|
"changed": "2026-11-16"
|
||||||
},
|
},
|
||||||
"reported": {
|
"reported": {
|
||||||
"sha256": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945",
|
"sha256": "4f53cda18c2baa0c0354bb5f9a3ecbe5ed12ab4d8e11ba873c2f11161202b945",
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2598
-1978
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
@@ -263,7 +263,7 @@ answer to `core/spec/form.yaml` (or the value to `data/dictionary.yaml`) and run
|
|||||||
|
|
||||||
## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle
|
## 10. Update, 2026-11-11: first attempts, the experiment plan, the automated cycle
|
||||||
|
|
||||||
Branch `first-attempt-cycle` (not merged, not pushed).
|
Branch `first-attempt-cycle` (pushed to Gitea; not merged into main yet).
|
||||||
|
|
||||||
- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first
|
- **The shadow is close to a first-attempt event.** Form submissions, gated tree runs: 69 shadows in 748 first
|
||||||
attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the
|
attempts (9.2%), 7 in 1,995 later ones (0.35%), in every week. Complete logs have only 28 first attempts at the
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ NEW = {
|
|||||||
"hour": "After walking an hour, what did you do?",
|
"hour": "After walking an hour, what did you do?",
|
||||||
"goal_v2": "Why are you doing this run?",
|
"goal_v2": "Why are you doing this run?",
|
||||||
"malek": "Would you seek Malek?",
|
"malek": "Would you seek Malek?",
|
||||||
|
"squad": "What squad were you assigned?",
|
||||||
**{f"enc_{c}": f"Did you attempt to bond? [{c}]" for c in coding.COLORS},
|
**{f"enc_{c}": f"Did you attempt to bond? [{c}]" for c in coding.COLORS},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -62,6 +63,10 @@ class Columns(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class Rows(unittest.TestCase):
|
class Rows(unittest.TestCase):
|
||||||
|
def test_squad_ignores_case(self):
|
||||||
|
for typed, coded in (("second", "Second"), (" THIRD ", "Third"), ("Don't know", "Don't know"), (None, "Unknown")):
|
||||||
|
self.assertEqual(code(NEW, company="Alone", squad=typed)["squad"], coded)
|
||||||
|
|
||||||
def test_form_submission_vs_bulk_import(self):
|
def test_form_submission_vs_bulk_import(self):
|
||||||
self.assertEqual(code(OLD, timestamp="2026-10-03 14:02:11")["source"], "Form")
|
self.assertEqual(code(OLD, timestamp="2026-10-03 14:02:11")["source"], "Form")
|
||||||
r = code(OLD, timestamp="Data Import")
|
r = code(OLD, timestamp="Data Import")
|
||||||
|
|||||||
Reference in New Issue
Block a user