Black report: Reddit comments (and reported runs) in the attempt table (#25)
CI / test (pull_request) Successful in 42s

- analysis/black/evidence.py: Reddit comments and "Report a run" are their own groups. The attempt table gets a
  column for each once it has a gated tree run (Reddit: 4 runs, 3 with a shadow; reported: none yet). A note says
  Reddit describes 17 shadows but only 3 say both gates and 3 the attempt, and that, like form posts, they show
  what shadow runs look like, not how often the shadow comes. black.json: by_attempt["Reddit comments"],
  reddit_shadows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
2026-11-19 20:47:00 +00:00
co-authored by Claude Opus 5.5
parent fbf42508a0
commit ee222dd775
3 changed files with 64 additions and 12 deletions
+22 -3
View File
@@ -35,9 +35,17 @@ def group(r):
return "bot"
if r["source"] == "reddit spreadsheet":
return "form" if r["entry"] == "Form" else "bulk logs"
if r["source"] == "reddit comment":
return "Reddit comments"
if r["source"] == "reported":
return "reported"
return "other"
GROUP_NAMES = {"form": "Form submissions", "bulk logs": "Bulk logs", "bot": "Bot", "Reddit comments": "Reddit comments",
"reported": "Reported on the Atlas"}
def tally(rows, key):
d = collections.OrderedDict()
for r in rows:
@@ -110,17 +118,20 @@ def build():
"another) are still fair if the over-reporting is the same on both sides.", ""]
# 1. by attempt
L += ["## 1. The shadow by attempt number", "", "| Attempt | Form submissions | Bulk logs | Bot |", "|---|---|---|---|"]
# the self-reported sources (Reddit comments, "Report a run") get a column once they have a gated tree run
groups = ["form", "bulk logs", "bot"] + [g for g in ("Reddit comments", "reported") if any(group(r) == g for r in G)]
L += ["## 1. The shadow by attempt number", "", "| Attempt | " + " | ".join(GROUP_NAMES[g] for g in groups) + " |",
"|---" * (len(groups) + 1) + "|"]
bands = [("1", 1, 1), ("2", 2, 2), ("3", 3, 3), ("4-10", 4, 10), ("11-30", 11, 30), ("31 or more", 31, 10 ** 9)]
for name, lo, hi in bands:
cells = []
for g in ("form", "bulk logs", "bot"):
for g in groups:
rs = [r for r in G if group(r) == g and r["attempt"] and lo <= r["attempt"] <= hi]
cells.append(frac(sum(bool(r["saw_shadow"]) for r in rs), len(rs)))
facts["by_attempt"].setdefault(g, {})[name] = [sum(bool(r["saw_shadow"]) for r in rs), len(rs)]
L.append(f"| {name} | " + " | ".join(cells) + " |")
cells = []
for g in ("form", "bulk logs", "bot"):
for g in groups:
rs = [r for r in G if group(r) == g and not r["attempt"]]
cells.append(frac(sum(bool(r["saw_shadow"]) for r in rs), len(rs)))
L.append("| not given | " + " | ".join(cells) + " |")
@@ -130,6 +141,14 @@ def build():
f"{len(complete_later)} later ones, with {sum(bool(r['saw_shadow']) for r in complete_first + complete_later)} shadows. "
f"Zero in {len(complete_later)} later attempts rules out a rate above {100 * upper(len(complete_later)):.1f}% for them; "
f"zero in {len(complete_first)} first attempts only rules out more than {100 * upper(len(complete_first)):.0f}%.", ""]
rc = [r for r in runs if r["source"] == "reddit comment" and r["saw_shadow"]]
if rc:
rg = [r for r in G if group(r) == "Reddit comments" and r["saw_shadow"]]
facts["reddit_shadows"] = {"all": len(rc), "gated": len(rg), "first_attempt": sum(r["attempt"] == 1 for r in rc)}
L += [f"Reddit comments describe {len(rc)} shadows in all, but a comment rarely says how the run began: only "
f"{len(rg)} say the rider helped their squadmate and climbed the tree (the table above), and "
f"{sum(r['attempt'] == 1 for r in rc)} say it was a first attempt. Like form submissions, they are posted by "
"choice, so they say what shadow runs look like, not how often the shadow comes.", ""]
# 2. the same route, first against later
F = [r for r in G if group(r) == "form" and r["attempt"]]
+11 -9
View File
@@ -4,18 +4,20 @@ Written by `analysis/black/evidence.py` from `data/public/runs_clean.csv`. Count
## 1. The shadow by attempt number
| Attempt | Form submissions | Bulk logs | Bot |
|---|---|---|---|
| 1 | 97 / 960 (10.1%) | 0 / 29 (0.0%) | 0 / 0 |
| 2 | 0 / 299 (0.0%) | 0 / 21 (0.0%) | 0 / 4 (0.0%) |
| 3 | 0 / 196 (0.0%) | 0 / 14 (0.0%) | 0 / 5 (0.0%) |
| 4-10 | 0 / 555 (0.0%) | 0 / 102 (0.0%) | 0 / 36 (0.0%) |
| 11-30 | 0 / 657 (0.0%) | 0 / 57 (0.0%) | 0 / 137 (0.0%) |
| 31 or more | 0 / 715 (0.0%) | 0 / 0 | 0 / 333 (0.0%) |
| not given | 0 / 1582 (0.0%) | 0 / 149 (0.0%) | 0 / 0 |
| Attempt | Form submissions | Bulk logs | Bot | Reddit comments |
|---|---|---|---|---|
| 1 | 97 / 960 (10.1%) | 0 / 29 (0.0%) | 0 / 0 | 1 / 1 (100.0%) |
| 2 | 0 / 299 (0.0%) | 0 / 21 (0.0%) | 0 / 4 (0.0%) | 0 / 0 |
| 3 | 0 / 196 (0.0%) | 0 / 14 (0.0%) | 0 / 5 (0.0%) | 0 / 0 |
| 4-10 | 0 / 555 (0.0%) | 0 / 102 (0.0%) | 0 / 36 (0.0%) | 0 / 0 |
| 11-30 | 0 / 657 (0.0%) | 0 / 57 (0.0%) | 0 / 137 (0.0%) | 0 / 0 |
| 31 or more | 0 / 715 (0.0%) | 0 / 0 | 0 / 333 (0.0%) | 0 / 0 |
| not given | 0 / 1582 (0.0%) | 0 / 149 (0.0%) | 0 / 0 | 2 / 3 (66.7%) |
Complete logs (bot and bulk) have 29 first attempts at the tree and 709 later ones, with 0 shadows. Zero in 709 later attempts rules out a rate above 0.4% for them; zero in 29 first attempts only rules out more than 10%.
Reddit comments describe 17 shadows in all, but a comment rarely says how the run began: only 3 say the rider helped their squadmate and climbed the tree (the table above), and 3 say it was a first attempt. Like form submissions, they are posted by choice, so they say what shadow runs look like, not how often the shadow comes.
## 2. The same route on a first attempt and on a later one
If the shadow depended only on the choices the form records (company, first direction, first move in the fight, the dagger, the bullies' fate, the fork), a route would do as well on a later attempt as on a first. It does not.
+31
View File
@@ -78,8 +78,39 @@
0,
333
]
},
"Reddit comments": {
"1": [
1,
1
],
"2": [
0,
0
],
"3": [
0,
0
],
"4-10": [
0,
0
],
"11-30": [
0,
0
],
"31 or more": [
0,
0
]
}
},
"reddit_shadows": {
"all": 17,
"gated": 3,
"first_attempt": 3
},
"same_route": {
"routes": 326,
"first_shadows": 93,