Reddit comments and the bot in the route model and the site odds; chosen deaths never count as failed bonds #40

Merged
kathleen merged 2 commits from atlas/reddit-in-model into main 2026-11-19 15:00:39 +00:00
Owner

Follows #38 (the first 82 Reddit comment runs). The goal is to use all the data that's possible and leave out only impossible runs (#21).

Route model (planner/policy.yaml, planner/build_policy.py)

  • reddit comment is now a model source: 77 of its 82 runs are used.
    • The 5 runs left out each break a game rule from #37's checks: Brown at the shadow (twice), Red at the shadow, Green after jumping on the Red, and Red in the meadow.
    • Partial runs are kept. The run whose only problem is an unusual first sighting is kept too, the same as form runs with that flag.
  • The Reddit runs get their own reporting-bias group, and they stay out of the low-bias sample, because riders posting a run in a thread choose the notable ones.
  • The odds move by at most 0.1 point. For example, Red goes from 92.4% to 92.5% and Brown from 9.6% to 9.7%. The low-bias sample is unchanged.

Site statistics (analysis/stats)

  • The statistics now use every source that records a route: the form, "Report a run", Reddit comments and the bot.

  • A deliberate death never counts as a failed bond, for any source. This is the route model's rule (burned at an encounter the rider wasn't attempting), which the summary cards already used. It now applies to:

    • encounter rates;
    • bond rates;
    • route rankings;
    • the route map's denominators.

    Biggest moves, for all runs: two Greens 8.9% → 29.1%, meadow Orange 9.2% → 25.2%, meadow Brown 14.3% → 28.3%, valley Red 79.2% → 93.3%. These come mostly from form riders who burned on purpose; the bot hardly moves the encounter rates.

  • Runs that aren't from the spreadsheet are grouped under their own source name, not as bulk logs. There are two new views: "Reddit comments only" and "DragonKind bot only".

  • site/atlas/app.js: the bulk-import share now counts only spreadsheet imports.

Black report

  • A shadow run counts as "off the gates" only when one of its answers contradicts a gate (3 runs). The 14 runs that don't say are counted separately, instead of being called "likely mis-entered".

Croissant

  • isBasedOn lists both megathreads (Part 3 and Part 4), via a new cite key in pipeline.yaml.

Checks

  • build --check, the experiment, Black, tree and known-answers checks, 29 collection tests and 21 JS tests passed on the Mac.
  • test_stats (it needs scipy) passed in a separate environment.
  • policy.json was rebuilt in the same environment that reproduces main's file byte for byte.

🤖 Generated with Claude Code

Follows #38 (the first 82 Reddit comment runs). The goal is to use all the data that's possible and leave out only impossible runs (#21). ## Route model (`planner/policy.yaml`, `planner/build_policy.py`) - `reddit comment` is now a model source: 77 of its 82 runs are used. - The 5 runs left out each break a game rule from #37's checks: Brown at the shadow (twice), Red at the shadow, Green after jumping on the Red, and Red in the meadow. - Partial runs are kept. The run whose only problem is an unusual first sighting is kept too, the same as form runs with that flag. - The Reddit runs get their own reporting-bias group, and they stay out of the low-bias sample, because riders posting a run in a thread choose the notable ones. - The odds move by at most 0.1 point. For example, Red goes from 92.4% to 92.5% and Brown from 9.6% to 9.7%. The low-bias sample is unchanged. ## Site statistics (`analysis/stats`) - The statistics now use every source that records a route: the form, "Report a run", Reddit comments and the bot. - **A deliberate death never counts as a failed bond, for any source.** This is the route model's rule (burned at an encounter the rider wasn't attempting), which the summary cards already used. It now applies to: - encounter rates; - bond rates; - route rankings; - the route map's denominators. Biggest moves, for all runs: two Greens 8.9% → 29.1%, meadow Orange 9.2% → 25.2%, meadow Brown 14.3% → 28.3%, valley Red 79.2% → 93.3%. These come mostly from form riders who burned on purpose; the bot hardly moves the encounter rates. - Runs that aren't from the spreadsheet are grouped under their own source name, not as bulk logs. There are two new views: "Reddit comments only" and "DragonKind bot only". - `site/atlas/app.js`: the bulk-import share now counts only spreadsheet imports. ## Black report - A shadow run counts as "off the gates" only when one of its answers contradicts a gate (3 runs). The 14 runs that don't say are counted separately, instead of being called "likely mis-entered". ## Croissant - `isBasedOn` lists both megathreads (Part 3 and Part 4), via a new `cite` key in `pipeline.yaml`. ## Checks - `build --check`, the experiment, Black, tree and known-answers checks, 29 collection tests and 21 JS tests passed on the Mac. - `test_stats` (it needs scipy) passed in a separate environment. - `policy.json` was rebuilt in the same environment that reproduces main's file byte for byte. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
kathleen added 2 commits 2026-11-19 14:59:35 +00:00
- planner/policy.yaml: `reddit comment` is a model source. Only runs that break a game rule (the rule_flags
  checks) are left out; partial runs are kept. Its own reporting-bias group, and out of the low-bias sample
  (riders posting a run pick the notable ones). 77 of the 82 runs are used.
- analysis/stats/measures.yaml: the site statistics use every source answered in the form's terms (form,
  "Report a run", Reddit comments).
- analysis/black/evidence.py: a shadow run is "off the gates" only when an answer contradicts a gate (3); runs
  that do not say are counted separately (14).
- croissant isBasedOn lists both megathreads (pipeline.yaml `cite`).
- Rebuilt: policy.json, stats.json, BLACK.md, black.json, croissant.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
- analysis/stats/measures.yaml: the bot is a statistics source. A deliberate death (`_deliberate`, the route
  model's rule: burned at an encounter the rider was not attempting) is left out of every bond rate, encounter
  rate and route ranking, for every source, as the model and the summary cards already did. Biggest moves (all
  runs): two Greens 8.9% -> 29.1%, meadow Orange 9.2% -> 25.2%, meadow Brown 14.3% -> 28.3%, valley Red 79.2% ->
  93.3%; mostly from form riders who burned on purpose, not from the bot.
- analysis/stats/site_stats.py: runs that are not from the spreadsheet are grouped as their own source
  ("Reddit comments", "DragonKind bot", "Reported on the Atlas"), not as bulk logs; two new presets (Reddit
  comments only, DragonKind bot only); the route map's bond denominators skip deliberate deaths.
- site/atlas/app.js: the bulk-import share counts only spreadsheet imports.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
kathleen merged commit d0141f2d9c into main 2026-11-19 15:00:39 +00:00
kathleen deleted branch atlas/reddit-in-model 2026-11-19 15:00:40 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DragonKind/threshing-atlas#40