Commit Graph
14 Commits
Author SHA1 Message Date
kathleen d7befc8e18 Merge pull request 'Review summary on Reddit scraper PRs; measure the form/Reddit overlap' (#44) from atlas/reddit-review into main
Publish the site (pages branch) / publish (push) Successful in 6s
CI / test (push) Successful in 40s
2026-11-19 16:50:48 +00:00
ekcSchianoandClaude Opus 5.5 98be27fc1f Reddit scraper PRs get a review summary; the form/Reddit overlap is measured (#23, #24)
CI / test (pull_request) Successful in 40s
- collection/etl/review_reddit.py (`python -m collection review-reddit`) and .gitea/workflows/reddit-review.yml:
  on every scraper pull request, one comment (refreshed on each push) listing the runs added, changed and gone, the
  runs that break a game rule (left out of the model and stats until corrected), other flags, runs that may also
  be in the form, and result pages already counted; plus how to reject / fix / add. forge.upsert_comment keeps it
  to one comment. collection/README.md: "Reviewing Reddit runs".
- analysis/overlap/overlap.py -> data/public/overlap.json: a Reddit run with at least 4 route answers "matches" a
  form row giving the same answers within a day; chance is the same test a few weeks off. 9 of 36 comparable runs
  match, ~7.3 by chance: about 2 of 82 look reported twice, too few to matter. Nothing is removed. Shown on the
  page's Data & methods tab; in the dataset metadata; checked in test:py; part of `rebuild`.
- Neither the form nor the register records a result-page id, so the only proven copy (comment <-> result page)
  stays the one #18 applies.
- data/public/stats.json: the statistics of #42 (impossible runs left out), which did not make it into that merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:44:52 +00:00
kathleenandClaude Opus 5.5 0ae29b5423 Publish the built site to a pages branch for Ken's server to serve (#27)
CI / test (pull_request) Successful in 39s
Every push to main builds the static site (npm run build -> dist/) and force-pushes it as the only commit of
the `pages` branch. Ken's web server serves that branch (a git-sync sidecar or a pull on a timer): nothing
on his server needs a secret, since the repository is public, and the site updates within minutes of every
merge, data PRs included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 16:36:08 +00:00
kathleenandClaude Opus 5.5 5f080e0a82 Infrastructure: stop the queued bot cycle, one source check at a time, track site/ (#27)
CI / test (pull_request) Successful in 39s
- bot-cycle.yml: no schedule. It needs a runner on the bot's machine that no one has added, so every 4 hours
  a run queued forever. The bot's data already arrives through the daily source check (it reads
  DragonKind/DragonKind) and the model goes back through push-policy. Still runnable by hand.
- update-sources.yml: a concurrency group, so two dispatches never build and push at once.
- .gitignore: drop `/site` (a Python template's mkdocs line). site/ is the Atlas page; new files there were
  silently ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 15:16:59 +00:00
ekcSchianoandClaude Opus 5.5 e10312ab02 Known answers for the bot: the guide's encounter answers and the bot's own results, by choice text (#35)
CI / test (pull_request) Successful in 2m3s
analysis/known/build_known.py writes data/public/known_answers.json from pdf_encounters.json and tree.json;
solver.js knownAnswer(known, offered, { target, sceneColour }) matches a screen's choices against it (the guide's
paraphrases by their words, the bot's texts exactly) and returns a pick when one choice is clearly on the guide's
recommended path to the target, plus the evidence for every choice to hand to Clef. Rebuilt daily, checked in CI,
and sent to the bot with the route model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 13:32:09 +00:00
kathleenandClaude Opus 5.5 66181b8489 Send a changed route PDF to the bot by pull request
CI / test (pull_request) Successful in 38s
The daily check keeps the community route PDF only in its private raw store (never published here). When that
copy differs from "Threshing Options.pdf" in the bot's private repo, `python -m collection send-pdf` opens (or
updates) a pull request there with the new PDF and the lines its text gained and lost, so Ken can update the
hand-written routes in src/pdfRoutes.js (the bot does not read the PDF itself).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 02:49:25 +00:00
kathleenandClaude Opus 5.5 3206c65d59 push-policy: install the requirements (python -m collection imports pandas)
CI / test (pull_request) Successful in 1m55s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 02:34:34 +00:00
kathleenandClaude Opus 5.5 78e069f03f Bot handoff: the daily check pulls Ken's bot repo; route model updates go to the bot by pull request
CI / test (pull_request) Successful in 39s
- update-sources: download DragonKind/DragonKind (read with ATLAS_BOT_TOKEN, or DRAGONKIND_TOKEN if set) and
  import it as the bot source. Needs DK_ACCOUNT_SALT (the same salt as earlier imports, so accounts keep their keys);
  without it the bot source is skipped as before.
- The bot load keeps one file, data/bot/trajectories-bot.jsonl: the bot's repo holds every account's full history,
  so each import is merged by attempt id (attempts already published stay; older per-import files are folded in).
  One copy of each attempt also keeps build-graph.js from counting an attempt twice.
- New workflow push-policy: when policy.json, solver.js or graph.js change on main, open (or update) a pull request
  on DragonKind/DragonKind that replaces vendor/threshing-atlas/ with them. Ken merges it.
- forge open-pr: --repo/--base for another repository, and reuse the open pull request for the same branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-19 02:20:35 +00:00
kathleenandClaude Opus 5.5 6fae8a9833 Daily update: no data pull request when the combined runs failed validation
CI / test (pull_request) Failing after 1m22s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-17 23:48:54 +00:00
kathleenandClaude Opus 5.5 47da9f1f82 Code the sheet's new answers; a failed build opens an issue instead of crashing the rebuild
The evening source check stopped (correctly) on two answers the coding rules did not know:
- first sighting "2 Greens": a new form option. Coded as Green, with a data flag that keeps the
  original answer (first_sighting_alias in core/spec/form.yaml).
- row label "Unknown" on a bulk-pasted row: added to the entry values in data/dictionary.yaml.

Workflow: when `collection update` reports the build failed, skip the rebuild (it would only fail
again) and always open the "Source check failed" issue, so new answers are reported, not lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-17 23:48:47 +00:00
kathleenandClaude Opus 5.5 bd89a8941b Daily update: one branch per run, so a leftover branch never blocks the push
CI / test (pull_request) Successful in 1m24s
Run #24 (2026-11-17) failed at the push: data/source-update-20261117 already
existed from an earlier run that day.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-17 23:37:14 +00:00
ekcSchianoandClaude Opus 5.5 93bdfcd4da One rebuild list for the refresh, the bot cycle and the daily workflows; graph.json rebuilt
CI / test (pull_request) Canceled after 0s
- python -m collection rebuild: graph, route model, experiment plan, the Black evidence, the screen
  tree, statistics, build. refresh, cycle and both update-sources workflows use it, so the morning
  update also rebuilds the reports the tests check (it rebuilt only graph, model and stats before)
- graph.json had not been rebuilt since the played-steps fix (no Node on the Mac)
- docs: the branch is pushed; README mentions rebuild

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-16 16:27:20 +00:00
ekcSchianoandClaude Fable 5.2 ab3664f369 Route model: first attempts are their own shadow roll; experiment plan and automated bot cycle
- game.yaml: `givens` (first or later attempt) and `distinct` levels that are not pooled; the shadow roll
  depends on the attempt first. Form submissions show the shadow on 9% of first attempts and 0.3% of later ones.
- Reporting bias: a roll whose bias cannot be measured can borrow another roll's (`borrow: {shadow: meadow}`),
  so form submissions inform the shadow again. An assumption, stated in planner/policy.yaml.
- policy.json: targets.<colour>.firstAttempt (this attempt, eventually). Black: about 1.4% on a first attempt,
  0.07% on a later one.
- solver.js: advisor options `attempt`, `fix`, sampling "uniform"; assign() and armFor() for the plan.
- planner/experiments.yaml + build_experiments.py -> data/public/experiments.json (progress and status per
  experiment, rebuilt from the bot's runs).
- python -m collection cycle: import bot runs, refit, re-plan, deliver the kit; .gitea/workflows/bot-cycle.yml.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014jqCC2LZ4x1WHQ1V57nTsv
2026-11-12 03:31:33 +00:00
ekcSchianoandClaude Opus 5.5 614bfcda7c Host on Gitea: forge.yaml, Gitea workflows, host-neutral links
- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
  the "Report a run" collector, the page's links (data/public/links.json, filled in by
  site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
  for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
  merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-12 02:46:17 +00:00