Files
threshing-atlas/docs/ROADMAP.md
T
ekcSchianoandClaude Opus 5.5 63b5d40e54 Step 5: data dictionary and Croissant metadata
- data/dictionary.yaml: every published file, and every column of runs_clean.csv
  with its type, meaning and allowed values
- collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from
  it, with a sha256 per file; `npm run metadata`, run by the daily update
- tests/core/test_dictionary.py: every column and value described, types hold,
  croissant.json up to date and checksums match
- attempt is now written as a whole number (was 1.0, 2.0); no statistic changes
- the site ships both files and the Methods text links them

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 01:37:10 +00:00

3.5 KiB

Roadmap

The aim is to move from summary statistics to path finding: for each dragon colour, the decision policy that maximises the chance of bonding it, with honest uncertainty.

  1. Repo and packaging (done): monorepo, CI, GitHub Pages, JS/Python parity tests.
  2. Graph format (core/, first version done): typed scene, chance and terminal nodes; history-context keys; per-source counts (bot, form, PDF, Discord); fingerprints and short labels only, no game prose. Importers in collection/bot for the bot's knowledge.json, runs.jsonl and encounters.jsonl (done), the form (done, via the crosswalk in graph.js), and a PDF scene crosswalk (to do). Daily source checks for the form, the register and the PDF (done).
  3. Context learning (analysis/): tests of history dependence (G-tests, BH); variable-order Markov / context tree with BIC; temporal-tiered DAG structure learning (BDeu/BIC); latent "character" model vs specific triggers vs memoryless baseline, compared on held-out likelihood; hierarchical Dirichlet back-off pooling.
  4. Solver (first version done: core/solver.js, analysis/reports/POLICY.md; still to do: richer history models from step 3): backward induction on the augmented state (scene, history summary), i.e. an influence diagram with perfect recall. Two objectives side by side per colour:
    • P(target colour on this attempt);
    • eventual success when deliberate deaths are free: maximise p_X / (p_X + p_other bond), solved by Dinkelbach iteration. Monte Carlo credible intervals, top-k routes, required steps and bottlenecks.
  5. Valley graph UI: every option at every point, with P(survive) and P(bond) per colour, the policy overlay for a chosen colour, and untested branches marked.
  6. Bot loop (advisor and sync published, see docs/BOT_INTEGRATION.md; experiments.json to do): the atlas publishes policy.json and experiments.json (ranked by value of information); the private DragonKind bot consumes them and returns anonymised runs.

Python first, declarative rules (November 2026)

The repo is moving to Python, with the rules and the game written as declarative spec files. JavaScript stays only where it has to run: the site's drawing and interaction, and a small route advisor the site and Ken's bot share.

  1. Coding rules: core/spec/form.yaml + core/atlas/coding.py, with rule tests. Done.
  2. Route model in Python, built from core/spec/game.yaml. Done (the bot's text, core/spec/crosswalk.yaml, moves with the bot importer and the JS advisor in step 4).
  3. Done. Fix the coding bugs found in the core review (letting the Red pass, "left your squadmates" before the redesigned form, missing bot text), each with a rule test.
  4. Done. The site shows precomputed statistics with fixed filter presets (analysis/stats/site_stats.py, measures.yaml, data/public/stats.json); the upload button and the click-to-select route map are gone; a small JS advisor (core/solver.js, about 250 lines) reads the game and the Python-built model from policy.json and replaces the 900-line JS solver; engine.js is gone.
  5. Done. data/dictionary.yaml describes every published file and every column of the coded runs (type, meaning, allowed values); collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from it, with a sha256 for every file. tests/core/test_dictionary.py fails when a column or value is undescribed or the metadata is stale. It also caught attempt being written as 1.0, 2.0: now a whole number.