# Roadmap The aim is to move from summary statistics to path finding: for each dragon colour, the decision policy that maximises the chance of bonding it, with honest uncertainty. 1. **Repo and packaging** (done): monorepo, CI, GitHub Pages, JS/Python parity tests. 2. **Graph format** (`core/`, first version done): typed scene, chance and terminal nodes; history-context keys; per-source counts (bot, form, PDF, Discord); fingerprints and short labels only, no game prose. Importers in `collection/bot` for the bot's `knowledge.json`, `runs.jsonl` and `encounters.jsonl` (done), the form (done, via the crosswalk in `graph.js`), and a PDF scene crosswalk (to do). Daily source checks for the form, the register and the PDF (done). 3. **Context learning** (`analysis/`): tests of history dependence (G-tests, BH); variable-order Markov / context tree with BIC; temporal-tiered DAG structure learning (BDeu/BIC); latent "character" model vs specific triggers vs memoryless baseline, compared on held-out likelihood; hierarchical Dirichlet back-off pooling. 4. **Solver** (first version done: `core/solver.js`, `analysis/reports/POLICY.md`; still to do: richer history models from step 3): backward induction on the augmented state (scene, history summary), i.e. an influence diagram with perfect recall. Two objectives side by side per colour: - P(target colour on this attempt); - eventual success when deliberate deaths are free: maximise p_X / (p_X + p_other bond), solved by Dinkelbach iteration. Monte Carlo credible intervals, top-k routes, required steps and bottlenecks. 5. **Valley graph UI**: every option at every point, with P(survive) and P(bond) per colour, the policy overlay for a chosen colour, and untested branches marked. 6. **Bot loop** (advisor and sync published, see `docs/BOT_INTEGRATION.md`; experiments.json to do): the atlas publishes `policy.json` and `experiments.json` (ranked by value of information); the private DragonKind bot consumes them and returns anonymised runs. ## Python first, declarative rules (November 2026) The repo is moving to Python, with the rules and the game written as declarative spec files. JavaScript stays only where it has to run: the site's drawing and interaction, and a small route advisor the site and Ken's bot share. 1. Coding rules: `core/spec/form.yaml` + `core/atlas/coding.py`, with rule tests. **Done.** 2. Route model in Python, built from `core/spec/game.yaml`. **Done** (the bot's text, `core/spec/crosswalk.yaml`, moves with the bot importer and the JS advisor in step 4). 3. **Done.** Fix the coding bugs found in the core review (letting the Red pass, "left your squadmates" before the redesigned form, missing bot text), each with a rule test. 4. **Done.** The site shows precomputed statistics with fixed filter presets (`analysis/stats/site_stats.py`, `measures.yaml`, `data/public/stats.json`); the upload button and the click-to-select route map are gone; a small JS advisor (`core/solver.js`, about 250 lines) reads the game and the Python-built model from `policy.json` and replaces the 900-line JS solver; `engine.js` is gone. 5. **Done.** `data/dictionary.yaml` describes every published file and every column of the coded runs (type, meaning, allowed values); `collection/etl/metadata.py` writes `data/croissant.json` (Croissant 1.0) from it, with a sha256 for every file. `tests/core/test_dictionary.py` fails when a column or value is undescribed or the metadata is stale. It also caught `attempt` being written as `1.0`, `2.0`: now a whole number. ## Data collection review (November 2026) Collection is an extract → transform → load pipeline declared in `collection/pipeline.yaml` (`python -m collection`). `data/public/runs_clean.csv` holds every run from every source with a `source` column (reddit spreadsheet, reddit thread, reported, bot, reddit pdf). The bot importer is Python (`collection/etl/transform/bot.py`, identical output to the old `bot.js`), and its crosswalk is `collection/specs/bot.yaml`, written into `core/graph.js` for the bot. Reddit goes through the official API; "Report a run" is a GitHub issue form generated from `collection/specs/report_form.yaml`. Only the previous raw version is kept, encrypted, in the Actions cache. Still to do: decide which non-spreadsheet sources join the statistics and the route model (`analysis/stats/measures.yaml` `sources`); the bot follows a strategy, so its runs need weighting rather than pooling.