The evening source check stopped (correctly) on two answers the coding rules did not know: - first sighting "2 Greens": a new form option. Coded as Green, with a data flag that keeps the original answer (first_sighting_alias in core/spec/form.yaml). - row label "Unknown" on a bulk-pasted row: added to the entry values in data/dictionary.yaml. Workflow: when `collection update` reports the build failed, skip the rebuild (it would only fail again) and always open the "Source check failed" issue, so new answers are reported, not lost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
151 lines
10 KiB
YAML
151 lines
10 KiB
YAML
# Data dictionary: every published file and every column of the coded runs.
|
|
#
|
|
# collection/etl/metadata.py builds data/croissant.json (Croissant, the MLCommons metadata format) from this
|
|
# file, collection/pipeline.yaml and the licences. tests/core/test_dictionary.py checks that every column of the coded runs
|
|
# is described here and that every value it takes is listed.
|
|
#
|
|
# Missing answers: "Unknown", "Unsure", "(not asked)" and the other values in core/spec/form.yaml missing_values carry
|
|
# no information about the choice; every analysis treats them, and empty cells, as missing.
|
|
version: 1
|
|
|
|
dataset:
|
|
name: Threshing Atlas
|
|
description: >-
|
|
Community-logged runs of the Dragonkind "threshing" (the dragon-bonding game based on Fourth Wing), coded into
|
|
one record per run: every choice, what the game rolled, and how the run ended. Also the route model built from
|
|
them, the statistics shown on the Threshing Atlas page, and the route graph shared with a DragonKind bot.
|
|
# the repository, the page and every file's link come from forge.yaml
|
|
license: https://creativecommons.org/licenses/by/4.0/
|
|
code_license: MIT
|
|
creator: ekcSchiano
|
|
keywords: [Dragonkind, Fourth Wing, threshing, decision tree, route planning, community data]
|
|
citation: "ekcSchiano and contributors (2026). Threshing Atlas. {repository}"
|
|
# Third-party sources keep their own terms (DATA_LICENSE.md); the route PDF is never redistributed.
|
|
|
|
files:
|
|
runs_clean:
|
|
path: data/public/runs_clean.csv
|
|
format: text/csv
|
|
description: >-
|
|
Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's
|
|
rules are core/spec/form.yaml). The `source` column says where each run came from.
|
|
form_responses:
|
|
path: data/public/form_responses.json
|
|
format: application/json
|
|
description: >-
|
|
The community "Threshing Routes" form export as published (headers and rows), with email addresses in
|
|
free-text answers redacted. The source of runs_clean.
|
|
reports:
|
|
path: data/public/reports.json
|
|
format: application/json
|
|
description: Runs reported through the Atlas's "Report a run" form (issues on its repository), answers only, no usernames.
|
|
register:
|
|
path: data/public/register.json
|
|
format: application/json
|
|
description: The Unofficial Dragon Register (which dragon each rider bonded), without rider names.
|
|
stats:
|
|
path: data/public/stats.json
|
|
format: application/json
|
|
description: >-
|
|
Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is
|
|
analysis/stats/measures.yaml).
|
|
policy:
|
|
path: data/public/policy.json
|
|
format: application/json
|
|
description: >-
|
|
The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision
|
|
tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).
|
|
graph:
|
|
path: data/public/graph.json
|
|
format: application/json
|
|
description: The route graph over every run in runs_clean.csv, counted per source (analysis/network/build-graph.js).
|
|
|
|
# The columns of runs_clean. type: text, integer, boolean or date. values: every value the column may take
|
|
# (empty cells are always allowed and mean "does not apply" or "not recorded").
|
|
columns:
|
|
run_id: { type: text, required: true, description: "Unique id of the run: sheet-<row of the form export>, thread-<result id>, reported-<issue number> or bot-<hash>." }
|
|
source:
|
|
type: text
|
|
required: true
|
|
description: >-
|
|
Where the run came from. "reddit spreadsheet": the community Threshing Routes form and spreadsheet linked from
|
|
the r/fourthwing megathreads. "reddit thread": a dragonkind.com result page posted in a megathread (bonded,
|
|
route not recorded). "reported": the Atlas's own "Report a run" form. "bot": an attempt logged by the
|
|
DragonKind bot. "register": a bonded dragon entered in the community's Unofficial Dragon Register (outcome
|
|
only, no route; the same run may also be in the spreadsheet). "reddit pdf": the community route PDF (a
|
|
flowchart of the routes, not a log of runs, so it adds no rows today).
|
|
values: [reddit spreadsheet, reddit thread, register, reported, bot, reddit pdf]
|
|
entry:
|
|
type: text
|
|
description: >-
|
|
For spreadsheet runs, the sheet's own row label: "Form" for a form submission (it has a timestamp);
|
|
bulk-pasted logs keep the label they were pasted with. Bulk logs record every run of the players who kept
|
|
them, so they are the least selectively reported rows. Empty for other sources.
|
|
values: [Form, Data Import, BulkUpload3, Manual Upload, Historical import, Unknown import, Unknown]
|
|
bot_mode:
|
|
type: text
|
|
description: >-
|
|
For bot runs, how the bot chose: "auto_learn" (its own rules and the route PDF), "atlas_policy" (the Atlas's
|
|
route model), "atlas_thompson" (the route model, sampling routes in proportion to how likely each is best), "explore" (trying untested options), "experiment" (a randomised test), "manual" (a person
|
|
clicked). Matters for analysis: the bot follows a strategy, so its choices are not like a player's. Empty for
|
|
other sources.
|
|
values: [auto_learn, atlas_policy, atlas_thompson, explore, experiment, manual]
|
|
date: { type: date, description: "Date of the run or its report (YYYY-MM-DD): the form submission, the issue, or the bot attempt. Empty for bulk-pasted rows and result pages." }
|
|
wing: { type: text, description: The rider's wing, from the form., values: [First, Second, Third, Fourth, Multiple, Unknown] }
|
|
section: { type: text, description: The rider's section within the wing (free text as answered). }
|
|
squad: { type: text, description: The rider's squad., values: [First, Second, Third, "Don't know", Unknown] }
|
|
attempt: { type: integer, description: Which attempt this was for the account (1 = first). Empty if not given. }
|
|
attempt_group: { type: text, description: First attempt or a later one ("I've lost count" counts as later)., values: [First attempt, Later attempt, Unknown] }
|
|
goal:
|
|
type: text
|
|
description: Which dragon the rider was trying to bond (older form) or why they did the run (redesigned form).
|
|
values: [Red, Green, Orange, Brown, Blue, Black, Blue or Black, Red (tree), Any (for fun), Daily run data, "(not asked/blank)"]
|
|
company: { type: text, description: Who entered the valley with the rider., values: [Alone, 1 Squadmate, 2 Squadmates, Unknown] }
|
|
left_squadmates:
|
|
type: boolean
|
|
description: >-
|
|
Whether a rider who entered with squadmates later left them. Only asked from 2026-10-27: earlier and undated
|
|
squad runs are empty (unknown).
|
|
direction: { type: text, description: First direction., values: [East, West, Center, Multiple, Unknown] }
|
|
first_sighting: { type: text, description: The first dragon that found the rider (a roll of the game)., values: [Red, Brown, Green, Orange, Unsure, Unknown] }
|
|
sighting_action:
|
|
type: text
|
|
description: How the rider reacted to the first dragon (redesigned form only).
|
|
values: [Hold still, Respect / face it, Retreat / run, Hide, Weapon, Meet its gaze, Protect squadmate, Approach, Stand alone, Unknown, "(not asked)"]
|
|
scream:
|
|
type: text
|
|
description: What the rider did about the scream, or that they died before it.
|
|
values: [Investigate & help, Investigate & leave, Ignore the scream, DIED at first sighting, Unknown]
|
|
resolution:
|
|
type: text
|
|
description: How the cadet encounter ended.
|
|
values: [Killed the bullies yourself, Squadmate's revenge, Left bullies to the dragons, Tourniquet, Left injured cadet, Took their weapons, Left them to it, DIED in combat, DIED at first sighting, Unknown]
|
|
combat_first:
|
|
type: text
|
|
description: The rider's first move in the bully fight.
|
|
values: [Assess the situation, Even the numbers (smaller cadet), Put yourself in the middle (bigger cadet), "(not recorded)"]
|
|
combat_second: { type: text, description: After assessing, what the rider did about the smaller cadet's dagger., values: [Engage the smaller cadet, Daggers or not (protect squadmate), "(not recorded)"] }
|
|
post_path: { type: text, description: The fork after the bully fight (only that path has one)., values: [Forest, West, East, (no fork), Unknown] }
|
|
hour: { type: text, description: After an hour's walk, the tallest tree or the valley centre., values: [Tree, Valley, DIED before the hour, Unknown] }
|
|
branch:
|
|
type: text
|
|
description: Which bonding encounter the run reached after the hour, worked out from the dragons recorded.
|
|
values: [Shadow (rare), "Descend quickly: Red", "Descend quickly: let the Red pass", "Descend slowly: two Greens", Valley Red, "Passed the Red: meadow"]
|
|
enc_Red: &enc { type: text, description: The rider's answer for this dragon in the bonding grid., values: [Bonded!, Incinerated, Changed my mind, Saved by another] }
|
|
enc_Green: *enc
|
|
enc_Orange: *enc
|
|
enc_Brown: *enc
|
|
enc_Blue: *enc
|
|
enc_Black: *enc
|
|
outcome: { type: text, description: How the run ended., values: [Bonded, Incinerated, Died before the hour, No outcome recorded, Ambiguous] }
|
|
outcome_color: { type: text, description: The dragon bonded, or the one that incinerated the rider (the last met)., values: [Red, Green, Orange, Brown, Blue, Black] }
|
|
rescued: { type: boolean, description: The Greens' encounter ended with an Orange or Brown swooping in and bonding the rider. }
|
|
saw_blue_meadow: { type: boolean, description: A Blue waited in the meadow (valley runs). }
|
|
saw_shadow: { type: boolean, description: The shadow appeared at the tree (tree runs). }
|
|
rare_reported: { type: text, description: The rider's own answer to "did you meet a rare dragon"., values: ["No", Blue (valley), Shadow, Unknown, "(not asked)"] }
|
|
malek_intent: { type: text, description: Whether the rider would seek Malek (die on purpose) rather than bond a dragon they did not want., values: ["Yes", "No", Unknown, "(not asked)"] }
|
|
tail: { type: text, description: The tail of the dragon bonded (redesigned form)., values: [Clubtail, Daggertail, Morningstartail, Scorpiontail, Swordtail] }
|
|
data_flags: { type: text, description: Inconsistencies found while coding the row, separated by "; ". Empty when none. }
|
|
death_note: { type: text, description: Free-text note from riders who died (email addresses removed). }
|
|
bond_note: { type: text, description: Free-text note from riders who bonded (email addresses removed). }
|