Step 5: data dictionary and Croissant metadata
- data/dictionary.yaml: every published file, and every column of runs_clean.csv with its type, meaning and allowed values - collection/publish_metadata.py writes data/croissant.json (Croissant 1.0) from it, with a sha256 per file; `npm run metadata`, run by the daily update - tests/core/test_dictionary.py: every column and value described, types hold, croissant.json up to date and checksums match - attempt is now written as a whole number (was 1.0, 2.0); no statistic changes - the site ships both files and the Methods text links them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
11b4a0398e
commit
63b5d40e54
@@ -0,0 +1,130 @@
|
||||
# Data dictionary: every published file and every column of the coded runs.
|
||||
#
|
||||
# collection/publish_metadata.py builds data/croissant.json (Croissant, the MLCommons metadata format) from this
|
||||
# file, data/sources.json and the licences. tests/core/test_dictionary.py checks that every column of the coded runs
|
||||
# is described here and that every value it takes is listed.
|
||||
#
|
||||
# Missing answers: "Unknown", "Unsure", "(not asked)" and the other values in core/spec/form.yaml missing_values carry
|
||||
# no information about the choice; every analysis treats them, and empty cells, as missing.
|
||||
version: 1
|
||||
|
||||
dataset:
|
||||
name: Threshing Atlas
|
||||
description: >-
|
||||
Community-logged runs of the Dragonkind "threshing" (the dragon-bonding game based on Fourth Wing), coded into
|
||||
one record per run: every choice, what the game rolled, and how the run ended. Also the route model built from
|
||||
them, the statistics shown on the Threshing Atlas page, and the route graph shared with a DragonKind bot.
|
||||
url: https://ekcschiano.github.io/threshing-atlas/
|
||||
repository: https://github.com/ekcSchiano/threshing-atlas
|
||||
license: https://creativecommons.org/licenses/by/4.0/
|
||||
code_license: MIT
|
||||
creator: ekcSchiano
|
||||
keywords: [Dragonkind, Fourth Wing, threshing, decision tree, route planning, community data]
|
||||
citation: "ekcSchiano and contributors (2026). Threshing Atlas. https://github.com/ekcSchiano/threshing-atlas"
|
||||
# Third-party sources keep their own terms (DATA_LICENSE.md); the route PDF is never redistributed.
|
||||
|
||||
files:
|
||||
runs_clean:
|
||||
path: data/derived/runs_clean.csv
|
||||
url: https://ekcschiano.github.io/threshing-atlas/data/runs_clean.csv
|
||||
format: text/csv
|
||||
description: One coded run per row of the community form (core/atlas/coding.py, rules in core/spec/form.yaml).
|
||||
form_responses:
|
||||
path: data/public/form_responses.json
|
||||
format: application/json
|
||||
description: >-
|
||||
The community "Threshing Routes" form export as published (headers and rows), with email addresses in
|
||||
free-text answers redacted. The source of runs_clean.
|
||||
register:
|
||||
path: data/public/register.json
|
||||
url: https://ekcschiano.github.io/threshing-atlas/data/register.json
|
||||
format: application/json
|
||||
description: The Unofficial Dragon Register (which dragon each rider bonded), without rider names.
|
||||
stats:
|
||||
path: data/public/stats.json
|
||||
url: https://ekcschiano.github.io/threshing-atlas/data/stats.json
|
||||
format: application/json
|
||||
description: >-
|
||||
Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is
|
||||
analysis/stats/measures.yaml).
|
||||
policy:
|
||||
path: data/public/policy.json
|
||||
url: https://ekcschiano.github.io/threshing-atlas/data/policy.json
|
||||
format: application/json
|
||||
description: >-
|
||||
The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision
|
||||
tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).
|
||||
graph:
|
||||
path: data/public/graph.json
|
||||
url: https://ekcschiano.github.io/threshing-atlas/data/graph.json
|
||||
format: application/json
|
||||
description: The route graph over every source (form and bot), counted per source (analysis/network/build-graph.js).
|
||||
|
||||
# The columns of runs_clean. type: text, integer, boolean or date. values: every value the column may take
|
||||
# (empty cells are always allowed and mean "does not apply" or "not recorded").
|
||||
columns:
|
||||
run_id: { type: integer, description: Row number in the form export (1 = first row). }
|
||||
source:
|
||||
type: text
|
||||
description: >-
|
||||
"Form" for a form submission (it has a timestamp). Bulk-pasted logs keep the label they were pasted with;
|
||||
they record every run of the players who kept them, so they are the least selectively reported rows.
|
||||
values: [Form, Data Import, BulkUpload3, Manual Upload, Historical import, Unknown import]
|
||||
date: { type: date, description: Date of a form submission (YYYY-MM-DD). Empty for bulk-pasted rows. }
|
||||
wing: { type: text, description: The rider's wing, from the form., values: [First, Second, Third, Fourth, Multiple, Unknown] }
|
||||
section: { type: text, description: The rider's section within the wing (free text as answered). }
|
||||
squad: { type: text, description: The rider's squad., values: [First, Second, Third, "Don't know", Unknown] }
|
||||
attempt: { type: integer, description: Which attempt this was for the account (1 = first). Empty if not given. }
|
||||
attempt_group: { type: text, description: First attempt or a later one ("I've lost count" counts as later)., values: [First attempt, Later attempt, Unknown] }
|
||||
goal:
|
||||
type: text
|
||||
description: Which dragon the rider was trying to bond (older form) or why they did the run (redesigned form).
|
||||
values: [Red, Green, Orange, Brown, Blue, Black, Blue or Black, Red (tree), Any (for fun), Daily run data, "(not asked/blank)"]
|
||||
company: { type: text, description: Who entered the valley with the rider., values: [Alone, 1 Squadmate, 2 Squadmates, Unknown] }
|
||||
left_squadmates:
|
||||
type: boolean
|
||||
description: >-
|
||||
Whether a rider who entered with squadmates later left them. Only asked from 2026-10-27: earlier and undated
|
||||
squad runs are empty (unknown).
|
||||
direction: { type: text, description: First direction., values: [East, West, Center, Multiple, Unknown] }
|
||||
first_sighting: { type: text, description: The first dragon that found the rider (a roll of the game)., values: [Red, Brown, Green, Orange, Unsure, Unknown] }
|
||||
sighting_action:
|
||||
type: text
|
||||
description: How the rider reacted to the first dragon (redesigned form only).
|
||||
values: [Hold still, Respect / face it, Retreat / run, Hide, Weapon, Meet its gaze, Protect squadmate, Approach, Stand alone, Unknown, "(not asked)"]
|
||||
scream:
|
||||
type: text
|
||||
description: What the rider did about the scream, or that they died before it.
|
||||
values: [Investigate & help, Investigate & leave, Ignore the scream, DIED at first sighting, Unknown]
|
||||
resolution:
|
||||
type: text
|
||||
description: How the cadet encounter ended.
|
||||
values: [Killed the bullies yourself, Squadmate's revenge, Left bullies to the dragons, Tourniquet, Left injured cadet, Took their weapons, Left them to it, DIED in combat, DIED at first sighting, Unknown]
|
||||
combat_first:
|
||||
type: text
|
||||
description: The rider's first move in the bully fight.
|
||||
values: [Assess the situation, Even the numbers (smaller cadet), Put yourself in the middle (bigger cadet), "(not recorded)"]
|
||||
combat_second: { type: text, description: After assessing, what the rider did about the smaller cadet's dagger., values: [Engage the smaller cadet, Daggers or not (protect squadmate), "(not recorded)"] }
|
||||
post_path: { type: text, description: The fork after the bully fight (only that path has one)., values: [Forest, West, East, (no fork), Unknown] }
|
||||
hour: { type: text, description: After an hour's walk, the tallest tree or the valley centre., values: [Tree, Valley, DIED before the hour, Unknown] }
|
||||
branch:
|
||||
type: text
|
||||
description: Which bonding encounter the run reached after the hour, worked out from the dragons recorded.
|
||||
values: [Shadow (rare), "Descend quickly: Red", "Descend quickly: let the Red pass", "Descend slowly: two Greens", Valley Red, "Passed the Red: meadow"]
|
||||
enc_Red: &enc { type: text, description: The rider's answer for this dragon in the bonding grid., values: [Bonded!, Incinerated, Changed my mind, Saved by another] }
|
||||
enc_Green: *enc
|
||||
enc_Orange: *enc
|
||||
enc_Brown: *enc
|
||||
enc_Blue: *enc
|
||||
enc_Black: *enc
|
||||
outcome: { type: text, description: How the run ended., values: [Bonded, Incinerated, Died before the hour, No outcome recorded, Ambiguous] }
|
||||
outcome_color: { type: text, description: The dragon bonded, or the one that incinerated the rider (the last met)., values: [Red, Green, Orange, Brown, Blue, Black] }
|
||||
rescued: { type: boolean, description: The Greens' encounter ended with an Orange or Brown swooping in and bonding the rider. }
|
||||
saw_blue_meadow: { type: boolean, description: A Blue waited in the meadow (valley runs). }
|
||||
saw_shadow: { type: boolean, description: The shadow appeared at the tree (tree runs). }
|
||||
rare_reported: { type: text, description: The rider's own answer to "did you meet a rare dragon"., values: ["No", Blue (valley), Shadow, Unknown, "(not asked)"] }
|
||||
malek_intent: { type: text, description: Whether the rider would seek Malek (die on purpose) rather than bond a dragon they did not want., values: ["Yes", "No", Unknown, "(not asked)"] }
|
||||
tail: { type: text, description: The tail of the dragon bonded (redesigned form)., values: [Clubtail, Daggertail, Morningstartail, Scorpiontail, Swordtail] }
|
||||
data_flags: { type: text, description: Inconsistencies found while coding the row, separated by "; ". Empty when none. }
|
||||
death_note: { type: text, description: Free-text note from riders who died (email addresses removed). }
|
||||
bond_note: { type: text, description: Free-text note from riders who bonded (email addresses removed). }
|
||||
Reference in New Issue
Block a user