site built from 44ccd7f7
This commit is contained in:
@@ -0,0 +1,164 @@
|
||||
# Data dictionary: every published file and every column of the coded runs.
|
||||
#
|
||||
# collection/etl/metadata.py builds data/croissant.json (Croissant, the MLCommons metadata format) from this
|
||||
# file, collection/pipeline.yaml and the licences. tests/core/test_dictionary.py checks that every column of the coded runs
|
||||
# is described here and that every value it takes is listed.
|
||||
#
|
||||
# Missing answers: "Unknown", "Unsure", "(not asked)" and the other values in core/spec/form.yaml missing_values carry
|
||||
# no information about the choice; every analysis treats them, and empty cells, as missing.
|
||||
version: 1
|
||||
|
||||
dataset:
|
||||
name: Threshing Atlas
|
||||
description: >-
|
||||
Community-logged runs of the Dragonkind "threshing" (the dragon-bonding game based on Fourth Wing), coded into
|
||||
one record per run: every choice, what the game rolled, and how the run ended. Also the route model built from
|
||||
them, the statistics shown on the Threshing Atlas page, and the route graph shared with a DragonKind bot.
|
||||
# the repository, the page and every file's link come from forge.yaml
|
||||
license: https://creativecommons.org/licenses/by/4.0/
|
||||
code_license: MIT
|
||||
creator: ekcSchiano
|
||||
keywords: [Dragonkind, Fourth Wing, threshing, decision tree, route planning, community data]
|
||||
citation: "ekcSchiano and contributors (2026). Threshing Atlas. {repository}"
|
||||
# Third-party sources keep their own terms (DATA_LICENSE.md); the route PDF is never redistributed.
|
||||
|
||||
files:
|
||||
runs_clean:
|
||||
path: data/public/runs_clean.csv
|
||||
format: text/csv
|
||||
description: >-
|
||||
Every run from every source, one per row, coded into the same columns (collection/etl/runs.py; the form's
|
||||
rules are core/spec/form.yaml). The `source` column says where each run came from.
|
||||
form_responses:
|
||||
path: data/public/form_responses.json
|
||||
format: application/json
|
||||
description: >-
|
||||
The community "Threshing Routes" form export as published (headers and rows), with email addresses in
|
||||
free-text answers redacted. The source of runs_clean.
|
||||
reddit_comments:
|
||||
path: data/public/reddit_comments.json
|
||||
format: application/json
|
||||
description: >-
|
||||
Runs described in the r/fourthwing megathread comments, read into the "Report a run" answers: comment id, day
|
||||
and answers only, no usernames or comment text.
|
||||
overlap:
|
||||
path: data/public/overlap.json
|
||||
format: application/json
|
||||
description: >-
|
||||
How many Reddit comment runs may also be in the community form (the same answers within a day), against what
|
||||
chance alone gives (analysis/overlap/overlap.py). Measured only: no run is removed.
|
||||
reports:
|
||||
path: data/public/reports.json
|
||||
format: application/json
|
||||
description: Runs reported through the Atlas's "Report a run" form (issues on its repository), answers only, no usernames.
|
||||
register:
|
||||
path: data/public/register.json
|
||||
format: application/json
|
||||
description: The Unofficial Dragon Register (which dragon each rider bonded), without rider names.
|
||||
stats:
|
||||
path: data/public/stats.json
|
||||
format: application/json
|
||||
description: >-
|
||||
Every number the Atlas page shows, for six filter presets (analysis/stats/site_stats.py; what is measured is
|
||||
analysis/stats/measures.yaml).
|
||||
policy:
|
||||
path: data/public/policy.json
|
||||
format: application/json
|
||||
description: >-
|
||||
The route model: the game (core/spec/game.yaml), the count tables behind every roll, and the best decision
|
||||
tree for every dragon colour under two goals, with posterior intervals (planner/build_policy.py).
|
||||
graph:
|
||||
path: data/public/graph.json
|
||||
format: application/json
|
||||
description: The route graph over every run in runs_clean.csv, counted per source (analysis/network/build-graph.js).
|
||||
|
||||
# The columns of runs_clean. type: text, integer, boolean or date. values: every value the column may take
|
||||
# (empty cells are always allowed and mean "does not apply" or "not recorded").
|
||||
columns:
|
||||
run_id: { type: text, required: true, description: "Unique id of the run, stable when rows are deleted or re-sorted: sheet-<hash of the form row's answers and timestamp>, register-<hash of the register row>, thread-<result id>, comment-<Reddit comment id>, reported-<issue number> or bot-<hash>." }
|
||||
source:
|
||||
type: text
|
||||
required: true
|
||||
description: >-
|
||||
Where the run came from. "reddit spreadsheet": the community Threshing Routes form and spreadsheet linked from
|
||||
the r/fourthwing megathreads. "reddit thread": a dragonkind.com result page posted in a megathread (bonded,
|
||||
route not recorded). "reddit comment": a run a rider described in a megathread comment, read automatically into
|
||||
the "Report a run" answers by the threshing-reddit scraper, which delivers them by pull request (hand
|
||||
corrections in collection/manual/reddit_comments.yaml). "reported": the Atlas's own "Report a run" form. "bot": an attempt logged by the
|
||||
DragonKind bot. "register": a bonded dragon entered in the community's Unofficial Dragon Register (outcome
|
||||
only, no route; the same run may also be in the spreadsheet). "reddit pdf": the community route PDF (a
|
||||
flowchart of the routes, not a log of runs, so it adds no rows today).
|
||||
values: [reddit spreadsheet, reddit thread, reddit comment, register, reported, bot, reddit pdf]
|
||||
entry:
|
||||
type: text
|
||||
description: >-
|
||||
For spreadsheet runs, the sheet's own row label: "Form" for a form submission (it has a timestamp);
|
||||
bulk-pasted logs keep the label they were pasted with. Bulk logs record every run of the players who kept
|
||||
them, so they are the least selectively reported rows. Empty for other sources.
|
||||
values: [Form, Data Import, BulkUpload3, Manual Upload, Historical import, Unknown import, Unknown]
|
||||
bot_mode:
|
||||
type: text
|
||||
description: >-
|
||||
For bot runs, how the bot chose: "auto_learn" (its own rules and the route PDF), "atlas_policy" (the Atlas's
|
||||
route model), "atlas_thompson" (the route model, sampling routes in proportion to how likely each is best), "explore" (trying untested options), "experiment" (a randomised test), "manual" (a person
|
||||
clicked). Matters for analysis: the bot follows a strategy, so its choices are not like a player's. Empty for
|
||||
other sources.
|
||||
values: [auto_learn, atlas_policy, atlas_thompson, explore, experiment, manual]
|
||||
date: { type: date, description: "Date of the run or its report (YYYY-MM-DD): the form submission, the issue, or the bot attempt. Empty for bulk-pasted rows and result pages." }
|
||||
wing: { type: text, description: The rider's wing, from the form., values: [First, Second, Third, Fourth, Multiple, Unknown] }
|
||||
section: { type: text, description: The rider's section within the wing (free text as answered). }
|
||||
squad: { type: text, description: The rider's squad., values: [First, Second, Third, "Don't know", Unknown] }
|
||||
attempt: { type: integer, description: Which attempt this was for the account (1 = first). Empty if not given. }
|
||||
attempt_group: { type: text, description: First attempt or a later one ("I've lost count" counts as later)., values: [First attempt, Later attempt, Unknown] }
|
||||
goal:
|
||||
type: text
|
||||
description: Which dragon the rider was trying to bond (older form) or why they did the run (redesigned form).
|
||||
values: [Red, Green, Orange, Brown, Blue, Black, Blue or Black, Red (tree), Any (for fun), Daily run data, "(not asked/blank)"]
|
||||
company: { type: text, description: Who entered the valley with the rider., values: [Alone, 1 Squadmate, 2 Squadmates, Unknown] }
|
||||
left_squadmates:
|
||||
type: boolean
|
||||
description: >-
|
||||
Whether a rider who entered with squadmates later left them. Only asked from 2026-10-27: earlier and undated
|
||||
squad runs are empty (unknown).
|
||||
direction: { type: text, description: First direction., values: [East, West, Center, Multiple, Unknown] }
|
||||
first_sighting: { type: text, description: The first dragon that found the rider (a roll of the game)., values: [Red, Brown, Green, Orange, Unsure, Unknown] }
|
||||
sighting_action:
|
||||
type: text
|
||||
description: How the rider reacted to the first dragon (redesigned form only).
|
||||
values: [Hold still, Respect / face it, Retreat / run, Hide, Weapon, Meet its gaze, Protect squadmate, Approach, Stand alone, Unknown, "(not asked)"]
|
||||
scream:
|
||||
type: text
|
||||
description: What the rider did about the scream, or that they died before it.
|
||||
values: [Investigate & help, Investigate & leave, Ignore the scream, DIED at first sighting, Unknown]
|
||||
resolution:
|
||||
type: text
|
||||
description: How the cadet encounter ended.
|
||||
values: [Killed the bullies yourself, Squadmate's revenge, Left bullies to the dragons, Tourniquet, Left injured cadet, Took their weapons, Left them to it, DIED in combat, DIED at first sighting, Unknown]
|
||||
combat_first:
|
||||
type: text
|
||||
description: The rider's first move in the bully fight.
|
||||
values: [Assess the situation, Even the numbers (smaller cadet), Put yourself in the middle (bigger cadet), "(not recorded)"]
|
||||
combat_second: { type: text, description: After assessing, what the rider did about the smaller cadet's dagger., values: [Engage the smaller cadet, Daggers or not (protect squadmate), "(not recorded)"] }
|
||||
post_path: { type: text, description: The fork after the bully fight (only that path has one)., values: [Forest, West, East, (no fork), Unknown] }
|
||||
hour: { type: text, description: After an hour's walk, the tallest tree or the valley centre., values: [Tree, Valley, DIED before the hour, Unknown] }
|
||||
branch:
|
||||
type: text
|
||||
description: Which bonding encounter the run reached after the hour, worked out from the dragons recorded.
|
||||
values: [Shadow (rare), "Descend quickly: Red", "Descend quickly: let the Red pass", "Descend slowly: two Greens", Valley Red, "Passed the Red: meadow"]
|
||||
enc_Red: &enc { type: text, description: The rider's answer for this dragon in the bonding grid., values: [Bonded!, Incinerated, Changed my mind, Saved by another] }
|
||||
enc_Green: *enc
|
||||
enc_Orange: *enc
|
||||
enc_Brown: *enc
|
||||
enc_Blue: *enc
|
||||
enc_Black: *enc
|
||||
outcome: { type: text, description: How the run ended., values: [Bonded, Incinerated, Died before the hour, No outcome recorded, Ambiguous] }
|
||||
outcome_color: { type: text, description: The dragon bonded, or the one that incinerated the rider (the last met)., values: [Red, Green, Orange, Brown, Blue, Black] }
|
||||
rescued: { type: boolean, description: The Greens' encounter ended with an Orange or Brown swooping in and bonding the rider. }
|
||||
saw_blue_meadow: { type: boolean, description: A Blue waited in the meadow (valley runs). }
|
||||
saw_shadow: { type: boolean, description: The shadow appeared at the tree (tree runs). }
|
||||
rare_reported: { type: text, description: The rider's own answer to "did you meet a rare dragon"., values: ["No", Blue (valley), Shadow, Unknown, "(not asked)"] }
|
||||
malek_intent: { type: text, description: Whether the rider would seek Malek (die on purpose) rather than bond a dragon they did not want., values: ["Yes", "No", Unknown, "(not asked)"] }
|
||||
tail: { type: text, description: The tail of the dragon bonded (redesigned form)., values: [Clubtail, Daggertail, Morningstartail, Scorpiontail, Swordtail] }
|
||||
data_flags: { type: text, description: Inconsistencies found while coding the row, separated by "; ". Empty when none. }
|
||||
death_note: { type: text, description: Free-text note from riders who died (email addresses removed). }
|
||||
bond_note: { type: text, description: Free-text note from riders who bonded (email addresses removed). }
|
||||
Reference in New Issue
Block a user