The same attempt reported in two places (form + Reddit): link proven copies, measure the overlap #24

Closed
opened 2026-11-19 03:29:30 +00:00 by kathleen · 0 comments
Owner

This is not about the same route being taken many times. Every attempt on a route is real data, and many riders will bond the same way on the same route. Nothing is ever dropped just because it looks like another run.

It's about one attempt reported twice: a rider logs their run in the Google Form and posts it in a megathread comment, and both copies become rows. That one attempt then counts twice, and the doubling falls hardest on rare outcomes, which people are more likely to share everywhere. That inflates rare-dragon rates and makes the evidence look stronger than it is.

We can't reliably tell a double post from two riders who took the same route on the same day (there are no player IDs), so:

Do

  1. Link only proven copies. A run counts once when two rows carry the same dragonkind.com result-page id. #18 already does this for Reddit comments vs result pages. Extend it to the form's result-link answers, if the form has them, and to the register where it records a result page.
  2. Measure the overlap, don't delete. Count how many Reddit-comment runs exactly match a form row from the same day: same answers, same outcome and colour. Report it in the scraper pull request summary (#23) and on the Data & methods tab.
    • If the overlap is small, nothing more is needed.
    • If it's large, treat it as reporting bias: give the reddit comment source its own rare-outcome weighting in planner/policy.yaml reporting_bias, as the form already has. Don't remove runs.

Don't

  • Don't drop or merge rows because their routes, dates and outcomes match. Identical routes are expected.
**This is not about the same route being taken many times.** Every attempt on a route is real data, and many riders will bond the same way on the same route. Nothing is ever dropped just because it looks like another run. It's about **one attempt reported twice**: a rider logs their run in the Google Form *and* posts it in a megathread comment, and both copies become rows. That one attempt then counts twice, and the doubling falls hardest on rare outcomes, which people are more likely to share everywhere. That inflates rare-dragon rates and makes the evidence look stronger than it is. We can't reliably tell a double post from two riders who took the same route on the same day (there are no player IDs), so: ## Do 1. **Link only proven copies.** A run counts once when two rows carry the same dragonkind.com result-page id. #18 already does this for Reddit comments vs result pages. Extend it to the form's result-link answers, if the form has them, and to the register where it records a result page. 2. **Measure the overlap, don't delete.** Count how many Reddit-comment runs exactly match a form row from the same day: same answers, same outcome and colour. Report it in the scraper pull request summary (#23) and on the Data & methods tab. - If the overlap is small, nothing more is needed. - If it's large, treat it as reporting bias: give the `reddit comment` source its own rare-outcome weighting in `planner/policy.yaml` `reporting_bias`, as the form already has. Don't remove runs. ## Don't - Don't drop or merge rows because their routes, dates and outcomes match. Identical routes are expected.
kathleen added the redditdata quality labels 2026-11-19 03:29:30 +00:00
kathleen changed title from Duplicate runs across sources (sheet, Reddit comments, register, result pages) to The same attempt reported in two places (form + Reddit): link proven copies, measure the overlap 2026-11-19 03:50:15 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: DragonKind/threshing-atlas#24