Files
threshing-atlas/docs/REPO_SETTINGS.md
T
ekcSchianoandClaude Opus 5.5 3b24a11f5a Collection as an ETL pipeline; runs_clean.csv from every source; Report a run
- collection/pipeline.yaml declares every source (extract kind, transform,
  privacy rules, load target, run source); `python -m collection update|build`
- extract keeps data/raw/<source>/current + previous only (git-ignored);
  data/manifest.json records the current raw checksum and change date
- transforms are pure; validate checks runs against data/dictionary.yaml
  before load; load writes byte-stable files only when they change
- data/public/runs_clean.csv: every run from every source with a new
  `source` column (reddit spreadsheet, reddit thread, reported, bot,
  reddit pdf); the sheet's own row label moves to `entry`; run_id is text
- bot importer ported to Python (same output as bot.js on the fixture);
  labels and crosswalk in collection/specs/bot.yaml, written into graph.js
- Reddit through the official API (result links to add by hand); the 12
  result pages move to collection/manual/reddit_results.yaml
- "Report a run": a GitHub issue form generated from
  collection/specs/report_form.yaml, collected as source "reported", plus a
  site tab pointing to it, the community spreadsheet and the megathread
- analysis and the route model read runs_clean.csv (sources in
  measures.yaml); stats.json and policy.json unchanged apart from the
  factor's new name
- daily workflow runs the pipeline; the raw store is kept between runs in
  the Actions cache, encrypted with RAW_ARCHIVE_KEY

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
2026-11-04 02:46:48 +00:00

3.8 KiB

Repository settings

Some settings live on GitHub, not in the code. Only the repository owner (ekcSchiano) can change them. They go together: the ruleset makes CI a gate on main, auto-merge lets a pull request merge itself once it passes, and the token lets the daily data update do that on its own.

1. Allow auto-merge (issue #4)

Settings → General → Pull Requests:

  • tick Allow auto-merge
  • tick Automatically delete head branches

2. Require CI to pass on main (issue #5)

Settings → Rules → Rulesets → New ruleset → New branch ruleset:

Field Value
Ruleset name main: CI must pass
Enforcement status Active
Bypass list leave empty
Target branches Add target → Include default branch
Restrict deletions on
Block force pushes on
Require a pull request before merging on, required approvals 0
Require status checks to pass on → Add checks → test (GitHub Actions)

test is the job in .github/workflows/ci.yml. It runs the JS and Python tests, the JS/Python parity checks, checks that data/public/graph.json is up to date, and builds the site.

From then on every change, including ours, goes through a pull request. Nothing can be pushed straight to main. To merge without waiting, open the PR and turn on auto-merge; it merges when test passes.

3. Access token for automation (issue #6)

The daily source check (.github/workflows/update-sources.yml) opens a pull request when the community data change. If it uses the built-in GITHUB_TOKEN, GitHub does not start other workflows from what it does. CI still runs on the PR, but after the merge the site would not redeploy. With a token it opens the PR, turns on auto-merge, and the merge redeploys the site.

Create the token at github.com → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new token:

Field Value
Token name threshing-atlas automation
Resource owner ekcSchiano
Expiration 90 days, with a calendar reminder to renew
Repository access Only select repositories → ekcSchiano/threshing-atlas
Repository permissions Contents: Read and write, Pull requests: Read and write (Metadata: Read-only is added automatically)

Then store it as a secret, never in a file, issue, chat or commit: repository Settings → Secrets and variables → Actions → New repository secret, name ATLAS_BOT_TOKEN.

Without the secret the workflow still works. It falls back to GITHUB_TOKEN and the PR waits for a manual merge.

If Ken's bot should also open pull requests here (for example to send new bot runs), create a second token with the same settings and store it as a secret in the bot's own repository. Separate tokens can be revoked separately, and a leak in one place does not expose the other.

Data collection secrets (optional)

Both are repository secrets (Settings → Secrets and variables → Actions → New repository secret). Without them the daily update still runs and says in its report what it skipped.

Secret What it is for How to make it
RAW_ARCHIVE_KEY Encrypts the private raw store (current and previous raw file of each source) kept in the Actions cache between runs. Any long random string, e.g. the output of openssl rand -base64 32. Keep a copy somewhere safe: without it the archive cannot be read.
REDDIT_CLIENT_ID, REDDIT_CLIENT_SECRET Reading the r/fourthwing megathreads through the official Reddit API. reddit.com/prefs/apps → "create another app…" → type script, any name, redirect uri http://localhost. The id is under the app name; the secret is labelled "secret".

The "Report a run" form adds the label run report to each report; the label must exist (Issues → Labels).