Host on Gitea: forge.yaml, Gitea workflows, host-neutral links

- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
  the "Report a run" collector, the page's links (data/public/links.json, filled in by
  site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
  for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
  merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-12 02:46:17 +00:00
co-authored by Claude Opus 5.5
parent 8713ccff1a
commit 614bfcda7c
20 changed files with 479 additions and 47 deletions
+5 -17
View File
@@ -64,22 +64,10 @@ def reddit(spec, env):
return json.dumps(threads, ensure_ascii=False, sort_keys=True).encode("utf-8")
def github_issues(spec, env):
"""Every issue with the label (open and closed), through the GitHub REST API."""
headers = {"Accept": "application/vnd.github+json"}
token = env.get("GITHUB_TOKEN") or env.get("GH_TOKEN")
if token:
headers["Authorization"] = f"Bearer {token}"
label = urllib.parse.quote(spec["label"])
issues, page = [], 1
while True:
url = f"https://api.github.com/repos/{spec['repo']}/issues?labels={label}&state=all&per_page=100&page={page}"
batch = json.loads(_get(url, headers=headers))
issues += [i for i in batch if "pull_request" not in i]
if len(batch) < 100:
break
page += 1
issues.sort(key=lambda i: i["number"])
def forge_issues(spec, env):
"""Every issue with the label (open and closed), from the repository's host (forge.yaml: Gitea or GitHub)."""
from . import forge
issues = forge.list_issues(spec["label"], env)
return json.dumps(issues, ensure_ascii=False, sort_keys=True).encode("utf-8")
@@ -119,7 +107,7 @@ def pack_local(spec, src):
return json.dumps(files, ensure_ascii=False, sort_keys=True).encode("utf-8")
EXTRACTORS = {"http": http, "reddit": reddit, "github_issues": github_issues, "local_dir": local_dir}
EXTRACTORS = {"http": http, "reddit": reddit, "forge_issues": forge_issues, "local_dir": local_dir}
# ---------------------------------------------------------------- the raw store