Host on Gitea: forge.yaml, Gitea workflows, host-neutral links

- forge.yaml: the repository's host (Gitea at git.kenschiano.net, limited visibility);
  the "Report a run" collector, the page's links (data/public/links.json, filled in by
  site/build.js) and the dataset metadata read it
- collection/etl/forge.py: issues, pull requests (merge when checks succeed) and labels
  for Gitea and GitHub; `python -m collection forge ...`
- .gitea/workflows: CI and the daily update (pushes a branch, opens a pull request that
  merges itself, opens an issue when a source breaks); .github/workflows kept for GitHub
- docs: Gitea setup (REPO_SETTINGS), the bot reading atlas files from the repository

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UCzTd31GphpMXDhQqWrCMQ
This commit is contained in:
ekcSchiano
2026-11-12 02:46:17 +00:00
co-authored by Claude Opus 5.5
parent 8713ccff1a
commit 614bfcda7c
20 changed files with 479 additions and 47 deletions
+7 -5
View File
@@ -7,6 +7,7 @@ Written by `python -m collection build` (a test checks it is up to date).
"""
import json
from . import forge
from .common import ROOT, pipeline, read_yaml, sha256_file
OUT = "data/croissant.json"
@@ -67,6 +68,7 @@ def field_description(spec):
def build(dictionary=None):
d = dictionary or load_dictionary()
ds = d["dataset"]
fg = forge.config()
sources = pipeline()["sources"]
distribution = []
@@ -77,7 +79,7 @@ def build(dictionary=None):
"@id": key,
"name": key,
"description": " ".join(spec["description"].split()),
"contentUrl": spec.get("url") or f"{ds['repository']}/blob/main/{spec['path']}",
"contentUrl": forge.file_url(spec["path"], fg),
"encodingFormat": spec["format"],
"sha256": sha256(path),
"contentSize": f"{path.stat().st_size} B",
@@ -102,12 +104,12 @@ def build(dictionary=None):
"conformsTo": "http://mlcommons.org/croissant/1.0",
"name": ds["name"],
"description": " ".join(ds["description"].split()),
"url": ds["url"],
"sameAs": ds["repository"],
"url": fg.get("site") or forge.repo_url(fg),
"sameAs": forge.repo_url(fg),
"license": ds["license"],
"creator": {"@type": "sc:Person", "name": ds["creator"], "url": f"https://github.com/{ds['creator']}"},
"creator": {"@type": "sc:Person", "name": ds["creator"]},
"keywords": ds["keywords"],
"citeAs": ds["citation"],
"citeAs": ds["citation"].format(repository=forge.repo_url(fg)),
"isLiveDataset": True,
"isBasedOn": [
{"@type": "sc:CreativeWork", "name": s["label"], "url": next(iter(s["links"].values()))}