novoyuuparosk-wiki/novoyuuparosk-auto-wiki

Files

T

tests / pytest (push) Successful in 7s

Details

Split the public banner URL from the runner-internal clone URL

Every auto-published page, in all three pipelines, carried a source link to
`localhost:3005`. The template was never at fault — it links `{{{source}}}`
verbatim; the bad URL was baked into each page's wikitext by the publisher.

Root cause was one variable serving two incompatible purposes. `URL_TO_GITEA`
has to be an address the runner container can clone from, and since the runner
uses host networking that is `http://localhost:3005`. All three workflows then
composed `GITEA_REPO_URL` — the base of the public banner link, written into
the wiki for anyone to read — out of that same loopback value.

Adds a second org variable, `PUBLIC_URL_TO_GITEA`, for the public base
(`https://git.novoyuuparosk.org`), and points the banner at it. Cloning keeps
using `URL_TO_GITEA`. The two are now documented as deliberately separate,
because collapsing them looks like tidying up.

`lib/wiki.public_source_base()` validates the banner base rather than trusting
it: unset, relative, loopback, mDNS (`*.local`) or private-range values raise
with the variable to set. A misconfiguration that would publish an unreachable
link now stops the run instead. Note the failure mode this creates on purpose:
until `PUBLIC_URL_TO_GITEA` exists, publishing aborts rather than proceeding
with a wrong URL.

Also builds the source path with `rel.as_posix()`, so the separator in a URL
does not depend on the OS the publisher runs on — it produced backslashes when
the renderer was exercised on Windows.

Prerequisites before this can publish:

1. Add the org variable `PUBLIC_URL_TO_GITEA` = https://git.novoyuuparosk.org
2. Flush the act_runner callee cache — `docker compose up -d --force-recreate
   act_runner`. Per the cross-cutting decisions log, edits to `publish-*.yml`
   keep executing the stale cached copy, and this change is in those files.

Pages already published still carry the old URL in their stored wikitext. They
are only corrected by re-publishing each pipeline in full.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

2026-09-09 16:37:06 +09:00

11 KiB

Raw Permalink Blame History

novoyuuparosk-auto-wiki

CI/CD pipelines that auto-apply commits to https://wiki.novoyuuparosk.org from upstream content repos.

Pipelines

Path	Source repo	Purpose	Status
`pipelines/songs/`	`novoyuuparosk-wiki/ncmr-songs`	Song lyric pages	v1 live
`pipelines/ses/`	`novoyuuparosk-wiki/ses-light-novel`	SES light novel pages	v1 in development
`pipelines/tech/`	`novoyuuparosk-wiki/tech-blogs`	Tech blog posts	v1 live

Per-pipeline READMEs cover everything specific to that pipeline (source schema, renderer, runtime, decisions). This root README covers only what's cross-cutting.

Architecture

Hybrid layout. The Gitea Actions trigger must live in the source repo (Gitea only fires workflows from .gitea/workflows/ of the pushed-to repo); the rendering logic lives here. The source-repo workflow is self-contained but clones this repo at runtime to get the renderer.

<source-repo>/
  .gitea/workflows/<name>.yml       <- workflow: clones this repo, runs renderer (deps pre-baked in job image)

novoyuuparosk-auto-wiki/             <- this repo
  .gitea/workflows/<pipeline>.yml    <- reusable workflow stubs (kept for reference; not actively called; may be stale)
  pipelines/<pipeline>/              <- per-pipeline code, schema, templates
  lib/                               <- shared modules (MediaWiki client, etc.)

Note: workflow_call across private repos was abandoned — the auto-generated run token is scoped to the triggering repo only and cannot clone a private callee. The source-repo workflow clones this repo directly using FAPAT.

Wiki

Base URL: https://wiki.novoyuuparosk.org
MediaWiki API: https://wiki.novoyuuparosk.org/api.php (confirmed)

Bot identity

MediaWiki BotPassword issued for user Mikkeli, bot name giteaBot. Login form: Mikkeli@giteaBot.

Mikkeli holds the bot right, so pipeline edits are flagged as bot edits and stay out of default Recent Changes — mwclient's page.save() requests the bot flag and the wiki honours it.

This section previously named Dubrowski@giteaAutomaton, but every bot-flagged revision in the wiki's history is attributed to Mikkeli, so that had been stale for some time — the live secret never matched the doc.

Credentials are stored in the Gitea org-scope secret vault under novoyuuparosk-wiki (WIKI_BOT_USER, WIKI_BOT_PASSWORD). Not stored in this repo.

Runner infrastructure

One act_runner instance serves all pipelines. Runs on a Pi 5 (Raspberry Pi OS Bookworm, aarch64) inside the same docker-compose stack that hosts the Gitea instance. Job execution is via the host Docker socket — runner is a container, jobs spawn as sibling containers.

act_runner build: linux-arm64, from the gitea/act_runner Docker image.

Container network mode: host — required so job containers can reach localhost:3005 (Gitea) and resolve mDNS hostnames.

Job container image

All pipelines share a single pre-built Docker image, served from the Gitea registry at pi5-16.local:3005/novoyuuparosk-wiki/novoyuuparosk-wiki-runner. The Dockerfile is at the repo root. It bakes in system deps (git, pandoc, ca-certificates) and all pipeline Python packages so job containers start instantly with no install steps.

The image builds automatically via .gitea/workflows/build-image.yml, which triggers on pushes that touch the Dockerfile, any pipeline requirements.txt, or that workflow itself. It uses kaniko (daemonless, unprivileged) to build and push two tags: an immutable :<short-sha> and a moving :latest.

A follow-up pin job then rewrites the image: pin in each publish-*.yml to the new :<short-sha> and commits it back to master (using FAPAT for contents write). The publish workflows therefore always reference an immutable tag, kept current automatically — no manual bump. The pin commit only touches workflow_call files, so it triggers no further runs.

Gitea Actions setup (cross-cutting)

Secrets and variables are scoped to the novoyuuparosk-wiki org, inherited by all repos under it. Runs belong to the caller repo, so a source repo calling a reusable workflow here resolves secrets and variables from its own owner — which is why they live at org scope rather than on this repo.

Secrets:

WIKI_BOT_USER = Mikkeli@giteaBot
WIKI_BOT_PASSWORD = the value from Bot identity above
FAPAT = Full-Access PAT owned by mikkeli, used by source-repo workflows to clone this repo at runtime. The PAT stays personal; only its storage scope moved to the org — hence the mikkeli: basic-auth username in the clone URLs.
PKGRW_PAT = package read/write PAT owned by mikkeli, used by build-image.yml to push to the container registry

Variables:

WIKI_BASE_URL = https://wiki.novoyuuparosk.org
WIKI_API_URL = https://wiki.novoyuuparosk.org/api.php
URL_TO_GITEA = runner-internal Gitea base URL, used only for cloning (e.g. http://localhost:3005 — the runner uses host networking). Named with URL_TO_ prefix — Gitea blocks variable names starting with GITEA_ or GITHUB_.
PUBLIC_URL_TO_GITEA = public Gitea base URL (https://git.novoyuuparosk.org), used only for the source link in each published page's banner.

These two must stay separate. URL_TO_GITEA is a loopback address, so composing the banner from it published every page — across all three pipelines — citing localhost:3005, which resolves for nobody reading the wiki. lib/wiki.public_source_base() now rejects a loopback, mDNS, or private-range value, so that misconfiguration fails the run instead of reaching a page.

Branch naming

This repo: automation/<pipeline-name> for pipeline-development branches (e.g., automation/songs).
Source repos: each pipeline's README defines the source-side branch convention (e.g., autowiki/<song-slug> in ncmr-songs).

Decisions log (cross-cutting)

Decision	Value	Date
Architecture	Hybrid: source-repo workflow clones this repo at runtime for the renderer	2026-06-09
Workflow pattern	Self-contained (not `workflow_call`) — cross-repo `workflow_call` blocked by token scoping on private repos	2026-06-09
Runner execution	Docker, added as a service to the existing Gitea docker-compose	2026-06-09
Runner network mode	`host` — job containers need to reach Gitea on localhost	2026-06-09
Secret/runner scope	User-level on `mikkeli` (no orgs on this instance)	2026-06-09
Ownership	This repo and all three source repos moved to the `novoyuuparosk-wiki` org. Secrets/variables re-created at org scope; runner re-registered instance-level so it serves org-owned runs	2026-08-11
This repo must stay public	`act_runner` resolves a cross-repo `uses:` by cloning the callee anonymously — the job token is not applied. A private callee therefore 404s with `repository not found`, regardless of the caller sharing its owner. Shared ownership does not satisfy the read requirement. Alternative if it must be private again: Settings → Actions → General → collaborative owners (Gitea 1.26+), untested here	2026-08-11
Runner cache masks this	The resolved callee is cached at `/root/.cache/act/<owner>-<repo>@<ref>`. Changing owner changes the key, so a working pipeline can break on a clone that had been served from cache for months. Suspect the cache before suspecting permissions	2026-08-11
Callee edits need a cache flush	The cache is not refreshed per run. Edits to `publish-*.yml` on `master` — including the pin job's own commits — keep executing the stale cached copy until the key changes or the cache is cleared. Observed directly: a run used the previous namespace and tag despite `master` being current. Flush with `docker compose up -d --force-recreate act_runner`; the cache is in the container layer, while `.runner` is in the `/data` volume, so registration survives	2026-08-11
kaniko cannot rebuild kaniko-act	The org rebuild exits 0 but the resulting image has no `/tmp` — kaniko does not persist the empty dir from `mkdir -p -m 1777 /tmp`, so `wget -O /tmp/...` fails. The bootstrap note's claim that version bumps "can be built by the build-image workflow itself" does not hold as written. `kaniko-act` therefore still lives under `mikkeli`; a real bump needs a host `docker build` + push, or a Dockerfile that forces the dir to materialise (e.g. writing a file inside it)	2026-08-11
Container registry	Gitea cannot transfer packages, so images were rebuilt into `pi5-16.local:3005/novoyuuparosk-wiki/*` rather than moved. `kaniko-act` bootstrapped via the new dispatch-only `build-kaniko-act.yml`, running in the old `mikkeli` copy; `novoyuuparosk-wiki-runner` came from a normal `build-image.yml` run. Registry auth is still the `mikkeli`-owned `PKGRW_PAT`, hence the `mikkeli:` username in the auth blob	2026-08-11
Old images left in place	The `mikkeli/*` package versions are orphaned but retained — nothing references them, and deleting a container version is irreversible. Safe to purge once the org images have proven themselves	2026-08-11
MediaWiki API path	`api.php` (classic action API)	2026-06-09
Branch naming (this repo)	`automation/<pipeline>` for pipeline-development branches	2026-06-09
Variable naming	`URL_TO_GITEA` not `GITEA_URL` — Gitea blocks `GITEA_`/`GITHUB_` prefixes	2026-06-09
Clone URL ≠ banner URL	Split into `URL_TO_GITEA` (runner-internal, for cloning) and `PUBLIC_URL_TO_GITEA` (public, for the page banner). One variable served both, and since host networking forces the clone address to `localhost:3005`, every auto-published page in every pipeline cited a link no reader could open. The banner value is validated by `public_source_base()` — loopback, mDNS and private-range addresses abort the run. Pages published before this carry the bad URL in their wikitext and need a full re-publish per pipeline to clear	2026-09-09
Source URL separators	`rel.as_posix()` in the banner URL, so the path separator never depends on the OS the publisher happens to run on	2026-09-09
Columns shorthand	Side-by-side columns authored as a ```columns fenced block, expanded post-Pandoc in shared `lib/wiki.py` (universal across pipelines). No wiki template or PHP extension — runs Pi-side before the API call	2026-06-14

Per-pipeline decisions live in each pipeline's README.

Setup checklist (cross-cutting)

Via the Gitea web UI logged in as mikkeli (an owner of novoyuuparosk-wiki):

Org-scoped secrets and variables set per Gitea Actions setup above
- WIKI_BOT_USER
- WIKI_BOT_PASSWORD
- FAPAT (Full-Access PAT — value not stored in this README; saved directly into the Gitea secret. Regenerate if lost.)
- WIKI_BASE_URL
- WIKI_API_URL
- URL_TO_GITEA
- PUBLIC_URL_TO_GITEA = https://git.novoyuuparosk.org — required; the publishers abort without it

With Pi access:

Add act_runner service to the existing Gitea docker-compose
Generate a runner registration token at /-/admin/actions/runners, bake into the compose env, docker compose up -d act_runner, confirm "online" in the Gitea UI

Per-pipeline setup lives in each pipeline's README. Start with pipelines/songs/.

11 KiB Raw Permalink Blame History