novoyuuparosk-auto-wiki/README.md

# novoyuuparosk-auto-wiki

CI/CD pipelines that auto-apply commits to https://wiki.novoyuuparosk.org from upstream content repos.

## Pipelines

| Path | Source repo | Purpose | Status |
|---|---|---|---|
| [`pipelines/songs/`](pipelines/songs/) | `novoyuuparosk-wiki/ncmr-songs` | Song lyric pages | v1 live |
| [`pipelines/ses/`](pipelines/ses/) | `novoyuuparosk-wiki/ses-light-novel` | SES light novel pages | v1 in development |
| [`pipelines/tech/`](pipelines/tech/) | `novoyuuparosk-wiki/tech-blogs` | Tech blog posts | v1 live |

Per-pipeline READMEs cover everything specific to that pipeline (source schema, renderer, runtime, decisions). This root README covers only what's cross-cutting.

## Architecture

Hybrid layout. The Gitea Actions trigger must live in the source repo (Gitea only fires workflows from `.gitea/workflows/` of the pushed-to repo); the rendering logic lives here. The source-repo workflow is self-contained but clones this repo at runtime to get the renderer.

```
<source-repo>/
  .gitea/workflows/<name>.yml       <- workflow: clones this repo, runs renderer (deps pre-baked in job image)

novoyuuparosk-auto-wiki/             <- this repo
  .gitea/workflows/<pipeline>.yml    <- reusable workflow stubs (kept for reference; not actively called; may be stale)
  pipelines/<pipeline>/              <- per-pipeline code, schema, templates
  lib/                               <- shared modules (MediaWiki client, etc.)
```

Note: `workflow_call` across private repos was abandoned — the auto-generated run token is scoped to the triggering repo only and cannot clone a private callee. The source-repo workflow clones this repo directly using `FAPAT`.

## Wiki

- Base URL: https://wiki.novoyuuparosk.org
- MediaWiki API: https://wiki.novoyuuparosk.org/api.php  *(confirmed)*

## Bot identity

MediaWiki BotPassword issued for user `Mikkeli`, bot name `giteaBot`. Login form: `Mikkeli@giteaBot`.

Both this and the previous `Dubrowski` identity hold the `bot` right, so pipeline edits stay flagged as bot edits and keep out of default Recent Changes — `mwclient`'s `page.save()` requests the bot flag and the wiki honours it.

Credentials are stored in the Gitea org-scope secret vault under `novoyuuparosk-wiki` (`WIKI_BOT_USER`, `WIKI_BOT_PASSWORD`). Not stored in this repo.

## Runner infrastructure

One `act_runner` instance serves all pipelines. Runs on a Pi 5 (Raspberry Pi OS Bookworm, `aarch64`) inside the same `docker-compose` stack that hosts the Gitea instance. Job execution is via the host Docker socket — runner is a container, jobs spawn as sibling containers.

`act_runner` build: `linux-arm64`, from the `gitea/act_runner` Docker image.

Container network mode: `host` — required so job containers can reach `localhost:3005` (Gitea) and resolve mDNS hostnames.

### Job container image

All pipelines share a single pre-built Docker image, served from the Gitea registry at `pi5-16.local:3005/mikkeli/novoyuuparosk-wiki-runner`. The `Dockerfile` is at the repo root. It bakes in system deps (git, pandoc, ca-certificates) and all pipeline Python packages so job containers start instantly with no install steps.

The image builds automatically via [`.gitea/workflows/build-image.yml`](.gitea/workflows/build-image.yml), which triggers on pushes that touch the `Dockerfile`, any pipeline `requirements.txt`, or that workflow itself. It uses kaniko (daemonless, unprivileged) to build and push two tags: an immutable `:<short-sha>` and a moving `:latest`.

A follow-up `pin` job then rewrites the `image:` pin in each `publish-*.yml` to the new `:<short-sha>` and commits it back to `master` (using `FAPAT` for contents write). The publish workflows therefore always reference an immutable tag, kept current automatically — no manual bump. The pin commit only touches `workflow_call` files, so it triggers no further runs.

## Gitea Actions setup (cross-cutting)

Secrets and variables are scoped to the `novoyuuparosk-wiki` org, inherited by all repos under it. Runs belong to the *caller* repo, so a source repo calling a reusable workflow here resolves secrets and variables from its own owner — which is why they live at org scope rather than on this repo.

**Secrets:**
- `WIKI_BOT_USER` = `Mikkeli@giteaBot`
- `WIKI_BOT_PASSWORD` = the value from *Bot identity* above
- `FAPAT` = Full-Access PAT owned by `mikkeli`, used by source-repo workflows to clone this repo at runtime. The PAT stays personal; only its storage scope moved to the org — hence the `mikkeli:` basic-auth username in the clone URLs.
- `PKGRW_PAT` = package read/write PAT owned by `mikkeli`, used by `build-image.yml` to push to the container registry

**Variables:**
- `WIKI_BASE_URL` = `https://wiki.novoyuuparosk.org`
- `WIKI_API_URL` = `https://wiki.novoyuuparosk.org/api.php`
- `URL_TO_GITEA` = Gitea instance base URL (e.g. `http://localhost:3005`). Named with `URL_TO_` prefix — Gitea blocks variable names starting with `GITEA_` or `GITHUB_`.

## Branch naming

- This repo: `automation/<pipeline-name>` for pipeline-development branches (e.g., `automation/songs`).
- Source repos: each pipeline's README defines the source-side branch convention (e.g., `autowiki/<song-slug>` in `ncmr-songs`).

## Decisions log (cross-cutting)

| Decision | Value | Date |
|---|---|---|
| Architecture | Hybrid: source-repo workflow clones this repo at runtime for the renderer | 2026-06-09 |
| Workflow pattern | Self-contained (not `workflow_call`) — cross-repo `workflow_call` blocked by token scoping on private repos | 2026-06-09 |
| Runner execution | Docker, added as a service to the existing Gitea docker-compose | 2026-06-09 |
| Runner network mode | `host` — job containers need to reach Gitea on localhost | 2026-06-09 |
| Secret/runner scope | User-level on `mikkeli` (no orgs on this instance) | 2026-06-09 |
| Ownership | This repo and all three source repos moved to the `novoyuuparosk-wiki` org. Keeping caller and callee under one owner avoids the cross-owner read gate on private reusable workflows (Settings → Actions → General → collaborative owners, Gitea 1.26+). Secrets/variables re-created at org scope; runner must serve org-owned runs | 2026-08-11 |
| Container registry | Packages are not transferable in Gitea, so images stay at `pi5-16.local:3005/mikkeli/*` for now. Registry auth still uses the `mikkeli`-owned `PKGRW_PAT` | 2026-08-11 |
| MediaWiki API path | `api.php` (classic action API) | 2026-06-09 |
| Branch naming (this repo) | `automation/<pipeline>` for pipeline-development branches | 2026-06-09 |
| Variable naming | `URL_TO_GITEA` not `GITEA_URL` — Gitea blocks `GITEA_`/`GITHUB_` prefixes | 2026-06-09 |
| Columns shorthand | Side-by-side columns authored as a ` ```columns ` fenced block, expanded post-Pandoc in shared `lib/wiki.py` (universal across pipelines). No wiki template or PHP extension — runs Pi-side before the API call | 2026-06-14 |

Per-pipeline decisions live in each pipeline's README.

## Setup checklist (cross-cutting)

Via the Gitea web UI logged in as `mikkeli` (an owner of `novoyuuparosk-wiki`):

- [x] Org-scoped secrets and variables set per *Gitea Actions setup* above
  - [x] `WIKI_BOT_USER`
  - [x] `WIKI_BOT_PASSWORD`
  - [x] `FAPAT` (Full-Access PAT — value not stored in this README; saved directly into the Gitea secret. Regenerate if lost.)
  - [x] `WIKI_BASE_URL`
  - [x] `WIKI_API_URL`
  - [x] `URL_TO_GITEA`

With Pi access:

- [x] Add `act_runner` service to the existing Gitea docker-compose
- [x] Generate a runner registration token at `/-/admin/actions/runners`, bake into the compose env, `docker compose up -d act_runner`, confirm "online" in the Gitea UI

Per-pipeline setup lives in each pipeline's README. Start with [`pipelines/songs/`](pipelines/songs/).