# pipelines/songs

Publishes lyric pages from `novoyuuparosk-wiki/ncmr-songs` to `https://wiki.novoyuuparosk.org`. v1, live.

- Source-file contract (frontmatter and body conventions): [SCHEMA.md](SCHEMA.md)
- Cross-cutting setup (wiki URL, bot identity, runner, Gitea secrets/variables): [repo root README](../../README.md)

## What the renderer does

- Parses YAML frontmatter per [SCHEMA.md](SCHEMA.md); files without `wiki.publish: true` are skipped silently
- Strips the body's first-line `# Heading`, then renders markdown → wikitext with Pandoc
- Expands ` ```columns ` fenced blocks into a flex row of `<poem>` columns (shared transform in `lib/wiki.py`)
- Resolves relative Markdown links to sibling `.md` files into `[[wiki page title|label]]`, using the target's own `title` frontmatter as the page name
- Prepends the `{{Auto-generated|source=…|commit=<sha>}}` banner
- Uploads a declared `lrc` companion file when its SHA1 has changed, and appends a `[[Media:…]]` footer link
- Injects `[[Category:Songs]]` + `[[Category:Album:<name>]]` when `album` is set, plus one `[[Category:…]]` per `categories` entry. Category pages themselves are user-written and never overwritten
- Writes via the MediaWiki bot API, skipping the write when the generated text already matches the page

Deliberately not in scope: multi-language metadata (the body conveys what languages exist; the renderer doesn't introspect), and auto-generated album index pages (album category-page descriptions are written freehand).

## Failure modes

Everything below aborts the run with a non-zero exit and the wiki untouched. Validation and rendering both complete before the first write, so a broken page never leaves the wiki half-updated.

| Condition | Result |
|---|---|
| Missing/empty `title`, non-boolean `wiki.publish`, malformed `release_date`, non-list `categories` | validation error |
| Two publishable files claiming the same `title` | validation error, checked repo-wide |
| A `.md` link whose target is missing, or exists but isn't published | render error |
| A `.md` link that survives rendering unresolved (unrecognised shape, or written inside a `columns` fence) | render error |
| `GITEA_REPO_URL` resolving to a loopback, mDNS or private-range host | run aborts before any wiki contact |

Two things degrade rather than fail: a missing or non-UTF-8 `lrc` file is skipped with a warning and the page still publishes; and a page whose existing wikitext does not start with `{{Auto-generated` is skipped with a warning, to avoid overwriting hand-written content. Delete the page or add the banner to hand it to the bot.

## Source repo and branch convention

Source repo: `novoyuuparosk-wiki/ncmr-songs`, triggering on push to `master`.

**Branch convention there**: per-song short-lived branches named `autowiki/<song-slug>` (e.g. `autowiki/pulse`). Create when staging edits, merge to `master` when ready to publish, delete after one cycle.

**Paths that don't trigger a run**: `wip/**`, `README.md`, `.gitignore`, `.claude/**`. Note that `wip/**` and `README.md` are excluded by the renderer too, so they can never publish even on a run triggered by something else.

## Invocation

`novoyuuparosk-wiki/ncmr-songs/.gitea/workflows/publish.yml` is a thin caller: it triggers, then `uses:` [`publish-songs.yml`](../../.gitea/workflows/publish-songs.yml) here, which clones both repos and runs `publish.py`.

Triggers:

- `push` to `master`, path-filtered as above
- `workflow_dispatch` — manual run from the Gitea Actions UI. Passes an empty base ref, which the pipeline reads as "publish everything", so this is the way to re-render every page after a renderer or template change

Editing `publish-songs.yml` requires a runner cache flush before it takes effect; the renderer itself is cloned fresh every run. See *Callee workflow cache* in the [root README](../../README.md).

### Publish scope

**Reading is always repo-wide; writing is scoped to the push.** Every run scans every publishable file, because link resolution and title-collision detection have to see the whole tree. What gets *written* is the push diff plus one hop in each link direction: any page a changed page links to (so a link can't point at an unpublished page), and any page that links to a changed page (so an edited `title` doesn't leave stale inbound links).

| Invocation | Publishes |
|---|---|
| `--all` | every publishable file. Used on first push and `workflow_dispatch` |
| `--files <paths…>` | those files plus their one-hop link neighbours. Used for an ordinary push |

Because the banner carries the commit SHA, every page in a run's write set is rewritten on a new commit; a page pulled in only by the link closure is a no-op API read when the run repeats at the same SHA.

### Environment

| Name | Kind | Purpose |
|---|---|---|
| `WIKI_API_URL` | variable, org-scoped | MediaWiki action API endpoint |
| `WIKI_BASE_URL` | variable, org-scoped | Wiki base URL |
| `WIKI_BOT_USER` | secret, org-scoped | Bot login (`Mikkeli@giteaBot`) |
| `WIKI_BOT_PASSWORD` | secret, org-scoped | BotPassword value |
| `URL_TO_GITEA` | variable, org-scoped | Runner-internal Gitea base URL, for cloning only |
| `PUBLIC_URL_TO_GITEA` | variable, org-scoped | Public Gitea base URL, for the banner source link only |
| `FAPAT` | secret, org-scoped | Full-Access PAT for cloning at runtime |
| `SOURCE_REF` | env, set in workflow from `github.sha` | Commit SHA, used in the banner and the edit summary |
| `GITEA_REPO_URL` | env, set in workflow from `PUBLIC_URL_TO_GITEA` + repo path | Base of the banner source link. Validated by `public_source_base()` |

## Dependencies

Job container image and its build are described in the [root README](../../README.md#job-container-image). Python packages for this pipeline are in `requirements.txt` here; changing it triggers an image rebuild.

## Wiki-side prerequisites

- **`Template:Auto-generated`** must exist on the wiki. Wikitext source: [`template_auto_generated.wikitext`](template_auto_generated.wikitext) in this directory.
- The bot user must be in the `bot` group, which grants `skipcaptcha`.