novoyuuparosk-auto-wiki/pipelines/songs/SCHEMA.md

SCHEMA — song source files

Source-file contract for the songs pipeline. Files in novoyuuparosk-wiki/ncmr-songs must follow this contract to be picked up by the auto-publisher.

The schema lives entirely in the YAML frontmatter block at the top of each .md file. The body below the frontmatter is plain markdown, rendered to MediaWiki wikitext by Pandoc — no special body conventions are enforced beyond the standard markdown grammar. There is no macro syntax and nothing to declare: a link to another song page is an ordinary relative Markdown link, resolved to a wikilink at publish time.

File layout

---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
release_date: 2026-06-18
lrc: pulse.lrc
---

# pulse under latent semantic envelopes

... body markdown; a related page is nominated as [development notes](pulse_dev.md) ...

The renderer parses the frontmatter, validates required fields, hands the body (with the first # Heading line removed) to Pandoc, then resolves .md links in the wikitext output to the wiki page titles of their targets.

Fields

Required

title (string)

The MediaWiki page name that the bot writes or updates. MediaWiki capitalises the first letter automatically; lowercase input is fine.

Example: title: pulse under latent semantic envelopes → wiki page Pulse under latent semantic envelopes at URL /wiki/Pulse_under_latent_semantic_envelopes.

The body should contain a matching # Title first-line heading for human readability when viewing the source file directly (in an editor, on Gitea's blob view, etc.). The renderer strips this first-line heading during conversion — the wiki page already supplies its own title.

wiki.publish (boolean)

Publishing gate. true means the pipeline writes this file to the wiki. false means the file is ignored.

The wiki key is a mapping rather than a flat field so future toggles (wiki.protected, wiki.summary_template, etc.) can be added without restructuring.

Files under wip/ are also skipped, regardless of wiki.publish.

Optional

album (string)

The full album name. Drives [[Category:Songs]] and [[Category:Album:<album name>]] injection at the bottom of the rendered wiki page.

Example: album: Shirakaba Express Service → category tags [[Category:Songs]] and [[Category:Album:Shirakaba Express Service]].

Pages without an album (companion documents, placeholders, songs without an assigned album) simply omit this field — neither category is injected unless categories provides one.

categories (list of strings)

Additional wiki categories to inject, beyond the album category. Each entry becomes [[Category:<value>]].

categories:
  - Song development notes
  - Work in progress

album and categories are additive — a page can have both.

release_date (ISO 8601 date)

The song's release date in YYYY-MM-DD format. Skipped if absent. Currently informational; may drive sort order on future auto-generated indexes.

lrc (string, relative path)

Pointer to a companion LRC file, relative to the .md file's location. The pipeline uploads the file to the wiki as <title>.lrc (if the content has changed) and appends a [[Media:<title>.lrc|Synced lyrics (.lrc)]] link to the page footer.

Example: in ses/pulse.md, lrc: pulse.lrc declares that ses/pulse.lrc is the companion synced-lyrics file.

If the LRC file is missing or not valid UTF-8, the upload is skipped with a warning — the page still publishes.

Body

Plain markdown. The renderer applies these transformations before and after Pandoc:

1. Pre-Pandoc: strip the first-line # Heading if present.
2. Post-Pandoc: expand columns fenced blocks into side-by-side wikitext (see below).
3. Post-Pandoc: resolve links to sibling .md files into wikilinks (see below).

Do not embed raw wikitext-specific syntax ({{Template}}, raw [[Wikilink]], etc.) in the body unless you intend the literal output. Use markdown idioms; the renderer adds the metadata-derived bits (banner, LRC link, categories, link resolution) around Pandoc's output.

Nominating a related co-publication

Link to it the ordinary way, with a relative Markdown link:

For more behind-the-scene shenanigans, see [dev gibberish](pulse_dev.md).

There is nothing to declare in the frontmatter. The link stays a working link in Gitea's blob view, in an editor preview, and anywhere else the raw file is read — and on the wiki it becomes a wikilink to the target's page:

For more behind-the-scene shenanigans, see [[pulse under latent semantic envelopes - development notes|dev gibberish]].

How the target becomes a page name. The path is resolved relative to the linking file's directory, and the target file's own title frontmatter field is used verbatim as the wikilink target — the same string the bot writes that page to. There is no second name format to keep in sync: rename a page by editing its title, and every link aimed at it follows on the next publish. MediaWiki handles the first-letter capitalisation and the space-to-underscore in URLs itself.

Written in the body	Rendered wikitext
[dev gibberish](pulse_dev.md)	[[<target's title>|dev gibberish]]
[pulse_dev.md](pulse_dev.md) (label same as path)	[[<target's title>]]
[notes](../ses/pulse_dev.md)	[[<target's title>|notes]]
[notes](pulse_dev.md#Section)	[[<target's title>#Section|notes]]
[site](https://example.com)	[https://example.com site] (untouched)

A .md link that cannot be resolved fails the run. Both cases abort with a non-zero exit before anything is published:

Condition	Result
Target file does not exist	error — file does not exist
Target exists but is not published (no wiki.publish: true, or under wip/)	error — file exists but is not published

Linking to a file does not publish it. Publication is always the target's own explicit wiki.publish: true; nothing is published as a side effect of being linked. If you want the link, opt the target in.

Scope of a run. A push publishes the files it changed, plus — one hop in each direction — any page a changed page links to, and any page that links to a changed page. The second is what keeps inbound links correct when a page's title moves. Regardless of how few pages a run publishes, the whole repository is always scanned first, so link resolution and title-collision checks see every publishable file.

Anchors (pulse_dev.md#Section) are passed through verbatim. MediaWiki derives its own section anchors from heading text, so an anchor that doesn't match a heading on the target page lands the reader at the top of it rather than at the section. Nothing validates this.

Links inside a columns fence do not work. The fence is verbatim by design (see below); Pandoc does not process Markdown in it, so a link written there would reach the wiki as literal text. The renderer detects this and fails the run rather than publishing it. Put the link in ordinary body prose.

Columns (side-by-side) shorthand

For parallel content — e.g. an original and its translation — write a fenced code block tagged columns and separate the columns with a line containing only ===:

```columns
**原題**

一行目
二行目
===
**Title**

first line
second line
```

The renderer turns this into a flexbox row of <poem> columns (one per ===-delimited section). Notes:

• Line breaks and blank lines are preserved verbatim — that's the point of using a fenced block; Pandoc passes the body through untouched, and each column is wrapped in <poem> so MediaWiki keeps the line breaks.
• N columns: use N−1 === separators. Two is the common case (original / translation).
• Emphasis in Markdown: write *italic*, **bold**, or ***bold-italic*** (asterisk style, single line) — these are converted to wikitext. Raw inline HTML such as <b>…</b> or <br> also still works (it's HTML-unescaped on the way out), but you shouldn't need it.
• This is a shared transform (lib/wiki.py), so it works for any pipeline, not just songs. Column width/gap styling currently lives in that transform.

Renderer behaviour

A run happens in three phases, in this order, and the wiki is not contacted until the third. Anything wrong in phase 1 or 2 aborts with a non-zero exit and the wiki untouched — never half-updated.

Phase 1 — scan and validate the whole repository. For every .md outside wip/ (and excluding README.md):

1. Parse the frontmatter; skip the file silently unless wiki.publish: true.
2. Validate the fields (see Validation errors below).
3. Register title and fail on a collision with any other publishable file, anywhere in the tree.
4. Build the path → page-title index every .md link resolves through.

Phase 2 — render every page that this run will publish. For each selected file:

1. Strip the leading # Heading from the body if present.
2. Pipe the body through pandoc -f markdown -t mediawiki, then expand any columns fenced blocks into side-by-side <poem> columns.
3. Resolve .md links to [[Page title|label]] wikilinks; fail on any that cannot be resolved, and on any link shape that survived rendering unresolved.
4. Prepend the auto-generated banner: {{Auto-generated|source=<source URL>|commit=<sha>}}.
5. If lrc is declared, append a [[Media:…]] link.
6. Append category tags: [[Category:Songs]] and [[Category:Album:<album>]] if album is set; [[Category:<cat>]] for each entry in categories.

Phase 3 — write. For each rendered page:

1. If lrc is declared: upload the LRC file if its SHA1 has changed.
2. Read the current wiki page content via the MediaWiki API; if identical to the generated output, skip the write (idempotency — keeps the wiki history clean).
3. If the page exists and does not start with {{Auto-generated, skip with a warning (manual page protection).
4. Otherwise, write the page with an edit summary referencing the source commit.

Implemented scope

• Frontmatter parsing and validation
• Repo-wide page-title collision detection, before any wiki write
• Pandoc-based markdown → wikitext body rendering
• First-line h1 stripping
• Banner template injection
• LRC file upload and footer link
• Relative .md link resolution ([label](notes.md) → [[wiki page title|label]]), repo-wide and batch-independent
• Album category injection ([[Category:Songs]] + [[Category:Album:<name>]]) when album is set
• Additional categories via categories list
• Render-before-write: a broken page fails the run instead of half-updating the wiki
• Idempotent writes (no-op skip when content matches)
• Manual-page protection (skip pages without auto-gen banner)
• Files without wiki.publish: true silently skipped

Not in scope (intentional)

• Multi-page aggregation: no album index pages generated; each file → its own wiki page. Album landing pages are wiki category pages, written freeform by the user.
• Language-aware rendering: the body conveys what languages exist; the renderer doesn't introspect.

Examples

A released song

---
title: pulse under latent semantic envelopes
album: Shirakaba Express Service
wiki:
  publish: true
release_date: 2026-06-18
lrc: pulse.lrc
---

Its body nominates the companion page with a plain relative link:

For more behind-the-scene shenanigans, see [dev gibberish](pulse_dev.md).

A companion document (no album)

---
title: pulse under latent semantic envelopes - development notes
wiki:
  publish: true
categories:
  - Song development notes
---

A placeholder (not published)

---
title: (placeholder)
album: kairo
wiki:
  publish: false
---

A WIP file (in wip/, skipped regardless)

---
title: 海淀
wiki:
  publish: false
---

Validation errors the renderer must produce

Files without a wiki.publish: true frontmatter field are silently skipped — no error. The pipeline fails fast and loud only on files that are opted in (wiki.publish: true) with invalid data:

• Missing or empty title on a file with wiki.publish: true
• wiki.publish present but not a boolean
• release_date (if present) does not parse as YYYY-MM-DD
• categories (if present) is not a list
• Two publishable source files declare the same title (page-name collision) — checked across the whole repository, not just the files a push touched
• A .md link whose target file does not exist
• A .md link whose target exists but is not published
• A .md link that survives rendering unresolved — a link shape the renderer does not recognise, or a Markdown link written inside a columns fence

Validation and rendering both run before any wiki write. A failure at either stage aborts the run with a non-zero exit and leaves the wiki untouched, including when only some of the run's pages are broken.

Frontmatter keys the schema does not define are ignored, so a leftover wiki.siblings block from the retired macro system is inert rather than an error. Delete them; they do nothing.