Removes the decision logs, the dated rows, the strikethrough retirements and
the setup progress checkboxes from all four READMEs. A decision log records how
the system came to be; a reader needs to know how it is. The two had drifted
far enough apart that the docs asserted things the shipped config contradicted.
Corrected, all of which were stale rather than merely verbose:
- the architecture section described the source-repo workflow as self-contained
and called the reusable workflows here "not actively called; may be stale".
They are the pipelines; the source-repo workflow is a thin caller.
- `workflow_call` was recorded as abandoned, and the instance as having no orgs.
Both pipelines run through cross-repo `workflow_call` under an org.
- tech-blogs was described as not yet initialised. It has published.
- ses and tech invocation sections claimed a self-contained workflow too.
Kept, rewritten as behaviour rather than as dated decisions, because they are
live operational constraints and not history:
- the callee workflow cache, why a `publish-*.yml` edit appears not to take,
and the flush command with its `--no-deps` and `--pull` caveats
- why this repo has to stay public
- the `URL_TO_GITEA` / `PUBLIC_URL_TO_GITEA` split and what validates it
- host network mode, and that `kaniko-act` cannot be rebuilt by build-image.yml
Also documents what the renderer, its failure modes and its publish scope
actually are, which the decision logs had been standing in for, and adds the
architecture note that renderer code is cloned fresh per run while workflow
YAML is cached — the distinction that governs whether a change takes effect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The publish workflows composed the {{Auto-generated}} banner's source link
from URL_TO_GITEA, the runner-internal clone address (http://localhost:3005),
so published pages cited a URL no reader could resolve. The banner URL now
comes from the PUBLIC_URL_TO_GITEA variable, validated by public_source_base()
in lib/wiki.py: unset, schemeless, loopback, *.local and private-range values
raise rather than reach the wiki. URL_TO_GITEA keeps its clone-address role.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QwzQyCzCCLriTgs7b9uxwY
Every auto-published page, in all three pipelines, carried a source link to
`localhost:3005`. The template was never at fault — it links `{{{source}}}`
verbatim; the bad URL was baked into each page's wikitext by the publisher.
Root cause was one variable serving two incompatible purposes. `URL_TO_GITEA`
has to be an address the runner container can clone from, and since the runner
uses host networking that is `http://localhost:3005`. All three workflows then
composed `GITEA_REPO_URL` — the base of the public banner link, written into
the wiki for anyone to read — out of that same loopback value.
Adds a second org variable, `PUBLIC_URL_TO_GITEA`, for the public base
(`https://git.novoyuuparosk.org`), and points the banner at it. Cloning keeps
using `URL_TO_GITEA`. The two are now documented as deliberately separate,
because collapsing them looks like tidying up.
`lib/wiki.public_source_base()` validates the banner base rather than trusting
it: unset, relative, loopback, mDNS (`*.local`) or private-range values raise
with the variable to set. A misconfiguration that would publish an unreachable
link now stops the run instead. Note the failure mode this creates on purpose:
until `PUBLIC_URL_TO_GITEA` exists, publishing aborts rather than proceeding
with a wrong URL.
Also builds the source path with `rel.as_posix()`, so the separator in a URL
does not depend on the OS the publisher runs on — it produced backslashes when
the renderer was exercised on Windows.
Prerequisites before this can publish:
1. Add the org variable `PUBLIC_URL_TO_GITEA` = https://git.novoyuuparosk.org
2. Flush the act_runner callee cache — `docker compose up -d --force-recreate
act_runner`. Per the cross-cutting decisions log, edits to `publish-*.yml`
keep executing the stale cached copy, and this change is in those files.
Pages already published still carry the old URL in their stored wikitext. They
are only corrected by re-publishing each pipeline in full.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Retires the wiki.siblings bill of materials in favour of ordinary relative
Markdown links, fixes the batch-scoped resolution that silently stripped
wikilinks on incremental pushes, and adds the test suite that should have
caught it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `wiki.siblings` bill of materials is gone. A related co-publication is
now nominated with an ordinary relative Markdown link — `[label](notes.md)` —
resolved at publish time against the target file's own `title` frontmatter.
No macro namespace, no per-file declaration, and the link still works in
Gitea's blob view and any editor preview.
The macro could not be made safe. Substitution was `str.replace(tag, …)` over
the whole rendered page: no word boundary, so a tag also occurring in lyrics or
a code block got rewritten; no collision check, so overlapping tags corrupted
each other in declaration order. SCHEMA.md documented the placeholder as
`[[TAG]]` while the code replaced the bare tag, so following the schema
produced `[[[[Title|label]]]]`.
Fixes a live corruption bug in the process. `sibling_map` was built only from
the files in the publish batch, and an unresolved sibling degraded to plain
text — so pushing `ses/pulse.md` alone republished it with its wikilink
silently stripped to the bare words "Dev gibberish". SCHEMA.md asserted the
workflow "always runs --all", which it never did on an ordinary push.
The repository is now always scanned in full, whatever a run publishes, so
link resolution and title-collision detection can never depend on which files
a push touched. Writes stay bounded to the push diff plus one hop in each link
direction: the page a changed page links to (so a link cannot point at an
unpublished page) and the pages linking to it (so an edited `title` does not
leave stale inbound links). Publishing is not instant; reading is cheap.
An unresolvable `.md` link is now a hard error, never graceful degradation, and
linking to a file still does not publish it — `wiki.publish: true` remains the
only gate. A leftover-link guard fails the run on any link shape the resolver
did not recognise, including a Markdown link written inside a verbatim
```columns fence, so a pandoc upgrade cannot silently publish literal text.
Also:
- render before write: every page in a run is rendered before the first is
published, so a broken page leaves the wiki untouched instead of
half-updated. `publish_one` is split into `render_page` (pure) and the write.
- pin pandoc's stdout to UTF-8. `text=True` decoded it with the ambient locale
encoding; on any non-UTF-8 locale the Japanese and Chinese lyrics died inside
subprocess's reader thread, `stdout` came back None, and the real error was
lost behind a TypeError. Latent on the Debian runner via PEP 538 only.
- title collisions are detected repo-wide, not just within a push, and before
any wiki contact.
- tests/, run by .gitea/workflows/test.yml in the publish runner image. The
pandoc output contract the resolver depends on is now pinned by a test
instead of by prose — which is how SCHEMA.md came to assert something false.
- drop an f-string with nothing to interpolate.
- quote the SOURCE_REF interpolation in the workflow instead of pasting an
input straight into shell.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The org rebuild exits 0 but produces an image without /tmp, so the
source fetch fails. Runner image still publishes to the org namespace;
only the builder stays put.
Build destination, both builder-image references and the three publish
pins now point at novoyuuparosk-wiki/*. The publish pins name a tag that
does not exist in the new namespace until this run's build job pushes it
and the pin job rewrites them, so publishes are briefly broken mid-run.
Bootstrap step for moving container images off the personal namespace.
Builds using the existing mikkeli/kaniko-act; the org copy is what this
run produces.
Correcting a wrong claim in the previous commit: sharing an owner with
the caller does not grant read access to a private callee. act_runner
clones the reusable workflow anonymously, so the repo has to be public.
Also notes the act cache key, which hid this for months, and the stale
Dubrowski bot identity.
This repo and the three source repos moved from the `mikkeli` account
into the `novoyuuparosk-wiki` org. Rewrites cover repo paths only —
the `mikkeli:` HTTP basic-auth usernames stay, since FAPAT/PKGRW_PAT
remain personally owned, as do the `3005/mikkeli/*` image paths
(Gitea cannot transfer packages).
Bot identity moves to `Mikkeli@giteaBot`; both accounts hold the bot
right, so edits stay flagged as bot edits.
Inside a columns fenced block (verbatim, so Pandoc doesn't process
Markdown), convert *italic* / **bold** / ***bold-italic*** to wikitext
emphasis. Asterisk style, single line; precedence bold-italic > bold >
italic. Raw inline HTML still works via the existing HTML-unescape path.
Docs + example updated to use ** ** rather than <b>.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Side-by-side columns shorthand: a 'columns' fenced block (columns split by a line of ===) is expanded post-Pandoc in shared lib/wiki.py into a flex div of poem columns. Universal across pipelines; no wiki template or PHP extension.
Authors write a ```columns fenced block (columns separated by a line of
===); Pandoc passes the body through verbatim as <pre class="columns">,
and a new post-Pandoc transform in lib.wiki.expand_columns expands it
into a flex <div> of <poem> columns. Runs entirely Pi-side before the
MediaWiki API write — no wiki template or PHP extension required.
The transform lives in shared lib/wiki.py (called from markdown_to_wikitext),
so it is universal across all pipelines. Stdlib only; no new deps.
Docs: SCHEMA.md author contract + songs/root decision logs.
Also ignore __pycache__/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build-image.yml now pushes a moving :latest tag alongside :<short-sha>,
and a follow-up pin job (running in :latest) rewrites the image pin in all
three publish-*.yml to the new sha and commits it back to master with
FAPAT. Publish workflows keep immutable sha pins, kept current with no
manual bump. No rebuild loop: the pin commit only touches workflow_call
files. Root README image-flow section updated to match.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dateless posts silently lost their Blog:<year> category. build_wikitext
now falls back to datetime.now().year when no date is declared, so every
published post lands in a year category. Docs corrected accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fetch uses the job's per-run auto-token (repo-scoped, nothing stored);
push moves to the PKGRW_PAT user secret, separating CI credentials
from the host's docker login token.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kaniko's scratch image has no /tmp; the build script and act's
RUNNER_TEMP both assume it exists.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
act starts job containers with entrypoint /bin/sleep, which kaniko's
scratch-based image lacks (busybox lives under /busybox). kaniko-act
adds the single missing symlink; bootstrap build is manual, documented
in the Dockerfile.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Daemonless image build inside the job container itself
(kaniko :debug). Source fetched via Gitea archive API, pushed to the
Gitea registry tagged with the short commit SHA. Triggered by changes
to Dockerfile, any pipeline requirements.txt, or this workflow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Workflows now pull pi5-16.local:3005/mikkeli/novoyuuparosk-wiki-runner:v0
instead of relying on the host dockerd image cache. Registry is the
authority; act_runner pull_image flipped to true accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Publishes mikkeli/tech-blogs to the wiki at bare titles (no prefix).
Every page gets Category:Tech blog; Category:Blog:<year> from the date
field; Category:<tag> per tag entry (no prefix).
Fields: title (required), wiki.publish (required), date (optional,
YYYY-MM-DD), tags (optional list).
Source repo tech-blogs not yet initialised — pipeline ships first.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Songs always carries a # Song title line so stripping makes sense there,
but SES body structure is freeform and may use h1 deliberately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New pipeline at pipelines/ses/ publishes mikkeli/ses-light-novel to
the SES: MediaWiki namespace. All pages get Category:SES and
Category:SES:<type> (type from frontmatter, fallback to parent dir name).
Shared functions (connect_wiki, markdown_to_wikitext, strip_first_h1,
AUTO_BANNER_PREFIX) extracted from songs/publish.py into lib/wiki.py;
songs refactored to import from there.
Also adds publish-ses.yml workflow stub and updates Dockerfile and
root README.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pages with a valid `album` field now get [[Category:Songs]] prepended
before [[Category:Album:<name>]]. Companion/sibling pages without an
album are unaffected. Updated SCHEMA.md and pipeline README accordingly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switch from python:3.12-slim + apt/pip installs to
novoyuuparosk-wiki-runner:latest. Removes the two install steps.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Credentials moved to Gitea user-scope vault — removed from README.
Architecture diagram and songs pipeline deps section updated to reflect
prebuilt novoyuuparosk-wiki-runner:latest image (no runtime dep installs).
Noted workflow stubs in .gitea/workflows/ may be stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `album` no longer required; category format changed to Category:Album:<name>
- New optional `categories` list for additional arbitrary categories
- `wiki.siblings` bill of materials: path+tag required for resolution, label optional
- Post-Pandoc [[TAG]] substitution with four-tier fallback (link+label, link, plain label, plain tag)
- Sibling map built from all published posts in the same run; unresolved siblings degrade gracefully
- SCHEMA.md fully updated; README.md deferred items moved to implemented
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All wiki pipelines share novoyuuparosk-wiki-runner:latest — a
locally-built image with git, pandoc, ca-certificates, and Python
packages pre-installed. Eliminates the apt-get and pip install
steps from every job run. Rebuild manually on the Pi when deps change.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For songs with an `lrc` field in frontmatter:
- Verifies the file exists and is valid UTF-8 (warns and skips otherwise)
- Uploads to wiki as `File:<title>.lrc` via MediaWiki file API
- Idempotent: skips upload if SHA1 matches existing wiki file
- Appends `[[Media:<title>.lrc|Synced lyrics (.lrc)]]` before the
category tag at the bottom of the wiki page
Songs without `lrc` in frontmatter are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
render.py was a misleading name — the script validates, renders,
and writes to MediaWiki (and will soon upload LRC files too).
publish.py better describes the full scope of the entry point.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If a wiki page exists but was written manually (no Auto-generated
banner), the bot now skips it instead of attempting an overwrite
that triggers MediaWiki CAPTCHA. Clear stderr warning tells user
to delete the page or add the banner to hand ownership to the bot.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Files with no frontmatter or wiki.publish not set should be silently
ignored rather than failing the whole pipeline run.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Timestamps made the idempotency check always false — every run would
rewrite every page. Commit SHA is stable: same source commit produces
identical wikitext, so unchanged pages are correctly skipped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Gitea blocks vars/secrets prefixed with GITEA_ or GITHUB_.
Also avoids relying on github.server_url which resolves to
github.com rather than the local Gitea instance on this runner.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
wan
mikkeli
wiki-runner image bot