Move plotting to pyqtgraph: interactive, overlay-capable render layer

Replace the fire-and-forget matplotlib pipeline (render() -> throwaway Figure ->
canvas teardown) with a three-stage architecture that supports zoom/pan, lin/log
toggling, and multi-file overlay:

  compute(audio_file) -> data        # heavy, worker thread, backend-neutral
  build_spec(data, view) -> PlotSpec # cheap, GUI thread, view-aware
  show_specs([(label, spec, color)]) # pyqtgraph, persistent PlotItem, overlay

- plotspec.py: backend-agnostic descriptors (Curve, Band, HLine, Heatmap,
  AxisSpec, PlotSpec) + ViewState (recompute-free lin/log)
- audio_visualization_widget.py: persistent pyqtgraph plot, never torn down;
  per-dataset colours for overlay; spectrogram log-freq via row resample
  (ImageItem is affine-only); ColorBarItem at a fixed cell
- Compare/overlay driven by file-list checkboxes; stable per-song colour by row
- Custom draggable reference lines (add/clear), persist across redraws
- Axis-constrained scroll zoom: Ctrl=time, Shift=value (_AxisZoomViewBox)
- RMS render no longer per-segment fill_between (was the slow path)

Fixes found in review/testing:
- FillBetweenItem needs penned child curves or it fills nothing (RMS/Waveform
  were blank); band fill verified by pixel count
- band overlay alpha was a no-op (QBrush.color() returns a copy)
- colorbar could stack across renders; now added/removed at a fixed layout cell

Deferred (per scope): stereo retention, deep perf rewrites (eager beat_track,
true-peak/crest loops, shared LUFS), per-song colour picker UI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Mikkeli Matlock
2026-06-14 00:35:10 +09:00
parent a322f08d0c
commit b400551321
9 changed files with 817 additions and 398 deletions
+59 -23
View File
@@ -14,7 +14,9 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
### Technical stack
- **Audio Processing**: librosa, numpy
- **Visualization**: matplotlib with custom colormaps and embedded Qt widgets
- **Visualization**: pyqtgraph — persistent, interactive (mouse zoom/pan, lin/log
toggle, multi-dataset overlay). matplotlib remains only for its colormaps
(consumed by pyqtgraph) and as a librosa dependency
- **GUI Framework**: PyQt5 with modular widget architecture
- **Metadata**: mutagen for audio tag reading
- **Font Support**: Custom font loading system with CJK fallback
@@ -34,8 +36,18 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
- Progress tracking and error handling
#### `audio_visualization_widget.py`
- Embedded matplotlib visualization with Qt integration
- Real-time plot updates and status display
- Persistent pyqtgraph plot — the PlotItem is reused across renders, never torn
down, so mouse zoom/pan and scale toggles survive every redraw
- `show_specs([(label, PlotSpec), ...], view)` draws one or more datasets onto
the shared axes, assigning a distinct colour per dataset for overlay/compare
- Spectrogram log-frequency is realised by resampling STFT rows onto a log grid
(`ImageItem` is affine-only and won't follow a log axis) — see `_render_heatmap`
#### `plotspec.py`
- Backend-agnostic drawing descriptors: `Curve`, `Band`, `HLine`, `Heatmap`,
`AxisSpec`, `PlotSpec`, plus the `ViewState` (recompute-free lin/log options)
- The seam that decouples metrics from the plotting library: metrics emit
*intent*, the renderer owns colour/layout/library specifics
#### `font_control_widget.py` & `font_manager.py`
- Unified font control system with clustered interface
@@ -45,11 +57,16 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
#### `plot_control_widget.py`
- Metric selector dropdown driven by the `metrics.METRICS` registry
- Houses the `Refresh Plot` button (foundation for upcoming style controls)
- Log-frequency toggle (view-state; recompute-free, currently honoured by the
spectrogram) and the `Refresh Plot` button
- Compare/overlay is *not* here — it is driven by the file-list checkboxes
#### `metrics.py`
- Pluggable `Metric` ABC: `compute(audio_file) -> data` (heavy, worker thread)
and `render(data, file_path) -> Figure` (cheap, GUI thread)
- Pluggable `Metric` ABC: `compute(audio_file) -> data` (heavy, worker thread,
backend-neutral numpy/scalars) and `build_spec(data, view) -> PlotSpec` (cheap,
GUI thread, view-aware). Metrics no longer touch the plotting library
- Compute-time vs view-time split: scale (lin/log) is a `ViewState` argument to
`build_spec`, so toggling it never recomputes
- Current registry:
- `RMSPowerMetric` — 10 s rolling RMS with adaptive colour scale
- `WaveformMetric` — min/max envelope, fixed ±1.1 y-range
@@ -58,11 +75,13 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
- `PSRMetric` — sample-peak minus short-term LUFS (3 s window)
- `TruePeakMetric` — 4× oversampled dBTP via `scipy.signal.resample_poly`
- `SpectrogramMetric` — log-frequency STFT heatmap; adaptive hop caps time
bins at ~4000, `N_FFT=4096`
- Shared render helpers: `_show_axis_extents(ax)` forces each axis's exact
min/max onto the ticks (so log-axis extremes like 22 kHz are always
labelled); `_fmt_tick` keeps those labels compact
- Drop in new ones (DR, spectral balance) by appending an instance to `METRICS`
bins at ~4000, `N_FFT=4096`. Log/linear frequency is a view toggle
- Drop in new ones (DR, spectral balance) by appending an instance to `METRICS`;
return a `PlotSpec` from `build_spec` (curves overlay automatically; heatmaps
show one dataset at a time)
- Note: the old matplotlib `_show_axis_extents` exact-endpoint tick labelling is
gone with the matplotlib render path. If wanted back, it belongs in the
renderer, applied uniformly to every metric — not per-metric
#### `master_core.py`
- Defines the `AudioFile` class: librosa loading, rolling RMS power, BPM detection
@@ -87,8 +106,20 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
### GUI features
- **File management**: Drag-and-drop and file dialog for audio selection
- **Compare/overlay**: each analysed file has a checkbox; the ticked set is
overlaid on one graph for the current metric (curve metrics overlay; the
spectrogram shows one track at a time). Highlighting a row drives the metadata
panel, independent of the overlay set
- **Interactive plot**: mouse drag-zoom, scroll-wheel zoom, pan, right-click menu
(pyqtgraph ViewBox); log/linear frequency toggle. Scroll zooms both axes;
**Ctrl+scroll** zooms time only, **Shift+scroll** zooms the value axis only
(`_AxisZoomViewBox`); scrolling over an axis also zooms just that axis
- **Custom reference lines**: "Add ref line" drops a draggable horizontal marker
on any metric (e.g. an eyeballed effective average); lines persist across
redraws/overlay changes and are cleared automatically when the metric changes
- **Font control**: Unified font selector with size control
- **Plot control**: Metric selector + refresh-plot button
- **Plot control**: Metric selector + log-frequency toggle + ref-line add/clear
+ refresh-plot button
- **Analysis display**: Real-time visualization with metadata panels
- **Modular architecture**: Self-contained widgets for easy layout management
@@ -105,10 +136,9 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
- Long-term average spectrum (LTAS) / tonal-balance curve
- Stereo metrics (correlation, mid/side) — needs `AudioFile` to retain stereo
2. **Interactive plot features**
2. **Interactive plot features** *(zoom/pan, axis-range select, lin/log done via
pyqtgraph)*
- GUI-controllable plotting styles (colormap, visualization type)
- Select axis ranges on the fly with automatic graph updates
- Zoom/pan controls for detailed analysis
- Export analysis results to CSV/JSON
3. **Advanced GUI controls**
@@ -121,11 +151,14 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
- Graphical logging text box
### Mid-to-long-term (very not urgent)
1. **Audio comparison system**
- Reference vs. comparee audio file analysis
- Side-by-side track comparison interface
1. **Audio comparison system** *(multi-file overlay done via file-list checkboxes;
each song has a stable palette colour keyed to its list row)*
- Per-song colour picker: clickable swatch in the file list (overlay already
accepts a caller-supplied colour per dataset via `show_specs`, so this is a
UI + override-map addition, not a render change)
- Reference vs. comparee designation (vs. flat overlay)
- Side-by-side track comparison interface (incl. spectrogram, which can't overlay)
- A/B testing for mastering versions
- Overlay visualization for comparative analysis
2. **Distribution & deployment**
- Self-contained executable releases
@@ -155,15 +188,18 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
### Dependencies
- librosa: Audio analysis and feature extraction
- numpy: Numerical computations
- scipy: Signal processing (true-peak polyphase oversampling)
- scipy: Signal processing (true-peak polyphase oversampling, spectrogram
log-frequency resample)
- pyloudnorm: BS.1770 loudness (LUFS, LRA)
- matplotlib: Plotting and visualization
- pyqtgraph: Interactive plotting (zoom/pan, overlay, lin/log)
- matplotlib: Colormaps only (consumed by pyqtgraph) + librosa dependency
- mutagen: Audio metadata extraction
- PyQt5: GUI framework
### Architecture considerations
- Analysis (`metrics.compute`) and visualization (`metrics.render`) are split
across the `Metric` ABC; compute runs on a worker thread, render on the GUI
- Three-stage split: `metrics.compute` (heavy, worker thread, backend-neutral
data) → `metrics.build_spec` (cheap, GUI thread, view-aware `PlotSpec`) →
`AudioVisualizationWidget.show_specs` (pyqtgraph rendering, overlay, colours)
- File path handling needs improvement for cross-platform compatibility
- Error handling should be enhanced for production use
- Consider moving from PyQt5 to PyQt6 or PySide for better licensing