Add spectrogram, native-rate loading, and always-labelled axis extremes

- SpectrogramMetric: log-frequency STFT power heatmap over time, magma
  colormap, -80 dB floor. Adaptive hop caps time bins at ~4000 so long
  tracks stay responsive on redraw; N_FFT=4096 keeps low-freq resolution.
- master_core: load audio at native sample rate (librosa.load sr=None)
  instead of librosa's 22050 Hz default, so the full band up to the
  file's own nyquist (~22 kHz at 44.1 kHz) is analysed. ~2x heavier on
  44.1/48 kHz files, by design.
- metrics: shared _show_axis_extents helper forces each axis's exact
  min/max onto the tick list with compact labels (_fmt_tick), so the
  true range is always readable -- notably the spectrogram's 22 kHz top,
  which otherwise sits unlabelled between log-scale decade ticks. Applied
  to all metric renders.
- Docs: README + CLAUDE updated for the new metric, native-rate loading,
  and axis-readability behaviour.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Mikkeli Matlock
2026-06-07 00:06:50 +09:00
parent 11182472e3
commit a322f08d0c
4 changed files with 164 additions and 18 deletions
+31 -11
View File
@@ -5,8 +5,8 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
## Current implementation
### Core features
- **Audio Analysis**: Uses librosa to analyze audio files (MP3/WAV/FLAC support)
- **Pluggable Metrics**: Switchable visualizations (RMS Power, Waveform, LUFS; DR next) via a `Metric` ABC
- **Audio Analysis**: Uses librosa to analyze audio files (MP3/WAV/FLAC support) at native sample rate (no resampling)
- **Pluggable Metrics**: Switchable visualizations (RMS Power, Waveform, LUFS, Crest Factor, PSR, True Peak, Spectrogram; DR next) via a `Metric` ABC
- **Metadata Extraction**: Reads ID3 tags from MP3 files for better file identification
- **Modular GUI Architecture**: Complete PyQt5 interface with drag-and-drop and file dialog support
- **Font Management**: Comprehensive CJK-compatible font system with user-provided font support
@@ -50,19 +50,37 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
#### `metrics.py`
- Pluggable `Metric` ABC: `compute(audio_file) -> data` (heavy, worker thread)
and `render(data, file_path) -> Figure` (cheap, GUI thread)
- Current registry: `RMSPowerMetric`, `WaveformMetric`, `LUFSMetric`
(BS.1770 short-term + integrated, via pyloudnorm) — drop in new ones (DR,
spectrum) by appending an instance to `METRICS`
- Current registry:
- `RMSPowerMetric` — 10 s rolling RMS with adaptive colour scale
- `WaveformMetric` — min/max envelope, fixed ±1.1 y-range
- `LUFSMetric` — BS.1770 short-term (3 s) + integrated + LRA, via pyloudnorm
- `CrestFactorMetric` — 20·log10(peak/RMS) per 1 s window
- `PSRMetric` — sample-peak minus short-term LUFS (3 s window)
- `TruePeakMetric` — 4× oversampled dBTP via `scipy.signal.resample_poly`
- `SpectrogramMetric` — log-frequency STFT heatmap; adaptive hop caps time
bins at ~4000, `N_FFT=4096`
- Shared render helpers: `_show_axis_extents(ax)` forces each axis's exact
min/max onto the ticks (so log-axis extremes like 22 kHz are always
labelled); `_fmt_tick` keeps those labels compact
- Drop in new ones (DR, spectral balance) by appending an instance to `METRICS`
#### `master_core.py`
- Defines the `AudioFile` class: librosa loading, rolling RMS power, BPM detection
- Loads at **native sample rate** (`librosa.load(..., sr=None)`) so the full
band is preserved — analysis runs ~2× heavier on 44.1/48 kHz files than the
old 22050 Hz default, by design
- No batch / CLI mode — all analysis is driven from `main.py` via `AnalysisResultsManager`
### Current analysis features
- **Native-rate loading**: full-band analysis up to the file's own nyquist
- **RMS power analysis**: 10-second rolling window with 2-second hops
- **Adaptive colour mapping**: Automatically adjusts scale based on detected headroom
- High dynamic range: 0-0.6 scale for loud masters
- Conservative mastering: 0-0.3 scale for quiet masters
- **Loudness metrics**: LUFS (short-term + integrated + LRA), PSR, Crest Factor
- **Peak analysis**: True Peak (4× oversampled dBTP)
- **Spectral view**: log-frequency spectrogram heatmap over time
- **Readable axes**: exact min/max of every axis is always labelled, even on log scale
- **BPM detection**: Automatic tempo analysis
- **Metadata display**: Artist and title from audio tags
- **Real-time visualization**: Embedded matplotlib plots with font-aware rendering
@@ -84,8 +102,8 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
### Short-term (not urgent)
1. **Enhanced metrics** *(plug new ones into `metrics.METRICS`)*
- Dynamic range measurement (DR meter)
- Peak-to-average ratio analysis
- Frequency spectrum analysis
- Long-term average spectrum (LTAS) / tonal-balance curve
- Stereo metrics (correlation, mid/side) — needs `AudioFile` to retain stereo
2. **Interactive plot features**
- GUI-controllable plotting styles (colormap, visualization type)
@@ -137,12 +155,15 @@ A custom mastering toolkit that provides metrics to evaluate audio masterings th
### Dependencies
- librosa: Audio analysis and feature extraction
- numpy: Numerical computations
- scipy: Signal processing (true-peak polyphase oversampling)
- pyloudnorm: BS.1770 loudness (LUFS, LRA)
- matplotlib: Plotting and visualization
- mutagen: Audio metadata extraction
- PyQt5: GUI framework
### Architecture considerations
- Current code mixes analysis and visualization - consider separation
- Analysis (`metrics.compute`) and visualization (`metrics.render`) are split
across the `Metric` ABC; compute runs on a worker thread, render on the GUI
- File path handling needs improvement for cross-platform compatibility
- Error handling should be enhanced for production use
- Consider moving from PyQt5 to PyQt6 or PySide for better licensing
@@ -173,6 +194,5 @@ The only entry point is `ujm` (defined in `pyproject.toml` as
### Planned usage enhancements
1. Interactive plot manipulation and style customization
2. LUFS and advanced metric analysis
3. Audio file comparison features
4. Self-contained executable releases
2. Audio file comparison features (reference vs. comparee)
3. Self-contained executable releases
+18 -5
View File
@@ -7,15 +7,28 @@ Developed with Claude Code assistance.
### Current
- **PyQt5 GUI**: drag-and-drop or file-dialog ingest of `.mp3`, `.wav`, `.flac`
- **RMS power analysis** on a 10 s rolling window with adaptive colour scale
- **Switchable metrics** via a dropdown, all sharing one analysis cache:
- **RMS Power** — 10 s rolling window with adaptive colour scale
- **Waveform** — min/max envelope, fixed ±1.1 scale
- **LUFS** — BS.1770 short-term (3 s) + integrated + loudness range (LRA)
- **Crest Factor** — peak-to-RMS spread over time
- **PSR** — peak-to-short-term-loudness ratio ("is it still breathing?")
- **True Peak** — 4× oversampled dBTP, catches inter-sample peaks
- **Spectrogram** — log-frequency STFT power heatmap over time
- **Always-labelled axis extremes**: every plot forces its exact min/max onto
the ticks, so you can read the true range even on a log axis (e.g. the
spectrogram's 22 kHz top, which otherwise falls between decade ticks)
- **Native sample rate**: audio is loaded without resampling, so the full band
(up to the file's own nyquist, e.g. ~22 kHz for 44.1 kHz files) is analysed
- **BPM detection** via librosa
- **CJK-safe font system** with custom fonts loaded from `fonts/` (gitignored), system fallbacks, and a live font selector
- **Background analysis thread** so the UI stays responsive
- **Background analysis thread** so the UI stays responsive; metric switches
compute off the GUI thread and cache, so re-selecting a metric is instant
- **Embedded matplotlib canvas** with auto-regenerated plots on font change
### Roadmap
See [CLAUDE.md](CLAUDE.md) for the full development roadmap. Near-term:
dynamic range, plot-style controls, interactive axis controls.
dynamic range (DR meter), plot-style controls, interactive axis controls.
## Quick start
@@ -49,8 +62,8 @@ through `uv.lock`. Python 3.10+.
| --- | --- |
| `main.py` | `MainWindow` + the `ujm` entry point |
| `analysis_results_manager.py` | Background `QThread` worker, result + metric-data cache |
| `master_core.py` | `AudioFile`: librosa loading, RMS rolling window, BPM |
| `metrics.py` | Pluggable `Metric` ABC + registry (RMS Power, Waveform, ) |
| `master_core.py` | `AudioFile`: native-rate librosa loading, RMS rolling window, BPM |
| `metrics.py` | Pluggable `Metric` ABC + registry (RMS, Waveform, LUFS, Crest, PSR, True Peak, Spectrogram) |
| `audio_visualization_widget.py` | Embedded `FigureCanvasQTAgg` host |
| `font_manager.py` | Custom + system CJK font discovery, matplotlib/Qt config |
| `font_control_widget.py` | Font picker + size slider |
+4 -2
View File
@@ -27,8 +27,10 @@ class AudioFile:
else:
self.song_name = safe_title(os.path.basename(self.file_path))
# librosa.load normalises to [-1.0, 1.0]
self.y, self.sr = librosa.load(file_path)
# librosa.load normalises to [-1.0, 1.0]. sr=None preserves the file's
# native sample rate; without it librosa resamples to 22050 Hz, which would
# discard everything above ~11 kHz (the entire top octave) before analysis.
self.y, self.sr = librosa.load(file_path, sr=None)
self.y_mono = librosa.to_mono(self.y)
self.max_amplitude = np.max(np.abs(self.y_mono))
self.avg_amplitude = np.mean(np.abs(self.y_mono))
+111
View File
@@ -20,6 +20,8 @@ import numpy as np
import matplotlib.colors as mcolors
import matplotlib.cm as cm
from matplotlib.figure import Figure
from matplotlib.ticker import FuncFormatter, NullFormatter
import librosa
import pyloudnorm as pyln
from scipy import signal as scipy_signal
@@ -36,6 +38,38 @@ def _to_dbfs(linear: np.ndarray | float) -> np.ndarray | float:
return 20.0 * np.log10(np.maximum(linear, _EPS))
def _fmt_tick(v, _pos=None) -> str:
"""Compact tick label: integer for big/whole values, trimmed decimals else."""
av = abs(v)
if v == 0 or av >= 100:
return f"{v:.0f}"
if av >= 1:
return f"{v:.1f}".rstrip("0").rstrip(".")
return f"{v:.3f}".rstrip("0").rstrip(".")
def _show_axis_extents(ax) -> None:
"""Force the exact min/max of each axis onto the tick list.
Matplotlib's locators often omit the extreme values — most visibly on a log
frequency axis, where the top (e.g. 22050 Hz) falls between decade ticks and
goes unlabelled. Union the endpoints into the existing in-range ticks so you
can always read where a plot actually starts and stops.
"""
fmt = FuncFormatter(_fmt_tick)
for is_log, get_lim, set_lim, get_ticks, set_ticks, mpl_axis in (
(ax.get_xscale() == "log", ax.get_xlim, ax.set_xlim, ax.get_xticks, ax.set_xticks, ax.xaxis),
(ax.get_yscale() == "log", ax.get_ylim, ax.set_ylim, ax.get_yticks, ax.set_yticks, ax.yaxis),
):
lo, hi = get_lim()
inside = [t for t in get_ticks() if lo <= t <= hi]
mpl_axis.set_major_formatter(fmt)
if is_log:
mpl_axis.set_minor_formatter(NullFormatter()) # keep minor marks unlabelled
set_ticks(sorted(set(inside) | {lo, hi}))
set_lim(lo, hi) # set_ticks can nudge the view; restore exact limits
class Metric(ABC):
"""A pluggable analysis metric."""
@@ -93,6 +127,7 @@ class RMSPowerMetric(Metric):
ax.set_ylabel("Power")
ax.set_xlabel("Time (seconds)")
ax.set_title(safe_title(os.path.basename(file_path)))
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -137,6 +172,7 @@ class WaveformMetric(Metric):
ax.set_ylabel("Amplitude")
ax.set_xlabel("Time (seconds)")
ax.set_title(safe_title(os.path.basename(file_path)))
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -236,6 +272,7 @@ class LUFSMetric(Metric):
ax.set_title(safe_title(os.path.basename(file_path)))
ax.grid(True, alpha=0.3)
ax.legend(loc="lower right", fontsize=8)
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -301,6 +338,7 @@ class CrestFactorMetric(Metric):
ax.set_title(safe_title(os.path.basename(file_path)))
ax.grid(True, alpha=0.3)
ax.legend(loc="lower right", fontsize=8)
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -373,6 +411,7 @@ class PSRMetric(Metric):
ax.set_title(safe_title(os.path.basename(file_path)))
ax.grid(True, alpha=0.3)
ax.legend(loc="lower right", fontsize=8)
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -444,6 +483,77 @@ class TruePeakMetric(Metric):
ax.set_title(safe_title(os.path.basename(file_path)))
ax.grid(True, alpha=0.3)
ax.legend(loc="lower right", fontsize=8)
_show_axis_extents(ax)
fig.tight_layout()
return fig
class SpectrogramMetric(Metric):
"""Log-frequency STFT spectrogram: frequency power distribution over time.
Each column is the magnitude spectrum of a short window, plotted in serial
as a colour-coded heatmap. The hop is chosen adaptively so long tracks don't
produce tens of thousands of columns (which would stall the GUI redraw): for
typical song lengths the hop lands around 50 ms, coarsening gracefully on
very long files.
"""
id = "spectrogram"
display_name = "Spectrogram"
N_FFT = 4096 # ~11 Hz bins at 44.1 kHz; keeps low-freq detail now
# that sr is native (nyquist ~22 kHz, not 11 kHz)
TARGET_COLUMNS = 4000 # cap on time bins, for render speed
DB_FLOOR = -80.0 # dynamic range shown, relative to peak
F_MIN = 20.0 # log axis can't show DC; clip the low edge here
def compute(self, audio_file: AudioFile):
y = audio_file.y_mono.astype(np.float32, copy=False)
sr = audio_file.sr
# Pick a hop that keeps the column count near TARGET_COLUMNS, but never
# finer than n_fft//4 (the usual 75%-overlap floor).
min_hop = self.N_FFT // 4
hop = max(min_hop, len(y) // self.TARGET_COLUMNS)
stft = librosa.stft(y, n_fft=self.N_FFT, hop_length=hop)
mag = np.abs(stft)
s_db = librosa.amplitude_to_db(mag, ref=np.max)
freqs = librosa.fft_frequencies(sr=sr, n_fft=self.N_FFT)
times = librosa.frames_to_time(
np.arange(s_db.shape[1]), sr=sr, hop_length=hop, n_fft=self.N_FFT
)
# Drop the DC bin (0 Hz) so the log frequency axis has no non-positive coord.
return {
"freqs": freqs[1:],
"times": times,
"s_db": s_db[1:, :],
"nyquist": sr / 2.0,
}
def render(self, data, file_path, figsize=(10, 4)) -> Figure:
freqs = data["freqs"]
times = data["times"]
s_db = data["s_db"]
nyquist = data["nyquist"]
fig = Figure(figsize=figsize, facecolor="white")
ax = fig.add_subplot(111)
mesh = ax.pcolormesh(
times, freqs, s_db,
cmap="magma", vmin=self.DB_FLOOR, vmax=0.0, shading="auto",
)
fig.colorbar(mesh, ax=ax, label="Power (dB)")
ax.set_yscale("log")
ax.set_ylim(self.F_MIN, nyquist)
ax.set_xlim(times[0], times[-1])
ax.set_ylabel("Frequency (Hz)")
ax.set_xlabel("Time (seconds)")
ax.set_title(safe_title(os.path.basename(file_path)))
_show_axis_extents(ax)
fig.tight_layout()
return fig
@@ -456,6 +566,7 @@ METRICS: dict[str, Metric] = {
CrestFactorMetric(),
PSRMetric(),
TruePeakMetric(),
SpectrogramMetric(),
)
}
DEFAULT_METRIC_ID = "rms_power"