https://github.com/lithdew/log-mel-spectrogram
My notes for computing log-Mel spectrograms on CPUs 129x faster.
https://github.com/lithdew/log-mel-spectrogram
Last synced: 11 months ago
JSON representation
My notes for computing log-Mel spectrograms on CPUs 129x faster.
- Host: GitHub
- URL: https://github.com/lithdew/log-mel-spectrogram
- Owner: lithdew
- Created: 2025-08-09T20:17:23.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-08-10T00:09:22.000Z (11 months ago)
- Last Synced: 2025-08-10T00:19:36.767Z (11 months ago)
- Language: C
- Homepage:
- Size: 1.44 MB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## log-mel-spectrogram
My notes for computing log-Mel spectrograms on CPUs ~533x faster. All rights reserved.
- **FFT**: 400-point, hop=160 at 16 kHz (25 ms window, 10 ms hop)
- **Mel**: 128 bins, librosa mel filterbank
- **Scale**: log10, clamp to max−8 dB, then (v+4)/4 normalization
### Accuracy
- **≤ 5e-5 max abs error vs OpenAI/librosa reference** on typical inputs
- Mean abs error ~1e-7; >99.9% of values within 1e-5
- Verified by [index_test.ts](index_test.ts) against [mel_example.json](mel_example.json) generated by [dump_mel_spec.py](dump_mel_spec.py) (PyTorch + librosa)
### Why it’s fast
- **Real‑FFT (RFFT) for N=400**: two complex 200‑pt FFTs (8x25) on even/odd, then a single combine for k=0..200
- **Manual unrolling** of small codelets (radix‑2 butterflies and radix‑5 DFTs)
- **OpenMP** across frames, plus SIMD pragmas on inner loops with aligned buffers
- **Thread-local scratch buffers** to avoid contention and allocations
- **Sparse mel dot-products** via precomputed CSR-style tables ([mel_mapping_tables.h](mel_mapping_tables.h))
- **Fused clamp + normalize** in a single pass over the output
- **Zero-copy FFI** from Bun to C, with optional output reuse to eliminate allocations
See [mel.c](mel.c) for the full implementation and comments. Generation of the mel filterbank and reference outputs: [dump_mel_filterbank.py](dump_mel_filterbank.py), [dump_mel_spec.py](dump_mel_spec.py).
### Research
Initial implementation ran in ~891 ms for the benchmark below.
#### Naive implementation (conceptually)
- Frame with 25 ms windows, 10 ms hop; Hann per frame
- For each frame:
- Allocate temporaries (windowed, spectrum, mel row)
- Compute DFT with fresh sin/cos per bin (or call a general FFT), keep full complex spectrum
- Convert to magnitudes via sqrt, then square again for power
- Dense multiply against a 128x201 mel matrix
- Apply log/clamp/normalize in multiple passes
- Row-major writes causing non-unit-stride stores per frame
Key issues: repeated trig, unnecessary roots, dense math, extra passes, allocations, and cache-unfriendly access.
#### What changed
- Precompute Hann; eliminate per-frame allocation
- Direct power (`re*re + im*im`) and `logf(x)*INV_LN10`
- Switched from a 16x25 mixed‑radix complex DFT to a split‑radix real‑FFT (400) via two 200‑pt (8x25) CFFTs + combine
- Manual unrolling of small FFT codelets (radix‑2, radix‑5)
- Sparse mel dot-products via CSR-like indices ([mel_mapping_tables.h](mel_mapping_tables.h))
- Frame-major writes (stride = n_frames) for unit-stride stores
- Thread-local scratch to avoid malloc/free and contention
- OpenMP across frames with SIMD in inner loops; static scheduling to reduce overhead
- Fused clamp + normalize in one pass
- Bun→C FFI with optional output reuse
Result: ~891 ms → **~1.67 ms average** on the benchmark below, while staying within the accuracy bounds above.
### Benchmark
```
% bun bench.ts
Benchmarking log-mel spectrogram
- file: samples_jfk.wav (176000 samples @ 16000 Hz)
- pad tail: 30s (480000 samples)
- frames: 4100 (128 mel bins → 524800 floats)
- engine: FFI call; output reused
- iterations: 30, warmup: 5, runs: 1
- env: Bun 1.2.19 | Darwin 24.5.0 | Apple M4 x10
Run 1/1: avg=1.671ms p50=1.656ms p90=1.776ms p99=2.092ms (min=1.530ms, max=2.199ms, sd=0.124)
Throughput: 2453382.0 frames/s | 392.541 MSamples/s | 1256.13 MB/s out
Memory: rss=47.05 MB (+160.00 KB) | heapUsed=1.11 MB (+0.00 B)
Done.
```
### Files
- [mel.c](mel.c): core implementation; compile to `libmel.{dylib,so}` (see top-of-file note)
- [mel_mapping_tables.h](mel_mapping_tables.h): generated sparse mel filterbank
- [index.ts](index.ts): Bun FFI; `logMelSpectrogram(audio, padTail)` convenience wrapper
- [bench.ts](bench.ts): benchmark runner (supports preallocated output and FFI/API modes)
- [dump_mel_filterbank.py](dump_mel_filterbank.py), [dump_mel_spec.py](dump_mel_spec.py): generate filterbank and reference spectrogram