{"id":30237788,"url":"https://github.com/lithdew/log-mel-spectrogram","last_synced_at":"2025-08-15T02:38:04.741Z","repository":{"id":309115368,"uuid":"1035175255","full_name":"lithdew/log-mel-spectrogram","owner":"lithdew","description":"My notes for computing log-Mel spectrograms on CPUs 129x faster.","archived":false,"fork":false,"pushed_at":"2025-08-10T00:09:22.000Z","size":1514,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-10T00:19:36.767Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lithdew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-09T20:17:23.000Z","updated_at":"2025-08-10T00:09:25.000Z","dependencies_parsed_at":"2025-08-10T00:29:58.859Z","dependency_job_id":null,"html_url":"https://github.com/lithdew/log-mel-spectrogram","commit_stats":null,"previous_names":["lithdew/log-mel-spectrogram"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/lithdew/log-mel-spectrogram","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lithdew%2Flog-mel-spectrogram","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lithdew%2Flog-mel-spectrogram/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lithdew%2Flog-mel-spectrogram/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lithdew%2Flog-mel-spectrogram/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lithdew","download_url":"https://codeload.github.com/lithdew/log-mel-spectrogram/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lithdew%2Flog-mel-spectrogram/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270515042,"owners_count":24598435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-15T02:00:12.559Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-15T02:38:00.308Z","updated_at":"2025-08-15T02:38:04.729Z","avatar_url":"https://github.com/lithdew.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"## log-mel-spectrogram\n\nMy notes for computing log-Mel spectrograms on CPUs ~533x faster. All rights reserved.\n\n- **FFT**: 400-point, hop=160 at 16 kHz (25 ms window, 10 ms hop)\n- **Mel**: 128 bins, librosa mel filterbank\n- **Scale**: log10, clamp to max−8 dB, then (v+4)/4 normalization\n\n### Accuracy\n\n- **≤ 5e-5 max abs error vs OpenAI/librosa reference** on typical inputs\n- Mean abs error ~1e-7; \u003e99.9% of values within 1e-5\n- Verified by [index_test.ts](index_test.ts) against [mel_example.json](mel_example.json) generated by [dump_mel_spec.py](dump_mel_spec.py) (PyTorch + librosa)\n\n### Why it’s fast\n\n- **Real‑FFT (RFFT) for N=400**: two complex 200‑pt FFTs (8x25) on even/odd, then a single combine for k=0..200\n- **Manual unrolling** of small codelets (radix‑2 butterflies and radix‑5 DFTs)\n- **OpenMP** across frames, plus SIMD pragmas on inner loops with aligned buffers\n- **Thread-local scratch buffers** to avoid contention and allocations\n- **Sparse mel dot-products** via precomputed CSR-style tables ([mel_mapping_tables.h](mel_mapping_tables.h))\n- **Fused clamp + normalize** in a single pass over the output\n- **Zero-copy FFI** from Bun to C, with optional output reuse to eliminate allocations\n\nSee [mel.c](mel.c) for the full implementation and comments. Generation of the mel filterbank and reference outputs: [dump_mel_filterbank.py](dump_mel_filterbank.py), [dump_mel_spec.py](dump_mel_spec.py).\n\n### Research\n\nInitial implementation ran in ~891 ms for the benchmark below.\n\n#### Naive implementation (conceptually)\n\n- Frame with 25 ms windows, 10 ms hop; Hann per frame\n- For each frame:\n  - Allocate temporaries (windowed, spectrum, mel row)\n  - Compute DFT with fresh sin/cos per bin (or call a general FFT), keep full complex spectrum\n  - Convert to magnitudes via sqrt, then square again for power\n  - Dense multiply against a 128x201 mel matrix\n  - Apply log/clamp/normalize in multiple passes\n  - Row-major writes causing non-unit-stride stores per frame\n\nKey issues: repeated trig, unnecessary roots, dense math, extra passes, allocations, and cache-unfriendly access.\n\n#### What changed\n\n- Precompute Hann; eliminate per-frame allocation\n- Direct power (`re*re + im*im`) and `logf(x)*INV_LN10`\n- Switched from a 16x25 mixed‑radix complex DFT to a split‑radix real‑FFT (400) via two 200‑pt (8x25) CFFTs + combine\n- Manual unrolling of small FFT codelets (radix‑2, radix‑5)\n- Sparse mel dot-products via CSR-like indices ([mel_mapping_tables.h](mel_mapping_tables.h))\n- Frame-major writes (stride = n_frames) for unit-stride stores\n- Thread-local scratch to avoid malloc/free and contention\n- OpenMP across frames with SIMD in inner loops; static scheduling to reduce overhead\n- Fused clamp + normalize in one pass\n- Bun→C FFI with optional output reuse\n\nResult: ~891 ms → **~1.67 ms average** on the benchmark below, while staying within the accuracy bounds above.\n\n### Benchmark\n\n```\n% bun bench.ts\nBenchmarking log-mel spectrogram\n- file: samples_jfk.wav (176000 samples @ 16000 Hz)\n- pad tail: 30s (480000 samples)\n- frames: 4100 (128 mel bins → 524800 floats)\n- engine: FFI call; output reused\n- iterations: 30, warmup: 5, runs: 1\n- env: Bun 1.2.19 | Darwin 24.5.0 | Apple M4 x10\nRun 1/1: avg=1.671ms p50=1.656ms p90=1.776ms p99=2.092ms (min=1.530ms, max=2.199ms, sd=0.124)\n  Throughput: 2453382.0 frames/s | 392.541 MSamples/s | 1256.13 MB/s out\n  Memory: rss=47.05 MB (+160.00 KB) | heapUsed=1.11 MB (+0.00 B)\nDone.\n```\n\n### Files\n\n- [mel.c](mel.c): core implementation; compile to `libmel.{dylib,so}` (see top-of-file note)\n- [mel_mapping_tables.h](mel_mapping_tables.h): generated sparse mel filterbank\n- [index.ts](index.ts): Bun FFI; `logMelSpectrogram(audio, padTail)` convenience wrapper\n- [bench.ts](bench.ts): benchmark runner (supports preallocated output and FFI/API modes)\n- [dump_mel_filterbank.py](dump_mel_filterbank.py), [dump_mel_spec.py](dump_mel_spec.py): generate filterbank and reference spectrogram\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flithdew%2Flog-mel-spectrogram","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flithdew%2Flog-mel-spectrogram","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flithdew%2Flog-mel-spectrogram/lists"}