{"id":51776456,"url":"https://github.com/42atomys/hpss-voice-denoiser","last_synced_at":"2026-07-20T06:30:37.449Z","repository":{"id":369894958,"uuid":"1125373671","full_name":"42atomys/hpss-voice-denoiser","owner":"42atomys","description":"a denoiser package using HPSS to clean voice from audio (usefull for STT, Diarization and embedding) ","archived":false,"fork":false,"pushed_at":"2025-12-30T23:03:49.000Z","size":48,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-07T11:32:31.807Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/42atomys.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-30T16:01:52.000Z","updated_at":"2026-01-13T23:51:50.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/42atomys/hpss-voice-denoiser","commit_stats":null,"previous_names":["42atomys/hpss-voice-denoiser"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/42atomys/hpss-voice-denoiser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/42atomys%2Fhpss-voice-denoiser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/42atomys%2Fhpss-voice-denoiser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/42atomys%2Fhpss-voice-denoiser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/42atomys%2Fhpss-voice-denoiser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/42atomys","download_url":"https://codeload.github.com/42atomys/hpss-voice-denoiser/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/42atomys%2Fhpss-voice-denoiser/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35676362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"ssl_error","status_checked_at":"2026-07-20T02:08:09.736Z","response_time":111,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-20T06:30:34.221Z","updated_at":"2026-07-20T06:30:37.442Z","avatar_url":"https://github.com/42atomys.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HPSS Voice Denoiser\n\nA production-ready audio denoising pipeline optimized for **ASR preprocessing** (Speech-to-Text, Speaker Diarization, Voice Embedding).\n\nBuilt on **Harmonic-Percussive Source Separation (HPSS)** with context-aware mixing to preserve voice quality while removing environmental noise.\n\n## Features\n\n- **Optimized for ASR**: Preserves voice characteristics critical for STT, diarization, and speaker embedding\n- **Stateless Processing**: Each audio chunk is processed independently (perfect for streaming)\n- **Voice-Preserving**: 99% voice band preservation, consonants intact\n- **Low Latency**: Suitable for real-time applications\n- **Simple API**: Easy to integrate as a library or use via CLI\n\n## Benchmark Results\n\nTested on real audio (88 seconds total). Run `benchmarks/benchmark.py` for full analysis.\n\n| Metric | Value | Description |\n|--------|-------|-------------|\n| **STT Confidence** | +16% improvement | Whisper word probability increased after denoising |\n| **Speaker Embedding** | 93.5% similar | Voice identity preserved (cosine similarity before/after) |\n| **Diarization** | 98% consistent | Speaker segments unchanged by denoising |\n| **Voice Band (300-3kHz)** | 75% preserved | Mid frequencies containing voice fundamentals |\n| **High Freq (3k-8kHz)** | 48% preserved | Reduced by design (noise lives here) |\n\n## Installation\n\n### From PyPI (recommended)\n\n```bash\npip install hpss-voice-denoiser\n```\n\n### With visualization support\n\n```bash\npip install hpss-voice-denoiser[visualization]\n```\n\n### From source\n\n```bash\ngit clone https://github.com/atomys/hpss-voice-denoiser.git\ncd hpss-voice-denoiser\npip install -e .\n```\n\n## Quick Start\n\n### As a Library\n\n```python\nfrom hpss_denoiser import HPSSDenoiser\n\n# Create denoiser with default settings\ndenoiser = HPSSDenoiser()\n\n# Process PCM audio (16kHz, 16-bit, mono)\nwith open(\"input.pcm\", \"rb\") as f:\n    pcm_data = f.read()\n\n# Denoise\ncleaned_pcm = denoiser.process(pcm_data)\n\n# Save result\nwith open(\"output.pcm\", \"wb\") as f:\n    f.write(cleaned_pcm)\n```\n\n### With NumPy Arrays\n\n```python\nimport numpy as np\nfrom hpss_denoiser import HPSSDenoiser\n\ndenoiser = HPSSDenoiser()\n\n# Float audio (-1.0 to 1.0)\naudio = np.random.randn(16000).astype(np.float64) * 0.1\n\n# Process\ncleaned = denoiser.process_array(audio)\n```\n\n### Custom Configuration\n\n```python\nfrom hpss_denoiser import HPSSDenoiser, DenoiserConfig\n\n# Adjust for your use case\nconfig = DenoiserConfig(\n    sample_rate=16000,\n    \n    # More aggressive noise reduction in silence\n    no_context_perc_gain=0.02,\n    \n    # Preserve more consonants\n    voice_context_perc_gain=0.25,\n)\n\ndenoiser = HPSSDenoiser(config)\n```\n\n### CLI Usage\n\n```bash\n# Basic usage\nhpss-denoise input.pcm output.pcm\n\n# Process with intermediate stages (for debugging)\nhpss-denoise input.pcm output.pcm --stages\n\n# Generate analysis visualization\nhpss-denoise input.pcm --analyze --output-image analysis.png\n\n# Custom sample rate\nhpss-denoise input.pcm output.pcm --sample-rate 8000\n\n# Show all options\nhpss-denoise --help\n```\n\n## Audio Format\n\nThe denoiser expects and produces:\n\n- **Format**: Raw PCM\n- **Sample Rate**: 16000 Hz (configurable)\n- **Bit Depth**: 16-bit signed integer\n- **Channels**: Mono\n\n### Converting from other formats\n\n```bash\n# WAV to PCM\nffmpeg -i input.wav -f s16le -acodec pcm_s16le -ar 16000 -ac 1 input.pcm\n\n# MP3 to PCM\nffmpeg -i input.mp3 -f s16le -acodec pcm_s16le -ar 16000 -ac 1 input.pcm\n\n# PCM to WAV (for playback)\nffmpeg -f s16le -ar 16000 -ac 1 -i output.pcm output.wav\n```\n\n## How It Works\n\n### Pipeline Architecture\n\n```\nAudio Input (PCM 16kHz, 16-bit)\n    │\n    ▼\n┌─────────────────────────────────────┐\n│  High-pass Filter (80 Hz)           │  Remove DC offset \u0026 rumble\n└─────────────────────────────────────┘\n    │\n    ▼\n┌─────────────────────────────────────┐\n│  STFT Analysis                      │  Time-frequency representation\n│  (25ms frames, 6ms hop)             │\n└─────────────────────────────────────┘\n    │\n    ▼\n┌─────────────────────────────────────┐\n│  HPSS Separation                    │  Split into harmonic (voice)\n│  (median filtering)                 │  and percussive (transients)\n└─────────────────────────────────────┘\n    │\n    ├─── Harmonic ───┐\n    │                ▼\n    │    ┌─────────────────────────────┐\n    │    │  Envelope Tightening        │  Reduce HPSS echo artifacts\n    │    │  (asymmetric follower)      │\n    │    └─────────────────────────────┘\n    │                │\n    │                ▼\n    │    ┌─────────────────────────────┐\n    ├───▶│  Context-Based Mixing       │  Detect voice activity\n    │    │  - Voice: keep 20% perc     │  Mix based on context\n    │    │  - Silence: keep 4% perc    │\n    │    └─────────────────────────────┘\n    │                │\n    └── Percussive ──┘\n                     │\n                     ▼\n┌─────────────────────────────────────┐\n│  Low-Frequency Denoising            │  Spectral subtraction \u003c350Hz\n│  (percentile-based)                 │\n└─────────────────────────────────────┘\n    │\n    ▼\n┌─────────────────────────────────────┐\n│  ISTFT Synthesis                    │  Reconstruct audio\n└─────────────────────────────────────┘\n    │\n    ▼\nAudio Output (PCM 16kHz, 16-bit)\n```\n\n### Why HPSS?\n\n**Harmonic-Percussive Source Separation** uses median filtering on the spectrogram:\n\n- **Harmonic components** (voice fundamentals, vowels) appear as horizontal lines\n- **Percussive components** (transients, consonants, noise) appear as vertical lines\n\nBy separating these, we can:\n1. Keep the harmonic component (clean voice)\n2. Selectively mix percussive based on voice context\n3. During speech: include percussive (consonants like 't', 's', 'k')\n4. During silence: suppress percussive (noise transients)\n\n### Key Innovation: Context-Aware Mixing\n\nThe challenge with HPSS for voice is that **consonants are percussive**. Naive suppression of the percussive component removes 't', 's', 'f', etc.\n\nOur solution: **detect voice context** using harmonic energy in the 200-4000 Hz band:\n- If voice is present: mix more percussive (preserve consonants)\n- If silence: aggressively suppress percussive (remove noise)\n\n## Configuration Reference\n\n```python\n@dataclass\nclass DenoiserConfig:\n    \"\"\"Configuration for HPSS voice denoiser.\"\"\"\n    \n    # Audio parameters\n    sample_rate: int = 16000          # Input/output sample rate\n    \n    # STFT parameters\n    frame_size_ms: int = 25           # Analysis frame size\n    hop_size_ms: int = 6              # Frame hop size\n    \n    # HPSS separation\n    harmonic_kernel: int = 9          # Median filter size (time)\n    percussive_kernel: int = 9        # Median filter size (freq)\n    hpss_margin: float = 2.5          # Separation hardness\n    \n    # Context detection\n    context_window: int = 10          # Frames to extend voice context\n    harmonic_threshold_db: float = -20.0  # Voice detection threshold\n    \n    # Percussive mixing\n    voice_context_perc_gain: float = 0.20  # Keep 20% during voice\n    no_context_perc_gain: float = 0.04     # Keep 4% during silence\n    \n    # Envelope tightening (echo reduction)\n    envelope_tightening: bool = True\n    envelope_attack_frames: int = 2\n    envelope_release_frames: int = 3\n    envelope_min_gain: float = 0.15\n    \n    # Low-frequency denoising\n    noise_reduction_strength: float = 0.8\n    noise_reduction_max_freq: float = 350.0\n```\n\n## Use Cases\n\n### Speech-to-Text (STT)\n\n```python\nfrom hpss_denoiser import HPSSDenoiser\nimport whisper\n\ndenoiser = HPSSDenoiser()\n\n# Denoise before transcription\nwith open(\"noisy_audio.pcm\", \"rb\") as f:\n    noisy = f.read()\n\ncleaned = denoiser.process(noisy)\n\n# Save and transcribe\nwith open(\"cleaned.pcm\", \"wb\") as f:\n    f.write(cleaned)\n\n# Use with Whisper\nmodel = whisper.load_model(\"base\")\nresult = model.transcribe(\"cleaned.wav\")\n```\n\n### Speaker Diarization\n\n```python\nfrom hpss_denoiser import HPSSDenoiser\n\n# Denoising improves speaker boundary detection\ndenoiser = HPSSDenoiser()\n\n# Process chunks for streaming diarization\nchunk_size = 30 * 16000 * 2  # 30 seconds\n\nwith open(\"meeting.pcm\", \"rb\") as f:\n    while chunk := f.read(chunk_size):\n        cleaned_chunk = denoiser.process(chunk)\n        # Send to diarization pipeline\n```\n\n### Voice Embedding\n\n```python\nfrom hpss_denoiser import HPSSDenoiser\n\n# Clean audio produces more stable embeddings\ndenoiser = HPSSDenoiser()\n\n# Process enrollment audio\nenrollment_clean = denoiser.process(enrollment_pcm)\n\n# Process verification audio\nverification_clean = denoiser.process(verification_pcm)\n\n# Compare embeddings (using your embedding model)\n```\n\n## Performance\n\n### Processing Speed\n\n| Audio Duration | Processing Time | Real-time Factor |\n|----------------|-----------------|------------------|\n| 1 second | ~44 ms | ~23x |\n| 10 seconds | ~420 ms | ~23x |\n| 88 seconds | ~3.8 s | ~23x |\n\n*Tested on macOS (Darwin), Python 3.12, single-threaded*\n\n### Memory Usage\n\n- ~50 MB base memory\n- ~2 MB per second of audio being processed\n- Streaming-friendly: process in chunks\n\n## Troubleshooting\n\n### Muffled output\n\nIncrease `voice_context_perc_gain`:\n\n```python\nconfig = DenoiserConfig(voice_context_perc_gain=0.30)\n```\n\n### Too much noise remaining\n\nDecrease `no_context_perc_gain`:\n\n```python\nconfig = DenoiserConfig(no_context_perc_gain=0.02)\n```\n\n### Echo/reverb artifacts\n\nReduce envelope release time:\n\n```python\nconfig = DenoiserConfig(envelope_release_frames=2)\n```\n\n### Consonants being cut\n\nIncrease context window:\n\n```python\nconfig = DenoiserConfig(context_window=15)\n```\n\n## Development\n\n### Setup\n\n```bash\ngit clone https://github.com/atomys/hpss-voice-denoiser.git\ncd hpss-voice-denoiser\npip install -e \".[dev]\"\n```\n\n### Run tests\n\n```bash\npytest\n```\n\n### Type checking\n\n```bash\nmypy src/hpss_denoiser\n```\n\n### Linting\n\n```bash\nruff check src/\nruff format src/\n```\n\n## Algorithm References\n\n- **HPSS**: Fitzgerald, D. (2010). \"Harmonic/Percussive Separation using Median Filtering\"\n- **Spectral Subtraction**: Boll, S. (1979). \"Suppression of Acoustic Noise in Speech Using Spectral Subtraction\"\n\n## License\n\nMIT License - see [LICENSE](LICENSE) for details.\n\n## Contributing\n\nContributions welcome! Please open an issue first to discuss what you would like to change.\n\n## Acknowledgments\n\nDeveloped to improve audio coming from wearable device project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F42atomys%2Fhpss-voice-denoiser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F42atomys%2Fhpss-voice-denoiser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F42atomys%2Fhpss-voice-denoiser/lists"}