awesome-diarization
A curated list of awesome Speaker Diarization papers, libraries, datasets, and other resources.
https://github.com/wq2012/awesome-diarization
Last synced: 15 days ago
JSON representation
-
Publications
-
Other
- An overview of automatic speaker diarization systems
- A spectral clustering approach to speaker diarization
- A study of the cosine distance-based mean shift for telephone speech diarization
- Stream-based speaker segmentation using speaker factors and eigenvoices
- AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario
- Speaker diarization using deep neural network embeddings
- Unsupervised methods for speaker diarization: An integrated and iterative approach
- A Speaker Diarization System for Studying Peer-Led Team Learning Groups
- Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks
- Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks
-
Special topics
- A Review of Speaker Diarization: Recent Advances with Deep Learning
- A review on speaker diarization systems and approaches
- Speaker diarization: A review of recent research
- DiarizationLM: Speaker Diarization Post-Processing with Large Language Models
- Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach
- Lexical speaker error correction: Leveraging language models for speaker diarization error correction
- TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization
- Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis
- End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings
- Supervised online diarization with sample mean loss for multi-domain data
- Discriminative Neural Clustering for Speaker Diarisation
- End-to-End Neural Speaker Diarization with Permutation-Free Objectives
- End-to-End Neural Speaker Diarization with Self-attention
- Fully Supervised Speaker Diarization
- A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings
- Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection
- Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR
- Joint Speech Recognition and Speaker Diarization via Sequence Transduction
- Says who? Deep learning models for joint speech recognition, segmentation and diarization
- Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox
- Online Speaker Diarization with Relation Network
- VoiceID on the Fly: A Speaker Recognition System that Learns from Scratch
- M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
- The Hitachi-JHU DIHARD III system: Competitive end-to-end neural diarization and x-vector clustering systems combined by DOVER-Lap
- ODESSA at Albayzin Speaker Diarization Challenge 2018
- Joint Discriminative Embedding Learning, Speech Activity and Overlap Detection for the DIHARD Challenge
- DyViSE: Dynamic Vision-Guided Speaker Embedding for Audio-Visual Speaker Diarization
- End-to-End Audio-Visual Neural Speaker Diarization
- MSDWild: Multi-modal Speaker Diarization Dataset in the Wild
- DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors
- AVA-AVD: Audio-Visual Speaker Diarization in the Wild
- A Review of Speaker Diarization: Recent Advances with Deep Learning
- Speaker diarization: A review of recent research
-
-
Software
-
Audio data augmentation
- pyroomacoustics
- gpuRIR
- rir_simulator_python
- WavAugment
- EEND_dataprep - to-end diarization models. |
-
Audio feature extraction
- LibROSA
- python_speech_features - speech-features.readthedocs.io/en/latest/ |
-
Clustering
- sklearn.cluster - ci.org/scikit-learn/scikit-learn.svg?branch=master)](https://travis-ci.org/scikit-learn/scikit-learn) | Python | scikit-learn clustering algorithms. |
- uis-rnn - rnn?style=social) [](https://travis-ci.org/google/uis-rnn) | Python & PyTorch | Google's Unbounded Interleaved-State Recurrent Neural Network (UIS-RNN) algorithm, for Fully Supervised Speaker Diarization. This clustering algorithm is **supervised**. |
- uis-rnn-sml - rnn-sml?style=social) | Python & PyTorch | A variant of UIS-RNN, for the paper Supervised Online Diarization with Sample Mean Loss for Multi-Domain Data. |
- DNC - based Discriminative Neural Clustering (DNC) for Speaker Diarisation. Like UIS-RNN, it is **supervised**. |
- SpectralCluster - ci.org/wq2012/SpectralCluster.svg?branch=master)](https://travis-ci.org/wq2012/SpectralCluster) | Python | Spectral clustering with affinity matrix refinement operations, auto-tune, and speaker turn constraints. |
- PLDA
- PLDA - source implementation of simplified PLDA (Probabilistic Linear Discriminant Analysis). |
- Auto-Tuning Spectral Clustering - Tuning-Spectral-Clustering?style=social) | Python | Auto-tuning Spectral Clustering method that does not need development set or supervised tuning. |
- Auto-Tuning Spectral Clustering - Tuning-Spectral-Clustering?style=social) | Python | Auto-tuning Spectral Clustering method that does not need development set or supervised tuning. |
-
Evaluation
- md-eval.pl - eval-v21.pl](https://github.com/jitendrab/btp/blob/master/c_code/single_diag_gaussian_no_viterbi/md-eval-v21.pl) from [jitendra](https://github.com/jitendrab); (3) [md-eval-22.pl](https://github.com/nryant/dscore/blob/master/scorelib/md-eval-22.pl) from [nryant](https://github.com/nryant) |
- Sequence Match Accuracy
- DiarizationLM - id?style=social) [](https://github.com/google/speaker-id/actions/workflows/python-app-diarizationlm.yml) | Python | Implements Word Error Rate (WER), Word Diarization Error Rate (WDER), and concatenated minimum-permutation Word Error Rate (cpWER). |
- pyannote-metrics - metrics?style=social) [](https://travis-ci.org/pyannote/pyannote-metrics) | Python| A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems. |
- SimpleDER
- dscore
- spyder
- CDER - phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://arxiv.org/abs/2208.08042) |
-
Framework
- SIDEKIT for diarization (s4d)
- LIUM SpkDiarization
- kaldi-asr - ci.com/kaldi-asr/kaldi.svg?branch=master)](https://travis-ci.com/kaldi-asr/kaldi) | Bash | Example scripts for speaker diarization on a portion of CALLHOME used in the 2000 NIST speaker recognition evaluation. |
- Alize LIA_SpkSeg
- MiniVox - source evaluation system for the online speaker diarization task. |
- SpeechBrain - source and all-in-one speech toolkit based on PyTorch. |
- pyAudioAnalysis
- AaltoASR - speech/speaker-diarization?style=social) | Python & Perl | Speaker diarization scripts, based on AaltoASR. |
- kaldi-speaker-diarization - lvl/kaldi-speaker-diarization?style=social) | Bash | Icelandic speaker diarization scripts using kaldi. |
- pyannote-audio - audio?style=social) | Python | Neural building blocks for speaker diarization: speech activity detection, speaker change detection, speaker embedding. |
- pyBK
- Speaker-Diarization - Diarization?style=social) | Python | Speaker diarization using uis-rnn and GhostVLAD. An easier way to support openset speakers. |
- EEND - speech/EEND?style=social) | Python & Bash & Perl | End-to-End Neural Diarization. |
- VBx - vectors diarization. x-vector extractor [recipe](https://github.com/phonexiaresearch/VBx-training-recipe) |
- RE-VERB - re-verb/RE-VERB?style=social) | Python & JavaScript | RE: VERB is speaker diarization system, it allows the user to send/record audio of a conversation and receive timestamps of who spoke when. |
- simple_diarizer
- Picovoice Falcon - diarization/#accuracy) speaker diarization engine written in C and available in Python, running on CPU with minimal overhead. |
- DiaPer - to-End Neural Diarization with Perceiver-Based Attractors](https://arxiv.org/pdf/2312.04324.pdf) including models pre-trained on free and public data. |
- sherpa-onnx - fsa/sherpa-onnx?style=social) | C++ & C & `C#` & Dart & Go & Java & JavaScript & Kotlin & Pascal & Python & Rust & Swift | Support speaker diarization, speech recognition, and text-to speech on various platforms with various language bindings. |
- FluidAudio - time audio processing with high accuracy. |
- FunASR - damo-academy/FunASR?style=social) | Python & PyTorch | FunASR is an open-source speech toolkit based on PyTorch, which aims at bridging the gap between academic researchs and industrial applications. |
- FunASR - grade speech recognition toolkit with built-in speaker diarization (cam++), VAD, ASR (SenseVoice/Paraformer), and punctuation. 50+ languages, 170x realtime. |
-
Other software
- VB Diarization - ci.org/wq2012/VB_diarization.svg?branch=master)](https://travis-ci.org/wq2012/VB_diarization) | Python | VB Diarization with Eigenvoice and HMM Priors. |
- DOVER-Lap - lap?style=social) | Python | Python package for combining diarization system outputs |
- Diar-az - di dataset. Kaldi to Gecko to Kaldi and corpus and back | |
-
Speaker change detection
- change_detection - Term Memory Networks. |
- tidydiarize
-
Speaker embedding
- resemble-ai/Resemblyzer - ai/Resemblyzer?style=social) | d-vector | Python & PyTorch | PyTorch implementation of generalized end-to-end loss for speaker verification, which can be used for voice cloning and diarization. |
- Speaker_Verification - vector | Python & TensorFlow | Tensorflow implementation of generalized end-to-end loss for speaker verification. |
- PyTorch_Speaker_Verification - vector | Python & PyTorch | PyTorch implementation of "Generalized End-to-End Loss for Speaker Verification" by Wan, Li et al. With UIS-RNN integration. |
- Real-Time Voice Cloning - Time-Voice-Cloning?style=social) | d-vector | Python & PyTorch | Implementation of "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" (SV2TTS) with a vocoder that works in real-time. |
- conformer-speaker-encoder - vector |Python & TFLite | Massively multilingual conformer-based speaker recognition models in TFLite format. |
- deep-speaker - speaker?style=social) | d-vector |Python & Keras | Third party implementation of the Baidu paper Deep Speaker: an End-to-End Neural Speaker Embedding System. |
- x-vector-kaldi-tf - zeinali/x-vector-kaldi-tf?style=social) | x-vector | Python & TensorFlow & Perl | Tensorflow implementation of x-vector topology on top of Kaldi recipe. |
- kaldi-ivector - ivector?style=social) | i-vector | C++ & Perl | Extension to Kaldi implementing the standard i-vector hyperparameter estimation and i-vector extraction procedure. |
- voxceleb-ivector - ivector?style=social) | i-vector |Perl | Voxceleb1 i-vector based speaker recognition system. |
- pytorch_xvectors - vector | Python & PyTorch | PyTorch implementation of Voxceleb x-vectors. Additionaly, includes meta-learning architectures for embedding training. Evaluated with speaker diarization and speaker verification. |
- ASVtorch - vector | Python & PyTorch | ASVtorch is a toolkit for automatic speaker recognition. |
- asv-subtools - subtools?style=social) | i-vector & x-vector | Kaldi & PyTorch | ASV-Subtools is developed based on Pytorch and Kaldi for the task of speaker recognition, language identification, etc. The 'sub' of 'subtools' means that there are many modular tools and the parts constitute the whole. |
- ReDimNet
- WeSpeaker - e2e/wespeaker?style=social) | x-vector & r-vector | Python & C++ & PyTorch | WeSpeaker is a research and production oriented speaker verification, recognition and diarization toolkit, which supports very strong recipes with on-the-fly data preparation, model training and evaluation, as well as runtime C++ codes. |
- x-vector-kaldi-tf - zeinali/x-vector-kaldi-tf?style=social) | x-vector | Python & TensorFlow & Perl | Tensorflow implementation of x-vector topology on top of Kaldi recipe. |
-
-
Star History
-
Video tutorials
- ![Star History Chart - history.com/#wq2012/awesome-diarization&Date)
- ![Star History Chart - history.com/#wq2012/awesome-diarization&Date)
-
Categories
Sub Categories
Video tutorials
69
Tech blogs
69
Other
47
Special topics
33
Framework
22
Speaker embedding
15
Speaker embedding training sets
13
Diarization datasets
10
Clustering
9
Evaluation
8
Audio data augmentation
5
Other software
3
Augmentation noise sources
2
Speaker change detection
2
Audio feature extraction
2
Books
1
Online courses
1
Keywords
speaker-diarization
8
audio
5
speaker-recognition
5
pytorch
5
diarization
4
speaker-verification
4
asr
4
python
4
voice-activity-detection
3
deep-learning
3
acoustics
3
speech-to-text
3
speaker-embedding
2
room-impulse-response
2
speaker-identification
2
tensorflow
2
image-source-model
2
vad
2
speech-recognition
2
speech-processing
2
macos
2
machine-learning
2
signal-processing
2
ios
2
speaker-change-detection
1
speech-activity-detection
1
linux
1
aarch64
1
mfc
1
lazarus
1
android
1
arm32
1
dotnet
1
object-pascal
1
cpp
1
onnx
1
raspberry-pi
1
risc-v
1
dsp
1
librosa
1
music
1
scipy
1
audio-analysis-tasks
1
audio-data
1
pyaudioanalysis
1
audio-processing
1
huggingface
1
language-model
1
speech-enhancement
1
speech-separation
1