https://github.com/idiap/zff_vad

Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering
https://github.com/idiap/zff_vad

audio-processing machine-learning noise-robust signal-processing speech-activity-detection voice-activity-detection

Last synced: about 1 month ago
JSON representation

Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering

Host: GitHub
URL: https://github.com/idiap/zff_vad
Owner: idiap
License: gpl-3.0
Created: 2022-11-22T14:12:07.000Z (over 2 years ago)
Default Branch: main
Last Pushed: 2023-10-19T15:07:27.000Z (over 1 year ago)
Last Synced: 2025-03-23T01:01:51.804Z (about 2 months ago)
Topics: audio-processing, machine-learning, noise-robust, signal-processing, speech-activity-detection, voice-activity-detection
Language: Python
Homepage:
Size: 631 KB
Stars: 19
Watchers: 6
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.rst
- License: LICENSE
- Citation: CITATION.cff

Awesome Lists containing this project

README

        ================================================================================================================

ZFF VAD

================================================================================================================

[Paper_]

[Poster_]

[Video_]

[Slides_]

|License| |OpenSource| |BlackFormat| |BanditSecurity| |iSortImports|

.. image:: img/figure.jpg

  :alt: Pipeline

Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering

---------------------------------------------------------------------------------------------------------------

This repository contains the code developed for the Interspeech accepted paper: `Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering`__ by E. Sarkar, R. Prasad, and M. Magimai Doss (2022).

Please cite the original authors for their work in any publication(s) that uses this work:

.. code:: bib

    @inproceedings{sarkar22_interspeech,

    author    = {Eklavya Sarkar and RaviShankar Prasad and Mathew Magimai Doss},

    title     = {{Unsupervised Voice Activity Detection by Modeling Source and System Information using Zero Frequency Filtering}},

    year      = {2022},

    booktitle = {Proc. Interspeech 2022},

    pages     = {4626--4630},

    doi       = {10.21437/Interspeech.2022-10535}

    }

Approach

---------

We jointly model voice source and vocal tract system information using zero-frequency filtering technique for the purpose of voice activity detection. This is computed by combining the ZFF filter outputs together to compose a composite signal carrying salient source and system information, such as the fundamental frequency :math:`$f_0$` and formants :math:`$F_1$` and :math:`$F_2$`, and then applying a dynamic threshold after spectral entropy-based weighting. Our approach operates purely in the time domain, is robust across a range of SNRs, and is much more computationally efficient than other neural methods. 

Installation

------------

This package has very few requirements. 

To create a new conda/mamba environment, install conda_, then mamba_ and simply follow the next steps:

.. code:: bash

    mamba env create -f environment.yml   # Create environment

    conda activate zff                    # Activate environment

    make install clean                    # Install packages

Command-line Usage

-------------------

To segment a single audio file into a .csv file:

.. code:: bash

    segment -w path/to/audio.wav -o path/to/save/segments

To segment a folder of audio files:

.. code:: bash

    segment -f path/to/folder/of/audio/files -o path/to/save/segments

For more options check:

.. code:: bash

    segment -h

*Note*: depending on the conditions of the given data, it will be necessary tune the smoothing and theta parameters.

Python Usage

-------------

To compute VAD on a given audio file:

.. code:: python

    from zff import utils

    from zff.zff import zff_vad

    # Read audio at native sampling rate

    sr, audio = utils.load_audio("audio.wav")

    # Get segments

    boundary = zff_vad(audio, sr)

    # Smooth

    boundary = utils.smooth_decision(boundary, sr)

    # Convert from sample to time domain

    segments = utils.sample2time(audio, sr, boundary)

    # Save as .csv file

    utils.save_segments("segments", "audio", segments)

To extract the composite signal from a given audio file:

.. code:: python

    from zff.zff import zff_cs

    from zff import utils

    # Read audio at native sampling rate

    fs, audio = utils.load_audio("audio.mp3")

    # Get composite signal

    composite = zff_cs(audio, sr)

    

    # Get all signals

    composite, y0, y1, y2, gcis = zff_cs(audio, sr, verbose=True)

Repository Structure

-----------------------------

.. code:: bash

    .

    ├── environment.yml          # Environment

    ├── img                      # Images

    ├── LICENSE                  # License

    ├── Makefile                 # Setup

    ├── MANIFEST.in              # Setup

    ├── pyproject.toml           # Setup

    ├── README.rst               # README

    ├── requirements.txt         # Setup

    ├── setup.py                 # Setup

    ├── version.txt              # Version

    └── zff                      # Source code folder

        ├── arguments.py            # Arguments parser

        ├── segment.py              # Main method

        ├── utils.py                # Utility methods

        └── zff.py                  # ZFF methods

Contact

-------

For questions or reporting issues to this software package, kindly contact the first author_.

    

.. _author: [email protected]

.. _Paper: https://www.isca-speech.org/archive/interspeech_2022/sarkar22_interspeech.html

.. _Poster: https://eklavyafcb.github.io/docs/Sarkar_Interspeech_2022_Poster_Landscape.pdf

.. _Video: https://youtu.be/hIHLu_7ESfM

.. _Slides: https://eklavyafcb.github.io/docs/Sarkar_Interspeech_2022_Presentation.pdf

.. _conda: https://conda.io

.. _mamba: https://mamba.readthedocs.io/en/latest/installation.html#existing-conda-install

__ https://www.isca-speech.org/archive/interspeech_2022/sarkar22_interspeech.html

.. |License| image:: https://img.shields.io/badge/License-GPLv3-blue.svg

    :target: https://github.com/idiap/ZFF_VAD/blob/master/LICENSE

    :alt: License

.. |OpenSource| image:: https://img.shields.io/badge/GitHub-Open%20source-green

    :target: https://github.com/idiap/ZFF_VAD/

    :alt: Open-Source

.. |BlackFormat| image:: https://img.shields.io/badge/code%20style-black-000000.svg

    :target: https://github.com/psf/black

    :alt: Style

.. |BanditSecurity| image:: https://img.shields.io/badge/security-bandit-yellow.svg

    :target: https://github.com/PyCQA/bandit

    :alt: Security

.. |iSortImports| image:: https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336

    :target: https://pycqa.github.io/isort

    :alt: Imports

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/idiap/zff_vad

Awesome Lists containing this project

README