https://github.com/emergenceai/kotlin_speech_features
This library provides common speech features for ASR including MFCCs and filterbank energies for Android and iOS.
https://github.com/emergenceai/kotlin_speech_features
android feature-extraction ios kotlin speech-feature-extraction speech-features speech-processing
Last synced: 4 months ago
JSON representation
This library provides common speech features for ASR including MFCCs and filterbank energies for Android and iOS.
- Host: GitHub
- URL: https://github.com/emergenceai/kotlin_speech_features
- Owner: EmergenceAI
- License: mit
- Created: 2022-09-15T12:02:16.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-09-20T07:12:41.000Z (over 3 years ago)
- Last Synced: 2025-10-21T09:55:20.662Z (5 months ago)
- Topics: android, feature-extraction, ios, kotlin, speech-feature-extraction, speech-features, speech-processing
- Language: Kotlin
- Homepage: https://merlynmind.github.io/kotlin_speech_features/
- Size: 8.09 MB
- Stars: 27
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
Kotlin Speech Features
---
Quick Links
# 📒 Introduction
This library is a complete port of python_speech_features in pure Kotlin available for Android and iOS projects.
It provides common speech features for Automated speech recognition (ASR) including MFCCs and filterbank energies.
To know more about MFCCs read more.
### Features
- [Mel Frequency Cepstral Coefficients (mfcc)](https://merlynmind.github.io/kotlin_speech_features/-kotlin%20-speech%20-features/org.merlyn.kotlinspeechfeatures/-speech-features/mfcc.html)
- [Filterbank Energies (fbank)](https://merlynmind.github.io/kotlin_speech_features/-kotlin%20-speech%20-features/org.merlyn.kotlinspeechfeatures/-speech-features/fbank.html)
- [Log Filterbank Energies (logfbank)](https://merlynmind.github.io/kotlin_speech_features/-kotlin%20-speech%20-features/org.merlyn.kotlinspeechfeatures/-speech-features/logfbank.html)
- [Spectral Subband Centroids (ssc)](https://merlynmind.github.io/kotlin_speech_features/-kotlin%20-speech%20-features/org.merlyn.kotlinspeechfeatures/-speech-features/ssc.html)
# 🙋 How to use
We support multiple platforms using Kotlin multiplatform.
Android
## Integration
Add jitpack.io to your project's repositories:
```gradle
allProjects {
repositories {
google()
maven { url 'https://jitpack.io' }
}
}
```
Add the dependency:
```gradle
dependencies {
implementation "com.github.MerlynMind:kotlin_speech_features:${version}"
}
```
## Example implementation
A sample app is included in this repo to help understand the implementation.
1. Convert your audio signal in the form of a float array. (A demo provided in the sample app)
2. Initialize speech features
```kotlin
private val speechFeatures = SpeechFeatures()
```
3. Perform any of the 4 operations:
```kotlin
val result = speechFeatures.mfcc(MathUtils.normalize(wav), nFilt = 64)
val result = speechFeatures.fbank(MathUtils.normalize(wav), nFilt = 64)
val result = speechFeatures.logfbank(MathUtils.normalize(wav), nFilt = 64)
val result = speechFeatures.ssc(MathUtils.normalize(wav), nFilt = 64)
```
4. The result will contain metrices with the expected features. Pass in these features for further processes (e.g. classification, speech recognition).
---
iOS
## Integration
1. In XCode, go to `File > Add Packages...`
2. Paste in the URL of this repo in the search box
3. Select the package found
4. Click `Add Package` button
## Example implementation
A sample app is included in this repo to help understand the implementation.
1. Convert your audio signal in the form of an `KotlinIntArray` and normalize it.
```swift
import KotlinSpeechFeatures
let signal = [Int](1...1000) // Example signal
let normalized = MathUtils.Companion.init().normalize(sig: toKotlinIntArray(arr: signal))
func toKotlinIntArray(arr: [Int]) -> KotlinIntArray {
let result = KotlinIntArray(size: Int32(arr.capacity))
for i in 0...(arr.count-1) {
result.set(index: Int32(i), value: Int32(arr[i]))
}
return result
}
```
2. Initialize speech features
```swift
let speechFeatures = SpeechFeatures()
```
3. Perform any of the 4 operations:
```swift
let result = speechFeatures.mfcc(signal: normalized, sampleRate: 16000, winLen: 0.025, winStep: 0.01, numCep: 13, nFilt: 64, nfft: 512, lowFreq: 0, highFreq: ni;, preemph: 0.97, ceplifter: 22, appendEnergy: true, winFunc: nil)
let result = speechFeatures.fbank(signal: normalized, sampleRate: 16000, winLen: 0.025, winStep: 0.01, nFilt: 64, nfft: 512, lowFreq: 0, highFreq: nil, preemph: 0.97, winFunc: nil)
let result = speechFeatures.logfbank(signal: normalized, sampleRate: 16000, winLen: 0.025, winStep: 0.01, nFilt: 64, nfft: 512, lowFreq: 0, highFreq: nil, preemph: 0.97, winFunc: nil)
let result = speechFeatures.ssc(signal: normalized, sampleRate: 16000, winLen: 0.025, winStep: 0.01, nFilt: 64, nfft: 512, lowFreq: 0, highFreq: nil, preemph: 0.97, winFunc: nil)
```
4. The result will contain metrices with the expected features. Pass in these features for further processes (e.g. classification, speech recognition).
JavaScript
```
Coming soon...
```
# ✍️ Contributing
Interested in contributing to the library? Thank you so much for your interest!
We are always looking for improvements to the project and contributions from open-source developers are greatly appreciated.
1. Clone repo and create a new branch:
```
git checkout https://github.com/merlynmind/kotlin_speech_features -b name_for_new_branch
```
2. Make changes and test
3. Submit Pull Request with comprehensive description of changes
# 🌟 Spread the word!
If you want to say thank you and/or support active development of this library:
- Add a GitHub Star to the project!
- Tweet about the project on your Twitter!
Tag @MerlynMind and/or #heyMerlnyn
Thank you so much for your interest in growing the reach of our library!
# 🧡 Credits
- [Arjun Sunil](https://github.com/arjun921) - Original Author of kotlin speech features
- [Raquib-Ul Alam](https://github.com/alamkanak) - For major refactoring and making the code presentable
- [Rob Smith](https://github.com/robmsmt) - For Mentoring and helping us to navigate through the task
# 📝 References
- Original library - [Python Speech Features](https://github.com/jameslyons/python_speech_features)
- Reference Library - [C Speech Features](https://github.com/Cwiiis/c_speech_features)
- Sample english.wav was obtained from
```
wget http://voyager.jpl.nasa.gov/spacecraft/audio/english.au
sox english.au -e signed-integer english.wav
```