https://github.com/tompaana/hls-transcription-sample
Sample demonstrating how to transcribe/live-subtitle HTTP Live Streaming media.
https://github.com/tompaana/hls-transcription-sample
c-sharp hls http-live-streaming transcription
Last synced: about 1 year ago
JSON representation
Sample demonstrating how to transcribe/live-subtitle HTTP Live Streaming media.
- Host: GitHub
- URL: https://github.com/tompaana/hls-transcription-sample
- Owner: tompaana
- License: mit
- Created: 2018-03-30T10:06:13.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-03-30T16:26:57.000Z (over 8 years ago)
- Last Synced: 2025-04-08T08:05:55.151Z (about 1 year ago)
- Topics: c-sharp, hls, http-live-streaming, transcription
- Language: C#
- Size: 26.4 KB
- Stars: 10
- Watchers: 2
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
HTTP Live Stream (HLS) Transcriber Sample
=========================================
This sample demonstrates how to transcribe/live-subtitle
[HTTP Live Streaming (HLS)](https://en.wikipedia.org/wiki/HTTP_Live_Streaming) media.
The project is developed for .NET Framework 4.7.1, but should be compatible with versions >= 4.6.
The audio extraction and format detection is implemented using
[NAudio library](https://github.com/naudio/NAudio). The transciption is done with
[Bing Speech API](https://azure.microsoft.com/en-us/services/cognitive-services/speech/).
## Getting started ##
1. Open the [HLSSample.sln](/HLSSample.sln) solution file with Visual Studio
2. Select the start-up project (`HSLConsoleTest.NETFramework` or `HSLWPFTest.NETFramework`):
Right-click the project name in **Solution Explorer** and select **Set as StartUp Project**
3. Insert the **playlist URL** (`.m3u8` file) as the value of `PlaylistUrl` string constant:
* To [Program.cs file](/HLSConsoleTest.NETFramework/Program.cs#L11) in the console app
* To [MainWindow.xaml.cs file](/HLSWPFTest.NETFramework/MainWindow.xaml.cs#L17) in the WPF app
4. Insert the **Bing Speech API** subscription key to enable transcription (can be omitted but so will then be the transcription)
5. Run and enjoy
## Implementation ##
The solution consists of three projects:
1. [HSTools.NETFramework](/HLSTools.NETFramework),
2. [HSLConsoleTest.NETFramework](/HLSConsoleTest.NETFramework) and
3. [HSLWPFTest.NETFramework](/HLSWPFTest.NETFramework).
**HSLTools.NETFramework** is a class library project containing the main functionality whilst the
two other (demo apps) serve as examples on how to use the aforementioned library.
### HSLTools ###
The **HSLTools** class library implements the following features:
* Loading and parsing HTTP Live Stream playlists (`.m3u8` files)
* Downloading `.ts` files into memory
* Extracting audio from `.ts` files utilizing [NAudio library](https://github.com/naudio/NAudio)
* Saving binary files on local disk
* Transcribing audio (bytes) with [Bing Speech API](https://azure.microsoft.com/en-us/services/cognitive-services/speech/)
The main class of the library is [HLSProcessor](/HLSTools.NETFramework/HLSProcessor.cs).
See the two demo application projects (HLSConsoleTest and HLSWPFTest) to learn how to use the class
library.
### HLSConsoleTest and HLSWPFTest projects ###
**HLSConsoleTest** processes the given playlist and displays the audio transcription in the
console window. **HLSWPFTest** plays and displays the video files in the playlist with
subtitles, which are produced by transcribing the audio in the media.
## What's missing? ##
This is not a production-ready piece of code, but rather a proof-of-concept.
Stuff missing/to consider:
* Universal Windows applications are not supported - please support/contribute to the awesome
[NAudio project](https://github.com/naudio/NAudio) in order to enable UWP compatibility.
* The media chunks are processed as they come (via the playlist). Thus, if/when the audio in
the chunk is terminated in the middle of a word, the transcription is incomplete
* "TODO" item here: Refactor/break the chunks in pieces based on the silent bits in the audio
* The [MediaElement](https://msdn.microsoft.com/en-us/library/system.windows.controls.mediaelement(v=vs.110).aspx)
in WPF applications does not support HTTP Live Stream out-of-the-box - the quick and dirty
approach taken here is to save the `.ts` files onto local disk and to feed the `MediaElement` the
local URIs.
## Credits ##
This project was one of the outcomes of a short hackfest and was developed by the following
fantastic team of people (in alphabetical order):
* [Wagner Alves](https://github.com/keyrox)
* [Toby Bradshaw](https://github.com/tobybrad)
* [João Goncalves](https://github.com/goncalvesj)
* [Tomi Paananen](https://github.com/tompaana)
* [Anita Ramanan](https://github.com/anraman)