https://github.com/phyulwin/unity-whisper-reading-prototype

Unity (6000.1.3f1) prototype for real-time speech-based reading practice using Whisper transcription, scoring, and live word feedback.
https://github.com/phyulwin/unity-whisper-reading-prototype

speech unity2d whisper-ai

Last synced: 27 days ago
JSON representation

Unity (6000.1.3f1) prototype for real-time speech-based reading practice using Whisper transcription, scoring, and live word feedback.

• Host: GitHub
• URL: https://github.com/phyulwin/unity-whisper-reading-prototype
• Owner: phyulwin
• License: mit
• Created: 2026-04-02T22:45:43.000Z (3 months ago)
• Default Branch: main
• Last Pushed: 2026-04-02T23:48:08.000Z (3 months ago)
• Last Synced: 2026-04-03T09:50:33.103Z (3 months ago)
• Topics: speech, unity2d, whisper-ai
• Language: ShaderLab
• Homepage:
• Size: 995 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Unity Whisper Reading Prototype

![](/Assets/Resources/screenshot1.png)

A standalone Unity 2D speech-reading prototype for testing real-time word recognition using the Whisper speech-to-text pipeline.

The application loads text content, parses it into sentences, and guides the user through each word sequentially with live feedback.

As the user reads aloud:

* Recognized words are validated against the current target word

* Correct words are highlighted in green

* Skipped or incorrect words are marked in red

* The underline advances automatically as progression continues

A live dashboard tracks:

* Total words

* Correct reads

* Skipped words

* Completion percentage

### Demo Video (YouTube)

[![Watch the demo on YouTube](https://img.youtube.com/vi/gigwffJc1PA/hqdefault.jpg)](https://youtu.be/gigwffJc1PA?si=gMmWmtmCd08HOleH)

---

## Whisper Integration

This project builds on the open-source **whisper.unity** package, which provides the underlying Whisper model integration and transcription pipeline used for microphone streaming and real-time text processing.

Credit to the original contributors for the foundational STT infrastructure.

![](/Assets/Resources/screenshot2.png)

---

## Updating Reading Content

To modify the reading content:

```

Assets/Resources/SampleText.txt

```

Replace the file contents with your desired text (story, dialogue, or sentence set).

The system will automatically:

* Load the file at runtime

* Parse it into sentences

* Update the reading flow

No code changes required.