https://github.com/scosman/voicebox
Exploration: using technology to aid people who lack both the ability to speak and fine motor control.
https://github.com/scosman/voicebox
accessibility gpt keyboards llm voicebox
Last synced: 15 days ago
JSON representation
Exploration: using technology to aid people who lack both the ability to speak and fine motor control.
- Host: GitHub
- URL: https://github.com/scosman/voicebox
- Owner: scosman
- Created: 2022-12-06T01:23:37.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2024-10-24T17:48:09.000Z (12 months ago)
- Last Synced: 2024-10-26T02:09:36.235Z (12 months ago)
- Topics: accessibility, gpt, keyboards, llm, voicebox
- Language: C
- Homepage:
- Size: 32.8 MB
- Stars: 12
- Watchers: 3
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[](https://github.com/scosman/voicebox/actions/workflows/test.yml)
# Mission
Mission: Explore using technology to aid people who lack both the ability to speak and fine motor control. This is common with cerebral palsy.
Our goal is to find systems which make a significant impact to some part of their life, and can be scaled to help many people. If explorations are successful, we will ship apps and/or tools to make this work widely accessible.
Success for this mission is a solution **many** people use **often** which is **much better than prior methods**. To achieve that, we ensure broadly useful, easily available (eg: App Store), easy to learn (consumer grade usability, in-app coaching), free, and marketed to reach the people who need it. For this project, we’d rather fail at this ambitious mission, than compromise and ship something incremental or inaccessible.
# Areas of Exploration
### 1) Realtime conversation
- Allow people with cerebral palsy to participate in conversation in real time. Precision of wording is flexible, speed of communicating their ideas is critical. We want them to be involved at the pace of a typical conversation, with full expressiveness.
- Explorations include: ML-LLM autocomplete, ML-LLM prediction/synthesis, ML grammar and expression tools, speech to text, text to speech, and novel combinations of the above.
### 2) Novel UIs for text entry and system control
- Build novel UIs that better accommodate lack of fine motor control. Leverage the dynamic screens of mobile devices, to allow typing and system control without moving your hands. Make accessible switch systems more powerful and learnable with consumer grade design.
- Explorations include: 2-switch UIs, 1-switch UIs, modal UIs, tree-based selection, ML predictions, consumer-grade onboarding.
# Development
### iOS Build instructions
To build this project you must have an OpenAPI key, which isn't included in the repo. Make a copy of `/ios/voicebox/Utils/AppSecrets.h.TEMPLATE` in the location `/ios/voicebox/Utils/AppSecrets.h`, then paste in your real API key.
### iOS Formatting Requirements
Any commits should be formatted with clang-format. Run `ios/format-code.sh` to format to project standards. The script `check-format.sh` checks for compliance with formatting standards. Copy the git hook in `ios/voicebox/Git Hooks` to your `.git/hooks` directory to ensure it's not missed.
# Author
Created at Chesterfield Laboratories Inc. by Steve Cosman