Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/natewu/neurospeak
natHACKS 23 submission
https://github.com/natewu/neurospeak
Last synced: 7 days ago
JSON representation
natHACKS 23 submission
- Host: GitHub
- URL: https://github.com/natewu/neurospeak
- Owner: natewu
- Created: 2023-11-09T00:01:58.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-11-19T16:57:13.000Z (12 months ago)
- Last Synced: 2024-05-01T16:16:39.738Z (7 months ago)
- Language: TypeScript
- Homepage:
- Size: 1.15 MB
- Stars: 5
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# NeuroSpeak
> [!NOTE]
For backend, go to the **gpt-backend** folder and run `npm install` to install all the dependencies. Then run `npm start` to start the server.
Don't forget to add the API key inside the `.env` file.
## Inspiration 💡
The inspiration behind NeuroSpeak is deeply rooted in the desire to bridge the communication gap for individuals with locked-in syndrome or paralysis. The realization that around 50,000 people in the U.S. alone live with locked-in syndrome, and globally, nearly 1 in 50 people are living with paralysis, propelled us to action. Recognizing the challenges these individuals face in expressing themselves, feeling isolated in their own bodies, we saw an urgent need for an accessible, empowering communication solution. NeuroSpeak is our answer to giving a voice to the voiceless.
## What it does 🤔
NeuroSpeak is a groundbreaking application that transforms the way individuals with paralysis or locked-in syndrome communicate. Utilizing advanced EEG and machine learning technologies, it captures brain signals corresponding to directional intentions (up, down, left, right) and converts these into meaningful actions. When spoken to, the app translates speech into text, which is then processed by a GPT language model to generate contextually relevant responses. Users can select their response using EEG signals, and the app vocalizes these choices, enabling real-time, meaningful communication. This technology is not just a tool; it's a bridge to the wider world, turning isolation into connection.
## How we built it ⚙️
The development of NeuroSpeak was a journey that combined expertise in EEG technology, machine learning, and natural language processing. We began by integrating EEG signal processing to capture and interpret brainwave patterns. This data was then used to interface with an AI-powered response generation system, utilizing OpenAI's GPT-4 for its advanced contextual understanding. Our development process involved rigorous testing to ensure seamless integration of these technologies, creating a user-friendly interface that could be operated on various devices including smartphones, tablets, and computers.
We drew significant inspiration from the updated Double-Diamond design process, incorporating a comprehensive research cycle. This approach entails uncovering and clearly defining the problem before embarking on the solution phase, ultimately leading to deployment.
## Design 🎨
Our focus is on translating our empathetic design into a tangible solution. We prioritize the development of an app with a simple and intuitive user interface (UI), specifically tailored to meet the unique communication needs of individuals experiencing locked-in syndrome. The emphasis is on ensuring that every interaction within the app is seamless and easily navigable, empowering users to express themselves effortlessly.
## Research 📚
Empathizing with users is crucial, and our project is built on the foundation of thorough research. Identifying our specific user group early on has set the course for our entire project. We drew valuable insights from various resources that played a significant role in shaping our approach.
- [link](https://news.northeastern.edu/2014/04/29/key-to-locked-in-syndrome/)
- [link](https://my.clevelandclinic.org/health/diseases/22462-locked-in-syndrome-lis)
- [link](https://www.merckmanuals.com/en-ca/professional/neurologic-disorders/coma-and-impaired-consciousness/locked-in-syndrome#:~:text=Patients%20with%20locked%2Din%20syndrome,or%20move%20their%20eyes%20laterally.)
## Challenges we ran into 😤
Our journey wasn't without challenges. One of the primary hurdles was ensuring the accuracy and responsiveness of EEG signal interpretation. We also faced complexities in integrating the GPT model to generate immediate, relevant responses. Ensuring device compatibility and creating an intuitive user interface posed additional challenges, as we aimed to make NeuroSpeak accessible to as many users as possible.
## Accomplishments that we're proud of ✨
We are immensely proud of creating a tool that empowers individuals with severe communication barriers. Seeing NeuroSpeak in action, enabling users to express their thoughts and engage in conversations, has been incredibly fulfilling. We’ve also successfully navigated the complex integration of EEG and AI technologies, a testament to our team's innovation and dedication.
## What we learned 🙌
This journey taught us a great deal about the intricacies of EEG technology, the capabilities of AI in real-world applications, and the profound impact technology can have on human lives. We've gained insights into the unique challenges faced by individuals with paralysis and locked-in syndrome, which have been invaluable in shaping NeuroSpeak.
## What's next for NeuroSpeak 🚀
NeuroSpeak's future involves global expansion through multilingual support and cultural adaptation, bolstered by partnerships in various regions. We're also developing a robust customer service system, incorporating user feedback for continuous app improvement. Technologically, we're enhancing AI capabilities for personalized response generation and introducing user profile customization. These steps aim to broaden NeuroSpeak's reach and enrich the communication experience for users worldwide.
