https://github.com/jacintogomez/google-hackathon-gemini
AI conversation bot for foreign language practice
https://github.com/jacintogomez/google-hackathon-gemini
flask google-gemini-ai google-translate-api google-vertex-ai openai-whisper pygame python
Last synced: 4 months ago
JSON representation
AI conversation bot for foreign language practice
- Host: GitHub
- URL: https://github.com/jacintogomez/google-hackathon-gemini
- Owner: jacintogomez
- Created: 2024-04-28T22:14:44.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-03-11T21:54:38.000Z (7 months ago)
- Last Synced: 2025-03-30T07:17:30.426Z (7 months ago)
- Topics: flask, google-gemini-ai, google-translate-api, google-vertex-ai, openai-whisper, pygame, python
- Language: HTML
- Homepage: https://google-hackathon-gemini.vercel.app
- Size: 125 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Google Hackathon - Foreign Language Practice Assistant
Language learning apps like Duolingo are great for early stage learning, but can fall short when someone wants to reach fluency. This leaves a gray area individuals can end up in where they have outgrown the language learning apps, but are still not ready yet to effectively converse with real native speakers in a non-practice setting.
This project is meant to address the issue by letting users have a verbal conversation with an AI bot in their chosen language. While there are obvious downsides to this, using an AI language model gives the user a playground so to speak where they can practice speech with a dummy conversation.
How to use:
1. Select your language (be aware that the less widely spoken languages will not have as well-trained models, and may give weird output at times).
2. Click "Record" and you will have 5 seconds to say what you want to the machine. After this the machine will resond, then you click record again and so on.
3. After the conversation ends the user can download a transcript of the conversation text to their device, in case there are any new words/phrases they would like to remember.
How it works:
- Google Gemini API is used to craft responses to the user input text. Google Vertex AI is used specifically so I could give a system instruction and so that chat history is remembered throughout the conversation
- Google Translate API is used to translate into and from English
- OpenAI Whisper is used to convert the user's speech to text
- Audio is played with Pygame
Features to add in the future:
- Alerts for grammatical mistakes
- Accent feedback and critiques
- Click text to show translation in English
- Nicer looking UI
# Steps to run:
1. Clone the repository
2. Create a .env file with the environment variables from the .env.template file (this will require a Google Cloud account)
3. Install requirements by running `pip install -r requirements.txt` in terminal
4. Run `python app.py` to start the server
5. Visit the localhost link shown in terminal