{"id":14117399,"url":"https://github.com/ntegrals/aura-voice","last_synced_at":"2025-05-14T20:02:21.708Z","repository":{"id":208927067,"uuid":"722811903","full_name":"ntegrals/aura-voice","owner":"ntegrals","description":"Aura is like Siri, but in your browser. An AI voice assistant optimized for low latency responses.","archived":false,"fork":false,"pushed_at":"2024-11-27T12:36:46.000Z","size":6282,"stargazers_count":1215,"open_issues_count":24,"forks_count":105,"subscribers_count":21,"default_branch":"main","last_synced_at":"2025-04-06T11:05:20.135Z","etag":null,"topics":["artificial-intelligence","elevenlabs","gpt-3","gpt-4","langchain","nextjs","openai","vercel","whisper","whisper-cpp"],"latest_commit_sha":null,"homepage":"https://voice.julianschoen.co","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ntegrals.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-24T02:31:09.000Z","updated_at":"2025-04-05T05:14:29.000Z","dependencies_parsed_at":"2024-01-07T23:26:39.504Z","dependency_job_id":"e2a279b8-c0a2-472f-b5b9-3094f0c9cdcf","html_url":"https://github.com/ntegrals/aura-voice","commit_stats":{"total_commits":44,"total_committers":6,"mean_commits":7.333333333333333,"dds":0.2727272727272727,"last_synced_commit":"a748b2e7d4be21d3f54c9cd595319876a5646c1e"},"previous_names":["ntegrals/voice-assistant-web","ntegrals/aura-voice"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntegrals%2Faura-voice","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntegrals%2Faura-voice/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntegrals%2Faura-voice/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ntegrals%2Faura-voice/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ntegrals","download_url":"https://codeload.github.com/ntegrals/aura-voice/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248724585,"owners_count":21151560,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","elevenlabs","gpt-3","gpt-4","langchain","nextjs","openai","vercel","whisper","whisper-cpp"],"created_at":"2024-08-14T18:01:16.234Z","updated_at":"2025-04-13T14:03:24.974Z","avatar_url":"https://github.com/ntegrals.png","language":"TypeScript","funding_links":["https://www.buymeacoffee.com/ntegrals"],"categories":["TypeScript","Browser-extensions"],"sub_categories":[],"readme":"\u003ca name=\"readme-top\"\u003e\u003c/a\u003e\n\n\u003cbr /\u003e\n\u003cdiv align=\"center\"\u003e\n\n\u003ch3 align=\"center\"\u003eSay hi to Aura 👋\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    Aura is a smart voice assistant optimized for low latency responses. It uses Vercel Edge Functions, Whisper Speech Recognition, GPT-4o and Eleven Labs TTS streaming.\n    \u003cbr /\u003e\n    \u003cbr /\u003e\n    \u003ca href=\"https://voice.julianschoen.co\"\u003eView Demo\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/ntegrals/aura-voice/issues/new?assignees=\u0026labels=bug\u0026projects=\u0026template=bug_report.md\u0026title=\"\u003eReport Bug\u003c/a\u003e\n    ·\n    \u003ca href=\"https://github.com/ntegrals/aura-voice/issues/new?assignees=\u0026labels=enhancement\u0026projects=\u0026template=feature_request.md\u0026title=\"\u003eRequest Feature\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/div\u003e\n\u003ca href=\"https://github.com/ntegrals/aura-voice\"\u003e\n    \u003cimg src=\".assets//header.png\" alt=\"Logo\"\u003e\n  \u003c/a\u003e\n\n## Features\n\n    ✅ A Siri-like voice assistant within your browser\n    ✅ Optimized for low latency responses\n    ✅ With the combined power of OpenAI, Whisper Speech Recognition and Eleven Labs\n\n## Demo\n\nYou can test the Aura here: [https://voice.julianschoen.co](https://voice.julianschoen.co)\n\n## Motivation\n\nVoice Assistants have become an integral part of our lives. They are everywhere. In our phones, in our cars, in our homes. Why not also on the web?\n\nUntil recently the main problem with voice assistants on the web was the latency. It took too long to send the audio to the server, generate an LLM completion and send speech back. The recent advances of OpenAI, Eleven Labs and Vercel have made it possible to build a voice assistant that is fast enough to be used on the web.\n\nI would love to for this repo to become the go-to place for people who want to build their own voice assistant. I've been working on this project for a while now and I'm really excited to share it with you.\n\n## Thoughts on latency and user experience\n\nThe latency of the voice assistant is the most important factor for a good user experience. Currently there are 3 main factors that contribute to the latency:\n\n- The time it takes to transcribe the audio (Via Whisper Speech Recognition)\n- The time it takes to generate the response (Via GPT-4o Mini)\n- The time it takes to stream the speech response (Via Eleven Labs TTS)\n\nBased on some tests I've done, the speech generation takes the most time. The longer the text to be synthesized, the longer it takes to generate the speech. The latency of the speech generation is also the most unpredictable.\n\nA possible mitigation strategy might be splitting the response into multiple parts and streaming them one after another. This would allow the user to start listening to the response while the rest of the response is being generated. I haven't implemented this yet, but it's something I'm considering. If you have any ideas on how to improve the latency, please let me know.\n\nAnother thing to keep in mind is perceived wait time. Based on some research, it seems that the perceived wait time is shorter if the user is given some kind of feedback while waiting. I've implemented a simple \"thinking\" notification that is shown while the assistant is processing the response, but I'm sure there are better ways to improve the perceived wait time.\n\n## Installation\n\n1. Clone the repo\n\n   ```sh\n   git clone https://github.com/ntegrals/aura-voice\n   ```\n\n2. Get an API Key from [https://openai.com/](https://openai.com/) and [https://elevenlabs.com/](https://elevenlabs.com/)\n\n   Copy the .env.example file to .env.local and add your API keys\n\n   ```sh\n   OPENAI_API_KEY=\"YOUR OPENAI API KEY\"\n   OPENAI_BASE_URL=(Optional)\n   NEXT_PUBLIC_ELEVENLABS_API_KEY=\"YOUR ELEVENLABS API KEY\"\n   NEXT_PUBLIC_ELEVENLABS_VOICE_ID=\"YOUR ELEVENLABS VOICE ID\"\n   ```\n\n3. Install the dependencies\n\n   ```sh\n   npm install\n   ```\n\n4. Run the app\n   ```sh\n   npm run dev\n   ```\n5. Deploy to vercel\n\n## Contact\n\nHi! Thanks for checking out and using this library. If you are interested in discussing your project, require mentorship, consider hiring me, or just wanna chat - I'm happy to talk.\n\nYou can send me an email to get in touch: j.schoen@mail.com or message me on Twitter: [@julianschoen](https://twitter.com/julianschoen)\n\nIf you'd just want to give something back, I've got a Buy Me A Coffee account:\n\n\u003ca href=\"https://www.buymeacoffee.com/ntegrals\"\u003e\n\u003cimg src=\".assets/buymeacoffee.png\" alt=\"buymeacoffee\" width=\"192\"\u003e\n\u003c/a\u003e\n\nThanks and have an awesome day 👋\n\n## Disclaimer\n\nVoice Assistant, is an experimental application and is provided \"as-is\" without any warranty, express or implied. By using this software, you agree to assume all risks associated with its use, including but not limited to data loss, system failure, or any other issues that may arise.\n\nThe developers and contributors of this project do not accept any responsibility or liability for any losses, damages, or other consequences that may occur as a result of using this software. You are solely responsible for any decisions and actions taken based on the information provided by Voice Assistant.\n\nPlease note that the use of the GPT-4 language model can be expensive due to its token usage. By utilizing this project, you acknowledge that you are responsible for monitoring and managing your own token usage and the associated costs. It is highly recommended to check your OpenAI API usage regularly and set up any necessary limits or alerts to prevent unexpected charges.\n\nBy using Voice Assistant, you agree to indemnify, defend, and hold harmless the developers, contributors, and any affiliated parties from and against any and all claims, damages, losses, liabilities, costs, and expenses (including reasonable attorneys' fees) arising from your use of this software or your violation of these terms.\n\n\u003c!-- LICENSE --\u003e\n\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntegrals%2Faura-voice","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fntegrals%2Faura-voice","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fntegrals%2Faura-voice/lists"}