{"id":27692220,"url":"https://github.com/nagarajpandith/aido","last_synced_at":"2025-10-27T15:42:43.973Z","repository":{"id":191217524,"uuid":"684171602","full_name":"nagarajpandith/aido","owner":"nagarajpandith","description":"3D multilingual chatbot with interactive gestures, contextual conversations, persistent chat history, and personalized medical and mental health support.","archived":false,"fork":false,"pushed_at":"2023-08-31T10:21:51.000Z","size":22929,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-09-01T00:34:46.250Z","etag":null,"topics":["chatbot","cloud-run","googlecloud","mixamo","nextjs","readyplayerme","shadcn-ui","t3-stack","tailwindcss","threejs","translate","trpc"],"latest_commit_sha":null,"homepage":"https://aido-ar2ndw3szq-uc.a.run.app/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nagarajpandith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-28T15:47:58.000Z","updated_at":"2023-09-01T00:35:09.855Z","dependencies_parsed_at":"2023-09-01T00:45:19.422Z","dependency_job_id":null,"html_url":"https://github.com/nagarajpandith/aido","commit_stats":null,"previous_names":["nagarajpandith/aido-client","nagarajpandith/aido"],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nagarajpandith%2Faido","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nagarajpandith%2Faido/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nagarajpandith%2Faido/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nagarajpandith%2Faido/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nagarajpandith","download_url":"https://codeload.github.com/nagarajpandith/aido/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250808297,"owners_count":21490641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","cloud-run","googlecloud","mixamo","nextjs","readyplayerme","shadcn-ui","t3-stack","tailwindcss","threejs","translate","trpc"],"created_at":"2025-04-25T11:31:46.947Z","updated_at":"2025-10-27T15:42:38.929Z","avatar_url":"https://github.com/nagarajpandith.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://res.cloudinary.com/dpfpk49oa/image/upload/c_pad,b_auto:predominant,fl_preserve_transparency/v1693113157/medical-check_lqv20o.jpg?_s=public-apps\" width=\"100\"/\u003e\n\n# Aido\n🏆 Awarded the 'Most Innovative Approach' at [NivHack](https://niveussolutions.com/hackathon) 2023 hosted at Niveus Solutions Pvt. Ltd., Mangalore.\n\n\u003cimg src=\"https://github.com/nagarajpandith/aido-client/assets/83623339/a36de8a0-41f1-43bf-93b4-51e2516d046d\" width=\"500\" /\u003e\n\n- [Live URL](https://aido-ar2ndw3szq-uc.a.run.app/)\n- VertexAi API Microservice was built with Flask -- [Repo](https://github.com/NidheeshaT/vertexai-api)\n- Both of the repos were dockerized \u0026 deployed on [Google Cloud Run](https://cloud.google.com/run)\n- [Final Presentation Pitch Deck](https://www.canva.com/design/DAFstFPq9mU/VrI2Bq1hMvutvrdiTBLdRw/view?utm_content=DAFstFPq9mU\u0026utm_campaign=designshare\u0026utm_medium=link\u0026utm_source=publishsharelink)\n\n\u003e **NOTE**\n\u003e The deployments might be down right now as the Cloud access was given only during the hackathon.\n\n## Features\n- Virtually Interactive 3D Character with speech \u0026 hand gestures\n- Multilingual Chat Sessions in different languages \n- Follow-up Questions based on previous conversation \n- Persistent chat sessions considering chat history of user \n- Personalised medical query \u0026 mental health support \n- Multilingual Voice Chat Support\n\n## Architecture\n![image](https://github.com/nagarajpandith/aido-client/assets/83623339/bfe71acf-7b0a-497a-9c9a-4023ba835823)\n\n## Tech Stack\n- Google PaLM2 based on model chat-bison using Google vertex AI\n- Deployed both client \u0026 server on Google Cloud Run\n- Database on Cloud SQL (PostgreSQL)\n- Google OAuth\n- Deepgram STT \u0026 ElevenLabs TTS\n- CI/CD using Google Cloud source repositories \u0026 Cloud build\n- Google Translate\n- T3 Stack: Next.js, tRPC, Prisma, Typescript\n- Flask Backend API\n\n## Screenshots\n![4](https://github.com/nagarajpandith/aido-client/assets/83623339/4934cc2a-d8dc-4d2b-8371-e8ca0c04c060)\n![5](https://github.com/nagarajpandith/aido-client/assets/83623339/afa9cc57-4c06-4805-80a2-2339b699a3a1)\n![6](https://github.com/nagarajpandith/aido-client/assets/83623339/f259b79e-7cec-48ed-88bf-c75c647f5a61)\n![7](https://github.com/nagarajpandith/aido-client/assets/83623339/88cff44a-f68a-4dfc-8173-62b696387708)\n![8](https://github.com/nagarajpandith/aido-client/assets/83623339/a925f607-f0be-4c32-9766-bb04ca311547)\n\n## Challenges Faced\n- No finetuning available for chat-bison model\n- It was our first time using \u0026 configuring GCP \u0026 Gitlab\n- No Direct support of CI/CD of Gitlab on Cloud Run\n- Didn't get access to Google TTS \u0026 STT due to Quota Limit\n- The TTS generation \u0026 lipsyncing had a lot of latency\n\n## Future Scope \n- MedPaLM integration for better accuracy.\n- Using 3D model for other usecases like kids' trainer, lifestyle expert etc.\n- Break the sentences according to fullstops \u0026 punctuations before processing further (to avoid latency)\n- Can extend the concept to Metaverse domain\n\n## Team Members - t3-tribe\n- Nagaraj Pandith\n- Swasthik Shetty\n- Nidheesha T\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnagarajpandith%2Faido","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnagarajpandith%2Faido","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnagarajpandith%2Faido/lists"}