{"id":15173186,"url":"https://github.com/jorgeandrespadilla/ai-ruby-track-hackathon","last_synced_at":"2026-02-06T16:06:52.963Z","repository":{"id":253701008,"uuid":"843674980","full_name":"jorgeandrespadilla/ai-ruby-track-hackathon","owner":"jorgeandrespadilla","description":"ComplaintSense: Ruby Track Hackathon for Headstarter AI","archived":false,"fork":false,"pushed_at":"2024-08-19T01:07:31.000Z","size":3304,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-13T10:49:37.310Z","etag":null,"topics":["assemblyai","google-vision-api","langchain","nextjs","openai","pinecone","postgresql","supabase","tailwindcss","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jorgeandrespadilla.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-17T04:39:35.000Z","updated_at":"2024-08-19T01:26:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"7046764a-495e-4eff-b3c0-b5f5a8aa3354","html_url":"https://github.com/jorgeandrespadilla/ai-ruby-track-hackathon","commit_stats":null,"previous_names":["jorgeandrespadilla/ai-ruby-track-hackathon"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jorgeandrespadilla/ai-ruby-track-hackathon","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jorgeandrespadilla%2Fai-ruby-track-hackathon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jorgeandrespadilla%2Fai-ruby-track-hackathon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jorgeandrespadilla%2Fai-ruby-track-hackathon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jorgeandrespadilla%2Fai-ruby-track-hackathon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jorgeandrespadilla","download_url":"https://codeload.github.com/jorgeandrespadilla/ai-ruby-track-hackathon/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jorgeandrespadilla%2Fai-ruby-track-hackathon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29167870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-06T15:38:29.831Z","status":"ssl_error","status_checked_at":"2026-02-06T15:37:48.592Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["assemblyai","google-vision-api","langchain","nextjs","openai","pinecone","postgresql","supabase","tailwindcss","typescript"],"created_at":"2024-09-27T10:42:09.437Z","updated_at":"2026-02-06T16:06:52.957Z","avatar_url":"https://github.com/jorgeandrespadilla.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Ruby Track Hackathon for Headstarter AI\n\n## Introduction\nThis project is built for the Ruby Track of the Headstarter AI Hackathon. The project demonstrates skills in AI Engineering and Full Stack Development by building a multi-modal complaint management system for Ruby, a financial technology company. The system leverages AI to analyze and categorize customer complaints, integrates with a relational database, and employs a Retrieval-Augmented Generation (RAG) pipeline with a vector database for advanced complaint retrieval.\n\n## Project Levels\n\n### Level 1\n**Task**: Use an LLM API to determine if a call is a complaint and create a summary of the complaint.\n\n**Implementation**:\n- Use an LLM API (e.g., OpenAI's GPT-4) to analyze text data.\n- Identify if the text is a complaint.\n- Generate a summary of the complaint.\n\n### Level 2\n**Task**: Assign a product category and a sub-product category similar to the sample data to the new complaint and save it to the database of complaints.\n\n**Implementation**:\n- Use the LLM API to categorize the complaint.\n- Map the complaint to predefined product and sub-product categories.\n- Save the categorized complaint to a relational database.\n\n### Level 3\n**Task**: Build a RAG pipeline using a vector database. Given a voice recording, find the most relevant complaints based on what is said in the voice recording.\n\n**Implementation**:\n- Convert voice recordings to text using a speech-to-text API.\n- Use a vector database (e.g., Pinecone, Faiss) to store and retrieve complaint vectors.\n- Implement a RAG pipeline to find the most relevant complaints.\n\n### Level 4\n**Task**: Make the inputs multi-modal. Handle voice, text, video, and text+picture inputs to identify and categorize complaints.\n\n**Implementation**:\n- Extend the system to handle multiple input modes:\n  - **Voice**: Analyze voice recordings.\n  - **Text**: Analyze screenshots of social media posts.\n  - **Video**: Analyze video content for complaints.\n  - **Text + Picture**: Analyze text and accompanying images (e.g., email with a screenshot).\n\n## Technologies Used\n- **Programming Language**: TypeScript\n- **Frameworks**: Next.js (React), LangChain\n- **APIs**: OpenAI GPT-4, AssemblyAI Speech-to-Text API, Google Vision API\n- **Databases**: PostgreSQL (Relational Database), Pinecone (Vector Database)\n\n## Architecture Diagram\n![image](https://github.com/user-attachments/assets/dfb60c4c-70e8-4daf-8130-7d933cad6761)\n\n## Getting Started\n\n1. Clone the repository.\n2. Install the dependencies by running `npm install`.\n3. Create a copy of the `.env.local.example` file and rename it to `.env.local`. Fill in the environment variables with your Firebase project configuration.\n4. Run the development server by running `npm run dev`.\n5. Open [http://localhost:3000](http://localhost:3000) in your browser to see the application.\n\n## Database Migrations\n\nTo generate a new migration, run the following command:\n\n```bash\nnpm run db:generate\n```\n\nTo run the migrations, use the following command:\n\n```bash\nnpm run db:migrate\n```\n\nTo check the consistency of the migrations, run the following command:\n\n```bash\nnpm run db:check\n```\n\n\u003e For more information, see the [Drizzle Kit documentation](https://orm.drizzle.team/learn/tutorials/drizzle-with-supabase).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjorgeandrespadilla%2Fai-ruby-track-hackathon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjorgeandrespadilla%2Fai-ruby-track-hackathon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjorgeandrespadilla%2Fai-ruby-track-hackathon/lists"}