{"id":30385336,"url":"https://github.com/the-data-dilemma/groqstreamchain","last_synced_at":"2025-08-21T02:14:25.446Z","repository":{"id":291621399,"uuid":"978211273","full_name":"The-Data-Dilemma/GroqStreamChain","owner":"The-Data-Dilemma","description":"GroqStreamChain is a real-time AI-powered chat app using FastAPI, WebSocket, and Groq. It streams AI responses for interactive, low-latency communication with session management and a clean, responsive frontend. Ideal for building AI-driven chatbots.","archived":false,"fork":false,"pushed_at":"2025-05-07T14:16:59.000Z","size":3758,"stargazers_count":32,"open_issues_count":0,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-25T01:37:20.474Z","etag":null,"topics":["ai","chat-gpt","fastapi","groq-api","groq-integration","langchain","llama3","llm","python","real-time","streaming","websocket"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/The-Data-Dilemma.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-05-05T16:31:07.000Z","updated_at":"2025-07-24T07:46:51.000Z","dependencies_parsed_at":"2025-05-05T18:01:11.712Z","dependency_job_id":"77fe4181-3b77-46d4-9dbe-7be650a9f473","html_url":"https://github.com/The-Data-Dilemma/GroqStreamChain","commit_stats":null,"previous_names":["pr0mila/groqstreamchain","the-data-dilemma/groqstreamchain"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/The-Data-Dilemma/GroqStreamChain","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/The-Data-Dilemma%2FGroqStreamChain","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/The-Data-Dilemma%2FGroqStreamChain/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/The-Data-Dilemma%2FGroqStreamChain/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/The-Data-Dilemma%2FGroqStreamChain/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/The-Data-Dilemma","download_url":"https://codeload.github.com/The-Data-Dilemma/GroqStreamChain/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/The-Data-Dilemma%2FGroqStreamChain/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271415496,"owners_count":24755639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","chat-gpt","fastapi","groq-api","groq-integration","langchain","llama3","llm","python","real-time","streaming","websocket"],"created_at":"2025-08-21T02:14:24.987Z","updated_at":"2025-08-21T02:14:25.436Z","avatar_url":"https://github.com/The-Data-Dilemma.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Python Version](https://img.shields.io/badge/python-3.9%2B-blue.svg)](https://www.python.org/downloads/)\n[![FastAPI](https://img.shields.io/badge/FastAPI-async%20framework-green)](https://fastapi.tiangolo.com/)\n[![WebSocket](https://img.shields.io/badge/WebSocket-Real--Time-orange)](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)\n[![LangChain](https://img.shields.io/badge/LangChain-Integration-yellowgreen)](https://www.langchain.com/)\n[![Groq](https://img.shields.io/badge/Groq-LPU--Powered-ff69b4)](https://groq.com/)\n[![MIT License](https://img.shields.io/github/license/pr0mila/GroqStreamChain)](https://github.com/pr0mila/GroqStreamChain/blob/main/LICENSE)\n[![Open Issues](https://img.shields.io/github/issues/pr0mila/GroqStreamChain)](https://github.com/pr0mila/GroqStreamChain/issues)\n[![Last Commit](https://img.shields.io/github/last-commit/pr0mila/GroqStreamChain)](https://github.com/pr0mila/GroqStreamChain/commits/main)\n# GroqStreamChain: Real-Time AI Chat System\n\n## Overview\n\n**GroqStreamChain** is a real-time, AI-powered chat application that uses WebSocket for low-latency communication and streams responses from a language model (LLM) powered by Groq and LangChain. The system is designed to provide a seamless conversational experience with real-time feedback and response streaming.\nHere is the final view of the **GroqStreamChain** chat application:\n\n![Chat App Final View](static/groqstreamchain.gif)\n\n## Project Structure\n\nThe system is organized into separate modules, each with specific responsibilities:\n\n- **`server.py`**: Main FastAPI application with WebSocket handling.\n- **`config.py`**: Configuration management (e.g., API keys, model settings).\n- **`models/chat.py`**: Data models for chats and messages.\n- **`services/llm_service.py`**: LLM service using Groq and LangChain for AI-powered responses.\n- **Frontend Files**: HTML, CSS, and JS files for the user interface.\n\n## Key Features\n\n1. **WebSocket Communication**: Provides real-time, low-latency communication between the client and server.\n2. **Streaming Responses**: The system streams LLM responses chunk by chunk to the client, improving perceived latency and enhancing the user experience.\n3. **Session Management**: Automatically creates and maintains unique chat sessions for each user interaction.\n4. **Clean Frontend**: A simple, responsive UI with typing indicators and message formatting for a smooth user experience.\n5. **Error Handling**: Handles WebSocket disconnections and reconnection attempts gracefully.\n\n## How to Run the Project\n\n### 1. Clone the project:\n\nFirst, clone the project from GitHub using the following command:\n\n```bash\ngit clone https://github.com/pr0mila/GroqStreamChain.git\n```\n\n### 2. Install the required dependencies:\n\n#### Create a Conda Environment (optional but recommended)\n\n```bash\nconda create --name streamchain python=3.9\nconda activate streamchain\n```\n\n#### Install dependencies via `requirements.txt`:\n\n```bash\npip install -r requirements.txt\n```\n\nOr install dependencies manually using `pip`:\n\n```bash\npip install fastapi uvicorn websockets python-dotenv groq langchain langchain-groq pydantic jinja2\n```\n\n### 3. Set up environment variables:\n\nCreate a `.env` file in the project directory and set the following variables:\n\n```bash\nGROQ_API_KEY=your_groq_api_key_here\nMODEL_NAME=llama-3.1-8b-instant\n```\n\nMake sure to replace `your_groq_api_key_here` with your actual Groq API key.\n\n### 4. Run the server:\n\n```bash\npython server.py\n```\n\nThis will start the FastAPI server, and you can access the application via the browser at:\n\n[http://localhost:8000](http://localhost:8000)\n\n\n\n\n## Running the Test Script (`test_groq.py`)\n\nTo test if the Groq API is working properly, you can run the test script `test_groq.py`. This script will send a test request to the Groq API and check if the response is correctly received.\n\n### Steps to Run `test_groq.py`:\n\n1. Make sure you have set up the `.env` file with your **Groq API key** and **model name**.\n\n2. Run the following command to execute the test:\n\n    ```bash\n    python test_groq.py\n    ```\n\n3. The script will print out the response from the Groq API. If the API is working properly, you should see a successful connection and the response from the model.\n\n4. If there are any issues with the API key or connectivity, the script will log an error message.\n\n\n## Performance Considerations\n\n- **WebSocket Connection**: The system maintains a persistent WebSocket connection to minimize connection overhead and ensure continuous communication between the client and server.\n- **Streaming Responses**: The system streams LLM responses as they are generated, showing partial responses to the user immediately, reducing perceived latency.\n- **Minimal Processing**: The server-side processing is kept to a minimum, relying on the Groq client to handle the bulk of the work.\n- **Efficient Message Handling**: Direct interaction with the Groq client ensures efficient token streaming\n\n## Blog Post\n\nFor a detailed explanation and walkthrough of the project, check out the blog post on Medium:\n\n[GroqStreamChain: Building a Real-Time AI Chat Application](https://medium.com/@promilaghoshmonty/groqstreamchain-chat-application-pngbuilding-491004f3e25d)\n\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthe-data-dilemma%2Fgroqstreamchain","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthe-data-dilemma%2Fgroqstreamchain","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthe-data-dilemma%2Fgroqstreamchain/lists"}