{"id":15063735,"url":"https://github.com/debapriya-source/llama-3.1-chatbot","last_synced_at":"2026-01-05T06:47:36.882Z","repository":{"id":256000006,"uuid":"854078916","full_name":"Debapriya-source/llama-3.1-chatbot","owner":"Debapriya-source","description":"This is a chatbot using llama-3.1-8b-instant model powered by groq, which is specifically designed to assist users to learn, understand and practice basic to complex coding problems.","archived":false,"fork":false,"pushed_at":"2024-09-25T09:17:19.000Z","size":18,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-03T00:16:31.984Z","etag":null,"topics":["artificial-intelligence","groq-api","llama3-1","streamlit"],"latest_commit_sha":null,"homepage":"https://the-tech-buddy.streamlit.app/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Debapriya-source.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-08T11:02:27.000Z","updated_at":"2025-01-14T16:44:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"49cb5596-a756-4e65-9417-f6a1ef409389","html_url":"https://github.com/Debapriya-source/llama-3.1-chatbot","commit_stats":null,"previous_names":["debapriya-source/llama-3.1-chatbot"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-chatbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-chatbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-chatbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Debapriya-source%2Fllama-3.1-chatbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Debapriya-source","download_url":"https://codeload.github.com/Debapriya-source/llama-3.1-chatbot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239236414,"owners_count":19604901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","groq-api","llama3-1","streamlit"],"created_at":"2024-09-25T00:06:39.662Z","updated_at":"2026-01-05T06:47:36.829Z","avatar_url":"https://github.com/Debapriya-source.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🦙Llama-3.1-chatbot🤖 powered by Groq hosted on Streamlit\n\nIn this tutorial, we'll build and deploy a personalised AI-powered chat application using Streamlit and the latest AI model llama-3.1-8b-instant. We'll use Groq for faster inference. Also we are going to **deploy it for free!**\nWe'll take you through the code, explaining each section and providing useful tips for customization.\n\n## Getting Started\n\nFirst sign in to [https://groq.com/](https://groq.com/) and click `start building`\n![Groq API](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wim3kjcetmo0vvrd2asr.png)\n\nClick on `Create API key` then create a new key, copy it and keep it somewhere safe.\n\n![Groq API Key](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/elwent9dzc33vyg14ai2.png)\n\nNow install the necessary libraries:\nCreate the requirements.txt file and paste this\n\n```txt\ngroq==0.9.0\nstreamlit==1.37.0\npython-dotenv\n```\n\nInstall these using\n\n```bash\npip install -r requirements.txt\n```\n\nLet's create our `main.py` file and import the required libraries:\n\n```python\nimport os\nfrom dotenv import dotenv_values\nimport streamlit as st\nfrom groq import Groq\n```\n\nWe'll use streamlit for building the chat interface, dotenv for handling environment variables, and groq for fast inference from the AI model.\n\n## Configuring the Page\n\nLet's set up the page configuration using Streamlit:\n\n```python\nst.set_page_config(\n    page_title=\"The Tech Buddy \",\n    page_icon=\"\",\n    layout=\"centered\",\n)\n```\n\nThis will give our chat application a professional look and feel.\n\n## Handling Environment Variables\n\nWe'll use environment variables to store sensitive information like API keys and and the application specific prompts.\nIn your root folder create a `.env` file like this:\n\n```env\nGROQ_API_KEY='YOUR_GROQ_API_KEY'\n\nINITIAL_RESPONSE=\"Enter what you want to show as the first response of your bot, example: Hello! my friend I am a painter from 70's. Whatsup?\"\n\nCHAT_CONTEXT=\"Enter how do you want to personalize your chatbot, example: You are a painter from the 70's and you are respond sentences with painting references.(This is for the system)\"\n\nINITIAL_MSG=\"Enter the first message from the assistant to initiate the chat history, example: Hey there! I know everything about painting, ask me anything.(This is for the assistant)\"\n```\n\nThis part is crucial to personalize your application as per your need. So play with it and explore.\n\nNow configure this environment variables in our python file:\n\n```python\ntry:\n    secrets = dotenv_values(\".env\")  # for dev env\n    GROQ_API_KEY = secrets[\"GROQ_API_KEY\"]\nexcept:\n    secrets = st.secrets  # for streamlit deployment\n    GROQ_API_KEY = secrets[\"GROQ_API_KEY\"]\n\n# Save the API key to environment variable\nos.environ[\"GROQ_API_KEY\"] = GROQ_API_KEY\n\nINITIAL_RESPONSE = secrets[\"INITIAL_RESPONSE\"]\nINITIAL_MSG = secrets[\"INITIAL_MSG\"]\nCHAT_CONTEXT = secrets[\"CHAT_CONTEXT\"]\n```\n\nIn the `try block` we are getting the environment variables from the `.env` file to run it and test it locally.\nBut when we'll deploy it using streamlit we will not get any access of the `.env` file. So that time we will store our secrets using streamlit and to access those secrets we will use `st.secretes` that returns a python `dict`, same like `dotenv_values(\".env\")`. So after deployment the `except block` gets executed.\n\n## Initializing the Chat Application\n\nLet's set up the chat history and initialize the AI model:\n\n- Copy your favourite AI-model's `Model ID` from [https://console.groq.com/docs/models](https://console.groq.com/docs/models):\n\n![Groq supported models](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2iake7ty12ygdud3vgmd.png)\n\nI used `llama-3.1-8b-instant` for my project.\n\n- Initialize your model:\n\n```python\n# Initialize the chat history if present as Streamlit session\nif \"chat_history\" not in st.session_state:\n    st.session_state.chat_history = [\n        {\"role\": \"assistant\",\n         \"content\": INITIAL_RESPONSE\n         },\n    ]\n\nclient = Groq()\n```\n\nWe'll store the chat history in the st.session_state object, which allows us to persist data across session refreshes.\n\n## Displaying the Chat Application\n\nLet's create the chat interface using Streamlit:\n\n```python\n# Page title\nst.title(\"Hey Buddy!\")\nst.caption(\"Let's go back in time...\")\n\n# Display chat history\nfor message in st.session_state.chat_history:\n    with st.chat_message(\"role\", avatar=''):\n        st.markdown(message[\"content\"])\n```\n\nWe'll use the st.chat_message function to display each message in the chat history.\n\n## User Input Field\n\nLet's create a text input field for the user to enter their question:\n\n```python\nuser_prompt = st.chat_input(\"Let's chat!\")\n```\n\nWhen the user submits their prompt, we'll append it to the chat history and generate a response from the AI model.\n\n## Generating a Response from the AI Model\n\nLet's create a response from the AI model using the Groq library:\n\n```python\ndef parse_groq_stream(stream):\n    for chunk in stream:\n        if chunk.choices:\n            if chunk.choices[0].delta.content is not None:\n                yield chunk.choices[0].delta.content\n\nif user_prompt:\n    with st.chat_message(\"user\", avatar=\"\"):\n        st.markdown(user_prompt)\n    st.session_state.chat_history.append(\n        {\"role\": \"user\", \"content\": user_prompt})\n\n    messages = [\n        {\"role\": \"system\", \"content\": CHAT_CONTEXT\n         },\n        {\"role\": \"assistant\", \"content\": INITIAL_MSG},\n        *st.session_state.chat_history\n    ]\n\n    stream = client.chat.completions.create(\n        model=\"llama-3.1-8b-instant\",\n        messages=messages,\n        stream=True  # for streaming the message\n    )\n    response = st.write_stream(parse_groq_stream(stream))\n    st.session_state.chat_history.append(\n        {\"role\": \"assistant\", \"content\": response})\n```\n\nWe'll use the `client.chat.completions.create()` method to generate a steam and then parse it to a actual response from the AI model, and then append it to the chat history.\n\n## Run it locally\n\nCongratulations! You've built a personalised AI-powered chat application using Streamlit, Groq, and a llama-3.1-8b-instant model.\n\nHere is the whole `main.py` file:\n\n```python\nimport os\nfrom dotenv import dotenv_values\nimport streamlit as st\nfrom groq import Groq\n\n\ndef parse_groq_stream(stream):\n    for chunk in stream:\n        if chunk.choices:\n            if chunk.choices[0].delta.content is not None:\n                yield chunk.choices[0].delta.content\n\n\n# streamlit page configuration\nst.set_page_config(\n    page_title=\"The 70's Painter\",\n    page_icon=\"🎨\",\n    layout=\"centered\",\n)\n\n\ntry:\n    secrets = dotenv_values(\".env\")  # for dev env\n    GROQ_API_KEY = secrets[\"GROQ_API_KEY\"]\nexcept:\n    secrets = st.secrets  # for streamlit deployment\n    GROQ_API_KEY = secrets[\"GROQ_API_KEY\"]\n\n# save the api_key to environment variable\nos.environ[\"GROQ_API_KEY\"] = GROQ_API_KEY\n\nINITIAL_RESPONSE = secrets[\"INITIAL_RESPONSE\"]\nINITIAL_MSG = secrets[\"INITIAL_MSG\"]\nCHAT_CONTEXT = secrets[\"CHAT_CONTEXT\"]\n\n\nclient = Groq()\n\n# initialize the chat history if present as streamlit session\nif \"chat_history\" not in st.session_state:\n    # print(\"message not in chat session\")\n    st.session_state.chat_history = [\n        {\"role\": \"assistant\",\n         \"content\": INITIAL_RESPONSE\n         },\n    ]\n\n# page title\nst.title(\"Hey Buddy!\")\nst.caption(\"Let's go back in time...\")\n# the messages in chat_history will be stored as {\"role\":\"user/assistant\", \"content\":\"msg}\n# display chat history\nfor message in st.session_state.chat_history:\n    # print(\"message in chat session\")\n    with st.chat_message(\"role\", avatar='🤖'):\n        st.markdown(message[\"content\"])\n\n\n# user input field\nuser_prompt = st.chat_input(\"Ask me\")\n\nif user_prompt:\n    # st.chat_message(\"user\").markdown\n    with st.chat_message(\"user\", avatar=\"🗨️\"):\n        st.markdown(user_prompt)\n    st.session_state.chat_history.append(\n        {\"role\": \"user\", \"content\": user_prompt})\n\n    # get a response from the LLM\n    messages = [\n        {\"role\": \"system\", \"content\": CHAT_CONTEXT\n         },\n        {\"role\": \"assistant\", \"content\": INITIAL_MSG},\n        *st.session_state.chat_history\n    ]\n\n    # Display assistant response in chat message container\n    with st.chat_message(\"assistant\", avatar='🤖'):\n        stream = client.chat.completions.create(\n            model=\"llama-3.1-8b-instant\",\n            messages=messages,\n            stream=True  # for streaming the message\n        )\n        response = st.write_stream(parse_groq_stream(stream))\n    st.session_state.chat_history.append(\n        {\"role\": \"assistant\", \"content\": response})\n\n```\n\nTo run it locally enter the following command in your terminal:\n\n```bash\nstreamlit run main.py\n```\n\n## Deployment\n\nWe are now all set to deploy our app.\nFirst upload the codebase in a [GitHub](https://github.com/) repository.\nThen [click here](https://streamlit.io/) to sign in to your streamlit account and go to `My Apps` section:\n\n- Click on `Create app` at the upper right corner.\n  ![Create streamlit app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kvk4x89migvt3nq17rc9.png)\n\n- Click on first option:\n  ![streamlit create app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ceimwm3jfu4l68je9526.png)\n\n- Locate your github repository:\n  ![streamlit create app - locate your github repo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/luia6wean5tbgeg3kr35.png)\n\n- Locate the your `main.py` file:\n  ![streamlit create app - locate your app file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mh7rv0fbj68bebxykift.png)\n\n- Create a custom url for your deployed app(optional):\n  ![streamlit create app - create custom url](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dxswptob2a5y1y33hg2x.png)\n\n- Click on `additional settings` and paste everything from your `.env` file (this is the `st.secrets`):\n  ![streamlit create app - configure the secrets](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oeqstxp26670fhsra8nd.png)\n\n- Click on deploy:\n  ![streamlit create app - deploy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tydebdjx9kzxa8cpqlik.png)\n\nCongrats! you have successfully deployed your own personalised AI app for free.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebapriya-source%2Fllama-3.1-chatbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdebapriya-source%2Fllama-3.1-chatbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebapriya-source%2Fllama-3.1-chatbot/lists"}