{"id":26034266,"url":"https://github.com/charlysan/llm_fastapi","last_synced_at":"2026-04-11T17:02:40.271Z","repository":{"id":281065826,"uuid":"944092671","full_name":"charlysan/llm_fastapi","owner":"charlysan","description":"LLM Assistant API Example built using Grok3 and Claude3.5 Sonnet","archived":false,"fork":false,"pushed_at":"2025-03-06T19:37:33.000Z","size":813,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-06T20:26:46.642Z","etag":null,"topics":["fastapi","grok3","llm","ollama","openai"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/charlysan.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-06T19:17:34.000Z","updated_at":"2025-03-06T19:41:37.000Z","dependencies_parsed_at":"2025-03-06T20:37:08.288Z","dependency_job_id":null,"html_url":"https://github.com/charlysan/llm_fastapi","commit_stats":null,"previous_names":["charlysan/llm_fastapi"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlysan%2Fllm_fastapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlysan%2Fllm_fastapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlysan%2Fllm_fastapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charlysan%2Fllm_fastapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/charlysan","download_url":"https://codeload.github.com/charlysan/llm_fastapi/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242320961,"owners_count":20108473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fastapi","grok3","llm","ollama","openai"],"created_at":"2025-03-07T02:49:45.594Z","updated_at":"2025-12-04T18:11:00.410Z","avatar_url":"https://github.com/charlysan.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM FastAPI Assistant APP\n(Generated from scratch with Grok-3 prompts and refined with Claude 3.5 Sonnet)\n\nThis is a simple FastAPI application that generates AWS CLI commands based on user requests using OpenAI or Ollama LLMs.\n\n\n![web](./img/web.gif)\n\n\n## Disclaimer\n\nThis app is an example of how LLMs can assist with tasks that involve converting user requests into actions or commands.  \nAll of the code in this repository was generated by Grok-3 and then refined using Claude 3.5 Sonnet in Cursor IDE. Additionally, about 90% of the documentation has been generated by an LLM.\n\nPlease note that this app is not intended for production use. Rather, it serves as a starting point for creating your own implementation tailored to a specific use case.\n\n## Table of Contents\n- [Disclaimer](#disclaimer)\n- [Prerequisites](#prerequisites)\n- [Setup and Running the App](#setup-and-running-the-app)\n  - [Configure Environment Variables](#configure-environment-variables)\n  - [Build and Run the App](#build-and-run-the-app)\n- [Web UI](#web-ui)\n- [Endpoints](#endpoints)\n- [Notes](#notes)\n- [How It Works](#how-it-works)\n- [Directory Structure](#directory-structure)\n- [Testing the App](#testing-the-app)\n- [Grok-3 Prompts](#grok-3-prompts)\n\n\n## Prerequisites\n\n- Docker and Docker Compose installed on your machine.\n- An OpenAI API key (if using OpenAI).\n- Access to an Ollama instance (if using Ollama).\n\n## Setup and Running the App\n\n1. **Configure Environment Variables**\n\n   Copy the example environment file and edit it:\n   ```bash\n   cp .env.example .env\n   ```\n   Then edit the `.env` file to update your configuration, especially the `OPENAI_TOKEN` which is required if using OpenAI.\n\n   For `OLLAMA_BASE_URL`, use the base URL of your Ollama instance (e.g. MacOS: `http://host.docker.internal:11434`, Linux: `http://172.17.0.1:11434`)\n\n2. **Build and Run the App**\n\n   Run the following command in the directory containing `docker-compose.yml`:\n\n   ```bash\n   docker-compose up --build\n   ```\n\n\nThe app will start and be available at http://localhost:8086.\n\n## Web UI\nVisit http://localhost:8086/ in your browser to access the web UI:\n- Ticket Request: Enter your AWS access request (e.g., \"access S3 for bucket /media\").\n- LLM: Select \"openai\" or \"ollama\".\n- Model: Enter the model name (e.g., \"gpt-4o-mini\").\n- User ARN: Enter the IAM user ARN (e.g., \"arn:aws:iam::123456789012:user/example\").\n- Submit: Click the button to get the AWS CLI command and notes displayed below the form.\n\n\n\n## Endpoints\n\n### GET /health\nDescription: Checks if the app is running.\n\nResponse: \n```json\n{\"status\": \"ok\"}\n```\n\nPOST /tickets\nDescription: Processes a user request to generate an AWS CLI command.\n\nRequest Body:\n```json\n{\n  \"llm\": \"openai\",           // or \"ollama\"\n  \"model\": \"gpt-4o-mini\",    // model name\n  \"request\": \"access S3 for resource for bucket /media\"\n}\n```\n\nResponse:\n```json\n{\n  \"model\": \"gpt-4o-mini\",\n  \"llm\": \"openai\",\n  \"tokens\": 150,\n  \"command\": \"aws s3api ...\",\n  \"notes\": \"Ensure the bucket exists.\"\n}\n```\n\n\n## Notes\n- The app assumes Ollama has an `/api/generate` endpoint. Adjust the `call_ollama` function if the API differs.\n- Token counts are included if provided by the LLM; otherwise, they may be null.\n- Ensure environment variables are set correctly to avoid runtime errors.\n\n\n## How It Works\n\n1. **Endpoints**:\n   - `GET /health`: Returns a simple status check.\n   - `POST /tickets`: Accepts a JSON payload, processes the request using the specified LLM, and returns an AWS CLI command with additional details.\n\n2. **LLM Integration**:\n   - Uses `requests` to call OpenAI or Ollama APIs based on the `llm` field.\n   - OpenAI uses the `/chat/completions` endpoint; Ollama assumes an `/api/generate` endpoint (adjust as needed).\n\n3. **Configurable Prompt**:\n   - The prompt is set via the `PROMPT_TEMPLATE` environment variable and includes placeholders for `account_id`, `user_iam_id`, `region`, and `request`.\n\n4. **Environment Variables**:\n   - `OPENAI_TOKEN`, `OLLAMA_BASE_URL`, and `OPENAI_BASE_URL` configure LLM access.\n   - AWS-related variables (`AWS_ACCOUNT_ID`, `AWS_USER_IAM_ID`, `AWS_REGION`) are used in the prompt.\n\n5. **Deployment**:\n   - The app runs on port 8086 inside a Docker container built from a Python Alpine image.\n   - Docker Compose simplifies running the app with pre-set environment variables.\n\n## Directory Structure\n\nAfter creating all files, your directory should look like this:\n\n```\n.\n├── Dockerfile\n├── LICENSE\n├── Readme.md\n├── docker-compose.yaml\n├── img\n│   └── web.gif\n├── main.py\n├── requirements.txt\n└── templates\n    └── index.html\n```\n\n### Testing the App\n\n1. Start the app with `docker-compose up --build`.\n2. Test the health endpoint:\n```bash\n   curl http://localhost:8086/health\n```\n\nExpected output:\n```json\n{\"status\": \"ok\"}\n```\n\nTest the tickets endpoint:\n\n```bash\ncurl -X POST http://localhost:8086/tickets \\\n-H \"Content-Type: application/json\" \\\n-d '{\"llm\": \"openai\", \"model\": \"gpt-4o-mini\", \"request\": \"Hi, I need access to S3 for resource for bucket /media. Thanks!\", \"user_arn\": \"arn:aws:iam::123456789012:user/example\"}'\n```\n\nExpected output:\n\n```json\n{\n    \"model\": \"gpt-4o-mini\",\n    \"llm\": \"openai\",\n    \"tokens\": 265,\n    \"command\": \"aws iam attach-user-policy --user-name example --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess\",\n    \"notes\": \"- This command grants the user named \\\"example\\\" full access to all S3 resources, including the specified bucket \\\"/media\\\".\\n- If you want to restrict access to only the \\\"/media\\\" bucket, consider creating a custom IAM policy that specifically allows actions on that bucket only, instead of using the managed policy `AmazonS3FullAccess`.\"\n}\n```\n\n## Grok-3 Prompts\n\nThis is the initial prompt used to generate the code:\n\n```\nLet's build a simple python app using FastAPI. \nThe app should expose two endpoints:GET /health -\u003e it should return 200\nPOST /tickets\nThis endpoint will receive a json payload:\n\n{\n    \"llm\": \"ollama/openai\",\n    \"model\": \"gpt-4o-mini\",\n    \"request\": \"\u003cuser_request\u003e\"\n}\n\n\u003cuser_request\u003e is a text, a ticket created by some user asking for certain AWS access \n(for example the user asks to access S3 for resource for bucket /media). \nYou should take that text and hit openai or ollama (use python Requests), \nand the llm should return the aws-cli command that the admin should run to accomplish that.\nThe endpoint should return a json response like this:\n\n{\n    \"model\": \"model used\",\n    \"llm\": \"llm_used\",\n    \"tokens\": \"tokens used (if available)\",\n    \"command\": \"aws command\",\n    \"notes\": \"extra notes if available\"\n}\n\nAdd a configurable prompt for the llm that takes into account the AWS account_id, \nuser_iam_id, aws_region. Add env var for openai_token, ollama_base_url, openai_base_url. \nCreate a single python file with everything, a dockerfile that uses python apline image, \nand a docker-compose file that I can use to run the app. The app should use port 8086. \nAdd a readme.md file with simple instructions on how to run the app.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharlysan%2Fllm_fastapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharlysan%2Fllm_fastapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharlysan%2Fllm_fastapi/lists"}