{"id":14964819,"url":"https://github.com/devxt-llc/ezlocalai","last_synced_at":"2026-02-21T16:03:24.361Z","repository":{"id":197825989,"uuid":"699411012","full_name":"DevXT-LLC/ezlocalai","owner":"DevXT-LLC","description":"ezlocalai is an easy to set up local artificial intelligence server with OpenAI Style Endpoints.","archived":false,"fork":false,"pushed_at":"2025-02-14T02:40:46.000Z","size":156640,"stargazers_count":83,"open_issues_count":4,"forks_count":16,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-28T17:57:28.895Z","etag":null,"topics":["ai","artificial-intelligence","cuda","llamacpp","local"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DevXT-LLC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Josh-XT"],"ko_fi":"joshxt"}},"created_at":"2023-10-02T15:22:42.000Z","updated_at":"2025-02-14T02:40:49.000Z","dependencies_parsed_at":"2023-10-05T07:47:06.980Z","dependency_job_id":"aac43e86-de43-4c47-a35d-837bfad0c948","html_url":"https://github.com/DevXT-LLC/ezlocalai","commit_stats":{"total_commits":734,"total_committers":5,"mean_commits":146.8,"dds":"0.016348773841961872","last_synced_commit":"2c267491605ad3c06ece7f839d934fb564dc383c"},"previous_names":["josh-xt/llamacpp-server","agixt/local-llm","agixt/ezlocalai","devxt-llc/ezlocalai","josh-xt/local-llm"],"tags_count":60,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevXT-LLC%2Fezlocalai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevXT-LLC%2Fezlocalai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevXT-LLC%2Fezlocalai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevXT-LLC%2Fezlocalai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DevXT-LLC","download_url":"https://codeload.github.com/DevXT-LLC/ezlocalai/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247266565,"owners_count":20910836,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","artificial-intelligence","cuda","llamacpp","local"],"created_at":"2024-09-24T13:33:49.644Z","updated_at":"2026-02-21T16:03:24.356Z","avatar_url":"https://github.com/DevXT-LLC.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/Josh-XT","https://ko-fi.com/joshxt"],"categories":[],"sub_categories":[],"readme":"# ezlocalai\n\n[![GitHub](https://img.shields.io/badge/GitHub-ezLocalai-blue?logo=github\u0026style=plastic)](https://github.com/DevXT-LLC/ezlocalai) [![Dockerhub](https://img.shields.io/badge/Docker-ezlocalai-blue?logo=docker\u0026style=plastic)](https://hub.docker.com/r/joshxt/ezlocalai)\n\nezlocalai is an easy set up artificial intelligence server that allows you to easily run multimodal artificial intelligence from your computer. It is designed to be as easy as possible to get started with running local models. It automatically handles downloading the model of your choice and configuring the server based on your CPU, RAM, and GPU specifications. It also includes [OpenAI Style](https://pypi.org/project/openai/) endpoints for easy integration with other applications using ezlocalai as an OpenAI API proxy with any model. Additional functionality is built in for voice cloning text to speech and a voice to text for easy voice communication as well as image generation entirely offline after the initial setup.\n\n## Prerequisites\n\n- [Python 3.10+](https://www.python.org/downloads/)\n- [Docker Desktop](https://docs.docker.com/docker-for-windows/install/) (Windows or Mac)\n- [CUDA Toolkit (May Need 12.4)](https://developer.nvidia.com/cuda-12-4-0-download-archive) (NVIDIA GPU only)\n- [ROCm](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/index.html) (AMD GPU only - Linux)\n\n\u003cdetails\u003e\n  \u003csummary\u003eAdditional Linux Prerequisites\u003c/summary\u003e\n\n- [Docker](https://docs.docker.com/get-docker/)\n- [Docker Compose](https://docs.docker.com/compose/install/)\n- [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html) (NVIDIA GPU only)\n- [ROCm](https://rocm.docs.amd.com/projects/install-on-linux/en/latest/index.html) (AMD GPU only - Radeon RX 6000/7000/9000 series, Radeon PRO, and Ryzen APUs)\n\n\u003c/details\u003e\n\n## Quick Start (Recommended)\n\nInstall the CLI and start ezlocalai with a single command:\n\n```bash\npip install ezlocalai\nezlocalai start\n```\n\nIt will take several minutes to download the models on the first run. Once running, access the API at \u003chttp://localhost:8091\u003e.\n\n### CLI Commands\n\n```bash\n# Start with defaults (auto-detects GPU, uses Qwen3-VL-4B)\nezlocalai start\n\n# Start with a specific model\nezlocalai start --model unsloth/gemma-3-4b-it-GGUF\n\n# Start with custom options\nezlocalai start --model unsloth/Qwen3-VL-4B-Instruct-GGUF \\\n                --uri http://localhost:8091 \\\n                --api-key my-secret-key \\\n                --ngrok \u003cyour-ngrok-token\u003e\n\n# Other commands\nezlocalai stop      # Stop the container\nezlocalai restart   # Restart the container\nezlocalai status    # Check if running and show configuration\nezlocalai logs      # Show container logs (use -f to follow)\nezlocalai update    # Pull/rebuild latest images\n\n# Send prompts directly from the CLI\nezlocalai prompt \"Hello, world!\"\nezlocalai prompt \"What's in this image?\" -image ./photo.jpg\nezlocalai prompt \"Explain quantum computing\" -m unsloth/Qwen3-VL-4B-Instruct-GGUF -temp 0.7\n```\n\n### CLI Options\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `--model`, `-m` | `unsloth/Qwen3-VL-4B-Instruct-GGUF` | HuggingFace GGUF model(s), comma-separated |\n| `--uri` | `http://localhost:8091` | Server URL |\n| `--api-key` | None | API key for authentication |\n| `--ngrok` | None | ngrok token for public URL |\n\n### Prompt Command Options\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `-m`, `--model` | Auto-detected | Model to use for the prompt |\n| `-temp`, `--temperature` | Model default | Temperature for response generation (0.0-2.0) |\n| `-tp`, `--top-p` | Model default | Top-p (nucleus) sampling parameter (0.0-1.0) |\n| `-image`, `--image` | None | Path to local image file or URL to include with prompt |\n| `-stats`, `--stats` | Off | Show statistics (tokens, speed, timing) after response |\n\nFor additional options (Whisper, image model, etc.), edit `~/.ezlocalai/.env`:\n\n### Data Persistence\n\nAll data is stored in `~/.ezlocalai/`:\n\n| Directory | Contents |\n|-----------|----------|\n| `~/.ezlocalai/data/models/` | Downloaded GGUF model files |\n| `~/.ezlocalai/data/hf/` | HuggingFace cache |\n| `~/.ezlocalai/data/voices/` | Voice cloning samples |\n| `~/.ezlocalai/data/outputs/` | Generated images/audio |\n| `~/.ezlocalai/.env` | Your configuration |\n\nModels persist across container updates - you won't re-download them when updating the CLI or rebuilding the CUDA image.\n\n## Benchmarks\n\nPerformance tested on Intel i9-12900KS + RTX 4090 (24GB):\n\n| Model | Size | Speed | Notes |\n|-------|------|-------|-------|\n| **Qwen3-VL-4B** | 4B | ~210 tok/s | Vision-capable, great for chat |\n| **Qwen3-Coder-30B** | 30B (MoE) | ~65 tok/s | Coding model, hot-swappable |\n\nBoth models pre-calibrate at startup and hot-swap in ~1 second.\n\n## Distributed Fallback / Multi-Machine Setup\n\nezlocalai supports a distributed fallback system where multiple instances can fall back to each other when local resources (VRAM/RAM) are exhausted, or fall back to any OpenAI-compatible API. This enables:\n\n- **Load balancing**: When one machine is busy, requests automatically route to another\n- **Redundancy**: If one server is overloaded, the fallback handles requests\n- **Resource optimization**: Each machine handles what it can, forwarding the rest\n- **Hybrid deployment**: Mix local ezlocalai instances with cloud APIs\n\n### Configuration\n\nSet these environment variables in your `.env` file or pass them to the container:\n\n```bash\n# Fallback server URL - can be another ezlocalai instance OR any OpenAI-compatible API\nFALLBACK_SERVER=http://192.168.1.100:8091  # Another ezlocalai instance\n# Or use a cloud provider:\n# FALLBACK_SERVER=https://api.openai.com/v1\n\n# Authentication for the fallback server\nFALLBACK_API_KEY=your-api-key\n\n# Optional: Override model for OpenAI-compatible fallback (pass-through by default)\n# If not set, the originally requested model is passed through to the fallback server\n# FALLBACK_MODEL=gpt-4o-mini\n\n# Combined memory threshold (VRAM + RAM) in GB - fallback triggers when below this\n# Models can offload to system RAM, so combined memory is more accurate than VRAM alone\nFALLBACK_MEMORY_THRESHOLD=8.0\n```\n\nThe system automatically detects whether `FALLBACK_SERVER` points to another ezlocalai instance or an OpenAI-compatible API by checking for the `/v1/resources` endpoint. If it's another ezlocalai server, full endpoint forwarding is used (preserving the original request). Otherwise, it falls back to standard OpenAI API calls, passing through the originally requested model (or using `FALLBACK_MODEL` if set as an override).\n\n### Example: Two-Machine Setup\n\n**Machine A** (Primary with RTX 4090):\n```bash\nEZLOCALAI_URL=http://0.0.0.0:8091\nEZLOCALAI_API_KEY=shared-key\nFALLBACK_SERVER=http://machine-b:8091\nFALLBACK_API_KEY=shared-key\n```\n\n**Machine B** (Fallback with RTX 3080):\n```bash\nEZLOCALAI_URL=http://0.0.0.0:8091\nEZLOCALAI_API_KEY=shared-key\nFALLBACK_SERVER=http://machine-a:8091\nFALLBACK_API_KEY=shared-key\n```\n\nBoth machines fall back to each other - creating a resilient two-node cluster.\n\n### Example: Local + Cloud Hybrid\n\nRun a local ezlocalai with OpenAI as the fallback:\n\n```bash\nEZLOCALAI_URL=http://0.0.0.0:8091\nFALLBACK_SERVER=https://api.openai.com/v1\nFALLBACK_API_KEY=sk-your-openai-key\nFALLBACK_MODEL=gpt-4o-mini\n```\n\n### Monitoring Fallback Status\n\nCheck the fallback status via API:\n```bash\n# Get resource status including fallback info\ncurl http://localhost:8091/v1/resources\n\n# Check fallback availability and models\ncurl http://localhost:8091/v1/fallback/status\n```\n\n### What Gets Forwarded\n\nWhen fallback is triggered to another ezlocalai instance, these endpoints are automatically forwarded:\n- `/v1/chat/completions` - Chat completions (including streaming)\n- `/v1/completions` - Text completions\n- `/v1/embeddings` - Text embeddings\n- `/v1/audio/transcriptions` - Speech-to-text\n- `/v1/audio/speech` - Text-to-speech\n- `/v1/images/generations` - Image generation\n\nFor OpenAI-compatible APIs, only chat completions and embeddings are forwarded.\n\n## Dedicated Voice Server\n\nezlocalai supports offloading TTS (text-to-speech) and STT (speech-to-text) processing to a dedicated voice server. This is useful when you want to:\n\n- **Separate workloads**: Run voice models on a dedicated GPU while the main server handles LLMs\n- **Optimize resources**: Keep voice models always loaded on a server with spare VRAM\n- **Reduce latency**: Avoid lazy loading delays for voice requests\n\n### Configuration\n\nSet the `VOICE_SERVER` environment variable:\n\n```bash\n# Option 1: Point to another ezlocalai server for voice processing\nVOICE_SERVER=http://192.168.1.100:8091\nVOICE_SERVER_API_KEY=your-api-key  # Optional, uses EZLOCALAI_API_KEY if not set\n\n# Option 2: Make THIS server the voice server (keeps TTS/STT loaded)\nVOICE_SERVER=true\n```\n\n### Voice Server Mode (`VOICE_SERVER=true`)\n\nWhen set to `true`, this server becomes a dedicated voice server:\n- TTS (Chatterbox) and STT (Whisper) models are **pre-loaded at startup** and stay resident\n- Voice models are **never unloaded** after requests (no lazy load/unload cycle)\n- LLM models are still lazy-loaded as needed\n- Ideal for a secondary server with a dedicated GPU for voice processing\n\n### Voice Passthrough Mode (`VOICE_SERVER=\u003curl\u003e`)\n\nWhen set to a URL, voice requests are forwarded to that server:\n- TTS and STT requests first try the voice server\n- If the voice server fails or is unavailable, falls back to local processing\n- LLM models run locally as usual\n- No voice models are loaded locally unless the voice server is unavailable\n\n### Example: Two-Machine Voice Offload Setup\n\n**Machine A** (Main LLM server with RTX 4090):\n```bash\nDEFAULT_MODEL=unsloth/Qwen3-Coder-30B-GGUF\nVOICE_SERVER=http://machine-b:8091\nVOICE_SERVER_API_KEY=shared-key\n```\n\n**Machine B** (Voice server with RTX 3090):\n```bash\nDEFAULT_MODEL=unsloth/Qwen3-4B-Instruct-GGUF  # Smaller LLM for basic tasks\nVOICE_SERVER=true  # Keep voice models loaded\n```\n\nMachine A handles LLM inference while Machine B handles all voice processing with models always ready.\n\n## Wake Word Detection and Training\n\nezlocalai includes a complete wake word training and inference system that enables custom voice activation for AI assistants. When `VOICE_SERVER=true`, the wake word endpoints are enabled.\n\n### Custom Wake Word Architecture\n\nezlocalai trains wake word models using TTS-generated samples and exports them in multiple formats for cross-platform deployment:\n\n```mermaid\ngraph LR\n    A[User selects wake word] --\u003e B[ezlocalai trains model]\n    B --\u003e C[Export: PyTorch .pt]\n    B --\u003e D[Export: ONNX .onnx]\n    B --\u003e E[Export: ESPDL .espdl]\n    C --\u003e F[Server inference]\n    D --\u003e G[Mobile App\u003cbr/\u003eONNX Runtime Mobile]\n    E --\u003e H[ESP32-S3\u003cbr/\u003eESP-DL Framework]\n```\n\n### Supported Export Formats\n\n| Format | Extension | Size | Target Platform | Framework |\n|--------|-----------|------|-----------------|-----------|\n| **PyTorch** | `.pt` | ~1.7MB | Server | PyTorch |\n| **ONNX** | `.onnx` | ~1.7MB | Mobile | ONNX Runtime Mobile |\n| **ESPDL** | `.espdl` | ~460KB | ESP32-S3 | ESP-DL v2.0+ |\n\nThe ESPDL format is quantized to int8 for efficient inference on microcontrollers, reducing the model size by ~75% while maintaining accuracy.\n\n### Model Architecture\n\n- **Input**: 40 MFCC coefficients × 150 frames (1.5 seconds of audio at 16kHz)\n- **Architecture**: Compact CNN with 3 conv layers + batch norm + max pooling\n- **Output**: Single sigmoid probability [0.0, 1.0]\n- **Typical Accuracy**: 95-98% validation accuracy\n\n### API Endpoints\n\n```bash\n# Train a new wake word model\ncurl -X POST \"http://localhost:8091/v1/wakeword/train\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"word\": \"hey jarvis\"}'\n\n# Check training status\ncurl \"http://localhost:8091/v1/wakeword/jobs/{job_id}\" \\\n  -H \"Authorization: Bearer your-api-key\"\n\n# List available models\ncurl \"http://localhost:8091/v1/wakeword/models\" \\\n  -H \"Authorization: Bearer your-api-key\"\n\n# Download model for mobile (ONNX format)\ncurl \"http://localhost:8091/v1/wakeword/models/hey%20jarvis?format=onnx\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -o hey_jarvis_model.onnx\n\n# Download model for ESP32-S3 (ESPDL format)\ncurl \"http://localhost:8091/v1/wakeword/models/hey%20jarvis?format=espdl\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -o hey_jarvis_model.espdl\n\n# Download PyTorch model for server-side inference\ncurl \"http://localhost:8091/v1/wakeword/models/hey%20jarvis?format=pytorch\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -o hey_jarvis_model.pt\n```\n\n### Client Integration\n\n**Mobile App (Flutter/Dart):**\n```dart\n// Download model from ezlocalai\nfinal response = await http.get(\n  Uri.parse('$serverUrl/v1/wakeword/models/$wakeWord?format=onnx'),\n  headers: {'Authorization': 'Bearer $apiKey'},\n);\n// Save and load with ONNX Runtime Mobile\nfinal session = await OrtSession.create(modelPath);\n```\n\n**ESP32-S3 (C with ESP-DL):**\n```c\n#include \"custom_wakeword.h\"\n\n// Initialize the custom wake word system\ncustom_wakeword_init();\n\n// Download model from server (stores in NVS)\ncustom_wakeword_download_model(\n    \"http://192.168.1.100:8091\",  // ezlocalai server URL\n    \"your-api-key\",               // API key (or NULL)\n    \"hey jarvis\"                  // Wake word to download\n);\n\n// Start detection with callback\nvoid on_wake_word(const char *word, float confidence, void *ctx) {\n    printf(\"Wake word '%s' detected (%.2f confidence)\\n\", word, confidence);\n}\ncustom_wakeword_start(on_wake_word, NULL);\n```\n\n### Training Process\n\nThe wake word trainer:\n1. **Generates TTS samples** using gTTS, Edge-TTS, and Chatterbox with various voices\n2. **Applies audio augmentation** (noise injection, speed/pitch variations, reverb)\n3. **Trains a CNN model** with MFCC features (40 coefficients)\n4. **Exports to all formats**:\n   - PyTorch (.pt) - full precision for server\n   - ONNX (.onnx) - for mobile deployment\n   - ESPDL (.espdl) - int8 quantized for ESP32\n\nTraining typically takes 3-5 minutes depending on your hardware.\n\n## OpenAI Style Endpoint Usage\n\nOpenAI Style endpoints available at `http://\u003cYOUR LOCAL IP ADDRESS\u003e:8091/v1/` by default. Documentation can be accessed at that \u003chttp://localhost:8091\u003e when the server is running.\n\n```python\nimport requests\n\nresponse = requests.post(\n    \"http://localhost:8091/v1/chat/completions\",\n    headers={\"Authorization\": \"Bearer your-api-key\"},  # Change this if you configured an API key\n    json={\n        \"model\": \"unsloth/Qwen3-VL-4B-Instruct-GGUF\",\n        \"messages\": [\n            {\n                \"role\": \"user\",\n                \"content\": [\n                    {\"type\": \"text\", \"text\": \"Describe each stage of this image.\"},\n                    {\n                        \"type\": \"image_url\",\n                        \"image_url\": {\n                            \"url\": \"https://www.visualwatermark.com/images/add-text-to-photos/add-text-to-image-3.webp\"\n                        },\n                    },\n                ],\n            },\n        ],\n        \"max_tokens\": 8192,\n        \"temperature\": 0.7,\n        \"top_p\": 0.8,\n    },\n)\nprint(response.json()[\"choices\"][0][\"message\"][\"content\"])\n```\n\nFor examples on how to use the server to communicate with the models, see the [Examples Jupyter Notebook](tests.ipynb) once the server is running. We also have an [example to use in Google Colab](ezlocalai-ngrok.ipynb).\n\n## Workflow\n\n```mermaid\ngraph TD\n   A[app.py] --\u003e B[FASTAPI]\n   B --\u003e C[Pipes]\n   C --\u003e D[LLM]\n   C --\u003e E[STT]\n   C --\u003e F[CTTS]\n   C --\u003e G[IMG]\n   D --\u003e H[llama_cpp]\n   D --\u003e I[tiktoken]\n   D --\u003e J[torch]\n   E --\u003e K[faster_whisper]\n   E --\u003e L[pyaudio]\n   E --\u003e M[webrtcvad]\n   E --\u003e N[pydub]\n   F --\u003e O[TTS]\n   F --\u003e P[torchaudio]\n   G --\u003e Q[diffusers]\n   Q --\u003e J\n   A --\u003e R[Uvicorn]\n   R --\u003e S[ASGI Server]\n   A --\u003e T[API Endpoint: /v1/completions]\n   T --\u003e U[Pipes.get_response]\n   U --\u003e V{completion_type}\n   V --\u003e|completion| W[LLM.completion]\n   V --\u003e|chat| X[LLM.chat]\n   X --\u003e Y[LLM.generate]\n   W --\u003e Y\n   Y --\u003e Z[LLM.create_completion]\n   Z --\u003e AA[Return response]\n   AA --\u003e AB{stream}\n   AB --\u003e|True| AC[StreamingResponse]\n   AB --\u003e|False| AD[JSON response]\n   U --\u003e AE[Audio transcription]\n   AE --\u003e AF{audio_format}\n   AF --\u003e|Exists| AG[Transcribe audio]\n   AG --\u003e E\n   AF --\u003e|None| AH[Skip transcription]\n   U --\u003e AI[Audio generation]\n   AI --\u003e AJ{voice}\n   AJ --\u003e|Exists| AK[Generate audio]\n   AK --\u003e F\n   AK --\u003e AL{stream}\n   AL --\u003e|True| AM[StreamingResponse]\n   AL --\u003e|False| AN[JSON response with audio URL]\n   AJ --\u003e|None| AO[Skip audio generation]\n   U --\u003e AP[Image generation]\n   AP --\u003e AQ{IMG enabled}\n   AQ --\u003e|True| AR[Generate image]\n   AR --\u003e G\n   AR --\u003e AS[Append image URL to response]\n   AQ --\u003e|False| AT[Skip image generation]\n   A --\u003e AU[API Endpoint: /v1/chat/completions]\n   AU --\u003e U\n   A --\u003e AV[API Endpoint: /v1/embeddings]\n   AV --\u003e AW[LLM.embedding]\n   AW --\u003e AX[LLM.create_embedding]\n   AX --\u003e AY[Return embedding]\n   A --\u003e AZ[API Endpoint: /v1/audio/transcriptions]\n   AZ --\u003e BA[STT.transcribe_audio]\n   BA --\u003e BB[Return transcription]\n   A --\u003e BC[API Endpoint: /v1/audio/generation]\n   BC --\u003e BD[CTTS.generate]\n   BD --\u003e BE[Return audio URL or base64 audio]\n   A --\u003e BF[API Endpoint: /v1/models]\n   BF --\u003e BG[LLM.models]\n   BG --\u003e BH[Return available models]\n   A --\u003e BI[CORS Middleware]\n   BJ[.env] --\u003e BK[Environment Variables]\n   BK --\u003e A\n   BL[setup.py] --\u003e BM[ezlocalai package]\n   BM --\u003e BN[LLM]\n   BM --\u003e BO[STT]\n   BM --\u003e BP[CTTS]\n   BM --\u003e BQ[IMG]\n   A --\u003e BR[API Key Verification]\n   BR --\u003e BS[verify_api_key]\n   A --\u003e BT[Static Files]\n   BT --\u003e BU[API Endpoint: /outputs]\n   A --\u003e BV[Ngrok]\n   BV --\u003e BW[Public URL]\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevxt-llc%2Fezlocalai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevxt-llc%2Fezlocalai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevxt-llc%2Fezlocalai/lists"}