{"id":15165432,"url":"https://github.com/mudler/localai","last_synced_at":"2026-05-14T00:02:30.652Z","repository":{"id":144887760,"uuid":"615869301","full_name":"mudler/LocalAI","owner":"mudler","description":":robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI,  running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference","archived":false,"fork":false,"pushed_at":"2025-05-08T14:45:06.000Z","size":19090,"stargazers_count":32451,"open_issues_count":459,"forks_count":2470,"subscribers_count":215,"default_branch":"master","last_synced_at":"2025-05-08T15:37:46.474Z","etag":null,"topics":["ai","api","audio-generation","distributed","gemma","gpt4all","image-generation","kubernetes","libp2p","llama","llama3","llm","mamba","mistral","musicgen","rerank","rwkv","stable-diffusion","text-generation","tts"],"latest_commit_sha":null,"homepage":"https://localai.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mudler.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["mudler"],"custom":["https://www.buymeacoffee.com/mudler"]}},"created_at":"2023-03-18T22:58:02.000Z","updated_at":"2025-05-08T15:20:38.000Z","dependencies_parsed_at":"2023-09-24T15:19:08.037Z","dependency_job_id":"a0550d97-39b1-4245-b4bc-3f737ce3976f","html_url":"https://github.com/mudler/LocalAI","commit_stats":{"total_commits":2844,"total_committers":117,"mean_commits":"24.307692307692307","dds":0.5868495077355838,"last_synced_commit":"fd4043266bf1369765ddffc6ca413feeae6c5d17"},"previous_names":["go-skynet/llama-cli","mudler/localai","go-skynet/localai"],"tags_count":112,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2FLocalAI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2FLocalAI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2FLocalAI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mudler%2FLocalAI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mudler","download_url":"https://codeload.github.com/mudler/LocalAI/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253508554,"owners_count":21919461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","api","audio-generation","distributed","gemma","gpt4all","image-generation","kubernetes","libp2p","llama","llama3","llm","mamba","mistral","musicgen","rerank","rwkv","stable-diffusion","text-generation","tts"],"created_at":"2024-09-27T04:01:23.317Z","updated_at":"2026-05-14T00:02:30.646Z","avatar_url":"https://github.com/mudler.png","language":"Go","funding_links":["https://github.com/sponsors/mudler","https://www.buymeacoffee.com/mudler","https://buymeacoffee.com/mudler"],"categories":["Chatbots"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  \u003cbr\u003e\n  \u003cimg width=\"300\" src=\"./core/http/static/logo.png\"\u003e \u003cbr\u003e\n\u003cbr\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://github.com/go-skynet/LocalAI/stargazers\" target=\"blank\"\u003e\n\u003cimg src=\"https://img.shields.io/github/stars/go-skynet/LocalAI?style=for-the-badge\" alt=\"LocalAI stars\"/\u003e\n\u003c/a\u003e\n\u003ca href='https://github.com/go-skynet/LocalAI/releases'\u003e\n\u003cimg src='https://img.shields.io/github/release/go-skynet/LocalAI?\u0026label=Latest\u0026style=for-the-badge'\u003e\n\u003c/a\u003e\n\u003ca href=\"LICENSE\" target=\"blank\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge\" alt=\"LocalAI License\"/\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://twitter.com/LocalAI_API\" target=\"blank\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/X-%23000000.svg?style=for-the-badge\u0026logo=X\u0026logoColor=white\u0026label=LocalAI_API\" alt=\"Follow LocalAI_API\"/\u003e\n\u003c/a\u003e\n\u003ca href=\"https://discord.gg/uJAeKSAGDy\" target=\"blank\"\u003e\n\u003cimg src=\"https://img.shields.io/badge/dynamic/json?color=blue\u0026label=Discord\u0026style=for-the-badge\u0026query=approximate_member_count\u0026url=https%3A%2F%2Fdiscordapp.com%2Fapi%2Finvites%2FuJAeKSAGDy%3Fwith_counts%3Dtrue\u0026logo=discord\" alt=\"Join LocalAI Discord Community\"/\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://trendshift.io/repositories/5539\" target=\"_blank\"\u003e\u003cimg src=\"https://trendshift.io/api/badge/repositories/5539\" alt=\"mudler%2FLocalAI | Trendshift\" style=\"width: 250px; height: 55px;\" width=\"250\" height=\"55\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n**LocalAI** is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.\n\n- **Drop-in API compatibility** — OpenAI, Anthropic, ElevenLabs APIs\n- **36+ backends** — llama.cpp, vLLM, transformers, whisper, diffusers, MLX...\n- **Any hardware** — NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only\n- **Multi-user ready** — API key auth, user quotas, role-based access\n- **Built-in AI agents** — autonomous agents with tool use, RAG, MCP, and skills\n- **Privacy-first** — your data never leaves your infrastructure\n\nCreated by [Ettore Di Giacinto](https://github.com/mudler) and maintained by the [LocalAI team](#team).\n\n\u003e [:book: Documentation](https://localai.io/) | [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) | [💻 Quickstart](https://localai.io/basics/getting_started/) | [🖼️ Models](https://models.localai.io/) | [❓FAQ](https://localai.io/faq/)\n\n## Guided tour\n\nhttps://github.com/user-attachments/assets/08cbb692-57da-48f7-963d-2e7b43883c18\n\n\u003cdetails\u003e\n\n\u003csummary\u003e\nClick to see more!\n\u003c/summary\u003e\n\n#### User and auth\n\nhttps://github.com/user-attachments/assets/228fa9ad-81a3-4d43-bfb9-31557e14a36c\n\n#### Agents\n\nhttps://github.com/user-attachments/assets/6270b331-e21d-4087-a540-6290006b381a\n\n#### Usage metrics per user\n\nhttps://github.com/user-attachments/assets/cbb03379-23b4-4e3d-bd26-d152f057007f\n\n#### Fine-tuning and Quantization\n\nhttps://github.com/user-attachments/assets/5ba4ace9-d3df-4795-b7d4-b0b404ea71ee\n\n#### WebRTC\n\nhttps://github.com/user-attachments/assets/ed88e34c-fed3-4b83-8a67-4716a9feeb7b\n\n\u003c/details\u003e\n\n## Quickstart\n\n### macOS\n\n\u003ca href=\"https://github.com/mudler/LocalAI/releases/latest/download/LocalAI.dmg\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/Download-macOS-blue?style=for-the-badge\u0026logo=apple\u0026logoColor=white\" alt=\"Download LocalAI for macOS\"/\u003e\n\u003c/a\u003e\n\n\u003e **Note:** The DMG is not signed by Apple. After installing, run: `sudo xattr -d com.apple.quarantine /Applications/LocalAI.app`. See [#6268](https://github.com/mudler/LocalAI/issues/6268) for details.\n\n### Containers (Docker, podman, ...)\n\n\u003e Already ran LocalAI before? Use `docker start -i local-ai` to restart an existing container.\n\n#### CPU only:\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 localai/localai:latest\n```\n\n#### NVIDIA GPU:\n\n```bash\n# CUDA 13\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13\n\n# CUDA 12\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12\n\n# NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar)\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64\n\n# NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark)\ndocker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13\n```\n\n#### AMD GPU (ROCm):\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas\n```\n\n#### Intel GPU (oneAPI):\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel\n```\n\n#### Vulkan GPU:\n\n```bash\ndocker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan\n```\n\n### Loading models\n\n```bash\n# From the model gallery (see available models with `local-ai models list` or at https://models.localai.io)\nlocal-ai run llama-3.2-1b-instruct:q4_k_m\n# From Huggingface\nlocal-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf\n# From the Ollama OCI registry\nlocal-ai run ollama://gemma:2b\n# From a YAML config\nlocal-ai run https://gist.githubusercontent.com/.../phi-2.yaml\n# From a standard OCI registry (e.g., Docker Hub)\nlocal-ai run oci://localai/phi-2:latest\n```\n\n\u003e **Automatic Backend Detection**: LocalAI automatically detects your GPU capabilities and downloads the appropriate backend. For advanced options, see [GPU Acceleration](https://localai.io/features/gpu-acceleration/).\n\nFor more details, see the [Getting Started guide](https://localai.io/basics/getting_started/).\n\n## Latest News\n\n- **April 2026**: [Voice recognition](https://github.com/mudler/LocalAI/pull/9500), [Face recognition, identification \u0026 liveness detection](https://github.com/mudler/LocalAI/pull/9480), [Ollama API compatibility](https://github.com/mudler/LocalAI/pull/9284), [Video generation in stable-diffusion.ggml](https://github.com/mudler/LocalAI/pull/9420), [Backend versioning with auto-upgrade](https://github.com/mudler/LocalAI/pull/9315), [Pin models \u0026 load-on-demand toggle](https://github.com/mudler/LocalAI/pull/9309), [Universal model importer](https://github.com/mudler/LocalAI/pull/9466), new backends: [sglang](https://github.com/mudler/LocalAI/pull/9359), [ik-llama-cpp](https://github.com/mudler/LocalAI/pull/9326), [TurboQuant](https://github.com/mudler/LocalAI/pull/9355), [sam.cpp](https://github.com/mudler/LocalAI/pull/9288), [Kokoros](https://github.com/mudler/LocalAI/pull/9212), [qwen3tts.cpp](https://github.com/mudler/LocalAI/pull/9316), [tinygrad multimodal](https://github.com/mudler/LocalAI/pull/9364)\n- **March 2026**: [Agent management](https://github.com/mudler/LocalAI/pull/8820), [New React UI](https://github.com/mudler/LocalAI/pull/8772), [WebRTC](https://github.com/mudler/LocalAI/pull/8790), [MLX-distributed via P2P and RDMA](https://github.com/mudler/LocalAI/pull/8801), [MCP Apps, MCP Client-side](https://github.com/mudler/LocalAI/pull/8947)\n- **February 2026**: [Realtime API for audio-to-audio with tool calling](https://github.com/mudler/LocalAI/pull/6245), [ACE-Step 1.5 support](https://github.com/mudler/LocalAI/pull/8396)\n- **January 2026**: **LocalAI 3.10.0** — Anthropic API support, Open Responses API, video \u0026 image generation (LTX-2), unified GPU backends, tool streaming, Moonshine, Pocket-TTS. [Release notes](https://github.com/mudler/LocalAI/releases/tag/v3.10.0)\n- **December 2025**: [Dynamic Memory Resource reclaimer](https://github.com/mudler/LocalAI/pull/7583), [Automatic multi-GPU model fitting (llama.cpp)](https://github.com/mudler/LocalAI/pull/7584), [Vibevoice backend](https://github.com/mudler/LocalAI/pull/7494)\n- **November 2025**: [Import models via URL](https://github.com/mudler/LocalAI/pull/7245), [Multiple chats and history](https://github.com/mudler/LocalAI/pull/7325)\n- **October 2025**: [Model Context Protocol (MCP)](https://localai.io/docs/features/mcp/) support for agentic capabilities\n- **September 2025**: New Launcher for macOS and Linux, extended backend support for Mac and Nvidia L4T, MLX-Audio, WAN 2.2\n- **August 2025**: MLX, MLX-VLM, Diffusers, llama.cpp now supported on Apple Silicon\n- **July 2025**: All backends migrated outside the main binary — [lightweight, modular architecture](https://github.com/mudler/LocalAI/releases/tag/v3.2.0)\n\nFor older news and full release notes, see [GitHub Releases](https://github.com/mudler/LocalAI/releases) and the [News page](https://localai.io/basics/news/).\n\n## Features\n\n- [Text generation](https://localai.io/features/text-generation/) (`llama.cpp`, `transformers`, `vllm` ... [and more](https://localai.io/model-compatibility/))\n- [Text to Audio](https://localai.io/features/text-to-audio/)\n- [Audio to Text](https://localai.io/features/audio-to-text/)\n- [Image generation](https://localai.io/features/image-generation)\n- [OpenAI-compatible tools API](https://localai.io/features/openai-functions/)\n- [Realtime API](https://localai.io/features/openai-realtime/) (Speech-to-speech)\n- [Embeddings generation](https://localai.io/features/embeddings/)\n- [Constrained grammars](https://localai.io/features/constrained_grammars/)\n- [Download models from Huggingface](https://localai.io/models/)\n- [Vision API](https://localai.io/features/gpt-vision/)\n- [Object Detection](https://localai.io/features/object-detection/)\n- [Reranker API](https://localai.io/features/reranker/)\n- [P2P Inferencing](https://localai.io/features/distribute/)\n- [Distributed Mode](https://localai.io/features/distributed-mode/) — Horizontal scaling with PostgreSQL + NATS\n- [Model Context Protocol (MCP)](https://localai.io/docs/features/mcp/)\n- [Built-in Agents](https://localai.io/features/agents/) — Autonomous AI agents with tool use, RAG, skills, SSE streaming, and [Agent Hub](https://agenthub.localai.io)\n- [Backend Gallery](https://localai.io/backends/) — Install/remove backends on the fly via OCI images\n- Voice Activity Detection (Silero-VAD)\n- Integrated WebUI\n\n## Supported Backends \u0026 Acceleration\n\nLocalAI supports **36+ backends** including llama.cpp, vLLM, transformers, whisper.cpp, diffusers, MLX, MLX-VLM, and many more. Hardware acceleration is available for **NVIDIA** (CUDA 12/13), **AMD** (ROCm), **Intel** (oneAPI/SYCL), **Apple Silicon** (Metal), **Vulkan**, and **NVIDIA Jetson** (L4T). All backends can be installed on-the-fly from the [Backend Gallery](https://localai.io/backends/).\n\nSee the full [Backend \u0026 Model Compatibility Table](https://localai.io/model-compatibility/) and [GPU Acceleration guide](https://localai.io/features/gpu-acceleration/).\n\n## Resources\n\n- [Documentation](https://localai.io/)\n- [LLM fine-tuning guide](https://localai.io/docs/advanced/fine-tuning/)\n- [Build from source](https://localai.io/basics/build/)\n- [Kubernetes installation](https://localai.io/basics/getting_started/#run-localai-in-kubernetes)\n- [Integrations \u0026 community projects](https://localai.io/docs/integrations/)\n- [Installation video walkthrough](https://www.youtube.com/watch?v=cMVNnlqwfw4)\n- [Media \u0026 blog posts](https://localai.io/basics/news/#media-blogs-social)\n- [Examples](https://github.com/mudler/LocalAI-examples)\n\n## Team\n\nLocalAI is maintained by a small team of humans, together with the wider community of contributors.\n\n- **[Ettore Di Giacinto](https://github.com/mudler)** — original author and project lead\n- **[Richard Palethorpe](https://github.com/richiejp)** — maintainer\n\nA huge thank you to everyone who contributes code, reviews PRs, files issues, and helps users in [Discord](https://discord.gg/uJAeKSAGDy) — LocalAI is a community-driven project and wouldn't exist without you. See the full [contributors list](https://github.com/mudler/LocalAI/graphs/contributors).\n\n## Citation\n\nIf you utilize this repository, data in a downstream project, please consider citing it with:\n\n```\n@misc{localai,\n  author = {Ettore Di Giacinto},\n  title = {LocalAI: The free, Open source OpenAI alternative},\n  year = {2023},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/go-skynet/LocalAI}},\n```\n\n## Sponsors\n\n\u003e Do you find LocalAI useful?\n\nSupport the project by becoming [a backer or sponsor](https://github.com/sponsors/mudler). Your logo will show up here with a link to your website.\n\nA huge thank you to our generous sponsors who support this project covering CI expenses, and our [Sponsor list](https://github.com/sponsors/mudler):\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.spectrocloud.com/\" target=\"blank\"\u003e\n    \u003cimg height=\"200\" src=\"https://github.com/user-attachments/assets/72eab1dd-8b93-4fc0-9ade-84db49f24962\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.premai.io/\" target=\"blank\"\u003e\n    \u003cimg height=\"200\" src=\"https://github.com/mudler/LocalAI/assets/2420543/42e4ca83-661e-4f79-8e46-ae43689683d6\"\u003e \u003cbr\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n### Individual sponsors\n\nA special thanks to individual sponsors, a full list is on [GitHub](https://github.com/sponsors/mudler) and [buymeacoffee](https://buymeacoffee.com/mudler). Special shout out to [drikster80](https://github.com/drikster80) for being generous. Thank you everyone!\n\n## Star history\n\n[![LocalAI Star history Chart](https://api.star-history.com/svg?repos=go-skynet/LocalAI\u0026type=Date)](https://star-history.com/#go-skynet/LocalAI\u0026Date)\n\n## License\n\nLocalAI is a community-driven project created by [Ettore Di Giacinto](https://github.com/mudler/) and maintained by the [LocalAI team](#team).\n\nMIT - Author Ettore Di Giacinto \u003cmudler@localai.io\u003e\n\n## Acknowledgements\n\nLocalAI couldn't have been built without the help of great software already available from the community. Thank you!\n\n- [llama.cpp](https://github.com/ggerganov/llama.cpp)\n- https://github.com/tatsu-lab/stanford_alpaca\n- https://github.com/cornelk/llama-go for the initial ideas\n- https://github.com/antimatter15/alpaca.cpp\n- https://github.com/EdVince/Stable-Diffusion-NCNN\n- https://github.com/ggerganov/whisper.cpp\n- https://github.com/rhasspy/piper\n- [exo](https://github.com/exo-explore/exo) for the MLX distributed auto-parallel sharding implementation\n\n## Contributors\n\nThis is a community project, a special thanks to our contributors!\n\u003ca href=\"https://github.com/go-skynet/LocalAI/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=go-skynet/LocalAI\" /\u003e\n\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmudler%2Flocalai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmudler%2Flocalai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmudler%2Flocalai/lists"}