{"id":49192412,"url":"https://github.com/vladimircuriel/locai-chat","last_synced_at":"2026-04-23T07:30:59.981Z","repository":{"id":313743581,"uuid":"1050207112","full_name":"vladimircuriel/locai-chat","owner":"vladimircuriel","description":"LocAI is a WebLLM web client with service workers to achieve offline connectivity","archived":false,"fork":false,"pushed_at":"2026-03-12T18:41:16.000Z","size":600,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-13T00:42:12.627Z","etag":null,"topics":["astrojs","chatbot","github-actions","github-pages","react","service-worker","webllm"],"latest_commit_sha":null,"homepage":"https://locai.o5d.dev/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vladimircuriel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-04T05:27:11.000Z","updated_at":"2026-03-12T18:41:13.000Z","dependencies_parsed_at":"2025-09-08T08:14:05.277Z","dependency_job_id":"a5752bb4-381b-4490-8cc3-3014662f28ac","html_url":"https://github.com/vladimircuriel/locai-chat","commit_stats":null,"previous_names":["vladimircuriel/locai-chat"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/vladimircuriel/locai-chat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Flocai-chat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Flocai-chat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Flocai-chat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Flocai-chat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vladimircuriel","download_url":"https://codeload.github.com/vladimircuriel/locai-chat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vladimircuriel%2Flocai-chat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32171220,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-23T02:19:40.750Z","status":"ssl_error","status_checked_at":"2026-04-23T02:17:55.737Z","response_time":53,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["astrojs","chatbot","github-actions","github-pages","react","service-worker","webllm"],"created_at":"2026-04-23T07:30:57.103Z","updated_at":"2026-04-23T07:30:59.964Z","avatar_url":"https://github.com/vladimircuriel.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n   LocAI - Local Multi-Model AI\n\u003c/h1\u003e\n\n\n\u003cp align=\"center\"\u003e \n  \u003cimg width=\"430\" alt=\"Vector\" src=\"https://github.com/user-attachments/assets/8fc35a35-6954-438f-b8c3-ffbe8f5fb23d\" /\u003e\n\u003c/p\u003e\n\n\n\u003cdiv align=\"center\"\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/Astro-000000?style=for-the-badge\u0026logo=astro\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/React-00D8FF?style=for-the-badge\u0026logo=react\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/TypeScript-3178C6?style=for-the-badge\u0026logo=typescript\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/WebGPU-5A31F4?style=for-the-badge\u0026logo=webgpu\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/IndexedDB-005A9C?style=for-the-badge\u0026logo=googlechrome\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/ServiceWorker-404040?style=for-the-badge\u0026logo=serviceworker\u0026logoColor=white\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/LocalStorage-F0DB4F?style=for-the-badge\u0026logo=javascript\u0026logoColor=black\" /\u003e  \n  \u003cimg src=\"https://img.shields.io/badge/WebLLM-1A1A1A?style=for-the-badge\u0026logo=github\u0026logoColor=white\" /\u003e  \n\u003c/div\u003e\n\n---\n\n**LocAI** is a fully client-side AI chat application that lets you download, run, and switch between multiple large language models directly in your browser—no signup or remote server required. It caches model weights and conversation history in IndexedDB, uses a Service Worker for offline support, and leverages WebGPU for accelerated inference. Every chat stays private on your device, and your last-used model, open conversation, and slider settings persist across reloads.\n\n## Table of Contents\n\n- [Features](#features)\n- [Application](#application)\n- [Tools Used](#tools-used)\n- [Installation](#installation)\n- [Areas for Improvement](#areas-for-improvement)\n\n## Features\n\n- **Client-Side Inference**: Download and run large language models entirely in your browser via WebGPU—no remote servers or API keys required.\n\n- **Multi-Model Support**: Browse, select, and switch between a growing catalog of MLC-AI models (e.g. Qwen2, Llama-3, Phi-3, Gemma-2) from the \u003chttps://mlc.ai/models\u003e directory.\n\n- **Offline-First**: Service Worker caches assets and model weights for offline use; once downloaded, models load instantly without network.\n\n- **Persistent Chats**: Conversations and messages are stored in IndexedDB; your chat history, open conversation, and slider settings persist across page reloads.\n\n- **Real-Time Streaming**: Partial responses stream in with live rendering and auto-formatted code blocks; dynamic sliders control temperature, top-p, penalties, max tokens, and choice count.\n\n- **WebGPU Enforcement**: Detects and blocks unsupported browsers (Firefox, Safari); guides users to Chrome/Chromium with WebGPU enabled.\n\n- **Responsive UI**: Modernized design with custom components (no external UI library), adaptive layout for desktop and mobile, and accessible modals for Info, Model Selection, Advanced Settings, and WebGPU errors.\n\n- **Localhost Mode**: Last‐used model and last open chat automatically reloaded from localStorage for seamless restarts.\n\n- **Privacy \u0026 Security**: All data remains on-device—clearing browser storage removes everything, ensuring zero-trust, zero-signup privacy.\n\n## Application\n\nhttps://github.com/user-attachments/assets/62283bfd-570b-42f8-9b3e-75c24fdac6ff\n\nA more extensive showcase is available in [my portfolio](https://vladimircuriel.com/posts/2_locai/)!\n\n## Tools Used\n\n- ![Astro](https://img.shields.io/badge/Astro-FF5D01?logo=astro\u0026logoColor=white\u0026style=flat-square) **Astro 5**: Static-site generator powering the modern, lightning-fast frontend.\n\n- ![React](https://img.shields.io/badge/React-61DAFB?logo=react\u0026logoColor=black\u0026style=flat-square) **React 19**: UI library for building interactive chat components.\n\n- ![TypeScript](https://img.shields.io/badge/TypeScript-3178C6?logo=typescript\u0026logoColor=white\u0026style=flat-square) **TypeScript**: Adds static typing across components, hooks, and data models.\n\n- ![WebGPU](https://img.shields.io/badge/WebGPU-5A31F4?style=flat-square) **WebGPU**: Browser API for GPU-accelerated inference of LLMs.\n\n- ![IndexedDB](https://img.shields.io/badge/IndexedDB-005A9C?logo=googlechrome\u0026logoColor=white\u0026style=flat-square) **IndexedDB**: Client-side storage of model weights and conversation history.\n\n- ![Service Worker](https://img.shields.io/badge/ServiceWorker-404040?style=flat-square) **Service Worker (PWA)**: Offline caching of assets and model files.\n\n- ![LocalStorage](https://img.shields.io/badge/LocalStorage-F0DB4F?logo=javascript\u0026logoColor=black\u0026style=flat-square) **LocalStorage**: Persists last-used model, open chat, and slider settings.\n\n- ![WebLLM](https://img.shields.io/badge/WebLLM-1A1A1A?style=flat-square) **@mlc-ai/web-llm**: Library for loading and running LLMs in the browser.\n\n- ![Framer Motion](https://img.shields.io/badge/Framer%20Motion-0055FF?logo=framer\u0026logoColor=white\u0026style=flat-square) **Framer Motion**: Animations and transitions for a polished UX.\n\n- ![Tailwind CSS](https://img.shields.io/badge/TailwindCSS-38B2AC?logo=tailwind-css\u0026logoColor=white\u0026style=flat-square) **Tailwind CSS**: Utility-first styling across the app.\n\n- ![React Markdown](https://img.shields.io/badge/React_Markdown-000000?style=flat-square) **react-markdown** + **remark-gfm**: Rendering of markdown formatted responses and code blocks.\n\n- ![Syntax Highlighter](https://img.shields.io/badge/SyntaxHighlighter-000000?style=flat-square) **react-syntax-highlighter**: Code formatting and highlighting during streaming.\n\n- ![PNPM](https://img.shields.io/badge/PNPM-F69220?logo=pnpm\u0026logoColor=white\u0026style=flat-square) **pnpm**: Fast, disk-efficient package manager.\n\n- ![GitHub Actions](https://img.shields.io/badge/GitHub_Actions-2088FF?logo=github-actions\u0026logoColor=white\u0026style=flat-square) **GitHub Actions**: CI/CD workflows for builds, tests, and deploys to GitHub Pages.\n\n## Installation\n\n### Prerequisites\n\n- **Docker**\n\n### Steps\n\n1. **Clone the repository**:\n\n```bash\ngit clone https://github.com/vladimircuriel/locai-chat\n```\n\n2. **Navigate to the project directory**:\n\n```bash\ncd locai-chat\n```\n   \n3. **Run the commands**:\n\n```bash\ndocker build -t locai:latest .\n```\n\n```bash\ndocker run -p 4321:4321 locai:latest\n```\n4. **Access the application**:\n\nOpen your browser and visit `http://localhost:4321` to access the user interface.\n\n## Areas for Improvement\n\n- Models must be re-downloaded when switching quantization or major versions, leading to extra wait and disk use.  \n- Very large models can exhaust device RAM/VRAM and may crash or hang browsers on lower-end hardware.  \n- Chat history in IndexedDB is unencrypted and tied to the browser; no built-in export or sync across devices.  \n- Mobile browsers with experimental or missing WebGPU support are blocked entirely—no lightweight fallback.  \n- No built-in search or filtering within long conversation histories.  \n- Context window is limited by model token capacity; very long chats may lose early context when truncated.  \n- No support for custom prompts or system-level instruction templates beyond a single “system” message.  \n- No collaborative or multi-user features—every session is isolated to the local device.  \n- Lack of model fine-tuning or personalization options; you’re limited to public pre-trained checkpoints.  \n- Clearing browser storage (IndexedDB/localStorage) deletes all chats and model caches without warning.  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimircuriel%2Flocai-chat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvladimircuriel%2Flocai-chat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvladimircuriel%2Flocai-chat/lists"}