{"id":51493934,"url":"https://github.com/kiritocode1/local-llm","last_synced_at":"2026-07-07T13:01:46.170Z","repository":{"id":337367800,"uuid":"1152292738","full_name":"kiritocode1/local-llm","owner":"kiritocode1","description":null,"archived":false,"fork":false,"pushed_at":"2026-02-16T11:32:30.000Z","size":198,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-16T12:36:55.971Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kiritocode1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-07T16:58:58.000Z","updated_at":"2026-02-16T11:32:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kiritocode1/local-llm","commit_stats":null,"previous_names":["kiritocode1/local-llm"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kiritocode1/local-llm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritocode1%2Flocal-llm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritocode1%2Flocal-llm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritocode1%2Flocal-llm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritocode1%2Flocal-llm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kiritocode1","download_url":"https://codeload.github.com/kiritocode1/local-llm/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kiritocode1%2Flocal-llm/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35228636,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-07T02:00:07.222Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-07T13:01:44.695Z","updated_at":"2026-07-07T13:01:46.136Z","avatar_url":"https://github.com/kiritocode1.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cstrong\u003e@blank-utils/llm\u003c/strong\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Run LLMs directly in your browser — zero server, zero API keys.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.npmjs.com/package/@blank-utils/llm\"\u003e\u003cimg src=\"https://img.shields.io/npm/v/@blank-utils/llm?style=flat-square\u0026color=0ea5e9\" alt=\"npm\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/kiritocode1/local-llm/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/npm/l/@blank-utils/llm?style=flat-square\" alt=\"license\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/react-%E2%89%A518-61dafb?style=flat-square\" alt=\"react\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/webgpu-supported-brightgreen?style=flat-square\" alt=\"webgpu\"\u003e\n\u003c/p\u003e\n\n---\n\n## Features\n\n- 🚀 **WebGPU acceleration** via [WebLLM](https://github.com/mlc-ai/web-llm) — falls back to WASM through [Transformers.js](https://github.com/huggingface/transformers.js)\n- ⚛️ **React hooks** — `useChat`, `useStream`, `useCompletion` with eager background loading\n- 🔤 **Type-safe model selection** — full autocomplete for 30+ supported models across both backends\n- 📝 **Streaming support** — real-time token output with abort control\n- 📄 **PDF \u0026 Image Processing** — Extract text from PDFs natively and easily pass multimodal image attachments\n- 🔄 **Message queueing** — users can type while models download; messages are processed once ready\n- 🧩 **Vanilla JS friendly** — works outside React with DOM helpers and a simple `createLLM()` API\n- ⚡ **Instant Builds** — Bundled via `tsup` for lightning-fast compilation\n- 📦 **Zero config** — auto-detects WebGPU/WASM and picks the best backend\n\n## Installation\n\n```bash\npnpm add @blank-utils/llm\n# or\nbun add @blank-utils/llm\n```\n\n\u003e React is an **optional** peer dependency. The core API works without it.\n\n### Import Styles (React)\n\nIf you are using the pre-built React components (`\u003cChat\u003e`, `\u003cChatApp\u003e`, `\u003cChatInput\u003e`), be sure to import the bundled CSS in your entry file (e.g. `index.tsx`, `App.tsx`, or `layout.tsx`):\n\n```typescript\nimport \"@blank-utils/llm/index.css\";\n```\n\n---\n\n## Quick Start\n\n### Quick Chat (Fully Featured App)\n\nThe fastest way to get started. `\u003cChatApp\u003e` includes the provider, model management, and UI in a single component:\n\n```tsx\nimport { ChatApp } from \"@blank-utils/llm/react\";\nimport \"@blank-utils/llm/index.css\"; // Required for UI styling\n\nexport default function App() {\n  return (\n    \u003cChatApp\n      defaultModel=\"qwen-2.5-0.5b\"\n      theme=\"dark\" // 'dark' | 'light'\n      systemPrompt=\"You are a helpful assistant.\"\n    /\u003e\n  );\n}\n```\n\n### Components (Custom Setup)\n\nIf you already have a provider or want more control, use the `\u003cChat\u003e` component:\n\n```tsx\nimport { LLMProvider, Chat } from \"@blank-utils/llm/react\";\nimport \"@blank-utils/llm/index.css\";\n\nexport default function App() {\n  return (\n    \u003cLLMProvider model=\"qwen-2.5-0.5b\"\u003e\n      \u003cChat\n        theme=\"dark\"\n        systemPrompt=\"You are a helpful assistant.\"\n        placeholder=\"Ask me anything...\"\n      /\u003e\n    \u003c/LLMProvider\u003e\n  );\n}\n```\n\n### Custom UI with Hooks\n\nBuild your own interface from scratch using our hooks:\n\n```tsx\nimport { LLMProvider, useChat, useLLM } from \"@blank-utils/llm/react\";\n// ... (rest of the hooks example)\n\nfunction App() {\n  return (\n    \u003cLLMProvider model=\"qwen-2.5-0.5b\"\u003e\n      \u003cChatUI /\u003e\n    \u003c/LLMProvider\u003e\n  );\n}\n\nfunction ChatUI() {\n  const { isLoading, loadProgress } = useLLM();\n  const {\n    messages,\n    input,\n    setInput,\n    send,\n    isGenerating,\n    isPending,\n    streamingText,\n  } = useChat({\n    systemPrompt: \"You are a helpful assistant.\",\n    queueWhileLoading: true,\n  });\n\n  return (\n    \u003cdiv\u003e\n      {isLoading \u0026\u0026 \u003cp\u003eLoading: {loadProgress?.progress}%\u003c/p\u003e}\n\n      {messages.map((m, i) =\u003e (\n        \u003cdiv key={i}\u003e\n          \u003cstrong\u003e{m.role}:\u003c/strong\u003e {m.content}\n        \u003c/div\u003e\n      ))}\n\n      {isPending \u0026\u0026 \u003cp\u003eWaiting for model...\u003c/p\u003e}\n      {isGenerating \u0026\u0026 \u003cp\u003eAI: {streamingText}\u003c/p\u003e}\n\n      \u003cinput\n        value={input}\n        onChange={(e) =\u003e setInput(e.target.value)}\n        onKeyDown={(e) =\u003e e.key === \"Enter\" \u0026\u0026 send()}\n        placeholder=\"Type a message...\"\n      /\u003e\n      \u003cbutton onClick={() =\u003e send()}\u003eSend\u003c/button\u003e\n    \u003c/div\u003e\n  );\n}\n```\n\n### Vanilla JavaScript\n\n```typescript\nimport { createLLM } from \"@blank-utils/llm\";\n\nconst llm = await createLLM({\n  model: \"qwen-2.5-0.5b\",\n  onLoadProgress: (p) =\u003e console.log(`Loading: ${p.progress}%`),\n});\n\n// Streaming\nawait llm.stream(\"Tell me a joke\", (token, fullText) =\u003e {\n  document.getElementById(\"output\")!.textContent = fullText;\n});\n\n// Non-streaming\nconst response = await llm.chat(\"Hello!\");\nconsole.log(response);\n```\n\n### Attach to DOM Elements\n\n```typescript\nimport { createLLM } from \"@blank-utils/llm\";\n\nconst llm = await createLLM({ model: \"smollm2-360m\" });\n\n// Wire up an input + output with one call\nconst cleanup = llm.attachToInput(\"#prompt-input\", \"#response-output\", {\n  triggerOnEnter: true,\n  clearOnSend: true,\n});\n```\n\n---\n\n## Usage in Next.js\n\nUsing `@blank-utils/llm` in Next.js Requires **two specific configurations** to allow WebAssembly multi-threading \u0026 WebWorker access for `SharedArrayBuffer` to work.\n\n### 1. Configure COOP/COEP Headers\n\nIn your `next.config.ts` or `next.config.js`, apply the following headers to allow cross-origin isolation:\n\n```typescript\n// next.config.ts\nimport type { NextConfig } from \"next\";\n\nconst nextConfig: NextConfig = {\n  async headers() {\n    return [\n      {\n        source: \"/(.*)\",\n        headers: [\n          {\n            key: \"Cross-Origin-Opener-Policy\",\n            value: \"same-origin\", // Required for SharedArrayBuffer\n          },\n          {\n            key: \"Cross-Origin-Embedder-Policy\",\n            value: \"require-corp\", // Required for SharedArrayBuffer\n          },\n        ],\n      },\n    ];\n  },\n};\n\nexport default nextConfig;\n```\n\n### 2. Dynamically Import UI Components\n\nWebLLM and WebGPU are exclusively client-side (browser) APIs. If you are using React Server Components (Next.js App Router default), you **MUST** dynamically import the chat interface with `ssr: false`.\n\n```tsx\n// app/page.tsx\n\"use client\";\n\nimport dynamic from \"next/dynamic\";\n\n// ✅ Correct: Prevents server-side rendering crashes\nconst LocalChat = dynamic(\n  () =\u003e import(\"@blank-utils/llm/react\").then((mod) =\u003e mod.ChatApp),\n  { ssr: false },\n);\n\nexport default function Page() {\n  return \u003cLocalChat defaultModel=\"phi-3.5-mini\" theme=\"dark\" /\u003e;\n}\n```\n\n---\n\n## Architecture\n\n```\n@blank-utils/llm\n├── src/\n│   ├── index.ts          # Main entry — re-exports everything\n│   ├── core.ts           # createLLM() factory, LocalLLM interface\n│   ├── models.ts         # Centralized model registry (single source of truth)\n│   ├── types.ts          # All TypeScript interfaces \u0026 types\n│   ├── detect.ts         # WebGPU / WASM capability detection\n│   ├── helpers.ts        # DOM utilities (attachToElements, createChatUI, etc.)\n│   ├── backends/\n│   │   ├── webllm.ts     # WebLLM backend (WebGPU)\n│   │   └── transformers.ts # Transformers.js backend (WASM / WebGPU)\n│   └── react/\n│       ├── index.tsx     # React context, provider, hooks\n│       ├── components.tsx # \u003cChat\u003e — ready-made chat interface\n│       └── chat-input.tsx # \u003cChatInput\u003e — auto-resizing input widget\n└── dist/                 # Built output (ESM)\n```\n\n### Dual Backend System\n\n|                   | **WebLLM**                | **Transformers.js** |\n| ----------------- | ------------------------- | ------------------- |\n| **Engine**        | MLC / TVM compiled models | ONNX Runtime        |\n| **Device**        | WebGPU only               | WebGPU or WASM      |\n| **Performance**   | Best (GPU-native)         | Good (CPU fallback) |\n| **Model source**  | MLC prebuilt cache        | HuggingFace Hub     |\n| **Auto-detected** | ✅ when WebGPU present    | ✅ fallback         |\n\nThe library auto-selects the best backend via `detectCapabilities()`. You can also force a backend:\n\n```tsx\n\u003cLLMProvider model=\"llama-3.2-3b\" backend=\"webllm\" /\u003e\n```\n\n---\n\n## React API\n\n### `\u003cLLMProvider\u003e`\n\nWrap your app to enable LLM functionality. All hooks must be used inside this provider.\n\n```tsx\n\u003cLLMProvider\n  model=\"qwen-2.5-0.5b\" // Model alias or full ID\n  backend=\"auto\" // 'webllm' | 'transformers' | 'auto'\n  autoLoad={true} // Start loading on mount\n  device=\"auto\" // 'webgpu' | 'wasm' | 'auto'\n  quantization=\"q4\" // 'q4' | 'q8' | 'fp16' | 'fp32'\n  systemPrompt=\"...\" // Default system prompt\n  onProgress={(p) =\u003e {}} // Loading progress\n  onLoad={(llm) =\u003e {}} // Called when model is ready\n  onError={(err) =\u003e {}} // Error handler\n\u003e\n  {children}\n\u003c/LLMProvider\u003e\n```\n\n\u003e **Tip:** Use `key={modelId}` on `\u003cLLMProvider\u003e` to force a full re-mount when switching models dynamically.\n\n### `useLLM()`\n\nAccess the raw LLM instance and loading state:\n\n```tsx\nconst {\n  llm, // LocalLLM | null\n  isLoading, // boolean — model is downloading\n  isReady, // boolean — model ready for inference\n  loadProgress, // { progress: number, status: string } | null\n  error, // Error | null\n  modelId, // string | null — current model ID\n  backend, // 'webllm' | 'transformers' | null\n  reload, // () =\u003e Promise\u003cvoid\u003e\n  unload, // () =\u003e Promise\u003cvoid\u003e\n} = useLLM();\n```\n\n### `useChat(options?)`\n\nFull chat conversation management with **eager loading** — users can send messages while the model downloads. Messages are queued and processed automatically once the model is ready.\n\nBy default, the `\u003cChat\u003e` component manages this hook entirely for you. However, you can call this manually to implement a head-less chat or custom interface. Multi-modal inputs (like arrays of `text` and `image_url` objects) can be passed directly to `send()` if you manage the image parsing yourself!\n\n```tsx\nconst {\n  messages, // ChatMessage[]\n  input, // string — controlled input value\n  setInput, // (value: string) =\u003e void\n  send, // (content?: string | any[]) =\u003e Promise\u003cstring\u003e\n  isGenerating, // boolean\n  isPending, // boolean — message queued, waiting for model\n  streamingText, // string — current partial response\n  stop, // () =\u003e void\n  clear, // () =\u003e void\n  append, // (message: ChatMessage) =\u003e void\n  reload, // () =\u003e Promise\u003cstring\u003e — regenerate last response\n} = useChat({\n  systemPrompt: \"You are a helpful assistant.\",\n  queueWhileLoading: true, // default: true\n  initialMessages: [],\n  generateOptions: { temperature: 0.7, maxTokens: 512 },\n  onStart: () =\u003e {},\n  onToken: (token, fullText) =\u003e {},\n  onFinish: (response) =\u003e {},\n  onError: (error) =\u003e {},\n});\n```\n\n### `useStream(options?)`\n\nSimple streaming generation without chat history management:\n\n```tsx\nconst { text, isStreaming, stream, stop, clear } = useStream({\n  onToken: (token, fullText) =\u003e {},\n  onFinish: (response) =\u003e {},\n  onError: (error) =\u003e {},\n  generateOptions: { temperature: 0.7 },\n});\n\nawait stream(\"Tell me a story\");\n// or with message array:\nawait stream([{ role: \"user\", content: \"Tell me a story\" }]);\n```\n\n### `useCompletion(options?)`\n\nNon-streaming, single-shot completion:\n\n```tsx\nconst { completion, isLoading, complete, clear } = useCompletion({\n  generateOptions: { maxTokens: 256 },\n});\n\nconst result = await complete(\"Summarize this text\");\n```\n\n### `\u003cLLMLoading\u003e` / `\u003cLLMReady\u003e`\n\nConditional rendering components:\n\n```tsx\n\u003cLLMLoading className=\"loading-state\"\u003e\n  \u003cp\u003eDownloading model...\u003c/p\u003e\n\u003c/LLMLoading\u003e\n\n\u003cLLMReady fallback={\u003cSpinner /\u003e}\u003e\n  \u003cChatInterface /\u003e\n\u003c/LLMReady\u003e\n```\n\n### Default Chat Interface explained\n\nThe `\u003cChat\u003e` and `\u003cChatApp\u003e` components provide a production-ready, minimalist, borderless \"Terminal\" interface inspired by brutalist and sacred computer aesthetics.\n\n**Key Features \u0026 Usage Points:**\n\n- **✨ Zero Config**: Just drop it in. No CSS files to import, no state to manage. All spacing, monospace typography, and layout are handled out-of-the-box.\n- **🎨 Rich Text Rendering**:\n  - **Global Markdown**: Bold, cursives, lists, tables.\n  - **Code Blocks**: Syntax highlighting for 20+ languages cleanly integrated without borders.\n  - **Diagrams**: Renders `mermaid` diagrams automatically.\n  - **Math**: Supports LateX expressions.\n- **⚡ Eager Interaction**: Users can type and send messages _while_ the model is still initializing. The chat controls the queue.\n- **🌗 Multimodal \u0026 Files**: Deeply integrated multimodal chat input. Simply drag and drop PDFs for automatic local text extraction, or drag-and-drop/paste images for seamless processing against vision models (e.g., `phi-3.5-vision`). WebLLM crash mitigations for dynamic image cropping are automatically handled.\n- **🔄 Model Switching**:\n  - If using `\u003cChatApp /\u003e`, a model selector dropdown is included automatically.\n  - If using `\u003cChat /\u003e`, pass `onModelChange` to enable the dropdown.\n- **🛠️ Extensible Toolbar**: Use the `inputActions` prop to add your own buttons (e.g., upload, clear) to the input area.\n\n### `\u003cChat\u003e` Component API\n\n```tsx\n\u003cChat\n  // Appearance\n  theme=\"dark\" // 'dark' | 'light'\n  maxHeight=\"600px\" // CSS max-height\n  className=\"my-chat\" // Extra classes\n  // Content\n  systemPrompt=\"...\" // Default: \"You are a helpful AI assistant...\"\n  welcomeMessage=\"...\" // Text shown when chat is empty\n  placeholder=\"...\" // Input placeholder\n  // Features\n  showHeader={true} // Toggle header/model info\n  showProgress={true} // Toggle loading progress bar\n  // Callbacks\n  onSend={(msg) =\u003e {}} // Listen to user messages\n  onResponse={(res) =\u003e {}} // Listen to AI responses\n  onModelChange={(id) =\u003e {}} // Enable model switching dropdown\n  inputActions={\n    \u003c\u003e\n      \u003cbutton\u003eClear\u003c/button\u003e\n    \u003c/\u003e\n  } // Add custom buttons\n/\u003e\n```\n\n### `\u003cChatApp\u003e` Component API\n\nWrapper that combines `LLMProvider` and `Chat`.\n\n```tsx\n\u003cChatApp\n  defaultModel=\"qwen-2.5-0.5b\"\n  defaultBackend=\"auto\" // 'webllm' | 'transformers'\n  autoLoad={true} // Start downloading immediately\n  {...chatProps} // All \u003cChat\u003e props are supported\n/\u003e\n```\n\n### `\u003cChatInput\u003e`\n\nStandalone auto-resizing input component. Use it to build custom chat layouts:\n\n```tsx\nimport { ChatInput } from \"@blank-utils/llm/react\";\n\n\u003cChatInput\n  value={input} // Controlled value\n  onChange={setInput} // Value change handler\n  onSend={handleSend} // Submit handler (Enter or button)\n  onStop={handleStop} // Stop generation\n  disabled={false} // Disable input\n  isGenerating={false} // Show stop button instead of send\n  placeholder=\"Type...\" // Placeholder text\n  maxRows={5} // Max rows before scroll\n  theme=\"dark\" // 'dark' | 'light'\n  actions={\u003cMyButtons /\u003e} // Custom toolbar actions\n/\u003e;\n```\n\n**Features:**\n\n- 📝 Auto-resizes up to `maxRows` then scrolls\n- ⌨️ Enter to send, Shift+Enter for newline\n- ⏹️ Stop button while generating\n- 🎨 Dark/light theme support\n- 📄 **Drag-and-drop or click to upload PDF files for automatic local text extraction (OCR)**\n- 🖼️ **Paste/upload images directly into multimodal models**\n\n---\n\n## Vanilla JS API\n\n### `createLLM(config?)`\n\nFactory function that auto-detects capabilities, picks a backend, loads a model, and returns a ready-to-use `LocalLLM` instance:\n\n```typescript\nimport { createLLM } from \"@blank-utils/llm\";\n\nconst llm = await createLLM({\n  model: \"phi-3.5-mini\",\n  backend: \"auto\",\n  systemPrompt: \"You are a helpful assistant.\",\n  onLoadProgress: (p) =\u003e console.log(`${p.status}: ${p.progress}%`),\n});\n\n// Chat (non-streaming)\nconst answer = await llm.chat(\"What is 2+2?\");\n\n// Stream\nawait llm.stream(\"Write a poem\", (token, fullText) =\u003e {\n  process.stdout.write(token);\n});\n\n// Attach to DOM\nconst cleanup = llm.attachToInput(\"#input\", \"#output\");\n\n// Free resources\nawait llm.unload();\n```\n\n### DOM Helpers\n\n```typescript\nimport {\n  createOutputStreamer,\n  attachToElements,\n  createChatUI,\n  createLoadingIndicator,\n} from \"@blank-utils/llm\";\n\n// Auto-scroll streaming output into an element\nconst streamer = createOutputStreamer(\"#output\", { scrollToBottom: true });\n\n// Create a full chat UI in one call\nconst { input, output, sendButton, cleanup } = createChatUI(\"#container\");\n\n// Progress indicator\nconst loading = createLoadingIndicator(\"#loading-container\");\nloading.show();\nloading.setProgress(50, \"Downloading weights...\");\nloading.hide();\n```\n\n### Capability Detection\n\n```typescript\nimport {\n  detectCapabilities,\n  logCapabilities,\n  isWebGPUSupported,\n} from \"@blank-utils/llm\";\n\nconst caps = await detectCapabilities();\n// { webgpu: true, wasm: true, recommendedBackend: 'webllm', recommendedDevice: 'webgpu' }\n\nawait logCapabilities(); // Pretty-prints to console\n\nif (await isWebGPUSupported()) {\n  console.log(\"WebGPU is available!\");\n}\n```\n\n---\n\n## Available Models\n\nAll models are defined in `src/models.ts` and exported as `WEBLLM_MODELS` and `TRANSFORMERS_MODELS`.  \nYou can use either the **alias** (short name) or the **full model ID** when specifying a model.\n\n### WebLLM Backend (WebGPU)\n\nThe WebLLM backend supports **over 80+ WebGPU-accelerated models**, organized into major ecosystems. All models are strongly typed and natively supported:\n\n- **Microsoft Phi**: `phi-3.5-mini`, `phi-3.5-vision` (Multimodal support!), `phi-3-mini-4k`, `phi-2`, `phi-1.5`\n- **Meta Llama**: `llama-3.2-1b`/`3b`, `llama-3.1-8b`/`70b`, `llama-3-8b`/`70b`, `llama-2-7b`/`13b`\n- **Alibaba Qwen**: `qwen3` (`0.6b` to `8b`), `qwen-2.5` (`0.5b` to `7b`), `qwen-2.5-coder` (`0.5b` to `7b`), `qwen-2.5-math` (`1.5b`)\n- **Google Gemma**: `gemma-2-2b`, `gemma-2-9b`, `gemma-2-2b-jpn`, `gemma-2b`\n- **HuggingFace SmolLM2**: `smollm2-135m` (~360MB VRAM!), `smollm2-360m`, `smollm2-1.7b`\n- **Mistral \u0026 Ministral**: `mistral-7b` (v0.2/v0.3), `ministral-3-3b-base`/`reasoning`/`instruct`\n- **DeepSeek Reasoning**: `deepseek-r1-qwen-7b`, `deepseek-r1-llama-8b`\n- **NousResearch Hermes**: `hermes-3-llama-3.2-3b`/`3.1-8b`, `hermes-2-theta/pro-llama`, `hermes-pro/openhermes-mistral`\n- **Other**: `tinyllama-1.1b`, `redpajama-3b`, `stablelm-2-zephyr-1.6b`, `wizardmath-7b`\n\n_Note: Many models have `-1k` variants (e.g. `llama-3.1-8b-1k`) with smaller context windows intentionally configured to require significantly less VRAM on constrained devices._\n\n### Transformers.js Backend (CPU / WASM)\n\n| Alias                 | HuggingFace Model ID                         | Notes        |\n| --------------------- | -------------------------------------------- | ------------ |\n| `qwen-2.5-0.5b`       | `onnx-community/Qwen2.5-0.5B-Instruct`       | **Default**  |\n| `qwen-2.5-1.5b`       | `onnx-community/Qwen2.5-1.5B-Instruct`       | Good quality |\n| `qwen-2.5-coder-0.5b` | `onnx-community/Qwen2.5-Coder-0.5B-Instruct` | Code         |\n| `qwen-2.5-coder-1.5b` | `onnx-community/Qwen2.5-Coder-1.5B-Instruct` | Code         |\n| `smollm2-135m`        | `HuggingFaceTB/SmolLM2-135M-Instruct`        | Ultra fast   |\n| `smollm2-360m`        | `HuggingFaceTB/SmolLM2-360M-Instruct`        | Fast         |\n| `smollm2-1.7b`        | `HuggingFaceTB/SmolLM2-1.7B-Instruct`        | Good         |\n| `phi-3-mini`          | `Xenova/Phi-3-mini-4k-instruct`              | Strong       |\n| `tinyllama`           | `Xenova/TinyLlama-1.1B-Chat-v1.0`            | Very fast    |\n\n### Type-Safe Model Selection\n\nThe `model` prop accepts any key from `WEBLLM_MODELS` or `TRANSFORMERS_MODELS` with full autocomplete, while still allowing arbitrary strings for custom models:\n\n```typescript\nimport type {\n  SupportedModel,\n  WebLLMModelID,\n  TransformersModelID,\n} from \"@blank-utils/llm\";\n\n// Full autocomplete for known models\nconst model: SupportedModel = \"qwen-2.5-0.5b\"; // ✅ autocomplete\n\n// Custom model IDs still work\nconst custom: SupportedModel = \"my-org/custom-model-onnx\"; // ✅ no error\n\n// Import the model maps for programmatic use\nimport { WEBLLM_MODELS, TRANSFORMERS_MODELS } from \"@blank-utils/llm\";\n\nObject.keys(WEBLLM_MODELS); // all WebLLM aliases\nObject.keys(TRANSFORMERS_MODELS); // all Transformers.js aliases\n```\n\n---\n\n## Build \u0026 Development\n\n```bash\n# Install dependencies\nbun install\n\n# Build (clean → bundle → assets → types)\nbun run build\n\n# Type-check only\nbun run typecheck\n\n# Run demo page\nbun run demo\n\n# Run tests\nbun test\n```\n\n### Build Pipeline\n\n| Script  | What it does                                               |\n| ------- | ---------------------------------------------------------- |\n| `clean` | Removes `dist/`                                            |\n| `build` | Super-fast bundling via `tsup`, compiling ESM code \u0026 types |\n\n### Package Exports\n\n```jsonc\n{\n  \".\": {\n    \"types\": \"./dist/index.d.ts\",\n    \"import\": \"./dist/index.js\",\n  },\n  \"./react\": {\n    \"types\": \"./dist/react/index.d.ts\",\n    \"import\": \"./dist/react/index.js\",\n  },\n}\n```\n\n---\n\n## Browser Requirements\n\n| Feature               | Minimum                    | Notes                          |\n| --------------------- | -------------------------- | ------------------------------ |\n| **WebGPU**            | Chrome 113+, Edge 113+     | Required for WebLLM backend    |\n| **WebAssembly**       | All modern browsers        | Fallback for Transformers.js   |\n| **SharedArrayBuffer** | Requires COOP/COEP headers | Needed for multi-threaded WASM |\n\nThe library automatically detects capabilities and picks the best backend. No manual configuration needed.\n\n---\n\n## License\n\nMIT © [blank](https://github.com/kiritocode1)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiritocode1%2Flocal-llm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkiritocode1%2Flocal-llm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkiritocode1%2Flocal-llm/lists"}