https://github.com/hilum-labs/local-llm-web
Web JavaScript SDK for local LLM inference with WebGPU and on-device AI.
https://github.com/hilum-labs/local-llm-web
client-side-ai javascript llm local-llm on-device-inference sdk typescript web web-ai webgpu
Last synced: 5 days ago
JSON representation
Web JavaScript SDK for local LLM inference with WebGPU and on-device AI.
- Host: GitHub
- URL: https://github.com/hilum-labs/local-llm-web
- Owner: hilum-labs
- Created: 2026-03-09T23:26:17.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2026-03-10T02:58:47.000Z (4 months ago)
- Last Synced: 2026-03-10T10:39:12.788Z (4 months ago)
- Topics: client-side-ai, javascript, llm, local-llm, on-device-inference, sdk, typescript, web, web-ai, webgpu
- Size: 1000 Bytes
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# local-llm-web
Web JavaScript SDK for local LLM inference with WebGPU.
`local-llm-web` is an open source SDK for running large language models in web applications with on-device inference, WebGPU acceleration, streaming generation, and local-first execution.
## What It Is
- Web SDK for local LLM inference
- JavaScript and TypeScript API for chat, generation, and model loading
- High-level package built on `local-llm-web-core`
- Local-first alternative to cloud-only LLM APIs
## Planned Capabilities
- Load quantized LLMs in web applications
- Run chat and text generation fully on-device
- Stream tokens with web-friendly APIs
- Use Web Workers for non-blocking inference
- Keep model execution private and local
## Use Cases
- Web AI applications
- Private on-device chat interfaces
- Local AI copilots
- Offline-capable LLM apps
- WebGPU-powered client-side inference
## Status
Active development. This repository is the public SDK layer for the web package family.
The lower-level runtime core lives in [`local-llm-web-core`](https://github.com/hilum-labs/local-llm-web-core).
## Package Direction
```bash
npm install local-llm-web
```
## Related Repositories
- [`local-llm-web-core`](https://github.com/hilum-labs/local-llm-web-core): WebGPU runtime core
- [`local-llm`](https://github.com/hilum-labs/local-llm): Node.js local LLM runtime
- [`hilum-local-llm-engine`](https://github.com/hilum-labs/hilum-local-llm-engine): native C/C++ inference engine