https://github.com/devidw/inferit
Visual inference exploration & experimentation playground
https://github.com/devidw/inferit
chrome-ai chromeai inference-client openai-compatible-api
Last synced: about 2 months ago
JSON representation
Visual inference exploration & experimentation playground
- Host: GitHub
- URL: https://github.com/devidw/inferit
- Owner: devidw
- License: agpl-3.0
- Created: 2024-11-05T02:49:19.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-11-29T22:44:23.000Z (6 months ago)
- Last Synced: 2025-04-02T08:09:54.318Z (2 months ago)
- Topics: chrome-ai, chromeai, inference-client, openai-compatible-api
- Language: Svelte
- Homepage: https://inferit.index.garden
- Size: 2.33 MB
- Stars: 88
- Watchers: 3
- Forks: 4
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: license.txt
Awesome Lists containing this project
README
inferit
_inferit_ is a visual take on llm inference. Most inference frontends are
limited to a single visual input/output "thread". This makes it hard to compare
output from different models, prompts and sampler settings. _inferit_ solves
this with its UI that allows for an unlimited number of side-by-side
generations. This makes it a perfect fit to compare and experiment with
different models, prompts and sampler settings.
Some example use cases:
- model exploration and comparison
- prompt engineering
- sampler setting optimizations
Supported Backends:
- Any local or remote backend that is compatible with the OpenAI API
- [Chrome's built-in llm](https://developer.chrome.com/docs/ai/built-in) (Gemini Nano)
## run it
1. I've deployed an instance for instant access here: https://inferit.index.garden (online)
2. Same code also ships as browser extension to run local and offline w/o having to start a process:
https://chromewebstore.google.com/detail/inferit/celkhcifjknihgjlieolcmchofdloagk
3. Run it locally from this repo:
```bash
pnpm install
pnpm build
pnpm preview
```
once running, set your api creds in the settings (stored on your device, [window.localStorage](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage)) and u are good to go
## development
```bash
pnpm dev
```
Contributions of any kind are warmly welcomed!
<3