https://github.com/drupol/wip-services-flake-llm
A Nix flake to run LLMs locally in a blink of an eye, without container but with privacy !
https://github.com/drupol/wip-services-flake-llm
Last synced: 8 months ago
JSON representation
A Nix flake to run LLMs locally in a blink of an eye, without container but with privacy !
- Host: GitHub
- URL: https://github.com/drupol/wip-services-flake-llm
- Owner: drupol
- Created: 2024-06-20T11:25:59.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-01-05T15:22:32.000Z (over 1 year ago)
- Last Synced: 2025-01-12T06:11:04.724Z (over 1 year ago)
- Language: Nix
- Homepage:
- Size: 72.3 KB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Services Flake LLM
A [Nix flake] to locally spawn the following services:
- [Ollama]
- [Tika]
- [Searxng]
- [Open-WebUI]
- [Docling-Serve]
The services are spawned using [`process-compose`].
The data are kept in the current directory you run the command.
To run it:
```shell
nix run github:drupol/wip-services-flake-llm
```
The services will be available at the following addresses:
- Ollama: http://localhost:11434
- Tika: http://localhost:9998
- Searxng: http://localhost:8080
- Open-WebUI: http://localhost:1111
- Docling-Serve: http://localhost:5001/ and http://localhost:5001/ui
[`process-compose`]: https://github.com/F1bonacc1/process-compose
[Ollama]: https://github.com/ollama/ollama
[Tika]: https://tika.apache.org/
[Searxng]: https://github.com/searxng/searxng
[Open-WebUI]: https://github.com/open-webui/open-webui
[Nix flake]: https://wiki.nixos.org/wiki/Flakes
[Docling-Serve]: https://github.com/docling-project/docling-serve