Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/endlessreform/voxdei
Personal LLM inference stack
https://github.com/endlessreform/voxdei
Last synced: about 6 hours ago
JSON representation
Personal LLM inference stack
- Host: GitHub
- URL: https://github.com/endlessreform/voxdei
- Owner: EndlessReform
- License: gpl-3.0
- Created: 2024-01-13T03:31:47.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-01-29T00:08:40.000Z (10 months ago)
- Last Synced: 2024-01-30T01:20:39.560Z (10 months ago)
- Language: TypeScript
- Size: 778 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Vox Dei
This monorepo is the platform for my personal inference stack:
- OpenAI-compatible backend wrapper for HuggingFace [text-generation-inference]()
- History microservice
- Chat frontend
- Annotation frontend
# Deploying this stack
TODO
## Local development setup
### Prerequisites
You should install the following if you don't have them:
- Git
- Node (through [nvm](https://github.com/nvm-sh/nvm))
- Python
- Python [Poetry](https://python-poetry.org/docs/#installing-with-the-official-installer)
### Setup
We're using [Nx](https://nx.dev) for monorepo management.
Install dependencies:
```bash
nvm install
# Install NPM dependencies using npm workspaces.
# TODO: Change install options?
npm i
# Install Python Poetry
curl -sSL https://install.python-poetry.org | python -
# Install python dependencies
npx nx run-many -t install
```
Everything in development uses `dev`.
```bash
# Run single project
npx nx run chat-ui:dev
# Run individual projects
npx nx run-many -t dev -p chat-ui chat-history-service
```
Build all projects:
```
npm run build
```
### Poetry
Add dependency to Python project:
```bash
npx nx chat-history-service:add dep
npx nx some_project:rm --name dep
```
Add local dependency:
```bash
# Add lib1 to proj2
npx nx run proj2:add --name lib1 --local
```
Create new Python project:
## Contributing
See ARCHITECTURE.md for details.