Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/skywardai/kirin
APIs aggregator for inference, fine-tuning and build models.
https://github.com/skywardai/kirin
ai api container conversational-ai fastapi fine-tuning llamacpp llm-inference llm-training rag sentence-embeddings vector-database
Last synced: about 3 hours ago
JSON representation
APIs aggregator for inference, fine-tuning and build models.
- Host: GitHub
- URL: https://github.com/skywardai/kirin
- Owner: SkywardAI
- License: apache-2.0
- Created: 2024-03-10T14:42:02.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2024-07-06T07:16:42.000Z (3 months ago)
- Last Synced: 2024-07-06T07:21:26.057Z (3 months ago)
- Topics: ai, api, container, conversational-ai, fastapi, fine-tuning, llamacpp, llm-inference, llm-training, rag, sentence-embeddings, vector-database
- Language: Python
- Homepage: https://skywardai.github.io/skywardai.io/
- Size: 405 KB
- Stars: 4
- Watchers: 2
- Forks: 6
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
Kirin
[](https://github.com/SkywardAI/chat-backend/actions/workflows/linter-and-builder.yaml) [](https://github.com/SkywardAI/chat-backend/actions/workflows/release-drafter.yml) [](https://github.com/SkywardAI/kirin/actions/workflows/release-image.yaml) [](https://github.com/SkywardAI/kirin/actions/workflows/github-code-scanning/codeql)
>[!Warning]
>**Kirin is currently in Development**: Expect breaking changes and bugs!
# Architecture
%20-%20APIs%20aggregator.svg)
# Doing inference on 12 CPUs
https://github.com/user-attachments/assets/3890b4cd-392b-489f-ad48-4e9692d4dc15
# Roadmap
- [x] **Inference mode(default)**: Chat with the small language model at consumer grade hardware with 8 CPUs
- [x] **RAG mode**: Chat with small language model based on pre-processed datasets
- [x] **CRM feature**s: Multiple users login, chat/chat(RAG), save chat history and share the chat history by JSON format
- [ ] **Neural Network define**
- [ ] Pre-define neural network as template, support load weights and fine-tune
- [ ] Support multiple neural network types
- [ ] Support custom simple neural network by UI
- [ ] **Training Factory**
- [ ] Train the neural network with the given dataset
- [ ] Quantize the model and upload it to the server with the given tokens
- [ ] Visualize the neural network and the training process
# Requirements
You need to make sure Docker and docker-compose on your environment. See [Requirements.md](./docs/Requirements.md)
# Quick setup
If you want to setup the project quickly, please follow the steps below:
Please check the [Requirements.md](./docs/Requirements.md) to prepare you env
```
git clone https://github.com/SkywardAI/kirin.git
cd kirin
make demo
```
# Build and deployment
[](https://youtu.be/63OxSmcBkhI?si=G82BOtcwRvQE7dLU)
And if you are interested in docker-in-docker development, see [Development.md](./docs/Development.md)
# Tech stack
This is a repository is the API aggregator of SkywardAI. It's using the following tech stack:
* 🐳 [Dockerized](https://www.docker.com/)
* 🐘 [Asynchronous PostgreSQL](https://www.postgresql.org/docs/current/libpq-async.html)
* 🐍 [FastAPI](https://fastapi.tiangolo.com/)
When the `Docker` is started, these are the URL addresses:
* Backend Application (API docs) $\rightarrow$ `http://localhost:8000/docs`
* Database editor (Adminer) $\rightarrow$ `http//localhost:8081`
## Why the above Tech-Stack?
Well, the easy answer is **Asynchronousity** and **Speed**!
* **FastAPI** is crowned as the fastest web framework for Python and thus we use it for our backend development.
* **Docker** is a technology that packages an application into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime.
## Other Technologies
The above-listed technologies are just the main ones. There are other technologies utilized in this project template to ensure that your application is robust and provides the best possible development environment for your team! These technologies are:
* [CodeCov](https://about.codecov.io/) $\rightarrow$ A platform that analyzes the result of your automated tests.
* [PyTest](https://docs.pytest.org/en/7.2.x/) $\rightarrow$ The testing framework for Python code.
* [DBDiagram](https://dbdiagram.io/home) $\rightarrow$ A platform that lets your design your database by writing SQL and converting it into ERD. This platform provides a complete symbol for entity relationships (not like many other platforms!).
* [GitHub Actions](https://github.com/features/actions) $\rightarrow$ The platform to setup our CI/CD by GitHub.
* [CODEOWNERS](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners) $\rightarrow$ A file for distributing the responsibilities in our project to each team/teammate.
## Acknowledgements
* [LocalAI](https://github.com/mudler/LocalAI)
* [FastAPI](https://fastapi.tiangolo.com/)
* [llama.cpp](https://github.com/ggerganov/llama.cpp)
* [LanceDB](https://lancedb.github.io/lancedb/)
* [HuggingFace Community](https://huggingface.co/aisuko)
* [RMIT Race Hub](https://race.rmit.edu.au)
* [Microsoft Phi-3 Series](https://huggingface.co/aisuko/Phi-3-mini-4k-instruct-gguf)
# LICENSE
This project is licensed under the terms of the Apache 2.0 license. See the [LICENSE](./LICENSE.md) file.