Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/intelligentnode/IntelliServer
AI models as scalable microservices, enabling evaluation of LLMs and offering end-to-end functions such as chatbot, semantic search, image generation and beyond.
https://github.com/intelligentnode/IntelliServer
ai chatbot cohere docker gpt4 image-generation llama2 llm microservice nodejs openai replicate semantic-search stable-diffusion
Last synced: about 2 months ago
JSON representation
AI models as scalable microservices, enabling evaluation of LLMs and offering end-to-end functions such as chatbot, semantic search, image generation and beyond.
- Host: GitHub
- URL: https://github.com/intelligentnode/IntelliServer
- Owner: intelligentnode
- License: mit
- Created: 2023-08-27T20:48:56.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-02-18T22:31:06.000Z (7 months ago)
- Last Synced: 2024-07-19T22:05:21.428Z (2 months ago)
- Topics: ai, chatbot, cohere, docker, gpt4, image-generation, llama2, llm, microservice, nodejs, openai, replicate, semantic-search, stable-diffusion
- Language: JavaScript
- Homepage: http://intelliserver.us-east-1.elasticbeanstalk.com/
- Size: 2.77 MB
- Stars: 26
- Watchers: 4
- Forks: 3
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- Awesome-LLM - IntelliServer - simplifies the evaluation of LLMs by providing a unified microservice to access and test multiple AI models. (LLM Deployment)
README
# IntelliServer
AI models as private microservice - ChatGPT, Cohere, Llama, Stability, Hugging inference and more.
Intelliserver is a microservice providing unified access to multiple AI models, allowing you to easily integrate cutting-edge AI into your project.
[](https://god.gw.postman.com/run-collection/29770592-caa8dae9-7eea-46a6-b64c-ada5c07be9f4?action=collection%2Ffork&source=rip_markdown&collection-url=entityId%3D29770592-caa8dae9-7eea-46a6-b64c-ada5c07be9f4%26entityType%3Dcollection%26workspaceId%3D0b88b327-c88a-4050-a084-cd4486f86b68#?env%5Bintelliserver-showcase%5D=W3sia2V5IjoidXJsIiwidmFsdWUiOiIiLCJlbmFibGVkIjp0cnVlLCJ0eXBlIjoiZGVmYXVsdCIsInNlc3Npb25WYWx1ZSI6IiIsInNlc3Npb25JbmRleCI6MH1d)
## Core Services
- **Chatbot**: chatbot functionalities using popular models like ChatGPT, Llama, and AWS Sagemaker models.
- **LLM Evaluation**: evaluate different AI models to choose the optimal solution for your requirements.
- **Semantic Search**: leverage context-aware semantic search capabilities across text documents.
- **Image Generation**: generate quality images based on described contexts using diffusion image models.
- **Chat Context**: get the relevant messages for the chatbot conversation.
- **Parsers**: convert documents to text such as PDF and word.
- **OCR**: extract text from images using AWS or Google vision.
## Installation
### Repository Setup
Instructions to run the microservice from GitHub repo:
**npm**
```bash
cd intelliserver
npm install
npm start
```
**docker**
```bash
# docker run
docker build -t intelliserver:latest .
docker run -p 80:80 intelliserver:latest
# or docker compose run
docker-compose up
```
### Release (Docker Hub)
To pull the release image from docker hub:
```bash
docker pull intellinode/intelliserver:latest
```
Run IntelliServer
```bash
# run with custom keys
API_KEY=
ADMIN_KEY=
docker run -p 80:80 -e API_KEY=$API_KEY -e ADMIN_KEY=$ADMIN_KEY intellinode/intelliserver:latest
# or run with the default key - only for testing
docker run -p 80:80 intellinode/intelliserver:latest
```
-----
**Mac M-series processors**
For Mac M-series users, pull the arm64 version:
```bash
docker pull intellinode/intelliserver:arm64
```
## Testing
To test Intelliserver, you can find the endpoints collection in the [postman repository](https://github.com/intelligentnode/IntelliServer/tree/main/postman).
Or access the swagger interactive docs: `localhost/api-docs/`
To customize the default keys and settings, create a `.env` file inside intelliserver cloned repo with the following values:
```
# api keys
API_KEY=
ADMIN_KEY=
# models keys - if not added, the user should send the key in the API call
OPENAI_API_KEY=
AZURE_OPENAI_API_KEY=
COHERE_API_KEY=
GOOGLE_API_KEY=
STABILITY_API_KEY=
HUGGING_API_KEY=
REPLICATE_API_KEY=
```
## Key Benefits
- **Unified Access:** Intelliserver provides a unified API for accessing different AI models. This allows for seamless switching between models using the same endpoint format.
- **Scalability:** Intelliserver utilizes microservices architecture, allowing the AI middleware to run as an independent service with dedicated resources.
- **Model Evaluation:** Intelliserver's design allows for seamless evaluation and comparison of different AI models using unified service. This facilitates data-driven decision when selecting the optimal model for specific use cases.
## License
Intelliserver is released under the [MIT License](https://github.com/intelligentnode/IntelliServer/blob/main/LICENSE)