Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/GaiaNet-AI/gaianet-node
https://github.com/GaiaNet-AI/gaianet-node
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/GaiaNet-AI/gaianet-node
- Owner: GaiaNet-AI
- License: gpl-3.0
- Created: 2024-03-06T10:57:45.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2024-04-13T04:50:50.000Z (8 months ago)
- Last Synced: 2024-04-14T02:17:09.294Z (8 months ago)
- Language: Shell
- Size: 28.9 MB
- Stars: 12
- Watchers: 6
- Forks: 6
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- AiTreasureBox - GaiaNet-AI/gaianet-node - 12-20_4853_0](https://img.shields.io/github/stars/GaiaNet-AI/gaianet-node.svg)|Install and run your own AI agent service| (Repos)
- awesome-homelab - GaiaNet - AI/gaianet-node?style=flat)  | Install and run your own AI agent service | (Apps / AI)
README
# Run your own GaiaNet node
[Japanese(日本語)](README-ja.md) | [Chinese(中文)](README-cn.md) | [Turkish (Türkçe)](README-tr.md) | [Farsi(فارسی)](README-fa.md) | [Arabic (العربية)](README-ar.md) | [Indonesia](README-id.md) | [Russian (русскийة)](README-ru.md) | [Portuguese (português)](README-pt.md) | We need your help to translate this README into your native language.
Like our work? ⭐ Star us!
Checkout our [official docs](https://docs.gaianet.ai/) and a [Manning ebook](https://www.manning.com/liveprojectseries/open-source-llms-on-your-own-computer) on how to customize open source models.
---
## Quick start
Install the default node software stack with a single line of command on Mac, Linux, or Windows WSL.
```bash
curl -sSfL 'https://github.com/GaiaNet-AI/gaianet-node/releases/latest/download/install.sh' | bash
```
Then, follow the prompt on your screen to set up the environment path. The command line will begin with `source`.
Initialize the node. It will download the model files and vector database files specified in the `$HOME/gaianet/config.json` file, and it could take a few minutes since the files are large.
```bash
gaianet init
```
Start the node.
```bash
gaianet start
```
The script prints the official node address on the console as follows.
You can open a browser to that URL to see the node information and then chat with the AI agent on the node.
```
... ... https://0xf63939431ee11267f4855a166e11cc44d24960c0.us.gaianet.network
```
To stop the node, you can run the following script.
```bash
gaianet stop
```
## Install guide
```bash
curl -sSfL 'https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/install.sh' | bash
```
The output should look like below:
```console
[+] Downloading default config file ...
[+] Downloading nodeid.json ...
[+] Installing WasmEdge with wasi-nn_ggml plugin ...
Info: Detected Linux-x86_64
Info: WasmEdge Installation at /home/azureuser/.wasmedge
Info: Fetching WasmEdge-0.13.5
/tmp/wasmedge.2884467 ~/gaianet
######################################################################## 100.0%
~/gaianet
Info: Fetching WasmEdge-GGML-Plugin
Info: Detected CUDA version:
/tmp/wasmedge.2884467 ~/gaianet
######################################################################## 100.0%
~/gaianet
Installation of wasmedge-0.13.5 successful
WasmEdge binaries accessible
The WasmEdge Runtime wasmedge version 0.13.5 is installed in /home/azureuser/.wasmedge/bin/wasmedge.
[+] Installing Qdrant binary...
* Download Qdrant binary
################################################################################################## 100.0%
* Initialize Qdrant directory
[+] Downloading the rag-api-server.wasm ...
################################################################################################## 100.0%
[+] Downloading dashboard ...
################################################################################################## 100.0%
```
By default, it installs into the `$HOME/gaianet` directory. You can also choose to install into an alternative directory.
```bash
curl -sSfL 'https://raw.githubusercontent.com/GaiaNet-AI/gaianet-node/main/install.sh' | bash -s -- --base $HOME/gaianet.alt
```
## Initialize the node
```
gaianet init
```
The output should look like below:
```bash
[+] Downloading Llama-2-7b-chat-hf-Q5_K_M.gguf ...
############################################################################################################################## 100.0%############################################################################################################################## 100.0%
[+] Downloading all-MiniLM-L6-v2-ggml-model-f16.gguf ...
############################################################################################################################## 100.0%############################################################################################################################## 100.0%
[+] Creating 'default' collection in the Qdrant instance ...
* Start a Qdrant instance ...
* Remove the existed 'default' Qdrant collection ...
* Download Qdrant collection snapshot ...
############################################################################################################################## 100.0%############################################################################################################################## 100.0%
* Import the Qdrant collection snapshot ...
* Recovery is done successfully
```
The `init` command initializes the node according to the `$HOME/gaianet/config.json` file. You can use some of our pre-set configurations. For example, the command below initializes a node with the llama-3 8B model with a London guidebook as knowledge base.
```bash
gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct_london/config.json
```
To see a list of pre-set configurations, you can do `gaianet init --help`.
Besides a pre-set configurations like `gaianet_docs`, you can also pass a URL to your own `config.json` for the node to be initialized to the state you'd like.
If you need to `init` a node installed in an alternative directory, do this.
```bash
gaianet init --base $HOME/gaianet.alt
```
## Start the node
```
gaianet start
```
The output should look like below:
```bash
[+] Starting Qdrant instance ...
Qdrant instance started with pid: 39762
[+] Starting LlamaEdge API Server ...
Run the following command to start the LlamaEdge API Server:
wasmedge --dir .:./dashboard --nn-preload default:GGML:AUTO:Llama-2-7b-chat-hf-Q5_K_M.gguf --nn-preload embedding:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf rag-api-server.wasm --model-name Llama-2-7b-chat-hf-Q5_K_M,all-MiniLM-L6-v2-ggml-model-f16 --ctx-size 4096,384 --prompt-template llama-2-chat --qdrant-collection-name default --web-ui ./ --socket-addr 0.0.0.0:8080 --log-prompts --log-stat --rag-prompt "Use the following pieces of context to answer the user's question.\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n"
LlamaEdge API Server started with pid: 39796
```
You can start the node for local use. It will be only accessible via `localhost` and not available on any of the GaiaNet domain's public URLs.
```bash
gaianet start --local-only
```
You can also start a node installed in an alternative base directory.
```bash
gaianet start --base $HOME/gaianet.alt
```
### Stop the node
```bash
gaianet stop
```
The output should look like below:
```bash
[+] Stopping WasmEdge, Qdrant and frpc ...
```
Stop a node installed in an alternative base directory.
```bash
gaianet stop --base $HOME/gaianet.alt
```
### Update configuration
Using `gaianet config` subcommand can update the key fields defined in the `config.json` file. You MUST run `gaianet init` again after you update the configuartion.
To update the `chat` field, for example, use the following command:
```bash
gaianet config --chat-url "https://huggingface.co/second-state/Llama-2-13B-Chat-GGUF/resolve/main/Llama-2-13b-chat-hf-Q5_K_M.gguf"
```
To update the `chat_ctx_size` field, for example, use the following command:
```bash
gaianet config --chat-ctx-size 5120
```
Below are all options of the `config` subcommand.
```console
$ gaianet config --help
Usage: gaianet config [OPTIONS]
Options:
--chat-url Update the url of chat model.
--chat-ctx-size Update the context size of chat model.
--embedding-url Update the url of embedding model.
--embedding-ctx-size Update the context size of embedding model.
--prompt-template Update the prompt template of chat model.
--port Update the port of LlamaEdge API Server.
--system-prompt Update the system prompt.
--rag-prompt Update the rag prompt.
--rag-policy Update the rag policy [Possible values: system-message, last-user-message].
--reverse-prompt Update the reverse prompt.
--domain Update the domain of GaiaNet node.
--snapshot Update the Qdrant snapshot.
--qdrant-limit Update the max number of result to return.
--qdrant-score-threshold Update the minimal score threshold for the result.
--base The base directory of GaiaNet node.
--help Show this help message
```
Have fun!