https://github.com/devflowinc/embedding-performance
Testing tps for embeddding servers
https://github.com/devflowinc/embedding-performance
Last synced: 6 months ago
JSON representation
Testing tps for embeddding servers
- Host: GitHub
- URL: https://github.com/devflowinc/embedding-performance
- Owner: devflowinc
- Created: 2024-01-12T01:04:33.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-14T04:11:46.000Z (almost 2 years ago)
- Last Synced: 2025-06-22T05:17:05.293Z (6 months ago)
- Language: Python
- Size: 13.7 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# hi
This is meant to test the performance of embedding servers
Using terraform only for the purpose of reproducability.
## Testing Setup
Create a markdown graph
```mermaid
graph LR
A[side_car] --> B((splade_server))
C[side_car] --> D((embedding_server))
```
## Setup terraform
Download terraform cli
Download aws cli
run `aws configure` You will see a prompt similar to this
```
user@machine $ aws configure
AWS Access Key ID [****************PYVK]: ****PYVK
AWS Secret Access Key [****************duMt]: ****duMt
Default region name [eu-central-1]:
Default output format [None]:
user@machine $
```
To get the ACCESS KEY and SECRET KEY you must create a user, give that user
admin perms and request for an access key for cli usage. The begining of this
guide here worked for me. https://spacelift.io/blog/terraform-tutorial
## Reproduce tests
```sh
terraform init
cp ./ssh-key.pub
terraform apply
```
You should see this as your output
```
side_car_ip = ""
splade_ip = ""
```
Set up for splade_server
```sh
ssh dev@
apt-get install python3.9 python3-is-python python-venv
git clone https://github.com/arguflow/embedding-performance
cd embedding-performance
python3 -m venv venv
. venv/bin/activate
pip install -r requirements.txt
granian embeddings:app --interface asgi --host 0.0.0.0 --port 7070
```
The side_car_ip is what we will test from
Set up for side_car
```sh
ssh dev@
sudo apt-get update -y && sudo apt-get install -y luajit
git clone https://github.com/wg/wrk
git clone https://github.com/arguflow/embedding-performance
cd wrk; make
cd ../embedding-performance/
./test.sh :7070
./test.sh :7070
```
To change the machine type
```sh
update the server-machine-type in main.tf
```
To bring down all servers
```sh
terraform destroy
```