{"id":28965952,"url":"https://github.com/devflowinc/embedding-performance","last_synced_at":"2025-06-24T07:10:42.013Z","repository":{"id":216821816,"uuid":"742205556","full_name":"devflowinc/embedding-performance","owner":"devflowinc","description":"Testing tps for embeddding servers","archived":false,"fork":false,"pushed_at":"2024-01-14T04:11:46.000Z","size":14,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-22T05:17:05.293Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devflowinc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2024-01-12T01:04:33.000Z","updated_at":"2024-01-12T01:04:41.000Z","dependencies_parsed_at":"2024-01-13T03:35:44.172Z","dependency_job_id":"bad1b61b-c5df-4d29-9b43-e5fb10cb11de","html_url":"https://github.com/devflowinc/embedding-performance","commit_stats":null,"previous_names":["arguflow/embedding-performance","devflowinc/embedding-performance"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/devflowinc/embedding-performance","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devflowinc%2Fembedding-performance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devflowinc%2Fembedding-performance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devflowinc%2Fembedding-performance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devflowinc%2Fembedding-performance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devflowinc","download_url":"https://codeload.github.com/devflowinc/embedding-performance/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devflowinc%2Fembedding-performance/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261624969,"owners_count":23186121,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-24T07:10:41.388Z","updated_at":"2025-06-24T07:10:41.999Z","avatar_url":"https://github.com/devflowinc.png","language":"Python","readme":"# hi\n\nThis is meant to test the performance of embedding servers\nUsing terraform only for the purpose of reproducability.\n\n## Testing Setup\n\nCreate a markdown graph\n```mermaid\ngraph LR\n    A[side_car] --\u003e B((splade_server))\n    C[side_car] --\u003e D((embedding_server))\n```\n\n## Setup terraform\n\nDownload terraform cli\nDownload aws cli\n\nrun `aws configure` You will see a prompt similar to this\n\n```\nuser@machine $ aws configure\nAWS Access Key ID [****************PYVK]: ****PYVK\nAWS Secret Access Key [****************duMt]: ****duMt\nDefault region name [eu-central-1]: \nDefault output format [None]: \n\nuser@machine $\n```\n\nTo get the ACCESS KEY and SECRET KEY you must create a user, give that user\nadmin perms and request for an access key for cli usage. The begining of this\nguide here worked for me. https://spacelift.io/blog/terraform-tutorial\n\n## Reproduce tests\n\n```sh\nterraform init\ncp \u003cyour-ssh-pub-key-location\u003e ./ssh-key.pub\nterraform apply\n```\n\nYou should see this as your output\n\n```\nside_car_ip = \"\u003cyour-side_car-ip\u003e\"\nsplade_ip = \"\u003cyour-splade-server-ip\u003e\"\n```\n\nSet up for splade_server\n```sh\n\nssh dev@\u003cyour-side_car-ip\u003e\n\napt-get install python3.9 python3-is-python python-venv\n\ngit clone https://github.com/arguflow/embedding-performance\ncd embedding-performance\npython3 -m venv venv\n. venv/bin/activate\npip install -r requirements.txt\ngranian embeddings:app --interface asgi --host 0.0.0.0 --port 7070\n```\n\nThe side_car_ip is what we will test from\n\nSet up for side_car\n```sh\nssh dev@\u003cyour-side_car-ip\u003e\n\nsudo apt-get update -y \u0026\u0026 sudo apt-get install -y luajit\ngit clone https://github.com/wg/wrk\ngit clone https://github.com/arguflow/embedding-performance\ncd wrk; make\ncd ../embedding-performance/\n\n./test.sh \u003cyour-splade-server-ip\u003e:7070\n./test.sh \u003cyour-embedding-server-ip\u003e:7070\n```\n\nTo change the machine type\n```sh\nupdate the server-machine-type in main.tf\n```\n\nTo bring down all servers \n\n```sh\nterraform destroy\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevflowinc%2Fembedding-performance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevflowinc%2Fembedding-performance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevflowinc%2Fembedding-performance/lists"}