{"id":19197117,"url":"https://github.com/vearch/python-algorithm-plugin","last_synced_at":"2025-09-03T08:38:15.575Z","repository":{"id":39739375,"uuid":"248474856","full_name":"vearch/python-algorithm-plugin","owner":"vearch","description":"Algorithm plugins of generating embedding vectors used for vector search","archived":false,"fork":false,"pushed_at":"2023-03-24T23:11:59.000Z","size":5996,"stargazers_count":16,"open_issues_count":4,"forks_count":11,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-06-25T05:06:22.794Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-03-19T10:33:36.000Z","updated_at":"2024-05-31T09:12:54.000Z","dependencies_parsed_at":"2025-04-20T10:52:10.074Z","dependency_job_id":null,"html_url":"https://github.com/vearch/python-algorithm-plugin","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vearch/python-algorithm-plugin","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vearch%2Fpython-algorithm-plugin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vearch%2Fpython-algorithm-plugin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vearch%2Fpython-algorithm-plugin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vearch%2Fpython-algorithm-plugin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vearch","download_url":"https://codeload.github.com/vearch/python-algorithm-plugin/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vearch%2Fpython-algorithm-plugin/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273415841,"owners_count":25101730,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-03T02:00:09.631Z","response_time":76,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T12:15:35.779Z","updated_at":"2025-09-03T08:38:15.539Z","avatar_url":"https://github.com/vearch.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Quickstart\n\n*This  Quickstart suit for those who want to search something but do not know how to extract image or text to features. Others refer to [APILowLevel.md](APILowLevel.md) .* \n\nVearch is aimed to build a simple and fast image retrieval system. Through this system, a image retrieval system could be easily built, including image object detection,  feature extraction and similarity search. This quickstart demonstrates how to use it.\n\n![images/main_process.gif](images/main_process.gif)\n\n\n\n## Before you begin\n\n1. Deploy Vearch system referred to [Deploy.md](https://github.com/vearch/vearch/blob/master/docs/Deploy.md).\n\n For testing you can download  [coco data](https://pjreddie.com/media/files/val2014.zip), or  use the images in images folder we choose from [coco data](https://pjreddie.com/media/files/val2014.zip). For more details, you can refer test folder in `plugin.src`\n\n## Different from APILowLevel.md\n\nThis API is similar to [APILowLevel.md](https://github.com/vearch/vearch/blob/master/docs/APILowLevel.md),  and plugin can perfectly adapt to it, Any method defined in APILowLevel.md can be used by plugin. However, if vector has existed, the APILowLevel.md is the fisrt choice.\n\nThe difference:\n\n- The name of db can not be one of  ['_cluster', 'list', 'db', 'space'].\n- Can not use `_msearch` method.\n- Replace the feature field with the object requiring the feature, refer to insert or search demo.\n\n\n## Deploy a plugin service\n\nThis requires only two operations:\n\n1. Modify parameters in `src/config.py`;\n2. Execution script:\n    For image, `bash ./bin/run.sh image` ;\n    For video, `bash ./bin/run.sh video`;\n    For text, `bash ./bin/run.sh text` ;\n\n\n## Create a database and space\n\nBefore inserting and searching, a database and space should be creating firstly. Use the following `curl` command to create a new database and space.\n\n```shell\n# create a db which name test\ncurl -XPUT -H \"content-type:application/json\" -d '{\"name\": \"test\"}' http://127.0.0.1:4101/db/_create\n\n# create a space in test db which name test too.\n## for image\ncurl -XPUT -H \"content-type: application/json\" -d '{\"name\": \"test\", \"partition_num\": 1, \"replica_num\": 1, \"engine\": {\"name\": \"gamma\", \"index_size\": 70000, \"max_size\": 10000000, \"id_type\": \"String\", \"retrieval_type\": \"IVFPQ\", \"retrieval_param\": { \"metric_type\": \"InnerProduct\", \"ncentroids\": 256, \"nsubvector\": 32  } }, \"properties\": {  \"url\": { \"type\": \"keyword\", \"index\": true }, \"feature1\": { \"type\": \"vector\", \"dimension\":512, \"format\": \"normalization\"  } }}  }' http://127.0.0.1:4101/space/test/_create\n\n## for text dimension is 768\ncurl -XPUT -H \"content-type: application/json\" -d '{\"name\": \"test\", \"partition_num\": 1, \"replica_num\": 1, \"engine\": {\"name\": \"gamma\", \"index_size\": 70000, \"max_size\": 10000000, \"id_type\": \"String\", \"retrieval_type\": \"IVFPQ\", \"retrieval_param\": { \"metric_type\": \"InnerProduct\", \"ncentroids\": 256, \"nsubvector\": 32  } }, \"properties\": {  \"text\": { \"type\": \"keyword\", \"index\": true }, \"feature1\": { \"type\": \"vector\", \"dimension\":768, \"format\": \"normalization\"  } }}  }' http://127.0.0.1:4101/space/test/_create\n```\n\nA successful response looks like this:\n\n```shell\n# create db\n{\"code\":200,\"msg\":\"success\",\"data\":{\"id\":1,\"name\":\"test\"}}\n\n# create space\n{\"code\":200,\"msg\":\"success\",\"data\":{\"id\":2,\"name\":\"test\",\"version\":2,\"db_id\":1,\"enabled\":true,\"partitions\":[{\"id\":3,\"space_id\":2,\"db_id\":1,\"partition_slot\":0,\"replicas\":[1]},{\"id\":4,\"space_id\":2,\"db_id\":1,\"partition_slot\":2147483647,\"replicas\":[1]}],\"partition_num\":2,\"replica_num\":1,\"properties\":{ \"url\": { \"type\": \"keyword\", \"index\":true}, \"feature1\": { \"type\": \"vector\", \"dimension\":512, \"format\": \"normalization\" }},\"engine\":{\"name\":\"gamma\",\"index_size\":100000,\"max_size\":100000,\"nprobe\":-1,\"metric_type\":\"InnerProduct\",\"ncentroids\":-1,\"nsubvector\":-1,\"nbits_per_idx\":-1}}}\n```\n\n\n\n## Delete a database and space\n\nIf the database and space is not need again, Use the following `curl` command to delete a database and space.\n\n```shell\n\ncurl -XDELETE http://127.0.0.1:4101/space/test/test\ncurl -XDELETE http://127.0.0.1:4101/db/test\n```\n\nA successful response looks like this:\n\n```shell\n{\"code\":200,\"msg\":\"success\"}\n```\n\n\n\n## Insert data into space\n\nUse the following `curl` command to insert single data into space.\n\nThe method of single import demo:\n\n```shell\n# single insert\n\n## image\ncurl -XPOST -H \"content-type: application/json\"  -d' { \"url\": \"../images/COCO_val2014_000000123599.jpg\", \"feature1\":{\"feature\":\"../images/COCO_val2014_000000123599.jpg\"}} ' http://127.0.0.1:4101/test/test/AW63W9I4JG6WicwQX_RC\n\n## text\ncurl -XPOST -H \"content-type: application/json\"  -d' { \"text\": \"感谢大家\", \"feature1\":{\"feature\":\"感谢大家\"} } ' http://127.0.0.1:4101/test/test/AW63W9I4JG6WicwQX_RC'\n```\n\nA successful response like this:\n\n```shell\n{\"_index\":\"test\",\"_type\":\"test\",\"_id\":\"AW63W9I4JG6WicwQX_RC\",\"status\":201,\"_version\":1,\"_shards\":{\"total\":0,\"successful\":1,\"failed\":0},\"result\":\"created\",\"_seq_no\":1,\"_primary_term\":1}\n```\n\n## Get record by ID\n\nUse the following `curl` command to get a record by ID\n\n```shell\n# request\ncurl -XGET http://127.0.0.1:4101/test/test/AW63W9I4JG6WicwQX_RC\n\n# response\n{\"_index\":\"test\",\"_type\":\"test\",\"_id\":\"AW63W9I4JG6WicwQX_RC\",\"found\":true,\"_version\":1,\"_source\":{\"url\":\"../images/COCO_val2014_000000123599.jpg\"}}\n```\n\n\n\n## Delete record by ID\n\nUse the following `curl` command to delete a record by ID\n\n```shell\n# request\ncurl -XDELETE http://127.0.0.1:4101/test/test/AWz2IFBSJG6WicwQVTog\n\n# response\n{\"_index\":\"test\",\"_type\":\"test\",\"_id\":\"AW63W9I4JG6WicwQX_RC\",\"status\":200,\"_version\":0,\"_shards\":{\"total\":0,\"successful\":1,\"failed\":0},\"result\":\"unknow\",\"_seq_no\":1,\"_primary_term\":1}\n```\n\n\n\n\n## Search similar result from space\n\nSearching by an image URI for an publicly accessible online image or an image stored in images folders.\n\nSearching by an image stored in images folders or image URI on Internet, using the following `curl` command to search  similar result from space\n\n```shell\ncurl -H \"content-type: application/json\" -XPOST -d '{ \"query\": { \"sum\": [{\"feature\":\"../images/COCO_val2014_000000123599.jpg\", \"field\":\"feature1\"}]}}' http://127.0.0.1:4101/test/test/_search\n\n```\n**Note**：If no index is created, you need add a parameter in request to enforce brute force search rather than index search, eg:\n```shell\ncurl -H \"content-type: application/json\" -XPOST -d '{ \"query\": { \"sum\": [{\"feature\":\"../images/COCO_val2014_000000123599.jpg\", \"field\":\"feature1\"}]},\"is_brute_search\":1}' http://127.0.0.1:4101/test/test/_search\n\n```\n\nA successful response looks like this:\n\n```shell\n{\"took\":14,\"timed_out\":false,\"_shards\":{\"total\":2,\"failed\":0,\"successful\":2},\"hits\":{\"total\":1,\"max_score\":0.9999997615814209,\"hits\":[{\"_index\":\"test\",\"_type\":\"test\",\"_id\":\"AW8OftTLJG6WicwQyAt2\",\"_score\":0.9999997615814209,\"_extra\":{\"vector_result\":[{\"field\":\"feature1\",\"source\":\"\",\"score\":0.9999997615814209}]},\"_version\":1,\"_source\":{\"url\":\"../images/COCO_val2014_000000123599.jpg\"}}]}}\n```\n\nsearch result look like this\n\n![images/COCO_val2014_000000123599.jpg](images/COCO_val2014_000000123599.jpg)\n\n![images/result.jpg](images/result.jpg)\n\n\n# Docker\n\n* build: `docker build . -t vearch/plugin:latest`\n  * add `-f Dockerfile.NonChina` to not use the Chinese docker index\n* run: `docker run -p 4101:4101 vearch/plugin`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvearch%2Fpython-algorithm-plugin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvearch%2Fpython-algorithm-plugin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvearch%2Fpython-algorithm-plugin/lists"}