{"id":19785394,"url":"https://github.com/michaelfeil/embed","last_synced_at":"2025-07-31T21:02:36.379Z","repository":{"id":249425472,"uuid":"831478114","full_name":"michaelfeil/embed","owner":"michaelfeil","description":"A stable, fast and easy-to-use inference library with a focus on a sync-to-async API","archived":false,"fork":false,"pushed_at":"2024-07-23T01:39:49.000Z","size":256,"stargazers_count":33,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-07-23T19:28:51.960Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/michaelfeil.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-20T17:05:28.000Z","updated_at":"2024-07-23T17:08:57.000Z","dependencies_parsed_at":"2024-07-20T18:44:58.659Z","dependency_job_id":null,"html_url":"https://github.com/michaelfeil/embed","commit_stats":null,"previous_names":["michaelfeil/embed"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fembed","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fembed/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fembed/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/michaelfeil%2Fembed/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/michaelfeil","download_url":"https://codeload.github.com/michaelfeil/embed/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224225081,"owners_count":17276436,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T06:14:32.183Z","updated_at":"2024-11-12T06:14:32.680Z","avatar_url":"https://github.com/michaelfeil.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# embed\nA stable, blazing fast and easy-to-use inference library with a focus on a sync-to-async API\n\n[![ci][ci-shield]][ci-url]\n[![Downloads][pepa-shield]][pepa-url]\n\n## Installation\n```bash\npip install embed\n```\n\n## Why embed?\n\nNote: Embed has a new url \u0026 moved to https://github.com/michaelfeil/infinity/tree/main/libs/embed_package.\n\nEmbed makes it easy to load any embedding, classification and reranking models from Huggingface. \nIt leverages [Infinity](https://github.com/michaelfeil/infinity) as backend for async computation, batching, and Flash-Attention-2.\n\n![CPU Benchmark Diagram](docs/l4_cpu.png)\nBenchmarking on an Nvidia-L4 instance. Note: CPU uses bert-small, CUDA uses Bert-large. [Methodology](https://michaelfeil.eu/infinity/0.0.51/benchmarking/).\n\n```python\nfrom embed import BatchedInference\nfrom concurrent.futures import Future\n\n# Run any model\nregister = BatchedInference(\n    model_id=[\n        # sentence-embeddings\n        \"michaelfeil/bge-small-en-v1.5\",\n        # sentence-embeddings and image-embeddings\n        \"jinaai/jina-clip-v1\",\n        # classification models\n        \"philschmid/tiny-bert-sst2-distilled\",\n        # rerankers\n        \"mixedbread-ai/mxbai-rerank-xsmall-v1\",\n    ],\n    # engine to `torch` or `optimum`\n    engine=\"torch\",\n    # device `cuda` (Nvidia/AMD) or `cpu`\n    device=\"cpu\",\n)\n\nsentences = [\"Paris is in France.\", \"Berlin is in Germany.\", \"A image of two cats.\"]\nimages = [\"http://images.cocodataset.org/val2017/000000039769.jpg\"]\nquestion = \"Where is Paris?\"\n\nfuture: \"Future\" = register.embed(\n    sentences=sentences, model_id=\"michaelfeil/bge-small-en-v1.5\"\n)\nfuture.result()\nregister.rerank(\n    query=question, docs=sentences, model_id=\"mixedbread-ai/mxbai-rerank-xsmall-v1\"\n)\nregister.classify(model_id=\"philschmid/tiny-bert-sst2-distilled\", sentences=sentences)\nregister.image_embed(model_id=\"jinaai/jina-clip-v1\", images=images)\n\n# manually stop the register upon termination to free model memory.\nregister.stop()\n```\n\nAll functions return `Futures(vector_embedding, token_usage)`, enables you to `wait` for them and removes batching logic from your code.\n\n```python\n\u003e\u003e\u003e embedding_fut = register.embed(sentences=sentences, model_id=\"michaelfeil/bge-small-en-v1.5\")\n\u003e\u003e\u003e print(embedding_fut)\n\u003cFuture at 0x7fa0e97e8a60 state=pending\u003e\n\u003e\u003e\u003e time.sleep(1) and print(embedding_fut)\n\u003cFuture at 0x7fa0e97e9c30 state=finished returned tuple\u003e\n\u003e\u003e\u003e embedding_fut.result()\n([array([-3.35943862e-03, ..., -3.22808176e-02], dtype=float32)], 19)\n```\n\n# Licence and Contributions\nembed is licensed as MIT. All contribrutions need to adhere to the MIT License. Contributions are welcome.\n\n\n\u003c!-- MARKDOWN LINKS \u0026 IMAGES --\u003e\n\u003c!-- https://www.markdownguide.org/basic-syntax/#reference-style-links --\u003e\n[contributors-shield]: https://img.shields.io/github/contributors/michaelfeil/embed.svg?style=for-the-badge\n[contributors-url]: https://github.com/michaelfeil/embed/graphs/contributors\n[forks-shield]: https://img.shields.io/github/forks/michaelfeil/embed.svg?style=for-the-badge\n[forks-url]: https://github.com/michaelfeil/embed/network/members\n[stars-shield]: https://img.shields.io/github/stars/michaelfeil/embed.svg?style=for-the-badge\n[stars-url]: https://github.com/michaelfeil/embed/stargazers\n[issues-shield]: https://img.shields.io/github/issues/michaelfeil/embed.svg?style=for-the-badge\n[issues-url]: https://github.com/michaelfeil/embed/issues\n[license-shield]: https://img.shields.io/github/license/michaelfeil/embed.svg?style=for-the-badge\n[license-url]: https://github.com/michaelfeil/embed/blob/master/LICENSE.txt\n[pepa-shield]: https://static.pepy.tech/badge/embed\n[pepa-url]: https://www.pepy.tech/projects/embed\n[ci-shield]: https://github.com/michaelfeil/infinity/actions/workflows/ci.yaml/badge.svg\n[ci-url]: https://github.com/michaelfeil/infinity/actions\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelfeil%2Fembed","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmichaelfeil%2Fembed","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmichaelfeil%2Fembed/lists"}