{"id":25725034,"url":"https://github.com/alimuhammadofficial/yabm25","last_synced_at":"2026-02-20T01:32:23.558Z","repository":{"id":278497802,"uuid":"933948167","full_name":"AliMuhammadOfficial/yabm25","owner":"AliMuhammadOfficial","description":null,"archived":false,"fork":false,"pushed_at":"2025-02-20T05:59:55.000Z","size":98,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-19T12:22:22.034Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AliMuhammadOfficial.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-02-17T02:41:36.000Z","updated_at":"2025-02-20T05:59:58.000Z","dependencies_parsed_at":"2025-05-07T06:31:09.977Z","dependency_job_id":null,"html_url":"https://github.com/AliMuhammadOfficial/yabm25","commit_stats":null,"previous_names":["alimuhammadofficial/yabm25"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/AliMuhammadOfficial/yabm25","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliMuhammadOfficial%2Fyabm25","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliMuhammadOfficial%2Fyabm25/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliMuhammadOfficial%2Fyabm25/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliMuhammadOfficial%2Fyabm25/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AliMuhammadOfficial","download_url":"https://codeload.github.com/AliMuhammadOfficial/yabm25/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AliMuhammadOfficial%2Fyabm25/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29638632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T22:32:43.237Z","status":"ssl_error","status_checked_at":"2026-02-19T22:32:38.330Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-25T22:17:47.883Z","updated_at":"2026-02-20T01:32:23.543Z","avatar_url":"https://github.com/AliMuhammadOfficial.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# YaBM25 - Python BM25 Search Engine\n\n[![PyPI version](https://badge.fury.io/py/yabm25.svg)](https://badge.fury.io/py/yabm25)\n[![Python Versions](https://img.shields.io/pypi/pyversions/yabm25.svg)](https://pypi.org/project/yabm25/)\n[![Downloads](https://static.pepy.tech/personalized-badge/yabm25?period=total\u0026units=international_system\u0026left_color=grey\u0026right_color=blue\u0026left_text=Downloads)](https://pepy.tech/project/yabm25)\n[![License](https://img.shields.io/github/license/alimuhammadofficial/yabm25.svg)](https://github.com/alimuhammadofficial/yabm25/blob/main/LICENSE)\n\nFast, scalable BM25 search engine implementation in Python with both in-memory and disk-based indexing. Perfect for RAG (Retrieval Augmented Generation), information retrieval, and search applications.\n\n## Key Features\n- 🚀 **High Performance**: Optimized implementation with vectorized operations\n- 💾 **Memory Efficient**: Optional disk-based indexing for large datasets\n- 🔄 **rank_bm25 Compatible**: Drop-in replacement for rank_bm25 with extended features\n- 📊 **Multiple Variants**: Supports BM25, BM25L, BM25Adpt\n- 🛠 **Production Ready**: Thread-safe with proper resource management\n- 📦 **Easy Integration**: Works with LangChain, LlamaIndex, and other RAG frameworks\n\n## Benchmarks\n| Dataset Size | Memory Usage | Index Time | Query Time |\n|-------------|--------------|------------|------------|\n|  x  |  y  | z  | qt  |\n\n## Installation\n```bash\npip install yabm25\n```\n\n## Quick Start\n\n### Simple In-Memory Usage\n```python\nfrom yabm25 import BM25Indexer\n\n# Initialize with corpus\ncorpus = [\n    \"Hello there good man!\",\n    \"It is quite windy in London\",\n    \"How is the weather today?\"\n]\ntokenized_corpus = [doc.split(\" \") for doc in corpus]\nbm25 = BM25Indexer(tokenized_corpus)\n\n# Search\nquery = \"windy London\"\ndoc_scores = bm25.get_scores(query.split(\" \"))\nprint(doc_scores)  # array([0., 0.93729472, 0.])\n\n# Get top document\ntop_docs = bm25.get_top_n(query.split(\" \"), corpus, n=1)\nprint(top_docs)  # ['It is quite windy in London']\n```\n\n### Large-Scale Usage\n```python\nfrom yabm25 import BM25Indexer, BM25Config\n\n# Configure disk-based index\nconfig = BM25Config(\n    index_dir=\"my_index\",\n    doc_chunk_size=500_000,\n    compression=\"ZSTD\"\n)\n\n# Build index\nindexer = BM25Indexer(config)\nindexer.build_index(large_corpus)\n\n# Search\nresults = indexer.query([\"term1\", \"term2\"])\n```\n\n## Documentation\n- [Examples](examples/README.md)\n- [API Reference](docs/api.md)\n- [Performance Guide](docs/performance.md)\n\n## Use Cases\n- 🤖 **RAG Applications**: Enhance LLM responses with relevant context\n- 🔍 **Search Systems**: Build powerful document search engines\n- 📚 **Information Retrieval**: Academic and research applications\n- 📊 **Text Analysis**: Document similarity and ranking\n\n## Comparison with Alternatives\n| Feature          | YaBM25 | rank_bm25 | Elasticsearch |\n|------------------|--------|-----------|---------------|\n| Memory Efficient | ✅     | ❌        | ✅           |\n| Disk-based      | ✅     | ❌        | ✅           |\n| Easy Setup      | ✅     | ✅        | ❌           |\n| Python Native   | ✅     | ✅        | ❌           |\n| RAG Optimized   | ✅     | ❌        | ❌           |\n\n## Citation\n```bibtex\n@software{yabm25,\n  title = {YaBM25: Yet Another BM25 Implementation},\n  author = {Muhammad, Ali},\n  year = {2025},\n  url = {https://github.com/alimuhammadofficial/yabm25}\n}\n```\n\n## Contributing\nContributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n## License\nMIT License. See [LICENSE](LICENSE) for details.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falimuhammadofficial%2Fyabm25","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falimuhammadofficial%2Fyabm25","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falimuhammadofficial%2Fyabm25/lists"}