{"id":29165018,"url":"https://github.com/yuniko-software/bge-m3-onnx","last_synced_at":"2025-08-26T08:10:45.902Z","repository":{"id":300834308,"uuid":"1004500740","full_name":"yuniko-software/bge-m3-onnx","owner":"yuniko-software","description":"ONNX implementation of the BGE-M3 multilingual embedding model and tokenizer with native C#, Java, and Python implementations. Generates all three embedding types: dense, sparse, and ColBERT vectors.","archived":false,"fork":false,"pushed_at":"2025-08-25T10:31:15.000Z","size":487,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-25T12:27:39.521Z","etag":null,"topics":["bge-m3","csharp","dotnet","embedding-models","hugginface","inference","java","machine-learning","onnx","python","pytorch","tokenizer","vector-database"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yuniko-software.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-18T18:10:35.000Z","updated_at":"2025-08-25T10:31:17.000Z","dependencies_parsed_at":"2025-07-08T16:26:21.472Z","dependency_job_id":"7b50d7e7-8b07-4f86-82fa-1a8770bc8251","html_url":"https://github.com/yuniko-software/bge-m3-onnx","commit_stats":null,"previous_names":["yuniko-software/bge-m3-onnx"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/yuniko-software/bge-m3-onnx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuniko-software%2Fbge-m3-onnx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuniko-software%2Fbge-m3-onnx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuniko-software%2Fbge-m3-onnx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuniko-software%2Fbge-m3-onnx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yuniko-software","download_url":"https://codeload.github.com/yuniko-software/bge-m3-onnx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yuniko-software%2Fbge-m3-onnx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272192669,"owners_count":24889452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-26T02:00:07.904Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bge-m3","csharp","dotnet","embedding-models","hugginface","inference","java","machine-learning","onnx","python","pytorch","tokenizer","vector-database"],"created_at":"2025-07-01T07:09:10.848Z","updated_at":"2025-08-26T08:10:45.894Z","avatar_url":"https://github.com/yuniko-software.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BGE-M3 ONNX\n\n![Build](https://github.com/yuniko-software/bge-m3-onnx/actions/workflows/ci-build.yml/badge.svg)\n\nThis repository demonstrates how to convert the complete [BGE-M3](https://github.com/FlagOpen/FlagEmbedding) model to [ONNX](https://github.com/microsoft/onnxruntime) format and use it in multiple programming languages with **full multi-vector functionality**.\n\n\u003cimg width=\"1589\" height=\"1180\" alt=\"image\" src=\"https://github.com/user-attachments/assets/c30cf557-4b54-42be-adc6-1c84bb704337\" /\u003e\n\n## Key Features\n\n- Generate all three BGE-M3 embedding types: dense, sparse, and ColBERT vectors\n- Reduced latency with local embedding generation  \n- Full control over the embedding pipeline with no external dependencies\n- Works offline without internet connectivity requirements\n- Cross-platform compatibility (C#, Java, Python)\n- CUDA GPU acceleration support\n\n## Repository Structure\n\n- `bge-m3-to-onnx.ipynb` - Jupyter notebook demonstrating the BGE-M3 conversion process\n- `/samples/dotnet` - C# implementation\n- `/samples/java` - Java implementation\n- `/samples/python` - Python implementation\n- `generate_reference_embeddings.py` - Script to generate reference embeddings for cross-language testing\n- `run_tests.sh` and `run_tests.ps1` - Test scripts for Linux/macOS and Windows\n\n## Getting Started\n\n1. Clone this repository:\n   ```bash\n   git clone https://github.com/yuniko-software/bge-m3-onnx.git\n   cd bge-m3-onnx\n   ```\n\n2. Get the BGE-M3 ONNX models:\n   - Option 1: Download from releases (recommended)\n     - Check the repository releases and download `onnx.zip`\n     - It already contains the bge-m3 embedding model and its tokenizer\n   \n   - Option 2: Generate yourself using the notebook\n     - Open and run `bge-m3-to-onnx.ipynb` - this is the most important file in the repository\n     - The notebook demonstrates how to convert BGE-M3 from FlagEmbedding to ONNX format\n     - This will create `bge_m3_tokenizer.onnx`, `bge_m3_model.onnx`, and `bge_m3_model.onnx_data` in the `/onnx` folder\n   \n   \u003e Note: This repository uses [`BAAI/bge-m3`](https://github.com/FlagOpen/FlagEmbedding) as the embedding model with its XLM-RoBERTa tokenizer.\n\n3. Generate reference embeddings (optional):\n   - Run `python generate_reference_embeddings.py` to create reference embeddings for testing\n\n4. Run the samples:\n   - Once you have the ONNX models in the `/onnx` folder, you can run any sample\n   - Try the .NET sample in `/samples/dotnet` or the Java sample in `/samples/java`\n\n5. Verify cross-language embeddings (optional):\n   - To ensure that .NET and Java embeddings match the Python-generated embeddings, you can run:\n   \n   - On Linux/macOS:\n     ```bash\n     chmod +x run_tests.sh\n     ./run_tests.sh\n     ```\n   \n   - On Windows:\n     ```powershell\n     ./run_tests.ps1\n     ```\n   \n   \u003e Note: These scripts require Python, .NET, Java, and Maven to be installed.\n\n## CUDA Support\n\nThis BGE-M3 ONNX model supports CUDA GPU acceleration for improved performance. To enable CUDA support:\n\n### Python\nInstall the ONNX Runtime with CUDA support:\n\n**Resource**: [ONNX Runtime CUDA Execution Provider Requirements](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#requirements)\n\nThis model is compatible with:\n- `pip install onnxruntime-gpu[cuda,cudnn]` - packages that include CUDA and cuDNN DLLs\n- [PyTorch packages that include CUDA and cuDNN DLLs](https://pytorch.org/get-started/locally/)\n\n### C# and Java\nFor C# and Java implementations, you need to install CUDA and cuDNN separately:\n\n**CUDA Installation:**\n- Linux: [CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux)\n- Windows: [CUDA Installation Guide for Windows](https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows)\n\n**cuDNN Installation:**\n- [cuDNN Backend Installation Guide](https://docs.nvidia.com/deeplearning/cudnn/installation/latest/backend.html)\n\n## Python Example\n\n```python\nimport onnxruntime as ort\nimport numpy as np\nfrom onnxruntime_extensions import get_library_path\n\n# Initialize BGE-M3 ONNX embedder\nembedder = OnnxBGEM3Embedder(\"onnx/bge_m3_tokenizer.onnx\", \"onnx/bge_m3_model.onnx\")\n\n# Generate all three embedding types\nresult = embedder.encode(\"Hello world!\")\n\nprint(f\"Dense: {len(result['dense_vecs'])} dimensions\")\nprint(f\"Sparse: {len(result['lexical_weights'])} tokens\")  \nprint(f\"ColBERT: {len(result['colbert_vecs'])} vectors\")\n\n# See full implementation in generate_reference_embeddings.py\n```\n\n## C# Example\n\n```csharp\nusing BgeM3.Onnx;\n\n// Initialize embedder\nusing var embedder = new M3Embedder(tokenizerPath, modelPath);\n\n// Generate all embedding types\nvar result = embedder.GenerateEmbeddings(\"Hello world!\");\n\nConsole.WriteLine($\"Dense: {result.DenseEmbedding.Length} dimensions\");\nConsole.WriteLine($\"Sparse: {result.SparseWeights.Count} tokens\");\nConsole.WriteLine($\"ColBERT: {result.ColBertVectors.Length} vectors\");\n\n// See full implementation in samples/dotnet\n```\n\n## Java Example\n\n```java\nimport com.yunikosoftware.bgem3onnx.M3Embedder;\n\n// Initialize embedder\ntry (M3Embedder embedder = new M3Embedder(tokenizerPath, modelPath)) {\n    // Generate all embedding types\n    M3EmbeddingOutput result = embedder.generateEmbeddings(\"Hello world!\");\n    \n    System.out.println(\"Dense: \" + result.getDenseEmbedding().length + \" dimensions\");\n    System.out.println(\"Sparse: \" + result.getSparseWeights().size() + \" tokens\");\n    System.out.println(\"ColBERT: \" + result.getColBertVectors().length + \" vectors\");\n}\n\n// See full implementation in samples/java\n```\n\n---\n\n⭐ **If you find this project useful, please consider giving it a star on GitHub!** ⭐ \n\nYour support helps make this project more visible to other developers who might benefit from BGE-M3's complete multi-vector functionality.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuniko-software%2Fbge-m3-onnx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyuniko-software%2Fbge-m3-onnx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyuniko-software%2Fbge-m3-onnx/lists"}