{"id":19023666,"url":"https://github.com/neuralmagic/nm-vllm-certs","last_synced_at":"2026-04-29T12:30:18.030Z","repository":{"id":249184224,"uuid":"830227630","full_name":"neuralmagic/nm-vllm-certs","owner":"neuralmagic","description":"General Information, model certifications, and benchmarks for nm-vllm enterprise distributions","archived":false,"fork":false,"pushed_at":"2025-02-15T06:59:02.000Z","size":898,"stargazers_count":11,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-18T14:51:57.153Z","etag":null,"topics":["vllm"],"latest_commit_sha":null,"homepage":"https://neuralmagic.github.io/nm-vllm-certs/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neuralmagic.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-17T21:19:05.000Z","updated_at":"2025-02-07T05:51:33.000Z","dependencies_parsed_at":"2024-09-18T10:00:06.245Z","dependency_job_id":"dddf559b-7d5a-450f-a5d2-3db59dbb2648","html_url":"https://github.com/neuralmagic/nm-vllm-certs","commit_stats":{"total_commits":15,"total_committers":4,"mean_commits":3.75,"dds":0.4666666666666667,"last_synced_commit":"dcf11e5490c46de252c13cb53979fd195303546b"},"previous_names":["neuralmagic/nm-vllm-certs"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neuralmagic%2Fnm-vllm-certs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neuralmagic%2Fnm-vllm-certs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neuralmagic%2Fnm-vllm-certs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neuralmagic%2Fnm-vllm-certs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neuralmagic","download_url":"https://codeload.github.com/neuralmagic/nm-vllm-certs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240072068,"owners_count":19743527,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["vllm"],"created_at":"2024-11-08T20:31:48.683Z","updated_at":"2026-04-29T12:30:17.971Z","avatar_url":"https://github.com/neuralmagic.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# nm-vllm-certs\n\n\n## Overview\nThe `nm-vllm` packages published in this repository are Neural Magic Enterprise Editions of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and Docker images. These are released as \"production level\" official releases and \"beta level\" nightly releases.\n\nOfficial releases are made at the discretion of Neural Magic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neural Magic's PyPI](https://pypi.neuralmagic.com).\n\nNightly builds are released every night given green runs in automation. The wheels are available at [Neural Magic's PyPI](https://pypi.neuralmagic.com).\n\n\n## Benchmarks\n\nPlease see how we are doing with our benchmark results [here]( https://neuralmagic.github.io/nm-vllm-certs/dev/bench/).\n\n\n## Installation\n\n\n### PyPI\nThe [nm-vllm PyPI package](https://pypi.neuralmagic.com/simple/nm-vllm/index.html) includes pre-compiled binaries for CUDA (version 12.1) kernels. For other PyTorch or CUDA versions, please compile the package from source.\n\nInstall it using pip:\n```bash\npip install nm-vllm --extra-index-url https://pypi.neuralmagic.com/simple\n```\n\nTo utilize the weight sparsity features, include the optional `sparse` dependencies.\n```bash\npip install nm-vllm[sparse] --extra-index-url https://pypi.neuralmagic.com/simple\n```\n\n\n### Docker\n\nThe `nm-vllm-ent` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-ent) includes premade docker images.\n\nLaunch the OpenAI-compatible server with:\n\n```bash\nMODEL_ID=Qwen/Qwen2-0.5B-Instruct\ndocker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-ent:latest --model $MODEL_ID\n```\n\n\n## Models\n\nNeural Magic maintains a variety of optimized models on our Hugging Face organization profiles:\n- [neuralmagic](https://huggingface.co/neuralmagic)\n- [nm-testing](https://huggingface.co/nm-testing)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneuralmagic%2Fnm-vllm-certs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneuralmagic%2Fnm-vllm-certs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneuralmagic%2Fnm-vllm-certs/lists"}