{"id":32436910,"url":"https://github.com/opendatalab/mineru-vl-utils","last_synced_at":"2026-04-13T08:02:41.503Z","repository":{"id":315347398,"uuid":"1052568611","full_name":"opendatalab/mineru-vl-utils","owner":"opendatalab","description":"A Python package for interacting with the MinerU Vision-Language Model.","archived":false,"fork":false,"pushed_at":"2026-04-11T05:30:53.000Z","size":339,"stargazers_count":109,"open_issues_count":0,"forks_count":30,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-11T07:20:58.557Z","etag":null,"topics":["mineru","utils","vlm"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/mineru-vl-utils/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opendatalab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-08T08:35:23.000Z","updated_at":"2026-04-11T05:30:56.000Z","dependencies_parsed_at":"2025-09-18T05:34:13.765Z","dependency_job_id":"f948b5f7-cc91-4dfe-aa7d-0984abaa9c70","html_url":"https://github.com/opendatalab/mineru-vl-utils","commit_stats":null,"previous_names":["opendatalab/mineru-vl-utils"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/opendatalab/mineru-vl-utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opendatalab%2Fmineru-vl-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opendatalab%2Fmineru-vl-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opendatalab%2Fmineru-vl-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opendatalab%2Fmineru-vl-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opendatalab","download_url":"https://codeload.github.com/opendatalab/mineru-vl-utils/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opendatalab%2Fmineru-vl-utils/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31744404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T06:26:45.479Z","status":"ssl_error","status_checked_at":"2026-04-13T06:26:44.645Z","response_time":93,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mineru","utils","vlm"],"created_at":"2025-10-25T23:56:51.677Z","updated_at":"2026-04-13T08:02:41.474Z","avatar_url":"https://github.com/opendatalab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mineru-vl-utils\n\nA Python package for interacting with the MinerU Vision-Language Model.\n\nIt's a lightweight wrapper that simplifies the process of sending requests\nand handling responses from the MinerU Vision-Language Model.\n\n## About Backends\n\nWe provides 6 different backends(deployment modes):\n\n1. **http-client**: A HTTP client for interacting with the OpenAI-compatible model server.\n2. **transformers**: A backend for using HuggingFace Transformers models. (slow but simple to install)\n3. **mlx-engine**: A backend for using Apple Silicon devices with macOS.\n4. **lmdeploy-engine**: A backend for using the LmDeploy engine.\n5. **vllm-engine**: A backend for using the VLLM synchronous batching engine.\n6. **vllm-async-engine**: A backend for using the VLLM asynchronous engine. (requires async programming)\n\n## About Output Format\n\nMinerU Vision-Language Model can handle document layout detection and\ntext/table/equation recognition tasks in a same model.\n\nThe output of the model is a list of `ContentBlock` objects, each representing\na detected block in the document with its content recognition results.\n\nEach `ContentBlock` contains the following attributes:\n\n- `type` (str): The type of the block, e.g., 'text', 'image', 'table', 'equation'.\n  - For a complete list of supported block types, please refer to [structs.py](mineru_vl_utils/structs.py).\n- `bbox` (list of floats): The bounding box of the block in the format [xmin, ymin, xmax, ymax],\n  with coordinates normalized to the range [0, 1].\n- `angle` (int or None): The rotation angle of the block, can be one of [0, 90, 180, 270].\n  - `0` means upward.\n  - `90` means rightward.\n  - `180` means upside down.\n  - `270` means leftward.\n  - `None` means the angle is not specified.\n- `content` (str or None): The recognized content of the block, if applicable.\n  - For 'text' blocks, this is the recognized text.\n  - For 'table' blocks, this is the recognized table in HTML format.\n  - For 'equation' blocks, this is the recognized LaTeX code.\n  - For 'image' blocks, this is `None`.\n\n## Installation\n\nFor `http-client` backend, just install the package via pip:\n\n```bash\npip install -U mineru-vl-utils\n```\n\nFor `transformers` backend, install the package with the `transformers` extra:\n\n```bash\npip install -U \"mineru-vl-utils[transformers]\"\n```\n\nFor `vllm-engine` and `vllm-async-engine` backend, install the package with the `vllm` extra:\n\n```bash\npip install -U \"mineru-vl-utils[vllm]\"\n```\n\nFor `mlx-engine` backend, install the package with the `mlx` extra:\n\n```bash\npip install -U \"mineru-vl-utils[mlx]\"\n```\n\nFor `lmdeploy-engine` backend, install the package with the `lmdeploy` extra:\n\n```bash\npip install -U \"mineru-vl-utils[lmdeploy]\"\n```\n\n\u003e [!NOTE]\n\u003e For using the `http-client` backend, you still need to have another \n\u003e `vllm`(or other LLM deployment tool) environment to serve the model as a http server.\n\n## Serving the Model (Optional)\n\n\u003e This is only needed if you want to use the `http-client` backend.\n\nYou can use `vllm` or another LLM deployment tool to serve the model.\nHere we only demonstrate how to use `vllm` to serve the model.\n\nWith vllm\u003e=0.10.1, you can use following command to serve the model.\nThe logits processor is used to support `no_repeat_ngram_size` sampling param,\nwhich can help the model to avoid generating repeated content.\n\n```bash\nvllm serve opendatalab/MinerU2.5-2509-1.2B --host 127.0.0.1 --port 8000 \\\n  --logits-processors mineru_vl_utils:MinerULogitsProcessor\n```\n\nIf you are using vllm\u003c0.10.1, `no_repeat_ngram_size` sampling param is not supported.\nYou still can serve the model without logits processor:\n\n```bash\nvllm serve opendatalab/MinerU2.5-2509-1.2B --host 127.0.0.1 --port 8000\n```\n\n## Using `MinerUClient` by Code\n\nNow you can use the `MinerUClient` class to interact with the model.\nFollowing are examples of using different backends.\n\n### `http-client` Example\n\n```python\nfrom PIL import Image\nfrom mineru_vl_utils import MinerUClient\n\nclient = MinerUClient(\n    backend=\"http-client\",\n    server_url=\"http://127.0.0.1:8000\"\n)\n\nimage = Image.open(\"/path/to/the/test/image.png\")\nextracted_blocks = client.two_step_extract(image)\nprint(extracted_blocks)\n```\n\n### `transformers` Example\n\n```python\nfrom transformers import AutoProcessor, Qwen2VLForConditionalGeneration\nfrom PIL import Image\nfrom mineru_vl_utils import MinerUClient\n\n# for transformers\u003e=4.56.0\nmodel = Qwen2VLForConditionalGeneration.from_pretrained(\n    \"opendatalab/MinerU2.5-2509-1.2B\",\n    dtype=\"auto\",\n    device_map=\"auto\"\n)\n\nprocessor = AutoProcessor.from_pretrained(\n    \"opendatalab/MinerU2.5-2509-1.2B\",\n    use_fast=True\n)\n\nclient = MinerUClient(\n    backend=\"transformers\",\n    model=model,\n    processor=processor\n)\n\nimage = Image.open(\"/path/to/the/test/image.png\")\nextracted_blocks = client.two_step_extract(image)\nprint(extracted_blocks)\n```\n\nIf you used an old version of `transformers`(`transformers\u003c4.56.0`),\nyou need to use `torch_dtype` instead of `dtype`.\n\n```python\nmodel = Qwen2VLForConditionalGeneration.from_pretrained(\n    \"opendatalab/MinerU2.5-2509-1.2B\",\n    torch_dtype=\"auto\",\n    device_map=\"auto\"\n)\n```\n\n### `mlx-engine` Example\n\n```python\nfrom mlx_vlm import load as mlx_load\nfrom PIL import Image\nfrom mineru_vl_utils import MinerUClient\n\nmodel, processor = mlx_load(\"opendatalab/MinerU2.5-2509-1.2B\")\n\nclient = MinerUClient(\n    backend=\"mlx-engine\",\n    model=model,\n    processor=processor\n)\n\nimage = Image.open(\"/path/to/the/test/image.png\")\nextracted_blocks = client.two_step_extract(image)\nprint(extracted_blocks)\n```\n\n### `lmdeploy-engine` Example\n\nFor default inference engine(`turbomind` by now).\n\n```python\nfrom lmdeploy.serve.vl_async_engine import VLAsyncEngine\nfrom mineru_vl_utils import MinerUClient\nfrom PIL import Image\n\nif __name__ == \"__main__\":\n    lmdeploy_engine = VLAsyncEngine(\"opendatalab/MinerU2.5-2509-1.2B\")\n\n    client = MinerUClient(\n        backend=\"lmdeploy-engine\",\n        lmdeploy_engine=lmdeploy_engine,\n    )\n\n    image = Image.open(\"/path/to/the/test/image.png\")\n    extracted_blocks = client.two_step_extract(image)\n    print(extracted_blocks)\n```\n\nFor pytorch inference engine and `ascend` accelerator.\n\n```python\nfrom lmdeploy import PytorchEngineConfig\nfrom lmdeploy.serve.vl_async_engine import VLAsyncEngine\nfrom mineru_vl_utils import MinerUClient\nfrom PIL import Image\n\nif __name__ == \"__main__\":\n    lmdeploy_engine = VLAsyncEngine(\n        \"opendatalab/MinerU2.5-2509-1.2B\",\n        backend=\"pytorch\",\n        backend_config=PytorchEngineConfig(\n            device_type=\"ascend\",\n        ),\n    )\n\n    client = MinerUClient(\n        backend=\"lmdeploy-engine\",\n        lmdeploy_engine=lmdeploy_engine,\n    )\n\n    image = Image.open(\"/path/to/the/test/image.png\")\n    extracted_blocks = client.two_step_extract(image)\n    print(extracted_blocks)\n```\n\n### `vllm-engine` Example\n\n```python\nfrom vllm import LLM\nfrom PIL import Image\nfrom mineru_vl_utils import MinerUClient\nfrom mineru_vl_utils import MinerULogitsProcessor  # if vllm\u003e=0.10.1\n\nllm = LLM(\n    model=\"opendatalab/MinerU2.5-2509-1.2B\",\n    logits_processors=[MinerULogitsProcessor]  # if vllm\u003e=0.10.1\n)\n\nclient = MinerUClient(\n    backend=\"vllm-engine\",\n    vllm_llm=llm\n)\n\nimage = Image.open(\"/path/to/the/test/image.png\")\nextracted_blocks = client.two_step_extract(image)\nprint(extracted_blocks)\n```\n\n### `vllm-async-engine` Example\n\n```python\nimport io\nimport asyncio\nimport aiofiles\n\nfrom vllm.v1.engine.async_llm import AsyncLLM\nfrom vllm.engine.arg_utils import AsyncEngineArgs\nfrom PIL import Image\nfrom mineru_vl_utils import MinerUClient\nfrom mineru_vl_utils import MinerULogitsProcessor  # if vllm\u003e=0.10.1\n\nasync_llm = AsyncLLM.from_engine_args(\n    AsyncEngineArgs(\n        model=\"opendatalab/MinerU2.5-2509-1.2B\",\n        logits_processors=[MinerULogitsProcessor]  # if vllm\u003e=0.10.1\n    )\n)\n\nclient = MinerUClient(\n  backend=\"vllm-async-engine\",\n  vllm_async_llm=async_llm,\n)\n\nasync def main():\n    image_path = \"/path/to/the/test/image.png\"\n    async with aiofiles.open(image_path, \"rb\") as f:\n        image_data = await f.read()\n    image = Image.open(io.BytesIO(image_data))\n    extracted_blocks = await client.aio_two_step_extract(image)\n    print(extracted_blocks)\n\nasyncio.run(main())\n\nasync_llm.shutdown()\n```\n\n## Other APIs\n\nBesides the `two_step_extract` method, `MinerUClient` also provides other APIs\nfor interacting with the model. Following are the main APIs:\n\n```python\nclass MinerUClient:\n\n    def layout_detect(self, image: Image.Image) -\u003e list[ContentBlock]:\n        ...\n\n    def batch_layout_detect(self, images: list[Image.Image]) -\u003e list[list[ContentBlock]]:\n        ...\n\n    async def aio_layout_detect(self, image: Image.Image) -\u003e list[ContentBlock]:\n        ...\n\n    async def aio_batch_layout_detect(self, images: list[Image.Image]) -\u003e list[list[ContentBlock]]:\n        ...\n\n    def two_step_extract(self, image: Image.Image) -\u003e list[ContentBlock]:\n        ...\n\n    def batch_two_step_extract(self, images: list[Image.Image]) -\u003e list[list[ContentBlock]]:\n        ...\n\n    async def aio_two_step_extract(self, image: Image.Image) -\u003e list[ContentBlock]:\n        ...\n\n    async def aio_batch_two_step_extract(self, images: list[Image.Image]) -\u003e list[list[ContentBlock]]:\n        ...\n```\n\n## Limitations\n\nThe `transformers` backend is slow and not suitable for production use.\n\nThe `MinerUClient` only supports standalone image(s) as input.\nPDF and DOCX files are not planned to be supported.\nCross-page and cross-document operations are not planned to be supported, too.\n\nFor production use cases, please use [MinerU](https://github.com/opendatalab/mineru),\nwhich is a more complete toolkit for document analyzing and data extraction.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopendatalab%2Fmineru-vl-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopendatalab%2Fmineru-vl-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopendatalab%2Fmineru-vl-utils/lists"}