{"id":26695685,"url":"https://github.com/sdpython/onnx-diagnostic","last_synced_at":"2026-01-23T18:03:39.780Z","repository":{"id":283653045,"uuid":"952423841","full_name":"sdpython/onnx-diagnostic","owner":"sdpython","description":"Investigate onnx models","archived":false,"fork":false,"pushed_at":"2026-01-18T12:40:03.000Z","size":2487,"stargazers_count":4,"open_issues_count":6,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-18T14:13:54.752Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sdpython.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOGS.rst","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-21T08:59:40.000Z","updated_at":"2026-01-18T12:40:07.000Z","dependencies_parsed_at":"2026-01-18T04:07:41.314Z","dependency_job_id":null,"html_url":"https://github.com/sdpython/onnx-diagnostic","commit_stats":null,"previous_names":["sdpython/onnx-diagnostic"],"tags_count":38,"template":false,"template_full_name":null,"purl":"pkg:github/sdpython/onnx-diagnostic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdpython%2Fonnx-diagnostic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdpython%2Fonnx-diagnostic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdpython%2Fonnx-diagnostic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdpython%2Fonnx-diagnostic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sdpython","download_url":"https://codeload.github.com/sdpython/onnx-diagnostic/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sdpython%2Fonnx-diagnostic/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28697425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-23T17:25:48.045Z","status":"ssl_error","status_checked_at":"2026-01-23T17:25:47.153Z","response_time":59,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-26T19:28:01.150Z","updated_at":"2026-01-23T18:03:39.744Z","avatar_url":"https://github.com/sdpython.png","language":"Python","readme":"\n.. image:: https://github.com/sdpython/onnx-diagnostic/raw/main/_doc/_static/logo.png\n    :width: 120\n\nonnx-diagnostic: investigate onnx models\n========================================\n\n.. image:: https://github.com/sdpython/onnx-diagnostic/actions/workflows/documentation.yml/badge.svg\n    :target: https://github.com/sdpython/onnx-diagnostic/actions/workflows/documentation.yml\n\n.. image:: https://badge.fury.io/py/onnx-diagnostic.svg\n    :target: http://badge.fury.io/py/onnx-diagnostic\n\n.. image:: https://img.shields.io/badge/license-MIT-blue.svg\n    :alt: MIT License\n    :target: https://opensource.org/license/MIT/\n\n.. image:: https://img.shields.io/github/repo-size/sdpython/onnx-diagnostic\n    :target: https://github.com/sdpython/onnx-diagnostic/\n    :alt: size\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n    :target: https://github.com/psf/black\n\n.. image:: https://codecov.io/gh/sdpython/onnx-diagnostic/graph/badge.svg?token=91T5ZVIP96 \n    :target: https://codecov.io/gh/sdpython/onnx-diagnostic\n\nThe main feature is about `patches \u003chttps://github.com/sdpython/onnx-diagnostic/tree/main/onnx_diagnostic/torch_export_patches\u003e`_:\nit helps exporting **pytorch models into ONNX**, mostly designed for LLMs using dynamic caches.\nPatches can be enabled as follows:\n\n.. code-block:: python\n\n  from onnx_diagnostic.torch_export_patches import torch_export_patches\n\n  with torch_export_patches(patch_transformers=True) as f:\n      ep = torch.export.export(model, args, kwargs=kwargs, dynamic_shapes=dynamic_shapes)\n      # ...\n\nDynamic shapes are difficult to guess for caches, one function\nreturns a structure defining all dimensions as dynamic.\nYou need then to remove those which are not dynamic in your model.\n\n.. code-block:: python\n\n  from onnx_diagnostic.export.shape_helper import all_dynamic_shapes_from_inputs\n\n  dynamic_shapes = all_dynamic_shapes_from_inputs(cache)\n\nIt also implements tools to investigate, validate exported models (ExportedProgramm, ONNXProgram, ...).\nSee `documentation of onnx-diagnostic \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/\u003e`_ and\n`torch_export_patches \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/api/torch_export_patches/index.html#onnx_diagnostic.torch_export_patches.torch_export_patches\u003e`_.\n\nGetting started\n+++++++++++++++\n\n::\n\n    git clone https://github.com/sdpython/onnx-diagnostic.git\n    cd onnx-diagnostic\n    pip install -e . -v\n\nor\n\n::\n\n    pip install onnx-diagnostic\n\nEnlightening Examples\n+++++++++++++++++++++\n\n**Where to start to export a model**\n\n* `Export microsoft/phi-2\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_tiny_phi2.html\u003e`_\n* `Export a LLM through method generate (with Tiny-LLM)\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_tiny_llm_method_generate.html\u003e`_\n\n**Torch Export**\n\n* `Use DYNAMIC or AUTO when exporting if dynamic shapes has constraints\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_with_dynamic_shapes_auto.html\u003e`_\n* `Find and fix an export issue due to dynamic shapes\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_locate_issue.html\u003e`_\n* `Export with DynamicCache and guessed dynamic shapes\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_with_dynamic_cache.html\u003e`_\n* `Steel method forward to guess the dynamic shapes (with Tiny-LLM)\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_tiny_llm.html\u003e`_\n* `Export Tiny-LLM with patches\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_export_tiny_llm_patched.html\u003e`_\n\n**Investigate ONNX models**\n\n* `Find where a model is failing by running submodels\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_failing_model_extract.html\u003e`_\n* `Intermediate results with (ONNX) ReferenceEvaluator\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_failing_reference_evaluator.html\u003e`_\n* `Intermediate results with onnxruntime\n  \u003chttps://sdpython.github.io/doc/onnx-diagnostic/dev/auto_examples/plot_failing_onnxruntime_evaluator.html\u003e`_\n\nSnapshot of usefuls tools\n+++++++++++++++++++++++++\n\n**torch_export_patches**\n\n.. code-block:: python\n\n  from onnx_diagnostic.torch_export_patches import torch_export_patches\n\n  with torch_export_patches(patch_transformers=True) as f:\n      ep = torch.export.export(model, args, kwargs=kwargs, dynamic_shapes=dynamic_shapes)\n      # ...\n\n**all_dynamic_shapes_from_inputs**\n\n.. code-block:: python\n\n  from onnx_diagnostic.export.shape_helper import all_dynamic_shapes_from_inputs\n\n  dynamic_shapes = all_dynamic_shapes_from_inputs(cache)\n\n**torch_export_rewrite**\n\n.. code-block:: python\n\n  from onnx_diagnostic.torch_export_patches import torch_export_rewrite\n\n  with torch_export_rewrite(rewrite=[Model.forward]) as f:\n      ep = torch.export.export(model, args, kwargs=kwargs, dynamic_shapes=dynamic_shapes)\n      # ...\n\n**string_type**\n\n.. code-block:: python\n\n    import torch\n    from onnx_diagnostic.helpers import string_type\n\n    inputs = (\n        torch.rand((3, 4), dtype=torch.float16),\n        [torch.rand((5, 6), dtype=torch.float16), torch.rand((5, 6, 7), dtype=torch.float16)],\n    )\n\n    # with shapes\n    print(string_type(inputs, with_shape=True))\n\n::\n\n    \u003e\u003e\u003e (T10s3x4,#2[T10s5x6,T10s5x6x7])\n\n**onnx_dtype_name**\n\n.. code-block:: python\n\n        import onnx\n        from onnx_diagnostic.helpers.onnx_helper import onnx_dtype_name\n\n        itype = onnx.TensorProto.BFLOAT16\n        print(onnx_dtype_name(itype))\n        print(onnx_dtype_name(7))\n\n::\n\n    \u003e\u003e\u003e BFLOAT16\n    \u003e\u003e\u003e INT64\n\n**max_diff**\n\n.. code-block:: python\n\n    import torch\n    from onnx_diagnostic.helpers import max_diff\n\n    print(\n        max_diff(\n            (torch.Tensor([1, 2]), (torch.Tensor([1, 2]),)),\n            (torch.Tensor([1, 2]), (torch.Tensor([1, 2]),)),\n        )\n    )\n\n::\n\n    \u003e\u003e\u003e {\"abs\": 0.0, \"rel\": 0.0, \"sum\": 0.0, \"n\": 4.0, \"dnan\": 0.0}s\n\n**guess_dynamic_shapes**\n\n.. code-block:: python\n\n    inputs = [\n        (torch.randn((5, 6)), torch.randn((1, 6))),\n        (torch.randn((7, 8)), torch.randn((1, 8))),\n    ]\n    ds = ModelInputs(model, inputs).guess_dynamic_shapes(auto=\"dim\")\n    print(ds)\n\n::\n\n    \u003e\u003e\u003e (({0: 'dim_0I0', 1: 'dim_0I1'}, {1: 'dim_1I1'}), {})\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsdpython%2Fonnx-diagnostic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsdpython%2Fonnx-diagnostic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsdpython%2Fonnx-diagnostic/lists"}