{"id":21105154,"url":"https://github.com/tchaton/sparsemlcallback","last_synced_at":"2025-03-14T08:45:03.168Z","repository":{"id":134496172,"uuid":"426935611","full_name":"tchaton/SparseMLCallback","owner":"tchaton","description":null,"archived":false,"fork":false,"pushed_at":"2021-11-11T09:56:03.000Z","size":1106,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-21T03:24:50.696Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tchaton.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-11T09:06:43.000Z","updated_at":"2021-11-14T20:42:42.000Z","dependencies_parsed_at":"2023-06-16T12:15:46.119Z","dependency_job_id":null,"html_url":"https://github.com/tchaton/SparseMLCallback","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2FSparseMLCallback","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2FSparseMLCallback/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2FSparseMLCallback/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tchaton%2FSparseMLCallback/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tchaton","download_url":"https://codeload.github.com/tchaton/SparseMLCallback/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243551230,"owners_count":20309291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-20T00:03:55.182Z","updated_at":"2025-03-14T08:45:03.133Z","avatar_url":"https://github.com/tchaton.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SparseML Callback\n\nAll credits to Sean Narenthiran. I am merely using his code for demonstrating purposes.\n\n\n[SparseML](https://docs.neuralmagic.com/sparseml/) allows you to leverage sparsity to improve inference times substantially.\n\nSparseML requires you to fine-tune your model with the ``SparseMLCallback`` + a SparseML Recipe. By training with the ``SparseMLCallback``, you can leverage the [DeepSparse](https://github.com/neuralmagic/deepsparse) engine to exploit the introduced sparsity, resulting in large performance improvements.\n\n\nThe SparseML callback requires the model to be ONNX exportable. This can be tricky when the model requires dynamic sequence lengths such as RNNs.\n\nTo use leverage SparseML \u0026 DeepSparse follow the below steps:\n\n## Choose your Sparse Recipe\n\nTo choose a recipe, have a look at [recipes](https://docs.neuralmagic.com/sparseml/source/recipes.html) and [Sparse Zoo](https://docs.neuralmagic.com/sparsezoo/).\n\nIt may be easier to infer a recipe via the UI dashboard using [Sparsify](https://github.com/neuralmagic/sparsify) which allows you to tweak and configure a recipe.\nThis requires to import an ONNX model, which you can get from your ``LightningModule`` by doing ``model.to_onnx(output_path)``.\n\n## Train with SparseMLCallback\n\n```py\n    from pytorch_lightning import LightningModule, Trainer\n    from pl_hub_sparse_ml_callack import SparseMLCallback\n\n    class MyModel(LightningModule):\n        ...\n\n    model = MyModel()\n\n    trainer = Trainer(\n        callbacks=SparseMLCallback(recipe_path='recipe.yaml')\n    )\n```\n\n## Export to ONNX!\n\nUsing the helper function, we handle any quantization/pruning internally and export the model into ONNX format.\nNote this assumes either you have implemented the property ``example_input_array`` in the model or you must provide a sample batch as below.\n\n```py\n    import torch\n\n    model = MyModel()\n    ...\n\n    # export the onnx model, using the `model.example_input_array`\n    SparseMLCallback.export_to_sparse_onnx(model, 'onnx_export/')\n\n    # export the onnx model, providing a sample batch\n    SparseMLCallback.export_to_sparse_onnx(model, 'onnx_export/', sample_batch=torch.randn(1, 128, 128, dtype=torch.float32))\n```\n\nOnce your model has been exported, you can import this into either [Sparsify](https://github.com/neuralmagic/sparsify) or [DeepSparse](https://github.com/neuralmagic/deepsparse).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftchaton%2Fsparsemlcallback","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftchaton%2Fsparsemlcallback","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftchaton%2Fsparsemlcallback/lists"}