{"id":37064242,"url":"https://github.com/dcato98/fastsparse","last_synced_at":"2026-01-14T07:30:25.017Z","repository":{"id":39870304,"uuid":"314243234","full_name":"dcato98/fastsparse","owner":"dcato98","description":"Fastai+PyTorch implementation of sparse model training methods (SET, SNFS, RigL) + customize-your-own.","archived":false,"fork":false,"pushed_at":"2022-10-20T19:08:27.000Z","size":1196,"stargazers_count":10,"open_issues_count":5,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-01T18:53:22.724Z","etag":null,"topics":["fastai","fastai-implementation","nbdev","pytorch","sparse"],"latest_commit_sha":null,"homepage":"https://dcato98.github.io/fastsparse/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dcato98.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-19T12:36:14.000Z","updated_at":"2023-02-04T06:29:09.000Z","dependencies_parsed_at":"2022-08-26T04:01:28.115Z","dependency_job_id":null,"html_url":"https://github.com/dcato98/fastsparse","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":"fastai/nbdev_template","purl":"pkg:github/dcato98/fastsparse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcato98%2Ffastsparse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcato98%2Ffastsparse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcato98%2Ffastsparse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcato98%2Ffastsparse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dcato98","download_url":"https://codeload.github.com/dcato98/fastsparse/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dcato98%2Ffastsparse/sbom","scorecard":{"id":329487,"data":{"date":"2025-08-11","repo":{"name":"github.com/dcato98/fastsparse","commit":"1a21fb95e1032f6578e196866bf8b14c5f4dd5cf"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.5,"checks":[{"name":"Code-Review","score":0,"reason":"Found 0/21 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/main.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:7: update your workflow using https://app.stepsecurity.io/secureworkflow/dcato98/fastsparse/main.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:8: update your workflow using https://app.stepsecurity.io/secureworkflow/dcato98/fastsparse/main.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:14","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:15","Info:   0 out of   2 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":0,"reason":"41 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-cr5q-6q9f-rq6q","Warn: Project is vulnerable to: GHSA-j6gc-792m-qgm2","Warn: Project is vulnerable to: GHSA-pj73-v5mw-pm9j","Warn: Project is vulnerable to: GHSA-jxhc-q857-3j6g","Warn: Project is vulnerable to: GHSA-48wp-p9qv-4j64","Warn: Project is vulnerable to: GHSA-4qw4-jpp4-8gvp","Warn: Project is vulnerable to: GHSA-636f-xm5j-pj9m","Warn: Project is vulnerable to: GHSA-7vh7-fw88-wj87","Warn: Project is vulnerable to: GHSA-fmx4-26r3-wxpf","Warn: Project is vulnerable to: GHSA-52p9-v744-mwjj","Warn: Project is vulnerable to: GHSA-286v-pcf5-25rc","Warn: Project is vulnerable to: GHSA-2qc6-mcvw-92cw","Warn: Project is vulnerable to: GHSA-2rr5-8q37-2w7h","Warn: Project is vulnerable to: GHSA-353f-x4gh-cqq8","Warn: Project is vulnerable to: GHSA-59gp-qqm7-cw4j","Warn: Project is vulnerable to: GHSA-5w6v-399v-w3cc","Warn: Project is vulnerable to: GHSA-7rrm-v45f-jp64","Warn: Project is vulnerable to: GHSA-cgx6-hpwq-fhv5","Warn: Project is vulnerable to: GHSA-crjr-9rc5-ghw8","Warn: Project is vulnerable to: GHSA-fq42-c5rg-92c2","Warn: Project is vulnerable to: GHSA-gx8x-g87m-h5q6","Warn: Project is vulnerable to: GHSA-jc36-42cf-vqwj","Warn: Project is vulnerable to: GHSA-jw9f-hh49-cvp9","Warn: Project is vulnerable to: GHSA-mrxw-mxhj-p664","Warn: Project is vulnerable to: GHSA-pxvg-2qj5-37jq","Warn: Project is vulnerable to: GHSA-r95h-9x8f-r3f7","Warn: Project is vulnerable to: GHSA-v4f8-2847-rwm7","Warn: Project is vulnerable to: GHSA-v6gp-9mmm-c6p5","Warn: Project is vulnerable to: GHSA-vr8q-g5c7-m54m","Warn: Project is vulnerable to: GHSA-vvfq-8hwr-qm4m","Warn: Project is vulnerable to: GHSA-xc9x-jj77-9p9j","Warn: Project is vulnerable to: GHSA-xh29-r2w5-wx8m","Warn: Project is vulnerable to: GHSA-xxx9-3xcr-gjj3","Warn: Project is vulnerable to: GHSA-2rxp-v6pw-ch6m","Warn: Project is vulnerable to: GHSA-4xqq-m2hx-25v8","Warn: Project is vulnerable to: GHSA-5866-49gr-22v4","Warn: Project is vulnerable to: GHSA-8cr8-4vfw-mr7h","Warn: Project is vulnerable to: GHSA-r55c-59qm-vjw6","Warn: Project is vulnerable to: GHSA-vg3r-rm7w-2xgh","Warn: Project is vulnerable to: GHSA-vmwr-mc7x-5vc3","Warn: Project is vulnerable to: GHSA-5cm2-9h8c-rvfx"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-18T03:14:10.618Z","repository_id":39870304,"created_at":"2025-08-18T03:14:10.618Z","updated_at":"2025-08-18T03:14:10.618Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28413297,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T05:26:33.345Z","status":"ssl_error","status_checked_at":"2026-01-14T05:21:57.251Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fastai","fastai-implementation","nbdev","pytorch","sparse"],"created_at":"2026-01-14T07:30:24.214Z","updated_at":"2026-01-14T07:30:25.006Z","avatar_url":"https://github.com/dcato98.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FastSparse\n\u003e Customizable Fastai+PyTorch implementation of sparse model training methods (SET, SNFS, RigL).\n\n\n## _Warning: this repo is undergoing active development_\n\n## Getting Started\n\n### Install\n\n`pip install fastsparse`\n\n### Sparse Algorithms\n\nThis network implements the following sparse algorithms:\n\n| Abbr. | Sparse Algorithm | in FastSparse | Notes |\n| :- | :- | :- | :- |\n|  | static sparsity baseline | omit `DynamicSparseTrainingCallback` |  |\n| SET | [Sparse Evolutionary Training](https://arxiv.org/abs/1901.09181) (Jan 2019) | `DynamicSparseTrainingCallback(**SET_presets)` |  |\n| SNFS | [Sparse Networks From Scratch](https://arxiv.org/abs/1907.04840) (Jul 2019) | `DynamicSparseTrainingCallback(**SNFS_presets)` | \\*redistribution not implemented |\n| RigL | [Rigged Lottery](https://arxiv.org/abs/1911.11134) (Nov 2019) | `DynamicSparseTrainingCallback(**RigL_presets)` |  |\n\n\\*Authors of the RigL paper demonstrate that using SNFS + Erdos-Renyi-Kernel distribution - redistribution outperforms SNFS + uniform sparsity + redistribution (at least on the measured benchmarks).\n\n### Fastai demo\n\nWith just 4 additional lines of code, you can train your model using the latest dynamic sparse training techniques. This example achieves \u003e99% accuracy on MNIST using a ResNet34 with only 1% of the weights.\n\n```python\n# (0) install the library\n# ! pip install fastsparse \n\nfrom fastai.vision.all import *\n\n# (1) import this package\nimport fastsparse as sparse\n\npath = untar_data(URLs.MNIST)\ndls = ImageDataLoaders.from_folder(path, 'training', 'testing')\nlearn = cnn_learner(dls, resnet34, metrics=error_rate, pretrained=False)\n\n# (2) sparsify initial model + enforce masks\nsparse_hooks = sparse.sparsify_model(learn.model, \n                                     model_sparsity=0.99,\n                                     sparse_f=sparse.erdos_renyi_sparsity)\n\n# (3) schedule dynamic mask updates\ncbs = [sparse.DynamicSparseTrainingCallback(**sparse.SNFS_presets, \n                                            batches_per_update=32)]\n\nlearn.fit_one_cycle(5, cbs=cbs)\n\n# (4) remove hooks that enforce masks\nsparse_hooks.remove()\n```\n\nSimply omit the `DynamicSparseTrainingCallback` to train a fixed-sparsity model as a baseline.\n\n### PyTorch demo (*not implemented yet*)\n\n```python\nimport torch\nfrom torchvision import models\n\ndata = ...\nmodel = ...\nopt = ...\nopt = DynamicSparseTrainingOptimizerWrapper(model, opt, **RigL_kwargs)\n\n### Modified training step\n# sparse_opt.step(...) will determine whether to:\n#  (A) take a regular opt step, or\n#  (B) update network connectivity\ndef sparse_train_step(model, xb, yb, loss_func, sparse_opt, step, pct_train):\n    preds = model(xb)\n    loss = loss_func(preds, yb)\n    loss.backward()\n    sparse_opt.step(step, pct_train)\n    sparse_opt.zero_grad()\n```\n\n### Save/Reload demo\n\nHere is an example of saving a model and reloading it to resume training.\n\n```python\nfrom fastai.vision.all import *\nfrom fastsparse import *\n\npath = untar_data(URLs.MNIST_TINY)\ndls = ImageDataLoaders.from_folder(untar_data(URLs.MNIST_TINY))\nlearn = cnn_learner(dls, resnet18, metrics=accuracy, pretrained=False)\nsparse_hooks = sparsify_model(learn.model, model_sparsity=0.9, sparse_f=erdos_renyi_sparsity)\ndst_kwargs = {**SNFS_presets, **{'batches_per_update': 8}}\ncbs = DynamicSparseTrainingCallback(**dst_kwargs)\n\nlearn.fit_flat_cos(5, cbs=cbs)\n\n# (0) save model as usual (masks are stored automatically)\nsave_model('sparse_tiny_mnist', learn.model, learn.opt)\n```\n\n\n\u003ctable border=\"1\" class=\"dataframe\"\u003e\n  \u003cthead\u003e\n    \u003ctr style=\"text-align: left;\"\u003e\n      \u003cth\u003eepoch\u003c/th\u003e\n      \u003cth\u003etrain_loss\u003c/th\u003e\n      \u003cth\u003evalid_loss\u003c/th\u003e\n      \u003cth\u003eaccuracy\u003c/th\u003e\n      \u003cth\u003etime\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e0\u003c/td\u003e\n      \u003ctd\u003e0.334119\u003c/td\u003e\n      \u003ctd\u003e0.679619\u003c/td\u003e\n      \u003ctd\u003e0.505007\u003c/td\u003e\n      \u003ctd\u003e00:03\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e1\u003c/td\u003e\n      \u003ctd\u003e0.271645\u003c/td\u003e\n      \u003ctd\u003e0.555170\u003c/td\u003e\n      \u003ctd\u003e0.848355\u003c/td\u003e\n      \u003ctd\u003e00:02\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e2\u003c/td\u003e\n      \u003ctd\u003e0.237115\u003c/td\u003e\n      \u003ctd\u003e0.072088\u003c/td\u003e\n      \u003ctd\u003e0.978541\u003c/td\u003e\n      \u003ctd\u003e00:02\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e3\u003c/td\u003e\n      \u003ctd\u003e0.220553\u003c/td\u003e\n      \u003ctd\u003e0.044927\u003c/td\u003e\n      \u003ctd\u003e0.987124\u003c/td\u003e\n      \u003ctd\u003e00:02\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e4\u003c/td\u003e\n      \u003ctd\u003e0.174585\u003c/td\u003e\n      \u003ctd\u003e0.006496\u003c/td\u003e\n      \u003ctd\u003e1.000000\u003c/td\u003e\n      \u003ctd\u003e00:02\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\n```python\n### manually restart notebook ###\n\n# (1) then recreate learner as usual\n\nfrom fastai.vision.all import *\nfrom fastsparse import *\n\npath = untar_data(URLs.MNIST_TINY)\ndls = ImageDataLoaders.from_folder(untar_data(URLs.MNIST_TINY))\nlearn = cnn_learner(dls, resnet18, metrics=accuracy, pretrained=False)\n\n# (2) re-sparsify model (this adds the masks to the parameters)\nsparse_hooks = sparsify_model(learn.model, model_sparsity=0.9, sparse_f=erdos_renyi_sparsity) # \u003c-- initial sparsity + enforce masks\n\n# (3) load model as usual\nload_model('sparse_tiny_mnist', learn.model, learn.opt)\n\n# (5) check validation loss \u0026 accuracy to verify we've loaded it successfully\nval_loss, val_acc = learn.validate()\nprint(f'validation loss: {val_loss}, validation accuracy: {val_acc}')\n\n# (4) optionally, continue training; otherwise remove sparsity-preserving hooks\nsparse_hooks.remove()\n```\n\n    /home/dc/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py:53: UserWarning: Could not load the optimizer state.\n      if with_opt: warn(\"Could not load the optimizer state.\")\n\n\n\n\n\n\n    validation loss: 0.006496043410152197, validation accuracy: 1.0\n\n\n### Training with Large Batch Sizes\n\nAuthors of the Rigged Lottery paper hypothesize that the effectiveness of using the gradient magnitude for determining which connections to grow is partly due to their large batch size (4096 for ImageNet). Those without access to multi-gpu clusters can achieve effective batch sizes of this size by using fastai's `GradientAccumulation` callback, which has been tested to be compatible with this package's `DynamicSparseTrainingCallback`.\n\n### Training with Small # of Epochs\n\nDynamic sparse training algorithms work by modifying the network connectivity during training, dropping some weights and allowing others to regrow. By default, network connectivity is modified at the end of each epoch. When training with few epochs, however, there will be few chances to explore which weights to connect. To update more frequently, in `DynamicSparseTrainingCallback`, set `batches_per_update` to a smaller # of batches than occur in one training epoch. Varying the number of batches per update trades off the frequency of updates with stability in making good updates.\n\n## Customization\n\nThere are many ways to implement and test your own dynamic sparse algorithms using FastSparse.\n\n### Custom Initial Sparsity Distribution:\n\nDefine your own initial sparsity distribution by setting `sparsify_method` in `sparsify_model` to a custom function. For example, this function (included in library) will keep the first layer dense and set the remaining layers to a fixed sparsity.\n\n```python\ndef first_layer_dense_uniform(params:list, model_sparsity:float):\n    sparsities = [1.] + [model_sparsity] * (len(params) - 1)\n    return sparsities\n```\n\n### Custom Drop Criterion\n\nWhile published papers like SNFS and RigL refer to 'drop criterion', this library implements the reverse, a 'keep criterion'. This is a function that returns a score for each weight, where the largest `M` scores will be and `M` is determined by the decay schedule. For example, both Sparse Networks From Scratch and Rigged Lottery both use the magnitude of the weights (in FastSparse: `weight_magnitude`).\n\nThis can easily be customized in FastSparse by defining your own keep score function:\n\n```python\ndef custom_keep_scoring_function(param, opt):\n    score = ...\n    assert param.shape == score.shape\n    return score\n```\n\nThen pass your custom function into the sparse training callback:\n\n```python\nDynamicSparseTrainingCallback(..., keep_score_f=custom_keep_scoring_function)\n```\n\n### Custom Grow Criterion\n\nThe grow criterion is a function that returns a score for each weight, where the largest `N` scores will be and `N` is determined by the decay schedule. For example, Sparse Networks From Scrath grows weights according to the momentum of the gradient, while Rigged Lottery uses the magnitude of the gradient (in FastSparse, `gradient_momentum` and `gradient_magnitude` respectively).\n\n```python\ndef custom_grow_scoring_function(param, opt):\n    score = ...\n    assert param.shape == score.shape\n    return score\n```\n\nThen pass your custom function into the sparse training callback:\n\n```python\nDynamicSparseTrainingCallback(..., grow_score_f=custom_grow_scoring_function)\n```\n\n## Replication Results\n\nIn machine learning, is very easy for seemingly insignificant differences in algorithmic implementation to have a noticeable impact on final results. Therefore, this section compares results from this implementation to results reported in published papers.\n\nTODO...\n\n## Under-The-Hood Details\n\nHere's what's going on. \n\nWhen you run `sparsify_model(learn.model, 0.9)`, this adds sparse masks and add pre_forward hooks to enforce masks on weights during forward pass.\n\n\u003e By default, a uniform sparsity distribution is used. Change the sparsity distribution to Erdos-Renyi with `sparsify_model(learn.model, 0.9, sparse_init_f=erdos_renyi)`, or pass in your custom function (see [Customization](#Customization)\n\n\u003e To avoid adding pre_forward hooks, use `sparsify_model(learn.model, 0.9, enforce_masks=False)`.\n\nWhen you add the `DynamicSparseTrainingCallback` callback, ... TODO complete section\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcato98%2Ffastsparse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdcato98%2Ffastsparse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdcato98%2Ffastsparse/lists"}