{"id":28381110,"url":"https://github.com/humancompatibleai/nn-clustering-pytorch","last_synced_at":"2025-06-24T21:31:16.475Z","repository":{"id":37198964,"uuid":"306158084","full_name":"HumanCompatibleAI/nn-clustering-pytorch","owner":"HumanCompatibleAI","description":"Checking the divisibility of neural networks, and investigating the nature of the pieces networks can be divided into.","archived":false,"fork":false,"pushed_at":"2023-02-11T02:20:04.000Z","size":663,"stargazers_count":6,"open_issues_count":2,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-05-30T03:40:20.546Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HumanCompatibleAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-21T22:02:19.000Z","updated_at":"2023-06-07T00:03:57.000Z","dependencies_parsed_at":"2023-01-25T20:46:41.181Z","dependency_job_id":null,"html_url":"https://github.com/HumanCompatibleAI/nn-clustering-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HumanCompatibleAI/nn-clustering-pytorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanCompatibleAI%2Fnn-clustering-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanCompatibleAI%2Fnn-clustering-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanCompatibleAI%2Fnn-clustering-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanCompatibleAI%2Fnn-clustering-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HumanCompatibleAI","download_url":"https://codeload.github.com/HumanCompatibleAI/nn-clustering-pytorch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HumanCompatibleAI%2Fnn-clustering-pytorch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261759067,"owners_count":23205488,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-30T03:38:07.131Z","updated_at":"2025-06-24T21:31:16.465Z","avatar_url":"https://github.com/HumanCompatibleAI.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# nn-clustering-pytorch\nChecking the divisibility of neural networks, and investigating the nature of the pieces networks can be divided into.\n\nNOTE: this code is still under active development. Use at your own peril.\n\nThings this codebase can do:\n\n- Train a neural network using gradient descent and pruning. The neural network can have convolutions, fully-connected layers, batch norm, dropout, and max pooling, as long as it uses only ReLU activations. This is done in `src/train_model.py`.\n- Turn such a neural network into a graph and apply graph clustering to it. This is done in `src/spectral_cluster_model.py`.\n- Compare the clusterability of a model to that of random shuffles of the model's weights. This is done in `src/shuffle_and_cluster.py`.\n- Regularize graph-clusterability during training, while normalizing weights. This is done in `src/clusterability_gradient.py` and `src/train_model.py`. Note: high values of the regularization weight might cause networks to lose connection between their inputs and outputs. I advise using values between 1e-4 and 1e-1.\n\nAfter cloning, enter the pipenv by running `pipenv shell`. To install dependencies:\n\n- if you don't want to contribute to development, just run `pipenv install`.\n- if you do want to contribute to development, install dependencies by running `pipenv install -d`, then install pre-commit hooks for `yapf`, `isort`, and `flake8` by running `pre-commit install`.\n\nYou should also probably create directories where results will be saved: `datasets`, `models`, `training_runs`, `clustering_runs`, and `shuffle_clust_runs`.\n\nThis codebase uses `sacred` extensively, which you can read about [here](https://sacred.readthedocs.io/en/stable/).\n\nNB: the codebase assumes that networks are defined in a very specific way. The operations must be grouped either into ModuleDicts (see [this pytorch doc page](https://pytorch.org/docs/stable/generated/torch.nn.ModuleDict.html)) or nn.Sequential modules.\n\nFor ModuleDicts:\n- One ModuleDict should be used per layer.\n- Batch norm modules must be in the ModuleDict of the linear transform they come after.\n- Within a ModuleDict, fully connected modules should be have the key `'fc'`, convolutional modules should have the key `'conv'`, and batch norm modules should have the key `'bn'`. Nothing else should have those keys.\n- ModuleDicts can't contain both a convolutional and a fully connected module.\n\nFor Sequential modules:\n- Sequential modules should be formed by taking in an ordered dict, where keys are names of operations and values are modules.\n- The keys of the ordered dict should be formatted `\u003cname of layer\u003e_\u003cname of module\u003e`, e.g. `0_fc`, `5_conv`, `layer6_relu`. Rules for names of modules are as for ModuleDicts.\n- One layer should contain at most one fully-connected or convolutional module.\n\nIn both cases, there can only be one batch norm module between two linear transformations. For examples of networks that meet these specifications, see `src/networks.py`.\n\nTODOs:\n  - Refactor code so that ablation_accuracy and compare_masks_clusters share functions\n  - Docstrings in compare_masks_clusters\n  - Move dataset stuff to its own separate file\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumancompatibleai%2Fnn-clustering-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhumancompatibleai%2Fnn-clustering-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumancompatibleai%2Fnn-clustering-pytorch/lists"}