{"id":13738205,"url":"https://github.com/Ma-Lab-Berkeley/ReduNet","last_synced_at":"2025-05-08T16:32:40.301Z","repository":{"id":37472439,"uuid":"366255999","full_name":"Ma-Lab-Berkeley/ReduNet","owner":"Ma-Lab-Berkeley","description":"ReduNet","archived":false,"fork":false,"pushed_at":"2022-02-17T07:57:11.000Z","size":100,"stargazers_count":539,"open_issues_count":6,"forks_count":81,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-04-05T09:09:59.931Z","etag":null,"topics":["white-box-architecture"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ma-Lab-Berkeley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-11T04:35:49.000Z","updated_at":"2025-03-25T11:13:20.000Z","dependencies_parsed_at":"2022-07-22T05:01:50.817Z","dependency_job_id":null,"html_url":"https://github.com/Ma-Lab-Berkeley/ReduNet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ma-Lab-Berkeley%2FReduNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ma-Lab-Berkeley%2FReduNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ma-Lab-Berkeley%2FReduNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ma-Lab-Berkeley%2FReduNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ma-Lab-Berkeley","download_url":"https://codeload.github.com/Ma-Lab-Berkeley/ReduNet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253105359,"owners_count":21855009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["white-box-architecture"],"created_at":"2024-08-03T03:02:14.371Z","updated_at":"2025-05-08T16:32:39.947Z","avatar_url":"https://github.com/Ma-Lab-Berkeley.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Deep Networks from the Principle of Rate Reduction\nThis repository is the official PyTorch implementation of the paper [ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction](https://arxiv.org/abs/2105.10446) (2021) \n\nby [Kwan Ho Ryan Chan](https://ryanchankh.github.io)* (UC Berkeley), [Yaodong Yu](https://yaodongyu.github.io/)* (UC Berkeley), [Chong You](https://sites.google.com/view/cyou)* (UC Berkeley), [Haozhi Qi](https://haozhi.io/) (UC Berkeley), [John Wright](http://www.columbia.edu/~jw2966/) (Columbia University), and [Yi Ma](http://people.eecs.berkeley.edu/~yima/) (UC Berkeley). \n\n\u003c!-- For the NumPy version of ReduNet, please go checkout: [https://github.com/ryanchankh/redunet_paper](https://github.com/ryanchankh/redunet_paper) --\u003e\n\n## What is ReduNet?\nReduNet is a deep neural network construcuted naturally by deriving the gradients of the Maximal Coding Rate Reduction (MCR\u003csup\u003e2\u003c/sup\u003e) [1] objective. Every layer of this network can be interpreted based on its mathematical operations and the network collectively is trained in a feed-forward manner only. In addition, by imposing shift invariant properties to our network, the convolutional operator can be derived using only the data and MCR\u003csup\u003e2\u003c/sup\u003e objective function, hence making our network design principled and interpretable. \n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"images/arch-redunet.jpg\" width=\"350\"\\\u003e\u003cbr\u003e\n\tFigure: Weights and operations for one layer of ReduNet\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\n[1] Yu, Yaodong, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. \"[Learning diverse and discriminative representations via the principle of maximal coding rate reduction](https://proceedings.neurips.cc/paper/2020/file/6ad4174eba19ecb5fed17411a34ff5e6-Paper.pdf)\" Advances in Neural Information Processing Systems 33 (2020). \n\n## Requirements\nThis codebase is written for `python3`. To install necessary python packages, run `conda create --name redunet_official --file requirements.txt`.\n\n## Demo\nFor a quick demonstration of ReduNet on Gaussian 2D or 3D cases, please visit the notebook by running one of the two commands: \n\n```\n$ jupyter notebook ./examples/gaussian2d.ipynb\n$ jupyter notebook ./examples/gaussian3d.ipynb\n```\n\n## Core Usage and Design\nThe design of this repository aims to be easy-to-use and easy-to-intergrate to the current framework of your experiment, as long as it uses PyTorch. The `ReduNet` object inherents from `nn.Sequential`, and layers `ReduLayers`, such as `Vector`, `Fourier1D` and `Fourier2D` inherent from `nn.Module`. Loss functions are implemented in `loss.py`. Architectures and Dataset options are located in `load.py` file. Data objects and pre-set architectures are loaded in folders `dataset` and `architectures`. Feel free to add more based on the experiments you want to run. We have provided basic experiment setups, located in `train_\u003cmode\u003e.py` and `evaluate_\u003cmode\u003e.py`, where `\u003cmode\u003e` is the type of experiment. For utility functions, please check out `functional.py` or `utils.py`. Feel free to email us if there are any issues or suggestions. \n\n\n## Example: Forward Construction\nTo train a ReduNet using forward construction, please checkout `train_forward.py`. For evaluating, please checkout `evaluate_forward.py`. For example, to train on 40-layer ReduNet on MNIST using 1000 samples per class, run:\n\n```\n$ python3 train_forward.py --data mnistvector --arch layers50 --samples 1000\n```\nAfter training, you can evaluate the trained model using `evaluate_forward.py`, by running:\n\n```\n$ python3 evaluate_forward.py --model_dir ./saved_models/forward/mnistvector+layers50/samples1000 \n```\n, which will evaluate using all available training samples and testing samples. For more training and testing options, please checkout the file `train_forward.py` and `evaluate_forward.py`.\n\n### Experiments in Paper\nFor code used to generate experimental empirical results listed in our paper, please visit our other repository: [https://github.com/ryanchankh/redunet_paper](https://github.com/ryanchankh/redunet_paper)\n\n## Reference\nFor technical details and full experimental results, please check the [paper](https://arxiv.org/abs/2105.10446). Please consider citing our work if you find it helpful to yours:\n\n```\n@article{chan2021redunet,\n  title={ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction},\n  author={Chan, Kwan Ho Ryan and Yu, Yaodong and You, Chong and Qi, Haozhi and Wright, John and Ma, Yi},\n  journal={arXiv preprint arXiv:2105.10446},\n  year={2021}\n}\n```\n\n## License and Contributing\n- This README is formatted based on [paperswithcode](https://github.com/paperswithcode/releasing-research-code).\n- Feel free to post issues via Github. \n\n## Contact\nPlease contact [ryanchankh@berkeley.edu](ryanchankh@berkeley.edu) and [yyu@eecs.berkeley.edu](yyu@eecs.berkeley.edu) if you have any question on the codes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMa-Lab-Berkeley%2FReduNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FMa-Lab-Berkeley%2FReduNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FMa-Lab-Berkeley%2FReduNet/lists"}