{"id":20679806,"url":"https://github.com/toshas/sttp","last_synced_at":"2025-04-19T23:52:49.397Z","repository":{"id":132938513,"uuid":"381995588","full_name":"toshas/sttp","owner":"toshas","description":"Spectral Tensor Train Parameterization of Deep Learning Layers","archived":false,"fork":false,"pushed_at":"2021-07-01T10:30:31.000Z","size":15068,"stargazers_count":15,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-19T23:52:44.112Z","etag":null,"topics":["aistats","compression","householder","orthogonality","pytorch","reparameterization","reproducibility","reproducible-research","stability-training","sttp","svd","svdp","tensor","tensortrain"],"latest_commit_sha":null,"homepage":"https://www.obukhov.ai/sttp","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/toshas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-01T10:28:51.000Z","updated_at":"2025-02-19T02:10:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"b9cdf94c-5e21-427d-874f-40bd4f392696","html_url":"https://github.com/toshas/sttp","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toshas%2Fsttp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toshas%2Fsttp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toshas%2Fsttp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/toshas%2Fsttp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/toshas","download_url":"https://codeload.github.com/toshas/sttp/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249830852,"owners_count":21331357,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aistats","compression","householder","orthogonality","pytorch","reparameterization","reproducibility","reproducible-research","stability-training","sttp","svd","svdp","tensor","tensortrain"],"created_at":"2024-11-16T21:27:43.087Z","updated_at":"2025-04-19T23:52:49.369Z","avatar_url":"https://github.com/toshas.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Spectral Tensor Train Parameterization of Deep Learning Layers\n\nThis repository is the official implementation of our AISTATS 2021 paper titled \"Spectral Tensor Train Parameterization \nof Deep Learning Layers\" by Anton Obukhov, Maxim Rakhuba, Alexander Liniger, Zhiwu Huang, Stamatios Georgoulis, Dengxin \nDai, and Luc Van Gool\n[[arXiv]](https://arxiv.org/abs/2103.04217) \n[[PMLR]](http://proceedings.mlr.press/v130/obukhov21a.html). \n\nIt demonstrates how to perform low-rank neural network reparameterization and its stable training in a compressed form. \nThe code provides all experiments (GAN and Image Classification) from the paper (see `configs/aistats21` directory) \nwith the following types of reparameterizations: SNGAN, SRGAN, SVDP, or STTP. \n\n![STTP teaser](doc/teaser.gif)\n\n## Installation\n\nAll experiments can be reproduced on a single 11Gb GPU.\n\nClone the repository, then create a new virtual environment, and install python dependencies into it:\n```bash\npython3 -m venv venv_sttp\nsource venv_sttp/bin/activate\npip3 install --upgrade pip\npip3 install -r requirements.txt\n```\n\nIn case of problems with generic requirements, fall back to \n[requirements_reproducibility.txt](doc/requirements_reproducibility.txt).\n\n## Logging\n\nThe code performs logging to the console, tensorboard file in the experiment log directory, and also Weights and Biases \n(wandb). Upon the first run, please enter your wandb credentials, which can be obtained by registering a free account \nwith the service.\n\n## Creating Environment Config\n\nThe training script allows specifying multiple `yml` config files, which will be concatenated during execution. \nThis is done to separate experiment configs from environment configs. \nTo start running experiments, create your own config file with a few environment settings, similar to \n[configs/env_lsf.yml](configs/env_lsf.yml). Generally, you only need to update paths; see other fields explained in the\n[config reference](doc/config.md).\n\n## Training\n\nChoose a preconfigured experiment from any of the `configs/aistats21` directories, or compose your own config \nusing the [config reference](doc/config.md), and run the following command:\n\n```shell\nCUDA_VISIBLE_DEVICES=0 python -m src.train --cfg configs/env_yours.yml --cfg configs/experiment.yml\n```\n\n## Poster\n\n![STTP poster](doc/poster.png)\n\n## Citation\n\nPlease cite our work if you found it useful:\n\n```\n@InProceedings{obukhov2021spectral,\n  title={Spectral Tensor Train Parameterization of Deep Learning Layers},\n  author={Obukhov, Anton and Rakhuba, Maxim and Liniger, Alexander and Huang, Zhiwu and Georgoulis, Stamatios and Dai, Dengxin and Van Gool, Luc},\n  booktitle={Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},\n  pages={3547--3555},\n  year={2021},\n  editor={Banerjee, Arindam and Fukumizu, Kenji},\n  volume={130},\n  series={Proceedings of Machine Learning Research},\n  month={13--15 Apr},\n  publisher={PMLR},\n  pdf={http://proceedings.mlr.press/v130/obukhov21a/obukhov21a.pdf},\n  url={http://proceedings.mlr.press/v130/obukhov21a.html}\n}\n```\n\n## License\nThis software is released under a CC-BY-NC 4.0 license, which allows personal and research use only. \nFor a commercial license, please contact the authors. You can view a license summary [here](LICENSE).\n\nPortions of source code taken from external sources are annotated with links to original files and their corresponding \nlicenses.\n\n## Acknowledgements\nThis work was supported by Toyota Motor Europe and was carried out at the TRACE Lab at ETH Zurich (Toyota Research on \nAutomated Cars in Europe - Zurich).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoshas%2Fsttp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftoshas%2Fsttp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftoshas%2Fsttp/lists"}