{"id":15637064,"url":"https://github.com/ducha-aiki/lsuvinit","last_synced_at":"2025-08-21T18:33:19.857Z","repository":{"id":71374142,"uuid":"46518240","full_name":"ducha-aiki/LSUVinit","owner":"ducha-aiki","description":"Reference caffe implementation of LSUV initialization","archived":false,"fork":false,"pushed_at":"2017-10-31T12:34:33.000Z","size":32354,"stargazers_count":112,"open_issues_count":1,"forks_count":33,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-12-10T21:36:46.130Z","etag":null,"topics":["caffe","convolutional-networks","convolutional-neural-networks","deep-learning","initialization","lsuv","lsuv-initialization"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ducha-aiki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-11-19T20:26:18.000Z","updated_at":"2024-07-28T20:03:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"3701a19d-4412-4f63-88aa-cc75ed0efa30","html_url":"https://github.com/ducha-aiki/LSUVinit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducha-aiki%2FLSUVinit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducha-aiki%2FLSUVinit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducha-aiki%2FLSUVinit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducha-aiki%2FLSUVinit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ducha-aiki","download_url":"https://codeload.github.com/ducha-aiki/LSUVinit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230527866,"owners_count":18240051,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["caffe","convolutional-networks","convolutional-neural-networks","deep-learning","initialization","lsuv","lsuv-initialization"],"created_at":"2024-10-03T11:09:47.173Z","updated_at":"2024-12-20T03:09:45.486Z","avatar_url":"https://github.com/ducha-aiki.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Layer-sequential unit-variance (LSUV) initialization \n\nThis is sample code for LSUV and Orthonormal initializations, implemented in python script within Caffe framework. \nThe original Caffe README is reproduced below the line.\nLSUV initialization is described in:\n\nMishkin, D. and Matas, J.,(2015). All you need is a good init. arXiv preprint [arXiv:1511.06422](http://arxiv.org/abs/1511.06422).\n\nOrthonormal initialization is described in:\n\nAndrew M. Saxe, James L. McClelland, Surya Ganguli (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint [arXiv:1312.6120](http://arxiv.org/abs/1312.6120)\n\n**upd.:** Torch re-implementation [https://github.com/yobibyte/torch-lsuv](https://github.com/yobibyte/torch-lsuv)\n\n**upd.:** Keras re-implementation [https://github.com/ducha-aiki/LSUV-keras](https://github.com/ducha-aiki/LSUV-keras)\n\n**New!** PyTorch re-implementation [https://github.com/ducha-aiki/LSUV-pytorch](https://github.com/ducha-aiki/LSUV-pytorch)\n\n**New!** Thinc re-implementation [LSUV-thinc](https://github.com/explosion/thinc/blob/e653dd3dfe91f8572e2001c8943dbd9b9401768b/thinc/neural/_lsuv.py)\n\nupdate: Why it is important to scale your input to var=1 before applying LSUV:\n\n![scale-no-scale](https://github.com/ducha-aiki/caffenet-benchmark/blob/master/logs/contrib/img/0_dataset_init.png)\n\n\n## Examples\n\nSee examples and sample log outputs in examples/lsuv. Run get_datasets.sh and training_*.sh for experiments reproduction. Note, than we haven`t freezed random seed and results may vary a little from run to run.\nInitialization script itself is in tools/extra/lsuv_init.py\n\n## Using with you current Caffe, or different framework installation\n\nIf you have Caffe installed already, you can simply copy script from   [tools/extra/lsuv_init.py](https://github.com/ducha-aiki/LSUVinit/blob/master/tools/extra/lsuv_init.py) and view its help (python lsuv_init.py -help) for further instructions. If you use a different framework, please, adapt that same script to your needs and notify us once you've got the code working, so we could let others know of existance of your solution if you're not against sharing your code. The script is self-explanatory, readable and adaptable.\n\n## Citation\n\nPlease cite us if you use this code:\n\n    @ARTICLE{LSUVInit2015,\n    author = {{Mishkin}, D. and {Matas}, J.},\n    title = \"{All you need is a good init}\",\n    journal = {arXiv preprint arXiv:1511.06422},\n    year = 2015,\n    month = nov\n    }\n\n----\n\n\n# Caffe\n\n[![Build Status](https://travis-ci.org/BVLC/caffe.svg?branch=master)](https://travis-ci.org/BVLC/caffe)\n[![License](https://img.shields.io/badge/license-BSD-blue.svg)](LICENSE)\n\nCaffe is a deep learning framework made with expression, speed, and modularity in mind.\nIt is developed by the Berkeley Vision and Learning Center ([BVLC](http://bvlc.eecs.berkeley.edu)) and community contributors.\n\nCheck out the [project site](http://caffe.berkeleyvision.org) for all the details like\n\n- [DIY Deep Learning for Vision with Caffe](https://docs.google.com/presentation/d/1UeKXVgRvvxg9OUdh_UiC5G71UMscNPlvArsWER41PsU/edit#slide=id.p)\n- [Tutorial Documentation](http://caffe.berkeleyvision.org/tutorial/)\n- [BVLC reference models](http://caffe.berkeleyvision.org/model_zoo.html) and the [community model zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo)\n- [Installation instructions](http://caffe.berkeleyvision.org/installation.html)\n\nand step-by-step examples.\n\n[![Join the chat at https://gitter.im/BVLC/caffe](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/BVLC/caffe?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge\u0026utm_content=badge)\n\nPlease join the [caffe-users group](https://groups.google.com/forum/#!forum/caffe-users) or [gitter chat](https://gitter.im/BVLC/caffe) to ask questions and talk about methods and models.\nFramework development discussions and thorough bug reports are collected on [Issues](https://github.com/BVLC/caffe/issues).\n\nHappy brewing!\n\n## License and Citation\n\nCaffe is released under the [BSD 2-Clause license](https://github.com/BVLC/caffe/blob/master/LICENSE).\nThe BVLC reference models are released for unrestricted use.\n\nPlease cite Caffe in your publications if it helps your research:\n\n    @article{jia2014caffe,\n      Author = {Jia, Yangqing and Shelhamer, Evan and Donahue, Jeff and Karayev, Sergey and Long, Jonathan and Girshick, Ross and Guadarrama, Sergio and Darrell, Trevor},\n      Journal = {arXiv preprint arXiv:1408.5093},\n      Title = {Caffe: Convolutional Architecture for Fast Feature Embedding},\n      Year = {2014}\n    }\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fducha-aiki%2Flsuvinit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fducha-aiki%2Flsuvinit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fducha-aiki%2Flsuvinit/lists"}