{"id":19066183,"url":"https://github.com/epfml/sparsifiedsgd","last_synced_at":"2025-04-28T12:28:02.117Z","repository":{"id":55106651,"uuid":"134734392","full_name":"epfml/sparsifiedSGD","owner":"epfml","description":"Sparsified SGD with Memory: https://arxiv.org/abs/1809.07599","archived":false,"fork":false,"pushed_at":"2018-10-25T09:22:17.000Z","size":3441,"stargazers_count":53,"open_issues_count":1,"forks_count":11,"subscribers_count":10,"default_branch":"master","last_synced_at":"2024-03-15T21:28:04.928Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-24T15:25:14.000Z","updated_at":"2024-01-04T16:23:20.000Z","dependencies_parsed_at":"2022-08-14T12:10:33.450Z","dependency_job_id":null,"html_url":"https://github.com/epfml/sparsifiedSGD","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FsparsifiedSGD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FsparsifiedSGD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FsparsifiedSGD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2FsparsifiedSGD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/sparsifiedSGD/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223772218,"owners_count":17199977,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:55:05.255Z","updated_at":"2024-11-09T00:55:05.854Z","avatar_url":"https://github.com/epfml.png","language":"Jupyter Notebook","readme":"# Sparsified SGD with Memory\n\nCode for the experimental part of the paper [Sparsified SGD with Memory](https://arxiv.org/abs/1809.07599). It contains the code for the following experiments:\n\n- Theoretical convergence with different sparsification operator\n- Comparison with QSGD\n- Multi-core experiments\n\nUse `notebooks/plots.ipynb` to visualize the results.\n\nPlease open an issue if you have questions or problems.\n\n### Environment set up\n\nInstall [Anaconda](https://anaconda.org) and create the `sparsifedSGD` environment\n```bash\nconda env create -f environment.yaml\nsource activate sparsifedSGD\n...\nsource deactivate # at the end\n```\n\nFor LaTeX support in plots\n\n```\nsudo apt-get install texlive-full msttcorefonts\n```\n\n\n\n### Reproduce the results\n\nTo reproduce the results, you can download the datasets from [LibSVM](https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html)\n\n```bash\nmkdir data\ncd data/\nwget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/rcv1_test.binary.bz2\nwget https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary/epsilon_normalized.bz2\n```\n\nWe decompress the libsvm file and use pickle format instead. It takes more space but is faster to load. You can create a file as follow\n\n```python\nimport pickle\nimport os\nfrom sklearn.datasets import load_svmlight_file\n\nif not os.path.exists('data'):\n    os.makedirs('data')\n\nX, y = load_svmlight_file('data/rcv1_test.binary.bz2')\nwith open('rcv1.pickle', 'wb') as f:\n    pickle.dump((X, y), f)\n\nX, y = load_svmlight_file('data/epsilon_normalized.bz2')\nwith open('epsilon.pickle', 'wb') as f:\n    pickle.dump((X, y), f)\n```\n\nYou can run the baseline\n\n```bash\npython experiments/baselines.py ./data results/baselines\n```\n\nRun our experiments, for example\n\n```bash\npython experiments/rcv-th.py ./data results/rcv-th\npython experiments/rcv-par.sh ./data results/rcv-par\n```\n\nAnd visualize the results with the notebooks.\n\n# Reference\nIf you use this code, please cite the following [paper](https://arxiv.org/abs/1809.07599)\n\n    @inproceedings{scj2018sparseSGD,\n      author = {Sebastian U. Stich and Jean-Baptiste Cordonnier and Martin Jaggi},\n      title = \"{Sparsified {SGD} with Memory}\",\n      booktitle = {NIPS 2018 - Advances in Neural Information Processing Systems},\n      year = 2018,\n      url = {https://arxiv.org/abs/1809.07599}\n    }\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fsparsifiedsgd","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Fsparsifiedsgd","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fsparsifiedsgd/lists"}