{"id":19066154,"url":"https://github.com/epfml/cola","last_synced_at":"2025-08-18T07:04:26.119Z","repository":{"id":72156776,"uuid":"144067301","full_name":"epfml/cola","owner":"epfml","description":"CoLa - Decentralized Linear Learning: https://arxiv.org/abs/1808.04883","archived":false,"fork":false,"pushed_at":"2021-11-30T05:17:22.000Z","size":46,"stargazers_count":20,"open_issues_count":0,"forks_count":5,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-28T12:52:44.496Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epfml.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-08-08T21:03:16.000Z","updated_at":"2025-01-18T09:13:38.000Z","dependencies_parsed_at":"2023-09-16T20:37:50.656Z","dependency_job_id":null,"html_url":"https://github.com/epfml/cola","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/epfml/cola","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fcola","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fcola/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fcola/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fcola/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epfml","download_url":"https://codeload.github.com/epfml/cola/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epfml%2Fcola/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270956437,"owners_count":24674934,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-18T02:00:08.743Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T00:54:46.795Z","updated_at":"2025-08-18T07:04:26.053Z","avatar_url":"https://github.com/epfml.png","language":"Python","readme":"# CoLa: Decentralized Linear Learning\n\nDecentralized machine learning is a promising emerging paradigm in view of global challenges of data ownership and privacy. We consider learning of linear classification and regression models, in the setting where the training data is decentralized over many user devices, and the learning algorithm must run on-device, on an arbitrary communication network, without a central coordinator. We propose COLA, a new decentralized training algorithm with strong theoretical guarantees and superior practical performance. Our framework overcomes many limitations of existing methods, and achieves communication efficiency, scalability, elasticity as well as resilience to changes in data and participating devices.\n\n[Paper appearing at NeurIPS 2018](https://arxiv.org/abs/1808.04883)\n\nThe CoLa algorithm scheme is communication efficient, as is its centralized counterpart [CoCoA](https://arxiv.org/abs/1611.02189). Processes communicate with their neighborhood nodes only after working on a data-local subproblem for a flexible amount of time, before exchanging a single parameter vector with neighbor nodes. The local solver can be any existing approximate solver: we e.g. provide coordinate solver in our implementation, with flexible number of local coordinate updates, here calling scikit-learn or Cython. Communication between nodes is done via pytorch / MPI.\n\n## Getting Started\nBuild a docker image or pull an existing one from dockerhub\n```bash\n# Build docker images\ndocker build -t cola .\n```\nRun the docker with\n```bash\n# Create a container with dataset directory mounted.\ndocker run --rm -it --mount type=bind,source=path_to_datasets,target=/datasets \\\n    cola:latest bash\n```\n\n### Dataset\nDownload the dataset. A local directory which will be mounted.\n```bash\nwget https://www.csie.ntu.edu.tw/\\~cjlin/libsvmtools/datasets/binary/webspam_wc_normalized_trigram.svm.bz2\nwget https://www.csie.ntu.edu.tw/\\~cjlin/libsvmtools/datasets/binary/epsilon_normalized.bz2\nwget https://www.csie.ntu.edu.tw/\\~cjlin/libsvmtools/datasets/binary/url_original.tar.bz2\nwget https://www.csie.ntu.edu.tw/\\~cjlin/libsvmtools/datasets/binary/url_combined.bz2\n```\nIt is recommended to split the dataset based on the total number of workers using `split_dataset.py`, e.g.\n```\npython split_dataset.py \\\n    --input_file /datasets/url_combined.bz2 \\\n    --K 10 \\\n    --outdir  /datasets/url_split\n```\n\n### Launch\nExecute MPI jobs in docker.\n```bash\n# Joblib can save the time spend on \nexport JOBLIB_CACHE_DIR='/cache'\nexport OUTPUT_DIR='./'\nURL_DATASET_PATH='/datasets/url_combined.bz2'\nEPSILON_DATASET_PATH='/datasets/epsilon_normalized.bz2'\n\nworld_size=20\nmpirun -n $world_size --oversubscribe python run_cola.py \\\n    --split_by 'features' \\\n    --max_global_steps 10 \\\n    --graph_topology 'complete' \\\n    --exit_time 1000.0 \\\n    --theta 1e-7 \\\n    --l1_ratio 1 \\\n    --lambda_ 1e-4 \\\n    --local_iters 10.0 \\\n    --output_dir ${OUTPUT_DIR} \\\n    --dataset_size 'all' \\\n    --ckpt_freq 2 \\\n    --dataset_path ${EPSILON_DATASET_PATH} \\\n    --dataset epsilon \\\n    --solvername ElasticNet \\\n    --algoritmname cola\n```\nNote that\n- If the dataset is already split by `split_dataset.py`, then you can add flag `--use_split_dataset` and specify the `--dataset_path` to `outdir/samples/K/` or `outdir/features/K/`.\n- You can set `--dataset test` or `--dataset test_sparse` to debug the code.\n- \n\n# Reference\nIf you use this code, please cite the following [paper](https://arxiv.org/abs/1808.04883)\n\n    @inproceedings{cola2018nips,\n       author = {He, L. and Bian, A. and Jaggi, M.},\n        title = \"{COLA: Decentralized Linear Learning}\",\n      booktitle = {NeurIPS 2018 - Advances in Neural Information Processing Systems},\n         year = 2018,\n    }\n\nWe thank [PyTorch](https://pytorch.org/) for the distributed communication module and [scikit-learn](http://scikit-learn.org/stable/) for the coordinate solver modules.\n\n* Buitinck, Lars, et al. \"API design for machine learning software: experiences from the scikit-learn project.\" arXiv preprint arXiv:1309.0238 (2013).\n* Pedregosa, Fabian, et al. \"Scikit-learn: Machine learning in Python.\" Journal of machine learning research 12.Oct (2011): 2825-2830.\n* Paszke, Adam, et al. \"Automatic differentiation in pytorch.\" (2017).\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fcola","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepfml%2Fcola","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepfml%2Fcola/lists"}