{"id":13737479,"url":"https://github.com/ryantd/veloce","last_synced_at":"2025-04-10T18:05:45.927Z","repository":{"id":53270015,"uuid":"438111664","full_name":"ryantd/veloce","owner":"ryantd","description":"WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.","archived":false,"fork":false,"pushed_at":"2022-08-04T16:03:45.000Z","size":9575,"stargazers_count":18,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-23T12:47:02.925Z","etag":null,"topics":["data-parallelism","deep-learning","distributed","distributed-computing","heterogeneity","model-parallelism","parameter-server","pytorch","ray","sparsity"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ryantd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-14T04:03:15.000Z","updated_at":"2023-11-18T10:25:05.000Z","dependencies_parsed_at":"2022-08-28T10:40:47.450Z","dependency_job_id":null,"html_url":"https://github.com/ryantd/veloce","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryantd%2Fveloce","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryantd%2Fveloce/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryantd%2Fveloce/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ryantd%2Fveloce/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ryantd","download_url":"https://codeload.github.com/ryantd/veloce/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248266166,"owners_count":21075110,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-parallelism","deep-learning","distributed","distributed-computing","heterogeneity","model-parallelism","parameter-server","pytorch","ray","sparsity"],"created_at":"2024-08-03T03:01:49.468Z","updated_at":"2025-04-10T18:05:45.912Z","avatar_url":"https://github.com/ryantd.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003e ‼️ Please note that this library may be recently changed its name and repository address ‼️\n\n\u003cbr /\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://github.com/ryantd/veloce\"\u003e\n    \u003cimg src=\"docs/images/logo.png\" alt=\"Veloce Logo\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\u003cbr /\u003e\n\n\u003e The project is currently under heavy development, and focusing on **PyTorch** and the **recommendation** scenario.\n\n## About\n**Veloce** is an `instant distributed computing` library based on the **Ray Train** and **Ray Data**, which is scalable, efficient, and easy-to-use. It accelerates the development of any ML/DL training workload, on any cloud or local, at any parallelism size. Previously named Enscale.\n\n### Goals\n- Launch any interactive ML/DL workloads **instantly** on your laptop or to any cloud\n- **Scale** your own single-machine neural network modules to a native **distributed** manner\n- Apply **heterogeneous** architecture\n- **Data scientist-friendly** API\n- **Sparse** and **dense feature** definitions\n\n### Non-Goals\n- Not a neural network library, there are only some benchmark modules provided.\n\n## Getting Started\n### Prerequisites\n- Python version \u003e= `3.7.1`\n- Packages\n  - requests \u003e= `2.26.0`\n  - ray \u003e= `1.9.2` and \u003c= `1.10`\n  - torch \u003e= `1.9.1`\n  - pandas \u003e= `1.3.5`\n  - pyarrow \u003e= `6.0.1`\n\n### Installation\n\n#### Using Pre-compiled Wheels\n```shell\n# CPU version\npip install veloce\n```\n\n#### From Source\n```shell\ngit clone https://github.com/ryantd/veloce\ncd veloce\npip install -e .\n```\n\n### Runtime environment\n\nThe library can launch **locally** or on any **cloud provider** with Ray set up.\n\n- If you want to launch on the cloud, go through this [doc](https://docs.ray.io/en/latest/cluster/cloud.html#launching-cloud-clusters) to set up your Ray Cluster. And then you can use `environ_validate(n_cpus=N, cluster_endpoint=\"ray://\u003chead_node_host\u003e:\u003cport\u003e\")` to connect your cluster.\n- Or just use `environ_validate(n_cpus=N)` to have a local experience.\n\nYou can add more native `ray.init` arguments, just put them into `environ_validate` call. Like `environ_validate(n_cpus=N, ignore_reinit_error=True)` to make Ray suppresses errors from calling `ray.init()` a second time.\n\n### Lightning example\n\n\u003e See more hands-on and advanced examples [here](examples/readme.md), like heterogeneous support and sparsity definition.\n\nThe following example requires `sklearn` to be installed. And `tqdm` is optional, which enables progress reporting.\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1mtDacq6Yty4k5tz_6iVA-lk83YnD46S3?usp=sharing)\n\n```python\nimport torch\nimport torch.nn as nn\nfrom sklearn.metrics import roc_auc_score\nfrom veloce.util import pprint_results, load_benchmark_dataset\nfrom veloce.model.ctr import DeepFM\nfrom veloce import NeuralNetTrainer, environ_validate\n\nN_WORKERS = 2\nN_DATA_PROCESSOR = 1\n\n# ray environment setup\nenviron_validate(n_cpus=N_DATA_PROCESSOR + N_WORKERS)\n# load dataset and sparsity definition pre-defined\ndatasets, feature_defs, dataset_options = load_benchmark_dataset(\n    # set your own dataset by `data_path=\"criteo_mini.txt\"`\n    separate_valid_dataset=False\n)\n# trainer setup\ntrainer = NeuralNetTrainer(\n    # module and dataset configs\n    module=DeepFM, # your own nn.Module or built in modules\n    module_params={\n        \"dense_feature_defs\": feature_defs[\"dense\"],\n        \"sparse_feature_defs\": feature_defs[\"sparse\"],\n    },\n    dataset=datasets,\n    dataset_options=dataset_options,\n    # trainer configs\n    epochs=5,\n    batch_size=512,\n    loss_fn=nn.BCELoss(),\n    optimizer=torch.optim.Adam,\n    metric_fns=[roc_auc_score],\n    # logger callbacks\n    callbacks=[\"json\"],\n    # computation abstract on distributed\n    num_workers=N_WORKERS,\n)\n# run and print results\nresults = trainer.run()\npprint_results(results)\n```\n\n## Architecture\n\n![arch](docs/images/arch.png)\n\n## Roadmap\n\n- Heterogeneous Strategy on Distributed Training\n  - [x] Sync Parameter Server\n  - [ ] Aync Parameter Server\n  - [ ] Hybird Phase 1: use sync or async for the dense or sparse component as you like, under homogeneous architecture\n  - [ ] Hybird Phase 2: you can choose async PS for the sparse component, and sync Ring Allreduce (like PyTorch's DDP) for the dense component\n- Framework Support\n  - [x] PyTorch: no specific plan to support other frameworks\n- Advanced Parallel Mechanism\n  - [ ] Heavy integrated [torchrec](https://github.com/pytorch/torchrec)\n- Accelerator Support\n  - [ ] GPU: complete inspection required\n\n## Reference\n- [Ray and Ray Train](https://github.com/ray-project/ray): Ray Train is a lightweight library for distributed deep learning, allowing you to scale up and speed up training for your deep learning models. Docs [here](https://docs.ray.io/en/master/train/train.html).\n- [DeepCTR-Torch](https://github.com/shenweichen/DeepCTR-Torch): Easy-to-use, modular and extendible package of deep-learning based CTR models.\n\n## License\nVeloce is MIT licensed, as found in the [LICENSE](LICENSE) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryantd%2Fveloce","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fryantd%2Fveloce","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fryantd%2Fveloce/lists"}