{"id":16936804,"url":"https://github.com/zhisbug/cavs","last_synced_at":"2025-06-13T18:07:06.174Z","repository":{"id":72781270,"uuid":"73146008","full_name":"zhisbug/Cavs","owner":"zhisbug","description":"Cavs: An Efficient Runtime System for Dynamic Neural Networks","archived":false,"fork":false,"pushed_at":"2020-09-18T06:31:45.000Z","size":859,"stargazers_count":14,"open_issues_count":1,"forks_count":3,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-25T14:51:20.949Z","etag":null,"topics":["deep-learning","gpu","neural-network"],"latest_commit_sha":null,"homepage":"https://github.com/zhisbug/Cavs","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhisbug.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-08T03:38:22.000Z","updated_at":"2025-02-10T17:13:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"08c99740-2e5f-439c-87de-d9b175c7356a","html_url":"https://github.com/zhisbug/Cavs","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhisbug%2FCavs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhisbug%2FCavs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhisbug%2FCavs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhisbug%2FCavs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhisbug","download_url":"https://codeload.github.com/zhisbug/Cavs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248465316,"owners_count":21108244,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","gpu","neural-network"],"created_at":"2024-10-13T20:57:59.703Z","updated_at":"2025-04-11T19:08:01.345Z","avatar_url":"https://github.com/zhisbug.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"cavs_logo.png\" width=256 /\u003e\u003c/p\u003e\n\n\nThis project is an implementation of the Cavs system presented in the paper: [Cavs: An Efficient Runtime System for Dynamic Neural Networks, ATC'18](https://www.usenix.org/system/files/conference/atc18/atc18-xu-shizhen.pdf), sponsored by [Petuum Inc](https://petuum.com/).\n\n\n## Introduction\nRecent deep learning (DL) models are moving more and more to dynamic neural network (NN) architectures, where the NN structure changes for every data sample. \nHowever, existing DL programming models are inefficient in handling dynamic network architectures because of: \n- substantial overhead caused by repeating dataflow graph construction and processing every example; \n- difficulties in batched execution of multiple samples;\n- inability to incorporate graph optimization techniques such as those used in static graphs. \n\nIn this paper, we present **Cavs**, a runtime system that overcomes these bottlenecks and achieves efficient training and inference of dynamic NNs. \nCavs represents a dynamic NN as a static vertex function \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{F}\"\u003e and a dynamic instance-specific graph \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{G}\"\u003e. \nIt avoids the overhead of repeated graph construction by only declaring and constructing \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{F}\"\u003e once, and allows for the use of static graph optimization techniques\non pre-defined operations in \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{F}\"\u003e. \nCavs performs training and inference by scheduling the execution of \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{F}\"\u003e following the dependencies in \u003cimg src=\"https://render.githubusercontent.com/render/math?math=\\large \\mathcal{G}\"\u003e, hence naturally exposing batched execution opportunities over different samples.\n\nExperiments comparing Cavs to state-of-the-art frameworks for dynamic NNs ([TensorFlow Fold](https://github.com/tensorflow/fold), [PyTorch](https://github.com/pytorch/pytorch) and [DyNet](https://github.com/clab/dynet)) demonstrate the efficacy of our approach: Cavs achieves a near one order of magnitude speedup on training of dynamic NN architectures, and ablations verify the effectiveness of our proposed design and optimizations.\n\n\n\n## How to cite \n```\n@inproceedings{xu2018cavs,\n  title={Cavs: An efficient runtime system for dynamic neural networks},\n  author={Xu, Shizhen and Zhang, Hao and Neubig, Graham and Dai, Wei and Kim, Jin Kyu and Deng, Zhijie and Ho, Qirong and Yang, Guangwen and Xing, Eric P},\n  booktitle={2018 $\\{$USENIX$\\}$ Annual Technical Conference ($\\{$USENIX$\\}$$\\{$ATC$\\}$ 18)},\n  pages={937--950},\n  year={2018}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhisbug%2Fcavs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzhisbug%2Fcavs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzhisbug%2Fcavs/lists"}