{"id":27918776,"url":"https://github.com/juliaparallel/dagger.jl","last_synced_at":"2025-05-14T15:11:28.088Z","repository":{"id":37502967,"uuid":"47311343","full_name":"JuliaParallel/Dagger.jl","owner":"JuliaParallel","description":"A framework for out-of-core and parallel execution","archived":false,"fork":false,"pushed_at":"2025-05-06T15:22:11.000Z","size":8434,"stargazers_count":665,"open_issues_count":111,"forks_count":73,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-05-06T18:50:18.985Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JuliaParallel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"custom":"https://numfocus.org/donate"}},"created_at":"2015-12-03T05:56:18.000Z","updated_at":"2025-04-30T18:34:00.000Z","dependencies_parsed_at":"2022-07-12T15:50:56.894Z","dependency_job_id":"a3984ad0-6ad5-4bda-a9b0-9552586f7744","html_url":"https://github.com/JuliaParallel/Dagger.jl","commit_stats":{"total_commits":718,"total_committers":47,"mean_commits":15.27659574468085,"dds":0.5947075208913649,"last_synced_commit":"8d6b234c6f30802a38e720dcd00b79ebb8092126"},"previous_names":[],"tags_count":83,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FDagger.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FDagger.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FDagger.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JuliaParallel%2FDagger.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JuliaParallel","download_url":"https://codeload.github.com/JuliaParallel/Dagger.jl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254170059,"owners_count":22026219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-06T18:25:31.535Z","updated_at":"2025-05-14T15:11:28.053Z","avatar_url":"https://github.com/JuliaParallel.png","language":"Julia","funding_links":["https://numfocus.org/donate"],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"docs/src/assets/logo-with-text.jpg\" alt=\"Dagger.jl\"\u003e\n\n-----\n\n\u003ch2 align=\"center\"\u003eA framework for out-of-core and parallel computing\u003c/h2\u003e\n\n| **Documentation**                       | **Build Status**                        |\n|:---------------------------------------:|:---------------------------------------:|\n| [![][docs-master-img]][docs-master-url] | [![Build Status][build-img]][build-url] |\n\n[docs-master-img]: https://img.shields.io/badge/docs-master-blue.svg\n[docs-master-url]: https://juliaparallel.github.io/Dagger.jl/dev\n[build-img]: https://badge.buildkite.com/d8f020afb67a5920709c2b0a29111cf596f3f052099b5b656f.svg?branch=master\n[build-url]: https://buildkite.com/julialang/dagger-dot-jl\n\nAt the core of Dagger.jl is a scheduler heavily inspired by [Dask](https://docs.dask.org/en/latest/). It can run computations represented as [directed-acyclic-graphs](https://en.wikipedia.org/wiki/Directed_acyclic_graph) (DAGs) efficiently on many Julia worker processes and threads, as well as GPUs via [DaggerGPU.jl](https://github.com/JuliaGPU/DaggerGPU.jl).\n\n*The **DTable** has been moved out of this repository. You can now find it [here](https://github.com/JuliaParallel/DTables.jl).*\n\n## Installation\n\nDagger.jl can be installed using the Julia package manager. Enter the Pkg REPL\nmode by typing \"]\" in the Julia REPL and then run:\n\n```julia\npkg\u003e add Dagger\n```\n\nOr, equivalently, install Dagger via the Pkg API:\n\n```julia\njulia\u003e import Pkg; Pkg.add(\"Dagger\")\n```\n\n## Usage\n\nOnce installed, the `Dagger` package can be loaded with `using Dagger`, or if\nyou want to use Dagger for distributed computing, it can be loaded as:\n\n```julia\nusing Distributed; addprocs() # Add one Julia worker per CPU core\nusing Dagger\n```\n\nYou can run the following example to see how Dagger exposes easy parallelism:\n\n```julia\n# This runs first:\na = Dagger.@spawn rand(100, 100)\n\n# These run in parallel:\nb = Dagger.@spawn sum(a)\nc = Dagger.@spawn prod(a)\n\n# Finally, this runs:\nwait(Dagger.@spawn println(\"b: \", b, \", c: \", c))\n```\n\n## Use Cases\n\nDagger can support a variety of use cases that benefit from easy, automatic\nparallelism, such as:\n\n- [Parallelizing Nested Loops](https://juliaparallel.org/Dagger.jl/dev/use-cases/parallel-nested-loops/#Use-Case:-Parallel-Nested-Loops)\n\nThis isn't an exhaustive list of the use cases that Dagger supports. There are\nmore examples in the docs, and more use cases examples are welcome (just file\nan issue or PR).\n\n## Contributing Guide\n\nPlease see the roadmap for missing features or known bugs:\n\n[Dagger Features and Roadmap](FEATURES_ROADMAP.md)\n\nOther resources:\n\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)\n[![GitHub issues](https://img.shields.io/github/issues/JuliaParallel/Dagger.jl)](https://github.com/JuliaParallel/Dagger.jl/issues)\n[![GitHub contributors](https://img.shields.io/github/contributors/JuliaParallel/Dagger.jl)](https://github.com/JuliaParallel/Dagger.jl/graphs/contributors)\n\nContributions are encouraged.\n\nThere are several ways to contribute to our project:\n\n**Reporting Bugs**: If you find a bug, please open an issue and describe the problem. Make sure to include steps to reproduce the issue and any error messages you receive regarding that issue.\n\n**Fixing Bugs**: If you'd like to fix a bug, please create a pull request with your changes. Make sure to include a description of the problem and how your changes will address it.\n\nAdditional examples and documentation improvements are also very welcome.\n\n## Resources\n\nList of recommended Dagger.jl resources:\n- Docs [![][docs-master-img]][docs-master-url]\n- Videos\n  - [Distributed Computing with Dagger.jl](https://youtu.be/capjmjVHfMU)\n  - [Easy, Featureful Parallelism with Dagger.jl](https://youtu.be/t3S8W6A4Ago)\n  - [Easier parallel Julia workflow with Dagger.jl](https://youtu.be/VrqzOsav61w)\n  - [Dagger.jl Development and Roadmap](https://youtu.be/G0Y62ysFbDk)\n\n## Help and Discussion\nFor help and discussion, we suggest asking in the following places:\n\n[Julia Discourse](https://discourse.julialang.org/c/domain/parallel/34) and on the [Julia Slack](https://julialang.org/slack/) in the `#dagger` channel.\n\n## References\n```bibtex\n@inproceedings{alomairy2024dynamic,\n  title={Dynamic Task Scheduling with Data Dependency Awareness Using Julia},\n  author={Alomairy, Rabab and Tome, Felipe and Samaroo, Julian and Edelman, Alan},\n  booktitle={2024 IEEE High Performance Extreme Computing Conference (HPEC)},\n  pages={1--7},\n  year={2024},\n  organization={IEEE}\n}\n```\n```bibtex\n@article{dagger2,\n  title={Efficient Dynamic Task Scheduling in Heterogeneous Environments with Julia},\n  author={Samaroo, Julian and Alomairy, Rabab and  and Giordano, Mose and Edelman, Alan},\n  year={2024},\n  publisher={MIT Open Access Articles}\n}\n```\n\n## Acknowledgements\n\nWe thank DARPA, Intel, and the NIH for supporting this work at MIT.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaparallel%2Fdagger.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuliaparallel%2Fdagger.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuliaparallel%2Fdagger.jl/lists"}