{"id":13582771,"url":"https://github.com/pegasus-isi/pegasus","last_synced_at":"2026-02-05T00:03:11.175Z","repository":{"id":6769642,"uuid":"8016431","full_name":"pegasus-isi/pegasus","owner":"pegasus-isi","description":"Pegasus Workflow Management System - Automate, recover, and debug scientific computations.","archived":false,"fork":false,"pushed_at":"2026-02-02T22:29:44.000Z","size":334796,"stargazers_count":210,"open_issues_count":46,"forks_count":88,"subscribers_count":5,"default_branch":"master","last_synced_at":"2026-02-03T09:43:19.438Z","etag":null,"topics":["bioinformatics","distributed-systems","hpc","workflow","workflow-management-system"],"latest_commit_sha":null,"homepage":"https://pegasus.isi.edu","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pegasus-isi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2013-02-04T21:18:31.000Z","updated_at":"2026-02-02T22:29:48.000Z","dependencies_parsed_at":"2025-10-08T06:06:56.277Z","dependency_job_id":"fd060376-24bc-492f-b20f-7a6636e13b17","html_url":"https://github.com/pegasus-isi/pegasus","commit_stats":{"total_commits":13122,"total_committers":50,"mean_commits":262.44,"dds":0.583752476756592,"last_synced_commit":"2ea99e04b2843d76f97fa00f7f0bc57cc1974a3c"},"previous_names":[],"tags_count":73,"template":false,"template_full_name":null,"purl":"pkg:github/pegasus-isi/pegasus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasus-isi%2Fpegasus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasus-isi%2Fpegasus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasus-isi%2Fpegasus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasus-isi%2Fpegasus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pegasus-isi","download_url":"https://codeload.github.com/pegasus-isi/pegasus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pegasus-isi%2Fpegasus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29101798,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-04T22:44:52.815Z","status":"ssl_error","status_checked_at":"2026-02-04T22:44:16.428Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bioinformatics","distributed-systems","hpc","workflow","workflow-management-system"],"created_at":"2024-08-01T15:03:00.821Z","updated_at":"2026-02-05T00:03:11.169Z","avatar_url":"https://github.com/pegasus-isi.png","language":"Java","funding_links":[],"categories":["Java"],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n      \u003cimg src=\"doc/sphinx/images/pegasusfront-black-reduced.png\" width=\"200\" alt=\"Pegasus WMS\" /\u003e\n\u003c/p\u003e\n\nPegasus Workflow Management System\n----------------------------------\n\u003cp align=\"left\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/license/pegasus-isi/pegasus?color=blue\u0026label=Licence\"/\u003e\n    \u003cimg src=\"https://img.shields.io/github/v/tag/pegasus-isi/pegasus?label=Latest\"/\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/dm/pegasus-wms?color=green\u0026label=PyPI%20Downloads\"/\u003e\n    \u003cimg src=\"https://img.shields.io/github/contributors-anon/pegasus-isi/pegasus?color=green\u0026label=Contributors\"/\u003e\n\u003c/p\u003e\n\nPegasus WMS is a configurable system for mapping and executing scientific\nworkflows over a wide range of computational infrastructures including laptops,\ncampus clusters, supercomputers, grids, and commercial and academic clouds.\nPegasus has been used to run workflows with up to 1 million tasks that process\ntens of terabytes of data at a time.\n\nPegasus WMS bridges the scientific domain and the execution environment by\nautomatically mapping high-level workflow descriptions onto distributed\nresources. It automatically locates the necessary input data and computational\nresources required by a workflow, and plans out all of the required data\ntransfer and job submission operations required to execute the workflow.\nPegasus enables scientists to construct workflows in abstract terms without\nworrying about the details of the underlying execution environment or the\nparticulars of the low-level specifications required by the middleware (Condor,\nGlobus, Amazon EC2, etc.). In the process, Pegasus can plan and optimize the\nworkflow to enable efficient, high-performance execution of large\nworkflows on complex, distributed infrastructures.\n\nPegasus has a number of features that contribute to its usability and\neffectiveness:\n\n* Portability / Reuse – User created workflows can easily be run in different\nenvironments without alteration. Pegasus currently runs workflows on top of\nCondor pools, Grid infrastructures such as Open Science Grid and XSEDE,\nAmazon EC2, Google Cloud, and HPC clusters. The same workflow can run on a\nsingle system or across a heterogeneous set of resources.\n* Performance – The Pegasus mapper can reorder, group, and prioritize tasks in\norder to increase overall workflow performance.\n* Scalability – Pegasus can easily scale both the size of the workflow, and\nthe resources that the workflow is distributed over. Pegasus runs workflows\nranging from just a few computational tasks up to 1 million. The number of\nresources involved in executing a workflow can scale as needed without any\nimpediments to performance.\n* Provenance – By default, all jobs in Pegasus are launched using the\nKickstart wrapper that captures runtime provenance of the job and helps in\ndebugging. Provenance data is collected in a database, and the data can be\nqueried with tools such as pegasus-statistics, pegasus-plots, or directly\nusing SQL.\n* Data Management – Pegasus handles replica selection, data transfers and\noutput registration in data catalogs. These tasks are added to a workflow as\nauxilliary jobs by the Pegasus planner.\n* Reliability – Jobs and data transfers are automatically retried in case of\nfailures. Debugging tools such as pegasus-analyzer help the user to debug the\nworkflow in case of non-recoverable failures.\n* Error Recovery – When errors occur, Pegasus tries to recover when possible\nby retrying tasks, by retrying the entire workflow, by providing workflow-level\ncheckpointing, by re-mapping portions of the workflow, by trying alternative\ndata sources for staging data, and, when all else fails, by providing a rescue\nworkflow containing a description of only the work that remains to be done.\nIt cleans up storage as the workflow is executed so that data-intensive\nworkflows have enough space to execute on storage-constrained resources.\nPegasus keeps track of what has been done (provenance) including the locations\nof data used and produced, and which software was used with which parameters.\n\n\nGetting Started\n---------------\n\nYou can find more information about Pegasus on the [Pegasus Website](https://pegasus.isi.edu).\n\nPegasus has an extensive [User Guide](https://pegasus.isi.edu/documentation/)\nthat documents how to create, plan, and monitor workflows.\n\nWe recommend you start by completing the Pegasus Tutorial from [Chapter 3 of the\nPegasus User Guide](https://pegasus.isi.edu/documentation/user-guide/tutorial.html).\n\nThe easiest way to install Pegasus is to use one of the binary packages\navailable on the [Pegasus downloads page](https://pegasus.isi.edu/downloads).\nConsult [Chapter 2 of the Pegasus User Guide](https://pegasus.isi.edu/documentation/user-guide/installation.html)\nfor more information about installing Pegasus from binary packages.\n\nRelease notes are also incorporated in the user guide and can be accessed from\nthe Table of Contents below the Reference Guide. The sources for it can\nbe found in `./doc/sphinx/release-notes` directory.\n\nThere is documentation on the Pegasus website for the Python, Java and R\n[Abstract Workflow Generator APIs](https://pegasus.isi.edu/documentation/reference-guide/api-reference.html).\nWe strongly recommend using the Python API which is feature complete, and also\nallows you to invoke all the pegasus command line tools.\n\nYou can use *pegasus-init* command line tool to run several examples\non your local machine. Consult [Chapter 4 of the Pegasus\nUser Guide](https://pegasus.isi.edu/documentation/user-guide/example-workflows.html)\nfor more information.\n\nThere are also examples of how to [Configure Pegasus for Different Execution\nEnvironments](https://pegasus.isi.edu/documentation/user-guide/execution-environments.html)\nin the Pegasus User Guide.\n\nIf you need help using Pegasus, please contact us. See the [contact page]\n(http://pegasus.isi.edu/contact) on the Pegasus website for more information.\n\n\nBuilding from Source\n--------------------\n\nPegasus can be compiled on any recent Linux or Mac OS X system.\n\n### Source Dependencies\n\nIn order to build Pegasus from source, make sure you have the following installed:\n\n* Git\n* Java 8 or higher\n* Python 3.6 or higher\n* Ant\n* gcc\n* g++\n* make\n* tox 3.14.5 or higher\n* mysql (optional, required to access MySQL databases)\n* postgresql (optional, required to access PostgreSQL databases)\n* Python pyyaml\n* Python GitPython\n\nOther packages may be required to run unit tests, and build MPI tools.\n\n### Compiling\n\nAnt is used to compile Pegasus.\n\nTo get a list of build targets run:\n\n    $ ant -p\n\nThe targets that begin with \"dist\" are what you want to use.\n\nTo build a basic binary tarball (excluding documentation), run:\n\n    $ ant dist\n\nTo build the release tarball (including documentation), run:\n\n    $ ant dist-release\n\nThe resulting packages will be created in the `dist` subdirectory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpegasus-isi%2Fpegasus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpegasus-isi%2Fpegasus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpegasus-isi%2Fpegasus/lists"}