{"id":27706208,"url":"https://github.com/snowplow/factotum","last_synced_at":"2025-10-08T14:03:42.721Z","repository":{"id":54613842,"uuid":"52620552","full_name":"snowplow/factotum","owner":"snowplow","description":"A system to programmatically run data pipelines","archived":false,"fork":false,"pushed_at":"2025-10-04T09:11:33.000Z","size":302,"stargazers_count":223,"open_issues_count":50,"forks_count":12,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-10-04T09:22:14.222Z","etag":null,"topics":["cron","dag","job","job-scheduler","rust"],"latest_commit_sha":null,"homepage":"http://snowplowanalytics.com/blog/2016/04/09/introducing-factotum-data-pipeline-runner/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/snowplow.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE-2.0.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2016-02-26T17:20:29.000Z","updated_at":"2025-09-15T15:20:08.000Z","dependencies_parsed_at":"2025-10-04T09:14:01.326Z","dependency_job_id":"4574739e-8a40-4355-bfa7-32d97e2cd05d","html_url":"https://github.com/snowplow/factotum","commit_stats":null,"previous_names":["snowplow-archive/factotum","snowplow/factotum"],"tags_count":32,"template":false,"template_full_name":null,"purl":"pkg:github/snowplow/factotum","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Ffactotum","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Ffactotum/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Ffactotum/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Ffactotum/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/snowplow","download_url":"https://codeload.github.com/snowplow/factotum/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/snowplow%2Ffactotum/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278956330,"owners_count":26075221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cron","dag","job","job-scheduler","rust"],"created_at":"2025-04-26T05:01:32.254Z","updated_at":"2025-10-08T14:03:42.715Z","avatar_url":"https://github.com/snowplow.png","language":"Rust","funding_links":[],"categories":["Applications","Rust","应用 Applications","应用程序 Applications","应用","Applications written in Rust"],"sub_categories":["贡献"],"readme":"# Factotum\n\n[![Release][release-image]][releases] [![Apache License 2.0][license-image]][license]\n\nA dag running tool designed for efficiently running complex jobs with non-trivial dependency trees. \n\n## The zen of Factotum\n\n1. A Turing-complete job is not a job, it's a program\n2. A job must be composable from other jobs\n3. A job exists independently of any job schedule\n\n## User quickstart\n\nAssuming you're running **64 bit Linux**: \n\n```{bash}\nwget https://github.com/snowplow/factotum/releases/download/0.6.0/factotum_0.6.0_linux_x86_64.zip\nunzip factotum_0.6.0_linux_x86_64.zip\n./factotum --version\n```\n\nFactotum requires one argument, which is a **[factotum factfile](/README.md#factfile-format)** that describes the job to run. For example, to run the sample **[sleep.factfile](https://raw.githubusercontent.com/snowplow/factotum/master/samples/sleep.factfile)**:\n\n```{bash}\nwget https://raw.githubusercontent.com/snowplow/factotum/master/samples/sleep.factfile\n./factotum run sleep.factfile\n```\nSpecifying variables in the job file can be done using `--env JSON` (or `-e JSON`). The `JSON` here is free-form and needs to correspond to the placeholders you've set in your job.\n\nFor example, the following will print \"hello world!\":\n\n```{bash}\nwget https://raw.githubusercontent.com/snowplow/factotum/master/samples/variables.factfile\n./factotum run variables.factfile --env '{ \"message\": \"hello world!\" }'\n```\n\nStarting from an arbitrary task can be done using the `--start TASK` or `-s TASK` arguments, where TASK is the name of the task you'd like to start at.\n\nFor example, to start at the \"echo beta\" task in [this job](https://raw.githubusercontent.com/snowplow/factotum/master/samples/echo.factfile), you can run the following:\n\n```{bash}\nwget https://raw.githubusercontent.com/snowplow/factotum/master/samples/echo.factfile\n./factotum run echo.factfile --start \"echo beta\"\n```\n\nTo get a quick overview of the options provided, you can use the `--help` or `-h` argument:\n\n```{bash}\n./factotum --help\n```\n \nFor more information on this file format and how to write your own jobs, see the **[Factfile format](/README.md#factfile-format)** section below.\n\n## Factfile format\n\nFactfiles are self-describing JSON which declare a series of tasks and their dependencies. For example: \n\n```{json}\n{\n    \"schema\": \"iglu:com.snowplowanalytics.factotum/factfile/jsonschema/1-0-0\",\n    \"data\": {\n        \"name\": \"Factotum demo\",\n        \"tasks\": [\n            {\n                \"name\": \"echo alpha\",\n                \"executor\": \"shell\",\n                \"command\": \"echo\",\n                \"arguments\": [ \"alpha\" ],\n                \"dependsOn\": [],\n                \"onResult\": {\n                    \"terminateJobWithSuccess\": [],\n                    \"continueJob\": [ 0 ]\n                }\n            },\n            {\n                \"name\": \"echo beta\",\n                \"executor\": \"shell\",\n                \"command\": \"echo\",\n                \"arguments\": [ \"beta\" ],\n                \"dependsOn\": [ \"echo alpha\" ],\n                \"onResult\": {\n                    \"terminateJobWithSuccess\": [],\n                    \"continueJob\": [ 0 ]\n                }\n            },\n            {\n                \"name\": \"echo omega\",\n                \"executor\": \"shell\",\n                \"command\": \"echo\",\n                \"arguments\": [ \"and omega!\" ],\n                \"dependsOn\": [ \"echo beta\" ],\n                \"onResult\": {\n                    \"terminateJobWithSuccess\": [],\n                    \"continueJob\": [ 0 ]\n                }\n            }\n        ]\n    }\n}\n```\n\nThis example defines three tasks that run shell commands - `echo alpha`, `echo beta` and `echo omega`. `echo alpha` has no dependencies - it will run immediately. `echo beta` depends\non the completion of the `echo alpha` task, and so will wait for `echo alpha` to complete. `echo omega` depends on the `echo beta` task, and so will wait for `echo beta` to be complete before \nexecuting. \n\nGiven the above, the tasks will be executed in the following sequence: `echo alpha`, `echo beta` and finally, `echo omega`. Tasks can have multiple dependencies in factotum, and tasks that are parallelizable will\nbe run concurrently. Check out **[the samples](/samples)** for more sample factfiles or **[the wiki](https://github.com/snowplow/factotum/wiki#creating-a-job)** for a more complete description of the factfile format. \n\n## Developer quickstart\n\nFactotum is written in **[Rust](https://www.rust-lang.org/)**.\n\n### Using Vagrant\n\n* Clone this repository - `git clone git@github.com:snowplow/factotum.git`\n* `cd factotum`\n* Set up a Vagrant box and ssh into it - `vagrant up \u0026\u0026 vagrant ssh`\n   * This will take a few minutes\n* `cd /vagrant`\n* Compile and run a demo - `cargo run -- run samples/echo.factfile` \n\n### Using stable Rust without Vagrant \n\n* **[Install Rust](https://www.rust-lang.org/downloads.html)**\n   * on Linux/Mac - `curl -sSf https://static.rust-lang.org/rustup.sh | sh`\n* Clone this repository - `git clone git@github.com:snowplow/factotum.git`\n* `cd factotum`\n* Compile and run a demo - `cargo run -- run samples/echo.factfile` \n\n## Copyright and license\n\nFactotum is copyright 2016-2021 Snowplow Analytics Ltd.\n\nLicensed under the **[Apache License, Version 2.0][license]** (the \"License\");\nyou may not use this software except in compliance with the License.\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n[license-image]: http://img.shields.io/badge/license-Apache--2-blue.svg?style=flat\n[license]: http://www.apache.org/licenses/LICENSE-2.0\n\n[release-image]: http://img.shields.io/badge/release-0.6.1-blue.svg?style=flat\n[releases]: https://github.com/snowplow/factotum/releases\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowplow%2Ffactotum","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsnowplow%2Ffactotum","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsnowplow%2Ffactotum/lists"}