{"id":13513693,"url":"https://github.com/commsor/titanoboa","last_synced_at":"2025-12-12T01:06:31.630Z","repository":{"id":37531441,"uuid":"132755236","full_name":"commsor/titanoboa","owner":"commsor","description":"Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.","archived":false,"fork":false,"pushed_at":"2022-09-16T12:57:43.000Z","size":9346,"stargazers_count":918,"open_issues_count":13,"forks_count":47,"subscribers_count":20,"default_branch":"cluster","last_synced_at":"2025-03-28T07:02:35.611Z","etag":null,"topics":["big-data","distributed","distributed-systems","esb","integrations","ipaas","jvm","low-code","service-bus","titanoboa","workflow","workflow-engine","workflow-platform"],"latest_commit_sha":null,"homepage":"https://titanoboa.io","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/commsor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-05-09T12:42:18.000Z","updated_at":"2025-03-26T08:21:57.000Z","dependencies_parsed_at":"2022-08-25T17:21:29.778Z","dependency_job_id":null,"html_url":"https://github.com/commsor/titanoboa","commit_stats":null,"previous_names":["mikub/titanoboa-lite"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/commsor%2Ftitanoboa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/commsor","download_url":"https://codeload.github.com/commsor/titanoboa/tar.gz/refs/heads/cluster","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247137301,"owners_count":20889845,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","distributed","distributed-systems","esb","integrations","ipaas","jvm","low-code","service-bus","titanoboa","workflow","workflow-engine","workflow-platform"],"created_at":"2024-08-01T05:00:35.726Z","updated_at":"2025-12-12T01:06:26.607Z","avatar_url":"https://github.com/commsor.png","language":"Clojure","funding_links":[],"categories":["Clojure"],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.com/mikub/titanoboa.svg?branch=master)](https://travis-ci.com/mikub/titanoboa)\n[![Join the chat at https://github.com/mikub/titanoboa/discussions](https://img.shields.io/badge/chat-github%20discussions-green.svg?logo=google-chat)](https://github.com/mikub/titanoboa/discussions)\n[![Demo](https://img.shields.io/badge/view-demo-green.svg)](https://www.titanoboa.io/demo.html)\n[![Free Hosted Instances](https://img.shields.io/badge/start-a%20free%20hosted%20instance%20with%20one%20click-orange.svg?logo=serverless)](https://cloud.titanoboa.io)\n[![Download](https://img.shields.io/badge/download-1.0.0--alfa.1-blue.svg)](https://github.com/mikub/titanoboa/releases/download/1.0.0-alfa.1/titanoboa-1.0.0-alfa.1_gui.zip)\n[![Docker Pulls](https://img.shields.io/docker/pulls/titanoboa/titanoboa.svg)](https://hub.docker.com/r/titanoboa/titanoboa/)\n\n## Synopsis\nTitanoboa makes complex workflows easy. \n\nIt is a fully distributed, highly scalable low-code workflow orchestration platform.\u003cbr/\u003e\nIt runs on the JVM and supports Java, Kotlin \u0026 Clojure. (Python is also coming!) \u003cbr/\u003e\nYou can run it anywhere: on-premises or in a cloud; on bare metal, VMs, Docker or K8s. \u003cbr/\u003e\nTitanoboa supports [clustering](https://github.com/mikub/titanoboa/wiki/Clustering) and you can scale to hundreds of nodes easily.\n\n![alt Logo](https://www.titanoboa.io/tb-logo-dark-nosub.svg)\n\nDue to its generic, distributed and easily extensible design, you can use titanoboa for wide variety of purposes. \u003cbr/\u003e\nYou can use it:\n* for **[Big Data](https://www.titanoboa.io/big-data.html)** processing \u0026 data pipelines automation\n* for **orchestrating Core Microservices** - as a replacement of Composite Microservices\n* as a full-featured [**iPaaS** / **Integration Platform**](https://titanoboa.io/using-titanoboa-for-cloud-integration.html)\n* for Data Transformations / **ETL**\n* for **[IT Automation](https://www.titanoboa.io/using-titanoboa-for-it-automation.html)**\n* as a **Service Bus** (ESB)\n* for **Batch Processing**\n\nSee also [titanoboa.io](https://titanoboa.io) and our [wiki](https://github.com/mikub/titanoboa/wiki). Predefined workflow steps are [here](https://github.com/mikub/titanoboa-tasklets).\n\n\u003cimg src=\"http://www.titanoboa.io/sample-graph.gif\" width=\"500\"/\u003e\n\n## Motivation\nTitanoboa has been created with aim to create a workflow platform, that would support all the usual features of complex workflow engines and majority of enterprise integration patterns including:\n* sequential and/or **parallel** step execution\n* configurable step **retry** upon error and advanced customizable error handling\n* advanced branching and conditional flow\n* potentially **cyclic** workflow graphs\n* **splitter** and **aggregator** (aka map/reduce) patterns which allow for processing of larger data sets\n* fully **transactional** nature that assures that all steps are executed even in case of a failover\n* full **extensibility** and ability to rapidly develop and deploy new workflows during runtime\n\nIn addition to this titanoboa also strives to honor **immutability and functional programming principles**. This enables it to be fully **distributed** and to be **highly available** with no central master server or database. This also helps lift performance limitations and thus titanoboa can be suitable for not only batch processing, but also for performance critical workflows.\n\nTitanoboa's in-browser [**GUI**](https://github.com/mikub/titanoboa/wiki/Getting-Started-with-GUI) can be used not only to monitor workflow jobs and server configuration, but provides an in-build IDE with [workflow visualisation](https://www.titanoboa.io/titanoboa-demo_drag_n_drop.mp4), advanced data and properties editor, [autcomplete functionality](https://www.titanoboa.io/titanoboa-demo_java.mp4) and a **[repl](https://www.titanoboa.io/titanoboa-demo_repl.mp4)** (even for [java](https://titanoboa.io/blog-java-support-repl.html)!) so as users can rapidly test-develop new workflows directly in there.\n\n\u003cimg width=\"42\" height=\"42\" src=\"https://github.com/mikub/titanoboa/blob/master/doc/clojure.svg\"\u003e\u003cimg width=\"54\" height=\"54\" src=\"https://github.com/mikub/titanoboa/blob/master/doc/java.svg\"\u003e\u003cbr/\u003e\nTitanoboa is **designed for both java \u0026 clojure developers** and we are striving to make it **[usable even for java developers with no prior clojure knowledge](#-developing-custom-workflow-steps-in-java)**.\n\nTo understand some of the concepts behind Titanoboa and how it can be used, feel free to watch this re:Clojure conference talk:\n\n[![recent talk at re:Clojure conference](https://img.youtube.com/vi/IRrh4VUHhNY/0.jpg)](https://www.youtube.com/watch?v=IRrh4VUHhNY)\n\n\n## Predefined Steps\nThough it is easy to rapidly develop new workflow steps directly from GUI (as shown in our [demo](https://www.titanoboa.io/demo.html)), there is a number of ready-made steps and tasklets in [this repository](https://github.com/mikub/titanoboa-tasklets).\nSome of the steps included are: \n- AWS (EC2, S3, SNS, SQS, SES)\n- JDBC Client\n- Http Client\n- Smtp Client\n- Sftp\n- PDF Generation \n\nand more.\n\nIf you [created your own custom steps](https://github.com/mikub/titanoboa/wiki/Designing-Workflows) (in [java](#-developing-custom-workflow-steps-in-java) or [clojure](https://github.com/mikub/titanoboa/wiki/Designing-Workflows)) and you wish to reuse them in other workflows it is super easy to add your own custom ready-made steps into the GUI - see our [wiki](https://github.com/mikub/titanoboa/wiki/Adding-Custom-Step-Types-and-Icons-to-GUI). You can even add custom icons!\n\n## Prerequisites\nJava 8 or higher.\n\n## Docker\n[![Docker Pulls](https://img.shields.io/docker/pulls/titanoboa/titanoboa.svg)](https://hub.docker.com/r/titanoboa/titanoboa/)\n\nDocker image is available in [titanoboa/titanoboa](https://hub.docker.com/r/titanoboa/titanoboa)\nYou can run it as follows:\n\n    docker run -d -p 3000:3000 --name titanoboa titanoboa/titanoboa:0.9.0\n\n## Installation\n**By the way, you don't need to install anything as you can just start a free hosted instance in your browser with one click! Just head to https://cloud.titanoboa.io**\n\nBut if you want, install away! We suggest giving titanoboa's GUI a try as well since it is the best starting point!\nDownload titanoboa's distribution including its GUI from https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1_gui.zip \n\n    curl --remote-name https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1_gui.zip\n\n**Note**: _The GUI is now also free for commercial use. Yay!_\n\n__Note__: _If you are intending on running titanoboa server on java 8 JRE, download a distribution for JRE instead:_\n\n    curl --remote-name https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1_jre_gui.zip\n\nUnzip the file:\n    \n    unzip titanoboa-0.9.1_gui.zip\n\nthen execute the start script:\n    \n     ./start\n\nIn your console you should see bunch of log messages and ultimately you will see\n     \n     INFO [main] - Started @2338ms\n\nwhich means the server started successfully. By default both the server and the GUI will start on port 3000 and you can open http://localhost:3000 in your browser.\n\nCongratulations! You have just started your titanoboa server!\n\nYou can go ahead and try to create a [sample workflow](https://github.com/mikub/titanoboa/wiki/Getting-Started-with-GUI) or [watch our demo](https://www.titanoboa.io/demo.html).\n\n### Installing server without GUI\nDownload the latest release from https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1.zip .\n\n    curl --remote-name https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1.zip\n\n__Note__: _If you are intending on running titanoboa server on java 8 JRE, download a distribution for JRE instead:_\n\n    curl --remote-name https://github.com/mikub/titanoboa/releases/download/0.9.1/titanoboa-0.9.1_jre.zip\n    \nAnd then follow the instructions above. By default the server will start on port 3000.\n     \n## Building from the repository\nThe steps are described on our [wiki](https://github.com/mikub/titanoboa/wiki/Building-from-the-repository).\n\n## Server Configuration\nServer configuration and external dependencies file can be specified by system properties `boa.server.config.path` and `boa.server.dependencies.path`:\n\n     java -Dboa.server.config.path=boa_server_config_local.clj -Dboa.server.dependencies.path=ext-dependencies.clj -cp \"./build/titanoboa.jar:./lib/*\" titanoboa.server\n     \nSee [Server configuration wiki](https://github.com/mikub/titanoboa/wiki/Server-Configuration) for more details.\n\n\n## Getting Started\nBefore you start, it might be a good idea to get familiar with titanoboa's [concepts](https://github.com/mikub/titanoboa/wiki) \u0026 [workflow design principles](https://github.com/mikub/titanoboa/wiki/Designing-Workflows).\n\nAlso feel free to watch Miro's [talk from re:Clojure 2020 conference](https://www.youtube.com/watch?v=IRrh4VUHhNY) which gives a pretty good overview of what Titanoboa can do.\n\n### Develop \u0026 Test Workflows with titanoboa GUI\nTitanoboa GUI is a good place to start devloping and testing workflows:\n\n[![]( https://github.com/mikub/titanoboa/blob/master/doc/generate-report17-change-details.png )](https://raw.githubusercontent.com/mikub/titanoboa/master/doc/generate-report17-change-details.png )\n\n[![]( https://github.com/mikub/titanoboa/blob/master/doc/generate-report19-jobs.png )](https://raw.githubusercontent.com/mikub/titanoboa/master/doc/generate-report19-jobs.png )\n\nSee an example in our wiki on how to create a [sample workflow](https://github.com/mikub/titanoboa/wiki/Getting-Started-with-GUI).\n\n ### \u003cimg width=\"48\" height=\"48\" src=\"https://github.com/mikub/titanoboa/blob/master/doc/java.svg\"\u003e Developing custom workflow steps in Java \n Titanoboa is also meant to be used by java developers who (apart from few concepts like [EDN](https://github.com/edn-format/edn)) do not need to be familiar with clojure. If you do not want to use clojure [java interop](https://clojure.org/reference/java_interop) to instantiate your objects and/or invoke your methods, you also have other options:\n \n To create a custom workflow step, simply add a (maven) dependency on [![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa/titanoboa-java.svg)](https://clojars.org/io.titanoboa/titanoboa-java) to your project.\n and create a class that will implement [io.titanoboa.java.IWorkloadFn](https://github.com/mikub/titanoboa-java/blob/master/src/main/java/io/titanoboa/java/IWorkloadFn.java) interface:\n ```java\n public interface IWorkloadFn {\n    public Object invoke (Map properties);\n}\n ```\n If you then add your project (or the corresponding maven artifact) to titanoboa's [external dependencies](https://github.com/mikub/titanoboa/wiki/Server-Configuration#external-dependencies), you can use your class name in the workflow-fn field. The class will be automatically instantiated as a singleton bean (so it has to have a constructor with no argumet) and all subsequent references to it from any workflow-fn will invoke its __invoke__ method:\n \n ```clojure\n :workload-fn io.titanoboa.java.SampleWorkloadImpl\n ```\nor\n\n ```clojure\n :workload-fn 'io.titanoboa.java.SampleWorkloadImpl\n ```\n or in GUI:\n[![]( https://github.com/mikub/titanoboa/blob/master/doc/java-workload.png )](https://raw.githubusercontent.com/mikub/titanoboa/master/doc/java-workload.png )\n\n### Java lambda support\nTo rapidly development and test new steps, you can also type a lambda function in the GUI and have titanoboa evaluate it during runtime. Read more about it [here](https://www.titanoboa.io/java-repl.html) or see how it is done in our [demo](https://www.titanoboa.io/demo.html).\n[![]( https://github.com/mikub/titanoboa/blob/master/doc/java-lambda-workload.png )](https://raw.githubusercontent.com/mikub/titanoboa/master/doc/java-lambda-workload.png )\n\n\n\n### \u003cimg width=\"42\" height=\"42\" src=\"https://github.com/mikub/titanoboa/blob/master/doc/clojure.svg\"\u003e Develop \u0026 Test Workflows Locally in Your Clojure REPL\nIf you cannot use GUI and do not want to use REST API, you can as well just start REPL locally and play with titanoboa there.\nEither build titanoboa from repo or get it as _leiningen_ or _maven_ dependency:\n\n[![Clojars Project](https://img.shields.io/clojars/v/io.titanoboa/titanoboa.svg)](https://clojars.org/io.titanoboa/titanoboa)\n\n```clojure\n[io.titanoboa/titanoboa \"0.9.0\"]\n```\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.titanoboa\u003c/groupId\u003e\n  \u003cartifactId\u003etitanoboa\u003c/artifactId\u003e\n  \u003cversion\u003e0.9.0\u003c/version\u003e\n\u003c/dependency\u003e\n```\nthen fire off a repl and start:\n\n#### Define a sample \"Hello World!\" workflow:\n```clojure\n(ns local-system-test\n  (:require [titanoboa.system]\n            [titanoboa.repo]\n            [titanoboa.system.local]\n            [titanoboa.processor]))\n\n(defn hello-fn [p]\n  {:message (str \"Hello \" (or (:name p) \"human\") \"!\")\n   :return-code (nil? (:name p))})\n\n(defn greet-fn [p]\n  {:message (str (:message p) \" Nice to meet you!\")})\n\n(defn fill-in-blanks [p]\n  {:message (str (:message p) \" What is your name?\")})\n  \n(def job-def {:first-step \"step1\"\n              :name       \"test\"\n              :properties {:name nil}\n              :steps      [{:id          \"step1\"\n                            :type :custom\n                            :supertype :tasklet\n                            :next [[false \"step2\"] [true \"step3\"]]\n                            :workload-fn 'local-system-test/hello-fn\n                            :properties  {}}\n                           {:id          \"step2\" :type :custom :supertype :tasklet\n                            :workload-fn 'local-system-test/greet-fn\n                            :next        []\n                            :properties  {}}\n                           {:id          \"step3\" :type :custom :supertype :tasklet\n                            :workload-fn 'local-system-test/fill-in-blanks\n                            :next        []\n                            :properties  {}}]})\n```\n\n#### Start a simple local system with workers that will process the job:\n```clojure\n(def new-jobs-chan (clojure.core.async/chan (clojure.core.async/dropping-buffer 1024)))\n(def jobs-chan (clojure.core.async/chan (clojure.core.async/dropping-buffer 1024)))\n(def finished-jobs-chan (clojure.core.async/chan (clojure.core.async/dropping-buffer 1024)))\n\n(titanoboa.system/start-system! :core-local\n                                  {:core-local {:system-def   #'titanoboa.system.local/local-core-system\n                                          :worker-def   #'titanoboa.system.local/local-worker-system\n                                          :worker-count 2}}\n                                  {:new-jobs-chan      new-jobs-chan\n                                   :jobs-chan          jobs-chan\n                                   :finished-jobs-chan finished-jobs-chan\n                                   :node-id            \"localhost\"\n                                   :eviction-interval  (* 1000 60 5)\n                                   :eviction-age       (* 1000 60 10)\n                                   :jobs-repo-path     \"repo-test/\"\n                                   :job-folder-path    \"job-folders/\"})\n                                   \n INFO [nREPL-worker-0] - Starting system :core-local ...\n INFO [nREPL-worker-0] - Starting CacheEvictionComponent...\n INFO [CacheEvictionComponent thread 0] - Starting CacheEvictionComponent thread [ CacheEvictionComponent thread 0 ].\n INFO [nREPL-worker-0] - Starting action processor pool...\n INFO [nREPL-worker-0] - Starting to watch repo folder for changes:  dev-resources/repo-test/\n INFO [nREPL-worker-0] - System :core-local started\n=\u003e true\n \n(titanoboa.system/start-workers! :core-local\n                                   {:core-local {:system-def   #'titanoboa.system.local/local-core-system\n                                           :worker-def   #'titanoboa.system.local/local-worker-system\n                                           :worker-count 2}})\n                                           \n INFO [nREPL-worker-1] - Starting 2 workers for system :core-local :\n INFO [nREPL-worker-1] - Starting a worker for system :core-local ...\n INFO [nREPL-worker-1] - Starting job worker....\n INFO [nREPL-worker-1] - Starting a worker for system :core-local ...\n INFO [nREPL-worker-1] - Starting job worker....\n=\u003e nil                                           \n```\n#### Start the job:\n```clojure\n(titanoboa.processor/run-job! :core-local\n                              {:jobdef job-def\n                               :properties {:name \"World\"}}\n                              true)\n\n INFO [nREPL-worker-2] - Submitting new job [] into new jobs channel...\n INFO [async-thread-macro-2] - Initializing a new job; First step will be: [ step1 ]\n INFO [async-thread-macro-2] - Retrieved job [ d673c759-4fc6-4af1-bdad-d1dfd0f50f22 ] from jobs channel; Starting step [ step1 ]\n INFO [async-thread-macro-2] - Next step is  step2 ; Submitting into jobs channel for next step's processing...\n INFO [async-thread-macro-1] - Initializing a next step; next step [ step2 ] was found among steps as step [ step2 ]\n INFO [async-thread-macro-2] - Acking main message for step  step1  with thread stack  nil\n INFO [async-thread-macro-1] - Retrieved job [ d673c759-4fc6-4af1-bdad-d1dfd0f50f22 ] from jobs channel; Starting step [ step2 ]\n INFO [async-thread-macro-1] - Looping through finalize-job! fn with thread-stack:  []\n INFO [async-thread-macro-1] - Acking main message for step  step2  with thread stack  nil\n INFO [async-thread-macro-1] - Job  d673c759-4fc6-4af1-bdad-d1dfd0f50f22  has finshed.\n=\u003e\n{:properties {:name \"World\", :message \"Hello World! Nice to meet you!\"},\n :step-start #inst\"2018-11-23T07:35:34.218-00:00\",\n :step-retries {},\n :tracking-id nil,\n  :step-state :completed,\n :start #inst\"2018-11-23T07:35:34.203-00:00\",\n :history [{:step-state :completed,\n            :start #inst\"2018-11-23T07:35:34.210-00:00\",\n            :duration 6,\n            :result false,\n            :node-id \"localhost\",\n            :id \"step1\",\n            :next-step \"step2\",\n            :exception nil,\n            :end #inst\"2018-11-23T07:35:34.216-00:00\",\n            :retry-count nil,\n            :thread-stack nil,\n            :message \"Step [step1] finshed with result [false]\\n\"}\n           {:id \"step2\",\n            :step-state :running,\n            :start #inst\"2018-11-23T07:35:34.218-00:00\",\n            :node-id \"localhost\",\n            :retry-count 0}\n           {:step-state :completed,\n            :start #inst\"2018-11-23T07:35:34.218-00:00\",\n            :duration 1,\n            :result nil,\n            :node-id \"localhost\",\n            :id \"step2\",\n            :next-step nil,\n            :exception nil,\n            :end #inst\"2018-11-23T07:35:34.219-00:00\",\n            :retry-count nil,\n            :thread-stack nil,\n            :message \"Step [step2] finshed with result []\\n\"}],\n :duration 16,\n :state :finished,\n :jobid \"d673c759-4fc6-4af1-bdad-d1dfd0f50f22\",\n :create-folder? true,\n :node-id \"localhost\",\n :next-step nil,\n :end #inst\"2018-11-23T07:35:34.219-00:00\"}\n```\n\nWhen you are done testing you may want to stop the system:\n```clojure\n(titanoboa.system/stop-all-systems!)\n\n INFO [nREPL-worker-3] - Stopping all workers for system :core-local\n INFO [nREPL-worker-3] - Stopping job worker gracefully; sending a stop signal to the worker via service bus....\n INFO [nREPL-worker-3] - Stopping job worker gracefully; sending a stop signal to the worker via service bus....\n INFO [nREPL-worker-3] - Stopping system :core-local ...\n INFO [nREPL-worker-3] - Stopping action processor pool...\n INFO [nREPL-worker-3] - Stopping CacheEvictionComponent thread [ CacheEvictionComponent thread 0 ]...\n ```\n\n## License\nCopyright © Commsor Inc.\n\nTitanoboa is licensed under AGPL license.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommsor%2Ftitanoboa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcommsor%2Ftitanoboa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcommsor%2Ftitanoboa/lists"}