{"id":19227494,"url":"https://github.com/spinnaker/orca","last_synced_at":"2025-05-15T07:06:41.779Z","repository":{"id":17069486,"uuid":"19834314","full_name":"spinnaker/orca","owner":"spinnaker","description":"Orchestration engine","archived":false,"fork":false,"pushed_at":"2025-04-09T03:02:38.000Z","size":22108,"stargazers_count":250,"open_issues_count":72,"forks_count":811,"subscribers_count":78,"default_branch":"master","last_synced_at":"2025-04-14T12:58:52.787Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spinnaker.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-05-15T20:33:00.000Z","updated_at":"2025-04-08T21:02:29.000Z","dependencies_parsed_at":"2023-12-12T11:25:56.931Z","dependency_job_id":"d3c6a602-5cb3-458a-a32e-9a2e335a0684","html_url":"https://github.com/spinnaker/orca","commit_stats":{"total_commits":5938,"total_committers":270,"mean_commits":"21.992592592592594","dds":0.8470865611316942,"last_synced_commit":"342dcc6e6ee1870506e5fc5bef6890d79ff741ec"},"previous_names":[],"tags_count":2363,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spinnaker%2Forca","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spinnaker%2Forca/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spinnaker%2Forca/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spinnaker%2Forca/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spinnaker","download_url":"https://codeload.github.com/spinnaker/orca/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254292042,"owners_count":22046426,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2024-11-09T15:23:35.946Z","updated_at":"2025-05-15T07:06:36.771Z","avatar_url":"https://github.com/spinnaker.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"Orca\n====\n[![Build Status](https://api.travis-ci.org/spinnaker/orca.svg?branch=master)](https://travis-ci.org/spinnaker/orca)\n\n![Orca Logo](logo.jpg?raw=true)\n\nOrca is the orchestration engine for Spinnaker.\nIt is responsible for taking an execution definition and managing the stages and tasks, coordinating the other Spinnaker services.\n\nOrca _Executions_ (either Pipelines or Orchestrations) are composed of _Stages_ which in turn are composed of _Tasks_.\nThe tasks of a stage share a common context which can be queried across the entire execution allowing multiple stages to coordinate.\nFor example a _bake_ stage publishes details of the image it creates which is then used by a _deploy_ stage.\n\nOrca is a stateless, horizontally scalable service.\nSpinnaker's execution state is persisted to a backend store and work is distributed evenly through a work queue.\nSo long as your backend persistence layer has capacity, you can freely add or remove Orca servers to match workload demands.\n\n## Internals Overview\n\n### Domain\n\nThe primary domain model is an `Execution`, of which there are two types: `PIPELINE` and `ORCHESTRATION`.\nThe `PIPELINE` type is for pipelines while `ORCHESTRATION`s is unscheduled, ad-hoc API-submitted actions and what you see in the UI under the \"Tasks\" tab in an Application.\n\nAt this point in Spinnaker’s life, these two types are nearly programmatically identical, the key difference being that a Pipeline has a predefined persistent configuration, whereas an Orchestration is arbitarily configured at request-time and serial in execution.\n\nFor one Execution, there are one or many `Stage`s organized as a DAG (Directed Acyclic Graph).\nAn Execution supports concurrent branching logic of Stages.\nThese Stages define higher-order, user-friendly actions and concepts, such as \"Resize Server Group\" or \"Deploy\" and can be chained together to form complex delivery workflows.\nWithin a Stage, there are one or more `Task`s (not to be confused with the Tasks tab in the UI: Internally these are Orchestrations).\nA Task can be considered an atomic unit of work, focused on a single action.\nIf you are familiar with the Spinnaker UI, you'll note that for every Stage, there are multiple line items that correlate to a Stage's runtime: These are often Tasks.\nTasks are always executed sequentially and serially.\n\nIt's important to note that a Stage is recursively composable.\nOne stage can have zero to many **Synthetic Stages**, that is, stages that can occur either `BEFORE` or `AFTER` its parent stage.\nThis is most easily illustrated in a Canary stage, which deploys multiple Server Groups (a baseline and canary).\nWhile to the end-user this appears to be a single stage, it's actually a composition of multiple Deploy stages with some extra logic on top.\nLastly, an Execution's stage graph does not need to be known ahead of time.\nAn Execution can lay its tracks down ahead of itself as it is running, which is how some of the more advanced functionality is implemented, like automatic rollbacks or canary behaviors.\n\n### Runtime\n\nOrca uses a distributed queue library, [Keiko](http://github.com/spinnaker/keiko), to manage its work.\nKeiko uses atomic messages to progress work through its lifecycle and allows Orca to first be resilient to node failure, as well as spread load evenly across the deployment making Orca easier to scale and operate.\n\nKeiko itself is an abstraction around a delay queue, meaning it can deliver **Messages** immediately or at specific times in the future, as well as reschedule messages that have already been added to the queue for a new delivery time.\nThis delay queue functionality is pivotal to Orca's performance and operational characteristics when dealing with long running operations.\n\nA Message is a fine-grained unit of work.\nAs an example, a `StartStage` message will verify that all upstream branches are complete and then perform any logic associated with preparing for the stage's execution.\nIt won't actually perform the stage's core actions.\nA `RunTask` is really the only message that actually performs stage work that a user would be familiar with.\n\nSome message types are re-deliverable and can be duplicated, whereas others are not.\nFor example, in a pipeline that has two concurrent stage branches that do not join at the end, there will be two `CompleteExecution` messages sent.\nThis is fine, because the `CompleteExecutionHandler` has logic to verify that indeed all stages are in a completed state before marking the entire execution as complete, whether it ultimately be a failure or success.\n\nFor every message type, there is a single correlated `Handler` that contains all of the logic for processing this `Message` type.\nA Message's contents are small and the bare minimum of information to process the request: For instance, an Execution is referenced by ID rather than containing its entire state.\nOrca builds atop these foundational Handlers in the form of `StageDefinitionBuilder` and other classes to build the DAG and implement actual Spinnaker orchestration logic.\n\nThe queue can be categorized into two parts: The `QueueProcessor`, and its Handlers.\nOrca uses a single thread to run `QueueProcessor`, which polls the oldest messages \"ready\" off the queue.\nA ready message is one whose delivery time is now or in the past: It is normal to have many more messages in the queue than those that are actually ready.\n\nA separate worker thread pool is used for Handlers.\nOrca depends on threads in that pool in order to scale sufficiently.\nYou can tune threads either by adjusting the pool size or by increasing the number of Orca instances.\nBy ignoring downstream bottlenecks Orca can scale horizontally to meet Spinnaker work demand.\n\nIf the queue starts to back up, Executions and their Tasks will begin to take longer to start or complete.\nSometimes the queue will back up because of downstream service pressure (for example, Clouddriver), but may also be due to Orca not having enough threads or persistence capacity.\n\nOne further note about the `QueueProcessor` and its thread pool.\nThe QueueProcessor polls the queue on a regular interval (default 10ms) and tries to fill its thread pool if there's work ready to process.\nIf the thread pool has a core size of 20, and 5 are already busy, it will attempt to take 15 messages off the queue immediately rather than one.\nThis is because in most cases a message takes less than 1ms to complete and we don't want Orca to idle unnecessarily.\n\n## Storage \u0026 Work Queue\n\nThe following storage backends are supported for storing Execution state:\n\n* Redis (default)\n* MySQL (recommended)\n\nThe following backends are supported for the work queue:\n\n* Redis\n\n## Running Orca\n\nOrca requires Redis (and optionally MySQL) to be up and running.\n\nStart Orca via `./gradlew bootRun`, or by following the instructions using the [Spinnaker installation scripts](https://www.github.com/spinnaker/spinnaker).\n\n### Debugging\n\nTo start the JVM in debug mode, set the Java system property `DEBUG=true`:\n\n```\n./gradlew -DDEBUG=true\n```\n\nThe JVM will then listen for a debugger to be attached on port `8183`.\nThe JVM will _not_ wait for the debugger to be attached before starting Orca; the relevant JVM arguments can be seen and modified as needed in `build.gradle`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspinnaker%2Forca","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspinnaker%2Forca","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspinnaker%2Forca/lists"}