{"id":28395188,"url":"https://github.com/databiosphere/stairway","last_synced_at":"2025-06-27T01:31:23.147Z","repository":{"id":40244837,"uuid":"238543572","full_name":"DataBiosphere/stairway","owner":"DataBiosphere","description":"Stairway saga transaction processor library","archived":false,"fork":false,"pushed_at":"2025-05-05T13:43:38.000Z","size":811,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":23,"default_branch":"develop","last_synced_at":"2025-05-05T14:49:56.590Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DataBiosphere.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-05T20:37:43.000Z","updated_at":"2025-05-05T13:43:41.000Z","dependencies_parsed_at":"2023-11-16T16:43:06.835Z","dependency_job_id":"90666aeb-60bd-491c-9cef-82c666231304","html_url":"https://github.com/DataBiosphere/stairway","commit_stats":null,"previous_names":[],"tags_count":109,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fstairway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fstairway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fstairway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fstairway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DataBiosphere","download_url":"https://codeload.github.com/DataBiosphere/stairway/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DataBiosphere%2Fstairway/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":257327563,"owners_count":22528582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-05-31T19:39:26.780Z","updated_at":"2025-06-27T01:31:23.134Z","avatar_url":"https://github.com/DataBiosphere.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=DataBiosphere_stairway\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=DataBiosphere_stairway)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=DataBiosphere_stairway\u0026metric=coverage)](https://sonarcloud.io/summary/new_code?id=DataBiosphere_stairway)\n\n# Stairway\nStairway is a library that provides a framework for running _saga transactions_. Saga\ntransactions, introduced by Hector Garcia-Molina in 1987, use _compensating operations_ to\nrollback the transaction rather than having a service that intermediates activity so is\nable to perform the rollback. The goal is to make a sequence of operations, often over disparate resources,\nrun as a transaction; either complete successfully or make no change.\n\nA typical example in our environment is an operation where we maintain metadata about\nan object via a write to a SQL database and create several cloud-platform objects\nto instantiate that object. If any of those operations fail, we want to clean up so\nthe state is as if the operation had never occurred.\n\nStairway provides a way for you to organize your code so that such operations are:\n * **Atomic** - they either happen or don't happen. If something goes wrong, the operation\n   is undone.\n * **Recoverable** - a failure of the service or system does not lose state. When the system recovers, the operation can proceed or rollback.\n\nStairway does not provide as strong a transaction guarantee as a database system. Most database systems are able to\nprovide:\n * **Isolation** -  one transaction runs as if it is the only transaction running\n * **Consistenty** - a transaction sees a consistent state of the underlying data\n\nStairway cannot intermediate access to underlying data and objects, so it cannot control\nthe view of those objects in the course of running the saga transaction.\n\nWith Stairway, you still have to write code with some care to make sure that it results\nin idempotent behavior. The benefit is that Stairway provides a common idiom, and handles\npersistent bookkeeping, retrying, logging, and recovery after failure.\n\n## References\nFor more information on developing flights in Stairway see: [Stairway Flight Developer Guide](FLIGHT_DEVELOPER_GUIDE.md)\n\nFor information on developing Stairway see [Developing Stairway](DEVELOPMENT.md)\n\nFor information on the StairCtl debug/recovery tool, see [StairCtl](STAIRCTL.md)\n\n## Overview\nThis section provides an overview of Stairway concepts.\n\n### Flights and Steps\nA `Step` contains a single operation. It provides two methods: `do` and `undo`. The `do` method performs the\noperation. The `undo` method removes the effects of the operation. Your step classes are derived from the `Step`\nbase class.\n\nA `Flight` a set of Steps (of course). The `Flight` provides overall atomicity by running the `do` and `undo`\nmethods, moving from step to step. Your flight classes are derived from the `Flight` base class.\n\nA `do` method can return three result states:\n- Success: all is well and we can continue forward.\n- Fatal: all is broken. We need to rollback.\n- Retry: a retryable error happened. Invoke the retry logic associated with the step.  \n\nEach Flight has two key-value maps that are accessible to each Step:\n- input parameters - an immutable `Map\u003cString, Object\u003e` containing input parameters specified when the flight\nwas launched\n- working parameters - a mutable `Map\u003cString, Object\u003e` containing data developed and used during execution of\nthe flight.\n\nWorking parameters are the way that steps communicate with each other or remember information needed to undo\ntheir work.\n\nWhen the flight completes, the working parameters can be retrieved. The result of the flight can be gathered\nfrom the working parameters. For example, TDR uses the convention that Flights store their results in an object\nwith the key \"result\". That object is serialized out as the response to the REST API endpoint.\n\n### Recovery\nThe Stairway system maintains persistent state about where it is in the Flight, so that a failure of\nthe system can be recovered. We use a SQL database for the persistent store.\n\nThe Flight stores its input parameters and working map at each step boundary. Stairway also persistently stores\nerrors, result states, and whether the flight is going forward or rolling back.\n\nOn recovery, Stairway can re-create the Flight object using its stored input parameters, re-create the execution\ncontext using the working map and other information, and re-launch the Flight. It starts running the flight at\nthe step it was on when the system failed.\n\nThat has a ramification for writing Step methods: they need to be written so that on a re-run,\nthey do the right thing. That is usually trickier for `do` methods.\n\n### Retrying\nEach step can be annotated with a RetryRule. A retryable error will cause an undo of the step.\nIt may sleep for a bit. Then the Flight will retry the `do` method.\nStairway comes with four built-in retry rules:\n- None - retry is not done. Retryable errors are treated as Fatal.\n- FixedInterval - retry is done K times with a fixed time interval between attempts\n- RandomBackoff - retry is done K times with a random time interval between attempts\n- ExponentialBackoff - retry is done K times with exponentially growing time interval between attempts\n\nYou can create your own derivation of the `RetryRule` class and implement your own retry algorithm.\n\n### Concurrency\nStairway is designed to provide atomic operations for one instance of one service. It does not coordinate any\nglobal or cross-service state. Therefore, it is up to the application or service to implement concurrency control on \nits objects.\n\n### Context Awareness and Logs\nStairway leverages the underlying logging system's mapped diagnostic context (MDC) if available.\n\nMDC manages contextual information on a per-thread basis: as Stairway submits a Flight for processing,\nit **passes along the MDC of the calling thread** so that context isn't lost.  It also further populates\nthe MDC with **flight-specific context** for the duration of the flight's execution on the thread,\nand **step-specific context** for the duration of each step's execution.\n\nFor more information on MDC, please see [Logback's MDC manual](https://logback.qos.ch/manual/mdc.html).\n\n# TODOs\n* Add a section on clusters, queuing, failure, and recovery\n* Add a section - perhaps in DEVELOPMENT.md describing the schema\n* Add a link to a flight example. Maybe https://github.com/DataBiosphere/terra-workspace-manager/blob/dev/src/main/java/bio/terra/workspace/service/resource/controlled/flight/create/CreateControlledResourceFlight.java\n* Best practices on serdes of FlightMap parameters\n* Guildance on developer testing of Stairway internal code and flights\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fstairway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabiosphere%2Fstairway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabiosphere%2Fstairway/lists"}