{"id":14235610,"url":"https://github.com/rtyler/otto","last_synced_at":"2025-03-17T09:30:46.093Z","repository":{"id":36808810,"uuid":"170025261","full_name":"rtyler/otto","owner":"rtyler","description":"Otto is your friendly continuous delivery companion.","archived":false,"fork":false,"pushed_at":"2023-03-25T00:05:34.000Z","size":1277,"stargazers_count":80,"open_issues_count":34,"forks_count":5,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-02-27T21:50:26.194Z","etag":null,"topics":["ci","ci-cd","continuous-delivery","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rtyler.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-10T20:46:24.000Z","updated_at":"2023-12-15T02:49:53.000Z","dependencies_parsed_at":"2024-10-27T12:48:49.593Z","dependency_job_id":"d3fa2224-ef01-43ac-b9ea-179db5e8c95a","html_url":"https://github.com/rtyler/otto","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtyler%2Fotto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtyler%2Fotto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtyler%2Fotto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rtyler%2Fotto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rtyler","download_url":"https://codeload.github.com/rtyler/otto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243858810,"owners_count":20359257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci","ci-cd","continuous-delivery","rust"],"created_at":"2024-08-20T21:02:08.996Z","updated_at":"2025-03-17T09:30:45.491Z","avatar_url":"https://github.com/rtyler.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":":toc: macro\n\n= Otto\n\n**link:https://webchat.freenode.net/?channels=#otto[Chat on IRC]**\n\nMeet Otto, your friendly continuous delivery companion.\n\nOtto is a robust distributed system for scalable continuous integration and\ndelivery. To accomplish this Otto is multi-process oriented and distributed by\ndefault; all system interactions must occur over process boundaries even if\nexecuting in the same logical host. This document outlines the high level\narchitecture but omits specific inter-process communication formats, protocols,\nand dependencies.\n\nOtto does not aim to be the center of the entire continuous delivery process,\nbut rather seeks to interoperate seamlessly with all the various components\nwhich make CD work for you.\n\ntoc::[]\n\n= Status\n\n**Otto is currently not usable.**\n\nThere are link:rfc[design documents] in the `rfc/` directory which can help\ndescribe the state of development for Otto.\n\nThe components are in different states of development. Please consult\nthe README document in the subfolders for their current purpose and status.\n\n= Development\n\nMuch of Otto is built in link:https://www.rust-lang.org/[Rust]. The project is\na \"cargo workspace\" which means that there are multiple binaries and libraries\ndefined in the source tree, which can all be built together via the root\n`Cargo.toml`. Much can be accomplished with `cargo build` and `cargo test`,\nwhich is easily scoped to a single project via the `-p` flag, for example\n`cargo test -p otto-parser`. There is also a `Makefile` which drives some\nhigher-level build system behavior.\n\nOtto is composed of many different services, which communicate via JSON over HTTP.\n\n== Make targets\n\nRunning `make` in the root directory will list some brief help output, but some useful make targets to be aware of are documented below:\n\n=== `release`\n\nBuild and strip release binaries to prepare for packaging. Most developers\nwon't need to execute this target.\n\n=== `run`\n\nLaunch the services defined in the `Procfile`. This requires a `cargo build`\nahead of time, and is really only useful for manual integration testing.\n\n=== `steps`\n\nThis target will build and package all the steps defined in `stdlib/`\n\n=== `test`\n\nRuns all the acceptance tests, typically implemented with\nlink:https://github.com/kward/shunit2/[shunit2]. This target will *not* run\n`cargo build` or `cargo test`. From a fresh clone, running the acceptance tests\nwill require an initial build, for example `cargo build \u0026\u0026 make test`.\n\n== Subdirectories\n\nEach subdirectory should have its own README with a little more information,\nbut at a glance:\n\n=== `cli/`\n\nThis directory contains all the command-line interfaces for Otto.\n\n=== `crates/`\n\nThis directory contains the various pieces of shared code\n\n=== `rfcs/`\n\nRFCs (Request for Comment) are design documents for different patterns or\nsubcomponents within Otto.\n\n=== `services/`\n\nProjects in this directory are Otto's mesh of services, which speak HTTP to\nprovide different aspects of functionality for the Otto project.\n\n=== `stdlib/`\n\nThe Otto step \"Standard Library.\" In essence, these are all the step libraries\nthat are assumed to be installed by default with Otto.\n\n\n\n== Problems to Solve\n\nBelow is an incomplete listing of the problems Otto aims to solve for:\n\n* **Existing tools do not model the entire continuous delivery process.** Using\n  an external tool such as Puppet, or relying on an external provider such as\n  AWS ECS, there can be a \"black hole\" in the deployment. A point where control\n  is delegated to an external system, and the orchestration tool (Otto), loses\n  sight of what is happening.\n* **Expecting \"one single instance\" to be the hub is unrealistic.** Many\n  deployment processes have \"development\" operated components, and \"ops\"\n  operated components, with little to no automated hand-off of control between\n  the two.\n* **Mixing of management and execution contexts causes a myriad of issues.**\n  Many tools allow the management/orchestrator process to run user-defined\n  workloads. This allows breaches of isolation between user-defined workloads\n  and administrator configuration and data.\n* **Non-deterministic runtime behavior adds instability.** Without being able to\n  \"explain\" a set of operations which should occur before runtime, it is\n  impossible to determine whether or not a given delivery pipeline is correctly\n  constructed.\n* **Blending runtime data and logic with process definition confuses users.** Related to the\n  problem above, Providing runtime data about the process in a manner which is\n  only accessible in the delivery process itself, overly complicates the parsing\n  and execution of a defined continuous delivery process.\n* **Modeling of the delivery process is blurred with build tooling.** Without a\n  clear separation of concerns between the responsibility of build tools like\n  GNU/Make, Maven, Gradle, etc and the continuous delivery process definition,\n  logic invariably bleeds between the two.\n* **Opinionated platform requirements prevent easy usage across different\n  environments.** Forcing a reliance on containers, or a runtime like the Java\n  Virtual Machine results in burdensome system configuration before starting to\n  do the real work of defining a continuous delivery process. Without gracefully\n  degrading in functionality depending on the system requirements which are\n  present, users are forced to hack around the platform requirements, or spent\n  significant worrying about and maintaining pre-requisites.\n* **Many tools are difficult to configure  by default.** For most application\n  stacks, there are common conventions which can be easily prescribed for the\n  80% use-case.  Ruby on Rails applications will almost all look identical, and\n  should require zero additional configuration.\n* **Secrets and credentials can be inadvertently leaked.** Many tools provide\n  some ability to configure secrets for the continuous delivery process, but\n  expose them to the process itself in insecure ways, which allow for leakage.\n* **Extensibility must not come at the expense of system integrity.** Systems\n  which allow for administrator, or user-injected code at runtime cannot avoid\n  system reliability and security problems. Extensibility is an important\n  characteristic to support, but secondary to system integrity.\n* **Usage cannot grow across an organization without user-defined extension**.\n  The operators of the system will not be able to provide for every eventual\n  requirement from users. Some mechanism for extending or consolidating aspects\n  of a continuous delivery process must exist.\n\n== Modeling Continuous Delivery\n\nSome characteristics of a continuous delivery process model which Otto must ensure:\n\n* **Deterministic ahead-of-time**. Without executing the full process, it must\n  be possible to \"explain\" what will happen.\n* **External interactions must be model-able.** Deferring control to an\n  external system must be accounted for in a user-defined model. For example,\n  submitting a deployment request, and then waiting for some external condition\n  to be made to indicate that the deployment has completed and the service is now\n  online. This should support both an evented model, wherein the external service\n  \"calls back\" and a polling model, where the process waits until some external\n  condition can be verified.\n* **Branching logic**, a user must be able to easily define branching logic.\n  For example, a web application's delivery may be different depending on\n  whether this is a production or a staging deployment.\n* **Describe, though not fully, environments.** All applications have at least\n  some concept of environments, whether it is a web application's concept of\n  staging/production, or a compiled asset's concept of debug/release builds.\n* **Safe credentials access**, credentials should not be exposed to in a way\n  which might allow the user-defined code to inadvertently leak the credential.\n* **Caching data between runs** must be describable in some form or fashion.\n  Taking Maven projects as an example, where successive runs of `mvn` on a\n  cold-cache will result in significant downloads of data, whereas caching\n  `~/.m2` will result in more acceptable performance.\n* **Refactor/extensibility support in-repo or externally.** Depending on\n  whether the source repository is a monorepo, or something more modular.\n  Common aspects of the process must be able to be templatized/parameterized in\n  some form, and shared within the repository or via an external repository.\n* **Scale down to near zero-configuration.** the simplest model possible should\n  simply define what platform's conventions to use. With Rails applications,\n  many applications are functionally in the same with their use of Bundler,\n  Rake, and RSpec.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtyler%2Fotto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frtyler%2Fotto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frtyler%2Fotto/lists"}