{"id":13602140,"url":"https://github.com/oxidecomputer/omicron","last_synced_at":"2025-07-22T23:38:28.752Z","repository":{"id":36989376,"uuid":"238580629","full_name":"oxidecomputer/omicron","owner":"oxidecomputer","description":"Omicron: Oxide control plane","archived":false,"fork":false,"pushed_at":"2025-07-17T23:44:52.000Z","size":70066,"stargazers_count":307,"open_issues_count":1147,"forks_count":45,"subscribers_count":36,"default_branch":"main","last_synced_at":"2025-07-17T23:48:33.825Z","etag":null,"topics":["api"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxidecomputer.png","metadata":{"files":{"readme":"README.adoc","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-02-06T01:04:16.000Z","updated_at":"2025-07-17T22:45:31.000Z","dependencies_parsed_at":"2024-01-18T05:26:09.618Z","dependency_job_id":"6af1c50d-cfce-4c6b-b942-ed5024b5be0c","html_url":"https://github.com/oxidecomputer/omicron","commit_stats":null,"previous_names":[],"tags_count":20,"template":false,"template_full_name":null,"purl":"pkg:github/oxidecomputer/omicron","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxidecomputer%2Fomicron","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxidecomputer%2Fomicron/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxidecomputer%2Fomicron/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxidecomputer%2Fomicron/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxidecomputer","download_url":"https://codeload.github.com/oxidecomputer/omicron/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxidecomputer%2Fomicron/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266591232,"owners_count":23953082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api"],"created_at":"2024-08-01T18:01:15.087Z","updated_at":"2025-07-22T23:38:28.726Z","avatar_url":"https://github.com/oxidecomputer.png","language":"Rust","funding_links":[],"categories":["api","Rust"],"sub_categories":[],"readme":":showtitle:\n:toc: left\n:icons: font\n\n= Oxide Control Plane\n\nThis repo houses the work-in-progress Oxide Rack control plane.\n\nimage::https://github.com/oxidecomputer/omicron/workflows/Rust/badge.svg[]\n\nOmicron is open-source. But we're pretty focused on our own goals for the foreseeable future and not able to help external contributors.  Please see xref:CONTRIBUTING.md[] for more information.\n\n== Documentation\n\nhttps://docs.oxide.computer/api[Docs are automatically generated for the public (externally-facing) API] based on the OpenAPI spec that itself is automatically generated from the server implementation.  You can generate your own docs for either the public API or any of the internal APIs by feeding the corresponding OpenAPI specs (in link:./openapi[]) into an OpenAPI doc generator.\n\nThere are some internal design docs in the link:./docs[] directory.  You might start with link:./docs/control-plane-architecture.adoc[].\n\nFor more design documentation and internal Rust API docs, see the https://rust.docs.corp.oxide.computer/omicron/[generated Rust documentation].  You can generate this yourself with:\n\n[source,text]\n----\n$ cargo doc --document-private-items\n----\n\nNote that `--document-private-items` is configured by default, so you can actually just use `cargo doc`.\n\nFolks with access to Oxide RFDs may find RFD 48 (\"Control Plane Requirements\") and other control plane RFDs relevant.  These are not currently publicly available.\n\n== Build and run\n\nOmicron has two modes of operation: \"simulated\" and \"non-simulated\".\n\nThe simulated version of Omicron allows the high-level control plane logic to run without\nactually managing any sled-local resources. This version can be executed on Linux, Mac, and illumos.\nThis mode of operation is provided for development and testing only.\n\nTo build and run the simulated version of Omicron, see: xref:docs/how-to-run-simulated.adoc[].\n\nThe non-simulated version of Omicron actually manages sled-local resources, and may only\nbe executed on hosts running Helios.\nThis mode of operation will be used in production.\n\nTo build and run the non-simulated version of Omicron, see: xref:docs/how-to-run.adoc[].\n\n=== Run tests with nextest\n\nThe supported way to run tests is via https://nexte.st/[cargo-nextest].\n\nNOTE: `cargo test` will not work for many of our tests, since they rely on nextest-specific features.\n\nIf you don't already have nextest installed, get started by https://nexte.st/book/pre-built-binaries[downloading a pre-built binary] or installing nextest via your package manager. Nextest has pre-built binaries for Linux, macOS and illumos.\n\nThen, run tests with:\n\n[source,text]\n----\n$ cargo nextest run\n----\n\nNextest https://github.com/nextest-rs/nextest/issues/16[does not support doctests]. Run doctests separately with `cargo test --doc`.\n\nSimilarly, you can run tests inside a https://github.com/oxidecomputer/falcon[Falcon] based VM. This is described in the `test-utils` https://github.com/oxidecomputer/omicron/tree/main/test-utils[README].\n\nThere's also a xref:./live-tests/README.adoc[`live-tests`] test suite that can be run by hand in a _deployed_ Omicron system.\n\n=== rustfmt and clippy\n\nYou can **format the code** using `cargo fmt`.  Make sure to run this before pushing changes.  The CI checks that the code is correctly formatted.\n\nYou can **run the https://github.com/rust-lang/rust-clippy[Clippy linter]** using `cargo xtask clippy`.  CI checks that code is clippy-clean.\n\n== Working in Omicron\n\nOmicron is a pretty large repo containing a bunch of related components.  (Why?  See xref:docs/repo.adoc[].)  If you just build the whole thing with `cargo build` or `cargo nextest run`, it can take a while, even for incremental builds.  Since most people are only working on a few of these components at a time, it's helpful to be know about Cargo's tools for working with individual packages in a workspace.\n\nNOTE: This section assumes you're already familiar with the prerequisites and environment setup needed to do _any_ work on Omicron.  See xref:docs/how-to-run-simulated.adoc[] or xref:docs/how-to-run.adoc[] for more on that.\n\n=== Key tips\n\n* Use `cargo check` when you just want to know if your code compiles.  It's _much_ faster than `cargo build` or `cargo nextest run`.\n* When using Cargo's check/build/test/clippy commands, you can use the `-p PACKAGE` flag to only operate on a specific package.  This often saves a lot of time for incremental builds.\n* When using Cargo's check/build/clippy commands, use `--all-targets` to make sure you're checking or building the test code, too.\n\nThese are explained a bit more below, along with some common pitfalls.\n\nHere's an example workflow.  Suppose you're working on some changes to the Nexus database model (`nexus-db-model` package, located at `nexus/db-model` from the root).  While you're actively writing and checking code, you might run:\n\n```\ncargo check --all-targets\n```\n\n_without_ any `-p` flag.  Running this incrementally is pretty fast even on the whole workspace.  This also uncovers places where your changes have broken code that uses this package.  (If you're making big changes, you might not want that right away.  In that case, you might choose to use `-p nexus-db-model` here.)\n\nWhen you're ready to test the changes you've made, start with building and running tests for the most specific package you've changed:\n\n```\ncargo nextest run -p nexus-db-model\n```\n\nOnce that works, check the tests for the next package up:\n\n```\ncargo nextest run -p omicron-nexus\n```\n\nWhen you're happy with things and want to make sure you haven't missed something, test everything:\n\n```\ncargo nextest run\n```\n\n=== Adding a new system library dependency\n\nWe check that certain system library dependencies are not leaked outside of their intended binaries via `cargo xtask verify-libraries` in CI. If you are adding a new dependency on a illumos/helios library it is recommended that you update xref:.cargo/xtask.toml[] with an allow list of where you expect the dependency to show up. For example some libraries such as `libnvme.so.1` are only available in the global zone and therefore will not be present in any other zone. This check is here to help us catch any leakage before we go to deploy on a rack. You can inspect a compiled binary in the target directory for what it requires by using `elfedit` - for example `elfedit -r -e 'dyn:tag NEEDED' /path/to/omicron/target/debug/sled-agent`.\n\n=== Checking feature flag combinations\n\nTo ensure that varying combinations of features compile, run `cargo xtask check-features`, which executes the https://github.com/taiki-e/cargo-hack[`cargo hack`] subcommand under the hood.\n\nThis `xtask` is run in CI using the `--ci` parameter , which automatically exludes certain `image-*` features that purposefully cause compiler errors if set and uses a pre-built binary.\n\nIf `cargo hack` is not already installed in omicron's `out/` directory, a pre-built binary will be installed automatically depending on your operating system and architecture.\n\nTo limit the max number of simultaneous feature flags combined for checking, run the `xtask` with the `--depth \u003cNUM\u003e` flag:\n\n[source,text]\n----\n$ cargo xtask check-features --depth 2\n----\n\n=== Rust packages in Omicron\n\nNOTE: The term \"package\" is overloaded: most programming languages and operating systems have their own definitions of a package.  On top of that, Omicron bundles up components into our own kind of \"package\" that gets delivered via the install and update systems.  These are described in the `package-manifest.toml` file in the root of the repo.  In this section, we're just concerned with Rust packages.\n\nNOTE: There's also confusion in the Rust world about the terms https://doc.rust-lang.org/book/ch07-01-packages-and-crates.html[\"packages\" and \"crates\"].  _Packages_ are the things that have a Cargo.toml file.  (Workspaces like Omicron itself have Cargo.toml files, too.)  Packages are also the things that you publish to crates.io (confusingly).  One package might have a library, a standalone executable binary, several examples, integration tests, etc. that are all compiled individually and produce separate artifacts.  These are what Rust calls _crates_.  We're generally just concerned with packages here, not crates.\n\nHere are some of the big components in the control plane that live in this repo:\n\n[cols=\"1,1,4\",options=\"header\"]\n|===\n|Main rust package\n|Component\n|Description\n\n|omicron-nexus\n|Nexus\n|Service responsible for handling external API requests and orchestrating the rest of the control plane.\n\n|omicron-sled-agent\n|Sled Agent\n|Service that runs on each compute sled (server) to manage resources on that Sled\n\n|dns-server\n|Internal DNS server, External DNS server\n|DNS server component used for both internal service discovery and external DNS\n\n|omicron-gateway\n|Management Gateway Service\n|Connects Nexus (and other control plane services) to services on the rack management network (e.g., service processors)\n\n|oximeter/oximeter\n|Oximeter\n|Collects telemetry from other services and stores it into Clickhouse\n\n|wicket/wicketd\n|Wicket\n|CLI interface made available to operators on the rack technician port for rack setup and recovery\n\n|===\n\nFor those with access to Oxide RFDs, RFD 61 discusses the organization principles and key components in more detail.\n\nMany of these components themselves are made up of other packages (e.g., `nexus-db-model` is under `omicron-nexus`).  There are also many more top-level packages than what's mentioned above.  These are used for common code, clients, tools, etc.  For more, see the Rustdoc for each module.  (Where docs are missing or incomplete, please contribute!)\n\nUse Cargo's `-p PACKAGE` to check/build/test only the package you're working on.  Since people are usually only working on one or two components at a time, you can usually iterate faster this way.\n\n=== Workspace management\n\nOmicron uses `cargo-hakari` to ensure that all workspace dependencies enable the same set of features. This dramatically improves compilation time when switching between different subsets of packages (e.g. `-p wicket` or `-p nexus-db-model`), because the sets of enabled features remain consistent.\n\n`cargo hakari` status is checked in CI; if the CI check fails, then update the configuration locally with\n\n```\ncargo install cargo-hakari --locked # only needed on the first run\ncargo hakari generate\ncargo hakari manage-deps\n```\n\n=== Why am I getting compile errors after I thought I'd already built everything?\n\nSay you're iterating on code, running `cargo build -p nexus-db-model` to build just that package.  You work through lots of compiler errors until finally it works.  Now you run tests: `cargo nextest run -p nexus-db-model`.  Now you see a bunch of compiler errors again!  What gives?\n\nBy default, Cargo does not operate on the tests.  Cargo's check/build/clippy commands ignore them.  This is another reason we suggest using `--all-targets` most of the time.\n\n=== Generated Service Clients and Updating\n\nEach service is a Dropshot server that presents an HTTP API. The description of\nthat API is serialized as an\nhttps://github.com/OAI/OpenAPI-Specification[OpenAPI] document which we store\nin link:./openapi[`omicron/openapi`] and check in to this repo. Checking in\nthese generated files allows us:\n\n. To catch accidental changes as test failures.\n. To explicitly observe API changes during code review (and in the git history).\n\nWe also use these OpenAPI documents as the source for the clients we generate\nusing https://github.com/oxidecomputer/progenitor[Progenitor]. Clients are\nautomatically updated when the coresponding OpenAPI document is modified.\n\nOpenAPI documents are tracked by the `cargo xtask openapi` command.\n\n* To regenerate all OpenAPI documents, run `cargo xtask openapi generate`.\n* To check whether all OpenAPI documents are up-to-date, run `cargo xtask\n  openapi check`.\n\nFor more information, see the documentation in\nlink:./dev-tools/openapi-manager[`dev-tools/openapi-manager`].\n\nNote that Omicron contains a nominally circular dependency:\n\n* Nexus depends on the Sled Agent client\n* The Sled Agent client is derived from the OpenAPI document emitted by Sled Agent\n* Sled Agent depends on the Nexus client\n* The Nexus client is derived from the OpenAPI document emitted by Nexus\n\nWe effectively \"break\" this circular dependency by virtue of the OpenAPI\ndocuments being checked in.\n\n=== Resolving merge conflicts in Cargo.lock\n\nWhen pulling in new changes from upstream \"main\", you may find conflicts in Cargo.lock.  The easiest way to deal with these is usually to take the upstream changes as-is, then trigger any Cargo operation that updates the lockfile.  `cargo metadata` is a quick one.  Here's an example:\n\n```\n# Pull in changes from upstream \"main\"\n$ git fetch\n$ git merge origin/main\n\n# Oh no!  We've got conflicts in Cargo.lock.  First, let's just take what's upstream:\n$ git show origin/main:Cargo.lock \u003e Cargo.lock\n\n# Now, run any command that causes Cargo to update the lock file as needed.\n$ cargo metadata \u003e /dev/null\n```\n\nWhen you do this, Cargo makes only changes to Cargo.lock that are necessary based on the various Cargo.toml files in the workspace and dependencies.\n\nHere are things you _don't_ want to do to resolve this conflict:\n\n* Run `cargo generate-lockfile` to generate a new lock file from scratch.\n* Remove `Cargo.lock` and let Cargo regenerate it from scratch.\n\nBoth of these will cause Cargo to make many more changes (relative to \"main\") than necessary because it's choosing the latest version of all dependencies in the whole tree.  You'll be inadvertently updating all of Omicron's transitive dependencies.  (You might conceivably want that.  But usually we update dependencies either as-needed for a particular change or via individual PRs via dependabot, not all at once because someone had to merge Cargo.lock.)\n\nYou can also resolve conflicts by hand.  It's tedious and error-prone.\n\n\n=== Configuring ClickHouse\n\nThe ClickHouse binary uses several sources for its configuration. The binary expects an XML\nconfig file, usually named `config.xml` to be available, or one may be specified with the\n`-C` command-line flag. The binary also includes a minimal configuration _embedded_ within\nit, which will be used if no configuration file is given or present in the current directory.\nThe server also accepts command-line flags for overriding the values of the configuration\nparameters.\n\nThe packages downloaded by `cargo xtask download clickhouse` include a `config.xml` file with them.\nYou should probably run ClickHouse via the `ch-dev` tool, but if you decide to run it\nmanually, you can start the server with:\n\n[source,text]\n$ /path/to/clickhouse server --config-file /path/to/config.xml\n\nThe configuration file contains a large number of parameters, but most of them are described\nwith comments in the included `config.xml`, or you may learn more about them\nhttps://clickhouse.tech/docs/en/operations/server-configuration-parameters/settings/[here]\nand https://clickhouse.tech/docs/en/operations/settings/[here]. Parameters may be updated\nin the `config.xml`, and the server will automatically reload them. You may also specify\nmany of them on the command-line with:\n\n[source,text]\n$ /path/to/clickhouse server --config-file /path/to/config.xml -- --param_name param_value ...\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxidecomputer%2Fomicron","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxidecomputer%2Fomicron","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxidecomputer%2Fomicron/lists"}