{"id":13542499,"url":"https://github.com/superfly/corrosion","last_synced_at":"2025-05-15T03:04:10.779Z","repository":{"id":195542037,"uuid":"631619767","full_name":"superfly/corrosion","owner":"superfly","description":"Gossip-based service discovery (and more) for large distributed systems.","archived":false,"fork":false,"pushed_at":"2025-05-14T12:19:36.000Z","size":35618,"stargazers_count":937,"open_issues_count":41,"forks_count":35,"subscribers_count":9,"default_branch":"main","last_synced_at":"2025-05-14T13:39:52.526Z","etag":null,"topics":["database","distributed","distributed-systems","gossip","rust","sqlite","swim"],"latest_commit_sha":null,"homepage":"https://superfly.github.io/corrosion/","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/superfly.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-04-23T15:42:33.000Z","updated_at":"2025-05-14T10:52:22.000Z","dependencies_parsed_at":"2023-10-16T23:15:30.226Z","dependency_job_id":"29da5b63-8431-4111-aa2d-c9df2b0da200","html_url":"https://github.com/superfly/corrosion","commit_stats":null,"previous_names":["superfly/corrosion"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superfly%2Fcorrosion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superfly%2Fcorrosion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superfly%2Fcorrosion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/superfly%2Fcorrosion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/superfly","download_url":"https://codeload.github.com/superfly/corrosion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254264749,"owners_count":22041792,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","distributed","distributed-systems","gossip","rust","sqlite","swim"],"created_at":"2024-08-01T10:01:09.085Z","updated_at":"2025-05-15T03:04:10.736Z","avatar_url":"https://github.com/superfly.png","language":"Rust","readme":"# Corrosion\nGossip-based service discovery (and more) for large distributed systems.\n\n## Why we built Corrosion\n\nWe built Corrosion specifically for service discovery across a large global network, replacing Consul’s central state database with eventually consistent state distributed across our hosts.\n\nOur new tool needed to deliver the following:\n\n### Fast reads and writes\n\nGetting state (data) from a central remote source can be incredibly expensive (at least 300ms for a round-trip to something on the other side of the world), but usually takes less than 1ms from a local source.\n\n### Fast, eventual, consistency\n\nMany use cases can cope with eventual consistency, especially if a consistent state is attained sooner than later. Raft fell short for some use cases at Fly.io where round-trips to a centralized location are too expensive.\n\n### Flexibility\n\nGlobal state for a distributed system isn't one-size-fits-all. Flexible schemas and queries are essential.\n\n## How Corrosion works\n\nIn a nutshell, Corrosion:\n\n- Maintains a SQLite database on each node\n- Gossips local changes throughout the cluster\n- Uses [CR-SQLite](https://github.com/vlcn-io/cr-sqlite) for conflict resolution with CRDTs\n- Uses [Foca](https://github.com/caio/foca) to manage cluster membership using a SWIM protocol\n- Periodically synchronizes with a subset of other cluster nodes, to ensure consistency\n\n## Features\n\n- A flexible API to read from and write to Corrosion's store using SQL statements\n- File-based schemas with on-the-fly updates\n- HTTP streaming subscriptions based on SQL queries\n- Live population of configuration files from Corrosion state with user-defined [Rhai](https://rhai.rs/) templates\n- Storage and propagation of state from locally registered Consul services, replacing the central database with Corrosion's distributed state\n- Secure peer-to-peer communication with the [QUIC](https://datatracker.ietf.org/doc/html/rfc9000) transport protocol (using [Quinn](https://github.com/quinn-rs/quinn))\n\n## Usage overview\n\nRun the Corrosion agent on every node/host in the cluster. Other programs running on the node use [Corrosion's HTTP API](https://superfly.github.io/corrosion/api/index.html) to query the local Corrosion SQLite database, add and update data, and subscribe to change notifications.\n\nThe [Corrosion CLI](https://superfly.github.io/corrosion/cli/index.html) provides commands for administration and access to database and features.\n\n### Quick start\n\n- [Prepare the Corrosion configuration file](https://superfly.github.io/corrosion/config/)\n- [Specify the initial database schema](https://superfly.github.io/corrosion/schema.html)\n- [Start the Corrosion agent](https://superfly.github.io/corrosion/cli/agent.html)\n\nSee the WIP [Corrosion documentation](https://superfly.github.io/corrosion/) for more details.\n\n## Building Corrosion\n\nClone [https://github.com/superfly/corrosion.git](https://github.com/superfly/corrosion.git).\n\nFrom within the repo directory:\n\n```\ncargo build --release \u0026\u0026 mv target/release/corrosion ./\n```\n","funding_links":[],"categories":["Rust","sqlite","rust","\u003ca name=\"Rust\"\u003e\u003c/a\u003eRust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperfly%2Fcorrosion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsuperfly%2Fcorrosion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsuperfly%2Fcorrosion/lists"}