{"id":19438801,"url":"https://github.com/spacejam/lathe","last_synced_at":"2026-03-01T07:31:44.811Z","repository":{"id":25307115,"uuid":"28733677","full_name":"spacejam/lathe","owner":"spacejam","description":"performant self-healing distributed micro-framework in rust","archived":false,"fork":false,"pushed_at":"2015-01-04T23:45:31.000Z","size":492,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-28T10:55:16.114Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/spacejam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-01-03T03:24:18.000Z","updated_at":"2019-12-29T19:40:29.000Z","dependencies_parsed_at":"2022-07-25T12:32:03.620Z","dependency_job_id":null,"html_url":"https://github.com/spacejam/lathe","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/spacejam/lathe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Flathe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Flathe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Flathe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Flathe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/spacejam","download_url":"https://codeload.github.com/spacejam/lathe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/spacejam%2Flathe/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29963752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T06:55:38.174Z","status":"ssl_error","status_checked_at":"2026-03-01T06:53:04.810Z","response_time":124,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T15:19:50.397Z","updated_at":"2026-03-01T07:31:44.787Z","avatar_url":"https://github.com/spacejam.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"Lathe\n=====\nLathe provides modular abstractions for running code in distributed environments.  You provide the business logic, and Lathe can handle deployment, monitoring, dynamic reconfiguration, and persistence.  It supports low-latency synchronous services, asynchronous data processing systems, persistence, caching, or any mix that you need.  It comes with some high level interfaces if you want to use them, but it makes no assumptions about how you need to use the core functionality of the system: service discovery and scheduling.  This is what makes it a micro-framework: it provides some modular tools but does not force \"the one true way\".  It is inspired by [Samza](https://github.com/apache/incubator-samza), [Storm](https://github.com/apache/storm), [Consul](https://github.com/hashicorp/consul), [groupcache](https://github.com/golang/groupcache) and [Circuit](https://github.com/gocircuit/circuit).\n###### Requirements \n1. Mesos\n2. Zookeeper\n3. Kafka is optional for asynchronous flows and persistent state replication\n\n### Core Components\n#### Scheduling\nYou can tag your tasks and provide affinities.  For instance, if you want to run several databases on your cluster, but want to guarantee that only one will ever be scheduled on a particular machine, you may tag each task with \"heavy_disk\" and set the task's bias for \"heavy_disk\" to \"never\".  Other choices are \"always\", \"prefer\", \"avoid\", and \"neutral\".  \"neutral\" is the default if you do not specify a bias on either task.  If you want a service to be colocated with a database, set the bias on either or both tasks for the other to \"prefer\" or \"always\".  When you submit a configuration, if the constraints that you specify are impossible to satisfy, you will be given an error and no action will be taken.\n#### Service Discovery\nA configurable number of metadata registry instances share the responsibilities of liveness monitoring, reliable configuration change broadcasting, and serving of a REST API for requesting current configuration details as well as sending metadata to a functional group of tasks.  There is also an agent daemon that you may optionally use for providing a large number of external hosts with a local copy of the metadata state.  The local agent uses a combination of gossip and periodic polling of the registries to receive updates.  This is useful if you have a number of services in your Lathe cluster that are relied on by external services.  For instance, if you run a cache in Lathe that your external web servers make requests to, and the task for the cache is restarted on a different host or port, you may configure the agent to execute a script that updates the web application's configuration when the cache membership changes.\n### Optional Abstractions\n#### Monitoring\nSimilar to how you may configure external agents to execute specified scripts when certain types of events happen, you may also configure the metadata registry to do this.  For example, if you already have a timeseries database that you run nagios checks against periodically, you may have one of the metadata registries execute a script that writes a value into the database whenever a task dies.  You may then have nagios check whether more than 5 tasks have died in the last hour, and generate an alert when the threshold is crossed.\n#### Connection Abstractions\nWhen a task dies and is restarted, this provides runtime reconfiguration.  Traffic cutover, naive load balancing and double-writing become configuration details rather than modifications to your code.\n#### Persistence\nInspired by Samza: creates a local RocksDB instance that is optionally replicated to Kafka to be inherited by the task's replacement should it die.\n#### Caching\nBased on groupcache.\n\nmy_topology.toml:\n```toml\n[task.http.lb]\ncrate = \"http_loadbalancer\"  # required\nversion = \"0.8.2\"            # required\ntask = \"my::LoadBalancer\"    # required\ntasks = 3                    # required\ntags = [ \"lb\", \"net\" ]       # optional, assists scheduler properly distribute\nbiases = [                   # optional scheduler biases based on tags\n  [\"lb\", \"avoid\"],\n  [\"frontpage\", \"prefer\"]\n]\nbackend_distribution = [     # you can provide your own keys to be accessed from the task\n  [\"task.http.frontpage.v4-2-1\", \"task.http.frontpage.v4-2-2\"],\n  [90, 10]\n]\n\n[task.http.frontpage.v4-2-1]\ncrate = \"http_frontpage\"\nversion = \"4.2.1\"\ntasks = 5\ntask = \"my::FrontPage\"\ntags = [ \"frontpage\" ]\n\n[task.http.frontpage.v4-2-2]\ncrate = \"http_frontpage\"\nversion = \"4.2.2\"\ntasks = 5\ntask = \"my::FrontPage\"\ntags = [ \"frontpage\" ]\n```\nsource for http_loadbalancer:\n```rust\nextern crate lathe;\nuse std::rand;\nuse std::collections::TreeMap;\nuse lathe::http::{HTTPTask,HTTPResult};\nuse lathe::config::{task,registry};\nuse lathe::task::{TaskRef};\n\nfn gethost() -\u003e TaskRef {\n  let destinations = registry.lookup_tasks(\"backend_distribution\").unwrap();\n  let mut total = 0i;\n  // don't be this lazy in production\n  let mut map = TreeMap::new();\n  for (task, weight) in destinations {\n    total += weight;\n    map.insert(weight, task);\n  }\n  let index = rand::random() * total;\n  registry.get_ref(map.find(index));  // this is cached locally, lazily populated on miss, and actively refreshed\n}\n\nstruct LoadBalancer;\nimpl HTTPTask for LoadBalancer {\n  fn handle(request: HTTPRequest) -\u003e Future\u003cHTTPResult\u003e {\n    request.forward(gethost())\n  }\n}\n```\nsource for one of the http_frontpage's:\n```rust\nextern crate lathe;\nuse lathe::http::{HTTPTask,HTTPResult,HTTPResponse};\n\nstruct FrontPage;\nimpl HTTPTask for FrontPage {\n  fn handle(request: HTTPRequest) -\u003e Future\u003cHTTPResult\u003e {\n    match request.path {\n      [\"get\", \"index\"] =\u003e Future.from_value(HTTPResponse(200, \":)\")),\n      _ =\u003e Future.from_value(HTTPResponse(400, \"invalid request\")),\n    }\n  }\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacejam%2Flathe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspacejam%2Flathe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspacejam%2Flathe/lists"}