{"id":19933735,"url":"https://github.com/bazelbuild/remote-apis","last_synced_at":"2025-05-14T23:00:22.554Z","repository":{"id":38805275,"uuid":"135763492","full_name":"bazelbuild/remote-apis","owner":"bazelbuild","description":"An API for caching and execution of actions on a remote system.","archived":false,"fork":false,"pushed_at":"2025-04-10T13:30:24.000Z","size":697,"stargazers_count":355,"open_issues_count":86,"forks_count":122,"subscribers_count":66,"default_branch":"main","last_synced_at":"2025-04-13T19:43:23.236Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bazelbuild.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-06-01T21:07:33.000Z","updated_at":"2025-04-10T13:30:28.000Z","dependencies_parsed_at":"2023-12-07T16:47:19.729Z","dependency_job_id":"8ae2507f-094a-4288-b333-b17355af3439","html_url":"https://github.com/bazelbuild/remote-apis","commit_stats":{"total_commits":176,"total_committers":55,"mean_commits":3.2,"dds":0.7897727272727273,"last_synced_commit":"9a0af1d31814bfcbd16dce545d10899bfc2445b7"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazelbuild%2Fremote-apis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazelbuild%2Fremote-apis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazelbuild%2Fremote-apis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bazelbuild%2Fremote-apis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bazelbuild","download_url":"https://codeload.github.com/bazelbuild/remote-apis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254243353,"owners_count":22038044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T23:14:42.064Z","updated_at":"2025-05-14T23:00:22.481Z","avatar_url":"https://github.com/bazelbuild.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# remote-apis\n\nThis repository contains a collection of APIs which work together to enable\nlarge scale distributed execution and caching on source code and other inputs.\nIt describes how to upload inputs, request the execution, monitor for results,\nand cache those results. It's overall aim is to enable large scale parallel executions\nthat wouldn't be feasible on a single system, while minimizing the amount of uploads\nand executions needed by storing data in a content-addressable format and caching results.\n\n### [Remote Execution API](build/bazel/remote/execution/v2/remote_execution.proto)\n\nThe Remote Execution API is an API that, at its most general, allows clients to request\nexecution of binaries on a remote system. It is intended primarily for use by build systems,\nsuch as [Bazel](bazel.build), to distribute build and test actions through a worker pool,\nand also provide a central cache of build results. This allows builds to execute\nfaster, both by reusing results already built by other clients and by allowing many\nactions to be executed in parallel, in excess of the resource limits of the machine\nrunning the build.\n\n### [Remote Asset API](build/bazel/remote/asset/v1/remote_asset.proto)\n\nThe Remote Asset API is an API to associate Qualifiers and URIs to Digests stored in\nContent Addressable Storage. It is primary intended to allow clients to use semantically\nrelevant identifiers, such as a git repository or tarball location, to get the corresponding\nDigest. This mapping may be pushed by a client directly, or dynamically resolved and added\nto CAS by the asset server when fetched by a client.\n\n### [Remote Logstream API](build/bazel/remote/logstream/v1/remote_logstream.proto)\n\nThe Remote Logstream API is an API supporting ordered reads and writes of `LogStream`\nresources. It is intented primarily for streaming the stdout and stderr of ongoing Action\nexecutions, enabling clients to view them while the Action is executing instead of waiting\nfor it's completion.\n\n## API users\n\nThere are a number of clients and services using these APIs, they are listed\nbelow.\n\n### Clients\nThese tools use the Remote Execution API to distribute builds to workers.\n\n* [Bazel](https://bazel.build)\n* [Buck2](https://github.com/facebook/buck2)\n* [BuildStream](https://buildstream.build/)\n* [Justbuild](https://github.com/just-buildsystem/justbuild) (via `--compatible`)\n* [Pants](https://www.pantsbuild.org)\n* [Please](https://please.build)\n* [Recc](https://gitlab.com/bloomberg/recc)\n* [Reclient](https://github.com/bazelbuild/reclient)\n* [Siso](https://chromium.googlesource.com/infra/infra/+/refs/heads/main/go/src/infra/build/siso/)\n\n### Servers\nThese applications implement the Remote Execution API to serve build requests\nfrom the clients above.\n\n* [Aspect Build](https://www.aspect.build/) (commercial)\n* [bazel-remote](https://github.com/buchgr/bazel-remote) (open source, cache only)\n* [Buildbarn](https://github.com/buildbarn) (open source)\n* [BuildBuddy](https://www.buildbuddy.io/) (commercial \u0026 open source)\n* [Buildfarm](https://github.com/bazelbuild/bazel-buildfarm) (open source)\n* [BuildGrid](https://buildgrid.build/) (open source)\n* [EngFlow](https://www.engflow.com/) (commercial)\n* [Flare Build Execution](https://flare.build/products/flare-build-execution) (commercial)\n* [Justbuild](https://github.com/just-buildsystem/justbuild/blob/master/doc/tutorial/just-execute.org) (via `--compatible`, open source)\n* [Kajiya](https://chromium.googlesource.com/infra/infra/+/refs/heads/main/go/src/infra/build/kajiya/) (open source)\n* [NativeLink](https://github.com/TraceMachina/nativelink) (open source)\n* [Scoot](https://github.com/twitter/scoot) (open source)\n\n### Workers\nServers generally distribute work to a fleet of workers.\nThe [Remote Worker API](https://docs.google.com/document/d/1s_AzRRD2mdyktKUj2HWBn99rMg_3tcPvdjx3MPbFidU)\ndefines a generic protocol for worker and server communication, although, \nthis API is considered too heavyweight for most use-cases. Because of that, \nmany implementations have designed their own protocols. Links to these APIs\nare provided as a reference below.\n*Adhering to any one of these protocols is not a requirement.*\n\n* [Buildfarm Operation Queues](https://bazelbuild.github.io/bazel-buildfarm/docs/architecture/queues/)\n  * Uses sets of queues for managing different payload requirements.\n* [Buildbarn Remote Worker](https://github.com/buildbarn/bb-remote-execution/blob/master/pkg/proto/remoteworker/remoteworker.proto)\n  * Uses a custom protocol for workers to connect to a scheduler and receive instructions.\n* [BuildGrid Bots](https://buildgrid.build/developer/data_model.html#rwapi)\n  * A server implementation of the Remote Workers API.\n* [Buildbox Worker](https://gitlab.com/BuildGrid/buildbox/buildbox-worker)\n  * A worker implementation of the Remote Workers API.\n\n## API Community\n\nThe [Remote Execution APIs\ngroup](https://groups.google.com/forum/#!forum/remote-execution-apis) hosts\ndiscussions related to the APIs in this repository.\n\nInterested parties meet monthly via VC to discuss issues related to the APIs,\nand several contributors have organized occasional meetups, hack-a-thons, and\nsummits. Joining the email discussion group will automatically add you to the\nGoogle Calendar invite for the monthly meeting.\n\n## Dependencies\n\nThe APIs in this repository refer to several general-purpose APIs published by\nGoogle in the [Google APIs\nrepository](https://github.com/googleapis/googleapis). You will need to refer to\npackages from that repository in order to generate code using this API. If you\nbuild the repository using the included `BUILD` files, Bazel will fetch the\nprotobuf compiler and googleapis automatically.\n\n## Using the APIs\n\nThe repository contains `BUILD` files to build the protobuf library with\n[Bazel](https://bazel.build/). If you wish to use them with your own project in\nBazel, you will possibly want to declare `cc_proto_library`,\n`java_proto_library`, etc. rules that depend on them.\n\nOther build systems will have to run protoc on the protobuf files, and link in\nthe googleapis and well-known proto types, manually.\n\n### Go (for non-Bazel build systems)\n\nThis repository contains the generated Go code for interacting with the API via\ngRPC. Get it with:\n\n```\ngo get github.com/bazelbuild/remote-apis\n```\n\nImport it with, for example:\n\n```\nrepb \"github.com/bazelbuild/remote-apis/build/bazel/remote/execution/v2\"\n```\n\n## Development\n\nEnable the git hooks to automatically generate Go proto code on commit:\n\n```\ngit config core.hooksPath hooks/\n```\n\nThis is a local setting, so applies only to this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazelbuild%2Fremote-apis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbazelbuild%2Fremote-apis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbazelbuild%2Fremote-apis/lists"}