{"id":13450738,"url":"https://github.com/buchgr/bazel-remote","last_synced_at":"2025-04-12T20:40:58.928Z","repository":{"id":27376605,"uuid":"101615462","full_name":"buchgr/bazel-remote","owner":"buchgr","description":"A remote cache for Bazel","archived":false,"fork":false,"pushed_at":"2025-04-12T14:29:09.000Z","size":3853,"stargazers_count":641,"open_issues_count":74,"forks_count":166,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-04-12T15:28:19.471Z","etag":null,"topics":["bazel","build","build-tool","caching","continous-integration","docker-container","goma","remote-caching"],"latest_commit_sha":null,"homepage":"https://bazel.build","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/buchgr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-28T07:20:47.000Z","updated_at":"2025-04-12T14:29:13.000Z","dependencies_parsed_at":"2023-02-12T12:16:12.102Z","dependency_job_id":"70ce4f35-b340-46dc-a3be-34cb98676a07","html_url":"https://github.com/buchgr/bazel-remote","commit_stats":{"total_commits":797,"total_committers":72,"mean_commits":"11.069444444444445","dds":0.3513174404015057,"last_synced_commit":"a1a92db56c47f0e262b39272cc12b5aec6b6f929"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buchgr%2Fbazel-remote","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buchgr%2Fbazel-remote/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buchgr%2Fbazel-remote/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buchgr%2Fbazel-remote/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/buchgr","download_url":"https://codeload.github.com/buchgr/bazel-remote/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631670,"owners_count":21136554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bazel","build","build-tool","caching","continous-integration","docker-container","goma","remote-caching"],"created_at":"2024-07-31T07:00:37.834Z","updated_at":"2025-04-12T20:40:58.917Z","avatar_url":"https://github.com/buchgr.png","language":"Go","readme":"![Build status](https://badge.buildkite.com/c11240e6e9519111f2380dfcf5fcb49e69fd5b2326c11a3059.svg?branch=master)\n\n# bazel-remote cache\n\nbazel-remote is a HTTP/1.1 and gRPC server that is intended to be used as a\nremote build cache for [REAPI](https://github.com/bazelbuild/remote-apis)\nclients like [Bazel](https://bazel.build) or as a component of a remote\nexecution service.\n\nThe cache contents are stored in a directory on disk with a maximum cache size,\nand bazel-remote will automatically enforce this limit as needed, by deleting\nthe least recently used files. S3, GCS and experimental Azure blob storage\nproxy backends are also supported.\n\nNote that while bazel-remote is consumable as a go module, we provide no\nguarantees on the stability or backwards compatibility of the APIs. We do\nattempt to keep the standalone executable backwards-compatible between\nreleases however, and cache directory format changes are only allowed in\nmajor version upgrades.\n\n**Project status**: bazel-remote has been serving TBs of cache artifacts per day since April 2018, both on\ncommodity hardware and AWS servers. Outgoing bandwidth can exceed 15 Gbit/s on the right AWS instance type.\n\n## HTTP/1.1 REST API\n\nCache entries are set and retrieved by key, and there are two types of keys that can be used:\n1. Content addressed storage (CAS), where the key is the lowercase SHA256 hash of the entry.\n   The REST API for these entries is: `/cas/\u003ckey\u003e` or with an optional but ignored instance name:\n   `/\u003cinstance\u003e/cas/\u003ckey\u003e`.\n2. Action cache, where the key is an arbitrary 64 character lowercase hexadecimal string.\n   Bazel uses the SHA256 hash of an action as the key, to store the metadata created by the action.\n   The REST API for these entries is: `/ac/\u003ckey\u003e` or with an optional instance name: `/\u003cinstance\u003e/ac/\u003ckey\u003e`.\n\nValues are stored via HTTP PUT requests, and retrieved via GET requests.\nHEAD requests can be used to confirm whether a key exists or not.\n\nIf GET requests specify `zstd` in the `Accept-Encoding` header, then\nzstandard-encoded data may be returned.\n\nTo upload zstandard compressed data, PUT requests must set\n`Content-Encoding: zstd` and include a custom `X-Digest-SizeBytes` header\nwith the size of the uncompressed entry. The key must also refer to\nthe uncompressed entry.\n\nIf the `--enable_ac_key_instance_mangling` flag is specified and the instance\nname is not empty, then action cache keys are hashed along with the instance\nname to produce the action cache lookup key. Since the URL path is processed\nwith Go's [path.Clean](https://golang.org/pkg/path/#Clean) function before\nextracting the instance name, clients should avoid using repeated slashes,\n`./` and `../` in the URL.\n\nValues stored in the action cache are validated as an ActionResult protobuf message as per the\n[Bazel Remote Execution API v2](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto)\nunless validation is disabled by configuration. The HTTP server also supports reading and writing JSON\nencoded protobuf ActionResult messages to the action cache by using HTTP headers `Accept: application/json`\nfor GET requests and `Content-type: application/json` for PUT requests.\n\n### Useful endpoints\n\n**/status**\n\nReturns the cache status/info.\n```\n$ curl http://localhost:8080/status\n{\n \"CurrSize\": 414081715503,\n \"ReservedSize\": 876400,\n \"MaxSize\": 8589934592000,\n \"NumFiles\": 621413,\n \"ServerTime\": 1588329927,\n \"GitCommit\": \"940d540d3a7f17939c3df0038530122eabef2f19\",\n \"NumGoroutines\": 12\n}\n```\n\n**/cas/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855**\n\nThe empty CAS blob is always available, even if the cache is empty. This can be used to test that\na bazel-remote instance is running and accepting requests.\n```\n$ curl --head --fail http://localhost:8080/cas/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\nHTTP/1.1 200 OK\nContent-Length: 0\nDate: Fri, 01 May 2020 10:42:06 GMT\n```\n\n### Prometheus Metrics\n\nTo query endpoint metrics see [github.com/slok/go-http-metrics's query examples](https://github.com/slok/go-http-metrics#prometheus-query-examples).\n\n## gRPC API\n\nbazel-remote also supports the ActionCache, ContentAddressableStorage and Capabilities services in the\n[Bazel Remote Execution API v2](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/execution/v2/remote_execution.proto),\nand the corresponding parts of the [Byte Stream API](https://github.com/googleapis/googleapis/blob/master/google/bytestream/bytestream.proto).\n\nWhen using the `--enable_ac_key_instance_mangling` feature, clients are\nadvised to avoid repeated slashes, `../` and `./` strings in the instance\nname, for consistency with the HTTP interface.\n\n### Prometheus Metrics\n\nTo query endpoint metrics see [github.com/grpc-ecosystem/go-grpc-prometheus's metrics documentation](https://github.com/grpc-ecosystem/go-grpc-prometheus#metrics).\n\n### Experimental Remote Asset API Support\n\nThere is (very) experimental support for a subset of the Fetch service in the\n[Remote Asset API](https://github.com/bazelbuild/remote-apis/blob/master/build/bazel/remote/asset/v1/remote_asset.proto)\nwhich can be enabled with the `--experimental_remote_asset_api` flag.\n\nTo use this with Bazel, specify\n[--experimental_remote_downloader=grpc://replace-with-your.host:port](https://docs.bazel.build/versions/master/command-line-reference.html#flag--experimental_remote_downloader).\n\n### Byte Stream compressed-blobs\n\nThis version of bazel-remote supports the\n[Byte Stream compressed-blobs REAPI feature](https://github.com/bazelbuild/remote-apis/pull/168),\nwhich provides a way for clients to upload and download CAS blobs compressed\nwith zstandard, in order to improve network efficiency.\n\nUploaded CAS blobs are stored in a zstandard compressed format by default,\nwhich can increase the effective cache size and reduce load on the server\nif clients also download blobs in zstandard compressed form. If you would\nrather store CAS blobs in uncompressed form, add `--storage_mode uncompressed`\nto your configuration.\n\n## Usage\n\nIf a YAML configuration file is specified by the `--config_file` command line\nflag or `BAZEL_REMOTE_CONFIG_FILE` environment variable, then other command\nline flags and environment variables are ignored. Otherwise, the flags and\nenvironment variables listed in the help text below can be specified (flags\noverride the corresponding environment variables).\n\nSee [examples/bazel-remote.service](examples/bazel-remote.service) for an\nexample (systemd) linux setup.\n\n### Command line flags\n\n```\n$ ./bazel-remote --help\nbazel-remote - A remote build cache for Bazel and other REAPI clients\n\nUSAGE:\n   bazel-remote [options]\n\nOPTIONS:\n   --config_file value Path to a YAML configuration file. If this flag is\n      specified then all other flags are ignored. [$BAZEL_REMOTE_CONFIG_FILE]\n\n   --dir value Directory path where to store the cache contents. This flag is\n      required. [$BAZEL_REMOTE_DIR]\n\n   --max_size value The maximum size of bazel-remote's disk cache in GiB.\n      This flag is required. (default: 0) [$BAZEL_REMOTE_MAX_SIZE]\n\n   --storage_mode value Which format to store CAS blobs in. Must be one of\n      \"zstd\" or \"uncompressed\". (default: \"zstd\") [$BAZEL_REMOTE_STORAGE_MODE]\n\n   --zstd_implementation value ZSTD implementation to use. Supported values:\n      \"cgo\", \"go\" (default: \"go\") [$BAZEL_REMOTE_ZSTD_IMPLEMENTATION]\n\n   --http_address value Address specification for the HTTP server listener,\n      formatted either as [host]:port for TCP or unix://path.sock for Unix\n      domain sockets. [$BAZEL_REMOTE_HTTP_ADDRESS]\n\n   --host value DEPRECATED. Use --http_address to specify the HTTP server\n      listener. [$BAZEL_REMOTE_HOST]\n\n   --port value DEPRECATED. Use --http_address to specify the HTTP server\n      listener. (default: 8080) [$BAZEL_REMOTE_PORT]\n\n   --grpc_address value Address specification for the gRPC server listener,\n      formatted either as [host]:port for TCP or unix://path.sock for Unix\n      domain sockets. Set to 'none' to disable. [$BAZEL_REMOTE_GRPC_ADDRESS]\n\n   --grpc_port value DEPRECATED. Use --grpc_address to specify the gRPC\n      server listener. Set to 0 to disable. (default: 9092)\n      [$BAZEL_REMOTE_GRPC_PORT]\n\n   --profile_address value Address specification for a http server to listen\n      on for profiling, formatted either as [host]:port for TCP or\n      unix://path.sock for Unix domain sockets. Off by default, but can also be\n      set to 'none' to disable explicitly. (default: \"\", ie profiling disabled)\n      [$BAZEL_REMOTE_PROFILE_ADDRESS]\n\n   --profile_host value DEPRECATED. Use --profile_address instead. A host\n      address to listen on for profiling, if enabled by a valid --profile_port\n      setting. (default: \"127.0.0.1\") [$BAZEL_REMOTE_PROFILE_HOST]\n\n   --profile_port value DEPRECATED. Use --profile_address instead. If a\n      positive integer, serve /debug/pprof/* URLs from\n      http://profile_host:profile_port. (default: 0, ie profiling disabled)\n      [$BAZEL_REMOTE_PROFILE_PORT]\n\n   --http_read_timeout value The HTTP read timeout for a client request in\n      seconds (does not apply to the proxy backends or the profiling endpoint)\n      (default: 0s, ie disabled) [$BAZEL_REMOTE_HTTP_READ_TIMEOUT]\n\n   --http_write_timeout value The HTTP write timeout for a server response in\n      seconds (does not apply to the proxy backends or the profiling endpoint)\n      (default: 0s, ie disabled) [$BAZEL_REMOTE_HTTP_WRITE_TIMEOUT]\n\n   --htpasswd_file value Path to a .htpasswd file. This flag is optional.\n      Please read https://httpd.apache.org/docs/2.4/programs/htpasswd.html.\n      [$BAZEL_REMOTE_HTPASSWD_FILE]\n\n   --min_tls_version value The minimum TLS version that is acceptable for\n      incoming requests (does not apply to proxy backends). Allowed values: 1.0,\n      1.1, 1.2, 1.3. (default: \"1.0\") [$BAZEL_REMOTE_MIN_TLS_VERSION]\n\n   --tls_ca_file value Optional. Enables mTLS (authenticating client\n      certificates), should be the certificate authority that signed the client\n      certificates. [$BAZEL_REMOTE_TLS_CA_FILE]\n\n   --tls_cert_file value Path to a pem encoded certificate file.\n      [$BAZEL_REMOTE_TLS_CERT_FILE]\n\n   --tls_key_file value Path to a pem encoded key file.\n      [$BAZEL_REMOTE_TLS_KEY_FILE]\n\n   --allow_unauthenticated_reads If authentication is enabled\n      (--htpasswd_file or --tls_ca_file), allow unauthenticated clients read\n      access. (default: false, ie if authentication is required, read-only\n      requests must also be authenticated) [$BAZEL_REMOTE_UNAUTHENTICATED_READS]\n\n   --idle_timeout value The maximum period of having received no request\n      after which the server will shut itself down. (default: 0s, ie disabled)\n      [$BAZEL_REMOTE_IDLE_TIMEOUT]\n\n   --max_queued_uploads value When using proxy backends, sets the maximum\n      number of objects in queue for upload. If the queue is full, uploads will\n      be skipped until the queue has space again. (default: 1000000)\n      [$BAZEL_REMOTE_MAX_QUEUED_UPLOADS]\n\n   --max_blob_size value The maximum logical/uncompressed blob size that will\n      be accepted from clients. Note that this limit is not applied to\n      preexisting blobs in the cache. (default: 9223372036854775807)\n      [$BAZEL_REMOTE_MAX_BLOB_SIZE]\n\n   --max_proxy_blob_size value The maximum logical/uncompressed blob size\n      that will be downloaded from proxies. Note that this limit is not applied\n      to preexisting blobs in the cache. (default: 9223372036854775807)\n      [$BAZEL_REMOTE_MAX_PROXY_BLOB_SIZE]\n\n   --num_uploaders value When using proxy backends, sets the number of\n      Goroutines to process parallel uploads to backend. (default: 100)\n      [$BAZEL_REMOTE_NUM_UPLOADERS]\n\n   --grpc_proxy.url value The base URL to use for the experimental grpc proxy\n      backend, e.g. grpc://localhost:9090 or grpcs://example.com:7070. Note that\n      this requires a backend with remote asset API support if you want http\n      client requests to work. [$BAZEL_REMOTE_GRPC_PROXY_URL]\n\n   --grpc_proxy.key_file value Path to a key used to authenticate with the\n      proxy backend using mTLS. If this flag is provided, then\n      grpc_proxy.cert_file must also be specified.\n      [$BAZEL_REMOTE_GRPC_PROXY_KEY_FILE]\n\n   --grpc_proxy.cert_file value Path to a certificate used to authenticate\n      with the proxy backend using mTLS. If this flag is provided, then\n      grpc_proxy.key_file must also be specified.\n      [$BAZEL_REMOTE_GRPC_PROXY_CERT_FILE]\n\n   --grpc_proxy.ca_file value Path to a certificate autority used to validate\n      the grpc proxy backend certificate. [$BAZEL_REMOTE_GRPC_PROXY_CA_FILE]\n\n   --http_proxy.url value The base URL to use for a http proxy backend.\n      [$BAZEL_REMOTE_HTTP_PROXY_URL]\n\n   --http_proxy.key_file value Path to a key used to authenticate with the\n      proxy backend using mTLS. If this flag is provided, then\n      http_proxy.cert_file must also be specified.\n      [$BAZEL_REMOTE_HTTP_PROXY_KEY_FILE]\n\n   --http_proxy.cert_file value Path to a certificate used to authenticate\n      with the proxy backend using mTLS. If this flag is provided, then\n      http_proxy.key_file must also be specified.\n      [$BAZEL_REMOTE_HTTP_PROXY_CERT_FILE]\n\n   --http_proxy.ca_file value Path to a certificate autority used to validate\n      the http proxy backend certificate. [$BAZEL_REMOTE_HTTP_PROXY_CA_FILE]\n\n   --gcs_proxy.bucket value The bucket to use for the Google Cloud Storage\n      proxy backend. [$BAZEL_REMOTE_GCS_BUCKET]\n\n   --gcs_proxy.use_default_credentials Whether or not to use authentication\n      for the Google Cloud Storage proxy backend. (default: false)\n      [$BAZEL_REMOTE_GCS_USE_DEFAULT_CREDENTIALS]\n\n   --gcs_proxy.json_credentials_file value Path to a JSON file that contains\n      Google credentials for the Google Cloud Storage proxy backend.\n      [$BAZEL_REMOTE_GCS_JSON_CREDENTIALS_FILE]\n\n   --ldap.url value The LDAP URL which may include a port. LDAP over SSL\n      (LDAPs) is also supported. Note that this feature is currently considered\n      experimental. [$BAZEL_REMOTE_LDAP_URL]\n\n   --ldap.base_dn value The distinguished name of the search base.\n      [$BAZEL_REMOTE_LDAP_BASE_DN]\n\n   --ldap.bind_user value The user who is allowed to perform a search within\n      the base DN. If none is specified the connection and the search is\n      performed without an authentication. It is recommended to use a read-only\n      account. [$BAZEL_REMOTE_LDAP_BIND_USER]\n\n   --ldap.bind_password value The password of the bind user.\n      [$BAZEL_REMOTE_LDAP_BIND_PASSWORD]\n\n   --ldap.username_attribute value The user attribute of a connecting user.\n      (default: \"uid\") [$BAZEL_REMOTE_LDAP_USER_ATTRIBUTE]\n\n   --ldap.groups_query value Filter clause for searching groups.\n      [$BAZEL_REMOTE_LDAP_GROUPS_QUERY]\n\n   --ldap.cache_time value The amount of time to cache a successful\n      authentication in seconds. (default: 3600) [$BAZEL_REMOTE_LDAP_CACHE_TIME]\n\n   --s3.endpoint value The S3/minio endpoint to use when using S3 proxy\n      backend. [$BAZEL_REMOTE_S3_ENDPOINT]\n\n   --s3.bucket value The S3/minio bucket to use when using S3 proxy backend.\n      [$BAZEL_REMOTE_S3_BUCKET]\n\n   --s3.bucket_lookup_type value The S3/minio bucket lookup type to use when\n      using S3 proxy backend. Allowed values: auto, dns, path. (default: \"auto\")\n      [$BAZEL_REMOTE_S3_BUCKET_LOOKUP_TYPE]\n\n   --s3.prefix value The S3/minio object prefix to use when using S3 proxy\n      backend. [$BAZEL_REMOTE_S3_PREFIX]\n\n   --s3.auth_method value The S3/minio authentication method. This argument\n      is required when an s3 proxy backend is used. Allowed values: iam_role,\n      access_key, aws_credentials_file. [$BAZEL_REMOTE_S3_AUTH_METHOD]\n\n   --s3.access_key_id value The S3/minio access key to use when using S3\n      proxy backend. Applies to s3 auth method(s): access_key.\n      [$BAZEL_REMOTE_S3_ACCESS_KEY_ID]\n\n   --s3.secret_access_key value The S3/minio secret access key to use when\n      using S3 proxy backend. Applies to s3 auth method(s): access_key.\n      [$BAZEL_REMOTE_S3_SECRET_ACCESS_KEY]\n\n   --s3.session_token value The S3/minio session token to use when using S3\n      proxy backend. Optional. Applies to s3 auth method(s): access_key.\n      [$BAZEL_REMOTE_S3_SESSION_TOKEN]\n\n   --s3.signature_type value Which type of s3 signature to use when using S3\n      proxy backend. Only applies when using the s3 access_key auth method.\n      Allowed values: v2, v4, v4streaming, anonymous. (default: v4)\n      [$BAZEL_REMOTE_S3_SIGNATURE_TYPE]\n\n   --s3.aws_shared_credentials_file value Path to the AWS credentials file.\n      If not specified, the minio client will default to '~/.aws/credentials'.\n      Applies to s3 auth method(s): aws_credentials_file.\n      [$BAZEL_REMOTE_S3_AWS_SHARED_CREDENTIALS_FILE,\n      $AWS_SHARED_CREDENTIALS_FILE]\n\n   --s3.aws_profile value The aws credentials profile to use from within\n      s3.aws_shared_credentials_file. Applies to s3 auth method(s):\n      aws_credentials_file. (default: \"default\") [$BAZEL_REMOTE_S3_AWS_PROFILE,\n      $AWS_PROFILE]\n\n   --s3.disable_ssl Whether to disable TLS/SSL when using the S3 proxy\n      backend. (default: false, ie enable TLS/SSL)\n      [$BAZEL_REMOTE_S3_DISABLE_SSL]\n\n   --s3.update_timestamps Whether to update timestamps of object on cache\n      hit. (default: false) [$BAZEL_REMOTE_S3_UPDATE_TIMESTAMPS]\n\n   --s3.iam_role_endpoint value Endpoint for using IAM security credentials.\n      By default it will look for credentials in the standard locations for the\n      AWS platform. Applies to s3 auth method(s): iam_role.\n      [$BAZEL_REMOTE_S3_IAM_ROLE_ENDPOINT]\n\n   --s3.region value The AWS region. Required when not specifying S3/minio\n      access keys. [$BAZEL_REMOTE_S3_REGION]\n\n   --s3.key_version value DEPRECATED. Key version 2 now is the only supported\n      value. This flag will be removed. (default: 2)\n      [$BAZEL_REMOTE_S3_KEY_VERSION]\n\n   --azblob.tenant_id value The Azure blob storage tenant id to use when\n      using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_TENANT_ID,\n      $AZURE_TENANT_ID]\n\n   --azblob.storage_account value The Azure blob storage storage account to\n      use when using azblob proxy backend.\n      [$BAZEL_REMOTE_AZBLOB_STORAGE_ACCOUNT]\n\n   --azblob.container_name value The Azure blob storage container name to use\n      when using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_CONTAINER_NAME]\n\n   --azblob.prefix value The Azure blob storage object prefix to use when\n      using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_PREFIX]\n\n   --azblob.update_timestamps Whether to update timestamps of object on cache\n      hit. (default: false) [$BAZEL_REMOTE_AZBLOB_UPDATE_TIMESTAMPS]\n\n   --azblob.auth_method value The Azure blob storage authentication method.\n      This argument is required when an azblob proxy backend is used. Allowed\n      values: client_certificate, client_secret, environment_credential,\n      shared_key, default. [$BAZEL_REMOTE_AZBLOB_AUTH_METHOD]\n\n   --azblob.shared_key value The Azure blob storage account access key to use\n      when using azblob proxy backend. Applies to AzBlob auth method(s):\n      shared_key. [$BAZEL_REMOTE_AZBLOB_SHARED_KEY, $AZURE_STORAGE_ACCOUNT_KEY]\n\n   --azblob.client_id value The Azure blob storage client id to use when\n      using azblob proxy backend. Applies to AzBlob auth method(s):\n      client_secret, client_certificate. [$BAZEL_REMOTE_AZBLOB_CLIENT_ID,\n      $AZURE_CLIENT_ID]\n\n   --azblob.client_secret value The Azure blob storage client secret key to\n      use when using azblob proxy backend. Applies to AzBlob auth method(s):\n      client_secret. [$BAZEL_REMOTE_AZBLOB_SECRET_CLIENT_SECRET,\n      $AZURE_CLIENT_SECRET]\n\n   --azblob.cert_path value Path to the certificates file. Applies to AzBlob\n      auth method(s): client_certificate. [$BAZEL_REMOTE_AZBLOB_CERT_PATH,\n      $AZURE_CLIENT_CERTIFICATE_PATH]\n\n   --disable_http_ac_validation Whether to disable ActionResult validation\n      for HTTP requests. (default: false, ie enable validation)\n      [$BAZEL_REMOTE_DISABLE_HTTP_AC_VALIDATION]\n\n   --disable_grpc_ac_deps_check Whether to disable ActionResult dependency\n      checks for gRPC GetActionResult requests. (default: false, ie enable\n      ActionCache dependency checks) [$BAZEL_REMOTE_DISABLE_GRPS_AC_DEPS_CHECK]\n\n   --enable_ac_key_instance_mangling Whether to enable mangling ActionCache\n      keys with non-empty instance names. (default: false, ie disable mangling)\n      [$BAZEL_REMOTE_ENABLE_AC_KEY_INSTANCE_MANGLING]\n\n   --enable_endpoint_metrics Whether to enable metrics for each HTTP/gRPC\n      endpoint. (default: false, ie disable metrics)\n      [$BAZEL_REMOTE_ENABLE_ENDPOINT_METRICS]\n\n   --http_metrics_prefix Whether to prefix http metrics with \"bazel_remote\"\n      or not (default: false, ie no prefix) [$BAZEL_REMOTE_HTTP_METRICS_PREFIX]\n\n   --experimental_remote_asset_api Whether to enable the experimental remote\n      asset API implementation. (default: false, ie disable remote asset API)\n      [$BAZEL_REMOTE_EXPERIMENTAL_REMOTE_ASSET_API]\n\n   --access_log_level value The access logger verbosity level. If supplied,\n      must be one of \"none\" or \"all\". (default: all, ie enable full access\n      logging) [$BAZEL_REMOTE_ACCESS_LOG_LEVEL]\n\n   --log_timezone value The timezone to use for log timestamps. If supplied,\n      must be one of \"UTC\", \"local\" or \"none\" for no timestamps. (default: UTC,\n      ie use UTC timezone) [$BAZEL_REMOTE_LOG_TIMEZONE]\n\n   --help, -h  show help\n```\n\n### Example configuration file\n\n```yaml\n# These two are the only required options:\ndir: path/to/cache-dir\nmax_size: 100\n\n# The form to store CAS blobs in (\"zstd\" or \"uncompressed\"):\n#storage_mode: zstd\n\n# The server listener address for HTTP/HTTPS. For TCP listeners,\n# use [host]:port, where host is optional (default 0.0.0.0) and can\n# be either a hostname or IP address. For Unix domain socket listeners,\n# use unix:///path/to/socket.sock, where /path/to/socket.sock can be\n# either an absolute or relative path to a socket path.\nhttp_address: 0.0.0.0:8080\n\n# The server listener address for gRPC (unix sockets are also supported\n# as described above):\n#grpc_address: 0.0.0.0:9092\n\n# If profile_address (or the deprecated profile_port and/or profile_host)\n# is specified, then serve /debug/pprof/* URLs here (unix sockets are also\n# supported as described above):\n#profile_address: 127.0.0.1:7070\n\n# HTTP read/write timeouts. Note that these do not apply to the proxy\n# backends or the profiling endpoint. Reasonable values might be twice\n# the length of time that you expect a client to read/write the largest\n# likely blob. Units can be one of: \"s\", \"m\", \"h\".\n#http_read_timeout: 15s\n#http_write_timeout: 20s\n\n# Specify a certificate if you want to use HTTPS and gRPCs:\n#tls_cert_file: path/to/tls.cert\n#tls_key_file:  path/to/tls.key\n# If you want to use mutual TLS with client certificates:\n#tls_ca_file: path/to/ca/cert.pem\n\n# Optionally specify the minimum supported TLS version for the\n# HTTPS/gRPCs servers (must be one of 1.0, 1.1, 1.2, 1.3):\n#min_tls_version: \"1.0\"\n\n# Alternatively, you can use simple authentication:\n#htpasswd_file: path/to/.htpasswd\n\n# At most one authentication mechanism can be used\n#ldap:\n#  url: ldaps://ldap.example.com:636\n#  base_dn: OU=My Users,DC=example,DC=com\n#  username_attribute: sAMAccountName      # defaults to \"uid\"\n#  bind_user: ldapuser\n#  bind_password: ldappassword\n#  cache_time: 3600                        # in seconds (default 1 hour)\n#  groups_query: (memberOf=CN=bazel-users,OU=Groups,OU=My Users,DC=example,DC=com)\n\n# If tls_ca_file or htpasswd_file are specified, you can choose\n# whether or not to allow unauthenticated read access:\n#allow_unauthenticated_reads: false\n\n# If specified, bazel-remote should exit after being idle\n# for this long. Time units can be one of: \"s\", \"m\", \"h\".\n#idle_timeout: 45s\n\n# If set to true, do not validate that ActionCache\n# items are valid ActionResult protobuf messages.\n#disable_http_ac_validation: false\n\n# If set to true, do not check that CAS items referred\n# to by ActionResult messages are in the cache.\n#disable_grpc_ac_deps_check: false\n\n# If set to true, enable metrics for each HTTP/gRPC endpoint.\n#enable_endpoint_metrics: false\n\n# Specify a custom list of histogram buckets for endpoint request duration metrics\n#endpoint_metrics_duration_buckets: [.5, 1, 2.5, 5, 10, 20, 40, 80, 160, 320]\n\n# At most one of the proxy backends can be selected:\n#\n# If this is 0, proxy backends won't upload blobs.\n#num_uploaders: 100\n# The maximum number of proxy uploads to queue, before dropping uploads.\n#max_queued_uploads: 1000000\n# The largest blob size that will be accepted, for example 10MB:\n#max_blob_size: 10485760\n#\n#gcs_proxy:\n#  bucket: gcs-bucket\n#  use_default_credentials: false\n#  json_credentials_file: path/to/creds.json\n#\n#s3_proxy:\n#  endpoint: minio.example.com:9000\n#  bucket: test-bucket\n#  prefix: test-prefix\n#  disable_ssl: true\n#  bucket_lookup_type: auto\n#\n# Provide exactly one auth_method (access_key, iam_role, or credentials_file) and accompanying configuration.\n#\n# Access key authenticaiton:\n#  auth_method: access_key\n#  access_key_id: EXAMPLE_ACCESS_KEY\n#  secret_access_key: EXAMPLE_SECRET_KEY\n#  session_token: EXAMPLE_SESSION_TOKEN\n#  signature_type: v4\n#\n# IAM Role authentication.\n#  auth_method: iam_role\n#  iam_role_endpoint: http://169.254.169.254\n#  region: us-east-1\n#\n# AWS credentials file.\n#  auth_method: credentials_file\n#  aws_shared_credentials_file: path/to/aws/credentials\n#  aws_profile: my-profile\n#\n#http_proxy:\n#  url: https://remote-cache.com:8080/cache\n# If you want to use mutual TLS with client certificates:\n#  cert_file: path/to/client.cert\n#  key_file:  path/to/client.key\n# If you want to use a custom CA:\n#  ca_file: path/to/ca.crt\n#\n# Note that the grpc proxy backend requires remote asset API support if\n# you want client -http-\u003e bazel-remote -grpc-\u003e backend requests to work.\n#grpc_proxy:\n#  url: grpc://remote-cache.com:9092\n# If you want to use mutual TLS with client certificates:\n#  cert_file: path/to/client.cert\n#  key_file:  path/to/client.key\n# If you want to use a custom CA:\n#  ca_file: path/to/ca.crt\n#\n#azblob_proxy:\n#  tenant_id: TENANT_ID\n#  storage_account: STORAGE_ACCOUNT\n#  container_name: CONTAINER_NAME\n#\n# Provide exactly one auth_method (client_certificate, client_secret, environment_credential,\n#shared_key, default) and accompanying configuration.\n#\n# Storage account shared key.\n#  auth_method: shared_key\n#  shared_key: APP_SHARED_KEY\n#\n# Client secret credentials.\n#  auth_method: client_secret\n#  client_id: APP_ID\n#  client_secret: APP_SECRET\n#\n# Client certificate credentials.\n#  auth_method: client_certificate\n#  cert_path: path/to/cert_file\n#\n# Default and environment methods don't have any additional parameters.\n#  auth_method: environment_credential\n#\n#  auth_method: default\n  \n# If set to a valid port number, then serve /debug/pprof/* URLs here:\n#profile_port: 7070\n# IP address to use, if profiling is enabled:\n#profile_host: 127.0.0.1\n\n# If true, enable experimental remote asset API support:\n#experimental_remote_asset_api: true\n\n# If supplied, controls the verbosity of the access logger (\"none\" or \"all\"):\n#access_log_level: none\n\n# If supplied, controls the timezone of the access logger (\"UTC\", \"local\" or \"none\"):\n#log_timezone: local\n```\n\n## Docker\n\n### Prebuilt Image\n\nWe publish docker images to [DockerHub](https://hub.docker.com/r/buchgr/bazel-remote-cache/)\nand [quay.io](https://quay.io/repository/bazel-remote/bazel-remote)\nthat you can use with `docker run`. The following commands will start bazel-remote with uid\nand gid `1000` on port `9090` for HTTP and `9092` for gRPC, with the default maximum cache\nsize of `5 GiB`.\n\n```bash\n# Dockerhub example:\n$ docker pull buchgr/bazel-remote-cache\n$ docker run -u 1000:1000 -v /path/to/cache/dir:/data \\\n\t-p 9090:8080 -p 9092:9092 buchgr/bazel-remote-cache \\\n\t--max_size 5\n```\n\n```bash\n# quay.io example:\n$ docker pull quay.io/bazel-remote/bazel-remote\n$ docker run -u 1000:1000 -v /path/to/cache/dir:/data \\\n\t-p 9090:8080 -p 9092:9092 quay.io/bazel-remote/bazel-remote \\\n\t--max_size 5\n```\n\nNote that you will need to change `/path/to/cache/dir` to a valid directory that is readable\nand writable by the specified user (or by uid/gid `65532` if no user was specified).\n\nIf you want the docker container to run in the background pass the `-d` flag right after `docker run`.\n\nYou can adjust the maximum cache size by appending `--max_size N`, where N is\nthe maximum size in Gibibytes.\n\n### Docker Compose notes\n\nSee [examples/docker-compose.yml](examples/docker-compose.yml) for an example configuration (modify the `--max_size` flag in there to suit your needs).\n\n### Kubernetes notes\n\n* See [examples/kubernetes.yml](examples/kubernetes.yml) for an example\n  configuration.\n\n* Don't name your deployment `bazel-remote`!\n  \n  Kubernetes sets some environment variables based on this name, which conflict\n  with the `BAZEL_REMOTE_*` environment variables that bazel-remote tries to\n  parse.\n\n* bazel-remote supports the `/grpc.health.v1.Health/Check` service, which you can\n  configure like so:\n  ```\n  alb.ingress.kubernetes.io/backend-protocol: HTTP\n  alb.ingress.kubernetes.io/backend-protocol-version: GRPC\n  alb.ingress.kubernetes.io/healthcheck-path: /grpc.health.v1.Health/Check\n  alb.ingress.kubernetes.io/healthcheck-port: 9092\n  alb.ingress.kubernetes.io/listen-ports: [{\"HTTPS\": 9092}]\n  alb.ingress.kubernetes.io/success-codes: 0\n  alb.ingress.kubernetes.io/target-type: ip\n  ```\n\n### Build your own docker image\n\nThe command below will build a docker image from source and install it into your local docker registry.\n\n```bash\n$ bazel build :bazel-remote-image-tarball \u0026\u0026 docker load -i bazel-bin/bazel-remote-image-tarball/tarball.tar\n$ docker run buchgr/bazel-remote-cache:tmp-amd64 --max_size 5 --dir /your/path/to/data\n```\n\n### ARM64 docker image\n\nBazel-remote can also run on ARM64 architecture devices, for example on a Raspberry Pi.\n\nTo build a docker image for ARM64:\n\n```bash\n$ bazel build :bazel-remote-image-arm64-tarball \u0026\u0026 docker load -i bazel-bin/bazel-remote-image-arm64-tarball/tarball.tar\n$ docker run buchgr/bazel-remote-cache:tmp-arm64 --max_size 5 --dir /your/path/to/data\n```\n\n## Build a standalone Linux binary\n\n```bash\n$ bazel build :bazel-remote\n```\n\n### Authentication\n\nbazel-remote defaults to allow unauthenticated access, but basic `.htpasswd`\nstyle authentication, mutual TLS authentication and (experimental) LDAP are\nalso supported.\n\nNote that only one authentication mechanism can be used at a time.\n\n#### htpasswd\n\nIn order to pass a `.htpasswd` and/or server key file(s) to the cache\ninside a docker container, you first need to mount the file in the\ncontainer and pass the path to the cache. The example below also\nconfigures TLS which is technically optional but highly recommended\nin order to not send passwords in plain text.\n\n```bash\n$ docker run -v /path/to/cache/dir:/data \\\n\t-v /path/to/htpasswd:/etc/bazel-remote/htpasswd \\\n\t-v /path/to/server_cert:/etc/bazel-remote/server_cert \\\n\t-v /path/to/server_key:/etc/bazel-remote/server_key \\\n\t-p 9090:8080 -p 9092:9092 buchgr/bazel-remote-cache \\\n\t--tls_cert_file=/etc/bazel-remote/server_cert \\\n\t--tls_key_file=/etc/bazel-remote/server_key \\\n\t--htpasswd_file /etc/bazel-remote/htpasswd --max_size 5\n```\n\n#### mTLS\n\nIf you prefer not using `.htpasswd` files it is also possible to\nauthenticate with mTLS (also can be known as \"authenticating client\ncertificates\"). You can do this by passing in the the cert/key the\nserver should use, as well as the certificate authority that signed\nthe client certificates:\n\n```bash\n$ docker run -v /path/to/cache/dir:/data \\\n\t-v /path/to/certificate_authority:/etc/bazel-remote/ca_cert \\\n\t-v /path/to/server_cert:/etc/bazel-remote/server_cert \\\n\t-v /path/to/server_key:/etc/bazel-remote/server_key \\\n\t-p 9090:8080 -p 9092:9092 buchgr/bazel-remote-cache \\\n\t--tls_ca_file=/etc/bazel-remote/ca_cert \\\n\t--tls_cert_file=/etc/bazel-remote/server_cert \\\n\t--tls_key_file=/etc/bazel-remote/server_key \\\n\t--max_size 5\n```\n\n#### LDAP\n\nThere is also an experimental LDAP authentication method. A configuration\nfile is advised to avoid leaking the ldap.bind_password value to local\nusers, but command line arguments are also supported.\n\nNote that the configuration options for this feature might change while\nthe feature is still considered \"experimental\".\n\n```bash\n$ docker run -v /path/to/cache/dir:/data \\\n   -p 9090:8080 -p 9092:9092 buchgr/bazel-remote-cache \\\n   --ldap.url=\"ldaps://ldap.example.com:636\" \\\n   --ldap.base_dn=\"OU=My Users,DC=example,DC=com\" \\\n   --ldap.groups_query=\"(|(memberOf=CN=bazel-users,OU=Groups,OU=My Users,DC=example,DC=com)(memberOf=CN=other-users,OU=Groups2,OU=Alien Users,DC=foo,DC=org))\" \\\n   --ldap.cache_time=100 \\\n   --ldap.bind_user=\"cn=readonly.username,ou=readonly,OU=Other Users,DC=example,DC=com\" \\\n   --ldap.bind_password=\"secret4Sure\" \\\n   --max_size 5\n```\n\n### Using bazel-remote with AWS Credential file authentication for S3 inside a docker container\n\nThe following demonstrates how to configure a docker instance of bazel-remote to use an AWS S3\nbackend, authenticating using the `supercool` profile from your `$HOME/.aws/credentials` file.\n\n```bash\n$ docker run -u 1000:1000 -v /path/to/cache/dir:/data -v $HOME/.aws:/aws-config \\\n   -p 9090:8080 -p 9092:9092 buchgr/bazel-remote-cache \\\n   --s3.auth_method=aws_credentials_file --s3.aws_profile=supercool \\\n   --s3.aws_shared_credentials_file=/aws-config/credentials \\\n   --s3.bucket=my-bucket --s3.endpoint=s3.us-east-1.amazonaws.com \\\n   --max_size 5\n```\n\nNote that if you use the `--s3.auth_method=iam_role` flag with docker, then in\norder to make the S3 host instance metadata service (located at 169.254.169.254)\nreachable, then you may need to use the docker flag `--network=host`.\n\n### Profiling\n\nTo enable pprof profiling, specify an address to listen to with\n`--profile_address`.\n\nIf running inside docker, you will need to use a profile_address value\nwith a host other than `127.0.0.1` and add a `-p` mapping to the docker\nrun commandline for the port.\n\nSee [Profiling Go programs with pprof](https://jvns.ca/blog/2017/09/24/profiling-go-with-pprof/)\nfor more details.\n\n## Configuring Bazel\n\nTo make bazel use remote cache, use the following flag:\n`--remote_cache=http://replace-with-your.host:port`. You can also use the\nfollowing protocols instead of http: https, grpc or grpcs (depending on your\nbazel-remote configuration).\n\nBasic username/password authentication can be added like so:\n\n`--remote_cache=http://user:pass@replace-with-your.host:port`\n\nTo avoid leaking your password in log files, you can place this flag in a\n[user-specific (and .gitignore'd) bazelrc file](https://docs.bazel.build/versions/master/best-practices.html#bazelrc).\n\nTo use mutual TLS with bazel, use a `grpcs` URL for the `--remote_cache`\nargument, and add the following flags:\n```bash\n\t--tls_certificate=path/to/ca.cert\n\t--tls_client_certificate=path/to/client/cert.cert\n\t--tls_client_key=path/to/client/cert.key\n```\n\nFor more details, see Bazel's [remote\ncaching](https://docs.bazel.build/versions/master/remote-caching.html#run-bazel-using-the-remote-cache)\ndocumentation.\n","funding_links":[],"categories":["Go","Tooling"],"sub_categories":["Remote caching and execution"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuchgr%2Fbazel-remote","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuchgr%2Fbazel-remote","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuchgr%2Fbazel-remote/lists"}