{"id":18533086,"url":"https://github.com/googleapis/storage-testbench","last_synced_at":"2025-04-09T14:31:55.312Z","repository":{"id":36989046,"uuid":"374765810","full_name":"googleapis/storage-testbench","owner":"googleapis","description":"A testbench for Google Cloud Storage client libraries.","archived":false,"fork":false,"pushed_at":"2025-03-06T23:39:57.000Z","size":1068,"stargazers_count":10,"open_issues_count":39,"forks_count":22,"subscribers_count":39,"default_branch":"main","last_synced_at":"2025-03-24T07:57:01.580Z","etag":null,"topics":["testing"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/googleapis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-07T18:36:47.000Z","updated_at":"2025-03-06T23:40:01.000Z","dependencies_parsed_at":"2023-11-21T22:25:22.961Z","dependency_job_id":"313317f6-6df6-425b-8cde-6b886394653d","html_url":"https://github.com/googleapis/storage-testbench","commit_stats":null,"previous_names":[],"tags_count":52,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleapis%2Fstorage-testbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleapis%2Fstorage-testbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleapis%2Fstorage-testbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleapis%2Fstorage-testbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/googleapis","download_url":"https://codeload.github.com/googleapis/storage-testbench/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055197,"owners_count":21040137,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["testing"],"created_at":"2024-11-06T19:09:12.163Z","updated_at":"2025-04-09T14:31:55.296Z","avatar_url":"https://github.com/googleapis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Storage Testbench\n\n**This is not an officially supported Google product**\n\nThis repository is used by Storage Client libraries to test integration tests locally\nand reproduce Storage API transient errors. The testbench emulates the Storage API and\nis expected to be used by Storage library maintainers.\n\n\n## Table of Contents\n- [Storage Testbench](#storage-testbench)\n  - [Table of Contents](#table-of-contents)\n  - [Issue Policy](#issue-policy)\n  - [What is this testbench?](#what-is-this-testbench)\n  - [When to use this testbench](#when-to-use-this-testbench)\n  - [How to use this testbench](#how-to-use-this-testbench)\n    - [Initial set up](#initial-set-up)\n    - [Run the testbench](#run-the-testbench)\n    - [Start the gRPC server](#start-the-gRPC-server)\n    - [Check that the testbench is running](#check-that-the-testbench-is-running)\n  - [Updating Proto Files](#updating-proto-files)\n  - [Force Failures](#force-failures)\n    - [return-broken-stream](#return-broken-stream)\n    - [return-corrupted-data](#return-corrupted-data)\n    - [stall-always](#stall-always)\n    - [stall-at-256KiB](#stall-at-256kib)\n    - [return-503-after-256K](#return-503-after-256k)\n    - [return-503-after-256K/retry-N](#return-503-after-256kretry-n)\n    - [redirect-send-token-T](#redirect-send-token-t)\n    - [redirect-send-handle-and-token-T](#redirect-send-handle-and-token-t)\n    - [redirect-expect-token-T](#redirect-expect-token-t)\n  - [Retry Test API](#retry-test-api)\n    - [Creating a new Retry Test](#creating-a-new-retry-test)\n    - [Get a Retry Test resource](#get-a-retry-test-resource)\n    - [Delete a Retry Test resource](#delete-a-retry-test-resource)\n    - [Causing a failure using x-retry-test-id header](#causing-a-failure-using-x-retry-test-id-header)\n    - [Forced Failures Supported](#forced-failures-supported)\n  - [Releasing the testbench](#releasing-the-testbench)\n\n## Issue Policy\n\nRepository provides no dedicated support for issues filed.\nIssues will be addressed when time permits.\n\n## What is this testbench?\n\nThis testbench fakes the Google Cloud Storage (GCS) API. You can configure the GCS client libraries to make calls to this fake rather than to the actual API.\n* The testbench fakes the JSON API, both over REST and gRPC. It has limited support for the XML API.\n* Generally, the error codes are similar to the ones generated by GCS, but the error messages are not.\n* The testbench performs far fewer error checks, and no permission checks (ACL/IAM).\n\n## When to use this testbench\n\nIn general, this testbench is best suited for integration tests that are hard (or just annoying) to reliably run against production. The primary example of this are errors that make the client library go through its retry path.\n\nThis testbench can be useful to test HMAC keys, which are really hard to test against production due to quota restrictions.\n\nIt is useful as well to test features that are not yet deployed to production: you can implement them in the testbench and then write the library code before production is \"ready\".\n\n## How to use this testbench\n\n### Initial set up\n\n1. [Set up python if you haven't already](https://cloud.google.com/python/docs/setup)\n2. [Clone this repository](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/cloning-a-repository-from-github/cloning-a-repository#cloning-a-repository)\n\n   From the terminal:\n   ```bash\n   git clone https://github.com/googleapis/storage-testbench.git\n   ```\n3. Switch to the cloned directory:\n   ```bash\n   cd storage-testbench\n    ```\n4. [Create a virtual environment](https://cloud.google.com/python/docs/setup#installing_and_using_virtualenv)\n    * keep this virtual environment active whenever you run the testbench\n5. Install dependencies:\n    ```bash\n    pip install -e .\n    ```\n\n### Run the testbench\n\nTo start the testbench, run this command from a terminal:\n\n On Non-Windows\n```bash\npython3 testbench_run.py localhost 9000 10\n```\n\nOn Windows\n```bash\npy testbench_run.py localhost 9000 10\n```\n\n\u003e ⚠️ Ensure that the virtual environment you created to install the dependencies is active.\n\n\n### Start the gRPC server\n\nIf you want to test the gRPC API, you must make a request which will start\nthe testbench's gRPC server. Also, the gRPC server must run on a different port\nthan the HTTP server. To serve gRPC requests on port 8888, run:\n\n```bash\ncurl -s --retry 5 --retry-max-time 40 \"http://localhost:9000/start_grpc?port=8888\"\n```\n\u003e ⚠️ The HTTP server must be kept running to start the gRPC server and to make calls to the testbench,\n\u003e  even after you start the gRPC server. Run this command in a different terminal window.\n\n### Check that the testbench is running\n\nEnsure the testbench is running by sending it a request from a different terminal, such as:\n\n```bash\ncurl -X GET localhost:9000\n```\n\nThe response you get should be: `OK`\n\nNow you can use the testbench (while it's running) with the client libraries.\n\n## Updating Proto Files\n\nFrom time to time you may need to update the files generated by protobuf and\ngRPC. To do so, clone the [protos](https://github.com/googleapis/googleapis) and\nrun the grpc_tools generator:\n\n```shell\ncd $HOME/storage-testbench\npip install --no-deps grpcio-tools\n./update-protos.sh $PWD/.googleapis\n```\n\nThen commit the files generated in `google/**`:\n\n```shell\ngit commit -m\"chore: update protos\" google\n```\n\n## Force Failures\n\nYou can force the following failures by using the `x-goog-emulator-instructions` header.\nThe `x-goog-testbench-instructions` header is deprecated, but supported for\nbackwards compatibility and provides the same functionality as\n`x-goog-emulator-instructions`, please change your code to use `x-goog-emulator-instructions` instead.\n\n### return-broken-stream\n\nSet request headers with `x-goog-emulator-instructions: return-broken-stream`.\nTestbench will fail after sending 1024*1024 bytes.\n\n### return-corrupted-data\n\nSet request headers with `x-goog-emulator-instructions: return-corrupted-data`.\nTestbench will return corrupted data.\n\n### stall-always\n\nSet request headers with `x-goog-emulator-instructions: stall-always`.\nTestbench will stall at the beginning.\n\n### stall-at-256KiB\n\nSet request headers with `x-goog-emulator-instructions: stall-at-256KiB`.\nTestbench will stall at 256KiB bytes.\n\n### return-503-after-256K\n\nSet request headers with `x-goog-emulator-instructions: return-503-after-256K`.\nTestbench will return a `HTTP 503` after sending 256KiB bytes.\n\n### return-503-after-256K/retry-N\n\nSet request headers with `x-goog-emulator-instructions: return-503-after-256K/retry-1` up to `x-goog-emulator-instructions: return-503-after-256K/retry-N`.\n\nFor N==1 and N==2 behave like `return-305-after-256K`, for `N\u003e=3` ignore the\nfailure instruction and return successfully. This is used to test failures during\nretry, the client cooperates by sending the retry counter in the failure\ninstructions.\n\n### redirect-send-token-T\n\nIn gRPC, set initial metadata with `x-goog-emulator-instructions: redirect-send-token-tokenval`.\nTestbench will fail redirectable RPCs with `routing_token` set to the value `T`\n(`tokenval` in the example).\n\n### redirect-send-handle-and-token-T\n\nIn gRPC, set initial metadata with `x-goog-emulator-instructions: redirect-send-handle-and-token-tokenval`.\nTestbench will fail redirectable RPCs with `routing_token` set to the value `T`\n(`tokenval` in the example) and a `write_handle` or `read_handle` depending on the method called( `storage.objects.insert` or `storage.objects.get` ).\n\n### redirect-expect-token-T\n\nIn gRPC, set initial metadata with `x-goog-emulator-instructions: redirect-expect-token-tokenval`.\nTestbench will consume the instruction if the RPC also specifies\n`x-goog-request-params: routing_token=T` (`routing_token=tokenval` in the\nexample).\n\nNote that `x-goog-request-params` supports multiple key-value pairs, encoded\nlike URL query parameters.\n\n\n## Retry Test API\n\nThe \"Retry Test API\" offers a mechanism to describe more complex retry scenarios\nwhile sending a single, constant header through all the HTTP requests from a\ntest program. Retry Test provides accounting of failures used to validate\nthe expected failures were experienced by the testbench and not accidentally missed.\n\nPrevious versions of the GCS testbench used a custom header in the RPC to\ncontrol the behavior of each RPC, for some test scenarios this required sending\ndifferent header with the first retry attempt vs. subsequent attempts. Producing\ndifferent headers in each attempt is not easy to implement with some client libraries.\n\nSending a constant header with all RPCs can be implemented across all client libraries,\nand to some degree decouples the test setup from the test execution.\n\n### Creating a new Retry Test\n\nThe following cURL request will create a Retry Test resource which emits a 503\nwhen a buckets list operation is received by the testbench HTTP server with the returned\nretry test ID.\n\nNote: A Retry Test resource can only apply to one transport, either `HTTP` or `GRPC`. It defaults\nto `HTTP` if not specified.\n\n```bash\ncurl -X POST \"http://localhost:9000/retry_test\" -H 'Content-Type: application/json' \\\n     -d '{\"instructions\":{\"storage.buckets.list\": [\"return-503\"]}, \"transport\": \"HTTP\"}'\n```\n\n### Get a Retry Test resource\n\nGet Retry Test resource by id \"1d05c20627844214a9ff7cbcf696317d\".\n\n```bash\ncurl -X GET \"http://localhost:9000/retry_test/1d05c20627844214a9ff7cbcf696317d\"\n```\n\n### Delete a Retry Test resource\n\nDelete Retry Test resource by id \"1d05c20627844214a9ff7cbcf696317d\".\n\n```bash\ncurl -X DELETE \"http://localhost:9000/retry_test/1d05c20627844214a9ff7cbcf696317d\"\n```\n\n### Causing a failure using x-retry-test-id header\n\nThe following cURL request will attempt to list buckets and the testbench will emit\na `503` error once in the HTTP server based on the Retry Test created above. Subsequent list buckets\noperations will succeed.\n\nIn this example, list buckets operations in the gRPC server will not be impacted since the Retry Test\ncreated above only applies to HTTP. Specify `\"transport\": \"GRPC\"` for a Retry Test Resource to apply\nto gRPC operations.\n\n```bash\ncurl -H \"x-retry-test-id: 1d05c20627844214a9ff7cbcf696317d\" \"http://localhost:9100/storage/v1/b?project=test\"\n```\n\n### Forced Failures Supported\n\n| Failure Id              | Description\n| ----------------------- | ---\n| return-X                                  | [HTTP] Testbench will fail with HTTP code provided for X, e.g. return-503 returns a 503  \u003cbr\u003e [GRPC] Testbench will fail with the equivalent gRPC error to the HTTP code provided for X (X currently supported in the gRPC context include `400`, `401`, `429`, `500`, `501`, `503`)\n| return-X-after-YK                         | [HTTP] Testbench will return X after YKiB of uploaded data \u003cbr\u003e [GRPC] Testbench will return the equivalent gRPC error to the HTTP code provided for X after YKiB of uploaded data (X currently supported in the gRPC context include `400`, `401`, `429`, `500`, `501`, `503`)\n| return-broken-stream-final-chunk-after-YB | [HTTP] Testbench will break connection on final chunk of a resumable upload after Y bytes\n| return-broken-stream                      | [HTTP] Testbench will fail after a few downloaded bytes \u003cbr\u003e [GRPC] Testbench will fail with `UNAVAILABLE` after a few downloaded bytes\n| return-broken-stream-after-YK             | [HTTP] Testbench will fail after YKiB of downloaded data \u003cbr\u003e [GRPC] Testbench will fail with `UNAVAILABLE` after YKiB of downloaded data\n| return-reset-connection                   | [HTTP] Testbench will fail with a reset connection \u003cbr\u003e [GRPC] Testbench will fail the RPC with `UNAVAILABLE`\n| stall-for-Ts-after-YK                     | [HTTP] Testbench will stall for T second after reading YKiB of downloaded/uploaded data, e.g. stall-for-10s-after-12K stalls after reading/writing 12KiB of data \u003cbr\u003e [GRPC] Not supported\n| redirect-send-token-T                     | [HTTP] Unsupported [GRPC] Testbench will fail the RPC with `ABORTED` and include appropriate redirection error details.\n| redirect-send-handle-and-token-T          | [HTTP] Unsupported [GRPC] Testbench will fail the RPC with `ABORTED` and include appropriate redirection error details.\n\n\n## Releasing the testbench\n\nThe repository currently uses [Github Tags](https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository#creating-a-release)\n\nSteps:\n1. Go to https://github.com/googleapis/storage-testbench/releases/new\n1. Create a new tag v0.x.x\n1. Title \"v0.x.x\"\n1. Click Generate release notes\n1. Make sure \"Set as the latest release\" is checked\n1. Click \"Publish Release\" to release","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleapis%2Fstorage-testbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogleapis%2Fstorage-testbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleapis%2Fstorage-testbench/lists"}