{"id":50385026,"url":"https://github.com/stackhpc/s3-active-storage-compliance-suite","last_synced_at":"2026-05-30T14:30:53.794Z","repository":{"id":61648690,"uuid":"553606290","full_name":"stackhpc/s3-active-storage-compliance-suite","owner":"stackhpc","description":"Unit tests and performance benchmarking tools for implementations of the s3-active-storage project","archived":false,"fork":false,"pushed_at":"2026-03-30T14:09:36.000Z","size":104,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-30T16:14:12.280Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stackhpc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-10-18T13:27:52.000Z","updated_at":"2026-03-30T14:10:04.000Z","dependencies_parsed_at":"2025-03-17T18:24:49.602Z","dependency_job_id":"037b6c48-a7ef-464b-b748-e18f60d1ab69","html_url":"https://github.com/stackhpc/s3-active-storage-compliance-suite","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/stackhpc/s3-active-storage-compliance-suite","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fs3-active-storage-compliance-suite","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fs3-active-storage-compliance-suite/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fs3-active-storage-compliance-suite/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fs3-active-storage-compliance-suite/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stackhpc","download_url":"https://codeload.github.com/stackhpc/s3-active-storage-compliance-suite/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackhpc%2Fs3-active-storage-compliance-suite/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33696681,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-30T02:00:06.278Z","response_time":92,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-05-30T14:30:53.088Z","updated_at":"2026-05-30T14:30:53.788Z","avatar_url":"https://github.com/stackhpc.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# S3 active storage compliance suite\n\nIntegration tests and performance benchmarking tools for implementations of S3 Active Storage servers.\n\nThere are currently two S3 active storage server implementations:\n\n* [Reductionist](https://github.com/stackhpc/reductionist-rs) - a performant server implementation\n* [S3 Active Storage Prototype](https://github.com/stackhpc/s3-active-storage-prototype) - a functional prototype\n\n## Compliance Suite Usage\n\nTo set up the testing environment run:\n\n```\npython -m venv venv\nsource ./venv/bin/activate\npip install -r requirements.txt\n```\n\nTo run the compliance test suite on your own implementation of an S3 active storage server, edit the following variables in `compliance/config.py`:\n\n- `S3_SOURCE` - The address of your S3 store (e.g. `https://s3-proxy.com/`). If you don't have an existing S3 store you can set up a temporary minio docker container by running `scripts/run-minio.sh` in a separate terminal, in which case you should leave the S3 source as localhost.\n  \n- `AWS_ID` - The AWS access key ID used to authenticate with the S3 source (defaults to 'minioadmin' for the minio docker container).\n\n- `AWS_PASSWORD` - The secret access key used to authenticate with the S3 source (defaults to 'minioadmin' for the minio docker container)\n\n- `PROXY_URL` - The address for your active storage proxy implementation (e.g. `https://s3-proxy.example.com/`)\n\n- `PROXY_CA_CERT` - Optional path to a CA certificate for your active storage proxy.\n\nThe compliance test suite can then be run by calling \n```\npytest\n```\nfrom within the project directory.\n\n### Testing older active storage servers\n\nWe aim to add tests for features as they are added to the S3 active storage server.\nThis does lead to problems when testing older versions of the server that may lack support for those features.\nThis is addressed through configuration variables in `compliance/config.py`.\n\n- `TEST_X_ACTIVESTORAGE_COUNT_HEADER` - Whether to test for the presence of the `x-activestorage-count` header in responses.\n- `COMPRESSION_ALGS` - List of names of compression algorithms to test. May be set to an empty list.\n- `FILTER_ALGS` - List of names of filter algorithms to test. May be set to an empty list.\n- `MISSING_DATA` - List of missing data description classes in `compliance/missing.py`. May be set to an empty list.\n- `TEST_BYTE_ORDER` - Whether to test data with different byte orders (endianness).\n- `TEST_PUBLIC_BUCKET` - Whether to test unauthenticated access to data stored in a public bucket.\n\n### Testing newer active storage servers\n\nAs work on the [Reductionist](https://github.com/stackhpc/reductionist-rs) continues, and the compliance suite evolves in parallel, we continue to allow progressive testing of new features with the following configuration variables in `compliance/config.py`.\n\n- `TEST_API_V2` - Whether to switch to `v2` of the `Reductionist` API where different object store backends are available\n- `TEST_HTTP_OBJECT_STORE` - Whether to use the newer `Reductionist` API to test a HTTP object store backend\n- `HTTP_SOURCE` - The URL of a HTTP object store\n- `HTTP_USERNAME` - Username for HTTP object store protected with [HTTP basic access authentication](https://en.wikipedia.org/wiki/Basic_access_authentication)\n- `HTTP_PASSWORD` - Password for authenticated HTTP object store\n\nTesting of the HTTP object storage backend is an optional addition to the compliance suite.\nS3 object storage is still primary and the accompanying scripts in the suite can start both a local [MiniO](https://www.min.io/) S3 object store and [nginx](https://nginx.org/) HTTP object store, populating both with test data before sending separate API requests to Reductionist to ensuire its reductions using both object stores yield the same result.\n\n### Implementation details\n\nTest data is currently generated as numpy arrays and then uploaded to the configured S3 source in binary format. Following this upload, requests are made to the active storage proxy and the proxy response is compared to the expected result based on the agreed API specification and the generated test arrays.\n\nThere are procedurally generated test cases to cover various combinations of reduction operation, data type, data shape and data slice parameters as well as testing of response codes and error messages informative error messages.\n\n\n\n## Performance Benchmarking\n\nThe `scripts/run-benchmarks.py` file can be used to benchmark running instances of the active storage proxy. To do so, edit the required config parameters at the top of the script (`PROXY_URLS`, `S3_SOURCE` \u0026 `AUTH`) and then run the script. \n\nThe benchmarking process will generate a few numpy arrays then upload them to a test bucket within the configured S3 source. The `sum` operation is then called repeatedly on each of these arrays to collect some timing statistics. For convenience, the script will also collect timing stats while *bypassing* the active storage proxy (i.e. fetching the full array from S3 and then performing the sum locally) to indicate how much of an improvement the active storage proxy is providing. Once all benchmark runs are finished, a boxplot figure will be generated (at the path `./benchmark--{timestamp}.png`) to summarize the benchmark results. Finally, all generated test data is removed from the S3 source.\n\n## Contributing\n\nThe following checks are run in CI (from the repository root):\n\n```\nblack .\nmypy compliance\nflake8\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackhpc%2Fs3-active-storage-compliance-suite","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstackhpc%2Fs3-active-storage-compliance-suite","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackhpc%2Fs3-active-storage-compliance-suite/lists"}