{"id":13738965,"url":"https://github.com/google/xarray-beam","last_synced_at":"2026-02-18T23:32:39.700Z","repository":{"id":39663272,"uuid":"366509361","full_name":"google/xarray-beam","owner":"google","description":"Distributed Xarray with Apache Beam","archived":false,"fork":false,"pushed_at":"2026-01-14T15:00:46.000Z","size":454,"stargazers_count":165,"open_issues_count":16,"forks_count":12,"subscribers_count":4,"default_branch":"main","last_synced_at":"2026-01-28T17:39:29.565Z","etag":null,"topics":["beam","dask","xarray","zarr"],"latest_commit_sha":null,"homepage":"https://xarray-beam.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-05-11T20:48:07.000Z","updated_at":"2026-01-14T15:00:48.000Z","dependencies_parsed_at":"2023-02-17T01:02:20.371Z","dependency_job_id":"70ec9371-9128-4a61-a406-f7f93a3ef0ab","html_url":"https://github.com/google/xarray-beam","commit_stats":{"total_commits":116,"total_committers":5,"mean_commits":23.2,"dds":0.5258620689655172,"last_synced_commit":"7f761b6dc132922031a5d824f233c4920322b6fe"},"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/google/xarray-beam","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fxarray-beam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fxarray-beam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fxarray-beam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fxarray-beam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google","download_url":"https://codeload.github.com/google/xarray-beam/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fxarray-beam/sbom","scorecard":{"id":159688,"data":{"date":"2025-08-11","repo":{"name":"github.com/google/xarray-beam","commit":"2ffeffdb89a89f0cc7b9238b96fadc7370b1d819"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.8,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/ci-build.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Maintained","score":10,"reason":"11 commit(s) and 3 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":5,"reason":"Found 11/19 approved changesets -- score normalized to 5","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci-build.yml:22: update your workflow using https://app.stepsecurity.io/secureworkflow/google/xarray-beam/ci-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/google/xarray-beam/ci-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yml:28: update your workflow using https://app.stepsecurity.io/secureworkflow/google/xarray-beam/ci-build.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci-build.yml:37: update your workflow using https://app.stepsecurity.io/secureworkflow/google/xarray-beam/ci-build.yml/main?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/ci-build.yml:34","Warn: pipCommand not pinned by hash: .github/workflows/ci-build.yml:45","Info:   0 out of   3 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/google/.github/SECURITY.md:1","Info: Found linked content: github.com/google/.github/SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: github.com/google/.github/SECURITY.md:1","Info: Found text in security policy: github.com/google/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":6,"reason":"4 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-cpwx-vrp4-4pq7","Warn: Project is vulnerable to: GHSA-gmj6-6f8f-6699","Warn: Project is vulnerable to: GHSA-h75v-3vvj-5mfj","Warn: Project is vulnerable to: GHSA-q2x7-8rv6-6q7h"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":2,"reason":"SAST tool is not run on all commits -- score normalized to 2","details":["Warn: 5 commits out of 21 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T12:59:17.233Z","repository_id":39663272,"created_at":"2025-08-16T12:59:17.234Z","updated_at":"2025-08-16T12:59:17.234Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29598246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T22:25:43.180Z","status":"ssl_error","status_checked_at":"2026-02-18T22:25:42.766Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beam","dask","xarray","zarr"],"created_at":"2024-08-03T04:00:21.896Z","updated_at":"2026-02-18T23:32:39.654Z","avatar_url":"https://github.com/google.png","language":"Python","readme":"# Xarray-Beam\n\nXarray-Beam is a Python library for building\n[Apache Beam](https://beam.apache.org/) pipelines with\n[Xarray](http://xarray.pydata.org/en/stable/) datasets.\n\nThe project aims to facilitate data transformations and analysis on large-scale\nmulti-dimensional labeled arrays, such as:\n\n-   Ad-hoc computation on Xarray data, by dividing a `xarray.Dataset` into many\n    smaller pieces (\"chunks\").\n-   Adjusting array chunks, using the\n    [Rechunker algorithm](https://rechunker.readthedocs.io/en/latest/algorithm.html).\n-   Ingesting large, multi-dimensional array datasets into an analysis-ready,\n    cloud-optimized format, namely [Zarr](https://zarr.readthedocs.io/) (see\n    also [Pangeo Forge](https://github.com/pangeo-forge/pangeo-forge-recipes)).\n-   Calculating statistics (e.g., \"climatology\") across distributed datasets\n    with arbitrary groups.\n\nFor more about our approach and how to get started,\n**[read the documentation](https://xarray-beam.readthedocs.io/)**!\n\n**Warning: Xarray-Beam is a sharp tool 🔪**\n\nXarray-Beam is relatively new, and focused on expert users:\n\n-   We use it extensively at Google for processing large-scale weather datasets,\n    but there is not yet a vibrant external community.\n-   It provides low-level abstractions that facilitate writing very large\n    scale data pipelines (e.g., 100+ TB), but by design it requires explicitly\n    thinking about how every operation is parallelized.\n\n## Installation\n\nXarray-Beam requires recent versions of immutabledict, Xarray, Dask, Rechunker,\nZarr, and Apache Beam. For best performance when writing Zarr files, use Xarray\n0.19.0 or later.\n\n## Disclaimer\n\nXarray-Beam is an experiment that we are sharing with the outside world in the\nhope that it will be useful. It is not a supported Google product. We welcome\nfeedback, bug reports and code contributions, but cannot guarantee they will be\naddressed.\n\nSee the \"Contribution guidelines\" for more.\n\n## Credits\n\nContributors:\n\n-   Stephan Hoyer\n-   Jason Hickey\n-   Cenk Gazen\n-   Alex Merose\n","funding_links":[],"categories":["Tools"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fxarray-beam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle%2Fxarray-beam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fxarray-beam/lists"}