https://github.com/cubed-dev/cubed

Scalable array processing with bounded memory
https://github.com/cubed-dev/cubed

distributed-computing ndimensional-arrays numpy python serverless xarray zarr

Last synced: 4 days ago
JSON representation

Scalable array processing with bounded memory

• Host: GitHub
• URL: https://github.com/cubed-dev/cubed
• Owner: cubed-dev
• License: apache-2.0
• Created: 2022-05-05T08:04:04.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-12-06T15:49:19.000Z (about 1 month ago)
• Last Synced: 2024-12-06T19:11:22.556Z (about 1 month ago)
• Topics: distributed-computing, ndimensional-arrays, numpy, python, serverless, xarray, zarr
• Language: Python
• Homepage: https://cubed-dev.github.io/cubed/
• Size: 17.5 MB
• Stars: 122
• Watchers: 8
• Forks: 14
• Open Issues: 84
• Metadata Files:
  • Readme: README.md
  • Contributing: docs/contributing.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Cubed

## Scalable array processing with bounded memory

Cubed is a Python library for scalable out-of-core multi-dimensional array processing with bounded memory.

- Cubed provides NumPy and Xarray APIs for processing your multi-dimensional array data

- Cubed is a drop-in replacement for Dask's Array API

- Cubed will tell you if your computation would run out of memory *before* running it

- Cubed is designed to be robust to failures and will reliably complete a computation

- Cubed can process hundreds of GB of array data on your laptop using all available cores

- Cubed is horizontally scalable and stateless, and can scale to multi-TB datasets in the cloud

[Documentation](https://cubed-dev.github.io/cubed/)