Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/apache/opendal
Apache OpenDAL: access data freely.
https://github.com/apache/opendal
azblob gcs hacktoberfest redis rust s3 storage
Last synced: 3 months ago
JSON representation
Apache OpenDAL: access data freely.
- Host: GitHub
- URL: https://github.com/apache/opendal
- Owner: apache
- License: apache-2.0
- Created: 2022-02-14T05:18:44.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-22T16:34:32.000Z (3 months ago)
- Last Synced: 2024-03-22T22:14:09.748Z (3 months ago)
- Topics: azblob, gcs, hacktoberfest, redis, rust, s3, storage
- Language: Rust
- Homepage: https://opendal.apache.org
- Size: 201 MB
- Stars: 2,765
- Watchers: 29
- Forks: 383
- Open Issues: 166
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Citation: CITATION.cff
- Codeowners: .github/CODEOWNERS
Lists
- awesome-stars - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- awesome-stars - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- awesome-rust-cloud-native - datafuselabs/opendal - in (Uncategorized / Uncategorized)
- awesome-stars - opendal
- awesome-stars - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- awesome-stars - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- stars - apache/opendal
- awesome-stars - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- awesome-ccamel - apache/opendal - Apache OpenDAL: access data freely. (Rust)
- awesome-stars - apache/opendal - `★2958` Apache OpenDAL: access data freely. (Rust)
- my-awesome - apache/opendal - 06 star:3.0k fork:0.4k Apache OpenDAL: access data freely. (Rust)
README
# Apache OpenDAL™: *Access Data Freely*
[](mailto:[email protected])
[](https://discord.gg/XQy8yGR2dg)
OpenDAL offers a unified data access layer, empowering users to seamlessly and efficiently retrieve data from diverse storage services. Our goal is to deliver a comprehensive solution for any languages, methods, integrations, and services.
## For *ANY* languages
| Name | Release |
|-----------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Rust Core](core/README.md) | [](https://crates.io/crates/opendal) |
| [C Binding](bindings/c/README.md) | - |
| [Cpp Binding](bindings/cpp/README.md) | - |
| [Dotnet Binding](bindings/dotnet/README.md) | - |
| [Go Binding](bindings/go/README.md) | - |
| [Haskell Binding](bindings/haskell/README.md) | - |
| [Java Binding](bindings/java/README.md) | [](https://central.sonatype.com/artifact/org.apache.opendal/opendal-java) |
| [Lua Binding](bindings/lua/README.md) | - |
| [Node.js Binding](bindings/nodejs/README.md) | [](https://www.npmjs.com/package/opendal) |
| [Ocaml Binding](bindings/ocaml/README.md) | - |
| [PHP Binding](bindings/php/README.md) | - |
| [Python Binding](bindings/python/README.md) | [](https://pypi.org/project/opendal/) |
| [Ruby Binding](bindings/ruby/README.md) | - |
| [Swift Binding](bindings/swift/README.md) | - |
| [Zig Binding](bindings/zig/README.md) | - |
## For *ANY* methods
| Name | Description | Release |
|--------------------------|--------------------------------------------------------------------|------------------------------------------------------------------------------|
| [oay](bin/oay/README.md) | Access data via API Gateway | [](https://crates.io/crates/oay) |
| [oli](bin/oli/README.md) | Access data via Command Line (alternative to s3cmd, s3cli, azcopy) | [](https://crates.io/crates/oli) |
| [ofs](bin/ofs/README.md) | Access data via POSIX file system API (alternative to s3fs) | [](https://crates.io/crates/ofs) |
## For *ANY* integrations
| Name | Description | Release |
|---------------------------------------------------------------------|--------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|
| [dav-server-opendalfs](integrations/dav-server/README.md) | Access data via integrations to [dav-server-rs](https://github.com/messense/dav-server-rs) | [](https://crates.io/crates/dav-server-opendalfs) |
| [object_store_opendal](integrations/object_store/README.md) | Access data via integrations to [object_store](https://docs.rs/object_store) | [](https://crates.io/crates/object_store_opendal) |
## For *ANY* services
| Type | Services |
|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Standard Storage Protocols | ftp, http, [sftp](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02), [webdav](https://datatracker.ietf.org/doc/html/rfc4918), .. |
| Object Storage Services | [azblob](https://azure.microsoft.com/en-us/services/storage/blobs/), [cos](https://www.tencentcloud.com/products/cos), [gcs](https://cloud.google.com/storage), [obs](https://www.huaweicloud.com/intl/en-us/product/obs.html), [oss](https://www.aliyun.com/product/oss), [s3](https://aws.amazon.com/s3/), [supabase](https://supabase.com/docs/guides/storage), [b2](https://www.backblaze.com/), [openstack swift](https://docs.openstack.org/swift/latest/), [upyun](https://www.upyun.com/), [vercel_blob](https://vercel.com/docs/storage/vercel-blob), .. |
| File Storage Services | fs, [alluxio](https://docs.alluxio.io/os/user/stable/en/api/REST-API.html), [azdls](https://azure.microsoft.com/en-us/products/storage/data-lake-storage/), [azfile](https://learn.microsoft.com/en-us/rest/api/storageservices/file-service-rest-api), [chainsafe](https://storage.chainsafe.io/), [dbfs](https://docs.databricks.com/en/dbfs/index.html), [gridfs](https://www.mongodb.com/docs/manual/core/gridfs/), [hdfs](https://hadoop.apache.org/docs/r3.3.4/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html), [ipfs](https://ipfs.tech/), [webhdfs](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/WebHDFS.html), .. |
| Consumer Cloud Storage Service | [gdrive](https://www.google.com/drive/), [onedrive](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage), [dropbox](https://www.dropbox.com/), [icloud](https://www.icloud.com/iclouddrive), [koofr](https://koofr.eu/), [pcloud](https://www.pcloud.com/), [seafile](https://www.seafile.com/), [yandex_disk](https://360.yandex.com/disk/), .. |
| Key-Value Storage Services | [cacache](https://crates.io/crates/cacache), [cloudflare_kv](https://developers.cloudflare.com/kv/), [dashmap](https://github.com/xacrimon/dashmap), memory, [etcd](https://etcd.io/), [foundationdb](https://www.foundationdb.org/), [persy](https://crates.io/crates/persy), [redis](https://redis.io/), [rocksdb](http://rocksdb.org/), [sled](https://crates.io/crates/sled), [redb](https://crates.io/crates/redb), [tikv](https://tikv.org/), [atomicserver](https://github.com/atomicdata-dev/atomic-server), .. |
| Database Storage Services | [d1](https://developers.cloudflare.com/d1/), [libsql](https://github.com/tursodatabase/libsql), [mongodb](https://www.mongodb.com/), [mysql](https://www.mysql.com/), [postgresql](https://www.postgresql.org/), [sqlite](https://www.sqlite.org/), .. |
| Cache Storage Services | [ghac](https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows), [memcached](https://memcached.org/), [mini_moka](https://github.com/moka-rs/mini-moka), [moka](https://github.com/moka-rs/moka), [vercel_artifacts](https://vercel.com/docs/concepts/monorepos/remote-caching), .. |
| Git Based Storage Services | [huggingface](https://huggingface.co/), .. |
> Welcome to add any services that are not currently supported [here](https://github.com/apache/opendal/issues/5).
## Examples
The examples are available at [here](./examples/).
## Documentation
The documentation is available at .
## Contribute
OpenDAL is an active open-source project. We are always open to people who want to use it or contribute to it. Here are some ways to go.
- Start with [Contributing Guide](CONTRIBUTING.md).
- Submit [Issues](https://github.com/apache/opendal/issues/new) for bug report or feature requests.
- Discuss at [dev mailing list](mailto:[email protected]) ([subscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20subscribe)) / [unsubscribe](mailto:[email protected]?subject=(send%20this%20email%20to%20unsubscribe)) / [archives](https://lists.apache.org/[email protected]))
- Asking questions in the [Discussions](https://github.com/apache/opendal/discussions/new?category=q-a).
- Talk to community directly at [Discord](https://discord.gg/XQy8yGR2dg).
## Who is using OpenDAL?
### Rust Core
- [apache/iceberg-rust](https://github.com/apache/iceberg-rust/): Native Rust implementation of [Apache Iceberg](https://iceberg.apache.org/), the open table format for analytic datasets.
- [Databend](https://github.com/datafuselabs/databend/): A modern Elasticity and Performance cloud data warehouse.
- [deepeth/mars](https://github.com/deepeth/mars): The powerful analysis platform to explore and visualize data from blockchain.
- [GreptimeDB](https://github.com/GreptimeTeam/greptimedb): An open-source, cloud-native, distributed time-series database.
- [mozilla/sccache](https://github.com/mozilla/sccache/): `sccache` is [`ccache`](https://github.com/ccache/ccache) with cloud storage
- [OctoBase](https://github.com/toeverything/OctoBase): the open-source database behind [AFFiNE](https://github.com/toeverything/affine), local-first, yet collaborative.
- [Pants](https://github.com/pantsbuild/pants): A fast, scalable, user-friendly build system for codebases of all sizes.
- [QuestDB](https://github.com/questdb/questdb): An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.
- [RisingWave](https://github.com/risingwavelabs/risingwave): A Distributed SQL Database for Stream Processing
- [Vector](https://github.com/vectordotdev/vector): A high-performance observability data pipeline.
### C Binding
- [Milvus](https://github.com/milvus-io/milvus): A cloud-native vector database, storage for next generation AI applications
### Java Binding
- [QuestDB](https://github.com/questdb/questdb): An open-source time-series database for high throughput ingestion and fast SQL queries with operational simplicity.
## Branding
The first and most prominent mentions must use the full form: **Apache OpenDAL™** of the name for any individual usage (webpage, handout, slides, etc.) Depending on the context and writing style, you should use the full form of the name sufficiently often to ensure that readers clearly understand the association of both the OpenDAL project and the OpenDAL software product to the ASF as the parent organization.
For more details, see the [Apache Product Name Usage Guide](https://www.apache.org/foundation/marks/guide).
## License and Trademarks
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.