https://github.com/apache/ozone
Scalable, reliable, distributed storage system optimized for data analytics and object store workloads.
https://github.com/apache/ozone
big-data hadoop kubernetes object-store s3 storage
Last synced: 2 months ago
JSON representation
Scalable, reliable, distributed storage system optimized for data analytics and object store workloads.
- Host: GitHub
- URL: https://github.com/apache/ozone
- Owner: apache
- License: apache-2.0
- Created: 2019-10-02T15:56:19.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2025-04-10T06:51:07.000Z (2 months ago)
- Last Synced: 2025-04-10T07:43:02.750Z (2 months ago)
- Topics: big-data, hadoop, kubernetes, object-store, s3, storage
- Language: Java
- Homepage: https://ozone.apache.org
- Size: 97.6 MB
- Stars: 898
- Watchers: 69
- Forks: 522
- Open Issues: 123
-
Metadata Files:
- Readme: README.md
- Changelog: HISTORY.md
- Contributing: CONTRIBUTING.md
- License: LICENSE.txt
- Security: SECURITY.md
Awesome Lists containing this project
README
Apache Ozone
===
Ozone is a scalable, redundant, and distributed object store for Hadoop and Cloud-native environments. Apart from scaling to billions of objects of varying sizes, Ozone can function effectively in containerized environments such as Kubernetes and YARN.
* MULTI-PROTOCOL SUPPORT: Ozone supports different protocols like S3 and Hadoop File System APIs.
* SCALABLE: Ozone is designed to scale to tens of billions of files and blocks and, in the future, even more.
* CONSISTENT: Ozone is a strongly consistent object store. This consistency is achieved by using protocols like RAFT.
* CLOUD-NATIVE: Ozone is designed to work well in containerized environments like YARN and Kubernetes.
* SECURE: Ozone integrates with Kerberos infrastructure for authentication, supports native ACLs and integrates with Ranger for access control and supports TDE and on-wire encryption.
* HIGHLY AVAILABLE: Ozone is a fully replicated system that is designed to survive multiple failures.
## Documentation
The latest documentation is generated together with the releases and hosted on the apache site.
Please check [the documentation page](https://ozone.apache.org/docs/) for more information.
## Contact
Ozone is a top level project under the [Apache Software Foundation](https://apache.org)
* Ozone [web page](https://ozone.apache.org)
* Mailing lists
* For any questions use: [[email protected]](https://lists.apache.org/[email protected])
* Chat: There are a few ways to interact with the community
* You can find the #ozone channel on the official ASF Slack. Invite link is [here](http://s.apache.org/slack-invite).
* You can use [GitHub Discussions](https://github.com/apache/ozone/discussions) to post questions or follow community syncs.
* There are Open [Weekly calls](https://cwiki.apache.org/confluence/display/OZONE/Ozone+Community+Calls) where you can ask anything about Ozone.
* Past meeting notes are also available from the wiki.
* Reporting security issues: Please consult with [SECURITY.md](./SECURITY.md) about reporting security vulnerabilities and issues.
## Download
Latest release artifacts (source release and binary packages) are [available](https://ozone.apache.org/downloads/) from the Ozone web page.
## Quick start
### Run Ozone from published Docker image
The easiest way to start a cluster with docker is:
```
docker run -p 9878:9878 apache/ozone
```
And you can use AWS S3 cli:
```
aws s3api --endpoint http://localhost:9878/ create-bucket --bucket=wordcount
aws s3 --endpoint http://localhost:9878 cp --storage-class REDUCED_REDUNDANCY /tmp/testfile s3://wordcount/testfile
```
### Run Ozone from released artifact
If you need a more realistic cluster, you can [download](https://ozone.apache.org/downloads/) the latest (binary) release package, and start a cluster with the help of docker-compose:
After you untar the binary:
```
cd compose/ozone
docker-compose up -d --scale datanode=3
```
The `compose` folder contains different sets of configured clusters (secure, HA, mapreduce example), you can check the various subfolders for more examples.
### Run on Kubernetes
Ozone is a first class citizen of the Cloud-Native environments. The binary package contains multiple sets of K8s resource files to show how it can be deployed.
## Build from source
Ozone can be built with [Apache Maven](https://maven.apache.org):
```
mvn clean install -DskipTests
```
And can be started with the help of Docker:
```
cd hadoop-ozone/dist/target/ozone-*/compose/ozone
docker-compose up -d --scale datanode=3
```
For more information, you can check the [Contribution guideline](./CONTRIBUTING.md)
## Contribute
All contributions are welcome.
1. Please open a [Jira](https://issues.apache.org/jira/projects/HDDS/issues) issue
2. And create a pull request
For more information, you can check the [Contribution guideline](./CONTRIBUTING.md)
## License
The Apache Ozone project is licensed under the Apache 2.0 License. See the [LICENSE](./LICENSE.txt) file for details.