{"id":13467054,"url":"https://github.com/Alluxio/alluxio","last_synced_at":"2025-03-26T00:31:50.898Z","repository":{"id":6051868,"uuid":"7276954","full_name":"Alluxio/alluxio","owner":"Alluxio","description":"Alluxio, data orchestration for analytics and machine learning in the cloud","archived":false,"fork":false,"pushed_at":"2025-02-18T19:15:58.000Z","size":205627,"stargazers_count":6962,"open_issues_count":1032,"forks_count":2945,"subscribers_count":439,"default_branch":"main","last_synced_at":"2025-03-24T22:20:15.151Z","etag":null,"topics":["alluxio","data-analysis","data-orchestration","hadoop","memory-speed","presto","spark","tensorflow","virtual-distributed-filesystem"],"latest_commit_sha":null,"homepage":"https://www.alluxio.io","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"jmlawrie/hotbox","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Alluxio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2012-12-21T17:43:46.000Z","updated_at":"2025-03-24T10:13:55.000Z","dependencies_parsed_at":"2023-10-15T12:00:11.922Z","dependency_job_id":"374319c0-16f1-4394-a3cd-f1dcefc7cd99","html_url":"https://github.com/Alluxio/alluxio","commit_stats":{"total_commits":27062,"total_committers":1457,"mean_commits":"18.573781743308167","dds":0.90950410169241,"last_synced_commit":"db91042a7d6cc3a15b28d4cb4919e2c140a44caa"},"previous_names":["amplab/tachyon"],"tags_count":160,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alluxio%2Falluxio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alluxio%2Falluxio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alluxio%2Falluxio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Alluxio%2Falluxio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Alluxio","download_url":"https://codeload.github.com/Alluxio/alluxio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245566099,"owners_count":20636391,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alluxio","data-analysis","data-orchestration","hadoop","memory-speed","presto","spark","tensorflow","virtual-distributed-filesystem"],"created_at":"2024-07-31T15:00:52.626Z","updated_at":"2025-03-26T00:31:50.889Z","avatar_url":"https://github.com/Alluxio.png","language":"Java","readme":"[![logo](docs/resources/alluxio_logo.png \"Alluxio\")](https://www.alluxio.io)\n\n[![Slack](https://img.shields.io/badge/slack-alluxio--community-blue.svg?logo=slack)](https://www.alluxio.io/slack)\n[![Release](https://img.shields.io/github/release/alluxio/alluxio/all.svg)](https://www.alluxio.io/download)\n[![Docker Pulls](https://img.shields.io/docker/pulls/alluxio/alluxio.svg)](https://hub.docker.com/r/alluxio/alluxio)\n[![Documentation](https://img.shields.io/badge/docs-reference-blue.svg)](https://www.alluxio.io/docs)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/Alluxio/alluxio/badge)](https://api.securityscorecards.dev/projects/github.com/Alluxio/alluxio)\n[![Twitter Follow](https://img.shields.io/twitter/follow/alluxio.svg?label=Follow\u0026style=social)](https://twitter.com/intent/follow?screen_name=alluxio)\n[![License](https://img.shields.io/github/license/alluxio/alluxio.svg)](https://github.com/Alluxio/alluxio/blob/master/LICENSE)\n\n## What is Alluxio\n[Alluxio](https://www.alluxio.io) (formerly known as Tachyon)\nis a virtual distributed storage system. It bridges the gap between\ncomputation frameworks and storage systems, enabling computation applications to connect to\nnumerous storage systems through a common interface. Read more about\n[Alluxio Overview](https://docs.alluxio.io/os/user/stable/en/Overview.html).\n\nThe Alluxio project originated from a research project called Tachyon at AMPLab, UC Berkeley,\nwhich was the data layer of the Berkeley Data Analytics Stack ([BDAS](https://amplab.cs.berkeley.edu/bdas/)).\nFor more details, please refer to Haoyuan Li's PhD dissertation\n[Alluxio: A Virtual Distributed File System](https://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-29.html).\n\n## Who Uses Alluxio\n\nAlluxio is used in production to manage Petabytes of data in many leading companies, with\nthe largest deployment exceeding 3,000 nodes. You can find more use cases at\n[Powered by Alluxio](https://www.alluxio.io/powered-by-alluxio) or visit our first community conference ([Data Orchestration Summit](https://www.alluxio.io/data-orchestration-summit-2019/)) to learn from other community members!\n\n## Who Owns and Manages Alluxio Project\n\nAlluxio Open Source Foundation is the owner of Alluxio project.\nProject operation is done by Alluxio Project Management Committee (PMC).\nYou can checkout more details in its structure and how to join Alluxio PMC \n[here](https://github.com/Alluxio/alluxio/wiki/Alluxio-Project-Management-Committee-(PMC)).\n\n## Community and Events\nPlease use the following to reach members of the community:\n\n* [Alluxio Community Slack Channel](https://www.alluxio.io/slack): post your questions here if you seek for help for general questions or issues using Alluxio.\n* [Special Interest Groups (SIG) for Alluxio users and developers](#contributing)\n* Community Events: [upcoming online office hours, meetups and webinars](https://www.alluxio.io/events)\n* Meetup Groups: [Global Online Meetup](https://www.meetup.com/Alluxio-Global-Online-Meetup/), [Bay Area Meetup](http://www.meetup.com/Alluxio),\n[New York Meetup](https://www.meetup.com/Alluxio-Open-Source-New-York-Meetup),\n[Beijing Alluxio Meetup](https://www.meetup.com/meetup-group-iLMBZGhS/), [Austin Meetup](https://www.meetup.com/Cloud-Data-Orchestration-Austin/)\n* [Alluxio Twitter](https://twitter.com/alluxio); [Alluxio Youtube Channel](https://www.youtube.com/channel/UCpibQsajhwqYPLYhke4RigA); [Alluxio Mailing List](https://groups.google.com/forum/?fromgroups#!forum/alluxio-users)\n\n## Download Alluxio\n\n### Binary download\n\nPrebuilt binaries are available to download at https://www.alluxio.io/download .\n\n### Docker\n\nDownload and start an Alluxio master and a worker. More details can be found in [documentation](https://docs.alluxio.io/os/user/stable/en/deploy/Running-Alluxio-On-Docker.html).\n\n```console\n# Create a network for connecting Alluxio containers\n$ docker network create alluxio_nw\n# Create a volume for storing ufs data\n$ docker volume create ufs\n# Launch the Alluxio master\n$ docker run -d --net=alluxio_nw \\\n    -p 19999:19999 \\\n    --name=alluxio-master \\\n    -v ufs:/opt/alluxio/underFSStorage \\\n    alluxio/alluxio master\n# Launch the Alluxio worker\n$ export ALLUXIO_WORKER_RAMDISK_SIZE=1G\n$ docker run -d --net=alluxio_nw \\\n    --shm-size=${ALLUXIO_WORKER_RAMDISK_SIZE} \\\n    --name=alluxio-worker \\\n    -v ufs:/opt/alluxio/underFSStorage \\\n    -e ALLUXIO_JAVA_OPTS=\"-Dalluxio.worker.ramdisk.size=${ALLUXIO_WORKER_RAMDISK_SIZE} -Dalluxio.master.hostname=alluxio-master\" \\\n    alluxio/alluxio worker\n```\n\n### MacOS Homebrew\n\n```console\n$ brew install alluxio\n```\n\n## Quick Start\n\nPlease follow the [Guide to Get Started](https://docs.alluxio.io/os/user/stable/en/Getting-Started.html)\nto run a simple example with Alluxio.\n\n## Report a Bug\n\nTo report bugs, suggest improvements, or create new feature requests, please open a [Github Issue](https://github.com/alluxio/alluxio/issues).\nIf you are not sure whether you run into bugs or simply have general questions with respect to Alluxio, post your questions on [Alluxio Slack channel](www.alluxio.io/slack).\n\n## Depend on Alluxio\n\nAlluxio project provides several different client artifacts for external projects to depend on Alluxio client:\n\n- Artifact `alluxio-shaded-client` is recommended generally for a project to use Alluxio client.\n  The jar of this artifact is self-contained (including all dependencies in a shaded form to prevent dependency conflicts),\n  and thus larger than the following two artifacts.\n- Artifact `alluxio-core-client-fs` provides\n  [Alluxio Java file system API](https://docs.alluxio.io/os/user/stable/en/api/Java-API.html#alluxio-java-api))\n  to access all Alluxio-specific functionalities.\n  This artifact is included in `alluxio-shaded-client`.\n- Artifact `alluxio-core-client-hdfs` provides\n  [HDFS-Compatible file system API](https://docs.alluxio.io/os/user/stable/en/api/Java-API.html#hadoop-compatible-java-client).\n  This artifact is included in `alluxio-shaded-client`.\n\nHere are examples to declare the dependecies on  `alluxio-shaded-client` using Maven:\n\n  ```xml\n  \u003cdependency\u003e\n    \u003cgroupId\u003eorg.alluxio\u003c/groupId\u003e\n    \u003cartifactId\u003ealluxio-shaded-client\u003c/artifactId\u003e\n    \u003cversion\u003e2.6.0\u003c/version\u003e\n  \u003c/dependency\u003e\n  ```\n\n## Contributing\n\nContributions via GitHub pull requests are gladly accepted from their original author. Along with\nany pull requests, please state that the contribution is your original work and that you license the\nwork to the project under the project's open source license. Whether or not you state this\nexplicitly, by submitting any copyrighted material via pull request, email, or other means you agree\nto license the material under the project's open source license and warrant that you have the legal\nauthority to do so.\nFor a more detailed step-by-step guide, please read\n[how to contribute to Alluxio](https://docs.alluxio.io/os/user/stable/en/contributor/Contributor-Getting-Started.html).\nFor new contributor, please take two [new contributor tasks](https://github.com/Alluxio/new-contributor-tasks).\n\nFor advanced feature requests and contributions, \nAlluxio core team is hosting regular online meetings with community users and developers to iterate the project in two special interest groups:\n\n* Alluxio and AI workloads: e.g., running Tensorflow, Pytorch on Alluxio through the POSIX API. Checkout the [meeting notes](https://docs.google.com/spreadsheets/d/1OlprIiUkGjMuZJ_6cLTJYVJpTGpnTWkFhHzX16tYNDQ/)\n* Alluxio and Presto workloads: e.g., running Presto on Alluxio. Checkout the [meeting notes](https://docs.google.com/spreadsheets/d/1V-fxqfG_oj3B1ZWSgbRWVuTHFvjL3pq6uXgAL-xvFQA/)\n\nSubscribe our [public calendar](https://calendar.google.com/calendar/embed?src=alluxio.com_g9ec8agk27baqu2nu692ft1m3s%40group.calendar.google.com\u0026ctz=America%2FLos_Angeles) to join us.\n\n## Useful Links\n\n- [Alluxio Website](https://www.alluxio.io/)\n- [Downloads](https://www.alluxio.io/download)\n- [Releases and Notes](https://www.alluxio.io/download/releases/)\n- [Documentation](https://www.alluxio.io/docs/)\n","funding_links":[],"categories":["HarmonyOS","Capabilities","Java","File System","Data Lake Storages","Data Storage Optimisation","II. Databases, search engines, big data and machine learning","人工智能","Repositories"],"sub_categories":["Windows Manager","Storage","Vector Database","1. Databases and storages"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlluxio%2Falluxio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAlluxio%2Falluxio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlluxio%2Falluxio/lists"}