{"id":13741567,"url":"https://github.com/apache/oozie","last_synced_at":"2025-03-30T16:02:12.324Z","repository":{"id":1657680,"uuid":"2383782","full_name":"apache/oozie","owner":"apache","description":"Mirror of Apache Oozie","archived":false,"fork":false,"pushed_at":"2025-01-27T08:38:13.000Z","size":32259,"stargazers_count":723,"open_issues_count":49,"forks_count":473,"subscribers_count":72,"default_branch":"master","last_synced_at":"2025-03-23T15:01:37.740Z","etag":null,"topics":["big-data","java","javascript","oozie"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-09-14T07:00:10.000Z","updated_at":"2025-03-11T19:57:04.000Z","dependencies_parsed_at":"2024-01-07T01:44:13.861Z","dependency_job_id":"cc28fbf8-081f-4aeb-96c1-add5d141710b","html_url":"https://github.com/apache/oozie","commit_stats":{"total_commits":2408,"total_committers":41,"mean_commits":58.73170731707317,"dds":0.8077242524916943,"last_synced_commit":"318fac5391eb1b7e9b868ee6fb64f4e9c49850cb"},"previous_names":[],"tags_count":51,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Foozie","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Foozie/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Foozie/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Foozie/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/oozie/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246270782,"owners_count":20750457,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["big-data","java","javascript","oozie"],"created_at":"2024-08-03T04:01:00.375Z","updated_at":"2025-03-30T16:02:12.236Z","avatar_url":"https://github.com/apache.png","language":"Java","readme":"Apache Oozie\n=============\n\nWhat is Oozie\n--------------\n\nOozie is an extensible, scalable and reliable system to define, manage, schedule, and execute complex Hadoop workloads via web services. More specifically, this includes:\n\n  * XML-based declarative framework to specify a job or a complex workflow of dependent jobs.\n  * Support different types of job such as Hadoop Map-Reduce, Pipe, Streaming, Pig, Hive and custom java applications.\n  * Workflow scheduling based on frequency and/or data availability.\n  * Monitoring capability, automatic retry and failure handing of jobs.\n  * Extensible and pluggable architecture to allow arbitrary grid programming paradigms.\n  * Authentication, authorization, and capacity-aware load throttling to allow multi-tenant software as a service.\n\nOozie Overview\n----------\n\nOozie is a server based Workflow Engine specialized in running workflow jobs with actions that run Hadoop Map/Reduce and Pig jobs.\n\nOozie is a Java Web-Application that runs in a Java servlet-container.\n\nFor the purposes of Oozie, a workflow is a collection of actions (i.e. Hadoop Map/Reduce jobs, Pig jobs) arranged in a control dependency DAG (Directed Acyclic Graph). \"control dependency\" from one action to another means that the second action can't run until the first action has completed.\n\nOozie workflows definitions are written in hPDL (a XML Process Definition Language similar to JBOSS JBPM jPDL).\n\nOozie workflow actions start jobs in remote systems (i.e. Hadoop, Pig). Upon action completion, the remote systems callback Oozie to notify the action completion, at this point Oozie proceeds to the next action in the workflow.\n\nOozie workflows contain control flow nodes and action nodes.\n\nControl flow nodes define the beginning and the end of a workflow ( start , end and fail nodes) and provide a mechanism to control the workflow execution path ( decision , fork and join nodes).\n\nAction nodes are the mechanism by which a workflow triggers the execution of a computation/processing task. Oozie provides support for different types of actions: Hadoop map-reduce, Hadoop file system, Pig, SSH, HTTP, eMail and Oozie sub-workflow. Oozie can be extended to support additional type of actions.\n\nOozie workflows can be parameterized (using variables like ${inputDir} within the workflow definition). When submitting a workflow job values for the parameters must be provided. If properly parameterized (i.e. using different output directories) several identical workflow jobs can concurrently.\n\nDocumentations :\n-----------------\nOozie web service is bundle with the built-in details documentation.\n\nMore information could be found at:\nhttp://oozie.apache.org/\n\nOozie Quick Start:\nhttp://oozie.apache.org/docs/5.0.0/DG_QuickStart.html\n\n\nSupported Hadoop Versions:\n----------------------------\n\nThis version of Oozie was primarily tested against Hadoop 2.4.x and 2.6.x.\n\n\n\nIf you have any questions/issues, please send an email to:\n\nuser@oozie.apache.org\n\nSubscribe using the link:\n\nhttp://oozie.apache.org/mail-lists.html\n","funding_links":[],"categories":["Data Pipeline","Data Workflow","大数据"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Foozie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Foozie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Foozie/lists"}