{"id":20714679,"url":"https://github.com/fabsta/oozie_hive_file_trigger","last_synced_at":"2026-02-04T09:36:40.885Z","repository":{"id":142929739,"uuid":"55209866","full_name":"fabsta/oozie_hive_file_trigger","owner":"fabsta","description":"Simple example on how to trigger hive jobs when hdfs is updated","archived":false,"fork":false,"pushed_at":"2016-04-01T07:12:43.000Z","size":4,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-11T06:50:18.770Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fabsta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-01T06:45:09.000Z","updated_at":"2016-04-01T06:45:09.000Z","dependencies_parsed_at":"2023-05-19T00:15:51.793Z","dependency_job_id":null,"html_url":"https://github.com/fabsta/oozie_hive_file_trigger","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fabsta/oozie_hive_file_trigger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabsta%2Foozie_hive_file_trigger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabsta%2Foozie_hive_file_trigger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabsta%2Foozie_hive_file_trigger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabsta%2Foozie_hive_file_trigger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fabsta","download_url":"https://codeload.github.com/fabsta/oozie_hive_file_trigger/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fabsta%2Foozie_hive_file_trigger/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266424014,"owners_count":23926123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-22T02:00:09.085Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T02:33:29.434Z","updated_at":"2026-02-04T09:36:35.863Z","avatar_url":"https://github.com/fabsta.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Short tutorial on how to run oozie jobs\n\nRequirements:\noozie\n\n### become oozie user\nall actions will be executed as user oozie\n```sh\n$su - oozie\n```\n### define oozie server url\n```sh\n$export OOZIE_URL=http://192.168.100.141:11000/oozie\n```\n\n### go to project folder\n```sh\n$cd /home/oozie/examples/apps/oozie-hive.\n```\nyou will find the following files:\n- job.properties: namenode, jobtracker (please adjust these) as well as URI of workflow || coordinator app\n- coordinator.xml: infos about scheduling and file trigger\n- script.q: example hive script\n- workflow.xml: the coordinator will execute what is in the workflow.xml (script.q) once the file exists\n\n\n### copy files to hdfs\nall project files - except job.properties - are expected to be on hdfs. So we copy the files to hdfs\n```sh\n$hdfs dfs -put -f /home/oozie/examples/apps/oozie-hive/ /user/oozie/examples/apps/\n```\n\n### start job\n```sh\n$oozie job -config job.properties -run\n```\nreturns the job id. e.g. 0000000-160331163827649-oozie-oozi-C\n\nWhat happens under the hood:\n- oozie reads job.properties and the get the url of the wf (workflow) or coord (coordinator) application (on hdfs).\n- (the only file not in hdfs is the job.properties file. keep that in mind when editing files. Update the hdfs via\n(hdfs dfs -put -f examples/apps/oozie-hive/ /user/oozie/examples/apps/))\n- oozie starts the wf/coord job.\n\nOk, the job is now running.\n\n### check job log\nThere are two ways to check what your oozie jobs are doing: via command line and via web interface\n#### command line\n```sh\n$oozie job  -log 0000000-160331163827649-oozie-oozi-C\n```\n#### web interface\n```sh\noozie-UI http://172.17.97.141:11000/oozie/)\n```\nChecking the log wills tell you that the job is currently waiting for the following file: \n```sh\nhdfs://biginsights01:8020/user/oozie/examples/input-data/rawLogs/2016/03/31//_SUCCESS\n```\nSo, to start the job, we create the file with:\n```sh\n$hdfs dfs -put /home/oozie/_SUCCESS /user/oozie/examples/input-data/rawLogs/2016/03/31/\n```\n\nnow if you check the file log again it should tell that it has found the file and that the job is running.\n\n#### hive action + job verification\nthe example hive-script (script.q) is supposed to create a 'test' table in hive.\nTo verify that the workflow has worked, just do a \n```sh\nhive -e 'show tables'\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabsta%2Foozie_hive_file_trigger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffabsta%2Foozie_hive_file_trigger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffabsta%2Foozie_hive_file_trigger/lists"}