{"id":18104886,"url":"https://github.com/springmt/apache-beam-gradle-sample","last_synced_at":"2025-04-06T06:18:32.590Z","repository":{"id":66034571,"uuid":"339647185","full_name":"SpringMT/apache-beam-gradle-sample","owner":"SpringMT","description":null,"archived":false,"fork":false,"pushed_at":"2021-02-17T18:09:53.000Z","size":66,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-12T11:52:27.511Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SpringMT.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-17T07:46:38.000Z","updated_at":"2021-02-17T18:09:56.000Z","dependencies_parsed_at":"2023-03-14T06:15:13.122Z","dependency_job_id":null,"html_url":"https://github.com/SpringMT/apache-beam-gradle-sample","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fapache-beam-gradle-sample","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fapache-beam-gradle-sample/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fapache-beam-gradle-sample/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SpringMT%2Fapache-beam-gradle-sample/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SpringMT","download_url":"https://codeload.github.com/SpringMT/apache-beam-gradle-sample/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247441221,"owners_count":20939266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-31T22:16:14.970Z","updated_at":"2025-04-06T06:18:32.585Z","avatar_url":"https://github.com/SpringMT.png","language":"Java","readme":"# apache beam gradle sample\n\n## Dataflowとは\nApache Beam がサポートする分散処理バックエンドのGCP版\n\nデータの並列処理パイプラインを構築するのがとても楽\n\nhttps://beam.apache.org/\n\n## Apache Beam\n### Apache Beamの概要\n* https://beam.apache.org/documentation/programming-guide/\n\n### SDKの選定\nJavaを使います。\n\n他にGoやpythonもありますが、型があること、Javaが優先されてサポートされている雰囲気からJavaにしています。\n\nGoは最近サポートされ始めたこともありなにか踏みそうな予感もしているので避けています\n\n\n## Deploy to Dataflow\n\n### Create Pipeline\n```\n./gradlew run \\\n  -Penv=foo \\\n  -PotherProject=bar \\\n  -Pgcp.project=baz \\\n  -PnumWorkers=1 \\\n  -PmaxNumWorkers=1 \\\n  -PserviceAccount=foobar@baz.iam.gserviceaccount.com \\\n  -PworkerMachineType=n1-standard-1\n```\n### Update Pipeline\n```\n./gradlew run \\\n  -Penv=foo \\\n  -PotherProject=bar \\\n  -Pgcp.project=baz \\\n  -PnumWorkers=1 \\\n  -PmaxNumWorkers=1 \\\n  -PserviceAccount=foobar@baz.iam.gserviceaccount.com \\\n  -PworkerMachineType=n1-standard-1\n  -Pupdate=true\n```\n\n## Reference\n* Gladle dependency locking\n  * http://blog.64p.org/entry/2020/05/13/100039\n* JSON to Java\n  * http://www.jsonschema2pojo.org/       ","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringmt%2Fapache-beam-gradle-sample","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fspringmt%2Fapache-beam-gradle-sample","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fspringmt%2Fapache-beam-gradle-sample/lists"}