{"id":13569953,"url":"https://github.com/LinMingQiang/flink-learn","last_synced_at":"2025-04-04T06:31:30.287Z","repository":{"id":40358157,"uuid":"100566231","full_name":"LinMingQiang/flink-learn","owner":"LinMingQiang","description":"Learning Flink : Flink CEP,Flink Core,Flink SQL","archived":false,"fork":false,"pushed_at":"2022-02-21T08:35:27.000Z","size":361633,"stargazers_count":71,"open_issues_count":5,"forks_count":35,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-05T02:36:05.771Z","etag":null,"topics":["bigdata","flink","sql","stream"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LinMingQiang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-08-17T05:48:09.000Z","updated_at":"2024-07-25T01:34:26.000Z","dependencies_parsed_at":"2022-08-19T00:21:11.894Z","dependency_job_id":null,"html_url":"https://github.com/LinMingQiang/flink-learn","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinMingQiang%2Fflink-learn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinMingQiang%2Fflink-learn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinMingQiang%2Fflink-learn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LinMingQiang%2Fflink-learn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LinMingQiang","download_url":"https://codeload.github.com/LinMingQiang/flink-learn/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247134439,"owners_count":20889397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigdata","flink","sql","stream"],"created_at":"2024-08-01T14:00:46.346Z","updated_at":"2025-04-04T06:31:26.640Z","avatar_url":"https://github.com/LinMingQiang.png","language":"Java","funding_links":[],"categories":["Java"],"sub_categories":[],"readme":"# Flink Learn\n---\n|   Flink                 | scala version      |Kafka version   |\n|:------------------:|:------------------:|:------------------:|\n| **1.14.2**    | **2.12+**             | **0.10+**             |\n---\n```\nMaven 模板 : https://www.mvnjar.com/org.apache.flink/flink-quickstart-scala/jar.html \u003cbr\u003e\n注意 ： 当在idea上调试时，先将 es-shade install。然后 ignore Project：es-shade ；否则有冲突问题\n```\n**Flink Submit**\n```\n！：1.10之后去掉 -yn ，加上会报错\n1：单独资源： flink run -m yarn-cluster --yarnname _wc -p 5 -yjm 1024 -ytm 1024 -ys 2 WordCount.jar\n2：共享空间： yarn-session.sh -n 5 -jm 1024 -tm 1024 -s 2 // 在yarn提前开辟一个资源空间 application_1567318548013_0001。 \u003cbr\u003e\n在开辟的空间上提交任务： flink run -m yarn-cluster --yarnname flink_wc -p 5 -yn 4 -yjm 1024 -ytm 1024 -ys 2 -yid application_1567318548013_0001 WordCount.jar \u003cbr\u003e\n```\n**测试的内容包括**\n```\nDemo和一些注解\n1: Flink DDL\n2: FLink Connector\n3: Flink Calcite\n4: Flink Join\n5: FLink 维表\n6: Flink Window\n7: FLink Datalake\n8: Flink Hive\n9: Flink CDC\n10: FLink 时态表\n11: Flink State\n12: Flink CEP\n13: Flink StateProcessApi\n14: Flink Bug\n```\n**Flink API** \u003cbr\u003e\n[官方文档 ： Table DataStream 互转](https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/common.html#convert-a-datastream-or-dataset-into-a-table)\n```\nStreamExecutionEnvironment : 流式相关。不能使用SQL的API。如果要在流里面用SQL，使用下面的\nStreamTableEnvironment ： 流式SQL相关。可以使用 SQL的API。如果要用Stream相关的，需要将tableData.toRetractStream[Row]\n```\n---\n```\nmvn编译参数加上： -Xmx2g -XX:MaxMetaspaceSize=1024m\nwindow 系统在 assembly.xml 里面使用 \u003coutputDirectory\u003e./\u003c/outputDirectory\u003e\n否则报错：\nOS=Windows and the assembly descriptor contains a *nix-specific root-relative-reference (starting with slash)\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLinMingQiang%2Fflink-learn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLinMingQiang%2Fflink-learn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLinMingQiang%2Fflink-learn/lists"}