{"id":14242079,"url":"https://github.com/642933588/jiron-cloud","last_synced_at":"2025-08-11T18:32:25.936Z","repository":{"id":247821334,"uuid":"826930937","full_name":"642933588/jiron-cloud","owner":"642933588","description":"该项目整合了多款优秀的开源产品，构建了一个功能全面的数据开发平台。平台提供了强大的数据集成、数据开发、数据查询、数据服务、数据质量管理、工作流调度和元数据管理功能。#dinky #dolphinscheduler #datavines #flinkcdc #openmetadata #flink #数据开发 #数据平台 # 数据开发平台 #大数据","archived":false,"fork":false,"pushed_at":"2024-08-21T07:45:43.000Z","size":344,"stargazers_count":98,"open_issues_count":0,"forks_count":29,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-21T08:57:32.374Z","etag":null,"topics":["datavines","dinky","dolphinscheduler","metadata","openmetadata"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/642933588.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":"http://doc.ruoyi.vip/ruoyi-cloud/other/donate.html"}},"created_at":"2024-07-10T17:04:29.000Z","updated_at":"2024-08-21T07:45:46.000Z","dependencies_parsed_at":"2024-08-21T08:52:33.562Z","dependency_job_id":null,"html_url":"https://github.com/642933588/jiron-cloud","commit_stats":null,"previous_names":["642933588/jiron-cloud"],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/642933588%2Fjiron-cloud","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/642933588%2Fjiron-cloud/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/642933588%2Fjiron-cloud/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/642933588%2Fjiron-cloud/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/642933588","download_url":"https://codeload.github.com/642933588/jiron-cloud/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":216304993,"owners_count":16005946,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["datavines","dinky","dolphinscheduler","metadata","openmetadata"],"created_at":"2024-08-21T12:00:47.466Z","updated_at":"2025-08-11T18:32:25.889Z","avatar_url":"https://github.com/642933588.png","language":"Java","funding_links":["http://doc.ruoyi.vip/ruoyi-cloud/other/donate.html"],"categories":["Java"],"sub_categories":[],"readme":"## 请各位朋友点个 ★star★ 支持一下，非常感谢~\n\n[【 Github地址：https://github.com/642933588/jiron-cloud 】](https://github.com/642933588/jiron-cloud)\n\n[【 Gitee 地址：https://gitee.com/642933588/jiron-cloud 】](https://gitee.com/642933588/jiron-cloud)\n\n[【 推荐实时数仓（数据湖）项目：https://github.com/Mrkuhuo/data-warehouse-learning 】](https://github.com/Mrkuhuo/data-warehouse-learning)\n\n【 公众号】Jiron开源\n\n这是一款功能全面的数据开发平台。平台提供了强大的数据集成、数据开发、数据查询、数据服务、数据质量管理、工作流调度和元数据管理功能。dinky ，dolphinscheduler ，datavines ，flinkcdc ，openmetadata ，flink ，数据开发 ，数据平台 ，数据开发平台 ，大数据\n\n## 目录\n\n- [功能](#功能)\n    - [数据质量管理 (Datavines)](#数据质量管理-datavines)\n    - [工作流调度 (DolphinScheduler)](#工作流调度-dolphinscheduler)\n    - [元数据管理 (OpenMetadata)](#元数据管理-openmetadata)\n    - [实时数据处理 (Dinky)](#实时数据处理-dinky)\n    - [数据集成（FlinkCDC）](#数据集成)\n    - [数据查询（Dataq）](#数据查询)\n    - [数据分析（DataAnalysis）](#数据分析)\n    - [数据服务（Dataservice）](#数据服务)\n- [架构](#架构)\n- [演示环境](#演示环境)\n- [功能展示](#功能展示)\n- [贡献](#贡献)\n- [许可](#许可)\n- [联系方式](#联系方式)\n\n## 功能\n\n### 数据质量管理 (Datavines)\n\n- **数据质量监控**：定义和执行数据质量规则，如空值检测、重复值检测、范围检查等，确保数据的准确性和完整性。\n- **告警和通知**：当数据质量出现问题时，Datavines 通过电子邮件、短信等方式发送告警通知，确保相关人员能够及时响应和处理问题。\n- **数据治理**：支持数据分类、标签和权限管理，确保数据的合规性和安全性。\n- **数据质量报告**：生成详细的数据质量报告，帮助用户了解和改进数据质量。\n\n### 工作流调度 (DolphinScheduler)\n\n- **分布式架构**：处理大规模的数据处理任务，确保系统的高可用性和扩展性。\n- **任务依赖管理**：定义复杂的任务依赖关系，确保任务按照预定的顺序执行。\n- **丰富的任务类型**：支持多种类型的任务，包括 Shell、Python 脚本、SQL、MapReduce、Spark、Flink 等。\n- **可视化工作流设计**：提供直观的图形化用户界面，通过拖拽方式设计和配置工作流。\n- **任务调度**：支持灵活的调度策略，包括定时调度、依赖触发、手动触发等。\n- **任务监控与告警**：实时监控任务的执行状态，并提供告警通知功能，确保问题及时处理。\n- **资源管理**：管理任务所需的资源，确保资源合理分配和使用。\n\n### 元数据管理 (OpenMetadata)\n\n- **元数据管理**：捕获和存储各种数据资产的元数据，包括表、列、数据集、仪表板、机器学习模型等。\n- **数据发现**：提供强大的数据发现工具，帮助用户搜索和发现企业内的数据资源。\n- **数据血缘分析**：支持详细的数据血缘分析，显示数据的来源和流向，提高数据透明度。\n- **数据治理**：定义和执行数据治理策略，包括数据分类、标签、访问控制、合规性检查等。\n- **数据质量管理**：与 Datavines 集成，监控和管理数据质量，检测数据中的错误和异常。\n- **集成与互操作性**：支持与多种数据源和数据工具的集成，适应多样化的数据环境。\n- **API 和 SDK**：提供丰富的 API 和 SDK，支持开发者扩展和定制功能。\n- **可视化界面**：提供用户友好的可视化界面，便于浏览和管理元数据，查看数据血缘关系图，配置数据治理策略等。\n\n### 实时数据处理 (Dinky)\n\n- **实时数据处理**：Dinky 提供了强大的实时数据处理能力，支持 Flink 等流处理框架。\n- **实时计算任务**：用户可以定义和执行实时计算任务，处理实时数据流。\n- **数据集成**：支持多种数据源和目标，能够将实时数据流集成到不同的数据系统中。\n- **可视化开发**：提供直观的图形化界面，用户可以通过拖拽方式设计和配置实时计算任务。\n- **监控与告警**：实时监控数据流处理状态，并在出现异常时提供告警通知。\n- **高可用性**：设计了高可用性和容错机制，确保实时数据处理的稳定性和可靠性。\n\n### 数据集成\n\n- **多数据源支持**：支持多种数据源的集成，包括数据库、API、文件系统等，能够灵活应对不同的数据来源。\n- **多执行引擎支持**：支持多种执行引擎，包括 DataX、Sqoop、Flink CDC，满足不同场景的数据同步需求。\n- **全库同步**：支持整个数据库的同步，方便全面的数据迁移和复制。\n- **多表同步**：支持多表同步，简化复杂的数据集成任务。\n- **全量和增量同步**：提供全量和增量同步功能，实现数据的一体化同步。\n- **断点续传**：支持断点续传机制，确保数据集成过程的可靠性和稳定性。\n\n### 数据查询\n\n- **灵活高效的数据查询**：基于 Kyuubi 提供灵活高效的数据查询功能，支持 SQL 和非 SQL 查询，满足各种数据检索需求。\n- **多数据源支持**：支持多种数据源的数据查询，适用于不同的数据存储系统。\n- **高性能**：优化的查询引擎，确保数据查询的高性能和低延迟。\n\n### 数据分析\n\n- **集成多种数据分析工具**：集成多种数据分析和可视化工具，帮助用户快速获取洞见。\n- **可视化分析**：提供丰富的可视化组件，用户可以通过图表、仪表板等方式展示分析结果。\n- **高级分析功能**：支持机器学习模型、统计分析等高级分析功能，满足复杂数据分析需求。\n\n### 数据服务\n\n- **数据服务接口**：提供数据服务接口，支持 RESTful API 和 GraphQL，方便数据的对外提供和集成。\n- **灵活的数据访问**：通过 API 提供灵活的数据访问方式，满足不同应用和系统的集成需求。\n- **高可用性和安全性**：设计了高可用性和安全性机制，确保数据服务的可靠性和数据的安全性。\n\n## 架构\n![架构图](https://github.com/642933588/dataworks/assets/10755257/c2f6a740-dc79-4e5c-8c3b-1a6f499c784b)\n\n\n## 演示环境\nJiron演示环境体验\n1. 关注公众号【Jiron开源】\n扫码关注公众号，即可获取演示环境的访问地址及账号信息，第一时间体验Jiron数据开发平台的强大功能。\n\u003cimg width=\"257\" alt=\"image\" src=\"https://github.com/user-attachments/assets/777b3ddf-f479-44c0-aa02-568856cd7958\"\u003e\n\n2. 加入知识星球，解锁 Jiron 数据开发平台，详尽安装与部署步骤，助你极速起飞！\n加入我们的知识星球社区，即可获取完整的Jiron平台本地部署文件，轻松在本地搭建专属环境，深入体验Jiron数据开发平台。\n\u003cimg width=\"257\" alt=\"image\" src=\"https://github.com/user-attachments/assets/8dc77cd1-f43b-4c55-8c31-7b081c012069\"\u003e\n\n想要随时获取最新功能更新、案例分享和技术干货，快来加入我们的社群吧！\n\n## 功能展示\n\n#### 首页\n![0](https://github.com/642933588/dataworks/assets/10755257/6681dcb9-3720-4762-9a77-76ee755689f9)\n\n#### 系统管理\n![0-0](https://github.com/642933588/dataworks/assets/10755257/bb4b7688-a026-4904-8459-59479ea5aa78)\n\n#### 数据开发\n![0-0](https://github.com/user-attachments/assets/d084ff22-c593-4a5d-98d2-faba4f266f86)\n![0-2](https://github.com/user-attachments/assets/9c74e761-b92f-4399-9bbf-4f6c070ff88c)\n![0-3](https://github.com/user-attachments/assets/41324e91-d6e1-4207-8e5d-01717797f10a)\n![0-4](https://github.com/user-attachments/assets/15860241-8cef-4448-8079-372d1776fdb9)\n![0-5](https://github.com/user-attachments/assets/db70c35a-11c4-4a9f-951c-3c8925d39ae6)\n![1-0](https://github.com/642933588/dataworks/assets/10755257/7027cc32-868a-4231-a8fc-9ef9b525e13e)\n![1-1](https://github.com/642933588/dataworks/assets/10755257/aa9cf079-8f5a-43cd-9f06-a9780e98d8e6)\n![1-2](https://github.com/642933588/dataworks/assets/10755257/a1d6a53a-6300-4025-9b83-89e5069acf8b)\n![2-0](https://github.com/642933588/dataworks/assets/10755257/4fef8c5c-0497-4b95-b654-bae1b574fda6)\n![3-0](https://github.com/642933588/dataworks/assets/10755257/f245e4de-ffae-4b1f-afd1-e7a279bb7c9e)\n![4-0](https://github.com/642933588/dataworks/assets/10755257/0e81d358-265a-4dab-b668-46462abbe1fc)\n![5-0](https://github.com/642933588/dataworks/assets/10755257/12000edc-8ab1-4303-a03a-8e71fd8b0b67)\n![5-1](https://github.com/642933588/dataworks/assets/10755257/7c0c6c87-dd13-44a0-a82b-b50123dfb14f)\n![5-2](https://github.com/642933588/dataworks/assets/10755257/7e09eb69-eb29-4983-86a6-3cb49642862b)\n![6-0](https://github.com/user-attachments/assets/6f32873e-bb86-43ee-9bcd-d974326cffd4)\n![6-1](https://github.com/user-attachments/assets/6721fa04-24a3-4f99-b47c-4df766f2c8b0)\n![6-2](https://github.com/user-attachments/assets/40a45190-d1d9-4b38-b3ba-cc10590d5f00)\n![6-3](https://github.com/642933588/dataworks/assets/10755257/2544a246-b1ed-4c5a-be87-3532e20e564c)\n![6-4](https://github.com/user-attachments/assets/662f814d-f6e0-429a-a2eb-e596d853141e)\n![6-5](https://github.com/user-attachments/assets/21747168-dbb4-4cd6-80dd-a8455c6406eb)\n![6-6](https://github.com/user-attachments/assets/72a80dcf-8841-4b47-bec9-c36f2b179777)\n\n\n#### 工作流开发\n![0-0](https://github.com/user-attachments/assets/b11562c9-5302-4507-8f89-70cfe6baa6fe)\n![0-1](https://github.com/user-attachments/assets/f3da75f1-90c0-4c68-a4ef-703209b0dddb)\n![0-2](https://github.com/user-attachments/assets/b272acd3-ee98-4566-8a08-6f7e6745117d)\n![0-3](https://github.com/user-attachments/assets/487adff9-8677-4e95-8684-275483787cdb)\n![0-4](https://github.com/user-attachments/assets/750084d3-0d99-4beb-9db4-84ad23640bee)\n![0-5](https://github.com/user-attachments/assets/7e9a54b3-047c-4de8-b742-73b3383599da)\n![0-6](https://github.com/user-attachments/assets/5687b29f-aa9e-4da8-97a5-31e41b866a60)\n![1-0](https://github.com/642933588/dataworks/assets/10755257/203c88e5-8dbc-406c-a554-373942ef267b)\n![2-0](https://github.com/642933588/dataworks/assets/10755257/fc5b6ac7-21f9-4fee-83e7-a1572e141281)\n![3-0](https://github.com/642933588/dataworks/assets/10755257/bdedd0b9-3f05-43e5-8252-15f3745e40ab)\n![4-0](https://github.com/user-attachments/assets/16401705-cc77-453c-8ac8-b69418fd7d3d)\n![4-1](https://github.com/user-attachments/assets/fe529eee-ffe6-4310-9cda-2f022a329b25)\n![4-2](https://github.com/user-attachments/assets/cb4b93e6-fbbd-4788-88ed-cd9629e02da7)\n![4-3](https://github.com/user-attachments/assets/15a4adb2-e28a-40e1-b508-8617660868ee)\n\n#### 数据质量\n![0](https://github.com/642933588/dataworks/assets/10755257/965a1a85-abad-4caf-b754-3792c2a00d72)\n![1](https://github.com/642933588/dataworks/assets/10755257/557e5ac9-d641-4eb8-a304-476cdc68be3e)\n![2](https://github.com/642933588/dataworks/assets/10755257/6150be5b-1930-4609-8e28-553417c6e7b8)\n![3-1](https://github.com/642933588/dataworks/assets/10755257/24cb8837-77a3-4e96-97dd-05689f834d9f)\n![3](https://github.com/642933588/dataworks/assets/10755257/3fc98712-492d-49ba-bdcf-7c54c621460c)\n![4](https://github.com/642933588/dataworks/assets/10755257/705e090a-69af-4082-b389-18424b57a45e)\n![5](https://github.com/642933588/dataworks/assets/10755257/50040d4d-c2da-4db9-8a8e-41f8fa4e9765)\n![6](https://github.com/642933588/dataworks/assets/10755257/bf35a1e5-142c-4897-b389-4dc91b2cd9df)\n![7](https://github.com/642933588/dataworks/assets/10755257/e818dab4-379b-4c03-83c5-059026b253cb)\n![8](https://github.com/642933588/dataworks/assets/10755257/442d9993-ff2f-485c-9cc6-9b171ac5dd90)\n\n#### 数据资产\n![0-1](https://github.com/642933588/dataworks/assets/10755257/5bac1d76-ef6f-45c0-9e79-be0f3ff38bec)\n![0-2](https://github.com/642933588/dataworks/assets/10755257/78343571-750b-45c9-90bd-e76eedd1dcc3)\n![1-1](https://github.com/642933588/dataworks/assets/10755257/6eef3f96-c7b3-4496-8f38-50c2e019ede4)\n\n![1-2](https://github.com/642933588/dataworks/assets/10755257/bb736cf5-f1b8-4afc-8e2a-0efe19309761)\n![1-3](https://github.com/642933588/dataworks/assets/10755257/8fb5614a-5807-4726-b674-ea10567ca6fc)\n![1-4](https://github.com/642933588/dataworks/assets/10755257/53155fb6-6273-414e-bbc2-8503bd79c88d)\n![1-5](https://github.com/642933588/dataworks/assets/10755257/7f605596-242b-4c8d-a08c-50edf5ce75ae)\n![1-6](https://github.com/642933588/dataworks/assets/10755257/9dfecbda-7d9c-40e9-b8b7-2340a58db2e4)\n![1-8](https://github.com/642933588/dataworks/assets/10755257/f6520567-1da5-4f7b-b12b-ac9e59c8a29b)\n![1-9](https://github.com/642933588/dataworks/assets/10755257/f08ea68e-a1b6-48d7-8516-71a8c72cad06)\n![1-10](https://github.com/642933588/dataworks/assets/10755257/2ba81950-b09e-4400-84a6-299cdd477c7a)\n![1-7](https://github.com/642933588/dataworks/assets/10755257/b5d268de-fb09-4c54-9e49-146540cf1b5a)\n![2-1](https://github.com/642933588/dataworks/assets/10755257/6bf835c8-d5fc-4907-bebe-1e27a5917118)\n![2-2](https://github.com/642933588/dataworks/assets/10755257/ac553d7f-820a-420a-bed1-b4cbf7dbcaab)\n![2-3](https://github.com/642933588/dataworks/assets/10755257/a7c0541e-cc16-419a-83ab-852de94a9b3e)\n![3-1](https://github.com/642933588/dataworks/assets/10755257/544b7a0c-b0e1-43ce-9d0e-619ba6dc9967)\n![4-1](https://github.com/642933588/dataworks/assets/10755257/4861963d-bbff-4a9e-a332-d5efebebf87d)\n![5-1](https://github.com/642933588/dataworks/assets/10755257/45ecc8c5-3170-4fd0-b4e0-e807350cbb41)\n![6-1](https://github.com/642933588/dataworks/assets/10755257/15cfa63a-730a-4c1e-80bd-3f27a78a643f)\n![9-1](https://github.com/642933588/dataworks/assets/10755257/81f52c8a-0793-49a7-a935-388dd467f19c)\n![9-2](https://github.com/642933588/dataworks/assets/10755257/375e8ab1-0748-4e8f-b015-cf2e83a8abc9)\n\n## 贡献指南\n\n我们欢迎任何形式的贡献！如果你有好的建议或发现了问题，请提交一个 Issue 或 Pull Request。\n\n## 许可\n\n本项目采用 apache-2.0 license 许可证。详情请参阅 [LICENSE](LICENSE)。\n\n## 联系我们\n\n- 邮箱: 810021796@qq.com（qq同号）\n- v信号：F810021796 （加v请注明来意）\n\n![image](https://github.com/user-attachments/assets/5b262636-9332-4b70-8ddd-ef0a5e0a1365)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F642933588%2Fjiron-cloud","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F642933588%2Fjiron-cloud","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F642933588%2Fjiron-cloud/lists"}