{"id":36418535,"url":"https://github.com/circustar/data-trans","last_synced_at":"2026-01-11T17:01:28.524Z","repository":{"id":44562518,"uuid":"359165162","full_name":"circustar/data-trans","owner":"circustar","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-25T12:02:52.000Z","size":251,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-18T14:24:43.513Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/circustar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-18T14:25:35.000Z","updated_at":"2024-10-25T12:02:56.000Z","dependencies_parsed_at":"2022-09-14T22:41:38.897Z","dependency_job_id":null,"html_url":"https://github.com/circustar/data-trans","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/circustar/data-trans","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circustar%2Fdata-trans","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circustar%2Fdata-trans/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circustar%2Fdata-trans/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circustar%2Fdata-trans/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/circustar","download_url":"https://codeload.github.com/circustar/data-trans/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/circustar%2Fdata-trans/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28314259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T14:58:17.114Z","status":"ssl_error","status_checked_at":"2026-01-11T14:55:53.580Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-11T17:01:24.420Z","updated_at":"2026-01-11T17:01:28.467Z","avatar_url":"https://github.com/circustar.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"### 1.特性\n* 数据抽取工具\n* 自动管理临时表生命周期\n* 自动管理索引\n* 支持多种数据库，包括Oracle、Mysql、DB2、SQLServer\n\n### 2.依赖\n* mybatis-plus\n\n### 3.引入依赖\n```xml\n    \u003cdependency\u003e\n        \u003cgroupId\u003ecom.circustar\u003c/groupId\u003e\n        \u003cartifactId\u003edata-trans\u003c/artifactId\u003e\n        \u003cversion\u003e1.0.0\u003c/version\u003e\n    \u003c/dependency\u003e\n```\n\n### 4.使用方法\n#### 4.1.定义一个数据转换组\n##### 4.1.1.DATA_TRANS_GROUP中定义数据转换组名称\n|  字段名   | 名称  | 类型  | 备注  |\n|  ----  | ----  | ----  | ----  |\n| DATA_TRANS_GROUP_NAME  | 分组名 | 文字 |  |\n| RECOVERABLE  | 是否可恢复 | 数字 | 1表示上次执行报错后，下次仍可继续执行 |\n| REMARK  | 备注 | 文字 |  |\n\n##### 4.1.2.DATA_TRANS表中定义分步步骤\n|  字段名   | 名称  | 类型  | 备注  |\n|  ----  | ----  | ----  | ----  |\n| DATA_TRANS_GROUP_NAME  | 分组名 | 文字 |  |\n| DATA_TRANS_ID  | 分步ID | 文字 |  |\n| DEPEND_DATA_TRANS_ID  | 分步依赖ID | 文字 | 定义执行顺序，表示在依赖的分步ID后执行 |\n| TABLE_NAME  | 处理数据的表名 | 文字 | 表示在那个表中处理数据 |\n| DISABLED  | 是否禁用 | 数字 | 0表示未禁用，1表示禁用 |\n| DROP_TABLE_FLAG  | 执行前是否需要Drop表 | 数字 | 0表示不会Drop，1表示Drop |\n| CREATE_TABLE_FLAG  | 执行前是否需要创建表 | 数字 | 0表示不会创建，1表示创建 |\n| TRUNCATE_TABLE_FLAG  | 执行前是否需要清空表数据 | 数字 | 0表示不会清空，1表示清空 |\n| UPDATE_TYPE  | 更新类型 | 文字 | 可选择INSERT/UPDATE/DELETE |\n| SELECT_PREFIX  | 执行Select时的前缀 | 文字 | 比如SqlServer的前缀：Top 1 |\n| SELECT_SUFFIX  | 执行Select时的后缀 | 文字 | 比如MySQL的后缀：Limit 1 |\n| SKIP_EXPRESSION  | 跳过条件 | 文字 | 支持SPEL表达式，表示什么情况下跳过该步骤 |\n| ADD_TO_PARAM_MAP  | 是否加入参数Map中 | 数字 | 1时回把更新结果放到参数Map中 |\n| REMARK  | 备注 | 文字 |  |\n\n##### 4.1.3.DATA_TRANS_SOURCE表中数据来源\n|  字段名   | 名称  | 类型  | 备注  |\n|  ----  | ----  | ----  | ----  |\n| DATA_TRANS_SOURCE_ID  | 表ID | 文字 |  |\n| DATA_TRANS_ID  | 分步ID | 文字 |  |\n| SOURCE_TABLE  | 数据来源表 | 文字 |  |\n| ALIAS  | 别名 | 文字 |  |\n| JOIN_TYPE  | 表连接类型 | 文字 | 可选inner join、left join等|\n| ON_STATEMENT  | 表连接条件 | 文字 |  |\n| WHERE_STATEMENT  | 过滤条件 | 文字 |  |\n\n##### 4.1.4.DATA_TRANS_COLUMN表中定义来源表、目标表字段对应关系\n|  字段名   | 名称  | 类型  | 备注  |\n|  ----  | ----  | ----  | ----  |\n| DATA_TRANS_COLUMN_ID  | 表ID | 文字 |  |\n| DATA_TRANS_ID  | 分步ID | 文字 |  |\n| COLUMN_NAME  | 目标列名 | 文字 |  |\n| COLUMN_TYPE  | 目标列列类型 | 文字 |  |\n| COLUMN_VALUE  | 目标列值 | 文字 |  |\n| GROUP_FLAG  | 分组标志 | 数字 |  |\n| PRIMARY_KEY  | 是否是主键 | 数字 |  |\n| INDEX_NAME_ORDERS  | 索引名以及字段顺序 | 文字 | 样例: idx1,1;idx2,2 |\n\n#### 4.2.执行\n```java\npublic class TestDataTrans {\n    @Autowired\n    private DataTransExecutorManager executorManager;\n\n    @Test\n    public void Test01(){\n        // 创建执行参数\n        Map\u003cString, String\u003e paramMap = new HashMap\u003c\u003e();\n        paramMap.put(\"param1\", \"testParam\");\n        Long execId = executorManager.addExecInfo(\"testGroup\", paramMap);\n\n        // 执行\n        executorManager.exec(execId);\n    }\n}\n```\n\n#### 4.3.DATA_TRANS_EXEC表中查看执行结果\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcircustar%2Fdata-trans","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcircustar%2Fdata-trans","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcircustar%2Fdata-trans/lists"}