{"id":14982322,"url":"https://github.com/dtstack/taier","last_synced_at":"2025-05-15T08:05:05.078Z","repository":{"id":37359938,"uuid":"343649088","full_name":"DTStack/Taier","owner":"DTStack","description":"Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display","archived":false,"fork":false,"pushed_at":"2024-08-07T03:06:51.000Z","size":154861,"stargazers_count":1239,"open_issues_count":72,"forks_count":336,"subscribers_count":33,"default_branch":"master","last_synced_at":"2025-04-03T14:08:47.055Z","etag":null,"topics":["azkaban","chunjun","cronjob-scheduler","dag","data-schedule","distributed-schedule-system","flink","hadoop","hive","job-scheduler","scheduler","spark","task-schedule","workflow-scheduling-system"],"latest_commit_sha":null,"homepage":"https://dtstack.github.io/Taier/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DTStack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-02T04:49:33.000Z","updated_at":"2025-04-03T05:46:07.000Z","dependencies_parsed_at":"2024-06-14T04:42:21.768Z","dependency_job_id":"f8db1f97-d55f-4f43-83b2-15fe3a118d24","html_url":"https://github.com/DTStack/Taier","commit_stats":{"total_commits":1036,"total_committers":62,"mean_commits":16.70967741935484,"dds":0.7258687258687259,"last_synced_commit":"deee65da4a4b84eb891b81d3184db4dea389d409"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DTStack%2FTaier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DTStack%2FTaier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DTStack%2FTaier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DTStack%2FTaier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DTStack","download_url":"https://codeload.github.com/DTStack/Taier/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248423540,"owners_count":21101001,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azkaban","chunjun","cronjob-scheduler","dag","data-schedule","distributed-schedule-system","flink","hadoop","hive","job-scheduler","scheduler","spark","task-schedule","workflow-scheduling-system"],"created_at":"2024-09-24T14:05:11.843Z","updated_at":"2025-04-11T14:49:29.388Z","avatar_url":"https://github.com/DTStack.png","language":"Java","readme":"\u003cdiv align=\"center\"\u003e\n         \u003ca href=\"https://dtstack.github.io/Taier/\" target=\"_blank\" rel=\"noopener noreferrer\"\u003e\n           \u003cimg src=\"website/static/img/logo.svg\" width=\"20%\" height=\"20%\" alt=\"Taier Logo\" /\u003e\n        \u003c/a\u003e\n \u003ch1\u003eTaier\u003c/h1\u003e\n \u003ch3\u003eA distributed dispatching system\u003c/h3\u003e\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/release/Dtstack/Taier.svg\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/stars/Dtstack/Taier\"\u003e\n  \u003cimg src=\"https://img.shields.io/github/forks/Dtstack/Taier\"\u003e\n  \u003ca href=\"https://www.apache.org/licenses/LICENSE-2.0.html\"\u003e\n   \u003cimg src=\"https://img.shields.io/badge/license-Apache%202-4EB1BA.svg\"\u003e\n  \u003c/a\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://dtstack.github.io/Taier/\"\u003eOffice Website\u003c/a\u003e |\n    \u003ca href=\"https://dtstack.github.io/Taier/docs/guides/introduction/\"\u003eDocument\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n[中文](./README_zh-CN.md) | [English](./README.md) \n\n## Introduction\n\n\u003e **Taier** , spelling in chinese is 太阿, is one of the ancient chinese celebrated swords.\n\n**Taier** is distributed dispatching system that focus on different tasks submitted and scheduled. It's aimed at reducing the **ETL**'s cost, making the complex dependencies between tasks clearly and reducing the labor cost about submitting, scheduling and O\u0026M.\n\nIt's unnecessary to concern about the complex dependencies between tasks and the underlying architecture about the big data platform at **Taier**, so that you can pay more attention into business.\n\n**Taier** provide an one-stop big data platform for submitting tasks, scheduling tasks, O\u0026M, presentation about indicators.\n\nThe core features for Taier are as follows:\n\n\n- Easy to distributed extend\n- Visualization config for DAG\n- With an IDE development platform designed for big-data users\n- Supports to develop your own plugins\n- Multiply task modes including guide mode and script mode\n- Supports to the dependencies between upstream/downstream tasks\n- Supports batch and stream tasks\n- Integrates various different versions of Hadoop\n- Easy to integrate Flink Standalone\n- Completely safe and non-intrusive to the cluster's environment\n- Isolation by tenants and clusters\n- Supports kerberos authentication\n- Different versions for tasks\n- Supports user-defined parameters for task\n- Real-time monitoring about cluster's resources\n- Real-time presented about data indicators\n- Restrict the task's resource\n\n## Architecture\n\n![architecture](/website/static/img/readme/taier-architecture.png)\n\n## Quick start\n\n#### [In Docker](https://dtstack.github.io/Taier/docs/quickstart/deploy/docker#2-%E4%BD%BF%E7%94%A8docker-compose)\n\n```shell\n$ wget https://raw.githubusercontent.com/DTStack/Taier/master/docker-compose.yml\n$ docker-compose up -d\n```\n\n#### [Online Preview](http://taier.dtstack.cn/)\n\n#### [Develop Tasks](https://dtstack.github.io/Taier/docs/quickstart/start)\n\n![main](/website/static/img/readme/main.png)\n\n## Tasks\n\n|       Tasks       |                                     Documentation                                     |\n| :---------------: | :-----------------------------------------------------------------------------------: |\n|     Work Flow     |       [Documentation](https://dtstack.github.io/Taier/docs/functions/task/workflow)   |\n|     Data Sync     |       [Documentation](https://dtstack.github.io/Taier/docs/functions/task/sync)       |\n| Data Acquisition  | [Documentation](https://dtstack.github.io/Taier/docs/functions/task/data-acquisition) |\n|       Flink       |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/flink)       |\n|       Shell       |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/shell)       |\n|      Python       |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/python)      |\n|     Spark SQL     |    [Documentation](https://dtstack.github.io/Taier/docs/functions/task/spark-sql)     |\n|     Hive SQL      |     [Documentation](https://dtstack.github.io/Taier/docs/functions/task/hive-sql)     |\n|     Flink SQL     |    [Documentation](https://dtstack.github.io/Taier/docs/functions/task/flink-sql)     |\n|   OceanBase SQL   |  [Documentation](https://dtstack.github.io/Taier/docs/functions/task/oceanbase-sql)   |\n|  ClickHouse SQL   |  [Documentation](https://dtstack.github.io/Taier/docs/functions/task/clickhouse-sql)  |\n|     Doris SQL     |    [Documentation](https://dtstack.github.io/Taier/docs/functions/task/doris-sql)     |\n|      TiDB SQL     |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/tidb-sql)    |\n|      MySQL SQL    |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/mysql-sql)   |\n|      Vertica SQL  |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/vertica-sql) |\n|      Postgre SQL  |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/postgre-sql) |\n|     SqlServer SQL |     [Documentation](https://dtstack.github.io/Taier/docs/functions/task/sqlserver-sql)|\n|   Greenplum SQL   |     [Documentation](https://dtstack.github.io/Taier/docs/functions/task/greenplum-sql)|\n|   MaxCompute SQL  |    [Documentation](https://dtstack.github.io/Taier/docs/functions/task/maxcompute-sql)|\n|     GaussDB SQL   |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/guassdb-sql) |\n|     DataX         |      [Documentation](https://dtstack.github.io/Taier/docs/functions/task/datax) |\n| User-defined Task |           [Documentation](https://dtstack.github.io/Taier/docs/expand/task)           |\n\n## Questions\n[FAQ Reference](https://dtstack.github.io/Taier/docs/quickstart/faq) For questions, bugs and supports please open an [issue](https://github.com/DTStack/Taier/issues/new/choose), we'll reply you in time.\n\n## Stay in touch\n\n- Slack https://join.slack.com/t/slack-p437975/shared_invite/zt-1iw5x1fw7-A6rVolqjP2z8V09~WwFUiA\n\n## Contribution\n\nPlease make sure to read the [Contributing Guide](https://dtstack.github.io/Taier/docs/contributing) before making a pull request.\n\n## Contributor\n\n\u003ca href=\"https://github.com/Dtstack/Taier/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=Dtstack/Taier\" /\u003e\n\u003c/a\u003e\n\n## License\n\n**Taier** is under the Apache 2.0 license. See the [LICENSE](http://www.apache.org/licenses/LICENSE-2.0) file for\ndetails.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdtstack%2Ftaier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdtstack%2Ftaier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdtstack%2Ftaier/lists"}