{"id":13989025,"url":"https://github.com/apache/inlong","last_synced_at":"2026-01-11T17:05:13.924Z","repository":{"id":36954798,"uuid":"231533573","full_name":"apache/inlong","owner":"apache","description":"Apache InLong - a one-stop, full-scenario integration framework for massive data","archived":false,"fork":false,"pushed_at":"2025-12-24T06:38:01.000Z","size":60957,"stargazers_count":1473,"open_issues_count":5,"forks_count":567,"subscribers_count":65,"default_branch":"master","last_synced_at":"2025-12-25T19:45:27.908Z","etag":null,"topics":["data-streaming","event-streaming","framework","full-scenario-service","inlong","massive-data-integration","one-stop-service"],"latest_commit_sha":null,"homepage":"https://inlong.apache.org/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGES.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-01-03T07:13:14.000Z","updated_at":"2025-12-24T06:38:06.000Z","dependencies_parsed_at":"2023-09-22T19:24:51.946Z","dependency_job_id":"a3d462a7-9f56-4ac9-9c5c-c06d8c9189fe","html_url":"https://github.com/apache/inlong","commit_stats":{"total_commits":4425,"total_committers":206,"mean_commits":"21.480582524271846","dds":0.895593220338983,"last_synced_commit":"643c790e0215f2ab6ebb264178d32c0d338f0586"},"previous_names":["apache/incubator-inlong","apache/incubator-tubemq"],"tags_count":43,"template":false,"template_full_name":null,"purl":"pkg:github/apache/inlong","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Finlong","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Finlong/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Finlong/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Finlong/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/inlong/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Finlong/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28314264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T14:58:17.114Z","status":"ssl_error","status_checked_at":"2026-01-11T14:55:53.580Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-streaming","event-streaming","framework","full-scenario-service","inlong","massive-data-integration","one-stop-service"],"created_at":"2024-08-09T13:01:29.207Z","updated_at":"2026-01-11T17:05:13.919Z","avatar_url":"https://github.com/apache.png","language":"Java","readme":"\u003c!--\n\n    Licensed to the Apache Software Foundation (ASF) under one\n    or more contributor license agreements.  See the NOTICE file\n    distributed with this work for additional information\n    regarding copyright ownership.  The ASF licenses this file\n    to you under the Apache License, Version 2.0 (the\n    \"License\"); you may not use this file except in compliance\n    with the License.  You may obtain a copy of the License at\n\n      http://www.apache.org/licenses/LICENSE-2.0\n\n    Unless required by applicable law or agreed to in writing,\n    software distributed under the License is distributed on an\n    \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n    KIND, either express or implied.  See the License for the\n    specific language governing permissions and limitations\n    under the License.\n\n--\u003e\n\n\n# [A one-stop, full-scenario integration framework for massive data](https://inlong.apache.org/)\n[![GitHub Actions](https://github.com/apache/inlong/actions/workflows/ci_build.yml/badge.svg)](https://github.com/apache/inlong/actions)\n[![CodeCov](https://codecov.io/gh/apache/inlong/branch/master/graph/badge.svg)](https://codecov.io/gh/apache/inlong)\n[![Maven Central](https://maven-badges.herokuapp.com/maven-central/org.apache.inlong/inlong/badge.svg)](http://search.maven.org/#search%7Cga%7C1%7Corg.apache.inlong)\n[![GitHub release](https://img.shields.io/badge/release-download-orange.svg)](https://inlong.apache.org/download/)\n[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)\n[![Twitter](https://img.shields.io/twitter/url?label=Follow%20ApacheInlong\u0026style=social\u0026url=https%3A%2F%2Ftwitter.com%2FApacheInlong)](https://twitter.com/ApacheInlong)\n[![Slack](https://img.shields.io/badge/%20Slack-InLong%20Channel-blueviolet)](https://the-asf.slack.com/archives/C01QAG6U00L)\n\n- [What is Apache InLong?](#what-is-apache-inlong)\n- [Features](#features)\n- [When should I use InLong?](#when-should-i-use-inlong)\n- [Build InLong](#build-inlong)\n- [Deploy InLong](#deploy-inlong)\n- [Contribute to InLong](#contribute-to-inlong)\n- [Contact Us](#contact-us)\n- [Documentation](#documentation)\n- [License](#license)\n\n# What is Apache InLong?\n|                                       **Stargazers Over Time**                                        |                                                                                                **Contributors Over Time**                                                                                                |\n|:-----------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|\n| [![Stargazers over time](https://starchart.cc/apache/inlong.svg)](https://starchart.cc/apache/inlong) | [![Contributor Over Time](https://contributor-overtime-api.git-contributor.com/contributors-svg?chart=contributorOverTime\u0026repo=apache/inlong)](https://git-contributor.com?chart=contributorOverTime\u0026repo=apache/inlong) |\n\n[Apache InLong](https://inlong.apache.org) is a one-stop, full-scenario integration framework for massive data that supports `Data Ingestion`, `Data Synchronization` and `Data Subscription`, and it provides automatic, secure and reliable data transmission capabilities. InLong also supports both batch and stream data processing at the same time, which offers great power to build data analysis, modeling and other real-time  applications based on streaming data.\n\nInLong (应龙) is a divine beast in Chinese mythology who guides the river into the sea, and it is regarded as a metaphor of the InLong system for reporting data streams.\n\nInLong was originally built at Tencent, which has served online businesses for more than 8 years, to support massive data (data scale of more than 80 trillion pieces of data per day) reporting services in big data scenarios. The entire platform has integrated 5 modules:  Ingestion, Convergence, Caching, Sorting, and Management, so that the business only needs to provide data sources, data service quality, data landing clusters and data landing formats, that is, the data can be continuously pushed from the source to the target cluster, which greatly meets the data reporting service requirements in the business big data scenario.\n\nFor getting more information, please visit our project documentation at https://inlong.apache.org/.\n![inlong-structure-en.png](https://github.com/apache/inlong-website/blob/master/static/img/inlong-structure-en.png)\n\n\n## Features\nApache InLong offers a variety of features:\n* **Ease of Use**: a SaaS-based service platform. Users can easily and quickly report, transfer, and distribute data by publishing and subscribing to data based on topics.\n* **Stability \u0026 Reliability**: derived from the actual online production environment. It delivers high-performance processing capabilities for 10 trillion-level data streams and highly reliable services for 100 billion-level data streams.\n* **Comprehensive Features**: supports various types of data access methods and can be integrated with different types of Message Queue (MQ). It also provides real-time data extract, transform, and load (ETL) and sorting capabilities based on rules. InLong also allows users to plug features to extend system capabilities.\n* **Service Integration**: provides unified system monitoring and alert services. It provides fine-grained metrics to facilitate data visualization. Users can view the running status of queues and topic-based data statistics in a unified data metric platform. Users can also configure the alert service based on their business requirements so that users can be alerted when errors occur.\n* **Scalability**: adopts a pluggable architecture that allows you to plug modules into the system based on specific protocols. Users can replace components and add features based on their business requirements.\n\n\n## When should I use InLong?\nInLong aims to provide a one-stop, full-scenario integration framework for massive data, users can easily build stream-based data applications. It supports `Data Ingestion`, `Data Synchronization` and `Data Subscription` at the same time, and is suitable for environments that need to quickly build a data reporting platform, as well as an ultra-large-scale data reporting environment that InLong is very suitable for, and an environment that needs to automatically sort and land the reported data.\n\nYou can use InLong in the following ways：\n- Integrate InLong, manage data streams through [SDK](https://inlong.apache.org/docs/next/sdk/manager-sdk/example).\n- Use [the InLong command-line tool](https://inlong.apache.org/docs/next/user_guide/command_line_tools) to view and create data streams.\n- Visualize your operations on [InLong dashboard](https://inlong.apache.org/docs/next/user_guide/dashboard_usage).\n\n## Supported Data Nodes (Updating)\n| Type         | Name              | Version                      |\n|--------------|-------------------|------------------------------|\n| Extract Node | Auto Push         | None                         |\n|              | File              | None                         |\n|              | Kafka             | 2.x                          |\n|              | MongoDB           | \u003e= 3.6                       |\n|              | MQTT              | \u003e= 3.1                       |\n|              | MySQL             | 5.6, 5.7, 8.0.x              |\n|              | Oracle            | 11,12,19                     |\n|              | PostgreSQL        | 9.6, 10, 11, 12              |\n|              | Pulsar            | 2.8.x                        |\n|              | Redis             | 2.6.x                        |\n|              | SQLServer         | 2012, 2014, 2016, 2017, 2019 |\n| Load Node    | Auto Consumption  | None                         |\n|              | ClickHouse        | 20.7+                        |\n|              | Elasticsearch     | 6.x, 7.x                     |\n|              | Greenplum         | 4.x, 5.x, 6.x                |\n|              | HBase             | 2.2.x                        |\n|              | HDFS              | 2.x, 3.x                     |\n|              | Hive              | 1.x, 2.x, 3.x                |\n|              | Iceberg           | 0.12.x                       |\n|              | Hudi              | 0.12.x                       |\n|              | Kafka             | 2.x                          |\n|              | MySQL             | 5.6, 5.7, 8.0.x              |\n|              | Oracle            | 11, 12, 19                   |\n|              | PostgreSQL        | 9.6, 10, 11, 12              |\n|              | SQLServer         | 2012, 2014, 2016, 2017, 2019 |\n|              | TDSQL-PostgreSQL  | 10.17                        |\n|              | Doris             | \u003e= 0.13                      |\n|              | StarRocks         | \u003e= 2.0                       |\n|              | Kudu              | \u003e= 1.12.0                    |\n|              | Redis             | \u003e= 3.0                       |\n|              | OceanBase         | \u003e= 1.0                       |\n\n## Build InLong\nMore detailed instructions can be found at [Quick Start](https://inlong.apache.org/docs/development/how_to_build) section in the documentation.\n\nRequirements:\n- Java [JDK 8](https://adoptopenjdk.net/?variant=openjdk8)\n- Maven 3.6.1+\n- [Docker](https://docs.docker.com/engine/install/) 19.03.1+\n\nCodeStyle:\n\n```\nmvn spotless:apply\n```\n\nCompile and install:\n```\nmvn clean install -DskipTests\n```\n(Optional) Compile using docker image:\n```\ndocker pull maven:3.6-openjdk-8\ndocker run -v `pwd`:/inlong  -w /inlong maven:3.6-openjdk-8 mvn clean install -DskipTests\n```\nafter compile successfully, you could find distribution file at `inlong-distribution/target`.\n\n## Deploy InLong\n- [Standalone for InLong](https://inlong.apache.org/docs/next/deployment/standalone)\n- [Docker Compose](https://inlong.apache.org/docs/next/deployment/docker)\n- [InLong on Kubernetes](https://inlong.apache.org/docs/next/deployment/k8s)\n- [Bare Metal](https://inlong.apache.org/docs/next/deployment/bare_metal)\n\n## Develop InLong\n- [Agent Plugin extends a Extract Data Node](https://inlong.apache.org/docs/next/design_and_concept/how_to_write_plugin_agent)\n- [Sort Plugin extends a Data Node](https://inlong.apache.org/docs/next/design_and_concept/how_to_extend_data_node_for_sort)\n- [Manager Plugin extends a Data Node](https://inlong.apache.org/docs/next/design_and_concept/how_to_extend_data_node_for_manager)\n- [Dashboard Plugin extends a Data Node page](https://inlong.apache.org/docs/next/design_and_concept/how_to_write_plugin_dashboard)\n\n## Contribute to InLong\n- Report any issue on [GitHub Issue](https://github.com/apache/inlong/issues)\n- Code pull request according to [How to contribute](https://inlong.apache.org/community/how-to-contribute).\n\n## Contact Us\n- Join Apache InLong mailing lists:\n    | Name                                                                          | Scope                           |                                                                 |                                                                     |                                                                              |\n    |:------------------------------------------------------------------------------|:--------------------------------|:----------------------------------------------------------------|:--------------------------------------------------------------------|:-----------------------------------------------------------------------------|\n    | [dev@inlong.apache.org](mailto:dev@inlong.apache.org)     | Development-related discussions | [Subscribe](mailto:dev-subscribe@inlong.apache.org)   | [Unsubscribe](mailto:dev-unsubscribe@inlong.apache.org)   | [Archives](http://mail-archives.apache.org/mod_mbox/inlong-dev/)   |\n- Ask questions on [Apache InLong Slack](https://the-asf.slack.com/archives/C01QAG6U00L)\n\n## Documentation\n- Home page: https://inlong.apache.org/\n- Issues: https://github.com/apache/inlong/issues\n\n## License\n© Contributors Licensed under an [Apache-2.0](LICENSE) license.\n\n\n","funding_links":[],"categories":["Java","大数据","Projects"],"sub_categories":["Data Integration"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Finlong","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Finlong","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Finlong/lists"}