{"id":15343723,"url":"https://github.com/wu-sheng/datacarrier","last_synced_at":"2026-03-07T18:31:22.503Z","repository":{"id":83979709,"uuid":"71865763","full_name":"wu-sheng/DataCarrier","owner":"wu-sheng","description":"DataCarrier is a light, embed, high-throughput, publish-subscribe MQ.","archived":false,"fork":false,"pushed_at":"2020-03-24T06:07:36.000Z","size":50,"stargazers_count":57,"open_issues_count":0,"forks_count":22,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-04-15T03:27:58.087Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wu-sheng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-10-25T06:19:45.000Z","updated_at":"2025-02-11T16:44:52.000Z","dependencies_parsed_at":"2023-10-20T23:51:18.575Z","dependency_job_id":null,"html_url":"https://github.com/wu-sheng/DataCarrier","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/wu-sheng/DataCarrier","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wu-sheng%2FDataCarrier","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wu-sheng%2FDataCarrier/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wu-sheng%2FDataCarrier/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wu-sheng%2FDataCarrier/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wu-sheng","download_url":"https://codeload.github.com/wu-sheng/DataCarrier/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wu-sheng%2FDataCarrier/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30226246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-07T18:12:09.766Z","status":"ssl_error","status_checked_at":"2026-03-07T18:11:58.786Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-01T10:49:33.829Z","updated_at":"2026-03-07T18:31:22.472Z","avatar_url":"https://github.com/wu-sheng.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DataCarrier\nDataCarrier is a light, embed, high-throughput, publish-subscribe MQ.\n\n**This project has been included and donated into Apache software foundation, as a part of [Apache SkyWalking](https://github.com/apache/incubator-skywalking).**\n\n[Sky-Walking APM](https://github.com/wu-sheng/sky-walking) uses **DataCarrier** in sky-walking agaent, one implementation version is used in OneAPM Ai commercial edition. See [skywalking version](https://github.com/wu-sheng/sky-walking/tree/master/apm-commons/apm-datacarrier).\n\n[![Coverage Status](https://coveralls.io/repos/github/wu-sheng/DataCarrier/badge.svg?branch=master\u0026q=2)](https://coveralls.io/github/wu-sheng/DataCarrier?branch=master\u0026q=3)\n[ ![Download](https://api.bintray.com/packages/wu-sheng/DataCarrier/com.a.eye.data-carrier/images/download.svg) ](https://bintray.com/wu-sheng/DataCarrier/com.a.eye.data-carrier/_latestVersion)\n\n## Why need DataCarrier\n- Publish-Subscribe In-Memory MQ. Support multi Producers and Consumer.\n- Light and Embed. A mini java lib, less than 20k, no other dependences.\n- High-throughput. Used in [Sky-Walking APM](https://github.com/wu-sheng/sky-walking).\n- Easy to use. Simple API\n- Only need jdk1.6\n\n## Download\n- [Download](https://bintray.com/wu-sheng/DataCarrier/com.a.eye.data-carrier/_latestVersion) latest version\n- Use Maven, Gradle, Ivy, SBT, etc. [set JCenter Center Repository](https://bintray.com/bintray/jcenter?filterByPkgName=com.a.eye.data-carrier)\n- maven\n```xml\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\" ?\u003e\n\u003csettings xsi:schemaLocation='http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd'\n          xmlns='http://maven.apache.org/SETTINGS/1.0.0' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'\u003e\n    \n    \u003cprofiles\u003e\n        \u003cprofile\u003e\n            \u003crepositories\u003e\n                \u003crepository\u003e\n                    \u003csnapshots\u003e\n                        \u003cenabled\u003efalse\u003c/enabled\u003e\n                    \u003c/snapshots\u003e\n                    \u003cid\u003ecentral\u003c/id\u003e\n                    \u003cname\u003ebintray\u003c/name\u003e\n                    \u003curl\u003ehttp://jcenter.bintray.com\u003c/url\u003e\n                \u003c/repository\u003e\n            \u003c/repositories\u003e\n            \u003cpluginRepositories\u003e\n                \u003cpluginRepository\u003e\n                    \u003csnapshots\u003e\n                        \u003cenabled\u003efalse\u003c/enabled\u003e\n                    \u003c/snapshots\u003e\n                    \u003cid\u003ecentral\u003c/id\u003e\n                    \u003cname\u003ebintray-plugins\u003c/name\u003e\n                    \u003curl\u003ehttp://jcenter.bintray.com\u003c/url\u003e\n                \u003c/pluginRepository\u003e\n            \u003c/pluginRepositories\u003e\n            \u003cid\u003ebintray\u003c/id\u003e\n        \u003c/profile\u003e\n    \u003c/profiles\u003e\n    \u003cactiveProfiles\u003e\n        \u003cactiveProfile\u003ebintray\u003c/activeProfile\u003e\n    \u003c/activeProfiles\u003e\n\u003c/settings\u003e\n```\n```\n\u003cdependency\u003e\n  \u003cgroupId\u003ecom.a.eye\u003c/groupId\u003e\n  \u003cartifactId\u003edata-carrier\u003c/artifactId\u003e\n  \u003cversion\u003ex.x\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n\n## How to use\n- create a new DataCarrier instance\n```java\n/**\n * channelSize = 5, bufferSize per channel = 5000\n */\nDataCarrier\u003cSampleData\u003e carrier = new DataCarrier\u003cSampleData\u003e(5, 5000);\n```\n\n- set message buffer strategy (optional)\n```java\n/**\n * default is BLOCKING\n * BLOCKING, waiting to set value to buffer, return when finished.\n * OVERRIDE, force to set value to buffer, return true forever.\n * IF_POSSIBLE, try to set value to buffer, return true when set successfully.\n */\ncarrier.setBufferStrategy(BufferStrategy.IF_POSSIBLE);\n```\n\n- set partitioner (optional)\n```java\n/**\n * default is SimpleRollingPartitioner\n * provided: ProducerThreadPartitioner, SimpleRollingPartitioner\n * you can create any partitioner, only need to implements IDataPartitioner interface\n */\ncarrier.setPartitioner(new ProducerThreadPartitioner\u003cSampleData\u003e());\n```\nref to [partitioner implements](src/main/java/com/a/eye/datacarrier/partition)\n\n- set consumer and start to consume data\n```java\n    /**\n     * set consumers to this Carrier.\n     * consumer begin to run when {@link DataCarrier\u003cT\u003e#produce(T)} begin to work.\n     *\n     * @param consumerClass class of consumer\n     * @param num           number of consumer threads\n     */\n    carrier.consume(SampleConsumer.class, 10);\n\nor\n\n    /**\n     * set consumers to this Carrier.\n     * consumer begin to run when {@link DataCarrier\u003cT\u003e#produce(T)} begin to work.\n     *\n     * @param consumer single instance of consumer, all consumer threads will all use this instance.\n     * @param num      number of consumer threads\n     * @return\n     */\n    carrier.consume(consumer, 10);\n```\n\n- create a consumer (sample)\n```java\npublic class SampleConsumer implements IConsumer\u003cSampleData\u003e {\n    public int i = 1;\n\n    @Override\n    public void init() {\n\n    }\n\n    @Override\n    public void consume(List\u003cSampleData\u003e data) {\n        for(SampleData one : data) {\n            one.setIntValue(this.hashCode());\n            ConsumerTest.buffer.offer(one);\n        }\n    }\n\n    @Override\n    public void onError(List\u003cSampleData\u003e data, Throwable t) {\n\n    }\n    \n    @Override\n    public void onExit() {\n\n    }\n}\n```\n\n- produce messages as you need (sample)\n```java\nfor (int i = 0; i \u003c 200; i++) {\n    carrier.produce(new SampleData());\n}\n```\n\n## Doc\n[(中文)SkyWalking子项目--DataCarrier 1.0 解读  ](http://wu-sheng.iteye.com/blog/2334404)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwu-sheng%2Fdatacarrier","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwu-sheng%2Fdatacarrier","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwu-sheng%2Fdatacarrier/lists"}