{"id":15127521,"url":"https://github.com/txn2/rxtx","last_synced_at":"2025-10-23T06:30:18.805Z","repository":{"id":101890157,"uuid":"129156604","full_name":"txn2/rxtx","owner":"txn2","description":"Queue based data collector, transmitter with store-and-forward. Useful for online/offline data collection, back pressure buffering or general queuing. REST / HTTP post.","archived":false,"fork":false,"pushed_at":"2024-12-27T20:18:48.000Z","size":3477,"stargazers_count":18,"open_issues_count":0,"forks_count":5,"subscribers_count":4,"default_branch":"v2.x","last_synced_at":"2025-02-05T10:45:03.861Z","etag":null,"topics":["data-collector","docker-container","docker-image","golang","iot","iot-application","iot-data","json","json-api","message-queue","mq","rest-api","rxtx","transmitter"],"latest_commit_sha":null,"homepage":"https://txn2.com","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/txn2.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-04-11T21:31:51.000Z","updated_at":"2024-12-27T20:18:49.000Z","dependencies_parsed_at":"2024-01-05T11:31:24.499Z","dependency_job_id":"0903cdd2-d4b0-4cfc-a65b-fdf8bb5b5beb","html_url":"https://github.com/txn2/rxtx","commit_stats":null,"previous_names":["cjimti/rxtx"],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/txn2%2Frxtx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/txn2%2Frxtx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/txn2%2Frxtx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/txn2%2Frxtx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/txn2","download_url":"https://codeload.github.com/txn2/rxtx/tar.gz/refs/heads/v2.x","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237784851,"owners_count":19365943,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-collector","docker-container","docker-image","golang","iot","iot-application","iot-data","json","json-api","message-queue","mq","rest-api","rxtx","transmitter"],"created_at":"2024-09-26T02:04:53.168Z","updated_at":"2025-10-23T06:30:18.465Z","avatar_url":"https://github.com/txn2.png","language":"Go","readme":"![rxtx data transmission](mast.jpg)\n[![irsync Release](https://img.shields.io/github/release/txn2/rxtx.svg)](https://github.com/txn2/rxtx/releases)\n[![Build Status](https://travis-ci.org/txn2/rxtx.svg?branch=master)](https://travis-ci.org/txn2/rxtx)\n[![Go Report Card](https://goreportcard.com/badge/github.com/txn2/rxtx)](https://goreportcard.com/report/github.com/txn2/rxtx)\n[![Maintainability](https://api.codeclimate.com/v1/badges/c4cbc94c46027f0e3161/maintainability)](https://codeclimate.com/github/txn2/rxtx/maintainability)\n[![GoDoc](https://godoc.org/github.com/txn2/irsync/rxtx?status.svg)](https://godoc.org/github.com/txn2/rxtx/rtq)\n[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Ftxn2%2Frxtx.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Ftxn2%2Frxtx?ref=badge_shield)\n\n[![Docker Container Image Size](https://shields.beevelop.com/docker/image/image-size/txn2/rxtx/latest.svg)](https://hub.docker.com/r/txn2/irsync/)\n[![Docker Container Layers](https://shields.beevelop.com/docker/image/layers/txn2/rxtx/latest.svg)](https://hub.docker.com/r/txn2/irsync/)\n[![Docker Container Pulls](https://img.shields.io/docker/pulls/txn2/rxtx.svg)](https://hub.docker.com/r/txn2/rxtx/)\n\n# rxtx\n**rxtx** is a queue based data collector \u003e data transmitter. Useful for online/offline data collection, back pressure buffering or general queuing. **rxtx** uses [bbolt](https://github.com/coreos/bbolt) maintained by CoreOs, a single file database for storing messages before they can be sent.\n\n[rtbeat](https://github.com/txn2/rtbeat) was developed to consume **rxtx** POST data and publish as events into [elasticsearch], [logstash], [kafka], [redis] or directly to log files.\n\n## Test on MacOs\n\n### Install with [brew]\n```bash\nbrew tap txn2/homebrew-tap\nbrew install rxtx\n```\n\n### Help\n```bash\nrxtx -h\n```\n\n## Test [Docker] Container\n\n### Help\n```bash\ndocker run --rm -it txn2/rxtx -h\n```\non arm 6/7 based device:\n```bash\ndocker run --rm -it txn2/rxtx:arm32v6-1.2.0 -h\n```\n\n\n## Test Source\n\n#### Help\n```bash\ngo run ./rxtx.go -h\n\nUsage of rxtx:\n  -batch int\n        Batch size. (default 5000)\n  -ingest string\n        Ingest server. (default \"http://localhost:8081/in\")\n  -interval int\n        Seconds between intervals. (default 30)\n  -maxq int\n        Max number of message in queue. (default 2000000)\n  -name string\n        Service name. (default \"rxtx\")\n  -path string\n        Directory to store database. (default \"./\")\n  -port string\n        Server port. (default \"8080\")\n\n```\n\n#### Start server on 8080\n```bash\ngo run ./rxtx.go \n```\n\n#### Add message to queue\n\nThe **rxtx** services accepts HTTP **POST** data to an API endpoint in the following form /rx/**PRODUCER**/**KEY**/**LABEL/...**/. One label is required, however as many labels as necessary may be added, separated by a forward slash.\n\n```bash\ncurl -w \"\\n\" -d \"{\\\"generic\\\": \\\"$RANDOM\\\"}\" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data\n```\n\n#### Add message to queue every second\n```bash\n while true; do curl -w \"\\n\" -d \"{\\\"generic\\\": \\\"$RANDOM\\\"}\" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data; sleep 1; done\n ```\n\n#### Add 1000 messages to the queue.\n```bash\n time for i in {1..1000}; do curl -w \"\\n\" -d \"{\\\"generic\\\": \\\"$RANDOM\\\"}\" -X POST http://localhost:8080/rx/me/generic_data/generic/test/data; done\n```\n#### Example Batch\n\n```json\n{  \n   \"uuid\":\"d7642975-4241-4f43-b704-b67621b184b8\",\n   \"size\":4,\n   \"messages\":[  \n      {  \n         \"seq\":\"2019072100000000038\",\n         \"time\":\"2019-07-21T02:44:49.560087Z\",\n         \"uuid\":\"a93cfe42-79c5-4a32-9dc0-6b77b68d0926\",\n         \"producer\":\"me\",\n         \"label\":\"/generic/test/data\",\n         \"key\":\"generic_data\",\n         \"payload\":{  \n            \"generic\":\"6955\"\n         }\n      },\n      {  \n         \"seq\":\"2019072100000000039\",\n         \"time\":\"2019-07-21T02:44:50.6214579Z\",\n         \"uuid\":\"66042da0-b0c1-447c-ab0b-66c887e8b56a\",\n         \"producer\":\"me\",\n         \"label\":\"/generic/test/data\",\n         \"key\":\"generic_data\",\n         \"payload\":{  \n            \"generic\":\"8923\"\n         }\n      },\n      {  \n         \"seq\":\"2019072100000000040\",\n         \"time\":\"2019-07-21T02:44:51.622978Z\",\n         \"uuid\":\"0ec3f6d7-905f-42bc-896e-515bf8f2a06d\",\n         \"producer\":\"me\",\n         \"label\":\"/generic/test/data\",\n         \"key\":\"generic_data\",\n         \"payload\":{  \n            \"generic\":\"28324\"\n         }\n      },\n      {  \n         \"seq\":\"2019072100000000046\",\n         \"time\":\"2019-07-21T02:44:57.8990519Z\",\n         \"uuid\":\"bfa2d06d-a8f4-4125-93bc-d08a070079d7\",\n         \"producer\":\"me\",\n         \"label\":\"/generic/test/data\",\n         \"key\":\"generic_data\",\n         \"payload\":{  \n            \"generic\":\"31460\"\n         }\n      }\n   ]\n}\n```\n\n\n### Profile\n\n```bash\ngo build ./rxtx.go \u0026\u0026 time ./rxtx --path=./data/ --cpuprofile=rxtxcpu.prof --memprofile=rxtxmem.prof\n```\n\nBrowser-based profile viewer:\n```bash\ngo tool pprof -http=:8081 rxtxcpu.prof\n```\n\n### Building and Releasing\n\n**rxtx** uses [GORELEASER] to build binaries and [Docker] containers.\n\n#### Test Release Steps\n\nInstall [GORELEASER] with [brew] (MacOS):\n```bash\nbrew install goreleaser/tap/goreleaser\n```\n\nBuild without releasing:\n```bash\ngoreleaser --skip-publish --rm-dist --skip-validate\n```\n\n#### Release Steps\n\n- Commit latest changes\n- [Tag] a version `git tag -a v2.0 -m \"Version 2.0\"`\n- Push tag `git push origin v2.0`\n- Run: `GITHUB_TOKEN=$GITHUB_TOKEN goreleaser --rm-dist`\n\n## Resources\n\n- [elasticsearch]\n- [logstash]\n- [kafka]\n- [redis]\n- [GORELEASER]\n- [Docker]\n- [homebrew]\n\n## License\n[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Ftxn2%2Frxtx.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Ftxn2%2Frxtx?ref=badge_large)\n\n[homebrew]: https://brew.sh/\n[brew]: https://brew.sh/\n[GORELEASER]: https://goreleaser.com/\n[Docker]: https://www.docker.com/\n[Tag]: https://git-scm.com/book/en/v2/Git-Basics-Tagging\n[elasticsearch]: https://www.elastic.co/\n[logstash]: https://www.elastic.co/products/logstash\n[kafka]: https://kafka.apache.org/\n[redis]: https://redis.io/\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftxn2%2Frxtx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftxn2%2Frxtx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftxn2%2Frxtx/lists"}