{"id":15998811,"url":"https://github.com/nikolaydubina/rchan","last_synced_at":"2025-10-21T05:30:28.657Z","repository":{"id":151420736,"uuid":"623951355","full_name":"nikolaydubina/rchan","owner":"nikolaydubina","description":"Go channel through Redis List","archived":true,"fork":false,"pushed_at":"2024-07-15T04:20:24.000Z","size":58,"stargazers_count":5,"open_issues_count":2,"forks_count":2,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-27T16:11:55.705Z","etag":null,"topics":["channel","experiment","go","optimism","performanse","queue","redis","redis-client","redis-queue","research"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nikolaydubina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-05T12:39:10.000Z","updated_at":"2024-09-26T04:55:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"a8291040-9a44-4984-b05d-c6da9672fe7b","html_url":"https://github.com/nikolaydubina/rchan","commit_stats":null,"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikolaydubina%2Frchan","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikolaydubina%2Frchan/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikolaydubina%2Frchan/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nikolaydubina%2Frchan/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nikolaydubina","download_url":"https://codeload.github.com/nikolaydubina/rchan/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237436607,"owners_count":19309946,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["channel","experiment","go","optimism","performanse","queue","redis","redis-client","redis-queue","research"],"created_at":"2024-10-08T08:22:42.618Z","updated_at":"2025-10-21T05:30:28.351Z","avatar_url":"https://github.com/nikolaydubina.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## rchan: Go channel through Redis List\n\n[![Go Report Card](https://goreportcard.com/badge/github.com/nikolaydubina/rchan)](https://goreportcard.com/report/github.com/nikolaydubina/rchan)\n[![Go Reference](https://pkg.go.dev/badge/github.com/nikolaydubina/rchan.svg)](https://pkg.go.dev/github.com/nikolaydubina/rchan)\n[![OpenSSF Scorecard](https://api.securityscorecards.dev/projects/github.com/nikolaydubina/rchan/badge)](https://securityscorecards.dev/viewer/?uri=github.com/nikolaydubina/rchan)\n\n\u003e Disclaimer: this is experiment to see channel based API for distributed queues. Channel technically is a function call that does not error out. So effectively sending to or reading from distributed queue through channel is _delaying_ or _buffering_ calls to distributied message broker. It is possible that this can lead to better performance and higher throughput. However, this project is more of an experiment of API design with goal to make workable solution with minimal code. Very likely you may want to use function based API in production.\n\n* 100 LOC\n* 30 _thousand_ RPS (send/receive individual message)\n* 1.4 _million_ RPS (send/receive batch pipeline)\n* graceful stop (no messages lost, unless error)\n* integration test\n\n```go\nrdb := redis.NewClient(\u0026redis.Options{Addr: \"localhost:6379\"})\n\n// Alice\nw := rchan.NewWriter[string](rdb, \"my-queue\", 10000, 10, 1, time.Millisecond*100)\nw \u003c- \"hello world 🌏🤍✨\"\n\n// ... 🌏 ...\n\n// Bob\nr, _ := rchan.NewReader[string](rdb, \"my-queue\", 10000, 10, 1, time.Millisecond*100)\nfmt.Println(\u003c-r)\n// Output: hello world 🌏🤍✨\n```\n\n## Benchmarks\n\n```bash\nREDIS_HOST=localhost REDIS_PORT=6379 go test -bench=. -benchmem .\n```\n\n```\ngoos: darwin\ngoarch: arm64\npkg: github.com/nikolaydubina/rchan\nBenchmarkBatchSendReceive/buff-10-batch-10-string--10 287922 4221 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.033 receive_MB/s 310 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-10-batch-10-bytes---10 249921 4009 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.141 receive_MB/s 350 B/op 10 allocs/op \nBenchmarkBatchSendReceive/buff-100-batch-10-string--10 280068 4131 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.078 receive_MB/s 310 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-100-batch-10-bytes---10 282595 4182 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.052 receive_MB/s 350 B/op 10 allocs/op \nBenchmarkBatchSendReceive/buff-100-batch-100-string--10 1021838 1108 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.745 receive_MB/s 256 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-100-batch-100-bytes---10 939798 1172 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.320 receive_MB/s 296 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-10-string--10 286066 4069 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.109 receive_MB/s 310 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-10-bytes---10 284227 4230 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.029 receive_MB/s 350 B/op 10 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-100-string--10 894435 1172 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.326 receive_MB/s 256 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-100-bytes---10 964352 1156 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.423 receive_MB/s 296 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-1000-string--10 1444040 767.7 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 11.18 receive_MB/s 259 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-1000-batch-1000-bytes---10 1451476 762.2 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 11.26 receive_MB/s 299 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-10-string--10 267046 4113 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.087 receive_MB/s 310 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-10-bytes---10 281936 4043 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 2.123 receive_MB/s 352 B/op 10 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-100-string--10 1039837 1095 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.835 receive_MB/s 256 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-100-bytes---10 974930 1144 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 7.503 receive_MB/s 296 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-1000-string--10 1462742 767.1 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 11.19 receive_MB/s 259 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-1000-bytes---10 1363063 811.4 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 10.58 receive_MB/s 299 B/op 8 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-10000-string--10 1622870 752.2 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 11.41 receive_MB/s 277 B/op 6 allocs/op \nBenchmarkBatchSendReceive/buff-10000-batch-10000-bytes---10 1630532 743.0 ns/op 100.0 receive+queue/% 100.0 receive/% 9.000 receive_B/op 11.55 receive_MB/s 317 B/op 8 allocs/op \nPASS\nok github.com/nikolaydubina/rchan 40.939s\n```\n\n### localhost\n\n8.7 GB/s\n\n```bash\niperf3 -s -p 3000 \n```\n\n```\n-----------------------------------------------------------\nServer listening on 3000 (test #1)\n-----------------------------------------------------------\nAccepted connection from ::1, port 59020\n[ 5] local ::1 port 3000 connected to ::1 port 59021\n[ ID] Interval Transfer Bitrate\n[ 5] 0.00-1.00 sec 7.43 GBytes 63.8 Gbits/sec \n[ 5] 1.00-2.00 sec 8.39 GBytes 72.0 Gbits/sec \n[ 5] 2.00-3.00 sec 8.79 GBytes 75.5 Gbits/sec \n[ 5] 3.00-4.00 sec 8.71 GBytes 74.8 Gbits/sec \n[ 5] 4.00-5.00 sec 8.75 GBytes 75.2 Gbits/sec \n[ 5] 5.00-6.00 sec 8.76 GBytes 75.3 Gbits/sec \n[ 5] 6.00-7.00 sec 8.26 GBytes 70.9 Gbits/sec \n[ 5] 7.00-8.00 sec 8.62 GBytes 74.0 Gbits/sec \n[ 5] 8.00-9.00 sec 8.49 GBytes 72.9 Gbits/sec \n[ 5] 9.00-10.00 sec 8.85 GBytes 76.0 Gbits/sec \n[ 5] 10.00-10.00 sec 2.06 MBytes 56.7 Gbits/sec \n- - - - - - - - - - - - - - - - - - - - - - - - -\n[ ID] Interval Transfer Bitrate\n[ 5] 0.00-10.00 sec 85.0 GBytes 73.0 Gbits/sec receiver\n-----------------------------------------------------------\n```\n\n```bash\niperf3 -c localhost -p 3000 -f M\n```\n\n```\nConnecting to host localhost, port 3000\n[ 7] local ::1 port 59021 connected to ::1 port 3000\n[ ID] Interval Transfer Bitrate\n[ 7] 0.00-1.00 sec 7.43 GBytes 7608 MBytes/sec \n[ 7] 1.00-2.00 sec 8.39 GBytes 8588 MBytes/sec \n[ 7] 2.00-3.00 sec 8.79 GBytes 8998 MBytes/sec \n[ 7] 3.00-4.00 sec 8.71 GBytes 8916 MBytes/sec \n[ 7] 4.00-5.00 sec 8.75 GBytes 8961 MBytes/sec \n[ 7] 5.00-6.00 sec 8.76 GBytes 8972 MBytes/sec \n[ 7] 6.00-7.00 sec 8.26 GBytes 8456 MBytes/sec \n[ 7] 7.00-8.00 sec 8.62 GBytes 8823 MBytes/sec \n[ 7] 8.00-9.00 sec 8.49 GBytes 8691 MBytes/sec \n[ 7] 9.00-10.00 sec 8.85 GBytes 9064 MBytes/sec \n- - - - - - - - - - - - - - - - - - - - - - - - -\n[ ID] Interval Transfer Bitrate\n[ 7] 0.00-10.00 sec 85.0 GBytes 8708 MBytes/sec sender\n[ 7] 0.00-10.00 sec 85.0 GBytes 8708 MBytes/sec receiver\n\niperf Done.\n```\n\n## Appendix A: Paths Not Taken\n\n### Single function that returns writer and reader channels\nSome call-sites do not use reader at all.\nUsually it is either only reader or writer, if you have both, you can use normal channel.\nReturning both reader and writer causes to read from Redis List into anonymous reader channel that is not used.\nIt keeps data there up to buffer size and it is not processed by anyone.\nSo to keep logic simple, instantiate what you need — either reader or writer.\nThis also helps to avoid duplication of parameters of buffering for reader vs writer.\n\n### Single channel return for reader\nWe need to tell reader goroutine that no more reads will ever be performed and send anything in channel back to Redis List.\nFor example, for graceful termination.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolaydubina%2Frchan","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnikolaydubina%2Frchan","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolaydubina%2Frchan/lists"}