{"id":13590326,"url":"https://github.com/cloudwego/shmipc-go","last_synced_at":"2025-04-08T09:06:01.606Z","repository":{"id":150556295,"uuid":"610154928","full_name":"cloudwego/shmipc-go","owner":"cloudwego","description":"A high performance inter-process communication golang library developed by CloudWeGo","archived":false,"fork":false,"pushed_at":"2024-10-28T06:31:21.000Z","size":132,"stargazers_count":445,"open_issues_count":10,"forks_count":34,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-01T07:45:13.340Z","etag":null,"topics":["networking","tcp","unix"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudwego.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-06T07:50:59.000Z","updated_at":"2025-04-01T03:53:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"eae43351-e267-4128-ac48-ca0ea162eb99","html_url":"https://github.com/cloudwego/shmipc-go","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fshmipc-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fshmipc-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fshmipc-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudwego%2Fshmipc-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudwego","download_url":"https://codeload.github.com/cloudwego/shmipc-go/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247809963,"owners_count":20999816,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["networking","tcp","unix"],"created_at":"2024-08-01T16:00:43.622Z","updated_at":"2025-04-08T09:06:01.579Z","avatar_url":"https://github.com/cloudwego.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\n# Shmipc\n\nEnglish | [中文](README_CN.md)\n\n## Introduction\n\nShmipc is a high performance inter-process communication library developed by ByteDance.\nIt is built on Linux's shared memory technology and uses unix or tcp connection to do process synchronization and finally implements zero copy communication across inter-processes. \nIn IO-intensive or large-package scenarios, it has better performance.\n\n## Features\n\n### Zero copy\n\nIn an industrial production environment, the unix domain socket and tcp loopback are often used in inter-process communication.The read operation or the write operation need copy data between user space buffer and kernel space buffer.But shmipc directly store data to the share memory, so no copy compared to the former.\n\n### batch IO\n\nAn IO queue was mapped to share memory, which describe the metadata of communication data.\nso that a process could put many request to the IO queue, and other process  could handle a batch IO per synchronization. It could effectively reduce the system calls which was brought by process synchronization.\n\n## Performance Testing\n\nThe source code bench_test.go, doing a performance comparison between shmipc and unix domain in ping-pong scenario with different package size. The result is as follows: having a performance improvement whatever small package or large package.\n\n```\ngo test -bench=BenchmarkParallelPingPong -run BenchmarkParallelPingPong\ngoos: linux\ngoarch: amd64\npkg: github.com/cloudwego/shmipc-go\ncpu: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz\nBenchmarkParallelPingPongByShmipc64B-40      \t  733821\t      1970 ns/op\t  64.97 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc512B-40     \t  536190\t      1990 ns/op\t 514.45 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc1KB-40      \t  540517\t      2045 ns/op\t1001.62 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc4KB-40      \t  509047\t      2063 ns/op\t3970.91 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc16KB-40     \t  590398\t      1996 ns/op\t16419.46 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc32KB-40     \t  607756\t      1937 ns/op\t33829.82 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc64KB-40     \t  609824\t      1995 ns/op\t65689.31 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc256KB-40    \t  622755\t      1793 ns/op\t292363.56 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc512KB-40    \t  695401\t      1993 ns/op\t526171.77 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc1MB-40      \t  538208\t      1873 ns/op\t1119401.64 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByShmipc4MB-40      \t  606144\t      1891 ns/op\t4436936.93 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds64B-40         \t  446019\t      2657 ns/op\t  48.18 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds512B-40        \t  450124\t      2665 ns/op\t 384.30 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds1KB-40         \t  446389\t      2680 ns/op\t 764.29 MB/s\t       0 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds4KB-40         \t  383552\t      3093 ns/op\t2648.83 MB/s\t       1 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds16KB-40        \t  307816\t      3884 ns/op\t8436.27 MB/s\t       8 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds64KB-40        \t  103027\t     10259 ns/op\t12776.17 MB/s\t     102 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds256KB-40       \t   25286\t     46352 ns/op\t11311.01 MB/s\t    1661 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds512KB-40       \t    9788\t    122873 ns/op\t8533.84 MB/s\t    8576 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds1MB-40         \t    4177\t    283729 ns/op\t7391.38 MB/s\t   40178 B/op\t       0 allocs/op\nBenchmarkParallelPingPongByUds4MB-40         \t     919\t   1253338 ns/op\t6693.01 MB/s\t  730296 B/op\t       1 allocs/op\nPASS\nok  \tgithub.com/cloudwego/shmipc\t42.138s\n\n```\n\n- BenchmarkParallelPingPongByUds, the ping-pong communication base on Unix domain socket.\n- BenchmarkParallelPingPongByShmipc, the ping-pong communication base on shmipc.\n- the suffix of the testing case name is the package size of communication, which from 64 Byte to 4 MB.\n\n### Quick start\n\n#### HelloWorld\n\n- [HelloWorldClient](https://github.com/cloudwego/shmipc-go/blob/main/example/helloworld/greeter_client/main.go)\n- [HelloWorldServer](https://github.com/cloudwego/shmipc-go/blob/main/example/helloworld/greeter_server/main.go)\n\n#### Integrate with application\n\n- [serialization and deserialization](https://github.com/cloudwego/shmipc-go/blob/main/example/best_practice/idl/example.go)\n- [client which using synchronous interface](https://github.com/cloudwego/shmipc-go/blob/main/example/best_practice/shmipc_client/main.go)\n- [server which using synchronous interface](https://github.com/cloudwego/shmipc-go/blob/main/example/best_practice/shmipc_server/main.go)\n- [client which using asynchronous interface](https://github.com/cloudwego/shmipc-go/blob/main/example/best_practice/shmipc_async_client/client.go)\n- [server which using asynchronous interface](https://github.com/cloudwego/shmipc-go/blob/main/example/best_practice/shmipc_async_server/server.go)\n\n#### HotRestart\n\n[hot restart demo](https://github.com/cloudwego/shmipc-go/blob/main/example/hot_restart_test/README.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudwego%2Fshmipc-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudwego%2Fshmipc-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudwego%2Fshmipc-go/lists"}