{"id":13413594,"url":"https://github.com/xtaci/gaio","last_synced_at":"2025-04-12T17:46:11.982Z","repository":{"id":45631291,"uuid":"229195107","full_name":"xtaci/gaio","owner":"xtaci","description":"High performance minimalism async-io(proactor) networking for Golang.","archived":false,"fork":false,"pushed_at":"2024-08-27T03:44:52.000Z","size":443,"stargazers_count":584,"open_issues_count":16,"forks_count":68,"subscribers_count":19,"default_branch":"master","last_synced_at":"2024-08-27T04:51:12.833Z","etag":null,"topics":["asyncio","c10k","context-switching","epoll","eventfd","kqueue","netpoll","proactor","syscall-dup"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xtaci.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-20T05:19:00.000Z","updated_at":"2024-08-27T03:44:55.000Z","dependencies_parsed_at":"2024-06-18T13:53:42.362Z","dependency_job_id":"b01a3d48-2c3d-4153-9882-853a2b95e37e","html_url":"https://github.com/xtaci/gaio","commit_stats":null,"previous_names":["xtaci/aio"],"tags_count":59,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xtaci%2Fgaio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xtaci%2Fgaio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xtaci%2Fgaio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xtaci%2Fgaio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xtaci","download_url":"https://codeload.github.com/xtaci/gaio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248609642,"owners_count":21132916,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","c10k","context-switching","epoll","eventfd","kqueue","netpoll","proactor","syscall-dup"],"created_at":"2024-07-30T20:01:44.059Z","updated_at":"2025-04-12T17:46:11.960Z","avatar_url":"https://github.com/xtaci.png","language":"Go","readme":"# gaio\n\n[![GoDoc][1]][2] [![MIT licensed][3]][4] [![Build Status][5]][6] [![Go Report Card][7]][8] [![Coverage Statusd][9]][10]\n\n[1]: https://godoc.org/github.com/xtaci/gaio?status.svg\n[2]: https://pkg.go.dev/github.com/xtaci/gaio\n[3]: https://img.shields.io/badge/license-MIT-blue.svg\n[4]: LICENSE\n[5]: https://img.shields.io/github/created-at/xtaci/gaio\n[6]: https://img.shields.io/github/created-at/xtaci/gaio\n[7]: https://goreportcard.com/badge/github.com/xtaci/gaio\n[8]: https://goreportcard.com/report/github.com/xtaci/gaio\n[9]: https://codecov.io/gh/xtaci/gaio/branch/master/graph/badge.svg\n[10]: https://codecov.io/gh/xtaci/gaio\n\n\u003cimg src=\"assets/logo.png\" alt=\"gaio\" height=\"300px\" /\u003e \n\n## Introduction\n[中文介绍](https://zhuanlan.zhihu.com/p/102890337)\n\nIn a typical Go network program, you start by accepting a connection with `conn := lis.Accept()`, then initiate a goroutine to handle incoming data using `go func(net.Conn)`. Next, you allocate a buffer with `buf := make([]byte, 4096)` and wait for data with `conn.Read(buf)`.\n\nFor a server managing over 10,000 connections with frequent short messages (e.g., \u003c512 bytes), the cost of context switching becomes significantly higher than that of receiving messages—each context switch can require over 1,000 CPU cycles or around 600 ns on a 2.1 GHz processor.\n\nBy eliminating one goroutine per connection through Edge-Triggered I/O Multiplexing, you can save the 2KB (R) + 2KB (W) stack space typically used per goroutine. Additionally, by employing an internal swap buffer, you can avoid the need for `buf := make([]byte, 4096)` at the expense of some performance.\n\nThe gaio library implements the proactor pattern, effectively addressing both memory constraints and performance objectives.\n\n## Features\n\n- **High Performance:** Tested in High Frequency Trading environments, handling 30K–40K RPS on a single HVM server.\n- **Scalability:** Designed for over C10K concurrent connections, optimizing both parallelism and single connection throughput.\n- **Flexible Buffering:** Use `Read(ctx, conn, buffer)` with a nil buffer to leverage the internal swap buffer.\n- **Non-Intrusive Integration:** Compatible with `net.Listener` and `net.Conn` (supports `syscall.RawConn`), allowing easy integration into existing applications.\n- **Efficient Context Switching:** Minimizes context switching costs for small messages, ideal for frequent chat message exchanges.\n- **Customizable Delegation:** Applications can choose when to delegate `net.Conn` to gaio, such as after a handshake or specific `net.TCPConn` settings.\n- **Back-Pressure Handling:** Applications can control when to submit read or write requests, enabling per-connection back-pressure to slow down sending when necessary, particularly useful for transferring data from a faster source (A) to a slower destination (B).\n- **Lightweight and Maintainable:** Approximately 1,000 lines of code, making it easy to debug.\n- **Cross-Platform Support:** Compatible with Linux and BSD.\n\n## Conventions\n\n- **Connection Delegation:** Once you submit an async read/write request with a related `net.Conn` to `gaio.Watcher`, that connection is delegated to the watcher. Subsequent calls to `conn.Read` or `conn.Write` will return an error, but TCP properties set by `SetReadBuffer()`, `SetWriteBuffer()`, `SetLinger()`, `SetKeepAlive()`, and `SetNoDelay()` will be retained.\n  \n- **Resource Management:** If you no longer need a connection, call `Watcher.Free(net.Conn)` to immediately close the socket and free resources. If you forget to call `Watcher.Free`, the runtime garbage collector will clean up system resources if `net.Conn` is not referenced elsewhere. Failing to call `Watcher.Close()` will lead the garbage collector to clean up all related resources if the watcher is unreferenced.\n\n- **Load Balancing:** For connection load balancing, create multiple `gaio.Watcher` instances to distribute `net.Conn` using your preferred strategy. For acceptor load balancing, utilize `go-reuseport` as the listener.\n\n- **Safe Read Requests:** When submitting read requests with a 'nil' buffer, the returned `[]byte` from `Watcher.WaitIO()` is safe to use until the next call to `Watcher.WaitIO()`.\n\n## TL;DR\n\n```go\npackage main\n\nimport (\n        \"log\"\n        \"net\"\n\n        \"github.com/xtaci/gaio\"\n)\n\n// this goroutine will wait for all io events, and sents back everything it received\n// in async way\nfunc echoServer(w *gaio.Watcher) {\n        for {\n                // loop wait for any IO events\n                results, err := w.WaitIO()\n                if err != nil {\n                        log.Println(err)\n                        return\n                }\n\n                for _, res := range results {\n                        switch res.Operation {\n                        case gaio.OpRead: // read completion event\n                                if res.Error == nil {\n                                        // send back everything, we won't start to read again until write completes.\n                                        // submit an async write request\n                                        w.Write(nil, res.Conn, res.Buffer[:res.Size])\n                                }\n                        case gaio.OpWrite: // write completion event\n                                if res.Error == nil {\n                                        // since write has completed, let's start read on this conn again\n                                        w.Read(nil, res.Conn, res.Buffer[:cap(res.Buffer)])\n                                }\n                        }\n                }\n        }\n}\n\nfunc main() {\n        w, err := gaio.NewWatcher()\n        if err != nil {\n              log.Fatal(err)\n        }\n        defer w.Close()\n\t\n        go echoServer(w)\n\n        ln, err := net.Listen(\"tcp\", \"localhost:0\")\n        if err != nil {\n                log.Fatal(err)\n        }\n        log.Println(\"echo server listening on\", ln.Addr())\n\n        for {\n                conn, err := ln.Accept()\n                if err != nil {\n                        log.Println(err)\n                        return\n                }\n                log.Println(\"new client\", conn.RemoteAddr())\n\n                // submit the first async read IO request\n                err = w.Read(nil, conn, make([]byte, 128))\n                if err != nil {\n                        log.Println(err)\n                        return\n                }\n        }\n}\n\n```\n\n### More examples\n\n\u003cdetails\u003e\n\t\u003csummary\u003e Push server \u003c/summary\u003e\n        package main\n\n```go\npackage main\n\nimport (\n        \"fmt\"\n        \"log\"\n        \"net\"\n        \"time\"\n\n        \"github.com/xtaci/gaio\"\n)\n\nfunc main() {\n        // by simply replace net.Listen with reuseport.Listen, everything is the same as in push-server\n        // ln, err := reuseport.Listen(\"tcp\", \"localhost:0\")\n        ln, err := net.Listen(\"tcp\", \"localhost:0\")\n        if err != nil {\n                log.Fatal(err)\n        }\n\n        log.Println(\"pushing server listening on\", ln.Addr(), \", use telnet to receive push\")\n\n        // create a watcher\n        w, err := gaio.NewWatcher()\n        if err != nil {\n                log.Fatal(err)\n        }\n\n        // channel\n        ticker := time.NewTicker(time.Second)\n        chConn := make(chan net.Conn)\n        chIO := make(chan gaio.OpResult)\n\n        // watcher.WaitIO goroutine\n        go func() {\n                for {\n                        results, err := w.WaitIO()\n                        if err != nil {\n                                log.Println(err)\n                                return\n                        }\n\n                        for _, res := range results {\n                                chIO \u003c- res\n                        }\n                }\n        }()\n\n        // main logic loop, like your program core loop.\n        go func() {\n                var conns []net.Conn\n                for {\n                        select {\n                        case res := \u003c-chIO: // receive IO events from watcher\n                                if res.Error != nil {\n                                        continue\n                                }\n                                conns = append(conns, res.Conn)\n                        case t := \u003c-ticker.C: // receive ticker events\n                                push := []byte(fmt.Sprintf(\"%s\\n\", t))\n                                // all conn will receive the same 'push' content\n                                for _, conn := range conns {\n                                        w.Write(nil, conn, push)\n                                }\n                                conns = nil\n                        case conn := \u003c-chConn: // receive new connection events\n                                conns = append(conns, conn)\n                        }\n                }\n        }()\n\n        // this loop keeps on accepting connections and send to main loop\n        for {\n                conn, err := ln.Accept()\n                if err != nil {\n                        log.Println(err)\n                        return\n                }\n                chConn \u003c- conn\n        }\n}\n\n```\n\u003c/details\u003e\n\n## Documentation\n\nFor complete documentation, see the associated [Godoc](https://godoc.org/github.com/xtaci/gaio).\n\n## Benchmarks\n\n| Test Case | Throughput test with 64KB buffer |\n|:-------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| Description | A client keep on sending 64KB bytes to server, server keeps on reading and sending back whatever it received, the client keeps on receiving whatever the server sent back until all bytes received successfully |\n| Command | `go test -v -run=^$ -bench Echo` |\n| Macbook Pro | 1695.27 MB/s 518 B/op 4 allocs/op|\n| Linux AMD64 | 1883.23 MB/s 518 B/op 4 allocs/op|\n| Raspberry Pi4 | 354.59 MB/s 334 B/op 4 allocs/op|\n\n| Test Case | 8K concurrent connection echo test |\n|:-------------:|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n|Description| Start 8192 clients, each client send 1KB data to server, server keeps on reading and sending back whatever it received, the client keeps on receiving whatever the server sent back until all bytes received successfully.|\n| Command | `go test -v -run=8k` |\n| Macbook Pro | 1.09s |\n| Linux AMD64 | 0.94s |\n| Raspberry Pi4 | 2.09s |\n\n## Testing Directives\nOn MacOS, you need to increase the max open files limit to run the benchmarks.\n\n```bash\nsysctl -w kern.ipc.somaxconn=4096\nsysctl -w kern.maxfiles=100000\nsysctl -w kern.maxfilesperproc=100000\nsysctl -w net.inet.ip.portrange.first=1024\nsysctl -w net.inet.ip.portrange.last=65535\n\nulimit -S -n 65536\n```\n\n### Regression\n\n![regression](assets/regression.png)\n\nX -\u003e number of concurrent connections, Y -\u003e time of completion in seconds\n\n```\nBest-fit values\t \nSlope\t8.613e-005 ± 5.272e-006\nY-intercept\t0.08278 ± 0.03998\nX-intercept\t-961.1\n1/Slope\t11610\n \n95% Confidence Intervals\t \nSlope\t7.150e-005 to 0.0001008\nY-intercept\t-0.02820 to 0.1938\nX-intercept\t-2642 to 287.1\n \nGoodness of Fit\t \nR square\t0.9852\nSy.x\t0.05421\n \nIs slope significantly non-zero?\t \nF\t266.9\nDFn,DFd\t1,4\nP Value\t\u003c 0.0001\nDeviation from horizontal?\tSignificant\n \nData\t \nNumber of XY pairs\t6\nEquation\tY = 8.613e-005*X + 0.08278\n```\n\n## FAQ\n1. if you encounter something like:\n\n```\n# github.com/xtaci/gaio [github.com/xtaci/gaio.test]\n./aio_linux.go:155:7: undefined: setAffinity\n./watcher.go:588:4: undefined: setAffinity\nFAIL\tgithub.com/xtaci/gaio [build failed]\nFAIL\n```\n\nmake sure you have gcc/clang installed.\n\n## License\n\n`gaio` source code is available under the MIT [License](/LICENSE).\n\n## References\n\n* https://zhuanlan.zhihu.com/p/102890337 -- gaio小记\n* https://github.com/golang/go/issues/15735 -- net: add mechanism to wait for readability on a TCPConn\n* https://en.wikipedia.org/wiki/C10k_problem -- C10K\n* https://golang.org/src/runtime/netpoll_epoll.go -- epoll in golang \n* https://www.freebsd.org/cgi/man.cgi?query=kqueue\u0026sektion=2 -- kqueue\n* https://idea.popcount.org/2017-02-20-epoll-is-fundamentally-broken-12/ -- epoll is fundamentally broken\n* https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Flow_control -- TCP Flow Control \n* http://www.idc-online.com/technical_references/pdfs/data_communications/Congestion_Control.pdf -- Back-pressure\n\n## Status\n\nStable\n","funding_links":[],"categories":["开源类库","Networking","Open source library","Go","网络","Relational Databases","网络相关库"],"sub_categories":["网络","Transliteration","Uncategorized","The Internet","音译","暂未分类这些库被放在这里是因为其他类别似乎都不适合。","暂未分类"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxtaci%2Fgaio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxtaci%2Fgaio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxtaci%2Fgaio/lists"}