{"id":37118415,"url":"https://github.com/ojaai/asyncfs","last_synced_at":"2026-01-14T13:50:11.030Z","repository":{"id":57649841,"uuid":"436177225","full_name":"ojaai/asyncfs","owner":"ojaai","description":"Golang library for asynchronous file I/O using io_uring / aio / OVERLAPPED interfaces","archived":false,"fork":false,"pushed_at":"2022-02-03T07:23:56.000Z","size":26,"stargazers_count":30,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-06-20T08:11:10.291Z","etag":null,"topics":["aio","asynchronous","asyncio","bsd","files","go","golang","io","io-uring","linux","overlapped","windows"],"latest_commit_sha":null,"homepage":"https://oja.ai","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ojaai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-08T08:48:19.000Z","updated_at":"2024-06-15T07:48:59.000Z","dependencies_parsed_at":"2022-09-05T03:00:36.168Z","dependency_job_id":null,"html_url":"https://github.com/ojaai/asyncfs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ojaai/asyncfs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ojaai%2Fasyncfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ojaai%2Fasyncfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ojaai%2Fasyncfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ojaai%2Fasyncfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ojaai","download_url":"https://codeload.github.com/ojaai/asyncfs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ojaai%2Fasyncfs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28422355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T13:30:50.153Z","status":"ssl_error","status_checked_at":"2026-01-14T13:29:08.907Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aio","asynchronous","asyncio","bsd","files","go","golang","io","io-uring","linux","overlapped","windows"],"created_at":"2026-01-14T13:50:10.417Z","updated_at":"2026-01-14T13:50:11.004Z","avatar_url":"https://github.com/ojaai.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Asyncfs is a library for asynchronous file I/O\n# Description\nThere is a problem in the Go runtime: since the goroutine scheduler works only in the context of the Go code itself, it needs to make a separate OS thread to call external functions such as cgo or syscalls. A typical example of syscalls is I/O operations with files. Unfortunately, epoll/kqueue can’t work with regular files, thus Go’s netpoller can’t help the Go's scheduler. The scheduler can’t switch a goroutine which execute the syscall and it is not clear how long it takes. Performing such operations in a separate OS thread is the only option the Go's scheduler has. If all the threads are busy, the new thread will be made. That is why if cgo is used frequently as well as with long syscalls (except for sockets, since there is a netpoller there) your Go program can create a lot of unnecessary OS threads.\nAsyncfs allows you to perform I/O operations with files using asynchronous interfaces: \n- **Linux, Android** - aio/io_uring;\n- **FreeBSD, MacOS** - aio;\n- **Windows** - OVERLAPPED;\n\nSo syscalls are processed in a short time and unnecessary OS threads are not created. The work with the files occurs at the core OS level and the goroutines are not blocked. Also you can work using synchronous mode. In this case you have an ability to set a limit that regulates the need to switch to a separate thread for the I/O operation.\n\n# Getting the Source\n```sh\ngo get -u github.com/ojaai/asyncfs\n```\n\n# Restrictions\n- The library is incompatible with the race detector\n- Using aio on Linux requires 512-bytes alignment\n\n# Example\n```go\nimport(\n\t\"fmt\"\n\t\"github.com/ojaai/asyncfs\"\n\t\"sync\"\n)\n\nfunc main() {\n\tpool := sync.Pool{\n\t\tNew: func() interface{} {\n\t\t\tb := make([]uint8, 0, 1024*128)\n\t\t\treturn b\n\t\t},\n\t}\n\n\tallocator := func(sz int) []uint8 {\n\t\tbuf, ok := pool.Get().([]uint8)\n\t\tif !ok || cap(buf) != sz {\n\t\t\treturn make([]uint8, 0, sz)\n\t\t}\n\t\treturn buf\n\t}\n\n\treleaser := func(buf []uint8) {\n\t\tfor i := range buf {\n\t\t\tbuf[i] = 0x0\n\t\t}\n\t\tbuf = buf[:0]\n\t\tpool.Put(buf)\n\t}\n\t\n\t// main ctx initialization\n\t// 1024 - asynchronous operations queue size;\n\t// 1024*4 - the byte limit that determines whether to switch to a new OS thread for synchronous I/O operation. \n\t// If the buffer size for I/O does not exceed this limit, then the Go's scheduler will not switch to a new OS thread;\n\t// allocator, releaser - memory operations for a buffer that can be used in an I/O operation;\n\t// allocator \u0026\u0026 releaser can be nil, if you don't use special allocators\n\tif err := asyncfs.NewCtx(1024, 1024*4, allocator, releaser); err != nil {\n\t\tpanic(err.Error())\n\t}\n\t// the last argument is the mode of working with the file: synchronous or asynchronous\n\tf, err := asyncfs.Create(\"/tmp/test_asyncfs\", asyncfs.ModeAsync)\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\n\t// aio on Linux requires I/O to be 512-byte aligned\n\t// AllocBuf make and align byte array\n\tbuf := asyncfs.AllocBuf(4096)\n\tdefer asyncfs.ReleaseBuf(buf)\n\n\t// fill 'buf' with your payload...\n\t// async mode: first 'n' will always be 0\n\t_, err = f.Write(buf)\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\t// do something ...\n\tfor {\n\t\tn, ok, err := f.LastOp()\n\t\tif !ok {\n\t\t\t// do something ...\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\tpanic(err.Error)\n\t\t}\n\t\tfmt.Printf(\"%d bytes has been written\\n\", n)\n\t\tbreak\n\t}\n  \n\t// ok, let's try to read\n\tf.Seek(0, 0)\n\tout := asyncfs.AllocBuf(4096)\n\tdefer asyncfs.ReleaseBuf(out)\n\t_, err = f.Read(out)\n\tif err != nil {\n\t\tpanic(err.Error())\n\t}\n\t// do something ...\n\tfor {\n\t\tn, ok, err := f.LastOp()\n\t\tif !ok {\n\t\t\t// do something ...\n\t\t\tcontinue\n\t\t}\n\t\tif err != nil {\n\t\t\tpanic(err.Error)\n\t\t}\n\t\tfmt.Printf(\"%d bytes has been read\\n\", n)\n\t\tbreak\n\t}\n\t// out has been filled\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fojaai%2Fasyncfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fojaai%2Fasyncfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fojaai%2Fasyncfs/lists"}