{"id":13879694,"url":"https://github.com/socketry/async-postgres","last_synced_at":"2025-07-16T15:32:50.537Z","repository":{"id":56842622,"uuid":"121703394","full_name":"socketry/async-postgres","owner":"socketry","description":null,"archived":true,"fork":false,"pushed_at":"2021-05-22T02:40:47.000Z","size":30,"stargazers_count":78,"open_issues_count":8,"forks_count":11,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-31T14:41:32.030Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/socketry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-16T01:21:13.000Z","updated_at":"2023-09-09T01:45:58.000Z","dependencies_parsed_at":"2022-09-01T06:31:47.244Z","dependency_job_id":null,"html_url":"https://github.com/socketry/async-postgres","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socketry%2Fasync-postgres","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socketry%2Fasync-postgres/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socketry%2Fasync-postgres/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/socketry%2Fasync-postgres/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/socketry","download_url":"https://codeload.github.com/socketry/async-postgres/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226143895,"owners_count":17580245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-06T08:02:29.305Z","updated_at":"2024-11-24T08:31:37.766Z","avatar_url":"https://github.com/socketry.png","language":"Ruby","funding_links":[],"categories":["Ruby"],"sub_categories":[],"readme":"# Async::Postgres\n\n**This gem is experimental and unmaintained. Please see https://github.com/socketry/db-postgres for an event driven driver for Postgres.**\n\n## Motivation\n\nWe have some IO bound web APIs generating statistics and we sometimes have issues when using [passenger] due to thread/process exhaustion. In addition, we make a lot of upstream HTTP RPCs and these are also IO bound.\n\nThis library, in combination with [async-http], ensure that we don't become IO bound in many cases. In addition, we don't need to tune the intermediate server as it will simply scale according to backend resource availability and IO throughput.\n\n[passenger]: https://github.com/phusion/passenger\n[async-http]: https://github.com/socketry/async-http\n\n## Installation\n\nAdd this line to your application's Gemfile:\n\n```ruby\ngem 'async-postgres'\n```\n\nAnd then execute:\n\n\t$ bundle\n\nOr install it yourself as:\n\n\t$ gem install async-postgres\n\n## Performance\n\nFor database-bound workloads, this approach yields significant improvements to throughput and ultimately latency.\n\nUsing the example provided, which sleeps in the database for 10x10ms, we expect 10 sequential requests/second:\n\n```ruby\nrun lambda {|env|\n\t10.times do\n\t\tActiveRecord::Base.connection.execute(\"SELECT pg_sleep(0.01)\")\n\tend\n\t\n\tActiveRecord::Base.clear_active_connections!\n\t\n\t[200, {}, []]\n}\n```\n\nWhen running on [puma], with 16 threads, we could expect roughly 16 threads * 10 sequential requests/second.\n\n```\n% puma\nPuma starting in single mode...\n* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song\n* Min threads: 0, max threads: 16\n* Environment: development\n* Listening on tcp://0.0.0.0:9292\nUse Ctrl-C to stop\n\n% wrk -c 512 -t 128 -d 30 http://localhost:9292\nRunning 30s test @ http://localhost:9292\n  128 threads and 512 connections\n  Thread Stats   Avg      Stdev     Max   +/- Stdev\n    Latency   105.77ms    4.05ms 176.61ms   98.61%\n    Req/Sec    37.83      5.64    40.00     84.64%\n  4544 requests in 30.09s, 230.75KB read\nRequests/sec:    151.00\nTransfer/sec:      7.67KB\n```\n\nWe can see we get close to the theoretical throughput given the number of available threads.\n\nIf we start [puma] with more threads, we get increased throughput.\n\n```\n% puma -t 128:128 \nPuma starting in single mode...\n* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song\n* Min threads: 128, max threads: 128\n* Environment: development\n* Listening on tcp://0.0.0.0:9292\nUse Ctrl-C to stop\n\n% wrk -c 512 -t 128 -d 30 http://localhost:9292\nRunning 30s test @ http://localhost:9292\n  128 threads and 512 connections\n  Thread Stats   Avg      Stdev     Max   +/- Stdev\n    Latency   153.92ms   27.06ms 597.36ms   95.11%\n    Req/Sec    26.34      9.14    40.00     70.04%\n  24985 requests in 30.10s, 1.24MB read\n  Socket errors: connect 0, read 49, write 0, timeout 0\nRequests/sec:    830.06\nTransfer/sec:     42.15KB\n```\n\nThe theoretical throughput in this case is 128 threads * 10 sequential requests/second. Unfortunately, [puma] in it's current configuration quickly becomes CPU bound:\n\n```\n% puma -t 512:512\nPuma starting in single mode...\n* Version 3.11.2 (ruby 2.5.0-p0), codename: Love Song\n* Min threads: 512, max threads: 512\n* Environment: development\n* Listening on tcp://0.0.0.0:9292\nUse Ctrl-C to stop\n\n% wrk -c 512 -t 128 -d 30 http://localhost:9292\nRunning 30s test @ http://localhost:9292\n  128 threads and 512 connections\n  Thread Stats   Avg      Stdev     Max   +/- Stdev\n    Latency   452.46ms   96.97ms   1.71s    80.84%\n    Req/Sec    10.12      5.52    40.00     77.06%\n  23343 requests in 30.10s, 1.16MB read\nRequests/sec:    775.49\nTransfer/sec:     39.38KB\n```\n\nWhen running with [falcon], we are limited by the database. Postgres has been configured with up to 1024 connections, and falcon runs one process per available (hyper-)core, 8 in this case. With up to 1024 connections, we could expect an upper bound of 512 connections * 10 sequential requests/second.\n\n```\n% falcon --quiet serve --forked\n\n% wrk -c 512 -t 128 -d 30 http://localhost:9292\nRunning 30s test @ http://localhost:9292\n  128 threads and 512 connections\n  Thread Stats   Avg      Stdev     Max   +/- Stdev\n    Latency   158.85ms   20.33ms 263.33ms   68.57%\n    Req/Sec    25.25      8.97    40.00     72.16%\n  96641 requests in 30.10s, 4.52MB read\nRequests/sec:   3210.90\nTransfer/sec:    153.65KB\n```\n\nWe get close to the theoretical 5120 requests/second limit, but at this point the entire test becomes CPU bound within Ruby/Falcon.\n\n## Usage\n\nIn theory, this is a drop-in replacement for ActiveRecord. But, it must be used with an [async] capable server like [falcon].\n\n[async]: https://github.com/socketry/async\n[falcon]: https://github.com/socketry/falcon\n[puma]: https://github.com/puma/puma\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new Pull Request\n\n## License\n\nCopyright, 2018, by Samuel G. D. Williams. \u003chttp://www.codeotaku.com\u003e\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in\nall copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\nTHE SOFTWARE.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocketry%2Fasync-postgres","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsocketry%2Fasync-postgres","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsocketry%2Fasync-postgres/lists"}