{"id":13522333,"url":"https://github.com/pinterest/bender","last_synced_at":"2025-03-31T22:31:08.252Z","repository":{"id":19682585,"uuid":"22936695","full_name":"pinterest/bender","owner":"pinterest","description":"An easy-to-use library for creating load testing applications","archived":false,"fork":false,"pushed_at":"2024-04-19T12:28:22.000Z","size":181,"stargazers_count":662,"open_issues_count":6,"forks_count":61,"subscribers_count":34,"default_branch":"master","last_synced_at":"2024-11-17T12:49:18.007Z","etag":null,"topics":["dns","http","load-testing","thrift"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"emulatorE2/snes9x-sdl","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pinterest.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-08-14T01:10:33.000Z","updated_at":"2024-11-01T09:50:54.000Z","dependencies_parsed_at":"2023-11-26T01:26:43.715Z","dependency_job_id":"e1a13afe-a8ae-493b-a12c-f1a2d7ba1d93","html_url":"https://github.com/pinterest/bender","commit_stats":{"total_commits":60,"total_committers":23,"mean_commits":2.608695652173913,"dds":0.6833333333333333,"last_synced_commit":"1085dcf323915a83e4cb07b500fa871236c87cef"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinterest%2Fbender","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinterest%2Fbender/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinterest%2Fbender/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pinterest%2Fbender/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pinterest","download_url":"https://codeload.github.com/pinterest/bender/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246552321,"owners_count":20795801,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dns","http","load-testing","thrift"],"created_at":"2024-08-01T06:00:45.759Z","updated_at":"2025-03-31T22:31:04.018Z","avatar_url":"https://github.com/pinterest.png","language":"Go","funding_links":[],"categories":["4. Load Generators and Synthetic Traffic","Uncategorized","Go"],"sub_categories":["Events \u0026 Problems","Uncategorized"],"readme":"Bender\n======\n\nBender makes it easy to build load testing applications for services using protocols like HTTP,\nThrift, Protocol Buffers and many more. Bender provides a library of flexible, powerful primitives\nthat can be combined (with plain Go code) to build load testers customized to any use case and that\nevolve with your service over time.\n\nBender provides two different approaches to load testing. The first, LoadTestThroughput, gives the\ntester control over the throughput (QPS), but not over the concurrency. This one is very well\nsuited for services that are open to the Internet, like web services, and even backend Thrift or\nProtocol Buffer services, since it will just keep sending requests, even if the service is\nstruggling. The second approach, LoadTestConcurrency, gives the tester control over the concurrency,\nbut not over the throughput. This approach is better suited to testing services that require lots of\nconcurrent connections, and need to be tested for resource limits.\n\nThat Bender is a library makes it flexible and easy to extend, but means it takes longer to create\nan initial load tester. As a result, we've focused on creating easy-to-follow tutorials that are\nwritten for people unfamiliar with Go, along with documentation for all the major functions in the\nlibrary.\n\n## Getting Started\n\nThe easiest way to get started with Bender is to use one of the tutorials:\n\n* [DHCPv6](https://github.com/pinterest/bender/blob/master/dhcpv6/TUTORIAL.md)\n* [DNS](https://github.com/pinterest/bender/blob/master/dns/TUTORIAL.md)\n* [HTTP](https://github.com/pinterest/bender/blob/master/http/TUTORIAL.md)\n* [TFTP](https://github.com/pinterest/bender/blob/master/tftp/TUTORIAL.md)\n* [Thrift](https://github.com/pinterest/bender/blob/master/thrift/TUTORIAL.md)\n\n## Documentation\n\nThe package documentation is available on [godoc.org](http://godoc.org/github.com/pinterest/bender).\nThe function and data structure documentation is also available there.\n\n## Performance\n\nWe have only informal, anecdotal evidence for the maximum performance of Bender. For example, in a\nvery simple load test using a Thrift server that just echoes a message, Bender was able to send\n7,000 QPS from a single EC2 m3.2xlarge host. At higher throughput the Bender \"overage\" counter\nincreased, indicating that either the Go runtime, the OS or the host was struggling to keep up.\n\nWe have found a few things that make a big difference when running load tests. First, the Go\nruntime needs some tuning. In particular, the Go GC is very immature, so we prefer to disable it\nusing the `GOGC=off` environment variable. In addition, we have seen some gains from setting\n`GOMAXPROCS` to twice the number of CPUs.\n\nSecondly, the Linux TCP stack for a default server installation is usually not tuned to high\nthroughput servers or load testers. After some experimentation, we have settled on adding these\nlines to `/etc/sysctl.conf`, after which you can run `sysctl -p` to load them (although it is\nrecommended to restart your host at this point to make sure these take effect).\n\n```\n# /etc/sysctl.conf\n# Increase system file descriptor limit\nfs.file-max = 100000\n\n# Increase ephermeral IP ports\nnet.ipv4.ip_local_port_range = 10000 65000\n\n# Increase Linux autotuning TCP buffer limits\n# Set max to 16MB for 1GE and 32M (33554432) or 54M (56623104) for 10GE\n# Don't set tcp_mem itself! Let the kernel scale it based on RAM.\nnet.core.rmem_max = 16777216\nnet.core.wmem_max = 16777216\nnet.core.rmem_default = 16777216\nnet.core.wmem_default = 16777216\nnet.core.optmem_max = 40960\nnet.ipv4.tcp_rmem = 4096 87380 16777216\nnet.ipv4.tcp_wmem = 4096 65536 16777216\n\n# Make room for more TIME_WAIT sockets due to more clients,\n# and allow them to be reused if we run out of sockets\n# Also increase the max packet backlog\nnet.core.netdev_max_backlog = 50000\nnet.ipv4.tcp_max_syn_backlog = 30000\nnet.ipv4.tcp_max_tw_buckets = 2000000\nnet.ipv4.tcp_tw_reuse = 1\nnet.ipv4.tcp_fin_timeout = 10\n\n# Disable TCP slow start on idle connections\nnet.ipv4.tcp_slow_start_after_idle = 0\n```\n\nThis is a slightly modified version of advice taken from this source:\nhttp://www.nateware.com/linux-network-tuning-for-2013.html#.VBjahC5dVyE\n\nIn addition, it helps to increase the open file limit with something like:\n\n```\nulimit -n 100000\n```\n\n## What Is Missing\n\nBender does not provide any support for sending load from more than one machine. If you need to\nsend more load than a single machine can handle, or you need the requests to come from multiple\nphysical hosts (or different networks, or whatever), you currently have to write your own tools. In\naddition, the histogram implementation used by Bender is inefficient to send over the network,\nunlike q-digest or t-digest, which we hope to implement in the future.\n\nBender does not provide any visualization tools, and has a relatively simple set of measurements,\nincluding a customizable histogram of latencies, an error rate and some other summary statistics.\nBender does provide a complete log of everything that happens during a load test, so you can use\nexisting tools to graph any aspect of that data, but nothing in Bender makes that easy right now.\n\nBender currently provides helper functions for DHCPv6, DNS, HTTP, Thrift and TFTP. We appreciate\nPull Requests for other protocols.\n\nThe load testers we have written internally with Bender have a lot of common command line arguments,\nbut we haven't finalized a set to share as part of the library.\n\n## Comparison to Other Load Testers\n\n#### JMeter\n\nJMeter provides a GUI to configure and run load tests, and can also be configured via XML (really,\nreally not recommended by hand!) and run from the command line. JMeter uses the same approach as\nLoadTestConcurrency in Bender, which is not a good approach to load testing services (see the Bender\ndocs and the Iago philosophy for more details on why that is). It isn't easy to extend JMeter to\nhandle new protocols, so it doesn't have support for Thrift or Protobuf. It is relatively easy to\nextend other parts of JMeter by writing Java code, however, and the GUI makes it easy to plug all\nthe pieces together.\n\n#### Iago\n\nIago is Twitter's load testing library and it is the inspiration for Bender's LoadTestThroughput\nfunction. Iago is a Scala library written on top of Netty and the Twitter Finagle libraries. As a\nresult, Iago is powerful, but difficult to understand, extend and configure. It was frustration with\nmaking Iago work that led to the creation of Bender.\n\n#### The Grinder\n\nThe Grinder has the same load testing approach as JMeter, but allows scripting via Jython, which\nmakes it more flexible and extensible. The Grinder uses threads, which limits the concurrency at\nwhich it can work, and makes it hard to implement things like Bender's LoadTestThroughput function.\nThe Grinder does have support for conveniently running distributed load tests.\n\n## Copyright\n\nCopyright 2014-2018 Pinterest, Inc.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\n## Attribution\n\nBender includes open source from the following sources:\n\n* Apache Thrift Libraries. Copyright 2014 Apache Software Foundation. Licensed under the Apache License v2.0 (http://www.apache.org/licenses/).\n* Miek Gieben DNS library for Go. Licensed under the BSD license (https://github.com/miekg/dns/blob/master/COPYRIGHT)\n* Insomniacslk DHCP library for Go. Licensed under the BSD license (https://github.com/insomniacslk/dhcp/blob/master/LICENSE)\n* Dmitri Popov TFTP library for Go. Licensed under the MIT license (https://github.com/pin/tftp/blob/master/LICENSE)\n* Go Libraries. Copyright 2012 The Go Authors. Licensed under the BSD license (http://golang.org/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinterest%2Fbender","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpinterest%2Fbender","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpinterest%2Fbender/lists"}