{"id":51180226,"url":"https://github.com/frappe/caffeine","last_synced_at":"2026-06-27T06:31:46.006Z","repository":{"id":267899148,"uuid":"902310572","full_name":"frappe/caffeine","owner":"frappe","description":"Microbenchmarks and load testing for Frappe Framework","archived":false,"fork":false,"pushed_at":"2026-05-30T06:04:40.000Z","size":78,"stargazers_count":63,"open_issues_count":8,"forks_count":20,"subscribers_count":10,"default_branch":"develop","last_synced_at":"2026-06-07T07:26:35.313Z","etag":null,"topics":["benchmarks","frappe","performance"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frappe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-12-12T10:15:58.000Z","updated_at":"2026-06-04T15:59:10.000Z","dependencies_parsed_at":"2024-12-16T14:44:33.047Z","dependency_job_id":"d26fa83e-a275-4cb6-8762-3af5a461ed7c","html_url":"https://github.com/frappe/caffeine","commit_stats":null,"previous_names":["ankush/caffeination","frappe/caffeination","frappe/caffeine"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/frappe/caffeine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frappe%2Fcaffeine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frappe%2Fcaffeine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frappe%2Fcaffeine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frappe%2Fcaffeine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frappe","download_url":"https://codeload.github.com/frappe/caffeine/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frappe%2Fcaffeine/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34844344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-27T02:00:06.362Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmarks","frappe","performance"],"created_at":"2026-06-27T06:31:45.550Z","updated_at":"2026-06-27T06:31:46.001Z","avatar_url":"https://github.com/frappe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"### Caffeine\n\nFrappes are usually not caffeinated enough, we want a lot of caffeine in ours.\n\n[![Sanity Test](https://github.com/frappe/caffeine/actions/workflows/ci.yml/badge.svg?event=schedule)](https://github.com/frappe/caffeine/actions/workflows/ci.yml)\n\n### Goal\n\nThis project has only one goal: **Speedup Frappe ecosystem by approximately by 2x.**\n\nApproximately, this boils down to:\n- Set up a good benchmark suite ranging from microbenchmarks to realistic traces.\n- Optimize EVERYTHING. Every 0.1% on critical path counts.\n- Make deployments resource efficient by tuning various knobs.\n\n### Running Microbenchmarks\n\nThis project uses [pyperf](https://pyperf.readthedocs.io/) to write various micro-benchmarks. Follow these steps to run the benchmarks:\n1. Install the app as usual: `bench get-app caffeine`\n3. Create a fresh site that hasn't been altered: `bench new-site bench.localhost`\n4. Allow running tests on this site: `bench --site bench.localhost set-config allow_tests true`\n5. Run benchmarks `bench --site bench.localhost run-microbenchmarks`\n\nNote: It can take up to an hour to run these benchmarks. The time requirement will only go up with more benchmarks.\n\nAdditional arguments:\n- `--filter=benchmark_name` can be used to filter benchmarks\n- `--help` will show you help about pyperf's inbuilt arguments. Refer [pyperf docs](https://pyperf.readthedocs.io/en/latest/runner.html) for more info.\n- Important pyperf commands:\n   - `-p5` can be used for a quick and dirty benchmark run consisting of only 5 outer runs.\n   - `-o output.json` can be used to store detailed results for analysis later.\n   - `pyperf compare_to` compares two results and applies statistical significance tests.\n   - `pyperf timeit` is useful for measuring tiny operations like setting an attribute on an object.\n\n#### Getting reliable results\n\nYour local setup might not be fit for benchmarking. Follow these steps before running benchmarks:\n\n0. Use a Linux machine. We don't run our servers on a Mac, so benchmarking on a Mac is sub-optimal.\n1. Stop all unnecessary running processes. Even your browser.\n2. If you're using a laptop, then plug it in. Do NOT benchmark on battery power.\n3. Disable SMT (HyperThreading) - `echo \"off\" | sudo tee /sys/devices/system/cpu/smt/control`\n4. Disable turbo boost. This is dependent on your CPU make and kernel version.\n5. Use `performance` governor. - [Arch Wiki](https://wiki.archlinux.org/title/CPU_frequency_scaling)\n6. Disable ASLR - `echo 0 | sudo tee /proc/sys/kernel/randomize_va_space`\n\nThis should get you roughly +/- 1% standard deviation results.\n\nYou can read this post for long-form explanations: https://ankush.dev/p/reliable-benchmarking\n\n\n#### Writing Microbenchmarks\n\n1. Find appropriate `bench_{module}.py` file.\n2. Add a new function with `bench_` prefix, the function body is your benchmark.\n3. If you need to measure something very small (\u003c1ms), then use `NanoBenchmark` class instead of function-based benchmarks.\n4. Be very cautious about how you write a benchmark, ensure that it _actually_ measures what you want to measure. E.g. If you want to measure the performance of `frappe.get_cached_doc` when it fetches data from Redis then you need to ensure that it's not just using a locally cached document.\n\n\n### E2E load testing\n\nThis repo contains a TPC-C inspired benchmark and load testing script for ERPNext.\n\n#### Pre-requisites\n\n- New site with ERPNext installed.\n- Grafana K6 - [Installation Guide](https://grafana.com/docs/k6/latest/set-up/install-k6/)\n- Basic understanding of how Frappe, ERPNext, and K6 work.\n\n#### Usage\n\nWe use k6 to generate load. You can read their [documentation](https://grafana.com/docs/k6/latest/get-started/) to get started.\n\n```bash\n# Setup dummy master data for load testing\nbench --site test_site setup-loadtest-data\n# Run the load test with 100 virtual concurrent users\nk6 run --vus 100 --duration 10m -e BASE_URL=http://sitename:port ./apps/caffeine/caffeine/loadtest/script.js\n```\n\nNote: fixture records have hard-coded naming to simplify writing tests without sending too many arguments. We intend to make it flexible over time if required.\n\n#### Tweaking / Extending\n\nIf you want to write custom benchmarks, consider forking the repository or creating a new script.\nThe goal of this project, as of now, is to just provide the boilerplate required for writing such tests.\n\nThe following variations are worth exploring in the future:\n\n- Realistic think times. This repo has an aggressive high-throughput API ingestion scenario. If your service will only be used by humans, then you can relax the think times.\n- Read heavy workloads. This repo has a disproportionately high W:R ratio. Most sites used by end users directly will have 1:10 write-to-read ratio.\n- Different modules and doctypes. Currently, we just do sales invoice -\u003e payment -\u003e Delivery simulation.\n- Randomness / Markov modelling on top of end-user traces / complex workflows\n\n\n### Contributing\n\nAt present, this repo is not accepting any external contributions.\n\n### License\n\nAGPL-3.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrappe%2Fcaffeine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrappe%2Fcaffeine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrappe%2Fcaffeine/lists"}