{"id":22546507,"url":"https://github.com/deephaven/benchmark","last_synced_at":"2025-06-25T16:05:09.658Z","repository":{"id":104451516,"uuid":"590620987","full_name":"deephaven/benchmark","owner":"deephaven","description":"Deephaven Benchmarking","archived":false,"fork":false,"pushed_at":"2025-06-05T18:58:25.000Z","size":1931,"stargazers_count":1,"open_issues_count":30,"forks_count":3,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-06-05T19:46:32.519Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deephaven.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-18T20:33:07.000Z","updated_at":"2025-06-05T18:58:27.000Z","dependencies_parsed_at":"2023-10-26T02:26:50.166Z","dependency_job_id":"c3efb294-32ce-415e-81cc-2ca21c64b2cf","html_url":"https://github.com/deephaven/benchmark","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/deephaven/benchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fbenchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fbenchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fbenchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fbenchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deephaven","download_url":"https://codeload.github.com/deephaven/benchmark/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deephaven%2Fbenchmark/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261906774,"owners_count":23228348,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T15:08:02.116Z","updated_at":"2025-06-25T16:05:09.637Z","avatar_url":"https://github.com/deephaven.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Deephaven Benchmark\n\n[Summary of Latest Successful Nightly Benchmarks](docs/NightlySummary.md)\n![Operation Rate Change Tracking By Release](https://storage.googleapis.com/deephaven-benchmark/nightly/deephaven/benchmark-summary.svg?)\n([See Other Deephaven Summaries Below](#other-deephaven-summaries))\n\nThe Benchmark framework provides support for gathering performance measurements and statistics for operations on tabular data.  It uses the JUnit\nframework as a runner and works from popular IDEs or from the command line. It is geared towards scale testing interfaces capable of ingesting \ntable data, transforming it, and returning tabular results. \n\nCurrently, most benchmarks that use the framework are aimed at broad coverage of single query operations executed in \n[Deephaven Community Core](https://deephaven.io/community/) through the Barrage Java Client. Tests focus on querying static parquet files, \nstreamed Kafka topics, and replayed data.\n\nThe project maintains several hundred standardized benchmarks for Deephaven query operations that are tracked both from release-to-release and \nnightly. Results are regularly published to a read-only GCloud bucket (*deephaven-benchmark*) available through the public storage API. \n\nThe typical workflow of a Benchmark test is... *Configure table/column generation* --\u003e *Execute Query* --\u003e *Measure Results*.  This is all done inside a JUnit test class.\n\nTests are designed to scale by changing a scale property value call *scale.row.count*, and per-test scale multipliers, so the same test can be used in multiple runs \nat different scales for comparison.  For ease of comparison, collected results use processing rates for benchmarked operations in addition to elapsed time. MXBean \nmetrics are also collected for each benchmark as well as details about the platform where each test ran.\n\nTests are run client-server, so the test runner does not need to be co-located with the Deephaven Engine. Measurements and statistics are taken directly \nfrom the engine(s) to reduce the affect of I/O and test setup on the results.\n\nResources:\n- [Getting Started](docs/GettingStarted.md) - Getting set up to run benchmarks against Deephaven Community Core\n- [Testing Concepts](docs/TestingConcepts.md) - Understanding what drives Benchmark development\n- [Test-writing Basics](docs/TestWritingBasics.md) - How to generate data and use it for tests\n- [Collected Results](docs/CollectedResults.md) - What's in the benchmark results\n- [Run the Release Distribution](docs/distro/BenchmarkDistribution.md) - How to run Deephaven benchmarks from a release tar file\n- [Run from the Command Line](docs/CommandLine.md) - How to run the benchmark jar with a test package\n- [Run Adhoc Github Workflows](docs/AdhocWorkflows.md) - Running benchmark sets on-demand from Github\n- [Set up a Benchmark Fork](docs/ForkSetup.md) - Set up and run from a Deephaven benchmark fork\n- [Published Results Storage](docs/PublishedResults.md) - How to grab and use Deephaven's published benchmarks\n\n## Other Deephaven Summaries\n\n[Summary of Comparison Benchmarks](docs/ComparisonSummary.md)\n![Operation Rate Product Comparison](https://storage.googleapis.com/deephaven-benchmark/compare/deephaven/benchmark-summary.svg?)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeephaven%2Fbenchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeephaven%2Fbenchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeephaven%2Fbenchmark/lists"}