{"id":15021079,"url":"https://github.com/kotlin/kotlinx-benchmark","last_synced_at":"2025-05-14T23:07:50.874Z","repository":{"id":38687749,"uuid":"162275279","full_name":"Kotlin/kotlinx-benchmark","owner":"Kotlin","description":"Kotlin multiplatform benchmarking toolkit","archived":false,"fork":false,"pushed_at":"2025-05-14T22:33:35.000Z","size":1492,"stargazers_count":579,"open_issues_count":71,"forks_count":42,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-05-14T23:07:39.807Z","etag":null,"topics":["benchmark","gradle","jmh","jvm","kotlin"],"latest_commit_sha":null,"homepage":"","language":"Kotlin","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Kotlin.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-12-18T11:05:59.000Z","updated_at":"2025-05-14T17:02:04.000Z","dependencies_parsed_at":"2023-11-20T10:38:38.586Z","dependency_job_id":"60133f06-f021-410a-b62c-c34fb32fcaf5","html_url":"https://github.com/Kotlin/kotlinx-benchmark","commit_stats":{"total_commits":451,"total_committers":37,"mean_commits":12.18918918918919,"dds":0.5809312638580931,"last_synced_commit":"64cf96b1ecb334ba307c900b048d2388719b6fe6"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kotlin%2Fkotlinx-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kotlin%2Fkotlinx-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kotlin%2Fkotlinx-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Kotlin%2Fkotlinx-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Kotlin","download_url":"https://codeload.github.com/Kotlin/kotlinx-benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254243363,"owners_count":22038046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","gradle","jmh","jvm","kotlin"],"created_at":"2024-09-24T19:56:06.618Z","updated_at":"2025-05-14T23:07:45.864Z","avatar_url":"https://github.com/Kotlin.png","language":"Kotlin","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kotlinx-benchmark\n\n[![Kotlin Alpha](https://kotl.in/badges/alpha.svg)](https://kotlinlang.org/docs/components-stability.html)\n[![JetBrains incubator project](https://jb.gg/badges/incubator.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)\n[![GitHub license](https://img.shields.io/github/license/kotlin/kotlinx-benchmark)](LICENSE)\n[![Build status](https://teamcity.jetbrains.com/guestAuth/app/rest/builds/buildType:(id:KotlinTools_KotlinxBenchmark_Build_All)/statusIcon.svg)](https://teamcity.jetbrains.com/viewType.html?buildTypeId=KotlinTools_KotlinxBenchmark_Build_All)\n[![Maven Central](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-benchmark-runtime.svg?label=Maven%20Central)](https://central.sonatype.com/artifact/org.jetbrains.kotlinx/kotlinx-benchmark-runtime)\n[![Gradle Plugin Portal](https://img.shields.io/maven-metadata/v?label=Gradle%20Plugin\u0026metadataUrl=https://plugins.gradle.org/m2/org/jetbrains/kotlinx/kotlinx-benchmark-plugin/maven-metadata.xml)](https://plugins.gradle.org/plugin/org.jetbrains.kotlinx.benchmark)\n\n`kotlinx-benchmark` is a toolkit for running benchmarks for multiplatform code written in Kotlin.\nIt is designed to work with Kotlin/JVM, Kotlin/JS, Kotlin/Native, and Kotlin/WasmJs (experimental) targets.\n\nTo get started, ensure you're using Kotlin 2.0.0 or newer and Gradle 7.4 or newer.\nHowever, because the Kotlin/WasmJs target is experimental and in an active development phase, it guarantees support\nonly for the specific Kotlin version used to build the library. For the latest version, this is Kotlin 2.0.20.\n\n## Features\n\n- Low noise and reliable results\n- Statistical analysis\n- Detailed performance reports\n\n## Table of contents\n\n\u003c!--- TOC --\u003e\n\n- [Setting Up a Kotlin Multiplatform Project for Benchmarking](#setting-up-a-kotlin-multiplatform-project-for-benchmarking)\n  - [Target-specific configurations](#target-specific-configurations)\n    - [Kotlin/JVM](#kotlinjvm)\n    - [Kotlin/JS](#kotlinjs)\n    - [Kotlin/Native](#kotlinnative)\n    - [Kotlin/Wasm](#kotlinwasm)\n  - [Writing Benchmarks](#writing-benchmarks)\n  - [Running Benchmarks](#running-benchmarks)\n  - [Benchmark Configuration Profiles](#benchmark-configuration-profiles)\n  - [Separate source set for benchmarks](#separate-source-set-for-benchmarks)\n- [Examples](#examples)\n- [Contributing](#contributing)\n\n\u003c!--- END --\u003e\n\n- **Useful guides**\n  - [Writing Benchmarks](docs/writing-benchmarks.md)\n  - [Configuring Benchmarks Execution](docs/configuration-options.md)\n  - [Setting Up Kotlin/JVM and Java Projects for Benchmarking](docs/kotlin-jvm-project-setup.md)\n  - [Setting Up a Separate Source Set for Benchmarks](docs/separate-benchmark-source-set.md)\n  - [Overview of Tasks Provided by kotlinx-benchmark Gradle Plugin](docs/tasks-overview.md)\n\n## Setting Up a Kotlin Multiplatform Project for Benchmarking\n\nTo configure a Kotlin Multiplatform project for benchmarking, follow the steps below. \nIf you want to benchmark only Kotlin/JVM and Java code, you may refer to our [comprehensive guide](docs/kotlin-jvm-project-setup.md) \ndedicated to setting up benchmarking in those specific project types.\n\n\u003cdetails open\u003e\u003csummary\u003eKotlin DSL\u003c/summary\u003e\n\n1.  **Applying Benchmark Plugin**: Apply the benchmark plugin.\n\n    ```kotlin\n    // build.gradle.kts\n    plugins {\n        id(\"org.jetbrains.kotlinx.benchmark\") version \"0.4.13\"\n    }\n    ```\n\n2.  **Specifying Plugin Repository**: Ensure you have the Gradle Plugin Portal for plugin lookup in the list of repositories:\n\n    ```kotlin\n    // settings.gradle.kts\n    pluginManagement {\n        repositories {\n            gradlePluginPortal()\n        }\n    }\n    ```\n\n3.  **Adding Runtime Dependency**: Next, add the `kotlinx-benchmark-runtime` dependency to the common source set:\n\n    ```kotlin\n    // build.gradle.kts\n    kotlin {\n        sourceSets {\n            commonMain {\n                dependencies {\n                    implementation(\"org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.13\")\n                }\n            }\n        }\n    }\n    ```\n\n4.  **Specifying Runtime Repository**: Ensure you have `mavenCentral()` for dependencies lookup in the list of repositories:\n\n    ```kotlin\n    // build.gradle.kts\n    repositories {\n        mavenCentral()\n    }\n    ```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eGroovy DSL\u003c/summary\u003e\n\n1.  **Applying Benchmark Plugin**: Apply the benchmark plugin.\n\n    ```groovy\n    // build.gradle\n    plugins {\n        id 'org.jetbrains.kotlinx.benchmark' version '0.4.13'\n    }\n    ```\n\n2.  **Specifying Plugin Repository**: Ensure you have the Gradle Plugin Portal for plugin lookup in the list of repositories:\n\n    ```groovy\n    // settings.gradle\n    pluginManagement {\n        repositories {\n            gradlePluginPortal()\n        }\n    }\n    ```\n\n3.  **Adding Runtime Dependency**: Next, add the `kotlinx-benchmark-runtime` dependency to the common source set:\n\n    ```groovy\n    // build.gradle\n    kotlin {\n        sourceSets {\n            commonMain {\n                dependencies {\n                    implementation 'org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.13'\n                }\n            }\n        }\n    }\n    ```\n\n4.  **Specifying Runtime Repository**: Ensure you have `mavenCentral()` for dependencies lookup in the list of repositories:\n\n    ```groovy\n    // build.gradle\n    repositories {\n        mavenCentral()\n    }\n    ```\n\n\u003c/details\u003e\n\n### Target-specific configurations\n\nTo run benchmarks on a platform ensure your Kotlin Multiplatform project targets that platform.\nFor different platforms, there may be distinct requirements and settings that need to be configured.\nThe guide below contains the steps needed to configure each supported platform for benchmarking.\n\n#### Kotlin/JVM\n\nTo run benchmarks in Kotlin/JVM:\n1.  Create a JVM target:\n\n    ```kotlin\n    // build.gradle.kts\n    kotlin {\n        jvm()\n    }\n    ```\n\n2.  Register `jvm` as a benchmark target:\n\n    ```kotlin\n    // build.gradle.kts\n    benchmark {\n        targets { \n            register(\"jvm\")\n        }\n    }\n    ```\n\n3.  Apply [allopen plugin](https://kotlinlang.org/docs/all-open-plugin.html) to ensure your benchmark classes and methods are `open`.\n\n    ```kotlin\n    // build.gradle.kts\n    plugins {\n        kotlin(\"plugin.allopen\") version \"2.0.20\"\n    }\n\n    allOpen {\n        annotation(\"org.openjdk.jmh.annotations.State\")\n    }\n    ```\n\n    \u003cdetails\u003e\u003csummary\u003e\u003cb\u003eExplanation\u003c/b\u003e\u003c/summary\u003e\n\n    Assume that you've annotated each of your benchmark classes with `@State(Scope.Benchmark)`:\n\n    ```kotlin\n    // MyBenchmark.kt\n    @State(Scope.Benchmark)\n    class MyBenchmark {\n        // Benchmarking-related methods and variables\n        @Benchmark\n        fun benchmarkMethod() {\n            // benchmarking logic\n        }\n    }\n    ```\n\n    In Kotlin, classes are `final` by default, which means they can't be overridden.\n    This conflicts with the Java Microbenchmark Harness (JMH) operation, which `kotlinx-benchmark` uses under the hood for running benchmarks on JVM.\n    JMH requires benchmark classes and methods to be `open` to be able to generate subclasses and conduct the benchmark.\n\n    This is where the `allopen` plugin comes into play. With the plugin applied, any class annotated with `@State` is treated as `open`, which allows JMH to work as intended:\n\n    ```kotlin\n    // build.gradle.kts\n    plugins {\n        kotlin(\"plugin.allopen\") version \"2.0.20\"\n    }\n\n    allOpen {\n        annotation(\"org.openjdk.jmh.annotations.State\")\n    }\n    ```\n\n    This configuration ensures that your `MyBenchmark` class and its `benchmarkMethod` function are treated as `open`.\n\n    \u003c/details\u003e\n\n    You can alternatively mark your benchmark classes and methods `open` manually, but using the `allopen` plugin enhances code maintainability.\n\n#### Kotlin/JS\n\nTo run benchmarks in Kotlin/JS:\n1.  Create a JS target with Node.js execution environment:\n\n    ```kotlin\n    // build.gradle.kts\n    kotlin {\n        js { \n            nodejs() \n        }\n    }\n    ```\n\n2.  Register `js` as a benchmark target:\n\n    ```kotlin\n    // build.gradle.kts\n    benchmark {\n        targets {\n            register(\"js\")\n        }\n    }\n    ```\n\n#### Kotlin/Native\n\nTo run benchmarks in Kotlin/Native:\n1.  Create a Native target:\n\n    ```kotlin\n    // build.gradle.kts\n    kotlin {\n        linuxX64()\n    }\n    ```\n\n2.  Register `linuxX64` as a benchmark target:\n\n    ```kotlin\n    // build.gradle.kts\n    benchmark {\n        targets {\n            register(\"linuxX64\")\n        }\n    }\n    ```\n    \nIt is possible to register multiple native targets. However, benchmarks can be executed only for the host target.\nThis library supports all [targets supported by the Kotlin/Native compiler](https://kotlinlang.org/docs/native-target-support.html). \n\n#### Kotlin/Wasm\n\nTo run benchmarks in Kotlin/Wasm:\n1.  Create a Wasm target with Node.js execution environment:\n\n    ```kotlin\n    // build.gradle.kts\n    kotlin {\n        wasm { \n            nodejs() \n        }\n    }\n    ```\n\n2.  Register `wasm` as a benchmark target:\n\n    ```kotlin\n    // build.gradle.kts\n    benchmark {\n        targets {\n            register(\"wasm\")\n        }\n    }\n    ```\n\nNote: Kotlin/Wasm is an experimental compilation target for Kotlin. It may be dropped or changed at any time. Refer to \n[Kotlin/Wasm documentation](https://kotlinlang.org/docs/wasm-overview.html) for up-to-date information about the target stability.\n\n### Writing Benchmarks\n\nAfter setting up your project and configuring targets, you can start writing benchmarks.\nAs an example, let's write a simplified benchmark that tests how fast we can add up numbers in an `ArrayList`:\n\n1. **Create Benchmark Class**: Create a class in your source set where you'd like to add the benchmark. Annotate this class with `@State(Scope.Benchmark)`.\n\n    ```kotlin\n    @State(Scope.Benchmark)\n    class MyBenchmark {\n\n    }\n    ```\n\n2. **Set Up Variables**: Define variables needed for the benchmark.\n\n    ```kotlin\n    private val size = 10\n    private val list = ArrayList\u003cInt\u003e()\n    ```\n\n3. **Initialize Resources**: Within the class, you can define any setup or teardown methods using `@Setup` and `@TearDown` annotations respectively. These methods will be executed before and after the entire benchmark run.\n\n    ```kotlin\n    @Setup\n    fun prepare() {\n        for (i in 0..\u003csize) {\n            list.add(i)\n        }\n    }\n\n    @TearDown\n    fun cleanup() {\n        list.clear()\n    }\n    ```\n\n4. **Define Benchmark Methods**: Next, create methods that you would like to be benchmarked within this class and annotate them with `@Benchmark`.\n\n    ```kotlin\n    @Benchmark\n    fun benchmarkMethod(): Int {\n        return list.sum()\n    }\n    ```\n\nYour final benchmark class will look something like this:\n\n```kotlin\nimport kotlinx.benchmark.*\n\n@State(Scope.Benchmark)\nclass MyBenchmark {\n    private val size = 10\n    private val list = ArrayList\u003cInt\u003e()\n\n    @Setup\n    fun prepare() {\n        for (i in 0..\u003csize) {\n            list.add(i)\n        }\n    }\n\n    @TearDown\n    fun cleanup() {\n        list.clear()\n    }\n\n    @Benchmark\n    fun benchmarkMethod(): Int {\n        return list.sum()\n    }\n}\n```\n\nNote: Benchmark classes located in the common source set will be run in all platforms, while those located in a platform-specific source set will be run only in the corresponding platform.\n\nSee [writing benchmarks](docs/writing-benchmarks.md) for a complete guide for writing benchmarks.\n\n### Running Benchmarks\n\nTo run your benchmarks in all registered platforms, run `benchmark` Gradle task in your project.\nTo run only on a specific platform, run `\u003ctarget-name\u003eBenchmark`, e.g., `jvmBenchmark`.\n\nFor more details about the tasks created by the `kotlinx-benchmark` plugin, refer to [this guide](docs/tasks-overview.md).\n\n### Benchmark Configuration Profiles\n\nThe `kotlinx-benchmark` library provides the ability to create multiple configuration profiles. The `main` configuration is already created by the toolkit.\nAdditional profiles can be created as needed in the `configurations` section of the `benchmark` block:\n\n```kotlin\n// build.gradle.kts\nbenchmark {\n    configurations {\n        named(\"main\") {\n            warmups = 20\n            iterations = 10\n            iterationTime = 3\n            iterationTimeUnit = \"s\"\n        }\n        register(\"smoke\") {\n            include(\"\u003cpattern of fully qualified name\u003e\")\n            warmups = 5\n            iterations = 3\n            iterationTime = 500\n            iterationTimeUnit = \"ms\"\n        }\n    }\n}\n```  \n\nRefer to our [comprehensive guide](docs/configuration-options.md) to learn about configuration options and how they affect benchmark execution.\n\n### Separate source set for benchmarks\n\nOften you want to have benchmarks in the same project, but separated from main code, much like tests.\nRefer to our [detailed documentation](docs/separate-benchmark-source-set.md) on configuring your project to set up a separate source set for benchmarks.\n\n## Examples\n\nTo help you better understand how to use the `kotlinx-benchmark` library, we've provided an [examples](examples) subproject. \nThese examples showcase various use cases and offer practical insights into the library's functionality.\n\n## Contributing\n\nWe welcome contributions to `kotlinx-benchmark`! If you want to contribute, please refer to our [Contribution Guidelines](CONTRIBUTING.md).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkotlin%2Fkotlinx-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkotlin%2Fkotlinx-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkotlin%2Fkotlinx-benchmark/lists"}