{"id":14969848,"url":"https://github.com/jejacks0n/activeexperiment","last_synced_at":"2025-10-08T18:10:26.935Z","repository":{"id":64575588,"uuid":"571419427","full_name":"jejacks0n/activeexperiment","owner":"jejacks0n","description":"Active Experiment is a framework for defining and running experiments in Ruby.","archived":false,"fork":false,"pushed_at":"2023-03-21T22:08:30.000Z","size":102,"stargazers_count":5,"open_issues_count":6,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-06T09:43:56.248Z","etag":null,"topics":["experiments","ruby","ruby-on-rails"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jejacks0n.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"MIT-LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-11-28T04:45:18.000Z","updated_at":"2025-09-29T18:27:47.000Z","dependencies_parsed_at":"2023-07-15T15:27:34.906Z","dependency_job_id":"cf14a737-d80d-4920-95f7-35bdb3ff088e","html_url":"https://github.com/jejacks0n/activeexperiment","commit_stats":null,"previous_names":["jejacks0n/active_experiment"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/jejacks0n/activeexperiment","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jejacks0n%2Factiveexperiment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jejacks0n%2Factiveexperiment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jejacks0n%2Factiveexperiment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jejacks0n%2Factiveexperiment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jejacks0n","download_url":"https://codeload.github.com/jejacks0n/activeexperiment/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jejacks0n%2Factiveexperiment/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278990364,"owners_count":26081262,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-08T02:00:06.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["experiments","ruby","ruby-on-rails"],"created_at":"2024-09-24T13:42:29.750Z","updated_at":"2025-10-08T18:10:26.913Z","avatar_url":"https://github.com/jejacks0n.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Active Experiment – Decide what to do next\n\n\u003cimg alt=\"Active Experiment\" width=\"150\" height=\"183\" src=\"https://user-images.githubusercontent.com/13765/208318101-b48c9493-15ed-4a99-b42f-b20720dd7c77.png\" align=\"right\" hspace=\"20\"\u003e\n\n[![Gem Version](https://badge.fury.io/rb/activeexperiment.svg)](https://badge.fury.io/rb/activeexperiment)\n[![License](https://img.shields.io/badge/license-MIT-brightgreen.svg)](https://opensource.org/licenses/MIT)\n[![Maintainability](https://img.shields.io/codeclimate/coverage-letter/jejacks0n/activeexperiment.svg?label=maintainability)](https://codeclimate.com/github/jejacks0n/activeexperiment/maintainability)\n[![Test Coverage](https://img.shields.io/codeclimate/coverage/jejacks0n/activeexperiment.svg?label=test%20coverage)](https://codeclimate.com/github/jejacks0n/activeexperiment/test_coverage)\n\nActive Experiment is a framework for defining and running experiments. It supports using a variety of rollout and reporting strategies and/or services.\n\nExperiments can be everything from determining which query has the best performance, to which feature gets the most engagement, to rolling out a canary version of a new api service.\n\nExperimentation is complex. There are a lot of different ways to run experiments, and even more ways to report on them. Active Experiment is designed to be flexible enough to support a variety of use cases, but also to be consistent and easy to use.\n\n## Usage\n\nDefine your experiments using easily testable classes:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\nend\n```\n\nThis experiment can be generated using the Rails generator:\n\n```bash\nrails generate experiment my_experiment red blue\n```\n\nRun the experiment with a context, like the current user, or the post being rendered:\n\n```ruby\nMyExperiment.run(current_user) # =\u003e \"red\" or \"blue\"\n```\n\nOptionally override the defaults using local scope and helpers:\n\n```ruby\nMyExperiment.run(current_user) do |experiment|\n  experiment.on(:red) { redirect_to red_path }\n  experiment.on(:blue) { redirect_to blue_path }\nend\n```\n\nThat's it! When this experiment is encountered by different users, half\u003csup\u003e\u0026#8224;\u003c/sup\u003e will get the red variant, half will get the blue variant, and each will always get the same.\n\n\u003csmall\u003e\u0026#8224; roughly half, for the statistically pedantic.\u003c/small\u003e \n\n## Download and Installation\n\nAdd this line to your Gemfile:\n\n```ruby\ngem \"activeexperiment\"\n```\n\nOr install the latest version with RubyGems:\n\n```bash\ngem install activeexperiment\n```\n\nSource code can be downloaded as part of the project on GitHub:\n\n* https://github.com/jejacks0n/activeexperiment\n\nAdapters can be added to integrate with various services:\n\n- [Unleash adapter](https://github.com/jejacks0n/activeexperiment-unleash) \n\n## Advanced experimentation\n\nThis area provides a high level overview of the tools that more complex experiments can benefit from.\n\nFor example, some experiments need to define a default variant (also known as a _control_) that will be assigned if the experiment is skipped:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n\n  # The term control is simply a convention that means the default variant, and\n  # any variant can be set as the default with +use_default_variant(:red)+\n  control { \"default\" }\nend\n```\n\nCallbacks can be used to hook into the lifecycle when experiments are run, and can be targeted to when a specific variant has been assigned:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  control { \"default\" }\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n\n  # Skipping an experiment will always assign the default variant, which could\n  # be nothing, but since there's a control defined, it will be used.\n  before_run { skip if context.admin? }\n  \n  # Only invoked when the red variant has been assigned.\n  before_variant(:red) { puts \"running the red variant\" }\n  \n  # Maybe there's cleanup or logging to do afterwards?\n  after_run { puts \"run complete with the #{variant} variant\" unless skipped? }\nend\n```\n\nSegment rules can be used to assign specific variants for certain cases:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  control { \"default\" }\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n\n  segment :admins, into: :red\n  segment :old_accounts, into: :control\n  \n  private\n  \n  def admins\n    context.admin?\n  end\n\n  def old_accounts\n    context.created_at \u003c 1.year.ago\n  end\nend\n```\n\n## Rollouts\n\nRollouts are a core concept in Active Experiment. They allow specifying how an experiment should be rolled out, and even if it should be skipped or not. For example, the default rollout in Active Experiment is percentage based and accepts distribution rules -- if no rules are provided, even distribution is used.\n\nA rollout can implement any number of different strategies, interact with services, and can be used on a per-experiment basis.\n\nHere's an example of using the default percent rollout with custom distribution rules:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n  variant(:green) { \"green\" }\n\n  # Will assign the green variant 80% of the time, red and blue 10% each.\n  use_rollout :percent, rules: { red: 10, blue: 10, green: 80 }\nend\n```\n\n### Defining custom rollouts\n\nProject specific rollouts can be defined and registered too. To illustrate, here's a custom rollout that inherits from the base rollout, uses a fictional feature flag library, and assigns a random variant.\n\n```ruby\nclass FeatureFlagRollout \u003c ActiveExperiment::Rollouts::BaseRollout\n  register_as :feature_flag\n\n  def skipped_for(experiment)\n    !Feature.enabled?(@rollout_options[:flag_name] || experiment.name)\n  end\n\n  def variant_for(experiment)\n    experiment.variant_names.sample\n  end\nend\n```\n\nThis rollout can now be used the same way the built-in rollouts are:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n\n  # Using a custom rollout with options.\n  use_rollout :feature_flag, flag_name: \"my_feature_flag\"\nend\n```\n\nCustom rollouts can be registered to autoload as well, so they're only loaded when needed:\n\n```ruby\nActiveExperiment::Rollouts.register(\n  :feature_flag, \n  \"lib/feature_flag_rollout.rb\"\n)\n```\n\nThere's a world of flexibility with custom rollouts. One creative and simple rollout concept is to use the experiment itself:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\n\n  def self.skipped_for(*)\n    false\n  end\n\n  def self.variant_for(*)\n    variant_names.sample\n  end\n  \n  use_rollout self\nend\n```\n\n## Reporting\n\nReporting is a core concept in Active Experiment. It allows for collecting data about experiments and variants, and can be used to track performance metrics, analyze results, and more.\n\nSome simple reporting strategies might simply be added to `after_run` callbacks, but more complex reporting strategies can be implemented using a subscriber.\n\nA subscriber can be used to listen for experiment events and report them to a service. For example, here's a subscriber that reports to a fictional analytics service:\n\n```ruby\nclass MyAnalyticsSubscriber \u003c ActiveSupport::Subscriber\n  attach_to :active_experiment\n\n  def process_run(event)\n    experiment = event.payload[:experiment]\n    return if experiment.skipped?\n\n    Analytics.report(\n      experiment.serialize,\n      error: event.payload[:exception_object]\n    )\n  end\nend\n```\n\nThe following Active Experiment events are available for subscribers:\n\n- `start_experiment` - The experiment has begun.\n- `process_segment_callbacks` - The experiment has processed all segment rules. A variant may have been resolved through this step.\n- `process_variant_steps` - An experiment variant has been run.\n- `process_variant_callbacks` - The experiment has processed variant callbacks.\n- `process_run_callbacks` - The experiment has processed run callbacks.\n- `process_run` - The experiment has completed and can be reported on.\n\nIn each of these events, the experiment instance is available in the `event.payload` hash.\n\n## Experiments in views\n\nExperiments can be used in views, just like in any other part of your application. Sometimes though, you might want to render markup inside your run block too, and to do this, you'll need to \"capture\" the experiment.\n\nTo accomplish this, you can ask the experiment to capture itself by providing the view scope. The following examples (HAML or ERB) help illustrate how to avoid duplicating markup within each variant block by putting it (the container div for instance) in the run block.\n\nRemember to include the `ActiveExperiment::Capturable` module in your experiment class:\n\n```ruby\nclass MyExperiment \u003c ActiveExperiment::Base\n  include ActiveExperiment::Capturable\n  \n  variant(:red) { \"red\" }\n  variant(:blue) { \"blue\" }\nend\n```\n\n\u003cdetails\u003e\n\u003csummary\u003eExpand HAML example\u003c/summary\u003e\n\n```haml\n!= MyExperiment.set(capture: self).run(current_user) do |experiment|\n  %div.container\n    = experiment.on(:red) do\n      %button.red-pill Red\n    = experiment.on(:blue) do\n      %button.blue-pill Blue\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eExpand ERB example\u003c/summary\u003e\n\n```erb\n\u003c%== MyExperiment.set(capture: self).run(current_user) do |experiment| %\u003e\n  \u003cdiv class=\"container\"\u003e\n    \u003c%= experiment.on(:red) do %\u003e\n      \u003cbutton class=\"red-pill\"\u003eRed\u003c/button\u003e\n    \u003c% end %\u003e\n    \u003c%= experiment.on(:blue) do %\u003e\n      \u003cbutton class=\"blue-pill\"\u003eBlue\u003c/button\u003e\n    \u003c% end %\u003e\n  \u003c/div\u003e\n\u003c% end %\u003e\n```\n\u003c/details\u003e\n\nIf you don't need to capture the experiment, simply run like you would anywhere else:\n\n```erb\n\u003c% MyExperiment.run(current_user) do |experiment| %\u003e\n  \u003c% experiment.on(:red) do %\u003e\n    \u003cbutton class=\"red-pill\"\u003eRed\u003c/button\u003e\n  \u003c% end %\u003e\n  \u003c% experiment.on(:blue) do %\u003e\n    \u003cbutton class=\"blue-pill\"\u003eBlue\u003c/button\u003e\n  \u003c% end %\u003e\n\u003c% end %\u003e\n```\n\n## Client side experimentation\n\nWhile Active Experiment doesn't include any specific tooling for client side experimentation at this time, it does provide the ability to surface experiments in the client layer.\n\nWhenever an experiment is run in the request lifecycle, it's stored so it can be provided to the client. This means that if an experiment is run in controller, a view, a helper, etc. it will be available to the client.\n\nIn the layout, the experiment data can be rendered as JSON for instance:\n\n```erb\n\u003ctitle\u003eMy App\u003c/title\u003e\n\u003cscript\u003e\n  window.experiments = \u003c%== ActiveExperiment::Executed.to_json %\u003e\n\u003c/script\u003e\n```\n\nOr each experiment can be iterated over and rendered individually:\n\n```erb\n\u003c% ActiveExperiment::Executed.as_array.each do |experiment| %\u003e\n  \u003cmeta name=\"\u003c%= experiment.name %\u003e\" content=\"\u003c%== experiment.serialize.to_json %\u003e\"\u003e\n\u003c% end %\u003e\n```\n\n## Testing\n\nActive Experiment provides a test helper that can be used to stub experiments and assert that the expected experiments have been run.\n\nTo use the test helper, include it in your test case:\n\n```ruby\nclass MyTestCase \u003c ActiveSupport::TestCase\n  include ActiveExperiment::TestHelper\nend\n```\n\nNow you can stub experiments in your tests:\n\n```ruby\ntest \"stubbing experiments\" do\n  stub_experiment(MyExperiment, :red) do\n    # Now all MyExperiment experiments will assign the :red variant.\n  end\n\n  stub_experiment(MyExperiment, skip: true) do\n    # Now all MyExperiment experiments will be skipped.\n  end\nend\n```\n\nAssertion helpers are also available:\n\n```ruby\ntest \"asserting experiments\" do\n  # Assert that no experiments have been run.\n  assert_no_experiments\n\n  MyExperiment.run(id: 1)\n\n  # Assert that 1 experiment has been run.\n  assert_experiments 1\n\n  # Assert that within the block, 2 experiments will be run.\n  assert_experiments 2 do\n    MyExperiment.run(id: 2)\n    MyExperiment.run(id: 3)\n  end\n  \n  # Assert an experiment has been run with a given context.\n  assert_experiment_with(MyExperiment, context: { id: 1 })\n  \n  # Assert that within the block, a matching experiment will be run.\n  assert_experiment_with(MyExperiment, variant: :red, context: { id: 4 }) do\n    MyExperiment.set(variant: :red).run(id: 4)\n  end\nend\n```\n\nRSpec support can be added by requiring `active_experiment/rspec` in the appropriate spec helper.\n\n## GlobalID support\n\nActive Experiment supports [GlobalID serialization](https://github.com/rails/globalid/) for experiment contexts. This is part of what makes it possible to utilize Active Record objects as context to consistently assign the same variant across multiple runs.\n\n## Similar and noteworthy projects\n\n- [Vanity](https://vanity.labnotes.org/) - Experiment Driven Development framework for Rails.\n- [Scientist](https://github.com/github/scientist) - A Ruby library for carefully refactoring critical paths.\n- [Gitlab::Experiment](https://gitlab.com/gitlab-org/ruby/gems/gitlab-experiment) - A framework for running experiments, by GitLab.\n- [Split](https://github.com/splitrb/split) - The Rack Based A/B testing framework.\n\n## License\n\nActive Experiment is released under the MIT license:\n\n* https://opensource.org/licenses/MIT\n\nCopyright 2022 [jejacks0n](https://github.com/jejacks0n)\n\n## Make Code Not War\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjejacks0n%2Factiveexperiment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjejacks0n%2Factiveexperiment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjejacks0n%2Factiveexperiment/lists"}