{"id":16131869,"url":"https://github.com/andrewvy/experiment","last_synced_at":"2025-03-18T14:31:29.220Z","repository":{"id":62429369,"uuid":"51781272","full_name":"andrewvy/experiment","owner":"andrewvy","description":":microscope: Elixir Library for carefully refactoring critical paths by performing experiments.","archived":false,"fork":false,"pushed_at":"2016-02-17T21:05:40.000Z","size":130,"stargazers_count":8,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-04-14T05:57:08.350Z","etag":null,"topics":["elixir","library","refactoring"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andrewvy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-15T19:54:17.000Z","updated_at":"2023-02-03T15:31:57.000Z","dependencies_parsed_at":"2022-11-01T20:07:10.085Z","dependency_job_id":null,"html_url":"https://github.com/andrewvy/experiment","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewvy%2Fexperiment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewvy%2Fexperiment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewvy%2Fexperiment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andrewvy%2Fexperiment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andrewvy","download_url":"https://codeload.github.com/andrewvy/experiment/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243933431,"owners_count":20370986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","library","refactoring"],"created_at":"2024-10-09T22:27:53.857Z","updated_at":"2025-03-18T14:31:28.946Z","avatar_url":"https://github.com/andrewvy.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Experiment\n\n[![Hex.pm](https://img.shields.io/hexpm/v/experiment.svg?style=flat-square)](https://hex.pm/packages/experiment)\n[![Hex.pm](https://img.shields.io/hexpm/dt/experiment.svg?style=flat-square)](https://hex.pm/packages/experiment)\n[![Inline docs](http://inch-ci.org/github/andrewvy/experiment.svg)](http://inch-ci.org/github/andrewvy/experiment)\n\nElixir Library for carefully refactoring critical paths, influenced heavily by [github/scientist](https://github.com/github/scientist).\n\n---\n\nRunning Elixir in production? ExUnit says all your tests pass? Can't wait to be\nutilizing the awesome hot-code reloading feature to push up your new changes?\nWell, hold on there, for critical parts of your application you may need more\nreassurance.\n\nExperiment allows you to run refactored code side-by-side (concurrently) with your previously\nwritten code, compare the outputs of each, and notifying when something didn't\nreturn as expected.\n\n---\n\n### Documentation\n\nDocumentation is available at [http://hexdocs.pm/experiment](http://hexdocs.pm/experiment)\n\n---\n\n### Installation\n\nAdd Experiment as a dependency to your project.\n\n```elixir\ndefp deps do\n  [{:experiment, \"~\u003e 0.0.3\"}]\nend\n```\n\nThen run `mix deps.get` to fetch it.\n\n---\n\n### Usage\n\nLet's say you're refactoring a super important controller. Tests can help, but\nsometimes you really want to pit your refactored code against your current code.\n\n```elixir\ndefmodule App.ImportantAPIController do\n  def index(conn, params) do\n    widget =\n      Experiment.new(\"returns widget for rendering\")\n      |\u003e Experiment.test(\u0026func_to_experiment/1, [\"foo\"])\n      |\u003e Experiment.test(\"Named Experiment\", \u0026another_func_to_experiment/0)\n      |\u003e Experiment.control(\u0026func_that_works/0)\n      |\u003e Experiment.compare(\u0026custom_compare_tests/2)\n      |\u003e Experiment.perform_experiment\n\n    render conn, widget: widget\n  end\n\n  def func_to_experiment(type) do\n    %{type: type}\n  end\n\n  def another_func_to_experiment do\n    %{type: \"bar\"}\n  end\n\n  def func_that_works do\n    %{type: \"foo\"}\n  end\n\n  # Override default behavior which `==` the control and candidate.\n  def custom_compare_tests(control, candidate) do\n    control.type == candidate.type\n  end\nend\n```\n\nBy default, the default Experiment adapter uses `Logger.info` for outputting. If an experiment crashes, a stacktrace is provided as it's result.\n\n```\n12:53:14.250 [info]  [Experiment] Example: returns widget for rendering - Test 1\n\nControl: {:ok, :bar}\nCandidate: {:ok, :foo}\n\n\n12:53:14.256 [info]  [Experiment] returns widget for rendering - Test 1\n\nControl: {:ok, :foo}\nCandidate: \"** (ErlangError) erlang error: ArgumentError\\n    test/experiment_test.exs:139: anonymous fn/1 in ExperimentTest.test handles experiment exceptions/1\\n    (experiment) lib/experiment/utils.ex:18: anonymous fn/2 in Experiment.Utils.bind/2\\n    (experiment) lib/experiment.ex:99: anonymous fn/1 in Experiment.perform_experiment/1\\n    (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2\\n    (elixir) lib/task/supervised.ex:40: Task.Supervised.reply/5\\n    (stdlib) proc_lib.erl:240: :proc_lib.init_p_do_apply/3\\n\"\n```\n\n---\n\n### Customizing Adapter\n\nYou can define your own Experiment adapter for recording results by using the `Experiment.Base` module.\n\n```elixir\ndefmodule App.ExperimentAdapter do\n  use Experiment.Base\n  require Logger\n\n  def record(lab, control, candidate) do\n    # Get the lab name\n    Logger.info lab.name\n\n    # Log out control's result\n    Logger.info \"#{control.name} - #{inspect control.result}\"\n\n    # Log out candidate's result\n    Logger.info \"#{candidate.name} - #{inspect candidate.result}\"\n\n    # Do something with the results, save them to the DB, etc.\n  end\nend\n```\n\nIn your config, you can specify your own Experiment adapter.\n\n```elixir\nconfig :experiment,\n  adapter: App.ExperimentAdapter\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrewvy%2Fexperiment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandrewvy%2Fexperiment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrewvy%2Fexperiment/lists"}