{"id":23613681,"url":"https://github.com/freckle/scientist-hs","last_synced_at":"2025-10-28T15:54:14.393Z","repository":{"id":38441372,"uuid":"465332660","full_name":"freckle/scientist-hs","owner":"freckle","description":"Haskell port of github/scientist","archived":false,"fork":false,"pushed_at":"2025-08-15T08:39:18.000Z","size":35,"stargazers_count":4,"open_issues_count":1,"forks_count":0,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-09-18T02:24:45.623Z","etag":null,"topics":["ghvm-managed"],"latest_commit_sha":null,"homepage":"","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/freckle.png","metadata":{"files":{"readme":"README.lhs","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-02T14:10:25.000Z","updated_at":"2025-08-15T08:39:20.000Z","dependencies_parsed_at":"2025-01-06T01:23:44.483Z","dependency_job_id":"c433bc51-6e92-4b84-929e-ba7cf6aa45db","html_url":"https://github.com/freckle/scientist-hs","commit_stats":{"total_commits":16,"total_committers":3,"mean_commits":5.333333333333333,"dds":0.375,"last_synced_commit":"debf175d0637050b16c7f258cb9d632492f24f71"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/freckle/scientist-hs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freckle%2Fscientist-hs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freckle%2Fscientist-hs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freckle%2Fscientist-hs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freckle%2Fscientist-hs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/freckle","download_url":"https://codeload.github.com/freckle/scientist-hs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/freckle%2Fscientist-hs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281467275,"owners_count":26506462,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-28T02:00:06.022Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ghvm-managed"],"created_at":"2024-12-27T17:18:45.972Z","updated_at":"2025-10-28T15:54:14.352Z","avatar_url":"https://github.com/freckle.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scientist\n\n[![Hackage](https://img.shields.io/hackage/v/scientist.svg?style=flat)](https://hackage.haskell.org/package/scientist)\n[![Stackage Nightly](http://stackage.org/package/scientist/badge/nightly)](http://stackage.org/nightly/package/scientist)\n[![Stackage LTS](http://stackage.org/package/scientist/badge/lts)](http://stackage.org/lts/package/scientist)\n[![CI](https://github.com/freckle/scientist-hs/actions/workflows/ci.yml/badge.svg)](https://github.com/freckle/scientist-hs/actions/workflows/ci.yml)\n\nHaskell port of\n[`github/scientist`](https://github.com/github/scientist#readme).\n\n## Usage\n\nThe following extensions are recommended:\n\n```haskell\n{-# LANGUAGE OverloadedStrings #-}\n```\n\n\u003c!--\n```haskell\n{-# LANGUAGE DeriveAnyClass #-}\n{-# LANGUAGE DeriveGeneric #-}\n{-# LANGUAGE DerivingStrategies #-}\n{-# LANGUAGE LambdaCase #-}\n\nmodule Main (module Main) where\n\nimport Prelude\n\nimport Control.Monad.IO.Class (MonadIO(..))\nimport Control.Monad.IO.Unlift (MonadUnliftIO)\nimport Data.Foldable (for_)\nimport Data.Text (Text)\nimport UnliftIO.Exception (SomeException, throwIO, displayException)\n```\n--\u003e\n\nMost usage will only require the top-level library:\n\n```haskell\nimport Scientist\n```\n\n\u003c!--\n```haskell\nimport Text.Markdown.Unlit ()\n\ntheOriginalCode :: m a\ntheOriginalCode = undefined\n\ntheExperimentalCode :: m a\ntheExperimentalCode = undefined\n```\n--\u003e\n\n1. Define a new `Experiment m c a b` with,\n\n   ```haskell\n   ex0 :: Functor m =\u003e Experiment m c a b\n   ex0 = newExperiment \"some name\" theOriginalCode\n   ```\n\n1. The type variables capture the following details:\n\n   - `m`: Some Monad to operate in, e.g. `IO`.\n   - `c`: Any context value you attach with `setExperimentContext`. It will then\n     be available in the `Result c a b` you publish.\n   - `a`: The result type of your original (aka \"control\") code, this is what is\n     always returned and so is the return type of `experimentRun`.\n   - `b`: The result type of your experimental (aka \"candidate\") code. It\n     probably won't differ (and must not to use a comparison like `(==)`), but\n     it can (provided you implement a comparison between `a` and `b`).\n\n1. Configure the Experiment as desired\n\n   ```haskell\n   ex1 :: (Functor m, Eq a) =\u003e Experiment m c a a\n   ex1 =\n     setExperimentPublish publish0\n       $ setExperimentCompare experimentCompareEq\n       $ setExperimentTry theExperimentalCode\n       $ newExperiment \"some name\" theOriginalCode\n\n   -- Increment Statsd, Log, store in Redis, whatever\n   publish0 :: Result c a b -\u003e m ()\n   publish0 = undefined\n   ```\n\n1. Run the experiment\n\n   ```haskell\n   run0 :: (MonadUnliftIO m, Eq a) =\u003e m a\n   run0 =\n     experimentRun\n       $ setExperimentPublish publish0\n       $ setExperimentCompare experimentCompareEq\n       $ setExperimentTry theExperimentalCode\n       $ newExperiment \"some name\" theOriginalCode\n   ```\n\n1. Explore things like `setExperimentIgnore`, `setExperimentEnabled`, etc.\n\n---\n\nThe rest of this README matches section-by-section to the ported project and\nshows only the differences in syntax for those features. Please follow the\nheader links for additional details, motivation, etc.\n\n## [How do I science?](https://github.com/github/scientist#how-do-i-science)\n\n\u003c!--\n```haskell\ndata Model = Model\n\ndata User = User\n\nuserLogin :: User -\u003e Text\nuserLogin = undefined\n\nuserCanRead :: User -\u003e Model -\u003e m Bool\nuserCanRead = undefined\n\nmodelCheckUserValid :: Model -\u003e User -\u003e m Bool\nmodelCheckUserValid = undefined\n```\n--\u003e\n\n```haskell\nmyWidgetAllows :: MonadUnliftIO m =\u003e Model -\u003e User -\u003e m Bool\nmyWidgetAllows model user = do\n  experimentRun\n    $ setExperimentTry\n        (userCanRead user model) -- new way\n    $ newExperiment \"widget-permissions\"\n        (modelCheckUserValid model user) -- old way\n```\n\n## [Making science useful](https://github.com/github/scientist#making-science-useful)\n\n```haskell\nrun1 :: MonadUnliftIO m =\u003e m a\nrun1 =\n  experimentRun\n    $ setExperimentEnabled (pure True)\n    $ setExperimentOnException onScientistException\n    $ setExperimentPublish (liftIO . putStrLn . formatResult)\n    $ setExperimentTry theExperimentalCode\n    $ newExperiment \"some name\" theOriginalCode\n\nonScientistException :: MonadIO m =\u003e SomeException -\u003e m ()\nonScientistException ex = do\n  liftIO $ putStrLn \"...\"\n\n  -- To re-raise\n  throwIO ex\n\nformatResult :: Result c a b -\u003e String\nformatResult = undefined\n```\n\n### [Controlling comparison](https://github.com/github/scientist#controlling-comparison)\n\n\u003c!--\n```haskell\nuserServiceFetch :: m [User]\nuserServiceFetch = undefined\n\nfetchAllUsers :: m [User]\nfetchAllUsers = undefined\n```\n---\u003e\n\n```haskell\nrun2 :: MonadUnliftIO m =\u003e m [User]\nrun2 =\n  experimentRun\n    $ setExperimentCompare (experimentCompareOn $ map userLogin)\n    $ setExperimentTry userServiceFetch\n    $ newExperiment \"users\" fetchAllUsers\n```\n\nWhen using `experimentCompareOn`, `By`, or `Eq`, if a candidate branch raises an\nexception, that will never compare equally.\n\n### [Adding context](https://github.com/github/scientist#adding-context)\n\nSee `setExperimentContext`.\n\n### [Expensive setup](https://github.com/github/scientist#expensive-setup)\n\nJust do it ahead of time.\n\n\u003c!--\n```haskell\ndata X = X\n\nexpensiveSetup :: m X\nexpensiveSetup = undefined\n\ntheOriginalCodeWith :: X -\u003e m a\ntheOriginalCodeWith = undefined\n\ntheExperimentalCodeWith :: X -\u003e m a\ntheExperimentalCodeWith = undefined\n```\n--\u003e\n\n```haskell\nrun3 :: MonadUnliftIO m =\u003e m a\nrun3 = do\n  x \u003c- expensiveSetup\n\n  experimentRun\n    $ setExperimentTry (theExperimentalCodeWith x)\n    $ newExperiment \"expensive\" (theOriginalCodeWith x)\n```\n\n### [Keeping it clean](https://github.com/github/scientist#keeping-it-clean)\n\nNot supported at this time. Format the value(s) as necessary when publishing.\n\n### [Ignoring mismatches](https://github.com/github/scientist#ignoring-mismatches)\n\nSee `setExperimentIgnore`.\n\n### [Enabling/disabling experiments](https://github.com/github/scientist#enablingdisabling-experiments)\n\nSee `setExperimentRunIf`.\n\n### [Ramping up experiments](https://github.com/github/scientist#ramping-up-experiments)\n\n```haskell\nrun4 :: MonadUnliftIO m =\u003e m a\nrun4 =\n  experimentRun\n    $ setExperimentEnabled (experimentEnabledPercent 30)\n    $ setExperimentTry theExperimentalCode\n    $ newExperiment \"some name\" theOriginalCode\n```\n\n### [Publishing results](https://github.com/github/scientist#publishing-results)\n\n\u003c!--\n```haskell\ndata Value = Value\n\ndata Pair = Pair\n\ntoJSON :: a -\u003e Value\ntoJSON = undefined\n\n(.=) :: Text -\u003e a -\u003e Pair\n(.=) = undefined\n\nobject :: [Pair] -\u003e Value\nobject = undefined\n\ndata MyContext = MyContext\n\ndata MyPayload = MyPayload\n  { name :: Text\n  , context :: Maybe MyContext\n  , control :: Value\n  , candidate :: Value\n  , execution_order :: [Text]\n  }\n\nstatsdTiming :: Text -\u003e a -\u003e m ()\nstatsdTiming = undefined\n\nstatsdIncrement :: Text -\u003e m ()\nstatsdIncrement = undefined\n\nredisLpush :: Text -\u003e Value -\u003e m ()\nredisLpush = undefined\n\nredisLtrim :: Text -\u003e Int -\u003e Int -\u003e m ()\nredisLtrim = undefined\n```\n--\u003e\n\n```haskell\nrun5 :: MonadUnliftIO m =\u003e m User\nrun5 =\n  experimentRun\n    $ setExperimentPublish publish1\n    $ setExperimentTry theExperimentalCode\n    $ newExperiment \"some name\" theOriginalCode\n\npublish1 :: MonadIO m =\u003e Result MyContext User User -\u003e m ()\npublish1 result = do\n  -- Details are present unless it's a ResultSkipped, which we'll ignore\n  for_ (resultDetails result) $ \\details -\u003e do\n    let eName = resultDetailsExperimentName details\n\n    -- Store the timing for the control value,\n    statsdTiming (\"science.\" \u003c\u003e eName \u003c\u003e \".control\")\n      $ resultControlDuration\n      $ resultDetailsControl details\n\n    -- for the candidate (only the first, see \"Breaking the rules\" below,\n    statsdTiming (\"science.\" \u003c\u003e eName \u003c\u003e \".candidate\")\n      $ resultCandidateDuration\n      $ resultDetailsCandidate details\n\n    -- and counts for match/ignore/mismatch:\n    case result of\n      ResultSkipped{} -\u003e pure ()\n      ResultMatched{} -\u003e do\n        statsdIncrement $ \"science.\" \u003c\u003e eName \u003c\u003e \".matched\"\n      ResultIgnored{} -\u003e do\n        statsdIncrement $ \"science.\" \u003c\u003e eName \u003c\u003e \".ignored\"\n      ResultMismatched{} -\u003e do\n        statsdIncrement $ \"science.\" \u003c\u003e eName \u003c\u003e \".mismatched\"\n        -- Finally, store mismatches in redis so they can be retrieved and\n        -- examined later on, for debugging and research.\n        storeMismatchData details\n\nstoreMismatchData :: Monad m =\u003e ResultDetails MyContext User User -\u003e m ()\nstoreMismatchData details = do\n  let\n    eName = resultDetailsExperimentName details\n    eContext = resultDetailsExperimentContext details\n\n    payload = MyPayload\n      { name = eName\n      , context = eContext\n      , control = controlObservationPayload $ resultDetailsControl details\n      , candidate = candidateObservationPayload $ resultDetailsCandidate details\n      , execution_order = resultDetailsExecutionOrder details\n      }\n\n    key = \"science.\" \u003c\u003e eName \u003c\u003e \".mismatch\"\n\n  redisLpush key $ toJSON payload\n  redisLtrim key 0 1000\n\ncontrolObservationPayload :: ResultControl User -\u003e Value\ncontrolObservationPayload rc =\n  object [\"value\" .= cleanValue (resultControlValue rc)]\n\ncandidateObservationPayload :: ResultCandidate User -\u003e Value\ncandidateObservationPayload rc = case resultCandidateValue rc of\n  Left ex -\u003e object [\"exception\" .= displayException ex]\n  Right user -\u003e object [\"value\" .= cleanValue user]\n\n-- See \"Keeping it clean\" above\ncleanValue :: User -\u003e Text\ncleanValue = userLogin\n```\n\nSee `Result`, `ResultDetails`, `ResultControl` and `ResultCandidate` for all the\navailable data you can publish.\n\n### [Testing](https://github.com/github/scientist#testing)\n\n**TODO**: `raise_on_mismatches`\n\n#### [Custom mismatch errors](https://github.com/github/scientist#custom-mismatch-errors)\n\n**TODO**: `raise_with`\n\n### [Handling errors](https://github.com/github/scientist#handling-errors)\n\n#### [In candidate code](https://github.com/github/scientist#in-candidate-code)\n\nCandidate code is wrapped in `tryAny`, resulting in `Either SomeException`\nvalues in the result candidates list. We use the [safer][blog]\n`UnliftIO.Exception` module.\n\n[blog]: https://www.fpcomplete.com/haskell/tutorial/exceptions/\n\n#### [In a Scientist callback](https://github.com/github/scientist#in-a-scientist-callback)\n\nSee `setExperimentOnException`.\n\n## [Breaking the rules](https://github.com/github/scientist#breaking-the-rules)\n\n### [Ignoring results entirely](https://github.com/github/scientist#ignoring-results-entirely)\n\n```haskell\nnope0 :: Experiment m c a b -\u003e Experiment m c a b\nnope0 = setExperimentIgnore (\\_ _ -\u003e True)\n```\n\nOr, more efficiently:\n\n```haskell\nnope1 :: Experiment m c a b -\u003e Experiment m c a b\nnope1 = setExperimentCompare (\\_ _ -\u003e True)\n```\n\n### [Trying more than one thing](https://github.com/github/scientist#trying-more-than-one-thing)\n\nIf you call `setExperimentTry` more than once, it will append (not overwrite)\ncandidate branches. If any candidate is deemed ignored or a mismatch, the\noverall result will be.\n\n`setExperimentTryNamed` can be used to give branches explicit names (otherwise,\nthey are \"control\", \"candidate\", \"candidate-{n}\"). These names are visible in\n`ResultControl`, `ResultCandidate`, and `resultDetailsExecutionOrder`.\n\n### [No control, just candidates](https://github.com/github/scientist#no-control-just-candidates)\n\nNot supported.\n\nSupporting the lack of a Control branch in the types would ultimately lead to a\nruntime error if you attempt to run such an `Experiment` without having and\nnaming a Candidate to use instead, or severely complicate the types to account\nfor that safely. In our opinion, this feature is not worth either of those.\n\n\u003c!--\n```haskell\nmain :: IO ()\nmain = pure ()\n```\n--\u003e\n\n---\n\n[LICENSE](./LICENSE) | [CHANGELOG](./CHANGELOG.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreckle%2Fscientist-hs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffreckle%2Fscientist-hs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffreckle%2Fscientist-hs/lists"}