{"id":19251039,"url":"https://github.com/onesignal/thesis","last_synced_at":"2025-10-04T16:19:59.505Z","repository":{"id":48095898,"uuid":"319834535","full_name":"OneSignal/thesis","owner":"OneSignal","description":"Rust library for controling experimental refactors","archived":false,"fork":false,"pushed_at":"2025-04-16T23:43:45.000Z","size":52,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":28,"default_branch":"main","last_synced_at":"2025-04-17T11:23:40.141Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OneSignal.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-12-09T03:48:27.000Z","updated_at":"2025-04-16T23:43:48.000Z","dependencies_parsed_at":"2024-11-09T18:21:36.755Z","dependency_job_id":"126e705b-80db-4bf1-b20f-c9b1673df501","html_url":"https://github.com/OneSignal/thesis","commit_stats":{"total_commits":19,"total_committers":3,"mean_commits":6.333333333333333,"dds":"0.10526315789473684","last_synced_commit":"2a54be597d572a835b6fe4b2679ca03530fb95cf"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OneSignal%2Fthesis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OneSignal%2Fthesis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OneSignal%2Fthesis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OneSignal%2Fthesis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OneSignal","download_url":"https://codeload.github.com/OneSignal/thesis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250056299,"owners_count":21367529,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T18:19:43.257Z","updated_at":"2025-10-04T16:19:54.432Z","avatar_url":"https://github.com/OneSignal.png","language":"Rust","readme":"# Thesis\n\nInspired by https://github.com/github/scientist\n\nThesis provides the `Experiment` struct, which represents an experiment to\nrun which compares the return values of multiple methods for accomplishing\nthe same task.\n\nLet's imagine that we already have a function called `load_data_from_db`,\nwhich loads some data from a database. We want to refactor this to instead\nload the same data from redis. We write a new function called\n`load_data_from_redis` to accomplish the same task, but with redis instead of\na DB. We want to use the redis version on only a very small percentage of\ntraffic, say 0.5% of incoming requests, and we want to log it out if the\nredis data doesn't match the DB data, treating the DB data as accurate and\ndiscarding the redis data if so. Here's how we can use an `Experiment` to do\nthis.\n\n```rust\nuse thesis::{Experiment, rollout::Percent};\n\nasync fn load_data_from_db(id: i32) -\u003e i32 { id }\nasync fn load_data_from_redis(id: i32) -\u003e i32 { id }\n\nlet id = 4;\nlet result = Experiment::new(\"load_data_from_db =\u003e load_data_from_redis\")\n    .control(load_data_from_db(id))\n    .experimental(load_data_from_redis(id))\n    .rollout_strategy(Percent::new(0.5))\n    .on_mismatch(|mismatch| {\n        eprintln!(\n            \"DB \u0026 Redis data differ - db={}, redis={}\",\n            mismatch.control,\n            mismatch.experimental,\n        );\n\n        // the `control` value here comes from the DB\n        mismatch.control\n    })\n    .run()\n    .await;\n\nassert_eq!(result, 4);\n```\n\n# Monitoring\n\nBecause thesis is designed to be used for refactoring operations in\nproduction systems, there are a few built-in features for monitoring and\nobservability. Some contextual information is provided via spans created with\nthe `tracing` crate, as well as some metrics via the `metrics` crate.\n\n## Metrics provided (with tags)\n\n- `thesis_experiment_run_total` - counter incremented each time the `run`\n  function is called\n    - `name` - name of the experiment provided to the constructor\n- `thesis_experiment_run_variant` - counter incremented each time a\n  variant (defined as control vs experimental) is run\n    - `name` - name of the experiment\n    - `kind` - one of `control`, `experimental`, `experimental_and_compare`\n- `thesis_experiment_outcome` - counter incremented each time an experiment\n  has an observable outcome\n    - `name` - name of the experiment\n    - `kind` - one of `control`, `experimental`, `experimental_and_compare`\n    - `outcome` - one of `ok`, `error`, `mismatch` (ok/error only produced\n    via `Experiment::run_result`)\n\n# Result handling\n\nIf your experimental (or control) methods may return an error, you should use\nthe `run_result` method on the `Experiment` builder. This method has special\nhandling and metrics reporting for `Result` types. When\n`RolloutDecision::UseControl` or `RolloutDecision::UseExperimental` are used,\n`run_result` works the same as `run`. Nothing special happens, even if the\nblock returns an error. Here's what happens when\n`RolloutDecision::UseExperimentalAndCompare` is used.\n\n| Control  | Experimental | Return Value             | Metrics (label values of `thesis_experiment_outcome`)                                                                   | Logs                                                                                                      |\n|----------|--------------|--------------------------|-------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------|\n| `Ok(X)`  | `Ok(X)`      | `Ok(X)`                  | `{kind=control, outcome=ok}`, `{kind=experimental, outcome=ok}`                                                         |                                                                                                           |\n| `Ok(X)`  | `Ok(Y)`      | Result of `on_mismatch`  | `{kind=control, outcome=ok}`, `{kind=experimental, outcome=ok}`, `{kind=experimental_and_compare, outcome=mismatch}`    |                                                                                                           |\n| `Ok(X)`  | `Err(e)`     | `Ok(X)`                  | `{kind=control, outcome=ok}`, `{kind=experimental, outcome=error}`, `{kind=experimental_and_compare, outcome=mismatch}` | `\"thesis experiment error\" kind=experimental, error=e`                                                    |\n| `Err(e)` | `Ok(x)`      | Result of  `on_mismatch` | `{kind=control, outcome=error}`, `{kind=experimental, outcome=ok}`, `{kind=experimental_and_compare, outcome=mismatch}` | `\"thesis experiment error\" kind=control, error=e`                                                         |\n| `Err(e)` | `Err(f)`     | `Err(e)`                 | `{kind=control, outcome=error}`, `{kind=experimental, outcome=error}`                                                   | `\"thesis experiment error\" kind=control, error=e`, `\"thesis experiment error\" kind=experimental, error=f` |\n\n# Limitations\n\n- The `control` and `experimental` futures must both have the same `Output`\n  types\n- There are no defaults provided for `control`, `experimental`, or\n  `rollout_strategy`, all of these methods must be called or the experiment\n  will not compile.\n- `control` and `experimental` must both be futures. A non-async version of\n  `Experiment` could be written, but this library does not currently provide\n  one.\n- The `name` provided to the experiment must be a `\u0026'static str`. We use the\n  `metrics` library for reporting metric information, which requires us to\n  either to use an owned `String` each time an `Experiment` is created, or to\n  require a static string. Allocating a `String` seems more wasteful than\n  limiting dynamicly created experiment names.\n- When using `run_result`, both `Result` types must have the same `Err` type.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonesignal%2Fthesis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fonesignal%2Fthesis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fonesignal%2Fthesis/lists"}