Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/facebookarchive/planout
PlanOut is a library and interpreter for designing online experiments.
https://github.com/facebookarchive/planout
Last synced: about 1 month ago
JSON representation
PlanOut is a library and interpreter for designing online experiments.
- Host: GitHub
- URL: https://github.com/facebookarchive/planout
- Owner: facebookarchive
- License: other
- Archived: true
- Created: 2014-01-26T00:07:54.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2021-03-19T22:24:45.000Z (over 3 years ago)
- Last Synced: 2024-05-19T07:25:25.545Z (6 months ago)
- Language: JavaScript
- Homepage: http://facebook.github.io/planout
- Size: 5.18 MB
- Stars: 1,674
- Watchers: 87
- Forks: 215
- Open Issues: 25
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
# PlanOut
[](https://pypi.org/project/planout/)
[](https://pypi.org/project/planout/)
[](https://travis-ci.com/facebook/planout)
PlanOut is a multi-platform framework and programming language for online field experimentation. PlanOut was created to make it easy to run and iterate on sophisticated experiments, while satisfying the constraints of deployed Internet services with many users.
Developers integrate PlanOut by defining experiments that detail how _units_ (e.g., users, cookie IDs) should get mapped onto conditions. For example, to create a 2x2 experiment randomizing both the color and the text on a button, you create a class like this in Python:
```python
class MyExperiment(SimpleExperiment):
def assign(self, params, userid):
params.button_color = UniformChoice(choices=['#ff0000', '#00ff00'], unit=userid)
params.button_text = UniformChoice(choices=['I voted', 'I am a voter'], unit=userid)
```
Then, in the application code, you query the Experiment object to find out what values the current user should be mapped onto:
```python
my_exp = MyExperiment(userid=101)
color = my_exp.get('button_color')
text = my_exp.get('button_text')
```
PlanOut takes care of randomizing each ``userid`` into the right bucket. It does so by hashing the input, so each ``userid`` will always map onto the same values for that experiment.
### What does the PlanOut distribution include?
The reference implementation for PlanOut is written in Python. It includes:
* Extensible classes for defining experiments. These classes make it easy to implement reliable, deterministic random assignment procedures, and automatically log key data.
* A basic implementation of namespaces, which can be used to manage multiple mutually exclusive experiments.
* The PlanOut interpreter, which executes serialized code generated by the PlanOut domain specific language.
* An interactive Web-based editor and compiler for developing and testing
PlanOut-language scripts.
Other production-ready versions of PlanOut are available for Java, JavaScript, and PHP, and can found in the `java/`, `js/`, and `php/` directories, respectively.
The `alpha/` directory contains implementations of PlanOut to other languages that are currently under development, including Go, Julia, and Ruby.
### Who is PlanOut for?
PlanOut designed for researchers, students, and small businesses wanting to run experiments. It is built to be extensible, so that it may be adapted for use with large production environments. The implementation here mirrors many of the key components of Facebook's Hack-based implementation of PlanOut which is used to conduct experiments with hundreds of millions of users.
### Full Example
To create a basic PlanOut experiment in Python, you subclass ``SimpleExperiment`` object, and implement an assignment method. You can use PlanOut's random assignment operators by setting ``e.varname``, where ``params`` is the first argument passed to the ``assign()`` method, and ``varname`` is the name of the variable you are setting.
```python
from planout.experiment import SimpleExperiment
from planout.ops.random import *
class FirstExperiment(SimpleExperiment):
def assign(self, params, userid):
params.button_color = UniformChoice(choices=['#ff0000', '#00ff00'], unit=userid)
params.button_text = WeightedChoice(
choices=['Join now!', 'Sign up.'],
weights=[0.3, 0.7], unit=userid)
my_exp = FirstExperiment(userid=12)
# parameters may be accessed via the . operator
print my_exp.get('button_text'), my_exp.get('button_color')
# experiment objects include all input data
for i in xrange(6):
print FirstExperiment(userid=i)
```
which outputs:
```
Join now! #ff0000
{'inputs': {'userid': 0}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 1}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 2}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 3}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#ff0000', 'button_text': 'Sign up.'}}
{'inputs': {'userid': 4}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Join now!'}}
{'inputs': {'userid': 5}, 'checksum': '22c13b16', 'salt': 'FirstExperiment', 'name': 'FirstExperiment', 'params': {'button_color': '#00ff00', 'button_text': 'Sign up.'}}
```
The ``SimpleExperiment`` class will automatically concatenate the name of the experiment, ``FirstExperiment``, the variable name, and the input data (``userid``) and hash that string to perform the random assignment. Parameter assignments and inputs are automatically logged into a file called ``firstexperiment.log'``.
### Installation
You can immediately install the reference implementation of PlanOut for Python using `pip` with:
```
pip install planout
```
See the `java/`, `php/`, `js/`, `alpha/golang`, `alpha/ruby`, `alpha/julia` directories for instructions on installing PlanOut for other languages.
### Learn more
Learn more about PlanOut visiting the [PlanOut website](http://facebook.github.io/planout/) or by [reading the PlanOut paper](http://arxiv.org/pdf/1409.3174v1.pdf). You can cite PlanOut as "Designing and Deploying Online Field Experiments". Eytan Bakshy, Dean Eckles, Michael S. Bernstein. Proceedings of the 23rd ACM conference on the World Wide Web. April 7–11, 2014, Seoul, Korea, or by copying and pasting the bibtex below:
``` bibtex
@inproceedings{bakshy2014www,
Author = {Bakshy, E. and Eckles, D. and Bernstein, M.S.},
Booktitle = {Proceedings of the 23rd ACM conference on the World Wide Web},
Organization = {ACM},
Title = {Designing and Deploying Online Field Experiments},
Year = {2014}
}
```