https://github.com/kessler/pickpick
An A/B testing engine
https://github.com/kessler/pickpick
ab-testing javascript node-module nodejs nodejs-modules npm-module npm-package
Last synced: about 1 month ago
JSON representation
An A/B testing engine
- Host: GitHub
- URL: https://github.com/kessler/pickpick
- Owner: kessler
- License: mit
- Created: 2018-07-14T02:20:33.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2024-02-19T15:10:44.000Z (about 1 year ago)
- Last Synced: 2025-03-17T08:09:25.516Z (about 2 months ago)
- Topics: ab-testing, javascript, node-module, nodejs, nodejs-modules, npm-module, npm-package
- Language: JavaScript
- Homepage:
- Size: 163 KB
- Stars: 37
- Watchers: 4
- Forks: 6
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ANNOUCEMENT: repository changing ownership
On the 16th of March 2024 this repository will change ownership and move to the personal account of one of it's maintainers, [Yaniv Kessler](https://github.com/kessler)
Ownership change will also occur on the npm registry to [Yaniv Kessler](https://www.npmjs.com/~kessler)
# pickpick
An A/B testing engine
[](https://circleci.com/gh/ironSource/pickpick)
[](https://coveralls.io/github/ironSource/pickpick?branch=master)
[](https://www.npmjs.com/package/pickpick)
To use this engine, create one or more experiments and stick them in a container. Each experiment is composed of a bunch of variations which are randomly picked/served to you. Once you obtain a variation, use it to render some content or make some sort of a decision.
The container's job is to select the correct experiments for each `visitor` based on each experiment's targeting expression. Thusly, to get a variation one would call `pick()` twice, once on the container and once on the experiment.
Further information about the targeting experssion syntax can be found here: [pickpick-targeting-compiler](https://github.com/ironSource/pickpick-targeting-compiler)
Also, please take a look at our [examples](./examples)
## example
`npm i -S pickpick`
Let's say we have a website with two pages `buy` and `index` and we want to run 3 experiments:
- on the `buy` page test `color button`
- on the `buy` page test `price`
- on the `index` page test `text`
```js
const { Experiment, ExperimentContainer } = require('pickpick')
// first create the experiments:
let e1 = Experiment.create({
name: 'buy page button color experiment',
id: '953d6fe0',
variations: [
{ object: '#ff0000', weight: 4 },
{ object: '#ff0000', weight: 1 },
{ object: '#00ff00', weight: 1 }
],
targeting: '_.path in ["buy", "index"]'
})
let e2 = Experiment.create({
name: 'buy page price experiment',
id: 'a40f09ac',
variations: [
{ object: 25 },
{ object: 35 },
{ object: 45 }
],
targeting: '_.path !== "home" && page !== "foo"'
})
let e3 = Experiment.create({
name: 'index text experiment',
id: 'ac49ef42',
variations: [
{ object: 'hi' },
{ object: 'hello' },
{ object: 'welcome' }
],
targeting: '_.path === "index"'
})
// now create a container:
let experiments = [e1, e2, e3]
let container = ExperimentContainer.create({ experiments })
// simulate a visitor that needs a determination about which variation of which experiment he gets:
let visitor = { page: 'index' }
for (let i = 0; i < 10; i++) {
let experiment = container.pick(visitor)
if (!experiment) {
// no experiment that targets this user
// handle this with defaults
console.log('default goes here')
} else {
console.log(`selected experiment '${experiment.name}' for '${JSON.stringify(visitor)}'`)
let variation = experiment.pick()
console.log(`selected variation is ${variation}`)
}
}
```
## API
#### Table of Contents
- [Experiment](#experiment)
- [Parameters](#parameters)
- [pick](#pick)
- [match](#match)
- [Parameters](#parameters-1)
- [add](#add)
- [Parameters](#parameters-2)
- [iterator](#iterator)
- [Targeting](#targeting)
- [Parameters](#parameters-3)
- [match](#match-1)
- [Parameters](#parameters-4)
- [expression](#expression)
- [iterator](#iterator-1)
- [has](#has)
- [Parameters](#parameters-5)
- [Variation](#variation)
- [Parameters](#parameters-6)
- [ExperimentContainer](#experimentcontainer)
- [Parameters](#parameters-7)
- [add](#add-1)
- [Parameters](#parameters-8)
- [pick](#pick-1)
- [Parameters](#parameters-9)
- [targetingFeatures](#targetingfeatures)
- [iterator](#iterator-2)
- [has](#has-1)
- [Parameters](#parameters-10)
- [hasId](#hasid)
- [Parameters](#parameters-11)
- [toJSON](#tojson)
### Experiment
[lib/Experiment.js:34-171](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Experiment.js#L34-L171 "Source code on GitHub")
An A/B test experiment contains one or more [variations](@Variation) and a definition of [targeting](@Targeting).
Experiments are serializable and can be created using classes from this engine or object literals. For example:
```js
const { Experiment } = require('pickpick')
const e1 = Experiment.create({
name: 'my experiment',
id: 'foo',
variations: [
{ object: 1, weight: 1 },
{ object: 2, weight: 1 },
{ object: 3, weight: 1 }
],
targeting: '_.geo === "US"'
})
```
#### Parameters
- `$0` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
- `$0.name`
- `$0.id`
- `$0.variations` (optional, default `[]`)
- `$0.targeting` (optional, default `Targeting.default()`)
- `$0.userData`
#### pick
[lib/Experiment.js:66-68](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Experiment.js#L66-L68 "Source code on GitHub")
randomly select one variation from the Variations set
Returns **Variant** the value contained within the selected variation
#### match
[lib/Experiment.js:76-78](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Experiment.js#L76-L78 "Source code on GitHub")
check if this experiment matches the input targeting
##### Parameters
- `targeting` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
Returns **[Boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
#### add
[lib/Experiment.js:84-95](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Experiment.js#L84-L95 "Source code on GitHub")
add another variation to this experiment
##### Parameters
- `variation` **([Variation](#variation) \| [Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object))**
#### iterator
[lib/Experiment.js:116-118](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Experiment.js#L116-L118 "Source code on GitHub")
iterate over the variations contained in this experiment
### Targeting
[lib/Targeting.js:11-80](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Targeting.js#L11-L80 "Source code on GitHub")
Targeting
#### Parameters
- `expression` **[String](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** see [pickpick-targeting-compiler](https://github.com/ironSource/pickpick-targeting-compiler) for more details
- `userEnvironment` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** (optional, default `{}`)
#### match
[lib/Targeting.js:31-39](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Targeting.js#L31-L39 "Source code on GitHub")
check if the input data is matched by this targeting instance
##### Parameters
- `inputTargeting` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)** is normally a simple js object
Returns **[Boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
#### expression
[lib/Targeting.js:45-47](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Targeting.js#L45-L47 "Source code on GitHub")
access this Targeting's expression
Returns **[String](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
#### iterator
[lib/Targeting.js:52-54](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Targeting.js#L52-L54 "Source code on GitHub")
iterate over the features that participate in the targeting
#### has
[lib/Targeting.js:61-63](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Targeting.js#L61-L63 "Source code on GitHub")
check if a feature is part of this targeting instance
##### Parameters
- `feature` **[String](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)** a name of a feature, e.g `geo`
Returns **[Boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
### Variation
[lib/Variation.js:9-99](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/Variation.js#L9-L99 "Source code on GitHub")
A variation attaches weight to a piece of data. Variations are used in Experiments and ExperimentContainers
#### Parameters
- `$0` **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**
- `$0.object`
- `$0.weight` (optional, default `1`)
### ExperimentContainer
[lib/ExperimentContainer.js:38-239](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L38-L239 "Source code on GitHub")
Contains one or more experiments and routes traffic evenly to each of them based on their
targeting. The following is an example of using a container to host several experiments, pick
on thats appropriate for a single visitor's targeting and then access a variation from the
selected experiment:
```js
const { ExperimentContainer, Experiment } = require('pickpick')
const experiments = [
Experiment.create(...),
Experiment.create(...),
Experiment.create(...)
]
const container = ExperimentContainer.create({ experiments })
let experiment = container.pick({ geo: 'US', page: 'index.html '})
if (experiment) {
let variation = experiment.pick()
// do something with the variation data
} else {
console.log('no experiments that match this targeting were found')
}
```
#### Parameters
- `__seed` **[number](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number)** just for testing / predictable engine results
#### add
[lib/ExperimentContainer.js:77-102](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L77-L102 "Source code on GitHub")
Add an experiment to this container. Inside a container experiments must
have unique ids. This method can accept different kinds of experiment expressions:
- an instance of Experiment:
```js
container.add(Experiment.create(...))
```
- An instance of Variation where it's object is an Experiment:
```js
container.add(Variation.create(Experiment.create(...)))
````
- An instance of Variation where it's object is an Expriment defined as an object literal:
```js
container.add(Variation.create({... experiment data ...}))
```
- A variation object literal wrapping an experiment object literal, this is useful in deserialization scenarios:
```js
container.add({ object: {... experiment data }, weight: 5 })
```
##### Parameters
- `experiments` **...any**
#### pick
[lib/ExperimentContainer.js:131-151](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L131-L151 "Source code on GitHub")
The pick method accepts a targeting object and randomly selects an experiment
from a set of experiments that match the targeting specification.
By default, selection is random and even, however, bias can be applied by specifying a weight
when adding an experiment to the container (see ExperimentContainer.add())
Weights are considered at the moment of selection from the current set of
matching experiments, therefor, careful planning of targeting is required to achieve
accurate traffic distribution betwee experiments.
For example, consider two experiments, `E1`, that targets `{ geo: 'US', page: '*' }` and `E2` that targets
`{ geo: 'US', page: 'index.html' }`. If both had the weight `1`, given the following stream
of visitors:
{ geo: 'US', page: 'sale.html' }
{ geo: 'US', page: 'index.html' }
{ geo: 'US', page: 'sale.html' }
{ geo: 'US', page: 'index.html' }
Then it is more likely that `E1` will receive more traffic than `E2` since `E1` competes with `E2` evenly on `index.html`
page but not on `sale.html`
##### Parameters
- `targeting` **[Targeting](#targeting)**
Returns **[Experiment](#experiment)** an experiment that matches this targeting or null if none is found.
#### targetingFeatures
[lib/ExperimentContainer.js:159-161](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L159-L161 "Source code on GitHub")
An iterator over all the targeting features from all the experiments
added to this container
Returns **Iterator**
#### iterator
[lib/ExperimentContainer.js:175-177](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L175-L177 "Source code on GitHub")
iterate over all the experiments in this container:
let container = ExperimentContainer.create(...)
for (let experiment of container) {
console.log(experiment.id)
}
Returns **ObjectIterator**
#### has
[lib/ExperimentContainer.js:184-188](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L184-L188 "Source code on GitHub")
check if this container contains the specified experiment
##### Parameters
- `experiment` **Expriment**
Returns **[Boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
#### hasId
[lib/ExperimentContainer.js:195-201](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L195-L201 "Source code on GitHub")
check if this container contains an experiment using an id
##### Parameters
- `experimentId` **[String](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String)**
Returns **[Boolean](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Boolean)**
#### toJSON
[lib/ExperimentContainer.js:208-215](https://[email protected]/:ReasonSoftware/pickpick/blob/a6938cd996f3f68216a96b210bf576e6e3ff6ece/lib/ExperimentContainer.js#L208-L215 "Source code on GitHub")
serialize this container with all it's experiments
Returns **[Object](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object)**