Winston is a AB/split/multivariate testing library utilizing redis and a basic machine learning algorithm for automatically displaying the most successful test variations.
pop/winston
Winston is a AB/split/multivariate testing library utilizing redis and a basic machine learning algorithm for automatically displaying the most successful test variations.
Winston is a AB/split testing library which utilizes Redis and basic machine learning. At it's core, Winston is a configurable roll-your-own A/B testing tool. Winston comes with several flavors of AB testing out of the box based on user-defined configuration options., (*1)
Features
Supports machine learning so you can set and forget variations and let Winston determine the best one.
Supports split testing (multiple tests per page at once)
Supports client side javascript events as well as code-triggered events
Minimal bloat to your codebase
Highly configurable
Composer based
Winston and Machine Learning
You can optionally tell Winston whether you'd like to enable machine learning algorithm or not. If it's enabled, Winston performs the following:, (*2)
It first checks if a test is reliably a favorite via confidence intervals.
If a test has no clear favorite, Winston decides on one of the following execution paths:
It falls back to picking a random test variation a certain percentage of the time (defaults to 10%)
It picks the current top performing variant the remaining percentage of the time (defaults to 90%).
If no test variations have any data collected, Winston will always pick at random.
The goal of Winston is to take the guess work out of displaying a top performing test variation. It's a set and forget operation on your part and Winston takes care of the rest. The default random percentages are entirely configurable., (*3)
Usage
Winston installation and setup is comprised of three core parts. Below is an overview of the three parts followed by details on configuring each of the parts., (*4)
Configuration:
You need to setup your Winston configuration file settings to your liking and create tests and variations. You can store your configuration settings however you want (array, JSON, Yaml, database, key/val store) so long as you can convert the settings to a properly formatted array when initializing Winston.
Client Side Code:
You need to add code to your client facing frontend website to display variations and track performance.
Server Side Code:
You need to create a new server side file (a controller route and action if you use MVC) which you grant Winston access to POST data to. There are two specific API endpoints you'll need for Winston which ultimately load an instance of Winston and call the following:
$winston->recordEvent($_POST);
$winston->recordPageview($_POST);
Configuration
Winston requires a fairly lengthy configuration array of settings, tests, and test variations. For a full picture of what a configuration array looks like, check out the basic example config file:, (*5)
This example uses short tags, but you don't have to if they aren't enabled. In this example, we're checking to see if varying the page's headline/tagline has any affect on the frequency of clicking a button directly below it., (*7)
This implementation is only an example with minimal routing support to give you an idea for how to tie in the endpoints from the config file., (*8)
recordEvent($data);
} else if ($uri == 'winston/pageview') {
$winston->recordPageview($data);
}
```
## Advanced Usage: Adding Events Within Variations via Templating
With Winston, you can add event bindings directly within your variation text/html. In each variation, you can use the syntax `{{EVENT_NAME}}` where `EVENT_NAME` is one of the supported client events found in the section below. Winston will internally find and replace these matching template strings with DOM event handlers. If the JavaScript event is triggered, the currently selected variation will trigger successfully and an AJAX request will fire to your backend indicating the success.
Here's an example of a test you can setup in your configuration file which utilizes the basic template engine:
```php
array(
'signup-submit-button-test' => array(
'description' => 'A sample test',
'variations' => array(
array(
'id' => 'submit-default',
'text' => ''
),
array(
'id' => 'submit-signup-now',
'text' => ''
),
)
)
)
);
```
## Supported Client Side Events
Winston supports triggering variation successes for all of the popular DOM events, however we suggest steering clear of mouse movement events given how frequently they trigger. The full list of supported events is `click`, `submit`, `focus`, `blur`, `change`, `mouseover`, `mouseout`, `mousedown`, `mouseup`, `keypress`, `keydown`, and `keyup`. Note that we do not handle preventing default event actions or stopping propagation. If you need that, add your own additional event bindings to the element(s).
To trigger an event in your client side code, simply call: `$winston->event('name-of-your-test', EVENT_TYPE)` where `EVENT_TYPE` is one of the events mentioned above. This method will then generate and return a DOM event string for you to output directly in your HTML, i.e.
```php
// returns: onclick="POP.Winston.event('name-of-your-test', 'SELECTED_VARIATION_ID', 'click');"
// where SELECTED_VARIATION_ID is the variation id found to be optimal/randomized by Winston
$winston->event('name-of-your-test', 'click');
```
Let's now bind a form submission event directly to a form as an example which will get attributed to the chosen variation. The order in which you call `event()` and `variation()` doesn't matter.
```php
Winston is a AB/split/multivariate testing library utilizing redis and a basic machine learning algorithm for automatically displaying the most successful test variations.