bentools/split-test-analyzer

Bayesian probability calculator for split testing / AB testing.

, _(*1)

Split Test Analyzer

This PHP tool will help you compute, for different versions of a split test, their probability to be the best one., _(*2)

It is a PHP port of the JS tool used on AB split test graphical Bayesian calculator., _(*3)

Concept

A Variation object is composed of 3 informations:, _(*4)

• A key to identify it (the Landing page / banner id for instance)
• The total number of actions (let's say, the unique visitors)
• The number of successful actions (let's say, the conversions)

Now you can compare those 3 objects and compute their probability to be the best., _(*5)

Usage

use BenTools\SplitTestAnalyzer\SplitTestAnalyzer;
use BenTools\SplitTestAnalyzer\Variation;

require_once __DIR__ . '/vendor/autoload.php';

$variations = [
    new Variation('LP #1', 8686347, 932),
    new Variation('LP #2', 7305026, 804),
    new Variation('LP #3', 4592639, 371),
    new Variation('LP #4', 4590186, 402),
    new Variation('LP #5', 4532325, 470),
    new Variation('LP #6', 4531653, 494),
];

$predictor = SplitTestAnalyzer::create()->withVariations(...$variations);
foreach ($predictor->getResult() as $key => $value) {
    printf('%s has %d%% chances to be the best one.' . PHP_EOL, $key, $value);
}

print PHP_EOL;

printf('The best one is: %s', $predictor->getBestVariation());

Output:, _(*6)

LP #1 has 15% chances to be the best one.
LP #2 has 43% chances to be the best one.
LP #3 has 0% chances to be the best one.
LP #4 has 0% chances to be the best one.
LP #5 has 7% chances to be the best one.
LP #6 has 34% chances to be the best one.

The best one is: LP #2

Because of the randomness used in Bayes calculator, results may slightly vary., _(*7)

To improve performance you can lower the amount of samples with SplitTestAnalyzer::create(100), but then you'll have to choose between performance and reliability., _(*8)

Installation

This library requires PHP 7.0+., _(*9)

composer require bentools/split-test-analyzer, _(*10)

Tests

./vendor/bin/phpunit, _(*11)

See also

bentools/cartesian-product, _(*12)

bentools/pager, _(*13)