2017 © Pedro Peláez
 

library etl

Extract / Transform / Load in PHP - Multiple formats - No dependency.

image

bentools/etl

Extract / Transform / Load in PHP - Multiple formats - No dependency.

  • Monday, February 19, 2018
  • by bpolaszek
  • Repository
  • 4 Watchers
  • 17 Stars
  • 1,415 Installations
  • PHP
  • 1 Dependents
  • 1 Suggesters
  • 2 Forks
  • 0 Open issues
  • 18 Versions
  • 67 % Grown

The README.md

Latest Stable Version License CI Workflow Coverage Total Downloads, (*1)

Okay, so you heard about the Extract / Transform / Load pattern, and you're looking for a PHP library to do the stuff. Alright, let's go!, (*2)

bentools/etl is a versatile PHP library for implementing the Extract, Transform, Load (ETL) pattern, designed to streamline data processing tasks., (*3)

Table of Contents

Concepts

Let's cover the basic concepts: - Extract: you have a source of data (a database, a CSV file, whatever) - an extractor is able to read that data and provide an iterator of items - Transform: apply transformation to each item. A transformer may generate 0, 1 or several items to load (for example, 1 item may generate multiple SQL queries) - Load: load transformed item to the destination. For example, extracted items have been transformed to SQL queries, and your loader will run those queries against your database., (*4)

Installation

composer require bentools/etl

[!WARNING] Current version (4.0) is a complete redesign and introduces significant BC (backward compatibility) breaks. Avoid upgrading from ^2.0 or ^3.0 unless you're fully aware of the changes., (*5)

Usage

Now let's have a look on how simple it is:, (*6)

use BenTools\ETL\EtlExecutor;

// Given
$singers = ['Bob Marley', 'Amy Winehouse'];

// Transform each singer's name to uppercase and process the array
$etl = (new EtlExecutor())
    ->transformWith(fn (string $name) => strtoupper($name));

// When
$report = $etl->process($singers);

// Then
var_dump($report->output); // ["BOB MARLEY", "AMY WINEHOUSE"]

OK, that wasn't really hard, here we basically don't have to extract anything (we can already iterate on $singers), and we're not loading anywhere, except into PHP's memory., (*7)

You may ask, "why don't you just array_map('strtoupper', $singers) ?" and you're totally right., (*8)

But sometimes, extracting, transforming and / or loading get a little more complex. You may want to extract from a file, a crawled content on the web, perform one to many transformations, maybe skip some items, or reuse some extraction, transformation or loading logic., (*9)

Here's another example of what you can do:, (*10)

use BenTools\ETL\EventDispatcher\Event\TransformEvent;
use BenTools\ETL\Loader\JSONLoader;

use function BenTools\ETL\extractFrom;

$executor = extractFrom(function () {
    yield ['firstName' => 'Barack', 'lastName' => 'Obama'];
    yield ['firstName' => 'Donald', 'lastName' => 'Trump'];
    yield ['firstName' => 'Joe', 'lastName' => 'Biden'];
})
    ->transformWith(fn (array $item) => implode(' ', array_values($item)))
    ->loadInto(new JSONLoader())
    ->onTransform(function (TransformEvent $event) {
        if ('Donald Trump' === $event->transformResult->value) {
            $event->state->skip();
        }
    });

$report = $executor->process();

dump($report->output); // string '["Barack Obama", "Joe Biden"]'

Or:, (*11)

$report = $executor->process(destination: 'file:///tmp/presidents.json');
var_dump($report->output); // string 'file:///tmp/presidents.json' - content has been written here

You get the point. Now you're up to write your own workflows!, (*12)

Continue reading the Getting Started Guide., (*13)

Contribute

Contributions are welcome! Don't hesitate to suggest recipes., (*14)

This library is 100% covered with Pest tests., (*15)

Please ensure to run tests using the command below and maintain code coverage before submitting PRs., (*16)

composer ci:check

License

MIT., (*17)

The Versions

19/02 2018

3.0.x-dev

3.0.9999999.9999999-dev

Extract / Transform / Load in PHP - Multiple formats - No dependency.

  Sources   Download

MIT

The Requires

  • php >=7.1

 

The Development Requires

by Beno!t POLASZEK

20/06 2017

dev-master

9999999-dev

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

08/06 2017

v2.5.1

2.5.1.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

07/06 2017

v2.5

2.5.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

05/05 2017

v2.4

2.4.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

26/04 2017

v2.3.2

2.3.2.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

08/03 2017

v2.3.1

2.3.1.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

08/03 2017

v2.3

2.3.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

08/03 2017

dev-develop

dev-develop

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

06/03 2017

v2.2

2.2.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

03/03 2017

v2.1

2.1.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

01/03 2017

v2.0.2

2.0.2.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

27/02 2017

v2.0.1

2.0.1.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

27/02 2017

v2.0

2.0.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

26/02 2016

v1.1.0

1.1.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

The Development Requires

by Beno!t POLASZEK

24/02 2016

v1.0.2

1.0.2.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

by Beno!t POLASZEK

12/02 2016

v1.0.1

1.0.1.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

by Beno!t POLASZEK

11/02 2016

v1.0.0

1.0.0.0

Some classes to implement the ETL pattern, with the ability to hook on each event.

  Sources   Download

MIT

The Requires

 

by Beno!t POLASZEK