2017 © Pedro PelĆ”ez
 

library extraload

Powerful ETL library.

image

umpirsky/extraload

Powerful ETL library.

  • Wednesday, May 9, 2018
  • by umpirsky
  • Repository
  • 7 Watchers
  • 26 Stars
  • 199 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 3 Forks
  • 7 Open issues
  • 9 Versions
  • 69 % Grown

The README.md

symfony upgrade fixer • twig gettext extractor • wisdom • centipede • permissions handler • extraload • gravatar • locurro • country list • transliterator , (*1)

Extraload Build Status Scrutinizer Code Quality

Powerful ETL library., (*2)

Examples

Dumping CSV data into the console

Input data is given in csv format:, (*3)

"99921-58-10-7", "Divine Comedy", "Dante Alighieri"
"9971-5-0210-0", "A Tale of Two Cities", "Charles Dickens"
"960-425-059-0", "The Lord of the Rings", "J. R. R. Tolkien"
"80-902734-1-6", "And Then There Were None", "Agatha Christie"

With:, (*4)

(new DefaultPipeline(
    new CsvExtractor(
        new \SplFileObject('books.csv')
    ),
    new NoopTransformer(),
    new ConsoleLoader(
        new Table(new ConsoleOutput())
    )
))->process();

It can be dumped as table to console:, (*5)

+---------------+--------------------------+------------------+
| 99921-58-10-7 | Divine Comedy            | Dante Alighieri  |
| 9971-5-0210-0 | A Tale of Two Cities     | Charles Dickens  |
| 960-425-059-0 | The Lord of the Rings    | J. R. R. Tolkien |
| 80-902734-1-6 | And Then There Were None | Agatha Christie  |
+---------------+--------------------------+------------------+

In this example NoopTransformer is used, but various transformations can be applied. Transformers can also be chained using TransformerChain., (*6)

Dumping a Doctrine query into the console

First of all make sure to load the fixtures into a database -- this example works with MySQL:, (*7)

mysql> source /home/standard/projects/Extraload/fixtures/mysql/books.sql

So the following code:, (*8)

(new DefaultPipeline(
    new QueryExtractor($conn, 'SELECT * FROM books'),
    new NoopTransformer(),
    new ConsoleLoader(
        new Table($output = new ConsoleOutput())
    )
))->process();

Will dump these results to the console:, (*9)

+---------------+--------------------------+------------------+
| 99921-58-10-7 | Divine Comedy            | Dante Alighieri  |
| 9781847493583 | La Vita Nuova            | Dante Alighieri  |
| 9971-5-0210-0 | A Tale of Two Cities     | Charles Dickens  |
| 960-425-059-0 | The Lord of the Rings    | J. R. R. Tolkien |
| 80-902734-1-6 | And Then There Were None | Agatha Christie  |
+---------------+--------------------------+------------------+

Dumping a Doctrine prepared query into the console

The following code:, (*10)

// ...

$sql = "SELECT * FROM books WHERE author = :author";
$values = [
    [
        'parameter' => ':author',
        'value' => 'Dante Alighieri',
        'data_type' => PDO::PARAM_STR // optional
    ]
];

(new DefaultPipeline(
    new QueryExtractor($conn, $sql, $values),
    new NoopTransformer(),
    new ConsoleLoader(
        new Table($output = new ConsoleOutput())
    )
))->process();

Will dump these results to the console:, (*11)

+---------------+---------------+-----------------+
| 99921-58-10-7 | Divine Comedy | Dante Alighieri |
| 9781847493583 | La Vita Nuova | Dante Alighieri |
+---------------+---------------+-----------------+

Dumping a Doctrine query into a table

The following code:, (*12)

// ...

(new DefaultPipeline(
    new QueryExtractor($conn, 'SELECT * FROM books'),
    new NoopTransformer(),
    new DbalLoader($conn, 'my_books')
))->process();

Will dump the results into the my_books table:, (*13)

mysql> select * from my_books;
+----------------+--------------------------+----------------------------+
| isbn           | title                    | author                     |
+----------------+--------------------------+----------------------------+
| 9781503262140  | Faust                    | Johann Wolfgang von Goethe |
| 978-0156949606 | The Waves                | Virgina Woolf              |
| 99921-58-10-7  | Divine Comedy            | Dante Alighieri            |
| 9781847493583  | La Vita Nuova            | Dante Alighieri            |
| 9971-5-0210-0  | A Tale of Two Cities     | Charles Dickens            |
| 960-425-059-0  | The Lord of the Rings    | J. R. R. Tolkien           |
| 80-902734-1-6  | And Then There Were None | Agatha Christie            |
+----------------+--------------------------+----------------------------+
7 rows in set (0.00 sec)

See more examples., (*14)

2. Inspiration

Inspired by php-etl and petl., (*15)

The Versions

09/05 2018

dev-SS/feature/mapping-fields-transformer

dev-SS/feature/mapping-fields-transformer http://umpirsky.com

Powerful ETL library.

  Sources   Download

MIT

The Requires

 

The Development Requires

export transform extract load convert import etl

12/03 2016

dev-feature/readme-header

dev-feature/readme-header http://umpirsky.com

Powerful ETL library.

  Sources   Download

MIT

The Requires

 

The Development Requires

export transform extract load convert import etl

25/01 2016

dev-analysis-zGDPZE

dev-analysis-zGDPZE http://umpirsky.com

Powerful ETL library.

  Sources   Download

MIT

The Requires

 

The Development Requires

export transform extract load convert import etl

23/01 2016

dev-feature/init

dev-feature/init http://umpirsky.com

Powerful ETL library.

  Sources   Download

MIT

The Requires

 

The Development Requires

export transform extract load convert import etl