2017 © Pedro Peláez
 

library doctrine-batch-utils

A set of utilities to operate with Doctrine ORM's batch processing functionality

image

ocramius/doctrine-batch-utils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  • Thursday, April 19, 2018
  • by Ocramius
  • Repository
  • 6 Watchers
  • 128 Stars
  • 15,887 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 1 Forks
  • 1 Open issues
  • 5 Versions
  • 99 % Grown

The README.md

DoctrineBatchUtils

This repository attempts to ease the pain of dealing with batch-processing in the context of Doctrine ORM transactions., (*1)

This repository is maintained by Patrick Reimers (PReimers)., (*2)

License Current release Build Status, (*3)

Installation

Supported installation method is via Composer:, (*4)

composer require ocramius/doctrine-batch-utils

Current features

As it stands, the only implemented utility in this repository is an IteratorAggregate that wraps around a DB transaction and calls ObjectManager#flush() and ObjectManager#clear() on the given EntityManager., (*5)

Example (array iteration)

It can be used as following:, (*6)

use DoctrineBatchUtils\BatchProcessing\SimpleBatchIteratorAggregate;

$object1  = $entityManager->find('Foo', 1);
$object2  = $entityManager->find('Bar', 2);

$iterable = SimpleBatchIteratorAggregate::fromArrayResult(
    [$object1, $object2], // items to iterate
    $entityManager,       // the entity manager to operate on
    100                   // items to traverse before flushing/clearing
);

foreach ($iterable as $record) {
    // operate on records here
}

$record freshness

Please note that the $record inside the loop will always be "fresh" (managed state), as the iterator re-fetches it on its own: this prevents you from having to manually call ObjectManager#find() on your own for every iteration., (*7)

Automatic flushing/clearing

In this example, the EntityManager will be flushed and cleared only once, but if there were more than 100 records, then it would flush (and clear) twice or more., (*8)

Example (query/iterators)

The previous example is still not memory efficient, as we are operating on a pre-loaded array of objects loaded by the ORM., (*9)

We can use queries instead:, (*10)

use DoctrineBatchUtils\BatchProcessing\SimpleBatchIteratorAggregate;

$iterable = SimpleBatchIteratorAggregate::fromQuery(
    $entityManager->createQuery('SELECT f FROM Files f'),
    100 // flush/clear after 100 iterations
);

foreach ($iterable as $record) {
    // operate on records here
}

Or our own iterator/generator:, (*11)

use DoctrineBatchUtils\BatchProcessing\SimpleBatchIteratorAggregate;

// This is where you'd persist/create/load your entities (a lot of them!)
$results = function () {
    for ($i = 0; $i < 100000000; $i += 1) {
        yield new MyEntity($i); // note: identifier must exist in the DB
    }
};

$iterable = SimpleBatchIteratorAggregate::fromTraversableResult(
    $results(),
    $entityManager,
    100 // flush/clear after 100 iterations
);

foreach ($iterable as $record) {
    // operate on records here
}

// eventually after all records have been processed, the assembled transaction will be committed to the database

Both of these approaches are much more memory efficient., (*12)

The Versions

19/04 2018

dev-master

9999999-dev https://github.com/Ocramius/DoctrineBatchUtils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  Sources   Download

MIT

The Requires

 

The Development Requires

orm doctrine doctrine2 batch processing

05/02 2018

1.1.x-dev

1.1.9999999.9999999-dev https://github.com/Ocramius/DoctrineBatchUtils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  Sources   Download

MIT

The Requires

 

The Development Requires

orm doctrine doctrine2 batch processing

05/02 2018

1.1.0

1.1.0.0 https://github.com/Ocramius/DoctrineBatchUtils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  Sources   Download

MIT

The Requires

 

The Development Requires

orm doctrine doctrine2 batch processing

09/10 2015

1.0.0

1.0.0.0 https://github.com/Ocramius/DoctrineBatchUtils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  Sources   Download

MIT

The Requires

 

The Development Requires

orm doctrine doctrine2 batch processing

09/10 2015

1.0.x-dev

1.0.9999999.9999999-dev https://github.com/Ocramius/DoctrineBatchUtils

A set of utilities to operate with Doctrine ORM's batch processing functionality

  Sources   Download

MIT

The Requires

 

The Development Requires

orm doctrine doctrine2 batch processing