2017 © Pedro Peláez
 

library jsonreader

JSON streaming reader (memory safety parser) with chunking option

image

webdevcraft/jsonreader

JSON streaming reader (memory safety parser) with chunking option

  • Saturday, October 29, 2016
  • by webdevcraft
  • Repository
  • 1 Watchers
  • 0 Stars
  • 19 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 1 Forks
  • 1 Open issues
  • 9 Versions
  • 19 % Grown

The README.md

JsonReader

Build Status, (*1)

JSON streaming reader (memory safety parser) with chunking option, (*2)

Purpose and Advantages

  1. Memory safety read of huge JSON files: solve "out of RAM" problem. Successfully tested with 1.5Gb JSON files
  2. Possibility to split large JSON file to smaller JSON chunks
  3. Custom traverse through JSON tree
  4. Read any JSON source: file, string, resource, character iterator
  5. Installation with Composer, no need any extra PHP extensions

Requirements

PHP 5.6, 7.0, (*3)

Installation

With Composer:, (*4)

composer require webdevcraft/jsonreader

Basic Usage

JSON Reader iterates through JSON structure elements, (*5)

Each $reader->read() iterates to next element until it returns false when finished, (*6)

So common usage is:, (*7)

while ($reader->read()) {
    // next element iteration
}

Reader provides information on each element:, (*8)

  1. DEPTH in JSON tree starting from 1: $reader->getDepth(), (*9)

  2. STATE (type of element): $reader->getState(). There is syntax sugar issers exist for each of states. State could be one of:, (*10)

    • Object Start: _JsonReaderInterface::STATE_OBJECT_START_ (or $reder->isObjectStartState()), (*11)

    • Object Key: _JsonReaderInterface::STATE_OBJECT_KEY_ (or $reder->isObjectKeyState()), (*12)

    • Object End _JsonReaderInterface::STATE_OBJECT_END_ (or $reder->isObjectEndState()), (*13)

    • Array Start _JsonReaderInterface::STATE_ARRAY_START_ (or $reder->isArrayStartState()), (*14)

    • Array End _JsonReaderInterface::STATE_ARRAY_END_ (or $reder->isArrayEndState()), (*15)

    • Value _JsonReaderInterface::STATE_VALUE_ (or $reder->isValueState()), (*16)

  3. VALUE of element by $reader->getValue(). Types:, (*17)

    1. For $reader->getState() === JsonReaderInterface::STATE_VALUE the type of value is automatically casted to PHP type: string, int, float, boolean, null, (*18)

    2. For JsonReaderInterface::STATE_OBJECT_KEY state is always string, (*19)

    3. For other states value is always null, (*20)

Example of elements sequence for test.json is presented in JsonReaderTest.php::testReadTokens(), (*21)

Chunks Usage

In case if your code is already dependent on JSON string denormanization you could use JsonReader library for chunking of huge file onto smaller JSON strings, (*22)

Just start buffering chunk on element position you wish by $reader->startWriteChunk() and get chunk by $reader->finishWriteChunk(). That returnes JSON chunk string and flushes buffer., (*23)

As chunk is buffered in memory make sure your JSON schema guarantees acceptable size of chunk, (*24)

That's nice practice to wrap chunks retrieve to iterator, (*25)

Example of test.json chunking is presented in JsonReaderTest.php::testChunks(), (*26)

Factory Usage

To create reader you have to use JsonReaderFactory, (*27)

Example of JsonReader creating:, (*28)

$factory = new JsonReaderFactory();
$reader = $factory->createByFilePath('/tmp/test.json');

Factory supports any sources by such methods:, (*29)

  1. createByFilePath by file path in local filesystem
  2. createByString by JSON string passing
  3. createByResource by opened resource link, for example with fopen()
  4. createByCharacterTraversable - if options above do not suite your needs feel free to write your custom characters iterator using any source you have to deal with. For example you could iterate PSR-7 Message\StreamInterface::read($acceptableSize) and iterate characters inside each of read()

Example of characters iterators, (*30)

The Versions

29/10 2016

dev-master

9999999-dev

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

29/10 2016

0.2.0

0.2.0.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

29/10 2016

0.1.6

0.1.6.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

29/10 2016

0.1.5

0.1.5.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

28/10 2016

0.1.4

0.1.4.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

28/10 2016

0.1.3

0.1.3.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

MIT

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

28/10 2016

0.1.2

0.1.2.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

28/10 2016

0.1.1

0.1.1.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

The Development Requires

by Iurii Barabash

json stream reader json parser memory safety huge json large json

28/10 2016

0.1.0

0.1.0.0

JSON streaming reader (memory safety parser) with chunking option

  Sources   Download

The Development Requires

by Iurii Barabash

parser json stream reader huge large memory safety