library recursive-pagination

xparse/recursive-pagination

Recursive Pagination

Abandoned !!!

See xparse/Parser, _(*1)

Recursive Pagination

Recursive Pagination allows you to parse website pages recursively. You need to pass link from where to start and set next page expression (xPath, css, etc)., _(*2)

Installation

You can install the package via Composer, _(*3)

``` bash $ composer require xparse/recursive-pagination, <sub>(*4)</sub>

## Basic Usage

Try to find all links to the repositories on github. Our query will be `xparse`.
With recursive pagination we can traverse all pagination links and process each resulting page to fetch repositories links.  

```php
  use Xparse\Parser\Parser;
  use Xparse\RecursivePagination\RecursivePagination;

  # init Parser
  $parser = new Parser();

  # set expression to pagination links
  $paginator = new RecursivePagination($parser, "//*[@class='pagination']//a/@href");
  # set initial page url
  $paginator->addToQueue('https://github.com/search?q=xparse');

  $allLinks = [];
  while (($page = $paginator->getNextPage())) {
    # set expression to fetch repository links
    $adsList = $page->value("//*[@class='repo-list-name']//a/@href")->getItems();
    # merge and remove duplicates
    $allLinks = array_values(array_unique(array_merge($allLinks, $adsList)));
  }
  print_r($allLinks);

Testing

bash $ vendor/bin/phpunit, <sub>(*5)</sub>

Credits

• Ganchev Anatolii
• All Contributors

License

The MIT License (MIT). Please see License File for more information., <sub>(*6)</sub>