2017 © Pedro Peláez
 

library easy-php-crawler

Simple yet flexible URL crawler.

image

lexxtor/easy-php-crawler

Simple yet flexible URL crawler.

  • Monday, February 20, 2017
  • by Lexxtor
  • Repository
  • 1 Watchers
  • 1 Stars
  • 5 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 0 Forks
  • 0 Open issues
  • 1 Versions
  • 0 % Grown

The README.md

easyPhpCrawler

It's a simple yet flexible crawler for parsing URLs and loading content., (*1)

Usage Example

``` php <?php, (*2)

use Lexxtor\EasyPhpCrawler\EasyPhpCrawler;, (*3)

require 'EasyPhpCrawler.php';, (*4)

EasyPhpCrawler::go('http://news.yandex.ru', [ 'beforeLoadUrl' => function($url, $crawler) { echo $crawler->currentUrlIndex . '/' . $crawler->getQueueSize() . " $url "; }, 'afterLoadUrlSuccess' => function($url, $content, $crawler) { echo 'loaded: ' . strlen($content) . "\n"; }, 'afterLoadUrlFail' => static function($url, $errorMessage, $crawler) { echo 'Error: ' . $errorMessage . "\n"; }, 'allowUrlRules' => [ '/\/\/news.yandex.ru\//', ], 'denyUrlRules' => [ '/search/', '/\/$/', '/maps/', '/themes/', '/\?redircnt=/', ], ]); ```, (*5)

The Versions

20/02 2017

dev-master

9999999-dev https://github.com/lexxtor/easy-php-crawler

Simple yet flexible URL crawler.

  Sources   Download

The Requires

  • php >=5.4

 

link crawler website