2017 © Pedro Peláez
 

library html-scraper

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

image

wykleph/html-scraper

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

  • Thursday, April 21, 2016
  • by Wykleph
  • Repository
  • 1 Watchers
  • 0 Stars
  • 6 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 0 Forks
  • 0 Open issues
  • 4 Versions
  • 0 % Grown

The README.md

HTMLScraper

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior in PHP., (*1)

This is great for creating scraping templates in no time at all.., (*2)

I have no affiliation with webscraper.io, so please refer to their documentation and their forums for anything you might need in regards to webscraper.io., (*3)

Installation : composer require wykleph/html-scraper, (*4)

Note: Child selectors are not supported yet, but it's on the docket!, (*5)

To use, require this project with composer, then download the webscraper.io extension for chrome. This is what we will use to generate our sitemap for crawling the html., (*6)

Once you have the webscraper.io extension, you will probably want to learn how to use the webscraper.io extension., (*7)

Once you have some selectors set up for your sitemap, click on Sitemap (sitemap-name)->Export Sitemap. The json output is what we will use to instantiate a SiteMap object:, (*8)

$SiteMap = new SiteMap($json);

The next step is to instantiate a HtmlScraper object to consume the SiteMap and the HTML you would like to crawl:, (*9)

$scraper = new HtmlScraper($SiteMap, $html);
$selections = $scraper->getSelections();

or:, (*10)

$selections = new HtmlScraper($SiteMap, $html)->getSelections();

The $selections array now contains all of the selections for the sitemap that you used for the given html., (*11)

The $selections array should also contain the name of the selector that you set up with webscraper.io as the key, so accessing your selections is as easy as grabbing something like $selections['username-field-name'] or $selections['phone']., (*12)

The Versions

21/04 2016

dev-master

9999999-dev

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

  Sources   Download

MIT

The Requires

 

by Parker Young

parser css html dom web crawler sitemap scraper selectors webscraper

05/02 2016

dev-composer-updates

dev-composer-updates

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

  Sources   Download

MIT

The Requires

 

by Parker Young

parser css html dom web crawler sitemap scraper selectors webscraper

04/02 2016

v0.1.0

0.1.0.0

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

  Sources   Download

MIT

The Requires

 

by Parker Young

04/02 2016

dev-namespace-changes

dev-namespace-changes

An API for taking json sitemaps generated by the webscraper.io extension, and emulating webscraper.io's scraping behavior.

  Sources   Download

MIT

The Requires

 

by Parker Young