2017 © Pedro Peláez
 

library serpscraper

PHP powered interface for querying the most popular Search Engines

image

athlon1600/serpscraper

PHP powered interface for querying the most popular Search Engines

  • Sunday, March 4, 2018
  • by Athlon1600
  • Repository
  • 7 Watchers
  • 23 Stars
  • 402 Installations
  • PHP
  • 0 Dependents
  • 0 Suggesters
  • 14 Forks
  • 10 Open issues
  • 2 Versions
  • 7 % Grown

The README.md

, (*1)

SerpScraper

The purpose of this library is to provide an easy, undetectable, and captcha resistant way to extract search results from popular search engines like Google and Bing., (*2)

Installation

The recommended way to install this is via Composer:, (*3)

composer require athlon1600/serpscraper "^4.0"

Extracting Search Results From Google

<?php

use SerpScraper\Engine\GoogleSearch;

$page = 1;

$google = new GoogleSearch();

// all available preferences for Google
$google->setPreference('results_per_page', 100);
//$google->setPreference('google_domain', 'google.lt');
//$google->setPreference('date_range', 'hour');

$results = array();

do {

    $response = $google->search("how to scrape google", $page);

    // error field must be empty otherwise query failed
    if(empty($response->error)){

        $results = array_merge($results, $response->results);
        $page++;

    } else if($response->error == 'captcha'){

        // read below
        break;
    }

} while ($response->has_next_page);

Solve Google Search captchas automatically

For this to work, you will need to register for 2captcha.com services, and get an API key. It is also highly recommended to use a proxy server.
Install a private proxy server on your own VPS here:
https://github.com/Athlon1600/useful#squid, (*4)

<?php

use SerpScraper\Engine\GoogleSearch;
use SerpScraper\GoogleCaptchaSolver;

$google = new GoogleSearch();

$browser = $google->getBrowser();
$browser->setProxy('PROXY:IP');

$solver = new GoogleCaptchaSolver($browser);

while(true){
    $response = $google->search('famous people born in ' . mt_rand(1500, 2020));

    if ($response->error == 'captcha') {

        echo "Captcha detected!" . PHP_EOL;

        $temp = $solver->solveUsingTwoCaptcha($response, '2CAPTCHA_API_KEY', 90);

        if ($temp->status == 200) {
            echo "Captcha solved successfully!" . PHP_EOL;
        } else {
            echo 'Solving captcha has failed...' . PHP_EOL;
        }

    } else {
        echo "OK. ";
    }

    sleep(2);
}

Extract Search Results from Bing

<?php

use SerpScraper\Engine\BingSearch;

$bing = new BingSearch();
$results = array();

for($page = 1; $page < 10; $page++){

    $response = $bing->search("search bing using php", $page);
    if($response->error == false){
        $results = array_merge($results, $response->results);
    }

    if($response->has_next_page == false){
        break;
    }
}

var_dump($results);

The Versions

04/03 2018

dev-master

9999999-dev

PHP powered interface for querying the most popular Search Engines

  Sources   Download

MIT

The Requires

  • ext-curl *

 

google search google scraper bing scraper bing search google captcha solver google proxy

30/11 2016

2.0

2.0.0.0

PHP powered interface for querying the most popular Search Engines

  Sources   Download

MIT

The Requires

 

google search google scraper bing scraper bing search google captcha solver google proxy