spatie/http-status-check

CLI tool to crawl a website and check HTTP status code

Check the HTTP status code of all links on a website

, _(*1)

This repository provides a tool to check the HTTP status code of every link on a given website., _(*2)

Support us

, _(*3)

We invest a lot of resources into creating best in class open source packages. You can support us by buying one of our paid products., _(*4)

We highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on our contact page. We publish all received postcards on our virtual postcard wall., _(*5)

Installation

This package can be installed via Composer:, _(*6)

composer global require spatie/http-status-check

Usage

This tool will scan all links on a given website:, _(*7)

http-status-check scan https://example.com

It outputs a line per link found. Here's an example on Laracast website scan:, _(*8)

, _(*9)

When the crawling process is finished a summary will be shown., _(*10)

By default the crawler uses 10 concurrent connections to speed up the crawling process. You can change that number by passing a different value to the --concurrency option:, _(*11)

http-status-check scan https://example.com --concurrency=20

You can also write all urls that gave a non-2xx or non-3xx response to a file:, _(*12)

http-status-check scan https://example.com --output=log.txt

When the crawler finds a link to an external website it will by default crawl that link as well. If you don't want the crawler to crawl such external urls use the --dont-crawl-external-links option:, _(*13)

http-status-check scan https://example.com --dont-crawl-external-links

By default, requests timeout after 10 seconds. You can change this by passing the number of seconds to the --timeout option:, _(*14)

http-status-check scan https://example.com --timeout=30

By default, the crawler will respect robots data. You can ignore them though with the --ignore-robots option:, _(*15)

http-status-check scan https://example.com --ignore-robots

If your site requires a basic authentification, you can use the --auth option:, _(*16)

http-status-check scan https://example.com --auth=username:password

Testing

To run the tests, first make sure you have Node.js installed. Then start the included node based server in a separate terminal window:, _(*17)

cd tests/server
npm install
node server.js

With the server running, you can start testing:, _(*18)

vendor/bin/phpunit

Changelog

Please see CHANGELOG for more information on what has changed recently., _(*19)

Contributing

Please see CONTRIBUTING for details., _(*20)

Security

If you've found a bug regarding security please mail security@spatie.be instead of using the issue tracker., _(*21)

Credits

• Freek Van der Herten
• Sebastian De Deyne
• All Contributors

License

The MIT License (MIT). Please see License File for more information., _(*22)