Deprecated
Please don't use this repo, use this one instead: https://github.com/mmerian/phpcrawl
composer require mmerian/phpcrawl
PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
Homepage Repository Packagist PHP
Please don't use this repo, use this one instead: https://github.com/mmerian/phpcrawl
composer require mmerian/phpcrawl