assets-crawler

Simple assets crawler that allows you to get all assets from a particular website via command-line

Install

$ npm install --global assets-crawler

Usage

$ assets-crawler --help

  Usage
    $ assets-crawler [url]

  Options
    --maxConnections  Maximum concurrent connections [Default: 1000]
    --requestInterval  Delay time in milliseconds before spawning a new connection [Default: 200]
    --timeout  Waiting time for a server to response[Default: 10000]
    --debug  If true, all errors from requests will be printed to stderr [Default: false]

  Examples
    $ assets-crawler facebook.github.io/graphql
     [
       {
         "url": "http://facebook.github.io/graphql",
         "assets": [
           "http://facebook.github.io/graphql/spec.css",
           "http://facebook.github.io/graphql/highlight.css"
         ]
       }
     ]

In case of a huge result, you might want to save an output to a file by using assets-crawler {url} > output.json.

License

MIT © Varayut Lerdkanlayanawat