Digs
Making easier the text crawling tasks over websites with depth levels.
Installation
pip install digs
or
pip install --upgrade digs
Usage
Common use will be extract the text from a website, the following call from terminal is the way to do that:
digs http://thewebsite.com
Also, you can add the option --depth=LEVEL to perform over the root domain (website) a crawling with the specific depth:
digs http://thewebsite.com --depth=3
Be careful, with high levels, the tree asociated to those crawlings grows exponentially in size.
And last but not the least, you can turn on a graphical interface (if you have installed PySide) with the following call from terminals:
digs -i
It will be look something like this:
About requirements
Look at requirements in the file: :
requirements.txt
digs was written by Jonathan S. Prieto C..