Web Scraping in Python using Bobik
This is a community-supported Bobik SDK for web scraping in Python.
Installing
You can install the SDK using pip:
pip install bobik_python_sdk
If you want to install the latest development version, use:
pip install -e git+https://github.com/emirkin/bobik_python_sdk#egg=bobik_python_sdk
Using
Here's a quick example to get you started
from bobik import Bobik
bobik_api = Bobik(YOUR_AUTH_TOKEN, debug=True)
def success_handler(response):
for site in response['results']:
print 'Results for %s' % site
for query in response['results'][site]:
print '\tQuery: %s' % query
for result in response['results'][site][query]:
print '\t\t%s' % result
def error_handler(error_list):
for err in error_list:
print err
query = {
'urls' : ['amazon.com', 'zynga.com', 'http://finance.yahoo.com'],
'queries' : ['//th', '//img/@src', 'return document.title', "return $('script').length"]
}
bobik_api.scrape(query, success_handler, error_handler)Documentation
Documentation can be found at http://usebobik.com/sdk/python/index.html.
The docs are generated with Sphinx. To generate
the docs, enter the docs folder and run:
$ make html
The documentation will be generated inside the _build/html directory.
Contributing
Write to support@usebobik.com to become a collaborator.
Bugs
Submit issues with the SDK here on Github.
