ScrapingClass

Installation

Last published version pip install scraping_class

Development version pip install git+https://github.com/elben10/ScrapingClass

Use

First we import the module and initiates the Connector class, and choose a file to write the log to.

import scraping_class
conn = scraping_class.Connector(logfile="log.csv", overwrite_log=False)

The code above creates a csv file named log.csv which looks like this

Then we make a requests for Github API.

url = "https://api.github.com/"
response, id_ = conn.get(url, "")

The reponse is a requests response, and the id_ is a request counter. Now the log file looks like this