Cruel
is a Python web-request library that allows you to extract detailed information from any webpage or site.
- Inbuilt beautiful soup instance
- ScraperAPI powered; so you don't have to worry about IP blocking
You can install the cruel
using pip:
pip install cruel
Below are examples of how to use the cruel
to extract data from Fiverr gig pages and user profiles.
from cruel import session
session.set_scraper_api_key("XYZ-SCRAPER_API_KEY")
response = session.get("https://www.fiverr.com/username/your-gig-slug") # your fiverr url should be here
print(response.soup) # gives you beautiful soup instance
# You can use `response.soup` to further extract your information.
Get your ScraperAPI key here.
The cruel
is organized into several modules to enhance code readability and maintainability:
-
cruel
-
__init__.py
: For exportingsession
-
-
cruel.utils
-
req.py
: Extending requests for Fiverr scraping -
scrape_utils.py
: Utilities for scraping
-
New pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Check more of my projects.