Full HTML WebPage Downloader, Parser and Data Extractor. Web Crawler, Server and Client Included.


License
BSD-2-Clause
Install
pip install FairWEB==4.5.0

Documentation

FWEB

FairWeb or FWEB is a complete webpage downloader and web-scrapping tool-kit.

  1. FusedDownloader -> Uses two libraries to attempt to download the URL.
    • v1 = NewsPaper.Article
    • v2 = FWEB.ArchiveDownloader_v1

USAGED: -> FusedDownloader.download_v2(url)

  1. StartCrawler -> Takes a URL and begins to "crawl" the website, downloading and saving every webpage it can.

    • Under the hood, it uses FusedDownloader to actually download and parse the webpage.
  2. Core.Tag -> Master Util Library for searching and finding anything inside the returned HTML.