GunScraper
A simple scraper for finding guns, according to search criteria, from Swedish gun shops.
Currently three shops are supported:
Setup
In order to install and setup GunScraper, follow the steps below:
- Create a virtual environment
- Install GunScraper and dependencies:
pip install gun_scraper - Download the configuration template
misc/config.yaml - Update the configuration
- Download
misc/runner.shand edit it with the path to the virtual environment and config file - Create a Cron Job to run
runner.shat desired interval
Example Cron Job, running every 12th hour:
0 */12 * * * <path-to-repo>/GunScraper/runner.sh >/tmp/stdout.log 2>/tmp/stderr.log
Config
The config.yaml follows the following syntax:
scraper:
filters:
# Dictionary defining which filters to apply
caliber: # Possible values: 22lr, 22WMR or 308win
handedness: # Possible values: left
sites:
- # List defining which sites to scrape. Supported values: 'torsbo', 'jg' and 'jaktmarken'
email:
sender: # email address that will appear as sender of the notification emails
receiver: # email that will receive notification emails
smtp_server: # hostname of smtp server used to send notifications
ssl_port: # SSL port of the 'smtp_server'
username: # username for the 'smtp_server'
password: # password for the 'smtp_server'
alive_msg_interval: # interval (in hours) to send notification in case no guns matching search criteria is found
data_folder: # folder to store persistent data in
logs_folder: # folder to store log output in
