ak_selenium
Selenium package with requests integration and anti-bot detection measures
Documentation · Report Bug · Request Feature
Table of Contents
- 1. About the Project
- 2. Getting Started
- 3. Usage
- 4. Roadmap
- 5. License
- 6. Contact
- 7. Acknowledgements
1. About the Project
ak_selenium is a Python package that provides an interface for automating browser tasks using Selenium. It comes with built-in functionalities for handling common tasks such as form filling, scrolling, and waiting for elements to load. Additionally, it has a built-in requests session that handles retries and timeouts, making it easier to send HTTP requests.
1.1. Features
- Chrome browser automation using Selenium WebDriver.
- Built-in methods for form filling, scrolling, and waiting for elements.
- Anti-bot detection measures
- Pass selenium headers/cookies to requests library
- Built-in requests session with retries and timeouts.
- Ability to use Chrome user data for browser automation.
- RAM optimization for browser options.
- Integrates Helium for easier automation
2. Getting Started
2.1. Installation
2.1.1. Production
Install with flit
pip install flit
flit install --deps productionAlternatively, you can use pip
pip install ak_selenium2.1.2. Development
Install with flit
flit install --pth-file3. Usage
from ak_selenium import Chrome, By, Keys
chrome = Chrome(headless=True) # Create a new Chrome browser instance
driver = chrome.driver #Get Chromedriver
chrome.get("https://example.com") # Navigate to a webpage
#Wait for element to load
locator = (By.TAG_NAME, "h1")
chrome.wait_for_locator(locator)
s = chrome.session # Pass selenium session to requests
s.get("https://www.iana.org/domains/reserved") # Get a website
# Get a list of websites
## Will randomize requests to not trigger bot detection
s.bulk_get(["https://www.iana.org/domains/reserved", "https://www.example.com"])Integrated with Helium to make it easier to set up automation.
Helium methods and functions can be used as intended in the original documentation
Example:
import helium
helium.wait_until(helium.Button('Download').exists)Alternatively, helium methods and classes have been collected into two classes Element and Action for convinience
Element exposes the following classes: Alert, Button, CheckBox, ComboBox, Image, Link, ListItem, RadioButton, Text, TextField and the method find_all
Action exposes the following methods: highlight, wait_until, refresh, attach_file, drag_file, combobox_select, hover, write.
Action also incorporates a Mouse sub-class that collect mouse-related methods.
Example:
from ak_selenium import Element, Action, Keys
import helium
chrome.get('https://google.com') #Go to website
Action.write('helium selenium github') #Enter text into text field
helium.press(Keys.ENTER) #Press Enter
Action.Mouse.click('mherrmann/helium') #Click
chrome.get('https://github.com/login') #Goto github
Action.write('username', into='Username') #Enter Username into Username field
Action.write('password', into='Password') #Enter Password into Password field
Action.Mouse.click('Sign in') #Click Sign-in
Action.Mouse.scroll(direction='down', num_pixels=100) #Scroll down 100px
helium.kill_browser() #Close the browser3.1. Additional Options
# Selenium Overrides
## Overide default useragent
chrome.USERAGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) \
AppleWebKit/537.36 (KHTML, like Gecko) \
Chrome/83.0.4103.53 Safari/537.36'
## Override implicit and max wait times for selenium
chrome.IMPLICITLY_WAIT_TIME = 3 #seconds
chrome.MAX_WAIT_TIME = 5 #seconds
# Requests.Session Override
s.MIN_REQUEST_GAP = 0.9 #seconds between requests4. Roadmap
- Add beautifulsoup integration
- Proxy
5. License
See LICENSE for more information.
6. Contact
Arun Kishore - @rpakishore
Project Link: https://github.com/rpakishore/ak_selenium
