Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rpakishore/ak_selenium
Selenium package with requests integration and anti-bot detection measures
https://github.com/rpakishore/ak_selenium
Last synced: 3 months ago
JSON representation
Selenium package with requests integration and anti-bot detection measures
- Host: GitHub
- URL: https://github.com/rpakishore/ak_selenium
- Owner: rpakishore
- License: mit
- Created: 2023-04-21T15:47:42.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-09-12T19:13:43.000Z (4 months ago)
- Last Synced: 2024-10-11T09:34:57.312Z (3 months ago)
- Language: Python
- Size: 171 KB
- Stars: 0
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
ak_selenium
Selenium package with requests integration and anti-bot detection measures
Documentation
·
Report Bug
·
Request Feature
[](https://github.com/rpakishore/ak_selenium/actions/workflows/test.yml)
Table of Contents
- [1. About the Project](#1-about-the-project)
- [1.1. Features](#11-features)
- [2. Getting Started](#2-getting-started)
- [2.1. Installation](#21-installation)
- [2.1.1. Production](#211-production)
- [2.1.2. Development](#212-development)
- [3. Usage](#3-usage)
- [3.1. Additional Options](#31-additional-options)
- [4. Roadmap](#4-roadmap)
- [5. License](#5-license)
- [6. Contact](#6-contact)
- [7. Acknowledgements](#7-acknowledgements)
## 1. About the Project
`ak_selenium` is a Python package that provides an interface for automating browser tasks using Selenium. It comes with built-in functionalities for handling common tasks such as form filling, scrolling, and waiting for elements to load. Additionally, it has a built-in requests session that handles retries and timeouts, making it easier to send HTTP requests.
### 1.1. Features
- Chrome browser automation using Selenium WebDriver.
- Built-in methods for form filling, scrolling, and waiting for elements.
- Anti-bot detection measures
- Pass selenium headers/cookies to requests library
- Built-in requests session with retries and timeouts.
- Ability to use Chrome user data for browser automation.
- RAM optimization for browser options.
- Integrates [Helium](https://github.com/mherrmann/helium) for easier automation
## 2. Getting Started
### 2.1. Installation
#### 2.1.1. Production
Install with flit
```bash
pip install flit
flit install --deps production
```
Alternatively, you can use pip
```bash
pip install ak_selenium
```
#### 2.1.2. Development
Install with flit
```bash
flit install --pth-file
```
## 3. Usage
```python
from ak_selenium import Chrome, By, Keys
chrome = Chrome(headless=True) # Create a new Chrome browser instance
driver = chrome.driver #Get Chromedriver
chrome.get("https://example.com") # Navigate to a webpage
#Wait for element to load
locator = (By.TAG_NAME, "h1")
chrome.wait_for_locator(locator)
s = chrome.session # Pass selenium session to requests
s.get("https://www.iana.org/domains/reserved") # Get a website
# Get a list of websites
## Will randomize requests to not trigger bot detection
s.bulk_get(["https://www.iana.org/domains/reserved", "https://www.example.com"])
```
Integrated with [Helium](https://github.com/mherrmann/helium) to make it easier to set up automation.
Helium methods and functions can be used as intended in the [original documentation](https://github.com/mherrmann/helium/blob/master/README.md)
Example:
```python
import helium
helium.wait_until(helium.Button('Download').exists)
```
Alternatively, helium methods and classes have been collected into two classes `Element` and `Action` for convinience
`Element` exposes the following classes: `Alert`, `Button`, `CheckBox`, `ComboBox`, `Image`, `Link`, `ListItem`, `RadioButton`, `Text`, `TextField` and the method `find_all`
`Action` exposes the following methods: `highlight`, `wait_until`, `refresh`, `attach_file`, `drag_file`, `combobox_select`, `hover`, `write`.
`Action` also incorporates a `Mouse` sub-class that collect mouse-related methods.
Example:
```python
from ak_selenium import Element, Action, Keys
import helium
chrome.get('https://google.com') #Go to website
Action.write('helium selenium github') #Enter text into text field
helium.press(Keys.ENTER) #Press Enter
Action.Mouse.click('mherrmann/helium') #Click
chrome.get('https://github.com/login') #Goto github
Action.write('username', into='Username') #Enter Username into Username field
Action.write('password', into='Password') #Enter Password into Password field
Action.Mouse.click('Sign in') #Click Sign-in
Action.Mouse.scroll(direction='down', num_pixels=100) #Scroll down 100px
helium.kill_browser() #Close the browser
```
### 3.1. Additional Options
```python
# Selenium Overrides
## Overide default useragent
chrome.USERAGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) \
AppleWebKit/537.36 (KHTML, like Gecko) \
Chrome/83.0.4103.53 Safari/537.36'
## Override implicit and max wait times for selenium
chrome.IMPLICITLY_WAIT_TIME = 3 #seconds
chrome.MAX_WAIT_TIME = 5 #seconds
# Requests.Session Override
s.MIN_REQUEST_GAP = 0.9 #seconds between requests
```
## 4. Roadmap
- [ ] Add beautifulsoup integration
- [ ] Proxy
## 5. License
See LICENSE for more information.
## 6. Contact
Arun Kishore - [@rpakishore](mailto:[email protected])
Project Link: [https://github.com/rpakishore/ak_selenium](https://github.com/rpakishore/ak_selenium)
## 7. Acknowledgements
- [Awesome README Template](https://github.com/Louis3797/awesome-readme-template/blob/main/README-WITHOUT-EMOJI.md)
- [Shields.io](https://shields.io/)
- [Helium](https://github.com/mherrmann/helium)