https://github.com/codingforentrepreneurs/supercharged-web-scraping-with-asyncio
Scrape websites asynchronously with Python 3.8+, Asyncio, & arsenic (aka Selenium for Async).
https://github.com/codingforentrepreneurs/supercharged-web-scraping-with-asyncio
arsenic async asynchronous asyncio python3 python38 selenium tutorial web-scraping
Last synced: 11 months ago
JSON representation
Scrape websites asynchronously with Python 3.8+, Asyncio, & arsenic (aka Selenium for Async).
- Host: GitHub
- URL: https://github.com/codingforentrepreneurs/supercharged-web-scraping-with-asyncio
- Owner: codingforentrepreneurs
- Created: 2020-08-04T04:47:56.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2021-04-17T00:11:07.000Z (about 5 years ago)
- Last Synced: 2025-06-27T04:14:57.888Z (11 months ago)
- Topics: arsenic, async, asynchronous, asyncio, python3, python38, selenium, tutorial, web-scraping
- Language: Jupyter Notebook
- Homepage: https://www.codingforentrepreneurs.com/projects/supercharged-web-scraping-with-asyncio
- Size: 337 KB
- Stars: 56
- Watchers: 4
- Forks: 22
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
[](https://www.codingforentrepreneurs.com/projects/supercharged-web-scraping-with-asyncio)
# Supercharged Web Scraping with Asyncio
Web scraping is simply automatically opening up any website and grabbing the data you find important on that website. It's fundamental to the internet, search engines, Data Science, automation, machine learning, and much more.
*Opening* websites and *extracting* data are only part of what makes web scraping great. It's the parsing of the data that's where the value is.
This project will cover:
- Basic web scraping with Python
- Web scraping with Selenium
- Sync vs Async
- Asynchronous Web scraping with Asyncio
Requirements:
- Python experience (at least the first 15 days of [this project](https://www.codingforentrepreneurs.com/projects/30-days-python-38/)).
- Selenium & [chromedriver](https://chromedriver.chromium.org/) installed (watch how in [this one](https://www.codingforentrepreneurs.com/projects/30-days-python-38/day-16-use-selenium-scape-automate-behind-password)).
#### [Watch the series](https://www.codingforentrepreneurs.com/projects/supercharged-web-scraping-with-asyncio)
### To use this code:
**1. Clone**
```bash
git clone https://github.com/codingforentrepreneurs/Supercharged-Web-Scraping-with-Asyncio supercharged
```
**2. Create Virtual Environment**
```bash
cd supercharged
python3.6 -m venv .
```
**3. Activate virtual environment and install requirements**
Mac/Linux
```
source bin/activate
```
Windows:
```
.\Scripts\activate
```
> If using **pipenv**, run `pipenv shell` && `pipenv install`
**Run jupyter**
```
jupyter notebook
```
or
```
python -m jupyter notebook
```
> If using **pipenv**, run `pipenv run jupyter notebook`