https://github.com/abenezeradane/mercury
A fast and efficient eBay webscraper implemented using Beautiful Soup, developed on commission while freelancing
https://github.com/abenezeradane/mercury
beautifulsoup cli cli-app command-line-tool dataclasses lxml multiprocessing python requests
Last synced: 8 months ago
JSON representation
A fast and efficient eBay webscraper implemented using Beautiful Soup, developed on commission while freelancing
- Host: GitHub
- URL: https://github.com/abenezeradane/mercury
- Owner: abenezeradane
- License: mit
- Created: 2022-08-31T00:40:59.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-11-10T03:33:05.000Z (almost 3 years ago)
- Last Synced: 2025-02-24T02:44:32.092Z (8 months ago)
- Topics: beautifulsoup, cli, cli-app, command-line-tool, dataclasses, lxml, multiprocessing, python, requests
- Language: Python
- Homepage:
- Size: 26.4 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Mercury
A fast and efficient eBay webscraper implemented using Beautiful Soup
### Installation
```bash
# Clone the repo
$ git clone https://github.com/PB020/Mercury.git
# Change the working directory
$ cd mercury
# Create virtual enviroment
$ python -m venv .env
$ .env/Scripts/activate # Windows
$ .env/bin/activate # Linux/MacOS
# Install requirements
$ pip install -U -r requirements.txt
```
### Usage
```bash
usage: mercury.py [-h] [--url URL] [--file FILE] [--output OUTPUT]
A fast and efficient eBay product webscraper
optional arguments:
-h, --help show this help message and exit
--url URL URL of an eBay product ('https://www.ebay.com/itm/...')
--file FILE Path to a file containing ONLY eBay product URLs
--output OUTPUT CSV filename where parsed data should be saved
```
**Note:** If no URL or file is provided, the script *will* terminate.
### Example
```bash
# Single URL
$ mercury.py --url "https://www.ebay.com/itm/..."
# File containing URLs
$ mercury.py --file "input.txt"
# Output to CSV
$ mercury.py --url "https://www.ebay.com/itm/..." --output "output.csv"
```
### Output
```csv
ITEM_NUMBER,NAME,CONDITION,PRICE,URL
...,...,...,...,...
```
### Dependencies
Web Scraping:
* [lxml](https://pypi.org/project/lxml/)
* [requests](https://pypi.org/project/requests/)
* [Beautiful Soup](https://www.crummy.com/software/BeautifulSoup/)
CLI:
* [argparse](https://pypi.org/project/argparse/)
* [colorama](https://pypi.org/project/colorama/)
Multiprocessing:
* [multiprocessing](https://pypi.org/project/multiprocessing/)
### License
[](https://opensource.org/licenses/MIT)