https://github.com/christoferrian/gsmarena-imagescrapper

Python script to scrap ONLY image link from gsmarena and then export it to .csv . if you want to download the image, there's also the downloader script
https://github.com/christoferrian/gsmarena-imagescrapper

csv gsmarena image python scraper scraping

Last synced: 2 days ago
JSON representation

Python script to scrap ONLY image link from gsmarena and then export it to .csv . if you want to download the image, there's also the downloader script

• Host: GitHub
• URL: https://github.com/christoferrian/gsmarena-imagescrapper
• Owner: ChristoferRian
• License: mit
• Created: 2024-09-18T03:45:13.000Z (16 days ago)
• Default Branch: main
• Last Pushed: 2024-09-18T04:08:37.000Z (16 days ago)
• Last Synced: 2024-10-01T19:28:31.759Z (2 days ago)
• Topics: csv, gsmarena, image, python, scraper, scraping
• Language: Python
• Homepage:
• Size: 41 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# gsmarena_scraper

SHOUT OUT TO : dbeley https://github.com/dbeley

This project is based from this page https://github.com/dbeley/gsmarena-scraper, with modifications to focus on downloading smartphone images.

The script extracts image links of phones from the gsmarena.com website to a CSV file, one for each brand. You can use a second script to download images from the generated CSV files.

To avoid spam detection, run with TOR (see below).

## Requirements

- python + pip

- docker + docker-compose

## Installation

Clone the repository:

```

git clone https://github.com/ChristoferRian/GSMArena-ImageScrapper

cd gsmarena-scraper

```

Install the python dependencies:

```

pip install requests beautifulsoup4 lxml pandas pysocks stem

```

If you prefer, you can also install the requirements in a virtual environment with pipenv (in order to run the python script, you will need to use `pipenv run python gsmarena-scraper.py` instead of `python gsmarena-scraper.py`):

```

pipenv install

```

## Usage

Run the docker container containing the tor proxy (you can tweak the torrc configuration file if you want, but the defaults should be good):

```

docker-compose up -d

```

Run the scrapper script:

```

python Scrapper.py

```

Download the images using:

```

python Downloader.py

```

After completion, you can stop the docker container with `docker-compose down`.

## Example Usage

Scrapper:

```

python Scrapper.py

input brand URL : https://www.gsmarena.com/google-phones-107.php

```

Downloader:

```

python Downloader.py

masukkan nama file csv nya : google

```

## Files exported

The `Scrapper.py` script will generate a CSV file with image links for each phone in the folder `Exports/{Brand_Name}.csv.`

To download the images, use the `Downloader.py` script with the CSV file of the desired brand.