Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/oxylabs/google-play-scraper

Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.
https://github.com/oxylabs/google-play-scraper

google google-news-api google-news-scraper google-play google-play-api google-play-scraper google-play-store google-search-scraper google-trends-api movies-api scraper

Last synced: about 1 month ago
JSON representation

Google Play Scraper for collecting public details about applications, games, movies & TV shows, books, and more.

Host: GitHub
URL: https://github.com/oxylabs/google-play-scraper
Owner: oxylabs
Created: 2023-09-01T07:19:24.000Z (over 1 year ago)
Default Branch: main
Last Pushed: 2024-09-16T11:04:27.000Z (3 months ago)
Last Synced: 2024-09-17T10:05:27.151Z (3 months ago)
Topics: google, google-news-api, google-news-scraper, google-play, google-play-api, google-play-scraper, google-play-store, google-search-scraper, google-trends-api, movies-api, scraper
Language: Python
Homepage: https://oxylabs.io/products/scraper-api/serp/google/play
Size: 52.7 KB
Stars: 2
Watchers: 1
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.md

Awesome Lists containing this project

README

        # Google Play Scraper

[![Oxylabs promo code](https://user-images.githubusercontent.com/129506779/250792357-8289e25e-9c36-4dc0-a5e2-2706db797bb5.png)](https://oxylabs.go2cloud.org/aff_c?offer_id=7&aff_id=877&url_id=112)

[![](https://dcbadge.vercel.app/api/server/eWsVUJrnG5)](https://discord.gg/GbxmdGhZjq)

This tutorial will show you how to gather public data from the Google Play store, including data points like **title**, **price**, **version number**, **download rates**, **reviews**, and more. In this repository, you can find a **free Google Play scraper tool**, designed for smaller-scale scraping tasks. If you want to increase your scraping scale, the second part of this guide will show you how to utilize a far more effective Oxylabs' [Scraper API](https://oxylabs.io/products/scraper-api). It comes with a **free trial**, which you can claim by registering a free account on the [dashboard](https://dashboard.oxylabs.io/).

- [Free Google Play Scraper](#free-google-play-scraper)

  * [Prerequisites](#prerequisites)

  * [Installation](#installation)

  * [Scraping Google Play](#scraping-google-play)

    + [1. Select a category:](#1-select-a-category)

    + [2. Use a search keyword:](#2-use-a-search-keyword)

  * [Notes](#notes)

- [Scrape Google Play data with Oxylabs Scraper API](#scrape-google-play-data-with-oxylabs-scraper-api)

  * [Python code example](#python-code-example)

  * [Output example](#output-example)

## Free Google Play Scraper

A free tool which you can use to get data for apps, books, or movies from Google Play using a specific search query.

### Prerequisites

To run this tool, you need to have Python `3.11` or later installed on your system.

### Installation

Open up a terminal window, navigate to this repository, and run this command:

```make install```

### Scraping Google Play

#### 1. Select a category:

To scrape data from Google Play, first choose one of these categories, that are available in Google Play:

- `apps`

- `movies`

- `books`

The default category in the tool is `apps`, so feel free to omit the `CATEGORY` parameter from the command if that's the category you need. 

If you prefer to choose a different category than `apps`, run this command in your terminal:

```bash

make scrape QUERY="" CATEGORY="

```

Otherwise, the command should look like this:

```bash

make scrape QUERY=""

```

> [!NOTE]

> Make sure the category name is in **lowercase**. 

#### 2. Use a search keyword:

For this example, let's try scraping Google Play results for movies about fishing. The command should look like this:

```bash

make scrape QUERY="fishing" CATEGORY="movies"

```

> [!NOTE]

> Make sure to enclose your query and category in quotation marks. Otherwise, the tool might have trouble parsing it.

After running the command, you should see a similar output in your terminal:



After the tool finishes running, you can find a file named `movies_play_items.csv` in your current working directory. This file contains Google Play items for the query and category you entered. The file name will always be in this format: `{category}_play_items.csv`. The generated CSV file contains these columns of data:

- `title` - The title of the movie.

- `price` - The price of the movie to rent.

- `rating` - The rating of the movie.

- `cover_url` - The URL to the image of the cover for the movie.

- `url` - The URL for the movie.

Here's an example of how the scraped and parsed data should look like:



### Notes

In case the code doesn't work or your project is of bigger scale, please refer to the second part of the tutorial. There, we showcase how to scrape public data with Oxylabs Scraper API.

## Scrape Google Play data with Oxylabs Scraper API

After purchasing access to the API or claiming your free trial, you'll have to use your API credentials for authentication.

You can retrieve Google Play results by providing your target URLs and

forming a `payload` with job parameters. [Scraper API](https://oxylabs.io/products/scraper-api) will return the **HTML** of

any public Google Play page you have provided.

### Python code example

The following examples demonstrate how you can get Google Play results

in HTML format. To begin, you need to send the request to the API using

the

[Push-Pull](https://developers.oxylabs.io/scraper-apis/serp-scraper-api/integration-methods/push-pull)

method (or [other methods](https://developers.oxylabs.io/scraper-apis/serp-scraper-api/integration-methods)):

```python

import requests

from pprint import pprint

# Structure payload.

payload = {

   'source': 'google',

   'url': 'https://play.google.com/store/games?hl=en_GB&gl=UK',

   'user_agent_type': 'desktop_edge',

   'render': 'html',

   'geo_location': 'United Kingdom',

   'locale': 'en-gb'

}

# Get response.

response = requests.request(

    'POST',

    'https://data.oxylabs.io/v1/queries',

    auth=('USERNAME', 'PASSWORD'), #Your credentials go here

    json=payload

)

# This will return a JSON response with job information and results URLs.

pprint(response.json())

```

Once the job is finished, you can then send another request to retrieve

the Google Play results. Here, you must use the **job ID** value that’s

provided in the response of the above code sample:

```python

import requests

from pprint import pprint

# Get response.

response = requests.request(

    'GET',

    'http://data.oxylabs.io/v1/queries/{job_id}/results',

    auth=('USERNAME', 'PASSWORD')

)

# This will return a JSON response with scraped results.

pprint(response.json())

```

Visit our

[documentation](https://developers.oxylabs.io/scraper-apis/serp-scraper-api/google/url)

for more information.

### Output example

The response will be in JSON format, containing HTML content and details about the job itself:

```json

{

  "results": [

    {

      "content": "email](mailto:[email protected]) if you need

assistance.

Read More Google Scraping Related Repositories: [Google Sheets for Basic Web Scraping](https://github.com/oxylabs/web-scraping-google-sheets), [How to Scrape Google Shopping Results](https://github.com/oxylabs/scrape-google-shopping), [How To Scrape Google Jobs](https://github.com/oxylabs/how-to-scrape-google-jobs), [Google News Scrpaer](https://github.com/oxylabs/google-news-scraper), [How to Scrape Google Scholar](https://github.com/oxylabs/how-to-scrape-google-scholar), [How to Scrape Google Flights with Python](https://github.com/oxylabs/how-to-scrape-google-flights), [How To Scrape Google Images](https://github.com/oxylabs/how-to-scrape-google-images), [Scrape Google Search Results](https://github.com/oxylabs/scrape-google-python), [Scrape Google Trends](https://github.com/oxylabs/how-to-scrape-google-trends)