https://github.com/DovarFalcone/google-takeout-location-parser

Easily parse location .json files provided by the Google Takeout service
https://github.com/DovarFalcone/google-takeout-location-parser

arcgis geospatial google google-history google-location-history google-maps google-takeout history-tracking json location-history-parser location-tracking parser personal-data

Last synced: 18 days ago
JSON representation

Easily parse location .json files provided by the Google Takeout service

• Host: GitHub
• URL: https://github.com/DovarFalcone/google-takeout-location-parser
• Owner: DovarFalcone
• License: mit
• Created: 2023-02-06T15:56:51.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-08-21T01:49:09.000Z (4 months ago)
• Last Synced: 2024-08-21T02:46:25.986Z (4 months ago)
• Topics: arcgis, geospatial, google, google-history, google-location-history, google-maps, google-takeout, history-tracking, json, location-history-parser, location-tracking, parser, personal-data
• Language: Python
• Homepage:
• Size: 1.06 MB
• Stars: 19
• Watchers: 1
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• project-awesome - DovarFalcone/google-takeout-location-parser - Easily parse location .json files provided by the Google Takeout service (Python)

README

        
# Location History Parser

There are three scripts in this repo, `semantic_location_parser.py`, `records_location_parser.py`, and `full_location_history_parser.py`.

Each file scans through the Takeouts directory and parses the Google location data.

The resulting csv files can be used in ArcGIS for spatial analysis and visualization. If you want to see your data visually fast, then convert the csv to a macro enabled `.xlms` file and use Excel's mapping tool.

![Fine Location data using records_location_parser.py](/assets/images/airport.png)

## Requirements

- Python 3.x

## Usage

1. Download and extract your Location History data from [Google Takeout](https://takeout.google.com/). Google will send you a link to download a `.zip` file. Unzip the file into a new directory.

2. Save the python scripts in the same folder that you extracted the Takeouts folder.

3. Open a terminal/command prompt and navigate to the location of the script and the data.

4. Run the script using the command `python semantic_location_parser.py`.

5. The resulting csv file will be saved in the same folder as the script and data with the name `semantic_location_history.csv`.

6. The steps are the same for each script.

![File Location](/assets/images/filelocation.png)

## Data Format

Google Takeout Location History data consists of two types of data: raw location history data and semantic location history data.

```

E:.

|   location_history_parser.py

|

\---Takeout

    |   archive_browser.html

    |

    \---Location History

        |   Records.json

        |   Settings.json

        |   Tombstones.csv

        |

        \---Semantic Location History

            +---2022

            |       2022_DECEMBER.json

            |       2022_NOVEMBER.json

            |

            \---2023

                    2023_FEBRUARY.json

                    2023_JANUARY.json

```

The `semantic_location_parser.py` script is used to parse Google Takeout Location History data into a csv file. The script can extract the `timestamp`, `address`, `placeId`, `name`, `latitudeE7` and `longitudeE7` values from the semantic location history data and store them in a csv file. 

The semantic location history data is found in the `Semantic Location History` folder and consists of more high-level and processed information compared to the raw location history data found in the `records.json` file. This data is partitioned by year in different subfolders and by month in different JSON files. Inside each semantic JSON file we can find a single flat `timelineObjects` array.

The `records_location_parser.py` is used to pull the less refined location data that is collected. This data can be used to highlight what roads you traveled along rather than just what location you visited.

The `full_location_history_parser.py`parses the same data as the `semantic_location_parser.py`, but it also parses the `records.json` file in the same script. The two files are appended together on the `timestamp`, `lat`, and `lon` columns. It's not the best way to store this data and not adivsed to be used.

This data set will have the following columns:

- `epoch_time`: Useful for APIs

- `timestamp`: sample start time from device

- `date_str`: timestamp to date

- `lat`: latitudinal coordinate. The values need to be divided by 107 to be in the expected range.

- `lon`: logitudinal coordinate. The values need to be divided by 107 to be in the expected range.

- `address`: Best guess address by Google

- `placeid`: Google place id

-  `name`: Name of location if it exists

-  `alt`: altitude

- `activity_type`: Most confident activity type

- `platformtype`: Platform used to access Google services

- `file_path`: Which folder the data came from

- `sys_time`: Comes from the `timestamp` key in the `records.json`, unique.

## Limitations

The script only extracts specific values from the raw location history data and the semantic location history data and stores them in their respective csv files. If you need to extract additional information, you may need to modify the script accordingly.