https://github.com/rubnsbarbosa/nasa-asteroids-extractor

ETL asteroids data extractor using some Google Cloud services
https://github.com/rubnsbarbosa/nasa-asteroids-extractor

bigquery bucket cloud-storage google-cloud nasa-api-neows

Last synced: 4 days ago
JSON representation

ETL asteroids data extractor using some Google Cloud services

• Host: GitHub
• URL: https://github.com/rubnsbarbosa/nasa-asteroids-extractor
• Owner: rubnsbarbosa
• License: mit
• Created: 2024-06-22T17:15:17.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-06-30T16:24:23.000Z (7 months ago)
• Last Synced: 2025-01-19T05:12:12.681Z (7 days ago)
• Topics: bigquery, bucket, cloud-storage, google-cloud, nasa-api-neows
• Language: Python
• Homepage:
• Size: 14.6 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


    





    Retrieve asteroids based on their closest approach to Earth data extractor from NASA API.



### Introduction

This repository provides an example of a data extractor etl:

* Extracts data from [Asteroids - NeoWs API](https://api.nasa.gov//).

`GET https://api.nasa.gov/neo/rest/v1/feed?start_date=START_DATE&end_date=END_DATE&api_key=API_KEY`

* Transform the JSON object, then load the csv file into a `gs://bucket`

* Load to Big Query from Google Cloud Storage

### Prerequisites

Ensure you have the following:

- Python 3.8 or later installed

- Google Cloud SDK installed and configured

- A Google Cloud project with billing enabled

### Setup

1. Clone the repository:

```bash

git clone https://github.com/rubnsbarbosa/nasa-asteroids-extractor.git

cd nasa-asteroids-extractor

```

2. Create a virtual environment and install dependencies:

```bash

python3 -m venv .venv

source .venv/bin/activate

pip install -r requirements.txt

```

### License

MIT.