https://github.com/imjasonh/bqrekor

Scraping Rekor into a BigQuery dataset
https://github.com/imjasonh/bqrekor

Last synced: about 1 month ago
JSON representation

Scraping Rekor into a BigQuery dataset

• Host: GitHub
• URL: https://github.com/imjasonh/bqrekor
• Owner: imjasonh
• License: apache-2.0
• Created: 2023-11-19T23:12:51.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-11-19T23:17:05.000Z (about 1 year ago)
• Last Synced: 2024-10-14T16:51:30.614Z (3 months ago)
• Language: HCL
• Size: 31.3 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# bqrekor

Scraping Rekor into a BigQuery dataset

_WORK IN PROGRESS_

## Setup

```

terraform init

terraform apply

```

This will prompt for a Google Cloud project ID and region.

It will create:

* A BigQuery dataset and table called `rekor_logs`

* A Cloud Run Job that scrapes the Rekor API and write the results into a GCS object as JSON

* A Cloud Scheduler config to run the Cloud Run Job every 10 minutes

* A BigQuery Data Transfer config to load the GCS object into the BigQuery table on a schedule

The result is an automatically updating BigQuery table containing Rekor entries seen by the app.