https://github.com/derekmeegan/mark_venaglia_edge_functions
Supabase Edge Functions for TripAdvisor review scraping and email handling with automatic GitHub Actions deployment.
https://github.com/derekmeegan/mark_venaglia_edge_functions
browserbase playwright supabase typescript webscraping
Last synced: 2 months ago
JSON representation
Supabase Edge Functions for TripAdvisor review scraping and email handling with automatic GitHub Actions deployment.
- Host: GitHub
- URL: https://github.com/derekmeegan/mark_venaglia_edge_functions
- Owner: derekmeegan
- Created: 2025-06-04T14:27:52.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-04T17:05:55.000Z (about 1 year ago)
- Last Synced: 2025-07-04T14:11:14.880Z (12 months ago)
- Topics: browserbase, playwright, supabase, typescript, webscraping
- Language: TypeScript
- Homepage: https://www.markvenaglia.com/
- Size: 17.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Venaglia Edge Functions
This repository contains Supabase Edge Functions for web scraping and email handling, deployed using GitHub Actions.
## Overview
The repository includes two main edge functions:
1. **scrape-reviews**: A Deno-based edge function that scrapes TripAdvisor reviews using Playwright and Browserbase, then stores the data in a Supabase database.
2. **send-email**: A simple edge function that processes contact form submissions and sends emails using SendGrid.
## Technologies Used
- [Deno](https://deno.land/) - The JavaScript/TypeScript runtime used for edge functions
- [Supabase](https://supabase.com/) - Backend-as-a-Service platform for hosting the edge functions
- [Playwright](https://playwright.dev/) - Browser automation library for web scraping
- [Browserbase](https://browserbase.com/) - Headless browser infrastructure for web scraping
- [SendGrid](https://sendgrid.com/) - Email delivery service
- [GitHub Actions](https://github.com/features/actions) - CI/CD for automatic deployment
## Edge Functions
### scrape-reviews
This function scrapes TripAdvisor reviews for a specific attraction:
- Uses Playwright with Browserbase for headless browser automation
- Scrapes multiple pages of reviews (configurable)
- Extracts detailed review data including:
- Reviewer information (name, profile, avatar)
- Review content (title, text, rating)
- Metadata (date, trip type, helpful votes)
- Stores the scraped data in a Supabase database table named "reviews"
- Handles pagination and error cases
#### Environment Variables Required:
- `BROWSERBASE_API_KEY` - API key for Browserbase
- `BROWSERBASE_PROJECT_ID` - Project ID for Browserbase
- `SUPABASE_URL` - URL of your Supabase project
- `SUPABASE_ANON_KEY` - Anonymous key for Supabase API access
### send-email
This function processes contact form submissions:
- Receives form data (name, email, subject, message)
- Forwards the information via SendGrid to a specified email address
- Handles CORS and provides appropriate responses
#### Environment Variables Required:
- `SENDGRID_API_KEY` - API key for SendGrid email service
## Deployment
The edge functions are automatically deployed to Supabase using GitHub Actions when changes are pushed to the main branch.
### GitHub Actions Workflow
The deployment workflow:
1. Triggers on push to main branch or manual workflow dispatch
2. Sets up the Supabase CLI
3. Deploys all functions to the specified Supabase project
#### Environment Variables Required for Deployment:
- `SUPABASE_ACCESS_TOKEN` - Access token for Supabase
- `PROJECT_ID` - Supabase project ID