https://github.com/toddbirchard/lambda-metadata-scraper

🟩 🌐 Netlify Lambda function to scrape metadata from a target URL.
https://github.com/toddbirchard/lambda-metadata-scraper

go golang lambda lambda-functions netlify scrape scraper serverless

Last synced: 7 months ago
JSON representation

🟩 🌐 Netlify Lambda function to scrape metadata from a target URL.

• Host: GitHub
• URL: https://github.com/toddbirchard/lambda-metadata-scraper
• Owner: toddbirchard
• License: mit
• Created: 2020-07-03T21:08:39.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2022-01-19T14:26:58.000Z (almost 4 years ago)
• Last Synced: 2025-03-27T23:11:22.584Z (7 months ago)
• Topics: go, golang, lambda, lambda-functions, netlify, scrape, scraper, serverless
• Language: Go
• Homepage:
• Size: 10.6 MB
• Stars: 4
• Watchers: 1
• Forks: 3
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# Lambda Metadata Scraper

![Go](https://img.shields.io/badge/Go-1.17.6-blue.svg?logo=go&longCache=true&logoColor=white&colorB=88C0D0&style=flat-square&colorA=4c566a)

![Goquery](https://img.shields.io/badge/Goquery-1.8.0-blue.svg?logo=go&longCache=true&logoColor=white&colorB=88C0D0&style=flat-square&colorA=4c566a)

![AWS Lambda](https://img.shields.io/badge/AWS--Lambda-1.28.0-blue.svg?logo=go&longCache=true&logoColor=white&colorB=88C0D0&style=flat-square&colorA=4c566a)

![GitHub Last Commit](https://img.shields.io/github/last-commit/google/skia.svg?style=flat-square&colorA=4c566a&colorB=a3be8c&logo=GitHub)

[![GitHub Issues](https://img.shields.io/github/issues/toddbirchard/lambda-metadata-scraper.svg?style=flat-square&colorA=4c566a&colorB=ebcb8b&logo=GitHub)](https://github.com/toddbirchard/lambda-metadata-scraper/issues)

[![GitHub Stars](https://img.shields.io/github/stars/toddbirchard/lambda-metadata-scraper.svg?style=flat-square&colorB=ebcb8b&colorA=4c566a&logo=GitHub)](https://github.com/toddbirchard/lambda-metadata-scraper/stargazers)

[![GitHub Forks](https://img.shields.io/github/forks/toddbirchard/lambda-metadata-scraper.svg?style=flat-square&colorA=4c566a&colorB=ebcb8b&logo=GitHub)](https://github.com/toddbirchard/lambda-metadata-scraper/network)

Lambda function which returns basic metadata for a single URL provided as a `?url` querystring parameter. Intended to be deployed as an endpoint served with Netlify functions.

### Example Usage

* Request Type: `GET`

* Endpoint: `https://hackersandslackers.com/.netlify/functions/scrape`

* Parameter: `?url=[TARGET_URL]`

```bash

$ curl https://hackersandslackers.com/.netlify/functions/scrape?url=https://toddbirchard.com \

   -H "Content-Type: application/json"

```

**Response:**

```json

{

   "Title": "Todd Birchard: Engineering, Product, Technology.",

   "Image": "https://storage.googleapis.com/toddbirchard-cdn/2019/08/cover.jpeg",

   "Description": "Giant reptile giving technology a good name. Occasional tangents of mass destruction. Made in Silicon Alley.",

   "Favicon": "/icons/icon-48x48.png"

}

```