{"id":17962154,"url":"https://github.com/skyzyx/lambda-htmlmeta","last_synced_at":"2026-01-22T17:01:58.776Z","repository":{"id":52057096,"uuid":"157163822","full_name":"skyzyx/lambda-htmlmeta","owner":"skyzyx","description":"Parse OpenGraph, OEmbed, Microdata, and Microformats 2 metadata from webpages via AWS Lambda.","archived":false,"fork":false,"pushed_at":"2025-11-18T20:19:23.000Z","size":20,"stargazers_count":3,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-18T22:20:54.894Z","etag":null,"topics":["api-gateway","aws","aws-lambda","cloudfront","golang","golang-application","lambda","microdata","microformats","microformats-2","microformats2","oembed","oembed-components","oembed-endpoints","oembed-services","opengraph","opengraph-data","opengraph-net","opengraph-tags","schema-org"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skyzyx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-11-12T06:00:27.000Z","updated_at":"2022-11-24T05:42:16.000Z","dependencies_parsed_at":"2022-08-30T22:40:48.929Z","dependency_job_id":null,"html_url":"https://github.com/skyzyx/lambda-htmlmeta","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/skyzyx/lambda-htmlmeta","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skyzyx%2Flambda-htmlmeta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skyzyx%2Flambda-htmlmeta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skyzyx%2Flambda-htmlmeta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skyzyx%2Flambda-htmlmeta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skyzyx","download_url":"https://codeload.github.com/skyzyx/lambda-htmlmeta/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skyzyx%2Flambda-htmlmeta/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28667299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T14:01:31.714Z","status":"ssl_error","status_checked_at":"2026-01-22T13:59:23.143Z","response_time":144,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-gateway","aws","aws-lambda","cloudfront","golang","golang-application","lambda","microdata","microformats","microformats-2","microformats2","oembed","oembed-components","oembed-endpoints","oembed-services","opengraph","opengraph-data","opengraph-net","opengraph-tags","schema-org"],"created_at":"2024-10-29T11:14:12.683Z","updated_at":"2026-01-22T17:01:58.750Z","avatar_url":"https://github.com/skyzyx.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HTML Metadata Parsing for AWS Lambda\n\nWill take a URL passed as a query string, and can generate output for OpenGraph, OEmbed, Microdata, and Microformats 2.\n\nThis is a _Serverless_ app, written in Go ([Golang]), running in AWS Lambda, with API Gateway in front of it, and AWS CloudFront in front of that (for caching).\n\n**This is an experiment.** Uptime is not guaranteed, and there is no SLA. _But_ all-in-all, it should be reasonably reliable.\n\n\u003e **NOTE:** I've tried to build this in a way that is very cheap to run. But if you find yourself using this more than just occasionally, consider kicking me down a few bucks to offset the cost of running this service. \u003chttps://cash.me/$rparman\u003e\n\n## Usage\n\nThe `https://metadata.ryanparman.com` hostname is a CloudFront caching distribution, in front of API Gateway, in front of a Lambda function.\n\nThere are four endpoints:\n\n* `GET /microdata` — This will return all [Microdata](https://schema.org/docs/gs.html)-formatted metadata (e.g., schema.org) as JSON.\n* `GET /microformats2` — This will return all [Microformats 2](http://microformats.org/wiki/microformats-2)-formatted metadata as JSON.\n* `GET /oembed` — This will return an [oEmbed](https://oembed.com) response, formatted as JSON.\n* `GET /opengraph` — This will return all [OpenGraph](http://ogp.me)-formatted metadata as JSON.\n\nAdditionally, it accepts one query-string parameter.\n\n| Parameter | Description |\n| --------- | ----------- |\n| `url` | (Required) The URL to parse the metadata from. |\n\n### Examples\n\n* [Microdata](https://metadata.ryanparman.com/microdata/?url=https://ryanparman.com/posts/2018/serving-bandwidth-friendly-video-with-hls/)\n* [Microformats 2](https://metadata.ryanparman.com/microformats2/?url=https://ryanparman.com/posts/2018/serving-bandwidth-friendly-video-with-hls/)\n* [oEmbed](https://metadata.ryanparman.com/oembed/?url=https://ryanparman.com/posts/2018/serving-bandwidth-friendly-video-with-hls/)\n* [OpenGraph](https://metadata.ryanparman.com/opengraph/?url=https://ryanparman.com/posts/2018/serving-bandwidth-friendly-video-with-hls/)\n\n## Developing/Deploying\n\n### Golang\n\nGo (when spoken) or [Golang] (when written) is a strongly-typed language from Google that \"blends the simplicity of Python with the performance of C\". Static binaries can be compiled for all major platforms, and many minor ones.\n\nIt is recommended that you install Golang using your system's package manager. If you don't have one (or if the version is too old), you can [install Golang from its website](https://golang.org/doc/install). Reading the [Getting Started](https://golang.org/doc/) documentation is a valuable exercise.\n\n```bash\nbrew update \u0026\u0026 brew install golang\n```\n\n### Glide\n\nGolang dependencies are managed with [Glide]. You should install them before compiling this project.\n\n```bash\ncurl https://glide.sh/get | sh\nglide install\n```\n\n### GoMetaLinter\n\n[GoMetaLinter] pulls together many popular linting tools, and can run them on a project.\n\n```bash\ngometalinter.v2 --install\n```\n\n### Serverless\n\n[Serverless] is a platform that wraps AWS Lambda and AWS CloudFormation, simplifying the deployment of Lambda apps. Serverless is written in Node.js, so you need to install that first.\n\nI recommend you install the [Node Version Manager][nvm], and use that to install the latest Node.js and npm. Once that's complete, install `serverless`.\n\n```bash\nnpm i -g serverless\n```\n\n### Developing\n\nThis app is small, and is self-contained in `main.go`.\n\n_By default_, it expects to be running in AWS Lambda, receiving HTTP requests coming from API Gateway.\n\nIf you are performing local development/testing, run `make build` to build for the local platform, then `bin/htmlinfo -mock` to run it. The local build reads from `mock.json` and treats it as an incoming request from API Gateway. You can change the query-string parameters to have the app respond to the documented query string parameters.\n\n```bash\nmake build\n\n# OpenGraph\nMETA_OPENGRAPH=true bin/htmlinfo -mock\n\n# oEmbed\nMETA_OEMBED=true bin/htmlinfo -mock\n\n# Microdata\nMETA_MICRODATA=true bin/htmlinfo -mock\n\n# Microformats 2\nMETA_MICROFORMATS2=true bin/htmlinfo -mock\n```\n\nMake sure that you run the linter to catch any issues.\n\n```bash\nmake lint\n```\n\n### Deployment\n\n`serverless` uses the same [local credentials](https://docs.aws.amazon.com/cli/latest/topic/config-vars.html) that the AWS CLI tools and the AWS SDKs use. If you haven't configured those yet, do that first.\n\nRun `make package` to build a binary for AWS Lambda. Then, `serverless deploy` to deploy the app to your environment.\n\n```bash\nmake deploy\n```\n\n\u003e **IMPORTANT:** Since this deployment uses CloudFront, deployments take _much_ longer than normal deployments. First-time deployments can take around 30 minutes. Subsequent deployments can still take several minutes.\n\n  [Glide]: https://glide.sh\n  [Golang]: https://golang.org\n  [GoMetaLinter]: https://github.com/alecthomas/gometalinter\n  [nvm]: https://github.com/creationix/nvm\n  [Serverless]: https://serverless.com/framework/docs/getting-started/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskyzyx%2Flambda-htmlmeta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskyzyx%2Flambda-htmlmeta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskyzyx%2Flambda-htmlmeta/lists"}