{"id":25417323,"url":"https://github.com/eteubert/web_inspector","last_synced_at":"2026-02-02T02:32:21.142Z","repository":{"id":139376540,"uuid":"166803803","full_name":"eteubert/web_inspector","owner":"eteubert","description":"Elixir web inspector to unfurl URLs","archived":false,"fork":false,"pushed_at":"2024-10-29T09:00:00.000Z","size":195,"stargazers_count":2,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-10-31T09:37:23.314Z","etag":null,"topics":["elixir"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eteubert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-21T11:35:40.000Z","updated_at":"2024-10-29T09:00:04.000Z","dependencies_parsed_at":null,"dependency_job_id":"633b8b9c-a4f6-4d64-b890-3810ac87d94c","html_url":"https://github.com/eteubert/web_inspector","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eteubert/web_inspector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eteubert%2Fweb_inspector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eteubert%2Fweb_inspector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eteubert%2Fweb_inspector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eteubert%2Fweb_inspector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eteubert","download_url":"https://codeload.github.com/eteubert/web_inspector/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eteubert%2Fweb_inspector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29001654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-02T01:32:03.847Z","status":"online","status_checked_at":"2026-02-02T02:00:07.448Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir"],"created_at":"2025-02-16T17:38:21.546Z","updated_at":"2026-02-02T02:32:21.126Z","avatar_url":"https://github.com/eteubert.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WebInspector\n\nElixir web inspector to unfurl URLs.\n\n## Installation\n\nThe package can be installed by adding `web_inspector` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:web_inspector, git: \"https://github.com/eteubert/web_inspector.git\"}\n  ]\nend\n```\n\n## Usage\n\n```elixir\nWebInspector.unfurl(\"https://podlove.org\")\n{:ok,\n %{\n   description: nil,\n   embed: nil,\n   icon: %{\n     height: \"32\",\n     type: \"icon\",\n     url: \"https://podlove.org/files/2014/06/cropped-podlove-avatar-bkd-1024-32x32.png\",\n     width: \"32\"\n   },\n   locations: [\"https://podlove.org\"],\n   original_url: \"https://podlove.org\",\n   providers: %{\n     misc: %{\n       \"canonical_url\" =\u003e nil,\n       \"icons\" =\u003e [\n         %{\n           height: \"32\",\n           type: \"icon\",\n           url: \"https://podlove.org/files/2014/06/cropped-podlove-avatar-bkd-1024-32x32.png\",\n           width: \"32\"\n         },\n         %{\n           height: \"192\",\n           type: \"icon\",\n           url: \"https://podlove.org/files/2014/06/cropped-podlove-avatar-bkd-1024-192x192.png\",\n           width: \"192\"\n         }\n       ],\n       \"title\" =\u003e \"Podlove | Personal Media Development\"\n     },\n     oembed: %{},\n     open_graph: %{},\n     twitter: %{}\n   },\n   site_name: \"podlove.org\",\n   site_url: \"https://podlove.org\",\n   title: \"Podlove | Personal Media Development\",\n   url: \"https://podlove.org\"\n }}\n```\n\n## Config\n\n### Puppeteer\n\nYou can setup your own Puppeteer server if you need full control over the page visit and extracted data. It should return data in the following format:\n\n```js\n{\n    site_name,\n    title,\n    description,\n    image,\n    url\n}\n```\n\nThe other extractors still run, but data extracted from the Puppeteer takes precedence.\n\nEnable Puppeteer and define its hostname in the config:\n\n```elixir\nconfig :web_inspector, puppeteer_enabled: true\nconfig :web_inspector, puppeteer_host: \"localhost:5000\"\n```\n\n### Scraping Ant\n\nInstead of plain HTTP requests, there is a [Scraping Ant](https://scrapingant.com/) adapter for more reliable results in case you run into bot detection issues. You need to put the API key into the environment variable `SCRAPINGANT_API_KEY`.\n\nFor example, create a `.env` file with the content:\n\n```\nSCRAPINGANT_API_KEY=abcde...789\n```\n\nIt will be loaded automatically. Or use your own way of setting environment variables.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feteubert%2Fweb_inspector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feteubert%2Fweb_inspector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feteubert%2Fweb_inspector/lists"}