{"id":51841636,"url":"https://github.com/seroperson/opengraph4s","last_synced_at":"2026-07-23T06:01:14.693Z","repository":{"id":365783910,"uuid":"712315888","full_name":"seroperson/opengraph4s","owner":"seroperson","description":"Deadly-simple library for fetching OpenGraph tags from a given webpage.","archived":false,"fork":false,"pushed_at":"2026-06-18T22:09:24.000Z","size":179,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-06-18T22:26:21.423Z","etag":null,"topics":["crawler","mill","opengraph","parser","scala","scrapper"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/seroperson.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-10-31T08:20:30.000Z","updated_at":"2026-06-18T22:09:25.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/seroperson/opengraph4s","commit_stats":null,"previous_names":["seroperson/opengraph4s"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/seroperson/opengraph4s","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seroperson%2Fopengraph4s","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seroperson%2Fopengraph4s/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seroperson%2Fopengraph4s/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seroperson%2Fopengraph4s/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/seroperson","download_url":"https://codeload.github.com/seroperson/opengraph4s/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/seroperson%2Fopengraph4s/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35791261,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-23T02:00:06.683Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","mill","opengraph","parser","scala","scrapper"],"created_at":"2026-07-23T06:01:14.186Z","updated_at":"2026-07-23T06:01:14.684Z","avatar_url":"https://github.com/seroperson.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"# opengraph4s\n\nDeadly-simple library for fetching OpenGraph tags from a given webpage.\n\n## Installation\n\nIn case if you use `sbt`:\n\n```sbt\nlibraryDependencies += \"me.seroperson\" %% \"opengraph4s\" % \"0.2.0\"\n```\n\nIn case of `mill`:\n\n```scala\nivy\"me.seroperson::opengraph4s::0.2.0\"\n```\n\n## How to use\n\nDefault implementation uses `sttp`, `fs2` and `scala-scrapper` to fetch and parse tags.\nShortly, to get started, it is enough to code something like this:\n\n```scala\nimport cats.effect.IO\nimport sttp.client4.httpclient.fs2.HttpClientFs2Backend\n\nimport me.seroperson.opengraph4s.client.SttpClient\nimport me.seroperson.opengraph4s.parser.ScalaScrapperParser\n\nval parser = ScalaScrapperParser[IO]()\nval backend = HttpClientFs2Backend.resource[IO]()\nval client = SttpClient(parser, backend)\n\nfor {\n  og \u003c- client.request(\"https://example.com/\")\n  _ \u003c- cats.effect.std.Console[IO].println(og)\n} yield og\n```\n\nIt initializes `parser`, which is responsible for parsing a fetched webpage, and `client`,\nwhich is responsible for fetching the webpage itself. The function\n`Client#request(String)` fetches a webpage and returns `F[OpenGraphParsed]` with all the\nparsed information.\n\nYou probably want to override headers on your requests to a webpage, such as\n`Accept-Language`, `User-Agent` and others, as long as many webpages response\ndifferently depending on user configuration. While using default implementation it is\nconfigurable easily by passing customized `backend` parameter to `SttpClient` (see\nexamples).\n\n## Examples\n\nThis repository contains `example/zio` and `example/cats` projects. You can run them like\nthis:\n\n```sh\n./millw example.__.run https://twitch.tv\n```\n\nHere `example/cats` uses `HtmlUnitBrowser` as parser and `IO` as effect type, while\n`example/zio` uses `JsoupBrowser` and `Task`.\n\n## Caveats\n\nDefault implementation works quite fast and parses simple pages correctly, but when it\ncomes to non-plain-html websites (like SPAs), `\u003chead\u003e` tag can be generated dynamically\nthen, which results in empty `OpenGraphParsed` response. Luckily, most of the cases can\nbe solved by using `HttpUnitBrowser` (from `scala-scrapper` library) with\n`ScalaScrapperParser#apply`. It renders JS and generates a html page for you, but take\nin mind that usually it works slower than default `JsoupBrowser` (because it renders JS\nactually) and it is not guaranteed that JS will be executed correctly.\n\nAlso, a webpage can have any kind of bot-protection: captcha, 403 for headless browser\nand so on.\n\nThis library doesn't handle such cases. If you are looking for any guarantees, probably\nit is better to consider using some SaaS web-scrappers then.\n\n## License\n\n```text\nMIT License\n\nCopyright (c) 2023 Daniil Sivak\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseroperson%2Fopengraph4s","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fseroperson%2Fopengraph4s","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fseroperson%2Fopengraph4s/lists"}