{"id":25595495,"url":"https://github.com/polywrap/web-scraper","last_synced_at":"2026-06-13T21:32:48.656Z","repository":{"id":179088577,"uuid":"662747640","full_name":"polywrap/web-scraper","owner":"polywrap","description":null,"archived":false,"fork":false,"pushed_at":"2023-09-26T13:55:40.000Z","size":137,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-09-27T04:00:27.366Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/polywrap.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-07-05T19:55:42.000Z","updated_at":"2023-09-27T04:00:27.367Z","dependencies_parsed_at":null,"dependency_job_id":"d13c285f-a31c-49b7-af05-ec0238ddfbcd","html_url":"https://github.com/polywrap/web-scraper","commit_stats":null,"previous_names":["polywrap/web-scraper"],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polywrap%2Fweb-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polywrap%2Fweb-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polywrap%2Fweb-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/polywrap%2Fweb-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/polywrap","download_url":"https://codeload.github.com/polywrap/web-scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240005473,"owners_count":19732759,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-21T11:29:31.120Z","updated_at":"2026-04-15T04:30:19.846Z","avatar_url":"https://github.com/polywrap.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Web-Scraper\nA wrap of the popular [scraper](https://crates.io/crates/scraper) crate.\n\n## Integrate\n### 1. Polywrap Setup\n`polywrap.graphql`:\n```graphql\n#import * into WebScraper from \"wrapscan.io/polywrap/web-scraper@1\"\n```\n\n`polywrap.yaml`:\n```yaml\nformat: 0.3.0\nproject:\n  name: my-app\n  type: app/typescript|python|rust|kotlin|swift\nsource:\n  schema: ./polywrap.graphql\n```\n\nCodegen:\n```bash\n$ polywrap codegen\n```\n\n### 2. Run The Wrap\n\n`app/typescript`:\n```typescript\nconst webScraper = new WebScraper();\n\nawait webScraper.get_text({\n  url: \"...\",\n});\n```\n\n`app/python`:\n```python\nweb_scraper = WebScraper()\n\nresult = web_scraper.get_text({\n    \"url\": \"...\"\n})\n```\n\n`app/rust`:\n```rust\nlet web_scraper = WebScraper::new();\n\nlet result = web_scraper.get_text(\n    \u0026WebScraperArgsGetText{\n        url: \"...\"\n    }\n).unwrap();\n```\n\n`app/kotlin`:\n```kotlin\nval webScraper = WebScraper(client)\n\nval result = webScraper.get_text(\n  WebScraperArgsGetText(\"...\")\n).getOrThrow()\n```\n\n`app/swift`:\n```swift\nlet web_scraper = WebScraper()\ntry? web_scraper.get_text(\n  args: WebScraperArgsGetText(url: \"...\")\n)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolywrap%2Fweb-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpolywrap%2Fweb-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpolywrap%2Fweb-scraper/lists"}