{"id":36987458,"url":"https://github.com/sharpapi/laravel-web-scraping-api","last_synced_at":"2026-02-21T12:02:38.387Z","repository":{"id":331651021,"uuid":"1131198267","full_name":"sharpapi/laravel-web-scraping-api","owner":"sharpapi","description":"Laravel package for SharpAPI.com - laravel-web-scraping-api","archived":false,"fork":false,"pushed_at":"2026-01-10T15:14:18.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-11T04:33:02.978Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sharpapi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-01-09T16:13:25.000Z","updated_at":"2026-01-10T15:14:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sharpapi/laravel-web-scraping-api","commit_stats":null,"previous_names":["sharpapi/laravel-web-scraping-api"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/sharpapi/laravel-web-scraping-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpapi%2Flaravel-web-scraping-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpapi%2Flaravel-web-scraping-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpapi%2Flaravel-web-scraping-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpapi%2Flaravel-web-scraping-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sharpapi","download_url":"https://codeload.github.com/sharpapi/laravel-web-scraping-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sharpapi%2Flaravel-web-scraping-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28405141,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-13T21:51:37.118Z","status":"ssl_error","status_checked_at":"2026-01-13T21:45:14.585Z","response_time":56,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-13T23:07:16.778Z","updated_at":"2026-01-13T23:07:17.415Z","avatar_url":"https://github.com/sharpapi.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"![SharpAPI GitHub cover](https://sharpapi.com/sharpapi-github-laravel-bg.jpg \"SharpAPI Laravel Client\")\n\n# Web Scraping API for Laravel\n\n## 🚀 Powerful web scraping capabilities for your Laravel applications.\n\n[![Latest Version on Packagist](https://img.shields.io/packagist/v/sharpapi/laravel-web-scraping-api.svg?style=flat-square)](https://packagist.org/packages/sharpapi/laravel-web-scraping-api)\n[![Total Downloads](https://img.shields.io/packagist/dt/sharpapi/laravel-web-scraping-api.svg?style=flat-square)](https://packagist.org/packages/sharpapi/laravel-web-scraping-api)\n\nCheck the details at SharpAPI's [Web Scraping API](https://sharpapi.com/en/catalog/utility/web-scraping-api) page.\n\n---\n\n## Requirements\n\n- PHP \u003e= 8.1\n- Laravel \u003e= 9.0\n\n---\n\n## Installation\n\nFollow these steps to install and set up the SharpAPI Laravel Web Scraping API package.\n\n1. Install the package via `composer`:\n\n```bash\ncomposer require sharpapi/laravel-web-scraping-api\n```\n\n2. Register at [SharpAPI.com](https://sharpapi.com/) to obtain your API key.\n\n3. Set the API key in your `.env` file:\n\n```bash\nSHARP_API_KEY=your_api_key_here\n```\n\n4. **[OPTIONAL]** Publish the configuration file:\n\n```bash\nphp artisan vendor:publish --tag=sharpapi-web-scraping-api\n```\n\n---\n## Key Features\n\n- **HTML Extraction**: Extract the full HTML content of any webpage.\n- **Structured Data Extraction**: Extract specific elements from webpages using CSS selectors.\n- **Screenshot Capture**: Take screenshots of webpages.\n- **Link Extraction**: Extract all links from a webpage.\n- **Image Extraction**: Extract all images from a webpage.\n- **JavaScript Rendering**: Properly render JavaScript-heavy websites.\n- **Proxy Support**: Use proxies to avoid IP blocking.\n\n---\n\n## Usage\n\nYou can inject the `WebScrapingApiService` class to access the web scraping functionality.\n\n### Basic Workflow\n\n1. **Scrape a Webpage**: Use `scrapeWebpage` to get the full HTML content of a webpage.\n2. **Extract Structured Data**: Use `extractStructuredData` to extract specific elements from a webpage using CSS selectors.\n3. **Take a Screenshot**: Use `takeScreenshot` to capture a screenshot of a webpage.\n4. **Extract Links**: Use `extractLinks` to extract all links from a webpage.\n5. **Extract Images**: Use `extractImages` to extract all images from a webpage.\n\n---\n\n### Controller Example\n\nHere is an example of how to use `WebScrapingApiService` within a Laravel controller:\n\n```php\n\u003c?php\n\nnamespace App\\Http\\Controllers;\n\nuse GuzzleHttp\\Exception\\GuzzleException;\nuse SharpAPI\\WebScrapingApi\\WebScrapingApiService;\n\nclass ScrapingController extends Controller\n{\n    protected WebScrapingApiService $scrapingService;\n\n    public function __construct(WebScrapingApiService $scrapingService)\n    {\n        $this-\u003escrapingService = $scrapingService;\n    }\n\n    /**\n     * @throws GuzzleException\n     */\n    public function scrapeWebpage(string $url)\n    {\n        $result = $this-\u003escrapingService-\u003escrapeWebpage($url);\n        \n        return response()-\u003ejson($result);\n    }\n\n    /**\n     * @throws GuzzleException\n     */\n    public function extractData(string $url)\n    {\n        $selectors = [\n            'title' =\u003e 'h1',\n            'paragraphs' =\u003e 'p',\n            'links' =\u003e 'a',\n        ];\n        \n        $result = $this-\u003escrapingService-\u003eextractStructuredData($url, $selectors);\n        \n        return response()-\u003ejson($result);\n    }\n\n    /**\n     * @throws GuzzleException\n     */\n    public function takeScreenshot(string $url)\n    {\n        $options = [\n            'width' =\u003e 1280,\n            'height' =\u003e 800,\n            'fullPage' =\u003e true,\n        ];\n        \n        $result = $this-\u003escrapingService-\u003etakeScreenshot($url, $options);\n        \n        return response()-\u003ejson($result);\n    }\n\n    /**\n     * @throws GuzzleException\n     */\n    public function extractLinks(string $url)\n    {\n        $result = $this-\u003escrapingService-\u003eextractLinks($url);\n        \n        return response()-\u003ejson($result);\n    }\n\n    /**\n     * @throws GuzzleException\n     */\n    public function extractImages(string $url)\n    {\n        $result = $this-\u003escrapingService-\u003eextractImages($url);\n        \n        return response()-\u003ejson($result);\n    }\n}\n```\n\n### Handling Guzzle Exceptions\n\nAll requests are managed by Guzzle, so it's helpful to be familiar with [Guzzle Exceptions](https://docs.guzzlephp.org/en/stable/quickstart.html#exceptions).\n\nExample:\n\n```php\nuse GuzzleHttp\\Exception\\ClientException;\n\ntry {\n    $result = $this-\u003escrapingService-\u003escrapeWebpage('https://example.com');\n} catch (ClientException $e) {\n    echo $e-\u003egetMessage();\n}\n```\n\n---\n\n## Optional Configuration\n\nYou can customize the configuration by setting the following environment variables in your `.env` file:\n\n```bash\nSHARP_API_KEY=your_api_key_here\nSHARP_API_BASE_URL=https://sharpapi.com/api/v1\n```\n\n---\n\n## Web Scraping Data Format Example\n\n```json\n{\n  \"url\": \"https://sharpapi.com/\",\n  \"timestamp\": \"2025-03-15T08:56:04.946195Z\",\n  \"scraped_data\": {\n    \"title\": \"AI-powered Workflow Automation API\",\n    \"detected_language\": \"en\",\n    \"headers\": {\n      \"charset\": \"utf-8\",\n      \"contentType\": null,\n      \"viewport\": [\"width=device-width\", \"initial-scale=1\"],\n      \"canonical\": \"https://sharpapi.com/\",\n      \"csrfToken\": \"ju6SYrG2vXRDCBVCXYQ1RWH5dn1qxnFmHC7dqNc9\"\n    },\n    \"meta_tags\": {\n      \"author\": null,\n      \"image\": null,\n      \"keywords\": [\"SharpAPI\", \"AI\", \"automation\", \"API\"],\n      \"description\": \"Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more.\"\n    },\n    \"open_graph\": {\n      \"og:title\": \"AI-powered Workflow Automation API\",\n      \"og:type\": \"website\",\n      \"og:URL\": \"https://sharpapi.com\",\n      \"og:image\": \"https://sharpapi.com/build/assets/sharpapi-website-preview-ARuIroBi.png\",\n      \"og:description\": \"Leverage AI API to streamline workflow in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more.\"\n    },\n    \"twitter_card\": {\n      \"twitter:card\": \"summary\",\n      \"twitter:site\": \"@sharpapi\",\n      \"twitter:creator\": \"@a2zwebltd\"\n    },\n    \"content_structured\": [\n      {\n        \"tag\": \"h1\",\n        \"content\": \"Automate workflows with AI-powered API\"\n      },\n      {\n        \"tag\": \"h2\",\n        \"content\": \"Leverage AI API for automation in E-Commerce, Marketing, Content Management, HR Tech, Travel, and more.\"\n      }\n    ],\n    \"links\": {\n      \"internal\": [\"https://sharpapi.com/register\", \"https://sharpapi.com/en/catalog\"],\n      \"external\": [\"https://github.com/sharpapi/\", \"https://chatgpt.com/g/g-4P3SFFu6Q-sharpapi-com-automate-with-ai\"]\n    }\n  }\n}\n```\n\n---\n\n## Support \u0026 Feedback\n\nFor issues or suggestions, please:\n\n- [Open an issue on GitHub](https://github.com/sharpapi/laravel-web-scraping-api/issues)\n- Join our [Telegram community](https://t.me/sharpapi_community)\n\n---\n\n## Changelog\n\nPlease see [CHANGELOG](CHANGELOG.md) for a detailed list of changes.\n\n---\n\n## Credits\n\n- [A2Z WEB LTD](https://github.com/a2zwebltd)\n- [Dawid Makowski](https://github.com/makowskid)\n- Enhance your [Laravel AI](https://sharpapi.com/) capabilities!\n\n---\n\n## License\n\nThe MIT License (MIT). Please see [License File](LICENSE.md) for more information.\n\n---\n\n## Follow Us\n\nStay updated with news, tutorials, and case studies:\n\n- [SharpAPI on X (Twitter)](https://x.com/SharpAPI)\n- [SharpAPI on YouTube](https://www.youtube.com/@SharpAPI)\n- [SharpAPI on Vimeo](https://vimeo.com/SharpAPI)\n- [SharpAPI on LinkedIn](https://www.linkedin.com/products/a2z-web-ltd-sharpapicom-automate-with-aipowered-api/)\n- [SharpAPI on Facebook](https://www.facebook.com/profile.php?id=61554115896974)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpapi%2Flaravel-web-scraping-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsharpapi%2Flaravel-web-scraping-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsharpapi%2Flaravel-web-scraping-api/lists"}