{"id":15063290,"url":"https://github.com/ne-lexa/roach-php-bundle","last_synced_at":"2025-04-10T10:50:48.339Z","repository":{"id":57024837,"uuid":"461933698","full_name":"Ne-Lexa/roach-php-bundle","owner":"Ne-Lexa","description":"Symfony bundle for roach-php/core","archived":false,"fork":false,"pushed_at":"2022-10-03T12:36:07.000Z","size":71,"stargazers_count":9,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-07T13:40:24.638Z","etag":null,"topics":["crawler","php","roach-php","scrapy","spider","symfony","symfony-bundle"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Ne-Lexa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-02-21T16:03:05.000Z","updated_at":"2023-12-18T12:26:46.000Z","dependencies_parsed_at":"2022-08-23T15:00:22.271Z","dependency_job_id":null,"html_url":"https://github.com/Ne-Lexa/roach-php-bundle","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ne-Lexa%2Froach-php-bundle","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ne-Lexa%2Froach-php-bundle/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ne-Lexa%2Froach-php-bundle/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Ne-Lexa%2Froach-php-bundle/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Ne-Lexa","download_url":"https://codeload.github.com/Ne-Lexa/roach-php-bundle/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248203098,"owners_count":21064512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","php","roach-php","scrapy","spider","symfony","symfony-bundle"],"created_at":"2024-09-24T23:54:35.644Z","updated_at":"2025-04-10T10:50:48.322Z","avatar_url":"https://github.com/Ne-Lexa.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\u003cimg src=\"logo.svg\" alt=\"roach-php-bundle\"/\u003e\u003c/div\u003e\n\n# roach-php-bundle\nSymfony bundle for [Roach PHP](https://roach-php.dev).\n\n[![Latest Stable Version](https://poser.pugx.org/nelexa/roach-php-bundle/v)](https://packagist.org/packages/nelexa/roach-php-bundle) [![PHP Version Require](https://poser.pugx.org/nelexa/roach-php-bundle/require/php)](https://packagist.org/packages/nelexa/roach-php-bundle)\n[![Tests](https://github.com/Ne-Lexa/roach-php-bundle/actions/workflows/build.yml/badge.svg)](https://github.com/Ne-Lexa/roach-php-bundle/actions/workflows/build.yml)\n[![Build Status](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/badges/build.png?b=main)](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/build-status/main)\n[![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/badges/quality-score.png?b=main)](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/?branch=main)\n[![Code Coverage](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/badges/coverage.png?b=main)](https://scrutinizer-ci.com/g/Ne-Lexa/roach-php-bundle/?branch=main)\n[![License](https://poser.pugx.org/nelexa/roach-php-bundle/license)](https://packagist.org/packages/nelexa/roach-php-bundle)\n\n\u003e Roach is a complete web scraping toolkit for PHP. It is ~~a shameless clone~~ heavily inspired by the popular Scrapy package for Python.\n\nThe Symfony bundle mostly provides the necessary container bindings for the various services Roach uses, as well as making certain configuration options available via a config file. To learn about how to actually start using Roach itself, check out the [rest of the documentation](https://roach-php.dev/docs/spiders).\n\n## Installing the Symfony bundle\nAdd `nelexa/roach-php-bundle` to your composer.json file:\n```bash\ncomposer require nelexa/roach-php-bundle\n```\n\n## Versions \u0026 Dependencies\n| Bundle version | roach-php/core version | Symfony version | PHP version(s) |\n|----------------|------------------------|-----------------|----------------|\n| 0.3.0          | 0.3.0                  | ^5.3 \\| ^6.0    | \u003e= 8.0         |\n| 1.0.0          | ^1.0.0                 | ^6.0            | \u003e= 8.0         |\n| 1.1.0          | 1.1.*                  | ^6.0            | \u003e= 8.0         |\n\n#### Register the bundle:\nRegister bundle into config/bundles.php (Flex did it automatically):\n```php\nreturn [\n    //...\n    \\Nelexa\\RoachPhpBundle\\RoachPhpBundle::class =\u003e ['all' =\u003e true],\n];\n```\n\n## Available Commands\nThe Symfony bundle of Roach registers a few console commands to make out development experience as pleasant as possible.\n\n### Run spider\n```bash\nphp bin/console roach:run\n```\nAfter that, you will get the entire list of available spiders.\n```text\n\n Choose a spider class:\n  [0] App\\Spider\\GoogleSpider\n  [1] App\\Spider\\FacebookSpider\n  [2] App\\Spider\\TwitterSpider\n\n```\nSimply select the desired spider (▼ or ▲) or enter its number and press Enter.\n\nYou can pass as the first argument the name spider class name to run or its alias.\nFor example, if you have a class `App\\Spider\\GoogleSpider`, then you can pass the following aliases: `GoogleSpider`, `google_spider` or `google`.\n```bash\nphp bin/console roach:run google\n```\nSometimes it is useful to override the number of concurrent requests and the pre-request delay. To do this, you can pass the `--concurrency` and `--delay` options.\n```bash\nphp bin/console roach:php google --concurrency 8 --delay 2\n```\nThese options override the `$concurrency` and `$requestDelay` public properties of your spider.\n\nAdd the `--output` (`-o`) option and you can save the collected data to a JSON file.\n```bash\nphp bin/console roach:php google --output 'path/to/data.json'\n```\n\n### Starting the REPL\n\nRoach ships with an [interactive shell](https://roach-php.dev/docs/repl) (often called Read-Evaluate-Print-Loop, or Repl for short) which makes prototyping our spiders a breeze. We can use the provided `roach:shell` command to launch a new Repl session.\n```bash\nphp bin/console roach:shell \"https://roach-php.dev/docs/introduction\"\n```\n\n### Generator classes\nFirst install [`Symfony MakerBundle`](https://symfony.com/bundles/SymfonyMakerBundle/current/index.html).\n```bash\ncomposer require --dev symfony/maker-bundle\n```\n\n#### Create a new roach spider class\n```bash\nphp bin/console make:roach:spider\n```\n\n#### Create a new roach extension class\n```bash\nphp bin/console make:roach:extension\n```\n\n#### Create a new roach item processor class\n```bash\nphp bin/console make:roach:item:processor\n```\n\n#### Create a new roach downloader request middleware class\n```bash\nphp bin/console make:roach:middleware:downloader:request\n```\n\n#### Create a new roach downloader response middleware class\n```bash\nphp bin/console make:roach:middleware:downloader:response\n```\n\n#### Create a new roach spider item middleware class\n```bash\nphp bin/console make:roach:middleware:spider:item\n```\n\n#### Create a new roach spider request middleware class\n```bash\nphp bin/console make:roach:middleware:spider:request\n```\n\n#### Create a new roach spider response middleware class\n```bash\nphp bin/console make:roach:middleware:spider:response\n```\n\n## Screencast\n[![asciicast](https://asciinema.org/a/naGrp60zyNVQakQxV0v0Vnwvc.svg)](https://asciinema.org/a/naGrp60zyNVQakQxV0v0Vnwvc)\n\n## Credits\n* [Ne-Lexa](https://github.com/Ne-Lexa)\n* [All contributors](https://github.com/Ne-Lexa/roach-php-bundle/graphs/contributors)\n\n## Changelog\nChanges are documented in the [releases page](https://github.com/Ne-Lexa/roach-php-bundle/releases).\n\n## License\nThe MIT License (MIT). Please see [LICENSE](LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fne-lexa%2Froach-php-bundle","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fne-lexa%2Froach-php-bundle","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fne-lexa%2Froach-php-bundle/lists"}