https://github.com/fadilxcoder/web-crawler
Crawl website to scrap data (TV shows / Foot Stream)
https://github.com/fadilxcoder/web-crawler
notes php poc
Last synced: about 1 year ago
JSON representation
Crawl website to scrap data (TV shows / Foot Stream)
- Host: GitHub
- URL: https://github.com/fadilxcoder/web-crawler
- Owner: fadilxcoder
- Created: 2021-08-29T17:58:12.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-03-01T17:56:28.000Z (about 3 years ago)
- Last Synced: 2025-01-04T19:23:26.474Z (over 1 year ago)
- Topics: notes, php, poc
- Language: PHP
- Homepage: https://my-json-server.typicode.com/fadilxcoder/web-crawler
- Size: 18.6 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Notes
- Install : `composer require fabpot/goutte`
- Crawlers
- - Alibaba website for watches names
- - Local car rental login success / failed (email dispatched in mailtrap)
----
- CLI : `php run` generate JSON db
- WEB GUI : `index.php` - for debugging
- API
- - https://my-json-server.typicode.com/fadilxcoder/web-crawler
- - https://my-json-server.typicode.com/fadilxcoder/web-crawler/db
- - https://my-json-server.typicode.com/fadilxcoder/web-crawler/person-of-interest-season-1
- Routes
- - https://my-json-server.typicode.com/fadilxcoder/streamer/movies
- - https://my-json-server.typicode.com/fadilxcoder/streamer/movies/1
## Docs
- https://github.com/FriendsOfPHP/Goutte (Github)
- https://symfony.com/doc/current/http_client.html (HTTP Client - Symfony components)
- https://symfony.com/doc/current/components/dom_crawler.html (The DomCrawler Component - Symfony components)
- https://symfony.com/components/BrowserKit (BrowserKit component)