https://github.com/almoggutin/web-crawler-worker-node-js
Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.
https://github.com/almoggutin/web-crawler-worker-node-js
aws backend-api docker express javascript jest kubernetes nodejs parser redis rest-api web-crawler web-scraper web-worker
Last synced: 3 months ago
JSON representation
Worker part of the Web Crawler application. The Web Crawler app takes an input from the user such as a link maximum number of pages and depth. At the end it shows in real time a tree of all the links and pages that the crawler found in the provided URL.
- Host: GitHub
- URL: https://github.com/almoggutin/web-crawler-worker-node-js
- Owner: almoggutin
- Created: 2021-04-25T16:12:16.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2021-05-18T13:13:20.000Z (about 5 years ago)
- Last Synced: 2023-05-20T19:19:57.525Z (about 3 years ago)
- Topics: aws, backend-api, docker, express, javascript, jest, kubernetes, nodejs, parser, redis, rest-api, web-crawler, web-scraper, web-worker
- Language: JavaScript
- Homepage:
- Size: 201 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0