https://github.com/dfirebaugh/web-crawler
a simple web scraper that creates a list of images and what pages they exist on
https://github.com/dfirebaugh/web-crawler
Last synced: 8 months ago
JSON representation
a simple web scraper that creates a list of images and what pages they exist on
- Host: GitHub
- URL: https://github.com/dfirebaugh/web-crawler
- Owner: dfirebaugh
- Created: 2018-09-27T19:53:59.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-09-27T19:54:27.000Z (over 7 years ago)
- Last Synced: 2024-12-27T19:29:55.797Z (over 1 year ago)
- Language: JavaScript
- Size: 5.86 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Web Crawler
==
This web crawler visits your home page and compiles a list of links on your site. It will then visit those links and create a list of images and what pages they reside on.
create a .env file with the root level url:
URL="http://example.com"
execute by running `node index.js`