https://github.com/Khusbu/mirror-website

A web crawler which mirrors a website.
https://github.com/Khusbu/mirror-website

Last synced: 3 months ago
JSON representation

A web crawler which mirrors a website.

• Host: GitHub
• URL: https://github.com/Khusbu/mirror-website
• Owner: Khusbu
• Created: 2015-09-18T06:37:07.000Z (about 9 years ago)
• Default Branch: master
• Last Pushed: 2015-12-07T14:37:07.000Z (almost 9 years ago)
• Last Synced: 2024-06-29T04:33:48.634Z (5 months ago)
• Language: Go
• Size: 3.08 MB
• Stars: 1
• Watchers: 4
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# mirror-website

A web crawler in Go which mirrors a website.

# Dependencies:

- Go Version- go1.5 linux/amd64

- Go package- net/http

# References:

- https://jdanger.com/build-a-web-crawler-in-go.html

- http://schier.co/blog/2015/04/26/a-simple-web-scraper-in-go.html

- https://golang.org/

# To run the program:

$go run webCrawler.go http://www.start-url.com