https://github.com/codehakase/crawl
A simple sitemap scrapper written in Go
https://github.com/codehakase/crawl
Last synced: 10 months ago
JSON representation
A simple sitemap scrapper written in Go
- Host: GitHub
- URL: https://github.com/codehakase/crawl
- Owner: codehakase
- Created: 2018-10-25T16:31:43.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2020-06-14T21:03:03.000Z (over 5 years ago)
- Last Synced: 2025-02-12T10:18:31.820Z (11 months ago)
- Language: Go
- Size: 52.7 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Crawl
A simple sitemap scrapper written in Go. Crawls a given url `u` and writes all
links (on same domain) to a `sitemap.xml` file.
## Installation
Via `go get`
```
$ go get github.com/codehakase/crawl
```
Run http server:
```
$ go build main.go
$ ./main # navigate to http://localhost:3000 to input url to crawl
```
Run from command line:
```
$ go run main.go https://example.com
```