https://github.com/aporcupine/go-rightmove
Implements a very basic scraper to read property details from RightMove.
https://github.com/aporcupine/go-rightmove
golang rightmove scraper
Last synced: 7 months ago
JSON representation
Implements a very basic scraper to read property details from RightMove.
- Host: GitHub
- URL: https://github.com/aporcupine/go-rightmove
- Owner: aporcupine
- License: mit
- Created: 2020-09-19T11:41:25.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-09-19T19:07:16.000Z (almost 6 years ago)
- Last Synced: 2024-06-20T16:50:08.316Z (about 2 years ago)
- Topics: golang, rightmove, scraper
- Language: Go
- Homepage:
- Size: 5.86 KB
- Stars: 3
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# go-rightmove
Implements a very basic scraper to read property details from RightMove.
This package extracts and unmarshals the JSON object that can be found at the bottom of a RightMove search page.
## Usage
1. Go to rightmove.co.uk and search for whatever type of properties you're interested in
1. Add your required filters to the search
1. Execute the search and copy the URL of the first results page (e.g. https://www.rightmove.co.uk/property-for-sale/find.html?searchType=SALE&locationIdentifier=REGION%5E87490&maxPrice=200000)
1. Call the `GetPropertiesFromSearchURL` function with the search URL
```golang
ctx := context.Background()
searchURL := "https://www.rightmove.co.uk/property-for-sale/find.html?searchType=SALE&locationIdentifier=REGION%5E87490&maxPrice=200000"
properties, err := rightmove.GetPropertiesFromSearchURL(ctx, searchURL)
if err != nil {
log.Fatalln(err)
}
fmt.Printf("Retrieved %d properties from RightMove\n", len(properties))
// Output: Retrieved 817 properties from RightMove
```
1. Process the resulting slice as necessary
## Legal
This implementation is a proof of concept only. The RightMove 'terms of use' states that the use of "scraping technology" is unauthorized. Do not use this library.