https://github.com/nstapelbroek/estate-crawler
Scraping the real estate agencies for up-to-date house listings as soon as they arrive!
https://github.com/nstapelbroek/estate-crawler
appartments crawler huurwoningen nederland python real-estate-agencies scrapy scrapy-crawler
Last synced: 5 months ago
JSON representation
Scraping the real estate agencies for up-to-date house listings as soon as they arrive!
- Host: GitHub
- URL: https://github.com/nstapelbroek/estate-crawler
- Owner: nstapelbroek
- License: bsd-3-clause
- Created: 2016-08-11T20:33:49.000Z (almost 10 years ago)
- Default Branch: latest
- Last Pushed: 2025-03-01T07:19:45.000Z (over 1 year ago)
- Last Synced: 2026-01-17T11:25:15.407Z (5 months ago)
- Topics: appartments, crawler, huurwoningen, nederland, python, real-estate-agencies, scrapy, scrapy-crawler
- Language: Python
- Homepage:
- Size: 878 KB
- Stars: 25
- Watchers: 3
- Forks: 6
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Estate Crawler
Finding a place to live can be tough. If for any reason you are unable
to buy a house, the search for a rental apartment can cost you a lot
of time and money. Real estate agencies will often charge you before
you've even visited your first potential "object".
The subscriptions they force upon you are unfair and do not
guarantee success of finding a place to live.
This project aims to relieve some pain by giving you insights of the
offers at several agencies. By collecting and comparing this data,
you can pick an agency that is best suited for your needs.
Saving you time and potentially money.
# Usage
You can run the estate crawler by executing the crawler.py file. Below is an example with Python3 and pipenv installed on your OS:
```bash
pipenv install && pipenv run ./crawler.py --region amsterdam,rotterdam,arnhem
```
If you have the luxury of Docker available on your system, you can run the crawler without installing any dependencies like so:
```bash
docker run --rm -v $(pwd):/app/build docker.io/nstapelbroek/estate-crawler --region amsterdam
```
Results of your crawl run are available in ./build/results.json after a successful run. You can change this path by passing
a `--output-file` argument. Note that the output file does not contain valid json, only valid json lines.