{"id":38207965,"url":"https://github.com/nstapelbroek/estate-crawler","last_synced_at":"2026-01-18T09:19:27.169Z","repository":{"id":12618764,"uuid":"65499795","full_name":"nstapelbroek/estate-crawler","owner":"nstapelbroek","description":"Scraping the real estate agencies for up-to-date house listings as soon as they arrive!","archived":false,"fork":false,"pushed_at":"2025-03-01T07:19:45.000Z","size":899,"stargazers_count":25,"open_issues_count":4,"forks_count":6,"subscribers_count":3,"default_branch":"latest","last_synced_at":"2026-01-17T11:25:15.407Z","etag":null,"topics":["appartments","crawler","huurwoningen","nederland","python","real-estate-agencies","scrapy","scrapy-crawler"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nstapelbroek.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-11T20:33:49.000Z","updated_at":"2025-08-09T12:07:06.000Z","dependencies_parsed_at":"2023-09-25T08:32:39.650Z","dependency_job_id":"c80fd5c0-eda4-477e-9422-678383f0a0ae","html_url":"https://github.com/nstapelbroek/estate-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/nstapelbroek/estate-crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nstapelbroek%2Festate-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nstapelbroek%2Festate-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nstapelbroek%2Festate-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nstapelbroek%2Festate-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nstapelbroek","download_url":"https://codeload.github.com/nstapelbroek/estate-crawler/tar.gz/refs/heads/latest","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nstapelbroek%2Festate-crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28534156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appartments","crawler","huurwoningen","nederland","python","real-estate-agencies","scrapy","scrapy-crawler"],"created_at":"2026-01-17T00:39:52.577Z","updated_at":"2026-01-18T09:19:27.154Z","avatar_url":"https://github.com/nstapelbroek.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Estate Crawler\n\nFinding a place to live can be tough. If for any reason you are unable \nto buy a house, the search for a rental apartment can cost you a lot \nof time and money. Real estate agencies will often charge you before\nyou've even visited your first potential \"object\". \nThe subscriptions they force upon you are unfair and do not \nguarantee success of finding a place to live.\n\nThis project aims to relieve some pain by giving you insights of the \noffers at several agencies. By collecting and comparing this data, \nyou can pick an agency that is best suited for your needs. \nSaving you time and potentially money.\n\n# Usage\nYou can run the estate crawler by executing the crawler.py file. Below is an example with Python3 and pipenv installed on your OS:\n\n```bash\npipenv install \u0026\u0026 pipenv run ./crawler.py --region amsterdam,rotterdam,arnhem\n```\n\nIf you have the luxury of Docker available on your system, you can run the crawler without installing any dependencies like so:\n```bash\ndocker run --rm -v $(pwd):/app/build docker.io/nstapelbroek/estate-crawler --region amsterdam\n```\n\nResults of your crawl run are available in ./build/results.json after a successful run. You can change this path by passing\na `--output-file` argument. Note that the output file does not contain valid json, only valid json lines. ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnstapelbroek%2Festate-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnstapelbroek%2Festate-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnstapelbroek%2Festate-crawler/lists"}