{"id":51001567,"url":"https://github.com/arberx/streeteasy-grep","last_synced_at":"2026-06-20T14:33:54.542Z","repository":{"id":47736678,"uuid":"396147918","full_name":"arberx/streeteasy-grep","owner":"arberx","description":"Search, store, and diff streeteasy.com rental queries programmatically with selenium.","archived":false,"fork":false,"pushed_at":"2023-10-17T22:45:03.000Z","size":30,"stargazers_count":0,"open_issues_count":4,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-03-17T10:00:52.582Z","etag":null,"topics":["diff","grep","parse","selenium","streeteasy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arberx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-14T22:31:49.000Z","updated_at":"2024-03-17T10:00:52.582Z","dependencies_parsed_at":"2022-07-26T17:45:19.102Z","dependency_job_id":null,"html_url":"https://github.com/arberx/streeteasy-grep","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arberx/streeteasy-grep","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arberx%2Fstreeteasy-grep","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arberx%2Fstreeteasy-grep/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arberx%2Fstreeteasy-grep/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arberx%2Fstreeteasy-grep/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arberx","download_url":"https://codeload.github.com/arberx/streeteasy-grep/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arberx%2Fstreeteasy-grep/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34573862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["diff","grep","parse","selenium","streeteasy"],"created_at":"2026-06-20T14:33:54.434Z","updated_at":"2026-06-20T14:33:54.526Z","avatar_url":"https://github.com/arberx.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## streeteasy-grep\n\nQuery [streeteasy.com](https://www.streeteasy.com) rentals and get the results in json format. Tool also always you to diff between different invocations of the same query.\n\n### Motivation\n\n`streeteasy` currently limits bots and scrapers from accessing the site programmitically with rate-limiting and captcha features.\n\nUsing [selenium](https://selenium-python.readthedocs.io/), we're able to replicate non-bot behavior and therefore allowing us to parse the site.\n\n**Note: Even this method gets rate limited sometimes.**\n### Installation\n\nFirst install the Chromium drivers locally (the directions can be found [here](https://selenium-python.readthedocs.io/)), and have a chromium based browser installed. Any browser can be used, but requires changing the driver setup in `parser.py`.\n\nSimple install on Debian based systems:\n\n```\napt-get install -y chromium-chromedriver\n```\n\nIt currently relies on [poetry](https://python-poetry.org/) for installation and dependency management. Follow the `poetry` docs for installation.\n\nOnce `poetry` is installed run:\n\n```\npoetry install\n```\n\nand\n\n```\npoetry shell\n```\n\nShould now be able to run the command `streeteasy-grep --help` in a console.\n\n### Usage\n\n```\nusage: streeteasy-grep [-h] [--location LOCATION] [--price-lower-bound PRICE_LOWER_BOUND] [--price-upper-bound PRICE_UPPER_BOUND] [--num-bedrooms NUM_BEDROOMS] [--has-fee]\n                       [--check-diff] [--num-pages NUM_PAGES]\n\nParse streeteasy rental results for a given query with given parameters.\n\noptional arguments:\n  -h, --help            show this help message and exit\n  --location LOCATION, -l LOCATION\n                        Location to search for. (default: ues)\n  --price-lower-bound PRICE_LOWER_BOUND, -pl PRICE_LOWER_BOUND\n                        Lower bound of rental price (default: 0)\n  --price-upper-bound PRICE_UPPER_BOUND, -pu PRICE_UPPER_BOUND\n                        Upper bound of rental price (default: 3000)\n  --num-bedrooms NUM_BEDROOMS, -nb NUM_BEDROOMS\n                        Number of bedrooms (default: 1)\n  --has-fee, -hf        Include apartments that have a signing fee. (default: False)\n  --check-diff, -cd     Check diff between queries if the same query file has been created before (default: False)\n  --num-pages NUM_PAGES, -np NUM_PAGES\n                        Number of pages to iterate through. (default: 1)\n  ```\n\n### Sample Usage\n\nBy default running `streeteasy-grep` will generate the query parameters:\n\n```\nues/price:0-3000|beds:1|no_fee|1\n```\n\nTranslating to querying for a 1 bedroom apartment on the Upper East Side, between $0-$3000/month, only including listing which are `no fee`, and only looking at the first page of results.\n\nThe results will be stored in a file called `results-{hash}.json` where `hash` is the hashed query string.\n\nIt'll contain a dictionary with the `links` keys, pointing to a object containing `address` and `price`:\n\n```\n{\"https://streeteasy.com/building/1600-3-avenue-new_york/5c\": {\"Address\": \"174 East 90th Street #5C\", \"Price\": \"$2,100\"}\n```\n\n### Checking diff\n\nThe tool is useful in checking for new listings using a previous invocation with **the same query**. Simply specifying:\n\n```\nstreeteasy-grep --check-diff\n```\n\nWill diff the new query contents with an existing json file (in the directory the tool is invoked) and print to `stdout`.\n\n### Motivation\n\nThe goal is you can setup a automated cron that runs the queries, then runs `--check-diff` and get immediate notifications if a new apartment is listed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farberx%2Fstreeteasy-grep","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farberx%2Fstreeteasy-grep","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farberx%2Fstreeteasy-grep/lists"}