{"id":17047704,"url":"https://github.com/residentmario/nyc-building-values","last_synced_at":"2026-02-13T19:32:25.173Z","repository":{"id":66134325,"uuid":"76480566","full_name":"ResidentMario/nyc-building-values","owner":"ResidentMario","description":null,"archived":false,"fork":false,"pushed_at":"2016-12-14T18:32:19.000Z","size":5382,"stargazers_count":4,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-22T21:32:20.025Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ResidentMario.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-14T17:10:51.000Z","updated_at":"2024-04-20T00:50:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"b81d7acd-128a-4e4f-bee2-e55baff1ffcc","html_url":"https://github.com/ResidentMario/nyc-building-values","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ResidentMario/nyc-building-values","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fnyc-building-values","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fnyc-building-values/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fnyc-building-values/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fnyc-building-values/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ResidentMario","download_url":"https://codeload.github.com/ResidentMario/nyc-building-values/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ResidentMario%2Fnyc-building-values/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29415603,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-13T06:24:03.484Z","status":"ssl_error","status_checked_at":"2026-02-13T06:23:12.830Z","response_time":78,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T09:50:06.331Z","updated_at":"2026-02-13T19:32:25.153Z","avatar_url":"https://github.com/ResidentMario.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NYC Building Values\n\n![splash](./figures/splash.png)\n\n## Background\n\nOne of the most hotly requested of the datasets that New York City maintains and releases is PLUTO – short for\n“Property Land Use Tax [Lot] Output” – a detailed record of every tract of land and every building standing on it in\nNew York City. The rows are unique tax lots, and each column describes an attribute — things like property value,\nnumber of buildings on the lot, square footage, and other information useful for tax assessment purposes. The\napplications of PLUTO (and its GIS shape-file companion, MapPLUTO) are huge, ranging from demographic studies to\ngovernment services mappings to real estate analysis to building age visualizations. We intend to focus on real\nestate analysis from the perspective of a land-user, e.g. the perspective of someone looking to own a house,\napartment, or other piece of property in New York City.\n\nThe PLUTO dataset does not directly embed any information on price. Luckily we have another data source: the NYC\nDepartment of Finance’s Rolling Sales dataset, which provides a rolling record of all real estate sales by price\nwhich occurred in the last 12 months. By merging this financial data with the PLUTO data we can set ground truths\nabout the value of specific homes in New York City, using which we can interpolate the values of residential\nbuildings not in the sales record, and, going one step further, predict the values of future homes to be built in\nthe city based on their characteristics with respect to the dataset.\n\nThis project additionally examines a third dataset, the New York City Department of Finance's RPAD dataset, which\ncontains information about the government-assessed financial values of units of property in the city. We are\ninterested in knowing how accurate these estimates are and how they gel when combined with a model based on other\nvariables.\n\n## Contents\n\nIn this project I built a linear regression classifier which ingests variables relating to a building unit (size,\ngeography, zoning class, and so on) and outputs an estimate of that unit's market value, based on a public-record\nhistory of real estate sales in New York City from within the lats year. The files constituting the project are\ncontained in `/notebooks`.\n\n## Cleaning\n\nBuilding a model based on this data ran into two major complexities. The first one is that we are provided with\ninformation about the sale value of a property, which we need to standardize using some metric in order to model\nsuccessfully. The obvious metric in this case is market value per square foot. Rolling sales data contains information\nabout the square footage of the property being sold\u0026mdash;**but it does not provide square footage for apartment\nsales**. This makes apartment sales impossible to standardize, and they had to be removed from the dataset entirely.\n\nThe second major complexity is that the rolling sales record contains many sales which are not truly \"sales\", in that\nthey are either actually 0\\$ or nominal-dollar **property transfers** (if you inherit a home from your parents, for\nexample, it still gets recorded in the property record as technically a sale), *or* they are **non-market sales** for\nbelow-market prices where the rest of the value of the property is made up by some kind of out-of-cash business\ncontract (an example would be when the city cits an in rem property to a private developer for next-to-nothing in\nreturn for very stingent requirements on what the developer can, cannot, and should do).\n\nAddressing the combination of these concerns required (or rather, it seemed prudent to require) culling out\napproximately 25% of non-apartment records in the sales record, in addition to all apartment sales. This resulted in\na dataset footprint that's heavily biased towards lower-value sectors of the outer boroughs:\n\n![splash](./figures/observations-by-zip.png)\n\nData points were confined to a very limited \"window\" of presumed accuracy, based on their colocation\nwith the estimated market values assigned in the RPAD data:\n\n![splash](./figures/window.png)\n\nThis is the data used to build our model.\n\n## Modelling\n\nA simple linear classifier was used. The dataset was modeled in two different configurations: one excluding the\ncity's value assessment data and one including it. These modeled had r^2 values of ~75% and ~80%, respectively. Here\nis a plot of the better of the two model's MSE performance; the band corresponds with one standard deviation worth of\nregression error:\n\n![splash](./figures/model-error-best.png)\n\nUnsurpisingly, most of the variance in the model was explained by geography, with certain effects, like zoning,\nhaving a significantly weaker effect than I had expected. \"Scale\" variables from the PLUTO dataset describing\nproperties of the *building*, as opposed to the *property unit* (a building might have many units; a sale might be\nfor all of them or just for some of them), had next-to-no effect in the model, and could be safely ignored.\n\nThe small size of the improvement of the model (5%) when adding in the city's assessment data was very surprising. It\nindicates that far more of the variation in price can be explained with simple variables like geography than I had\ninitially expected, and that, furthermore, the city's value assessments, while containing price information, don't\ncontain *as much more* of it respective to simple classification as I had expected.\n\nIt is unlikely that chosing a different classifier will improve performance (with the possible exception of\ngeospatial linear regression). This is because basically all of the important variables for the model turned out to\nbe categorical variables, which are exceptionally well-handled by a linear model; and because the R^2 is already very\nhigh, given the complexity of what we are trying to measure.\n\nWhile the high r^2 values and fit statistics are encouraging, the extreme lengths we had to go to in order to get data\nthat we are reasonably sure can be reliably fitted by a model in the first place make it difficult to extend the\nmodel beyond the dataset we generated here. Having apartment sales' square footage data would allow greatly expanding\nthe \"reach\" of the model, and working on acquiring that would be a logical next step to take.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresidentmario%2Fnyc-building-values","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fresidentmario%2Fnyc-building-values","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fresidentmario%2Fnyc-building-values/lists"}