{"id":22538200,"url":"https://github.com/cfpb/mapusaurus","last_synced_at":"2025-04-09T20:15:36.074Z","repository":{"id":17095274,"uuid":"19860601","full_name":"cfpb/mapusaurus","owner":"cfpb","description":"Real-time geospatial mapping of HMDA data. This project is being deprecated as HMDA data schema are changing in calendar year 2019","archived":false,"fork":false,"pushed_at":"2018-12-20T19:50:31.000Z","size":5743,"stargazers_count":12,"open_issues_count":5,"forks_count":23,"subscribers_count":23,"default_branch":"master","last_synced_at":"2025-04-09T20:15:29.854Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cfpb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-05-16T15:06:07.000Z","updated_at":"2022-04-26T04:48:44.000Z","dependencies_parsed_at":"2022-09-23T07:41:25.754Z","dependency_job_id":null,"html_url":"https://github.com/cfpb/mapusaurus","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fmapusaurus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fmapusaurus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fmapusaurus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fmapusaurus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cfpb","download_url":"https://codeload.github.com/cfpb/mapusaurus/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248103872,"owners_count":21048245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-07T11:10:46.063Z","updated_at":"2025-04-09T20:15:36.049Z","avatar_url":"https://github.com/cfpb.png","language":"JavaScript","readme":"Mapusaurus [![Build Status](https://travis-ci.org/cfpb/mapusaurus.png)](https://travis-ci.org/cfpb/mapusaurus) [![Coverage Status](https://coveralls.io/repos/cfpb/mapusaurus/badge.svg)](https://coveralls.io/r/cfpb/mapusaurus)\n\n![Mapusaurus screenshot](screenshot.png)\n\n\n## Description\n\nThis repository provides data and scripts to set up an API endpoint for serving Home Mortgage Disclosure Act data as well as front-end and back-end application components that feed off this data.\nFinancial institution data is loaded from raw HMDA files and welded to National Information Center data to allow for more robust analysis in the front-end application.\n\nThe Mapusaurus back-end is a Python/Django application. Additional requirements are defined below.\n\n\n## Data\n\nThe data you can load is:\n\n* HMDA Transmittal Sheet\n* HMDA Reporter Panel\n\nBoth are available from the FFIEC.\n\nHere are the 2013 files:\n\nTransmittal sheet:\nhttp://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAInstitutionRecords.zip\n\nReporter panel:\nhttp://www.ffiec.gov/hmdarawdata/OTHER/2013HMDAReporterPanel.zip\n\n## Requirements\n\nThis currently uses:\nDjango 1.7\nPython 2.7.x\n\nYou will also need:\nPostgreSQL 9.3\nPostGIS 2.1.x\nElasticSearch\n\nThere's also a requirements.txt file in the repository root directory that can be installed with pip.\n\n\n## Loading the data\n\n\nTo create the tables, you need to run:\n\n```\n    python manage.py migrate respondents\n```\n\nThere's also a fixture that you need to load some information from:\n\n```\n    python manage.py loaddata agency\n```\n\nThis loads static regulator agency data.\n\nDownload the two transmittal sheet and reporter panel flat files.\n\nThere are two management commands that will load data, and need to be run\nin the following order:\n\n```\n1. python manage.py load_transmittal \u003cpath/to/transmittal_sheet\u003e\n2. python manage.py load_reporter_panel \u003cpath/to/reporter_panel\u003e\n```\n\n\n## GEO\n\nThe 'geo' application requires GeoDjango and PostGIS. Follow the instructions\nfor installing GeoDjango.\n\nHere are some separate instructions for running the geo application.\n\n```\n    python manage.py migrate geo\n```\n\nCurrently, we load census tract, county, CBSA, and metropolitan division files.\nYou can download them from the census.gov FTP site:\n\n```\nftp://ftp2.census.gov/geo/tiger/TIGER2013/TRACT/\nftp://ftp2.census.gov/geo/tiger/TIGER2013/COUNTY/\nftp://ftp2.census.gov/geo/tiger/TIGER2013/CBSA/\nftp://ftp2.census.gov/geo/tiger/TIGER2013/METDIV/\n```\n\nThis is how you load the data:\n\n```\n    # This example only loads census tracts from IL (FIPS code: 17); repeat\n    # for other states as needed\n    python manage.py load_geos_from /path/to/tl_2013_17_tract.shp\n    python manage.py load_geos_from /path/to/tl_2013_us_county.shp\n    python manage.py load_geos_from /path/to/tl_2013_us_cbsa.shp\n    python manage.py load_geos_from /path/to/tl_2013_us_metdiv.shp\n```\n\nThese import scripts are set up to update geos in place -- no need to delete\nrecords manually.\n\nOnce census tracts and counties are loaded, run the following command to\nassociate census tracts with their CBSAs.\n\n```\n    python manage.py set_tract_csa_cbsa\n```\n\n\n## Census Data\n\nThe 'censusdata' app loads census data to the census tracts found in the 'geo'\napplication. As such, 'censusdata' relies on 'geo'.\n\nFirst, run migrate to create the appropriate tables\n\n```\n    python manage.py migrate censusdata\n```\n\nYou'll then want to import census data related to the tracts you've loaded\nwhile setting up the 'geo' app. Go to\n```\nhttp://www2.census.gov/census_2010/04-Summary_File_1/\n```\nand select the state you care about. Download the associated `*.sf1.zip` file,\nwhich you should then unzip.\n\nLoading the data looks like this:\n```\n    python manage.py load_summary_one /path/to/XXgeo2010.sf1\n```\n\nWarning: currently, data will not be updated in place; to re-import, you'll\nneed to delete everything from the `censusdata_census2010*` tables.\n\n\n## HMDA Data\n\nThe 'hmda' app loads HMDA data to the census tracts found in the 'geo'\napplication. As such, 'hmda' relies on 'geo'. In fact, 'hmda' will only store\ndata for states that are loaded via the 'geo' app.\n\nFirst, run migrate to create the appropriate tables\n\n```\n    python manage.py migrate hmda\n```\n\nNext, download a flat file representing all of the HMDA LAR data:\n```\nhttp://www.ffiec.gov/hmda/hmdaflat.htm\n```\nand download the zip file. Unzip it and then:\n```\n    python manage.py load_hmda /path/to/2013HMDALAR\\ -\\ National.csv\n```\n\nNote that this process takes several minutes (though you will receive progress\nnotifications). This import can be run repeatedly (if additional geos are\nadded later, for example).\n\nWarning: At the moment, the import assumes a single year of information.\nThat's a todo.\n\nAlternatively, the load_hmda script can read a directory of CSV files and load them one by one.\nThere is also the option of removing these files after they are processed.\n\n```\n    split -l 50000 -d  \"/path/to/2013HMDALAR\\ -\\ National.csv\" hmda_csv_\n    python manage.py load_hmda /path/to/2013HMDALAR/  delete_file:true\n```\n\n\nYou will most likely want to pre-calculate the median number of loans for a\nparticular lender X city pair -- this speeds up map loading quite a bit.\n\n```\n    python manage.py calculate_loan_stats\n```\n\n\n## Styles\n\nWhile the base application attempts to appear \"acceptable\", you will likely\nwish to provide your own icons, colors, etc. We provide an example app\n(`basestyle`) which you can modify directly or copy into a separate Django\napp. If you go the latter route, remember to activate your new app and\ndeactivate the basestyle.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Fmapusaurus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcfpb%2Fmapusaurus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Fmapusaurus/lists"}