{"id":13608204,"url":"https://github.com/cfpb/api","last_synced_at":"2025-04-09T20:15:28.333Z","repository":{"id":10160655,"uuid":"12241595","full_name":"cfpb/api","owner":"cfpb","description":"Documentation to support upcoming data platform API and data sets","archived":false,"fork":false,"pushed_at":"2023-05-09T17:04:19.000Z","size":6930,"stargazers_count":31,"open_issues_count":17,"forks_count":78,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-04-09T20:15:22.916Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://cfpb.github.io/api/hmda/","language":"CSS","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"tensorflow/tensorflow","license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cfpb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-08-20T11:33:30.000Z","updated_at":"2025-03-18T17:00:24.000Z","dependencies_parsed_at":"2024-01-19T10:12:14.439Z","dependency_job_id":"b5e149e0-1ee0-4461-879e-587dd8a7f806","html_url":"https://github.com/cfpb/api","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fapi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fapi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fapi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cfpb%2Fapi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cfpb","download_url":"https://codeload.github.com/cfpb/api/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248103872,"owners_count":21048245,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T19:01:25.171Z","updated_at":"2025-04-09T20:15:28.308Z","avatar_url":"https://github.com/cfpb.png","language":"CSS","funding_links":[],"categories":["CSS"],"sub_categories":[],"readme":"CFPB Public Data API\n====================\n\nThis is the source project for the CFPB public data API at http://api.consumerfinance.gov. It contains up-to-date load scripts for populating data in that API. Note, however, that the `resources/static/` directory is a work in progress; refer to https://github.com/cfpb/qu/tree/master/resources/static for the current static resources.\n\n## Create a config file\n\n```sh\ncp sample_config.edn config.edn\n```\n\nEdit that file with appropriate values, including MongoDB connection information.\n\n## Loading data\n\nTo load the HMDA dataset, run `lein repl` and enter the following:\n\n```clj\n(-main \"config.edn\") ;; use the name of your config file. config.edn is an example\n(require 'qu.loader)\n(in-ns 'qu.loader)\n(load-dataset \"hmda\")\n```\nThen close your REPL, using `Ctrl-D`.\nThis does start the web server, so you may want to use a different configuration that starts it on an unpublished port.\n\n## Speeding up data load\nYou can take advantage of more CPU and Ram by concurrently loading chunks of data by splitting the large data file \ninto smaller files with 1 million records each. \n\n```./hmda_split_csv.sh hmda_lar_all_2012.csv split_hmda_2012_``` \n\nThen update the definition.json file to load these split files instead of the single CSV. Edit the hmda_lar sources \nsection and replace it with a list of all the split files for the year you're targeting. \n\nFor example, for 2012, it looks like this:\n```\n   \"sources\": [\n\"split_hmda_2012_aa\",\n\"split_hmda_2012_ab\",\n\"split_hmda_2012_ac\",\n\"split_hmda_2012_ad\",\n\"split_hmda_2012_ae\"\n]\n```\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Fapi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcfpb%2Fapi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcfpb%2Fapi/lists"}