{"id":24590904,"url":"https://github.com/blmoore/summerdatachallenge","last_synced_at":"2025-04-30T08:10:20.350Z","repository":{"id":20302066,"uuid":"23575819","full_name":"blmoore/summerdatachallenge","owner":"blmoore","description":"My entry for: http://summerdatachallenge.com (I came 3rd)","archived":false,"fork":false,"pushed_at":"2014-11-07T12:20:37.000Z","size":403751,"stargazers_count":5,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-30T14:11:37.251Z","etag":null,"topics":["analytics","data-science","london","r","real-estate","rstats"],"latest_commit_sha":null,"homepage":"http://blm.io/datarea","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"seven1m/30-days-of-elixir","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blmoore.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-09-02T11:17:47.000Z","updated_at":"2019-09-05T00:26:46.000Z","dependencies_parsed_at":"2022-09-07T02:21:46.408Z","dependency_job_id":null,"html_url":"https://github.com/blmoore/summerdatachallenge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blmoore%2Fsummerdatachallenge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blmoore%2Fsummerdatachallenge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blmoore%2Fsummerdatachallenge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blmoore%2Fsummerdatachallenge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blmoore","download_url":"https://codeload.github.com/blmoore/summerdatachallenge/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251666283,"owners_count":21624293,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","data-science","london","r","real-estate","rstats"],"created_at":"2025-01-24T09:26:44.733Z","updated_at":"2025-04-30T08:10:20.330Z","avatar_url":"https://github.com/blmoore.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n\u003cimg src=\"writeup/images/logo3_hires.png\" /\u003e\u003cbr /\u003e\n\u003cimg src=\"writeup/images/logotext.png\" width=\"200\" /\u003e\n\u003c/p\u003e\n\nThis repo containts my entry for Imperial create lab's [summerdatachallenge](http://summerdatachallenge.com). The challenge was to apply data science techniques to one or more of their supplied datasets. I chose **London house prices**, listings for all house sales within 100 km of the City of London from 2009 up to 2014.\n\nA short written report can be found under [`report/sdc_report.pdf`](report/sdc_report.pdf) and there's an accompanying site at [blm.io/datarea](http://blm.io/datarea).\n\n## How to run\n\nClone the github repository or download as a zip archive then run as described below. Note that the competition data is **not** included in this repository due to its terms of use, so to run these analyses you must first place the file `Houseprice_2009_100km_London.csv` (137 MB) in directory `houseprices/`. Scripts are all in the `R` directory, so can then be run with e.g. `Rscript R/fractal_context.R`, but are best played with interactively through an R IDE such as [RStudio](http://www.rstudio.com/).\n\nThe main scripts are briefly described here, more information is available in source code comments:\n\n* `fractal_context.R` generates a series of visualisations (namely `plots/FC*`) that relate a specific area to its neighbouring sector, district and area in terms of, is it the most or least expensive in a given locale? An outlier? Unexpectedly underpriced? Figures FC0-4 were combined for the final report using inkscape.\n* `arima_model.R` — after some background work, fits AR|I|MA models to house price time series and plots the forecast of a given sector ([`plots/forecast.pdf`](plots/forecast.pdf)) as well as a random selection for comparison ([`plots/grid_forecasts.svg`](plots/grid_forecasts.svg)).\n* `investment_grade.R` — fits ARIMA models to all sectors in dataset (2500+?) and calculates historical volatility to be combined into an investment grade. Saves the top 5 sectors ([`plots/top5_investments.svg`](plots/top5_investments.svg)) and a summary dataframe R object ([`rds/invest_grade.rds`](rds/invest_grade.rds)).\n\nOther more minor scripts include:\n* `postcode_map.R` — draws a series of monthly png bitmap images then stitches them together into animated gifs (via ImageMagick commandline) to show the entire dataset of house sales over time.\n* `interactive.R` — builds the basic javascript plots using rCharts (and dimple.js) of investment grading used online.\n* `report_viz.R`— just draws the introductory overview map ([`plots/report_overview.pdf`](plots/report_overview.pdf))for the written report.\n* `gmap.R` — outputs a csv ([`gmap/fusion_kml.csv`](gmap/fusion_kml.csv)) for use with fusion tables and the Google Maps API in order to build the interactive map overlay shown in the online report.\n\nThe directory `wip/` contains work in progress scripts or analyses that didn't make the final report. `writeup/` contains a version of the online report (current version at: [blm.io/datarea](http://blm.io/datarea)) and `report/` contains the LaTeX written report.\n\n## sessionInfo()\n\nBelow is the output of `sessionInfo()` which shows loaded package versions, the OS and R version (3.1.1) under which these scripts were written. For CRAN snapshots, these analyses were performed around mid October 2014.\n\n```\nR version 3.1.1 (2014-07-10)\nPlatform: x86_64-apple-darwin13.1.0 (64-bit)\n\nlocale:\n[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8\n\nattached base packages:\n[1] grid      stats     graphics  grDevices utils     datasets  methods   base     \n\nother attached packages:\n [1] rgeos_0.3-6      maptools_0.8-30  mapdata_2.2-3    maps_2.3-9       gpclib_1.5-5    \n [6] gridExtra_0.9.1  ggplot2_1.0.0    forecast_5.6     timeDate_3010.98 zoo_1.7-11      \n[11] dplyr_0.3.0.2    sp_1.0-15       \n\nloaded via a namespace (and not attached):\n [1] assertthat_0.1   codetools_0.2-9  colorspace_1.2-4 DBI_0.3.1        digest_0.6.4    \n [6] foreign_0.8-61   fracdiff_1.4-2   gtable_0.1.2     lattice_0.20-29  magrittr_1.0.1  \n[11] MASS_7.3-35      munsell_0.4.2    nnet_7.3-8       parallel_3.1.1   plyr_1.8.1      \n[16] proto_0.3-10     quadprog_1.5-5   rCharts_0.4.5    Rcpp_0.11.3      reshape2_1.4    \n[21] RJSONIO_1.3-0    scales_0.2.4     stringr_0.6.2    tools_3.1.1      tseries_0.10-32 \n[26] whisker_0.3-2    yaml_2.1.13 \n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblmoore%2Fsummerdatachallenge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblmoore%2Fsummerdatachallenge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblmoore%2Fsummerdatachallenge/lists"}