{"id":15662344,"url":"https://github.com/weiji14/nz_convnet","last_synced_at":"2025-05-05T23:25:33.237Z","repository":{"id":96183614,"uuid":"121210111","full_name":"weiji14/nz_convnet","owner":"weiji14","description":"A U-net based ConvNet for NZ imagery to classify building outlines.","archived":false,"fork":false,"pushed_at":"2018-03-15T23:34:40.000Z","size":85944,"stargazers_count":19,"open_issues_count":0,"forks_count":6,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-31T00:41:18.431Z","etag":null,"topics":["aerial-imagery","big-data","binder","computer-vision","convolutional-neural-networks","deep-learning","jupyter-notebook","keras"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weiji14.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null},"funding":{"github":["weiji14"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2018-02-12T06:40:55.000Z","updated_at":"2024-08-19T15:39:29.000Z","dependencies_parsed_at":"2023-09-26T03:46:21.339Z","dependency_job_id":null,"html_url":"https://github.com/weiji14/nz_convnet","commit_stats":{"total_commits":10,"total_committers":1,"mean_commits":10.0,"dds":0.0,"last_synced_commit":"368619f8a81a8cf5c4cddabaf1f2d8455a8256bd"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weiji14%2Fnz_convnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weiji14%2Fnz_convnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weiji14%2Fnz_convnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weiji14%2Fnz_convnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weiji14","download_url":"https://codeload.github.com/weiji14/nz_convnet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252591783,"owners_count":21773156,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aerial-imagery","big-data","binder","computer-vision","convolutional-neural-networks","deep-learning","jupyter-notebook","keras"],"created_at":"2024-10-03T13:31:56.774Z","updated_at":"2025-05-05T23:25:33.214Z","avatar_url":"https://github.com/weiji14.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/weiji14"],"categories":[],"sub_categories":[],"readme":"Convolutional Neural Network (ConvNet) trained on open data from Land Information New Zealand (LINZ). Specifically, we train the ConvNet on aerial photography to detect building outlines.\n\n# Getting started\n\n## Quickstart\n\nLaunch Binder\n\n[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/weiji14/nz_convnet/master)\n\n## Installation\n\n    git clone https://github.com/weiji14/nz_convnet.git\n    cd nz_convnet\n    conda env create -f environment.yml\n\n## Running the jupyter notebook\n\n    source activate nz_convnet\n    python -m ipykernel install --user  #to install conda env properly\n    jupyter kernelspec list --json      #see if kernel is installed\n    jupyter notebook\n\n## Where the data goes\n\n|Folder                 | Example of a file inside                         | extension | Notes          |\n|:--------------------- |:------------------------------------------------ |:---------:|:-------------- |\n| data/vector           | nz-building-outlines-pilot.shp                   | \\*.shp    | see section Training data/Mask |\n| data/raster/downloads | lds-tile-2015-bk39-5000-0401-rgb-GTiff.zip       | \\*.zip    | see section Training data/Images  |\n| data/raster           | 2015_BK39_5000_0401_RGB.tif                      | \\*.tif    | unzipped files from data/raster/downloads |\n| data/train            | X_2015_BK39_5000_0401_RGB.hdf5                   | \\*.hdf5   | binary of tif file to load into numpy array |\n| data/test             | wellington-03m-rural-aerial-photos-2012-2013.tif | \\*.tif    | unzipped files similar to those in data/raster |\n\n## Prediction\n\n### In near realtime!\n\n    source activate nz_convnet\n    python predict.py\n    \n    #You can also set 2 integer parameters:\n    #  1st argument - output pixel size e.g. 256, 512, 1024 (default: 256)\n    #  2nd argument - prediction threshold e.g from least accurate 0% accept anything to 100% won't output much (default: 50)\n    python predict.py 512 50 \n\nLive testing on imagery of Karori, Wellington.\n\n![livesample1](https://user-images.githubusercontent.com/23487320/36468063-aed6c1bc-1746-11e8-8337-51a6a62ec796.gif)\n\n### To a raster geotiff (which you can vectorize to a polygon)\n\nOnce you clone the repository, open the [jupyter notebook](nz_convnet.ipynb) and follow the instructions to run 'Part 5 - Save Results'.\nYou will need to have some geotiffs inside the data/test folder, and you may need to tweak the (img_height, img_width) parameter.\n\nThere might need to be some fiddling on your part to get this parameter right, so that the input RGB image will be tiled perfectly.\nThe algorithm will create a prediction on each tile, and join in back together, so if it is not tiled perfectly due to the user setting, it will raise an error.\n\nBelow is a visualization in [QGIS 3.0](https://qgis.org/) of a sample test image and the predicted raster mask output\n\n![qgissample1](https://user-images.githubusercontent.com/23487320/37496053-2e7a79ce-2915-11e8-9732-fd27592ba237.gif)\n\nMask was styled using singleband pseudocolor, linear interpolation, with the OrRd color ramp in equal-interval mode.\nOpacity set to 0% for values 0, 0.25 and 0.5, and 50% for values 0.75 and 1.0.\nGIF was recorded using [Peek](https://github.com/phw/peek)\n\n### More output examples\n\nSample outputs on cross validation dataset plotted with matplotlib inside the [jupyter notebook](nz_convnet.ipynb) environment.\nLeft is input RGB image, Middle is ConvNet model output, Right is the Mask.\n\n![sample1](https://user-images.githubusercontent.com/23487320/36362177-17747d88-1597-11e8-8c17-167b8037cb71.png)\n![sample2](https://user-images.githubusercontent.com/23487320/36362245-9dd6fa04-1597-11e8-959b-87ed3217e131.png)\n![sample3](https://user-images.githubusercontent.com/23487320/36362261-bfc48046-1597-11e8-81c9-c4139569cde0.png)\n\n\n\n# Data sources used to train the [keras](https://github.com/keras-team/keras) model\n\nUsing freely available data from [LINZ Data Service](https://data.linz.govt.nz/). As there is a 3.5GB limit, we resort to using region crops using the 'Set a crop' tool on the top right. Not ideal but it ensures a little bit of reproducibility.\n\n## Training data\n\n### Images\n\n|Region Crop Type                       |Region Name                          |LINZ Data Source|\n| ------------------------------------- |:-----------------------------------:| --------------:|\n| General Electorate Boundaries 2014    | Wigram                              | [Canterbury 0.3m Rural Aerial Photos (2015-16)](https://data.linz.govt.nz/layer/53519-canterbury-03m-rural-aerial-photos-2015-16/) |\n| Manual Tile Selection\\*               | Hastings 2015_BK39_5000_{XXXX}_RGB  | [0401](https://data.linz.govt.nz/x/vnGVkg) [0402](https://data.linz.govt.nz/x/aA5XSv) [0403](https://data.linz.govt.nz/x/DYsY9B) [0404](https://data.linz.govt.nz/x/qvgapR) [0405](https://data.linz.govt.nz/x/VKVcWf) [0501](https://data.linz.govt.nz/x/8hJeCu) [0502](https://data.linz.govt.nz/x/k57ftA) [0503](https://data.linz.govt.nz/x/QTuhaQ) [0504](https://data.linz.govt.nz/x/3qijGe) [0505](https://data.linz.govt.nz/x/gEXkwt) [0601](https://data.linz.govt.nz/x/KcLnd9) [0602](https://data.linz.govt.nz/x/wy9pLP) [0603](https://data.linz.govt.nz/x/bNwq2d) [0604](https://data.linz.govt.nz/x/Ekkshs) [0605](https://data.linz.govt.nz/x/r9ZuP8) |\n| Manual Tile Selection\\*               | Tuakau bb32_{XXXX}                  | [4630](https://data.linz.govt.nz/x/9s9M9A) [4631](https://data.linz.govt.nz/x/nGwPpQ) [4632](https://data.linz.govt.nz/x/RekRWe) [4633](https://data.linz.govt.nz/x/43ZTCt) [4634](https://data.linz.govt.nz/x/hRNUs9) [4635](https://data.linz.govt.nz/x/LoBWaP) [4636](https://data.linz.govt.nz/x/yByYGd) [4637](https://data.linz.govt.nz/x/Fwbbd8) [4638](https://data.linz.govt.nz/x/tLQdLN) [4639](https://data.linz.govt.nz/x/XiDe2c) [4730](https://data.linz.govt.nz/x/oUpiP7) [4731](https://data.linz.govt.nz/x/Srdj6M) [4732](https://data.linz.govt.nz/x/6FSmmb) [4733](https://data.linz.govt.nz/x/yAvvdB) [4734](https://data.linz.govt.nz/x/idFoTq) [4735](https://data.linz.govt.nz/x/Mz4p96) [4736](https://data.linz.govt.nz/x/zPrrqL) [4737](https://data.linz.govt.nz/x/dmftXa) [4738](https://data.linz.govt.nz/x/HAUvDp) [4739](https://data.linz.govt.nz/x/uYHwt5) [4830](https://data.linz.govt.nz/x/pgh3xo) [4831](https://data.linz.govt.nz/x/T5W5e4) [4832](https://data.linz.govt.nz/x/7TK7MJ) [4833](https://data.linz.govt.nz/x/jp883Y) [4834](https://data.linz.govt.nz/x/PDwAin) [4835](https://data.linz.govt.nz/x/2bkCQ3) [4836](https://data.linz.govt.nz/x/eyZD7H) [4837](https://data.linz.govt.nz/x/JNNFnX) [4838](https://data.linz.govt.nz/x/vkBHUm) [4839](https://data.linz.govt.nz/x/Z8yKA2) [4930](https://data.linz.govt.nz/x/DWnLrG) [4931](https://data.linz.govt.nz/x/qtbNYW) [4932](https://data.linz.govt.nz/x/VHQQEk) [4933](https://data.linz.govt.nz/x/8fDRuz) [4934](https://data.linz.govt.nz/x/k32TcF) [4935](https://data.linz.govt.nz/x/QRpVJV) [4936](https://data.linz.govt.nz/x/3odWyj) [4937](https://data.linz.govt.nz/x/gCSYfy) [4938](https://data.linz.govt.nz/x/KaFaNE) [4939](https://data.linz.govt.nz/x/ww4b4U) [5030](https://data.linz.govt.nz/x/bLrdji) [5031](https://data.linz.govt.nz/x/EiffRx) [5032](https://data.linz.govt.nz/x/r7Ug8D) [5033](https://data.linz.govt.nz/x/WVHioT) [5034](https://data.linz.govt.nz/x/9r6kVh) [5035](https://data.linz.govt.nz/x/nFtnBw) [5036](https://data.linz.govt.nz/x/RdhosC) [5037](https://data.linz.govt.nz/x/42WqZS) [5038](https://data.linz.govt.nz/x/hQKsFg) [5039](https://data.linz.govt.nz/x/Lm8tvv) \n\n* Manual tile selection selects tiles manually from the tiles table, e.g. [here](https://data.linz.govt.nz/layer/53401-hawkes-bay-03m-rural-aerial-photos-2014-15/data/)\n\n### Mask\n\n- [NZ Building Outlines (Pilot)](https://data.linz.govt.nz/layer/53413-nz-building-outlines-pilot/) in shapefile format.\n\n## Test data\n\n|Region Crop Type                       |Region Name             |LINZ Data Source|\n| ------------------------------------- |:----------------------:| --------------:|\n| NZ Topo 50 Map Sheets                 | BP31 - Porirua         | [Wellington 0.3m Rural Aerial Photos (2012-2013)](https://data.linz.govt.nz/layer/51870-wellington-03m-rural-aerial-photos-2012-2013/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweiji14%2Fnz_convnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweiji14%2Fnz_convnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweiji14%2Fnz_convnet/lists"}