{"id":16691159,"url":"https://github.com/daniel-j-h/nedem","last_synced_at":"2025-04-05T03:24:41.120Z","repository":{"id":89361068,"uuid":"470266877","full_name":"daniel-j-h/nedem","owner":"daniel-j-h","description":"Neural implicit digital elevation model","archived":false,"fork":false,"pushed_at":"2022-05-25T15:18:53.000Z","size":1246,"stargazers_count":1,"open_issues_count":10,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-01-13T03:54:45.696Z","etag":null,"topics":["dem","digital-elevation-model","flax","glo-30","jax","machine-learning","optax"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/daniel-j-h.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-03-15T17:34:44.000Z","updated_at":"2022-05-25T15:19:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"bcd80738-2704-46d1-9e11-d6e0f10b1e6d","html_url":"https://github.com/daniel-j-h/nedem","commit_stats":{"total_commits":7,"total_committers":1,"mean_commits":7.0,"dds":0.0,"last_synced_commit":"c18fed99687bea64ad4b29fadde122c0acef5b6c"},"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daniel-j-h%2Fnedem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daniel-j-h%2Fnedem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daniel-j-h%2Fnedem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/daniel-j-h%2Fnedem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/daniel-j-h","download_url":"https://codeload.github.com/daniel-j-h/nedem/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247282861,"owners_count":20913448,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dem","digital-elevation-model","flax","glo-30","jax","machine-learning","optax"],"created_at":"2024-10-12T16:07:05.894Z","updated_at":"2025-04-05T03:24:41.099Z","avatar_url":"https://github.com/daniel-j-h.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neural Implicit Digital Elevation Model\n\nWe fit a neural network on a digital elevation model; we can reconstruct the digital elevation model by using the learned weights to predict elevation for every coordinate.\n\nBenefits\n- Flexible up/down scaling of the model's parameters to increase/decrease details\n- Weighted sampling during training to focus on details in regions of interest\n\nExamples, with a very small neural net (~15k parameters), 2k iterations, and different [fourier feature](https://arxiv.org/abs/2006.10739) scales\n\n![](./assets/nedem.png)\n\nSee [this issue](https://github.com/daniel-j-h/nedem/issues/2) for the respective training videos.\n\nNote: this project is experimental and a playground for me to learn Jax/Flax, neural implicits, and fourier features.\n\n\n## Dataset\n\nWe use the [Copernicus GLO-30](https://registry.opendata.aws/copernicus-dem/) digital elevation model for experiments.\n\n    aws s3 cp s3://copernicus-dem-30m/Copernicus_DSM_COG_10_N46_00_E008_00_DEM/Copernicus_DSM_COG_10_N46_00_E008_00_DEM.tif . --no-sign-request\n\n\n## Quadkeys\n\nWe work with quadkey tifs as the unit of abstraction.\n\n    rio warp in.tif 3857.tif --dst-crs EPSG:3857\n    rio bounds 3857.tif | mercantile tiles 10 \u003e z10.txt\n\n    while read tile; do\n      rio clip 3857.tif $(mercantile quadkey \"$tile\").tif --with-complement --bounds \"$(mercantile shapes --extents --mercator \"$tile\")\"\n    done \u003c z10.txt\n\nThis will warp and cut the raster tifs into quadkeys like `1202213002.tif`.\n\n\n## Training\n\nYou will need docker, docker-compose, and the nvidia-docker plugin set up to run training on a GPU.\n\nWe provide a self-contained and reproducible development environment with the correct drivers, versions, and dependencies.\n\n    make\n    make sh\n\nTo fit a neural net on the quadkey `1202213002.tif`, in the self-contained development environment run\n\n    ./main.py data/1202213002.tif\n\n\n## Hillshade\n\nThe digital elevation model is best visualized as a hillshaded tif; you can use `gdaldem` from `gdal-bin` for hill shading\n\n    for p in step-*.tif; do gdaldem hillshade $p -multidirectional hillshade-$p ; done\n\nThe model fitting happens over multiple steps; you can use `ffmpeg` to animate the hill shaded tifs\n\n    ffmpeg -y -loglevel error -r 15 -f image2 -pattern_type glob -i \"hillshade-step-*.tif\" -vf 'crop=w=512:h=512' -c:v libx264 -crf 23 -profile:v high -preset veryslow -pix_fmt yuv420p -movflags faststart hillshade-step.mp4\n\n\n## References\n\n- https://arxiv.org/abs/2009.09808\n- https://arxiv.org/abs/2006.10739\n- https://registry.opendata.aws/copernicus-dem/\n\n\n## License\n\nCopyright © 2022 Daniel J. Hofmann\n\nDistributed under the MIT License (MIT).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaniel-j-h%2Fnedem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdaniel-j-h%2Fnedem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdaniel-j-h%2Fnedem/lists"}