{"id":23796873,"url":"https://github.com/gher-uliege/DINCAE","last_synced_at":"2025-09-06T12:31:57.037Z","repository":{"id":39730270,"uuid":"193079989","full_name":"gher-uliege/DINCAE","owner":"gher-uliege","description":"DINCAE (Data-Interpolating Convolutional Auto-Encoder) is a neural network to reconstruct missing data in satellite observations.","archived":false,"fork":false,"pushed_at":"2025-04-24T08:01:27.000Z","size":214,"stargazers_count":51,"open_issues_count":4,"forks_count":20,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-07-09T04:06:52.604Z","etag":null,"topics":["interpolation","netcdf4","neural-network","oceanography","python","remote-sensing","satellite-observations","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gher-uliege.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-21T10:33:01.000Z","updated_at":"2025-05-13T08:16:41.000Z","dependencies_parsed_at":"2023-12-15T22:08:19.735Z","dependency_job_id":"7de10f7c-a7b8-4d04-b1e3-50313cd58a14","html_url":"https://github.com/gher-uliege/DINCAE","commit_stats":{"total_commits":157,"total_committers":2,"mean_commits":78.5,"dds":0.006369426751592355,"last_synced_commit":"a3d70ba43f985326055138dcddf724b7663f5416"},"previous_names":["gher-ulg/dincae"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/gher-uliege/DINCAE","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gher-uliege%2FDINCAE","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gher-uliege%2FDINCAE/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gher-uliege%2FDINCAE/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gher-uliege%2FDINCAE/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gher-uliege","download_url":"https://codeload.github.com/gher-uliege/DINCAE/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gher-uliege%2FDINCAE/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273904853,"owners_count":25188635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interpolation","netcdf4","neural-network","oceanography","python","remote-sensing","satellite-observations","tensorflow"],"created_at":"2025-01-01T20:11:50.833Z","updated_at":"2025-09-06T12:31:56.891Z","avatar_url":"https://github.com/gher-uliege.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Project Status: Unsupported – The project has reached a stable, usable state but the author(s) have ceased all work on it. A new maintainer may be desired.](https://www.repostatus.org/badges/latest/unsupported.svg)](https://www.repostatus.org/#unsupported)\n![GitHub last commit](https://img.shields.io/github/last-commit/gher-uliege/DINCAE)\n![GitHub top language](https://img.shields.io/github/languages/top/gher-uliege/DINCAE)\n![GitHub language count](https://img.shields.io/github/languages/count/gher-uliege/DINCAE)\n\n[![documentation latest](https://img.shields.io/badge/docs-latest-blue.svg)](https://gher-uliege.github.io/DINCAE/)\n[![DOI](https://zenodo.org/badge/193079989.svg)](https://zenodo.org/badge/latestdoi/193079989)\n[![codecov.io](http://codecov.io/github/gher-ulg/DINCAE/coverage.svg?branch=master)](http://codecov.io/github/gher-ulg/DINCAE?branch=master)\n\n# DINCAE\n\n\nDINCAE (Data-Interpolating Convolutional Auto-Encoder) is a neural network to\nreconstruct missing data in satellite observations which is described in the following open access paper:\nhttps://doi.org/10.5194/gmd-13-1609-2020\n\n\n*Note that this code is no longer maintained and has been superseeded by https://github.com/gher-ulg/DINCAE.jl*\n\n\n\n## Installation\n\nPython 3.6 or 3.7 with the modules:\n* numpy (https://docs.scipy.org/doc/numpy/user/install.html)\n* netCDF4 (https://unidata.github.io/netcdf4-python/netCDF4/index.html)\n* TensorFlow 1.15 with GPU support (https://www.tensorflow.org/install)\n\nTested versions:\n\n* Python 3.6.8\n* netcdf4 1.4.2\n* numpy 1.15.4\n* Tensorflow version 1.15 (DINCAE does not work with TensforFlow 2.0; TensorFlow 1.5 does not work on python 3.8)\n\nYou can install those packages either with `pip3` or with `conda`.\n\n\n## Documentation\n\nThe document is available at https://gher-uliege.github.io/DINCAE/.\n\n## Input format\n\nThe input data should be in netCDF with the variables:\n* `lon`: longitude (degrees East)\n* `lat`: latitude (degrees North)\n* `time`: time (days since 1900-01-01 00:00:00)\n* `mask`: boolean mask where true means the data location is valid\n* `SST` (or any other varbiable name): the data\n\n\nThis is the example output from `ncdump -h`:\n\n```\nnetcdf avhrr_sub_add_clouds {\ndimensions:\n\ttime = UNLIMITED ; // (5266 currently)\n\tlat = 112 ;\n\tlon = 112 ;\nvariables:\n\tdouble lon(lon) ;\n\tdouble lat(lat) ;\n\tdouble time(time) ;\n\t\ttime:units = \"days since 1900-01-01 00:00:00\" ;\n\tint mask(lat, lon) ;\n\tfloat SST(time, lat, lon) ;\n\t\tSST:_FillValue = -9999.f ;\n}\n```\n\nAn example for how to create this file in the examples folder:\n* [python example](https://github.com/gher-ulg/DINCAE/blob/master/examples/create\\_input\\_file.py)\n* [julia example](https://github.com/gher-ulg/DINCAE/blob/master/examples/create\\_input\\_file.jl)\n\n\n## Running DINCAE\n\nCopy the template file `run_DINCAE.py` and adapt the filename, variable name and the output directory and possibly optional arguments for the reconstruction method as mentioned in the [documentation](https://gher-ulg.github.io/DINCAE/).\nThe code can be run as follows:\n\n```bash\npython3 run_DINCAE.py\n```\n\nThe output NetCDF files are contain the variables:\n* `meandata` the time average of the input data used to compute the anomalies\n* `mean_rec` and `sigma_rec`: the mean and standard deviation of the Gaussian probability distribution function of the reconstruction.  \n\nIn [Barth et al., 2020](https://doi.org/10.5194/gmd-13-1609-2020) the best results were obtained by averaging all the NetCDF files.\n\n## Reducing GPU memory\n\nConvolutional neural networks can require \"a lot\" of GPU memory. These parameters can affect GPU memory utilisation:\n\n* reduce the mini-batch size\n* use fewer layers (e.g. `enc_nfilter_internal` = [16,24,36] or [16,24])\n* use less filters (reduce the values of the optional parameter enc_nfilter_internal)\n* reduce `frac_dense_layer`, a parameter controlling the width of the dense layer in the bottleneck\n* use a smaller domain or lower resolution\n\n\n## Example results\n\n[Link to animation](http://data-assimilation.net/upload/Alex/DINCAE/data-avg-DINCAE-AVHRR.gif)\n\n\nMore information about this result is given in the [linked paper](https://www.geosci-model-dev-discuss.net/gmd-2019-128/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgher-uliege%2FDINCAE","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgher-uliege%2FDINCAE","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgher-uliege%2FDINCAE/lists"}