{"id":13949185,"url":"https://github.com/TUW-GEO/esa_cci_sm","last_synced_at":"2025-07-20T11:31:07.127Z","repository":{"id":42063023,"uuid":"152270055","full_name":"TUW-GEO/esa_cci_sm","owner":"TUW-GEO","description":"Conversion of ESA CCI SM images into time series","archived":false,"fork":false,"pushed_at":"2024-11-07T15:53:46.000Z","size":4891,"stargazers_count":16,"open_issues_count":2,"forks_count":11,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-07-06T23:03:43.175Z","etag":null,"topics":["cci","climers","esa","preprocess","python","read","sm","soil-moisture"],"latest_commit_sha":null,"homepage":"https://www.esa-soilmoisture-cci.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TUW-GEO.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.rst","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-10-09T14:53:05.000Z","updated_at":"2025-07-02T08:51:02.000Z","dependencies_parsed_at":"2023-09-21T19:13:03.963Z","dependency_job_id":"31e90cb1-285d-480c-9353-9aa075bbd74d","html_url":"https://github.com/TUW-GEO/esa_cci_sm","commit_stats":{"total_commits":127,"total_committers":8,"mean_commits":15.875,"dds":"0.44881889763779526","last_synced_commit":"17d9e0901dc2497b2a41ee2d781d0b0d834cf749"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/TUW-GEO/esa_cci_sm","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUW-GEO%2Fesa_cci_sm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUW-GEO%2Fesa_cci_sm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUW-GEO%2Fesa_cci_sm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUW-GEO%2Fesa_cci_sm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TUW-GEO","download_url":"https://codeload.github.com/TUW-GEO/esa_cci_sm/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TUW-GEO%2Fesa_cci_sm/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264923074,"owners_count":23683717,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cci","climers","esa","preprocess","python","read","sm","soil-moisture"],"created_at":"2024-08-08T05:01:40.173Z","updated_at":"2025-07-20T11:31:07.108Z","avatar_url":"https://github.com/TUW-GEO.png","language":"Python","funding_links":[],"categories":["Natural Resources"],"sub_categories":["Soil and Land"],"readme":"============\nesa_cci_sm\n============\n\n|ci| |cov| |pip| |doc|\n\n.. |ci| image:: https://github.com/TUW-GEO/esa_cci_sm/actions/workflows/ci.yml/badge.svg?branch=master\n   :target: https://github.com/TUW-GEO/esa_cci_sm/actions\n\n.. |cov| image:: https://coveralls.io/repos/TUW-GEO/esa_cci_sm/badge.png?branch=master\n  :target: https://coveralls.io/r/TUW-GEO/esa_cci_sm?branch=master\n\n.. |pip| image:: https://badge.fury.io/py/esa_cci_sm.svg\n    :target: http://badge.fury.io/py/esa_cci_sm\n\n.. |doc| image:: https://readthedocs.org/projects/esa_cci_sm/badge/?version=latest\n   :target: http://esa-cci-sm.readthedocs.org/\n\n\nTools to convert `ESA CCI SM \u003chttps://climate.esa.int/en/projects/soil-moisture/\u003e`_ image files into a time series format.\n\nInstallation\n------------\n\nThis python package and all required dependencies can be installed from\n`pypi \u003chttps://pypi.org/project/esa-cci-sm/\u003e`_\nvia `pip \u003chttps://pip.pypa.io/en/stable/getting-started/\u003e`_\n\n.. code-block:: shell\n\n    pip install esa_cci_sm\n\nOn macOS if you get ``ImportError: Pykdtree failed to import its C extension``,\nthen it might be necessary to install the `pykdtree` package from conda-forge\n\n.. code-block:: shell\n\n    conda install -c conda-forge pykdtree\n\nData download\n-------------\n\nDownload ESA CCI SM data files either from the public\n`CEDA data store via https \u003chttps://catalogue.ceda.ac.uk/uuid/779f116d0477439db1874592add5848c/\u003e`_\nor the CEDA FTP server using, e.g. `FileZilla \u003chttps://filezilla-project.org/\u003e`_\nor `wget \u003chttps://www.gnu.org/software/wget/\u003e`_\n\n**Host (FTP)**: ``anon-ftp.ceda.ac.uk`` no password or user required\n\n**Directory (FTP)**: ``/neodc/esacci/soil_moisture``)\n\nE.g the following command will download v9.1 COMBINED data for the year 2023 from the CEDA FTP via wget\n\n.. code-block:: shell\n\n    wget -np -nH --cut-dirs 7 -r ftp://anon-ftp.ceda.ac.uk/neodc/esacci/soil_moisture/data/daily_files/COMBINED/v09.1/2023/\n\nTime series creation\n--------------------\n\nAfter installing this package via pip, you have access to the command line tool to convert ESA CCI SM image files\nto CF conform time series. We use an Orthogonal multidimensional array representation\nas implemented in the `pynetCF \u003chttps://github.com/TUW-GEO/pynetcf\u003e`_ python library.\n\nNote that we assume that the downloaded images are stored in yearly subfolders\nlike\n\n.. code-block::\n\n    /tmp/img/\n    ├── 1978/\n    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781101000000-fv09.1.nc\n    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-19781102000000-fv09.1.nc\n    │   ├── ...\n    ...\n    ├── 2023/\n    │   ├── ...\n    │   ├── ESACCI-SOILMOISTURE-L3S-SSMV-PASSIVE-20231231000000-fv09.1.nc\n\nThe following command would then take the daily images from 1991\nto 2023 in the path ``/tmp/img`` and convert data for grid cells over land into times series.\nTime series are then stored in ``/tmp/ts``.\n\n.. code-block:: shell\n\n    ccism_reshuffle /tmp/img /tmp/ts 1991-01-01 2023-12-31 --land_points True\n\nAfterwards, in python, the data can be read as pandas DataFrames.\n\n.. code-block:: python\n\n    \u003e\u003e from esa_cci_sm.interface import CCITs\n    \u003e\u003e ds = CCITs(\"/tmp/ts\", ioclass_kws={'read_bulk': True})\n    \u003e\u003e ds.read(15, 45)  # lon, lat\n                      sm  sm_uncertainty  flag  ...  mode  sensor            t0\n    1991-01-01  0.424880        0.094507     0  ...     1       2   7670.175000\n    1991-01-02       NaN             NaN    24  ...     2       2           NaN\n    1991-01-03       NaN             NaN     8  ...     0       2           NaN\n    ...              ...             ...   ...  ...   ...     ...           ...\n    2023-12-29  0.495448        0.039983     0  ...     3   21536  19720.051575\n    2023-12-30  0.426107        0.055060     0  ...     3   16416  19721.147066\n    2023-12-31  0.390103        0.030294     0  ...     3   21600  19722.117129\n\n\nSupported Products\n==================\n\nAt the moment this package supports ESA CCI soil moisture data versions\n3 to 9 in netCDF format (reading and time series creation)\nwith a spatial sampling of 0.25 degrees.\n\nContribute\n==========\n\nWe are happy if you want to contribute. Please raise an issue explaining what\nis missing or if you find a bug. We will also gladly accept pull requests\nagainst our master branch for new features or bug fixes.\n\nSetup\n-----\nSetup of a complete development environment with `conda\n\u003chttp://conda.pydata.org/miniconda.html\u003e`_ can be performed using the following\ncommands:\n\n.. code-block:: shell\n\n  git clone git@github.com:TUW-GEO/esa_cci_sm.git --recursive esa_cci_sm\n  cd ./esa_cci_sm\n  conda create -n esa_cci_sm python=3.12\n  conda activate esa_cci_sm\n  pip install -e .[testing]\n\nTo checkout our testdata files, you need to have\n`Git LFS \u003chttps://git-lfs.github.com/\u003e`_ installed on your machine.\n\nGuidelines\n----------\n\nIf you want to contribute please follow these steps:\n\n- Fork the esa_cci_sm repository to your account\n- Clone the repository, make sure you use ``git clone ... --recursive`` to also get\n  the test data repository.\n- make a new feature branch from the esa_cci_sm master branch\n- Add your feature\n- Please include tests for your contributions in one of the test directories.\n  We use pytest so a simple function called test_my_feature is enough\n- submit a pull request to our master branch\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTUW-GEO%2Fesa_cci_sm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTUW-GEO%2Fesa_cci_sm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTUW-GEO%2Fesa_cci_sm/lists"}