https://github.com/TUW-GEO/ecmwf_models
Python package for downloading ECMWF reanalysis data and converting it into a time series format.
https://github.com/TUW-GEO/ecmwf_models
climers earth-science ecmwf era5 geo modelling python remote-sensing
Last synced: 19 days ago
JSON representation
Python package for downloading ECMWF reanalysis data and converting it into a time series format.
- Host: GitHub
- URL: https://github.com/TUW-GEO/ecmwf_models
- Owner: TUW-GEO
- License: mit
- Created: 2016-08-29T12:44:50.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2025-03-10T09:46:28.000Z (about 1 month ago)
- Last Synced: 2025-03-30T09:29:57.679Z (21 days ago)
- Topics: climers, earth-science, ecmwf, era5, geo, modelling, python, remote-sensing
- Language: Python
- Homepage: https://ecmwf-models.readthedocs.io
- Size: 6.16 MB
- Stars: 40
- Watchers: 5
- Forks: 15
- Open Issues: 2
-
Metadata Files:
- Readme: README.rst
- Changelog: CHANGELOG.rst
- Contributing: CONTRIBUTING.rst
- License: LICENSE.txt
- Authors: AUTHORS.rst
Awesome Lists containing this project
- awesome_time_series_in_python - ecmwf_models
README
============
ecmwf_models
============
|ci| |cov| |pip| |doc|
.. |ci| image:: https://github.com/TUW-GEO/ecmwf_models/actions/workflows/ci.yml/badge.svg?branch=master
:target: https://github.com/TUW-GEO/ecmwf_models/actions
.. |cov| image:: https://coveralls.io/repos/TUW-GEO/ecmwf_models/badge.png?branch=master
:target: https://coveralls.io/r/TUW-GEO/ecmwf_models?branch=master
.. |pip| image:: https://badge.fury.io/py/ecmwf-models.svg
:target: https://badge.fury.io/py/ecmwf-models
.. |doc| image:: https://readthedocs.org/projects/ecmwf-models/badge/?version=latest
:target: https://ecmwf-models.readthedocs.io/en/latest/
Readers and converters for ECMWF reanalysis (`ERA5 `_
and `ERA5-Land `_) data.
Written in Python.
Works great in combination with `pytesmo `_.
Installation
============
This package has been tested on Linux, Windows and macOS for
python ``3.10``, ``3.11``, and ``3.12``.
Ideally you should use one of the supported python versions (the package might
still work for older python versions).
Use ``pip`` to install all required python dependencies
as well as the ``ecmwf_models`` package from `pypi `_.
.. code::
pip install ecmwf_models
On Windows systems, it might be necessary to install required C-libraries via conda.
For installation we recommend `Miniconda `_:
.. code::
conda install -c conda-forge pygrib netcdf4 pyresample pykdtree
Quick Start
===========
Download image data from CDS (`set up API `_ first) using the ``era5 download`` and ``era5land download``
console command (see ``era5 download --help`` for all options) ...
.. code-block:: shell
era5land download /tmp/era5/img -s 2024-04-01 -e 2024-04-05 -v swvl1,swvl2 --h_steps 0,12
... and convert them to time series (ideally for a longer period). Check ``era5 reshuffle --help``
.. code-block:: shell
era5land reshuffle /tmp/era5/img /tmp/era5/ts -s 2024-04-01 -e 2024-04-05 --land_points True
Finally, in python, read the time series data for a location as a pandas
DataFrame.
.. code-block:: python
>> from ecmwf_models.interface import ERATs
>> ds = ERATs('/tmp/era5/ts')
>> ds.read(18, 48) # (lon, lat)
swvl1 swvl2
2024-04-01 00:00:00 0.318054 0.329590
2024-04-01 12:00:00 0.310715 0.325958
2024-04-02 00:00:00 0.360229 0.323502
... ... ...
2024-04-04 12:00:00 0.343353 0.348755
2024-04-05 00:00:00 0.350266 0.346558
2024-04-05 12:00:00 0.343994 0.344498
More programs are available to keep an exisiting image and time series record
up-to-date. Type ``era5 --help`` and ``era5land --help`` to see all available
programs.
CDS API Setup
=============
In order to download data from CDS, this package uses the CDS API
(https://pypi.org/project/cdsapi/). You can either pass your credentials
directly on the command line (which might be unsafe) or set up a
.cdsapirc file in your home directory (recommended).
Please see the description at https://cds.climate.copernicus.eu/how-to-api.
Supported Products
==================
At the moment this package supports
- **ERA5**
- **ERA5-Land**
reanalysis data in **grib** and **netcdf** format (download, reading, time
series creation) with a default spatial sampling of 0.25 degrees (ERA5),
and 0.1 degrees (ERA5-Land).
It should be easy to extend the package to support other ECMWF reanalysis
products. This will be done as need arises.
Docker image
============
We provide a docker image for this package. This contains all
pre-installed dependencies and can simply be pulled via
.. code-block:: shell
$ docker pull ghcr.io/tuw-geo/ecmwf_models:latest
Alternatively, to build the image locally using the provided Dockerfile, call
from the package root
.. code-block:: shell
$ docker buildx build -t ecmwf_models:latest . 2>&1 | tee docker_build.log
Afterwards, you can execute the ``era5`` and ``era5land`` commands directly in
the container (after mounting some volumes to write data to).
The easiest way to set the API credentials in this case is via the
``CDSAPI_KEY`` container variable or the ``--cds_token`` option as below.
.. code-block:: shell
$ docker run -v /data/era5/img:/container/path ecmwf_models:latest bash -c \
'era5land update_img /container/path --cds_token xxxx-xxx-xxx-xx-xxxx'
You can use this together with a task scheduler to regularly pull new data.
Citation
========
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.593533.svg
:target: https://doi.org/10.5281/zenodo.593533
If you use the software in a publication then please cite it using the Zenodo DOI.
Be aware that this badge links to the latest package version.
Contribute
==========
We are happy if you want to contribute. Please raise an issue explaining what
is missing or if you find a bug.
Please take a look at the `developers guide `_.