https://github.com/PCMDI/pcmdi_metrics

Open-source Python package for Systematic Evaluation of Climate and Earth System Models
https://github.com/PCMDI/pcmdi_metrics
climate climate-analysis climate-data climate-model climate-model-evaluation climate-models climate-science climate-variability python
Last synced: 3 months ago
JSON representation
Open-source Python package for Systematic Evaluation of Climate and Earth System Models
Host: GitHub
URL: https://github.com/PCMDI/pcmdi_metrics
Owner: PCMDI
License: bsd-3-clause
Created: 2013-05-28T20:48:38.000Z (over 12 years ago)
Default Branch: main
Last Pushed: 2025-07-02T20:49:44.000Z (3 months ago)
Last Synced: 2025-07-07T00:41:58.411Z (3 months ago)
Topics: climate, climate-analysis, climate-data, climate-model, climate-model-evaluation, climate-models, climate-science, climate-variability, python
Language: Python
Homepage: http://pcmdi.github.io/pcmdi_metrics/
Size: 626 MB
Stars: 114
Watchers: 9
Forks: 44
Open Issues: 48
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.rst
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Citation: CITATION.cff
- Support: docs/supporting-data.rst
Awesome Lists containing this project

open-sustainable-technology - PCMDI Metrics Package - Open-source Python package for Systematic Evaluation of Climate and Earth System Models. (Climate Change / Earth and Climate Modeling)
README

          














# PCMDI Metrics Package (PMP)

[![latest version](https://img.shields.io/conda/vn/conda-forge/pcmdi_metrics.svg?kill_cache=1)](https://anaconda.org/conda-forge/pcmdi_metrics/)

[![Last updated](https://anaconda.org/conda-forge/pcmdi_metrics/badges/latest_release_date.svg?kill_cache=1)](https://anaconda.org/conda-forge/pcmdi_metrics/files)

![platforms](https://img.shields.io/badge/platforms-linux%20|%20osx-lightgrey.svg)

[![DOI](https://zenodo.org/badge/doi/10.5281/zenodo.592790.svg)](https://doi.org/10.5281/zenodo.592790)

[![License](https://anaconda.org/conda-forge/pcmdi_metrics/badges/license.svg)](https://github.com/PCMDI/pcmdi_metrics/blob/main/LICENSE)

[![Formatted with black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)

[![All Contributors](https://img.shields.io/github/all-contributors/PCMDI/pcmdi_metrics?color=ee8449&style=flat-square)](#contributors)

Conda-forge (CURRENT, recommended):

[![Download](https://anaconda.org/conda-forge/pcmdi_metrics/badges/downloads.svg?kill_cache=1)](https://anaconda.org/conda-forge/pcmdi_metrics/)

PCMDI Conda Channel (old, deprecated):

[![Download](https://anaconda.org/pcmdi/pcmdi_metrics/badges/downloads.svg?kill_cache=1)](https://anaconda.org/pcmdi/pcmdi_metrics)

The PCMDI Metrics Package (PMP) is used to provide "quick-look" objective comparisons of Earth System Models (ESMs) with one another and available observations.  Results are produced in the context of all model simulations contributed to CMIP6 and earlier CMIP phases.  Among other purposes, this enables modeling groups to evaluate changes during the development cycle in the context of the structural error distribution of the multi-model ensemble. Currently, the comparisons emphasize metrics of large- to global-scale annual cycle, tropical and extra-tropical modes of variability, ENSO, MJO, regional monsoons, high frequency characteristics of simulated precipitation, and cloud feedback.

**PCMDI uses the PMP to produce [quick-look simulation summaries across generations of CMIP](https://pcmdi.llnl.gov/research/metrics/).**

The metrics package consists of the following parts: 

* Analysis software

* Observation-based reference database of global (or near global, land or ocean) [time series and climatologies](https://github.com/PCMDI/PCMDIobs-cmor-tables/tree/master/catalogue)

* [Package documentation](http://pcmdi.github.io/pcmdi_metrics/) and [interactive jupyter notebook demos](https://github.com/PCMDI/pcmdi_metrics/blob/main/doc/jupyter/Demo/README.md)

* [Database](https://github.com/PCMDI/pcmdi_metrics_results_archive) of performance metrics computed for CMIP models

The package expects model data to be [CF-compliant](http://cfconventions.org/). To successfully use the package some input data "conditioning" may be required. We provide several demo scripts within the package.

Documentation

-------------

### Getting Started

* Installation requirements and instructions are available on the [Install](http://pcmdi.github.io/pcmdi_metrics/install.html) page

* Users will need to contact the PMP developers (pcmdi-metrics@llnl.gov) to obtain supporting datasets and get started using the package.

* An overview for using the package and template scripts are detailed on the [Using-the-package](http://pcmdi.github.io/pcmdi_metrics) page

* [View Demo](https://github.com/PCMDI/pcmdi_metrics/blob/main/doc/jupyter/Demo/README.md)

### References

Latest: 

* Lee, J., Gleckler, P. J., Ahn, M.-S., Ordonez, A., Ullrich, P. A., Sperber, K. R., Taylor, K. E., Planton, Y. Y., Guilyardi, E., Durack, P., Bonfils, C., Zelinka, M. D., Chao, L.-W., Dong, B., Doutriaux, C., Zhang, C., Vo, T., Boutte, J., Wehner, M. F., Pendergrass, A. G., Kim, D., Xue, Z., Wittenberg, A. T., and Krasting, J.: Systematic and objective evaluation of Earth system models: PCMDI Metrics Package (PMP) version 3, Geosci. Model Dev., 17, 3919–3948, https://doi.org/10.5194/gmd-17-3919-2024, **2024**. 

Earlier versions:

* Gleckler, P. J., Doutriaux, C., Durack, P. J., Taylor, K. E., Zhang, Y., Williams, D. N., Mason, E., and Servonnat, J.: A more powerful reality test for climate models, Eos T. Am. Geophys. Un., 97, https://doi.org/10.1029/2016eo051663, **2016**. 

* Gleckler, P. J., Taylor, K. E., and Doutriaux, C.: Performance metrics for climate models, J. Geophys. Res., 113, D06104, https://doi.org/10.1029/2007jd008972, **2008**. 

Contact

-------

[Report Bug](https://github.com/PCMDI/pcmdi_metrics/issues)

[Request Feature](https://github.com/PCMDI/pcmdi_metrics/issues)

Some installation support for CMIP participating modeling groups is available: pcmdi-metrics@llnl.gov

Acknowledgement

---------------

Content in this repository is developed by climate and computer scientists from the Program for Climate Model Diagnosis and Intercomparison ([PCMDI][PCMDI]) at Lawrence Livermore National Laboratory ([LLNL][LLNL]). This work is sponsored by the Regional and Global Model Analysis ([RGMA][RGMA]) program, of the Earth and Environmental Systems Sciences Division ([EESSD][EESSD]) in the Office of Biological and Environmental Research ([BER][BER]) within the [Department of Energy][DOE]'s [Office of Science][OS]. The work is performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.

LLNL-CODE-2004137 

DOE CODE ID: #153383



     

     

    



[PCMDI]: https://pcmdi.llnl.gov/

[LLNL]: https://www.llnl.gov/

[RGMA]: https://climatemodeling.science.energy.gov/program/regional-global-model-analysis

[EESSD]: https://science.osti.gov/ber/Research/eessd

[BER]: https://science.osti.gov/ber

[DOE]: https://www.energy.gov/

[OS]: https://science.osti.gov/

License

-------

Distributed under the BSD 3-Clause License. See [`LICENSE`](https://github.com/PCMDI/pcmdi_metrics/blob/main/LICENSE) for more information.

Release Notes and History

-------------------------

| 
[Versions] | Update summary   |

| ------------- | ------------------------------------- |

| [v3.9.1]      | New capability (**new modes for modes of variability metrics: EA, SCA**) and technical update

| [v3.9]        | New capability (**Decision-Relevant metrics, Database access API**) and new demo notebooks

| [v3.8.2]      | Technical update

| [v3.8.1]      | Technical update with new figure (modes of variability multi-panel plot)

| [v3.8]        | New capability (**figure generation for ENSO**, xCDAT migration completed for **Monsoon Wang** with figure generation), major dependency update (`numpy` >= 2.0)

| [v3.7.2]      | Technical update

| [v3.7.1]      | Technical update with documentation improvements

| [v3.7]        | New capability (**figure generation for mean climate**) and technical update

| [v3.6.1]      | Technical update, additional QC repair functions

| [v3.6]        | New capability (**regional application of precip variability**) and technical update

| [v3.5.2]      | New capability (**QC**, **new modes for modes of variability metrics: PSA1, PSA2**) and technical update

| [v3.5.1]      | Technical update

| [v3.5]        | Technical update: MJO and Monsoon Sperber [xCDAT](https://xcdat.readthedocs.io/en/latest/) conversion

| [v3.4.1]      | Technical update

| [v3.4]        | Technical update: Modes of variability [xCDAT](https://xcdat.readthedocs.io/en/latest/) conversion

| [v3.3.4]      | Technical update

| [v3.3.3]      | Technical update

| [v3.3.2]      | Technical update

| [v3.3.1]      | Technical update

| [v3.3]        | New metric added: **Sea-Ice**

| [v3.2]        | New metric added: **Extremes**

| [v3.1.2]      | Technical update

| [v3.1.1]      | Technical and documentation update

| [v3.1]        | New metric added: **Precipitation Benchmarking -- distribution bimodality**

| [v3.0.2]      | Minor patch and more documentation added

| [v3.0.1]      | Minor technical patch                 

| [v3.0.0]      | New metric added: **Cloud feedback metric** by @mzelinka. [**xCDAT**](https://xcdat.readthedocs.io/en/latest/) implemented for mean climate metrics

  Click here for older versions

| 
[Versions] | Update summary   |

| ------------- | ------------------------------------- |

| [v2.5.1]      | Technical update

| [v2.5.0]      | New metric added: **Precipitation Benchmarking -- distribution**. Graphics updated

| [v2.4.0]      | New metric added: **AMO** in variability modes

| [v2.3.2]      | CMEC interface updates

| [v2.3.1]      | Technical update

| [v2.3]        | New graphics using [archived PMP results](https://github.com/PCMDI/pcmdi_metrics_results_archive)

| [v2.2.2]      | Technical update

| [v2.2.1]      | Minor update

| [v2.2]        | New metric implemented: **precipitation variability across time scale**

| [v2.1.2]      | Minor update

| [v2.1.1]      | Simplified dependent libraries and CI process

| [v2.1.0]      | [**CMEC**](https://cmec.llnl.gov/) driver interfaced added.

| [v2.0]        | New capabilities: **ENSO** metrics, demos, and documentations.

| [v1.2]        | Tied to CDAT 8.0. Extensive regression testing added. New metrics: **Diurnal cycle and intermittency of precipitation**, sample **monsoon** metrics.

| [v1.1.2]      | Now managed through Anaconda, and tied to UV-CDAT 2.10.  Weights on bias statistic added. Extensive provenance information incorporated into json files.

| [v1.1]        | First public release, emphasizing **climatological statistics**, with development branches for ENSO and regional monsoon precipitation indices

| [v1.0]        | Prototype version of the PMP

[Versions]: https://github.com/PCMDI/pcmdi_metrics/releases

[v3.9.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.9.1

[v3.9]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.9

[v3.8.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.8.2

[v3.8.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.8.1

[v3.8]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.8

[v3.7.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.7.2

[v3.7.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.7.1

[v3.7]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.7

[v3.6.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.6.1

[v3.6]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.6

[v3.5.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.5.2

[v3.5.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.5.1

[v3.5]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.5

[v3.4.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.4.1

[v3.4]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.4

[v3.3.4]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.3.4

[v3.3.3]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.3.3

[v3.3.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.3.2

[v3.3.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.3.1

[v3.3]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.3

[v3.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.2

[v3.1.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1.2

[v3.1.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1.1

[v3.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.1

[v3.0.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.0.2

[v3.0.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.0.1

[v3.0.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v3.0.0

[v2.5.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.5.1

[v2.5.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.5.0

[v2.4.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.4.0

[v2.3.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.3.2

[v2.3.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.3.1

[v2.3]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.3

[v2.2.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.2.2

[v2.2.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.2.1

[v2.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.2

[v2.1.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.1.2

[v2.1.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.1.1

[v2.1.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.1.0

[v2.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v2.0

[v1.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v1.2

[v1.1.2]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/1.1.2

[v1.1]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v1.1

[v1.0]: https://github.com/PCMDI/pcmdi_metrics/releases/tag/v1.0

Current Core Development Team

-----------------------------

* [Jiwoo Lee](https://people.llnl.gov/lee1043) ([LLNL](https://www.llnl.gov/), PMP Lead)

* [Peter Gleckler](https://pcmdi.llnl.gov/staff/gleckler/) ([LLNL](https://www.llnl.gov/))

* [Paul Ullrich](https://people.llnl.gov/ullrich4) ([LLNL](https://www.llnl.gov/), [PCMDI](https://pcmdi.llnl.gov/) Project PI)

* [Bo Dong](https://people.llnl.gov/dong12) ([LLNL](https://www.llnl.gov/))

* [Kristin Chang](https://people.llnl.gov/chang61) ([LLNL](https://www.llnl.gov/))

* [Shixuan Zhang](https://www.pnnl.gov/science/staff/staff_info.asp?staff_num=9376) ([PNNL](https://www.pnnl.gov/))

All Contributors

----------------

Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):

  

    

      
_{Jiwoo Lee}
💻 📖 👀 ⚠️ ✅ 🔬 🤔 🚇

      
_{Peter Gleckler}
💻 📖 🔬 👀 ⚠️ 🔣 🤔

      
_{Ana Ordonez}
💻 📖 👀 ⚠️ ✅ 🚇

      
_{Min-Seop Ahn}
💻 📖 👀 ⚠️ ✅ 🔬

      
_{Paul Ullrich}
🤔 🔬

      
_{Charles Doutriaux}
💻

      
_{Karl Taylor}
🔬 🤔

    

    

      
_{Paul J. Durack}
💻

      
_{Mark Zelinka}
💻

      
_{Celine Bonfils}
🔬

      
_{Curtis C. Covey}
💻 🔬

      
_{Zeshawn Shaheen}
💻

      
_{Lina Muryanto}
🚇

      
_{Tom Vo}
🚇

    

    

      
_{Jason Boutte}
🚇

      
_{Jeffrey Painter}
🔣 🚇 💻

      
_{Stephen Po-Chedley}
🔣 🚇

      
_{Xylar Asay-Davis}
🚇

      
_{John Krasting}
💻 ⚠️

      
_{Angeline G Pendergrass}
💻 🔬 🤔

      
_{Michael Wehner}
💻 🔬

    

    

      
_{Daehyun Kim}
💻 🔬

      
_{Bo Dong}
💻

      
_{Shixuan Zhang}
💻

      
_{Kristin Chang}
💻

      
_{Alex Jonko}
💻

    

  

This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification.
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/PCMDI/pcmdi_metrics

Awesome Lists containing this project

README
