{"id":26055849,"url":"https://github.com/ncar/cupid","last_synced_at":"2025-04-03T03:13:33.116Z","repository":{"id":210784646,"uuid":"719248925","full_name":"NCAR/CUPiD","owner":"NCAR","description":"CUPiD is a “one stop shop” that enables and integrates timeseries file generation, data standardization, diagnostics, and metrics from all CESM components.","archived":false,"fork":false,"pushed_at":"2025-03-31T22:10:48.000Z","size":16152,"stargazers_count":30,"open_issues_count":54,"forks_count":28,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-31T23:23:02.345Z","etag":null,"topics":["cesm","diagnostics","framework","python"],"latest_commit_sha":null,"homepage":"https://ncar.github.io/CUPiD/","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NCAR.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-15T19:20:24.000Z","updated_at":"2025-03-31T22:10:55.000Z","dependencies_parsed_at":"2023-12-13T00:28:56.719Z","dependency_job_id":"80b1dfce-07e1-4e1c-8cc6-b6ca046a24ed","html_url":"https://github.com/NCAR/CUPiD","commit_stats":{"total_commits":209,"total_committers":13,"mean_commits":"16.076923076923077","dds":0.7177033492822966,"last_synced_commit":"d2cc4d0513ca60402b4e50192826af4f6c756a33"},"previous_names":["ncar/cupid"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCAR%2FCUPiD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCAR%2FCUPiD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCAR%2FCUPiD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NCAR%2FCUPiD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NCAR","download_url":"https://codeload.github.com/NCAR/CUPiD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246927839,"owners_count":20856198,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cesm","diagnostics","framework","python"],"created_at":"2025-03-08T10:23:19.341Z","updated_at":"2025-04-03T03:13:33.072Z","avatar_url":"https://github.com/NCAR.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# \u003cimg src=\"images/logo.png\" alt=\"CUPiD Logo\" width=100 /\u003e CUPiD: CESM Unified Postprocessing and Diagnostics\n\nPython Framework for Generating Diagnostics from CESM\n\n## Project Vision\n\nCUPiD is a “one stop shop” that enables and integrates timeseries file generation, data standardization, diagnostics, and metrics from all CESM components.\n\nThis collaborative effort aims to simplify the user experience of running diagnostics by calling post-processing tools directly from CUPiD, running all component diagnostics from the same tool as either part of the CIME workflow or independently, and sharing python code and a standard conda environment across components.\n\n## Installing\n\nTo install CUPiD, you need to check out the code and then set up a few environments.\nThe initial examples have hard-coded paths that require you to be on `casper`.\n\nThe code relies on submodules to install a few packages that are still being developed,\nso the `git clone` process requires `--recurse-submodules`:\n\n``` bash\n$ git clone --recurse-submodules https://github.com/NCAR/CUPiD.git\n```\n\nThen `cd` into the `CUPiD` directory and build the necessary conda environments with\n\n``` bash\n$ cd CUPiD\n$ mamba env create -f environments/cupid-infrastructure.yml\n$ conda activate cupid-infrastructure\n$ which cupid-diagnostics\n$ mamba env create -f environments/cupid-analysis.yml\n```\n\nNotes:\n\n1. As of version 23.10.0, `conda` defaults to using `mamba` to solve environments.\nIt still feels slower than running `mamba` directly, hence the recommendation to install with `mamba env create` rather than `conda env create`.\nIf you do not have `mamba` installed, you can still use `conda`... it will just be significantly slower.\n(To see what version of conda you have installed, run `conda --version`.)\n1. If the subdirectories in `externals/` are all empty, run `git submodule update --init` to clone the submodules.\n1. For existing users who cloned `CUPiD` prior to the switch from manage externals to git submodule, we recommend removing `externals/` before checking out main, running `git submodule update --init`, and removing `manage_externals` (if it is still present after `git submodule update --init`).\n1. If `which cupid-diagnostics` returned the error `which: no cupid-diagnostics in ($PATH)`, then please run the following:\n\n   ``` bash\n   $ conda activate cupid-infrastructure\n   $ pip install -e .  # installs cupid\n   ```\n\n1. In the `cupid-infrastructure` environment, run `pre-commit install` to configure `git` to automatically run `pre-commit` checks when you try to commit changes from the `cupid-infrastructure` environment; the commit will only proceed if all checks pass. Note that CUPiD uses `pre-commit` to ensure code formatting guidelines are followed, and pull requests will not be accepted if they fail the `pre-commit`-based Github Action.\n1. If you plan on contributing code to CUPiD,\nwhether developing CUPiD itself or providing notebooks for CUPiD to run,\nplease see the [Contributor's Guide](https://ncar.github.io/CUPiD/contributors_guide.html).\n\n## Running\n\nCUPiD currently provides an example for generating diagnostics.\nTo test the package out, try to run `examples/key-metrics`:\n\n``` bash\n$ conda activate cupid-infrastructure\n$ cd examples/key_metrics\n$ # machine-dependent: request multiple compute cores\n$ cupid-diagnostics\n$ cupid-webpage  # Will build HTML from Jupyter Book\n```\n\nAfter the last step is finished, you can use Jupyter to view generated notebooks in `${CUPID_ROOT}/examples/key-metrics/computed_notebooks`\nor you can view `${CUPID_ROOT}/examples/key-metrics/computed_notebooks/_build/html/index.html` in a web browser.\n\nNotes:\n\n1. Occasionally users report the following error the first time they run CUPiD: `Environment cupid-analysis specified for \u003cYOUR-NOTEBOOK\u003e.ipynb could not be found`. The fix for this is the following:\n   ``` bash\n   $ conda activate cupid-analysis\n   (cupid-analysis) $ python -m ipykernel install --user --name=cupid-analysis\n   ```\n\nFurthermore, to clean the `computed_notebooks` folder which was generated by the `cupid-diagnostics` and `cupid-webpage` commands, you can run the following command:\n\n``` bash\n$ cupid-clean\n```\n\nThis will clean the `computed_notebooks` folder which is at the location pointed to by the `run_dir` variable in the `config.yml` file.\n\n### CUPiD Options\n\nMost of CUPiD's configuration is done via the `config.yml` file, but there are a few command line options as well:\n\n```bash\n(cupid-infrastructure) $ cupid-diagnostics -h\nUsage: cupid-diagnostics [OPTIONS] CONFIG_PATH\n\n  Main engine to set up running all the notebooks.\n\nOptions:\n  -s, --serial        Do not use LocalCluster objects\n  -ts, --time-series  Run time series generation scripts prior to diagnostics\n  -atm, --atmosphere  Run atmosphere component diagnostics\n  -ocn, --ocean       Run ocean component diagnostics\n  -lnd, --land        Run land component diagnostics\n  -ice, --seaice      Run sea ice component diagnostics\n  -glc, --landice     Run land ice component diagnostics\n  -rof, --river-runoff Run river runoff component diagnostics\n  --config_path       Path to the YAML configuration file containing specifications for notebooks (default config.yml)\n  -h, --help          Show this message and exit.\n```\n\n#### Running in serial\n\nBy default, several of the example notebooks provided use a dask `LocalCluster` object to run in parallel.\nHowever, the `--serial` option will pass a logical flag to each notebook that can be used to skip starting the cluster.\n\n```py3\n# Spin up cluster (if running in parallel)\nclient=None\nif not serial:\n  cluster = LocalCluster(**lc_kwargs)\n  client = Client(cluster)\n\nclient\n```\n\n#### Specifying components\n\nIf no component flags are provided, all component diagnostics listed in `config.yml` will be executed by default. Multiple flags can be used together to select a group of components, for example: `cupid-diagnostics -ocn -ice`.\n\n\n### Timeseries File Generation\nCUPiD also has the capability to generate single variable timeseries files from history files for all components. To run timeseries, edit the `config.yml` file's timeseries section to fit your preferences, and then run `cupid-timeseries`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fncar%2Fcupid","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fncar%2Fcupid","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fncar%2Fcupid/lists"}