{"id":22197078,"url":"https://github.com/openscm/openscm-calibration_examples","last_synced_at":"2025-03-24T22:43:43.802Z","repository":{"id":157852111,"uuid":"632305731","full_name":"openscm/OpenSCM-Calibration_examples","owner":"openscm","description":"Long-running examples using OpenSCM-Calibration","archived":false,"fork":false,"pushed_at":"2024-08-14T00:23:01.000Z","size":120126,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-30T02:21:31.497Z","etag":null,"topics":["calibration","climate","examples","openscm","python"],"latest_commit_sha":null,"homepage":"https://openscm.github.io/OpenSCM-Calibration_examples/","language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openscm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-25T06:22:44.000Z","updated_at":"2023-04-29T05:19:19.000Z","dependencies_parsed_at":null,"dependency_job_id":"f76af6a4-733a-4e93-8cdc-de55b350f826","html_url":"https://github.com/openscm/OpenSCM-Calibration_examples","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"03057c0d7ef92c910d94c836551c20b00c92b672"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscm%2FOpenSCM-Calibration_examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscm%2FOpenSCM-Calibration_examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscm%2FOpenSCM-Calibration_examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscm%2FOpenSCM-Calibration_examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openscm","download_url":"https://codeload.github.com/openscm/OpenSCM-Calibration_examples/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245366195,"owners_count":20603438,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["calibration","climate","examples","openscm","python"],"created_at":"2024-12-02T14:18:20.031Z","updated_at":"2025-03-24T22:43:43.777Z","avatar_url":"https://github.com/openscm.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OpenSCM-Calibration-examples\n\nLong-running examples using [OpenSCM-Calibration](https://github.com/openscm/OpenSCM-Calibration).\n\n## Installation\n\nAfter cloning the repository, we recommend installing with [poetry](https://python-poetry.org/).\n\n```bash\npoetry install\n```\n\n## Running the examples\n\nAll the examples can be run with (and the docs built with)\n\n```bash\npoetry run jupyter-book build book\n```\n\n## Rationale\n\nAs a user of [OpenSCM-Calibration](https://github.com/openscm/OpenSCM-Calibration)\n(or indeed any repository), it is helpful to have examples of how it is used\nin production and the outputs it produces. However, including such examples in\nthe core development repository comes with a problem: it adds a very slow step\nto any continuous integration, which quickly becomes really annoying for\ndevelopment. A secondary problem is that you also end up with output in the\nrepository, which quickly bloats it (even notebooks can be megabytes in size\nif they contain many plots).\n\nThe solution we use here is to house our production-style examples in a\nseparate repository. We don't run these examples every time we change the code\nbase, however we do run them regularly and check/update them when the core\nrepository makes new releases.\n\n### Further details\n\nEven in this repository, we don't currently store outputs in the notebooks.\nThe reason is that outputs can quickly become bloated and storing the outputs\ndiscourages re-running i.e. testing the notebooks.\n\nHaving looked around, we haven't found a good solution that allows us to run\nour notebooks once as part of a test, then use the run output directly in a\ndocs build without rebuilding. [nbmake](https://github.com/treebeardtech/nbmake)\nclaims to support this use case, but in our experience jupyter-book didn't\nrecognise the nbmake output and tried to re-run the notebooks anyway. Perhaps\nwe were just using the combination of tools like [nbmake](https://github.com/treebeardtech/nbmake),\n[jupyter-cache](https://github.com/executablebooks/jupyter-cache) and\n[jupyter-book](https://github.com/executablebooks/jupyter-book) incorrectly\n(one thing to keep in mind if trying to make this work is that we want the\nexecution time of the notebook to appear in our docs too).\n\nInstead, we combine the docs building and testing steps. We build the docs\nusing jupyter-book, and include, in our notebooks, assertion cells that act\neffectively as tests. We avoid polluting the entire notebook with these\nassertions by making them hidden by default (and use jupyter-book's support\nfor showing and hiding cells to give users and readers the chance to check\nthem if they wish). We like this solution because it makes clear to developers\nwhere the assertion is (other solutions hide the assertions in the cell's\nJSON, which feels like a hack to us) and allows users to look at it if they\nwish while making clear that they aren't actually necessary for the example to\nrun.\n\nWe do check the notebook formatting as a separate step. This is easy to do and\nvery cheap using [blacken-docs](https://github.com/adamchainz/blacken-docs).\n\n### Really long-running notebooks\n\nIf we want to add super long-running notebooks to this repository, one\npossible problem is that they are too long-running to reasonably run in CI\n(perhaps they take 3 days to run). In this case, one solution could be to\nstart tracking some of the outputs of our notebook cache. This would make the\nnotebook untested in most cases (because the cache would be used instead of\nrunning the notebook) but this might be the best compromise where running the\nnotebook is truly not an option. We don't have such a use case yet so we\nhaven't implemented this, but we think the current solution doesn't shut the\ndoor on really long-running notebooks so is a good choice for now.\n\n## For developers\n\nFor development, we rely on [poetry](https://python-poetry.org) for all our\ndependency management. For all of work, we use our `Makefile`.\nYou can read the instructions out and run the commands by hand if you wish,\nbut we generally discourage this because it can be error prone and doesn't\nupdate if dependencies change (e.g. the environment is updated).\nIn order to create your environment, run `make virtual-environment -B`.\n\nIf there are any issues, the messages from the `Makefile` should guide you\nthrough. If not, please raise an issue in the\n[issue tracker](https://github.com/openscm/OpenSCM-Calibration_examples/issues).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenscm%2Fopenscm-calibration_examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenscm%2Fopenscm-calibration_examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenscm%2Fopenscm-calibration_examples/lists"}