{"id":20652428,"url":"https://github.com/thesofakillers/nowcastlib","last_synced_at":"2025-06-17T22:38:30.619Z","repository":{"id":62577772,"uuid":"337752010","full_name":"thesofakillers/nowcastlib","owner":"thesofakillers","description":"🧙‍♂️🔧 Utils that can be reused and shared across and beyond the ESO Nowcast project","archived":false,"fork":false,"pushed_at":"2021-08-27T15:53:32.000Z","size":4834,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-05-07T19:03:32.083Z","etag":null,"topics":["eso","eso-nowcast","nowcast"],"latest_commit_sha":null,"homepage":"https://giuliostarace.com/nowcastlib","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thesofakillers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-10T14:38:00.000Z","updated_at":"2021-08-26T13:43:44.000Z","dependencies_parsed_at":"2022-11-03T19:28:54.370Z","dependency_job_id":null,"html_url":"https://github.com/thesofakillers/nowcastlib","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/thesofakillers/nowcastlib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thesofakillers%2Fnowcastlib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thesofakillers%2Fnowcastlib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thesofakillers%2Fnowcastlib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thesofakillers%2Fnowcastlib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thesofakillers","download_url":"https://codeload.github.com/thesofakillers/nowcastlib/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thesofakillers%2Fnowcastlib/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":260452734,"owners_count":23011540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["eso","eso-nowcast","nowcast"],"created_at":"2024-11-16T17:34:56.951Z","updated_at":"2025-06-17T22:38:25.605Z","avatar_url":"https://github.com/thesofakillers.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nowcast Library\n\n🧙‍♂️🔧 Utils that can be reused and shared across and beyond the ESO Nowcast\nproject\n\nThis is a public repository hosted on GitHub via a push mirror setup in the\n[internal ESO GitLab repository](https://gitlab.eso.org/gstarace/nowcastlib/)\n\n## Installation\n\nSimply run\n\n```console\npip install nowcastlib\n```\n\n## Usage and Documentation\n\nNowcast Library (nowcastlib) consists in a collection of functions organized in\nsubmodules (API) and a tool accessible via the command line (CLI). The latter is\nprimarily intended for accessing the Nowcast Library Pipeline, an opinionated\nyet configurable set of processing steps for wrangling data and evaluating\nmodels in a consistent and rigorous way. More information can be found on the\nnowcastlib pipeline index page\n([link to markdown](nowcastlib/pipeline/README.md) and\n[link to hosted docs](https://giuliostarace.com/nowcastlib/pipeline))\n\nPlease refer to the\n[examples folder](https://github.com/thesofakillers/nowcastlib/tree/master/examples)\non GitHub for usage examples.\n\n### API\n\nHere is a quick example of how one may import nowcastlib and access to one of\nthe functions:\n\n```python\n\"\"\"Example showing how to access compute_trig_fields function\"\"\"\nimport nowcastlib as ncl\nimport pandas as pd\nimport numpy as np\n\ndata_df = pd.DataFrame(\n    np.array([[0, 3, 4, np.NaN], [32, 4, np.NaN, 4], [56, 8, 0, np.NaN]]).T,\n    columns=[\"A\", \"B\", \"C\"],\n    index=pd.date_range(start=\"1/1/2018\", periods=4, freq=\"2min\"),\n)\n\nresult = ncl.datasets.compute_trig_fields(data_df, [\"A\", \"C\"])\n```\n\nMore in-depth API documentation can be found\n[here](https://giuliostarace.com/nowcastlib/).\n\n### CLI\n\nSome of the library's functionality is bundled in configurable subcommands\naccessible via the terminal with the command `nowcastlib`:\n\n```console\nusage: nowcastlib [-h] [-v]\n                  {triangulate,preprocess,sync,postprocess,datapipe} ...\n\npositional arguments:\n  {triangulate,preprocess,sync,postprocess,datapipe}\n                        available commands\n    triangulate         Run `nowcastlib triangulate -h` for further help\n    preprocess          Run `nowcastlib preprocess -h` for further help\n    sync                Run `nowcastlib sync -h` for further help\n    postprocess         Run `nowcastlib postprocess -h` for further help\n    datapipe            Run `nowcastlib datapipe -h` for further help\n\noptional arguments:\n  -h, --help            show this help message and exit\n  -v, --verbose         increase verbosity level from INFO to DEBUG\n```\n\n### Repository Structure\n\nThe following output is generated with `tree . -I 'dist|docs|*.pyc|__pycache__'`\n\n```bash\n.\n├── LICENSE\n├── Makefile # currently used to build docs\n├── README.md\n├── de421.bsp # not committed\n├── docs/ # html files for the documentation static website\n├── examples\n│   ├── README.md\n│   ├── cli_triangulate_config.yaml\n│   ├── data/  # not committed\n│   ├── datasync.ipynb\n│   ├── output/ # not committed\n│   ├── pipeline_datapipe.json\n│   ├── pipeline_preprocess.json\n│   ├── pipeline_sync.json\n│   ├── signals.ipynb\n│   └── triangulation.ipynb\n├── images\n│   └── pipeline_flow.png\n├── nowcastlib # the actual source code for the library\n│   ├── __init__.py\n│   ├── cli\n│   │   ├── __init__.py\n│   │   └── triangulate.py\n│   ├── datasets.py\n│   ├── dynlag.py\n│   ├── gis.py\n│   ├── pipeline\n│   │   ├── README.md\n│   │   ├── __init__.py\n│   │   ├── cli.py\n│   │   ├── process\n│   │   │   ├── __init__.py\n│   │   │   ├── postprocess\n│   │   │   │   ├── __init__.py\n│   │   │   │   ├── cli.py\n│   │   │   │   └── generate.py\n│   │   │   ├── preprocess\n│   │   │   │   ├── __init__.py\n│   │   │   │   └── cli.py\n│   │   │   └── utils.py\n│   │   ├── split\n│   │   │   └── __init__.py\n│   │   ├── structs.py\n│   │   ├── sync\n│   │   │   ├── __init__.py\n│   │   │   └── cli.py\n│   │   └── utils.py\n│   ├── signals.py\n│   └── utils.py\n├── poetry.lock # lock file generated by python poetry for dependency mgmt\n└── pyproject.toml # general information file, handled by python poetry\n```\n\n#### Directories and Files not Committed\n\nThere are a number of files and folders that are not committed due to their\nlarge and static nature that renders them inappropriate for git version control.\nThe following files and folder warrant a brief explanation.\n\n-   Certain functions (time since sunset, sun elevation) of the Nowcast Library\n    rely on the use of a .bsp file, containing information on the locations\n    through time of various celestial bodies in the sky. This file will be\n    automatically downloaded upon using one of these functions for the first\n    time.\n-   The examples scripts make use of a `data/` directory containing a series of\n    csv files. Most of the data used in the examples can be downloaded from the\n    [ESO Ambient Condition Database](http://archive.eso.org/cms/eso-data/ambient-conditions.html).\n    Users can then change the paths set in the examples to fit their needs. For\n    users interested in replicating the exact structure and contents of the data\n    directory, a compressed copy of it (1.08 GB) is available to ESO members\n    through\n    [this Microsoft Sharepoint link](https://europeansouthernobservatory.sharepoint.com/:u:/t/OpticalTurbulenceandWeatherNowcast/EeH844GlBgdBjc63uaPiO4ABrh7ylH54zH3dJV9WSIhakA?e=gPBWJ7).\n-   At times the examples show the serialization functionality of the nowcastlib\n    pipeline or need to output some data. In these situations the `output/`\n    directory in the examples folder is used.\n\n## Development Setup\n\nThis repository relies on [Poetry](https://python-poetry.org/) for tracking\ndependencies, building and publishing. It is therefore recommended that\ndevelopers [install poetry](https://python-poetry.org/docs/#installation) and\nmake use of it throughout their development of the project.\n\n### Dependencies\n\nMake sure you are in the right Python environment and run\n\n```console\npoetry install\n```\n\nThis reads [pyproject.toml](./pyproject.toml), resolves the dependencies, and\ninstalls them.\n\n### Deployment\n\nThe repository is published to [PyPi](https://pypi.org/), so to make it\naccessible via a `pip install` command as mentioned [earlier](#install).\n\nTo publish changes follow these steps. Ideally this process is automated via a\nCI tool triggered by a push/merge to the master branch:\n\n1. Optionally run\n   [`poetry version`](https://python-poetry.org/docs/cli/#version) with the\n   appropriate argument based on [semver guidelines](https://semver.org/).\n\n2. Update the documentation by running\n\n    ```console\n    make document\n    ```\n\n3. Prepare the package by running\n\n    ```console\n    poetry build\n    ```\n\n4. Ensure you have [TestPyPi](https://test.pypi.org/) and PyPi configured as\n   your poetry repositories:\n\n    ```console\n    poetry config repositories.testpypi https://test.pypi.org/legacy/\n    poetry config repositories.pypi https://pypi.org/\n    ```\n\n5. Publish the repository to TestPyPi, to see that everything works as expected:\n\n    ```console\n    poetry publish -r testpypi\n    ```\n\n6. Stage, commit and push your changes (to master) with git.\n7. Publish the repository to PyPi:\n\n    ```console\n    poetry publish -r pypi\n    ```\n\nUpon successful deployment, the library should be available for install via\n`pip`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthesofakillers%2Fnowcastlib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthesofakillers%2Fnowcastlib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthesofakillers%2Fnowcastlib/lists"}