{"id":41435644,"url":"https://github.com/ecmwf-projects/cads-obs-catalogue-manager","last_synced_at":"2026-05-27T13:03:12.955Z","repository":{"id":220950307,"uuid":"736947994","full_name":"ecmwf-projects/cads-obs-catalogue-manager","owner":"ecmwf-projects","description":null,"archived":false,"fork":false,"pushed_at":"2026-03-30T14:27:33.000Z","size":2720,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-30T15:07:14.332Z","etag":null,"topics":["cads"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ecmwf-projects.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-12-29T10:45:36.000Z","updated_at":"2026-03-30T14:08:23.000Z","dependencies_parsed_at":"2024-02-05T11:33:13.080Z","dependency_job_id":"2dd1c837-808e-49eb-9f8f-3c90b3e0590a","html_url":"https://github.com/ecmwf-projects/cads-obs-catalogue-manager","commit_stats":null,"previous_names":["ecmwf-projects/cads-obs-catalogue-manager"],"tags_count":14,"template":false,"template_full_name":null,"purl":"pkg:github/ecmwf-projects/cads-obs-catalogue-manager","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecmwf-projects%2Fcads-obs-catalogue-manager","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecmwf-projects%2Fcads-obs-catalogue-manager/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecmwf-projects%2Fcads-obs-catalogue-manager/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecmwf-projects%2Fcads-obs-catalogue-manager/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ecmwf-projects","download_url":"https://codeload.github.com/ecmwf-projects/cads-obs-catalogue-manager/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecmwf-projects%2Fcads-obs-catalogue-manager/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31508282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T03:10:19.677Z","status":"ssl_error","status_checked_at":"2026-04-07T03:10:13.982Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cads"],"created_at":"2026-01-23T14:37:28.775Z","updated_at":"2026-04-07T10:00:32.016Z","avatar_url":"https://github.com/ecmwf-projects.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CopDS Observations repository catalogue manager\n\n## Instalation\n\nThe CLI services will be used directly on the VM.\n\n### Installation with conda\n\nTo install the package, simply clone it and install its dependencies via conda.\n\n```commandline\ngit clone git@github.com:ecmwf-projects/cads-obs-catalogue-manager.git\ncd cads-obs-catalogue-manager\nconda create -n cads-obs -c conda-forge python=3.10\nconda activate cads-obs\nconda env update --file environment.yml\npip install --no-deps .\n```\n\n### Installation with pip\n\nInstall python 3.12 with your package manager, together with [the netCDF library](https://github.com/Unidata/netcdf-c). Other dependencies may be needed depending\non the Linux version to be used. Then run.\n\n```commandline\ngit clone git@github.com:ecmwf-projects/cads-obs-catalogue-manager.git\ncd cads-obs-catalogue-manager\npython -m pip install .\n```\n\n## Workflow for developers/contributors\n\nFor best experience create a new conda environment (e.g. DEVELOP) with Python 3.10:\n\n```commandline\nconda create -n DEVELOP -c conda-forge python=3.12\nconda activate DEVELOP\n```\n\nBefore pushing to GitHub, run the following commands:\n\n1. Update conda environment: `make conda-env-update`\n1. Install this package: `pip install -e .`\n1. Sync with the latest [template](https://github.com/ecmwf-projects/cookiecutter-conda-package) (optional): `make template-update`\n1. Run quality assurance checks: `make qa`\n1. Run tests: `make unit-tests`\n1. Run the static type checker: `make type-check`\n1. Build the documentation (see [Sphinx tutorial](https://www.sphinx-doc.org/en/master/tutorial/)): `make docs-build`\n\n### Writing tests\n\nWe use the pytest framework and fixtures.\n\n### Test data preparison\n\nThe tests are meant to work against test databases, so a few steps are needed to\nset them up:\n\nFirst, download the dump: [test_ingestiondb.sql](https://cloud.predictia.es/s/R9a6z8fBZQcPrAQ)\nfile and store it at tests/docker.\n\nDownload also [cuon_data.tar.gz](https://cloud.predictia.es/s/dTb87RQXfgJ6S6S) and\nextract it in tests/data/cuon_data. This contains a couple of netCDFs from the CUON\nsoundings dataset for the tests.\n\nAdd a .env file at tests/docker containing the pass credentials for the 3 db:\n\n```commandline\nTEST_INGESTION_DB_PASS=xxxx\nCATALOGUE_PASSWORD=xxxx\nSTORAGE_PASSWORD=xxxx\n```\n\nNote that credentials must coincide with the ones at the cdsobs_config template.\n\nFinally start the docker containers by running:\n\n```commandline\ncd tests/docker\ndocker compose up -d\n```\n\nIngestion db tables available right now:\n\n- woudc_ozonesonde_header\n- woudc_ozonesonde_data\n- woudc_totalozone_header\n- woudc_totalozone_data\n- guan_data_header\n- guan_data_value\n- header\n- harmonized_data\n- gruan_data_header\n- gruan_data_value\n- uscrn.unc_subhourly\n- uscrn.unc_hourly\n- uscrn.unc_daily\n- uscrn.unc_monthly\n\n### Test data update\n\nThis requires SSH access to the VM and a .env file defining the DB connection\ndetails. It will replicate the ingestion DB (right now only the Ozone dataset),\ntruncating the data tables at 10k rows, and dump it to a .sql file.\n\n```commandline\ncd tests/scripts\nbash make_test_ingestiondb.sh\n```\n\n### HTTP API\n\nTo deploy the HTTP API, unicorn needs to be installed. Please note that it should\nnot be exposed to the internet without a proxy. The deploy command is:\n\n```\npython cdsobs/api_rest/main.py\n```\n\nThe /doc endpoint can be opened in a browser to check the automatically generated\ndocumentation.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecmwf-projects%2Fcads-obs-catalogue-manager","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fecmwf-projects%2Fcads-obs-catalogue-manager","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecmwf-projects%2Fcads-obs-catalogue-manager/lists"}