{"id":37403495,"url":"https://github.com/oslokommune/okdata-metadata-api","last_synced_at":"2026-01-16T05:48:24.847Z","repository":{"id":37868480,"uuid":"376821627","full_name":"oslokommune/okdata-metadata-api","owner":"oslokommune","description":"API for posting, updating and retrieving metadata in the dataplatform","archived":false,"fork":false,"pushed_at":"2026-01-12T17:08:54.000Z","size":1575,"stargazers_count":0,"open_issues_count":3,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2026-01-12T18:08:45.753Z","etag":null,"topics":["dataplatform"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oslokommune.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2021-06-14T12:51:15.000Z","updated_at":"2026-01-12T08:26:07.000Z","dependencies_parsed_at":"2024-05-21T08:48:06.081Z","dependency_job_id":"b971a28f-c799-43ce-bbe0-37cd69fe2f50","html_url":"https://github.com/oslokommune/okdata-metadata-api","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/oslokommune/okdata-metadata-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-metadata-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-metadata-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-metadata-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-metadata-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oslokommune","download_url":"https://codeload.github.com/oslokommune/okdata-metadata-api/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oslokommune%2Fokdata-metadata-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28477420,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T03:13:13.607Z","status":"ssl_error","status_checked_at":"2026-01-16T03:11:47.863Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataplatform"],"created_at":"2026-01-16T05:48:24.738Z","updated_at":"2026-01-16T05:48:24.826Z","avatar_url":"https://github.com/oslokommune.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Metadata-api\n============\n\nREST API for managing dataset metadata, versions, editions and distributions.\n\n## Setup\n\n1. Install [Serverless Framework](https://serverless.com/framework/docs/getting-started/)\n2. Setup venv\n```\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -r requirements.txt\n```\n3. Install Serverless plugins: `make init`\n4. Install Python toolchain: `python3 -m pip install (--user) tox black pip-tools`\n   - If running with `--user` flag, add `$HOME/.local/bin` to `$PATH`\n\n\n## Input Validation\nThe input is validated with json schema, see the models under `schema/`\n\n\n## Formatting code\n\nCode is formatted using [black](https://pypi.org/project/black/): `make format`\n\n## Running tests\n\nTests are run using [tox](https://pypi.org/project/tox/): `make test`\n\nFor tests and linting we use [pytest](https://pypi.org/project/pytest/), [flake8](https://pypi.org/project/flake8/) and [black](https://pypi.org/project/black/).\n\n## Deploy\n\nDeploy to both dev and prod is automatic via GitHub Actions on push to main. You\ncan alternatively deploy from local machine with: `make deploy` or `make\ndeploy-prod`.\n\n# Concept\nThe metadata API is structured around the following base concept - the `dataset`:\n```\n+-- dataset-id=my-dataset\n|   +-- version=1\n|       +-- edition=20190101T105900\n|           +-- distribution=filename.txt\n|           +-- distribution=foo.txt\n|       +-- edition=20200101T105900\n|          +-- distribution=presentation.md\n|   +-- version=2\n|       +-- edition=20200101T105900\n|           +-- distribution=otherfile.md\n|       +-- edition=20210101T105900\n```\n`dataset/version/edition` - `my-dataset/1/20190101T105900`\n\nEach version and edition keeps a version named `latest` (a reserved name for a version and edition), that always contains the latest version/edition POSTed to that resource, and can be accessed with `datasets/my-dataset/versions/latest` to get the latest version and `datasets/my-dataset/version/1/editions/latest`\n\n\n## API usage\nThe correct schema definition that is used for validation in the examples below: see `schema/*.json`\n\n### Access\n* Create dataset: valid keycloack access token in header: `\"Authorization\": f\"Bearer {accessToken}\"`\n* Create or update version or edition: valid keycloack access token and owner-access to `:dataset-id`\n* List dataset/version/edition: Logged in user\n\n### List all datasets\n\n```\nGET /datasets\n```\nAll available datasets. An optional query parameter `parent_id` is accepted for filtering by parent dataset.\n\n### Create dataset\n\n```\nPOST /datasets\n\n{\n    \"title\": \"Besøksdata gjenbruksstasjoner\",\n    \"description\": \"Sensordata fra tellere på gjenbruksstasjonene\",\n    \"keywords\": [\"avfall\", \"besøkende\", \"gjenbruksstasjon\"],\n    \"frequency\": \"hourly\",\n    \"accessRights\": \"public\",\n    \"privacyLevel\": \"green\",\n    \"objective\": \"Formålsbeskrivelse\",\n    \"contactPoint\": {\n        \"name\": \"Tim\",\n        \"email\": \"tim@oslo.kommune.no\",\n        \"phone\": \"98765432\"\n    },\n    \"publisher\": \"REN\"\n}\n```\nThis will create a dataset with ID=besoksdata-gjenbruksstasjoner, the id is derived from the title of the dataset. If another dataset exists with the same ID, a ID will be created with a random set of characters at the end of the id (eg: besoksdata-gjenbruksstasjoner-5C5uX)\n\n### Update dataset\n\n#### Replace\n\n```\nPUT /datasets/:dataset-id\n\n{\n    \"title\": \"Besøksdata gjenbruksstasjoner oppdatert tittel\",\n    \"description\": \"Sensordata fra tellere på gjenbruksstasjonene\",\n    \"keywords\": [\"avfall\", \"besøkende\", \"gjenbruksstasjon\"],\n    \"frequency\": \"hourly\",\n    \"accessRights\": \"public\",\n    \"privacyLevel\": \"green\",\n    \"objective\": \"Formålsbeskrivelse\",\n    \"contactPoint\": {\n        \"name\": \"Tim\",\n        \"email\": \"tim@oslo.kommune.no\",\n        \"phone\": \"11111111\"\n    },\n    \"publisher\": \"REN\"\n}\n```\nUpdates a single `dataset-id`, replaces old json document\n\n#### Partial update\n\n```\nPATCH /datasets/:dataset-id\n\n{\n    \"title\": \"Besøksdata gjenbruksstasjoner kun oppdatert tittel\"\n}\n```\n\nPartially updates a single `dataset-id`. **Note** that patching is top-level shallow, i.e. updates inside deep structure values will behave as a PUT.\n\nE.g. `phone` must be supplied in the following PATCH, even though we are just changing `name` and `email`. If `phone` was not supplied, it would be removed.\n\n```\nPATCH /datasets/:dataset-id\n\n{\n    \"contactPoint\": {\n        \"name\": \"Kim\",\n        \"email\": \"kim@oslo.kommune.no\",\n        \"phone\": \"11111111\"\n    }\n}\n```\n\n### Get a single dataset\n\n```\nGET /datsets/:dataset-id\n```\n\n### Create version for a dataset\n\n```\nPOST /datasets/:dataset-id/versions\n\n{\n    \"version\": \"1\",\n    \"schema\": {},\n    \"transformation\": {}\n}\n```\n`version` will become  `:version-id` in the examples below\n\n### Update version\n\n```\nPUT /datasets/:dataset-id/versions/:version-id\n\n{\n    \"version\": \"1\",\n    \"schema\": {},\n    \"transformation\": {}\n}\n```\nUpdates a single `version-id`, replaces old json document, `version` key must maintain same value as `:version-id`\n\n### Get a version\n\n```\nGET /datasets/:dataset-id/versions/:version-id\n```\n\n### Get latest version\n\n```\nGET /datasets/:dataset-id/versions/latest\n```\nGet the latest version created on `dataset-id`\n\n### Create new edition\n\n```\nPOST /datasets/:dataset-id/versions/:version-id/editions\n\n{\n    \"description\": \"Data for one hour\",\n    \"startTime\": \"2018-12-21T08:00:00+01:00\", // inclusive\n    \"endTime\": \"2018-12-21T09:00:00+01:00\"    // exclusive\n}\n```\n\n### Get edition\n\n```\nGET /datasets/:dataset-id/versions/:version-id\n```\n\n### Get latest edition\n\n```\nGET /datasets/:dataset-id/versions/:version-id/latest\n```\nGet the latest edition created on `:version-id`\n\n### Create new distribution\n\n```\nPOST /datasets/:dataset-id/versions/:version-id/editions/:edition-id/distributions\n\n{\n    \"filename\": \"visitors.csv\",\n    \"format\": \"text/csv\",\n    \"checksum\": \"...\"\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslokommune%2Fokdata-metadata-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foslokommune%2Fokdata-metadata-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foslokommune%2Fokdata-metadata-api/lists"}