{"id":35109894,"url":"https://github.com/akvaplan-niva/dois","last_synced_at":"2026-04-20T23:31:58.747Z","repository":{"id":45114592,"uuid":"394984202","full_name":"akvaplan-niva/dois","owner":"akvaplan-niva","description":"Akvaplan-niva DOI register","archived":false,"fork":false,"pushed_at":"2024-04-16T16:20:21.000Z","size":1658,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-08T17:41:27.722Z","etag":null,"topics":["crossref","crossref-api","deno","doi","metadata","openalex","unpaywall"],"latest_commit_sha":null,"homepage":"https://dois.deno.dev","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akvaplan-niva.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-08-11T12:39:37.000Z","updated_at":"2025-02-03T15:01:13.000Z","dependencies_parsed_at":"2023-09-29T16:21:50.904Z","dependency_job_id":"98d61dfb-74e0-496e-9232-fb741144c0d2","html_url":"https://github.com/akvaplan-niva/dois","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/akvaplan-niva/dois","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akvaplan-niva%2Fdois","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akvaplan-niva%2Fdois/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akvaplan-niva%2Fdois/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akvaplan-niva%2Fdois/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akvaplan-niva","download_url":"https://codeload.github.com/akvaplan-niva/dois/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akvaplan-niva%2Fdois/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32070627,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-20T21:26:33.338Z","status":"ssl_error","status_checked_at":"2026-04-20T21:26:22.081Z","response_time":94,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crossref","crossref-api","deno","doi","metadata","openalex","unpaywall"],"created_at":"2025-12-27T19:28:55.362Z","updated_at":"2026-04-20T23:31:58.733Z","avatar_url":"https://github.com/akvaplan-niva.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Akvaplan-niva DOIs\n\nThis [git repository](https://github.com/akvaplan-niva/dois) contains metadata for Akvaplan-niva publications with a DOI.\n\n## Service\nThe [Deno Deploy](https://deno.com/deploy) service https://dois.deno.dev/ is connected to `main` on this repository and is the data source of https://akvaplan.no/en/publications\n\n## Pipeline\nThe metadata is collected by an automated [pipeline](bin/doi-pipeline) and stored as NDJSON in the `slim` directory.\n\nThe pipeline finds Akvaplan-niva publications in Crossref, CRISTIN, and OpenAlex.\n\n## Use\n\n### Run pipeline\n```sh\n./bin/doi-pipeline\n```\n### Add DOIs (to local disk)\nManually add DOIs by adding/editing a NDJSON file in `doi/*/*.ndjson` and run pipeline.\n\n### Update service data \nInspect and push approved changes in `slim/*.ndjson`.\n\nUpdate the KV store in the data service: \n```sh\n$ curl --netrc -XPOST https://dois.deno.dev/ingest\n```\n\n```json\n{\"ingested\":1669,\"total\":1669,\"elapsed\":198.636,\"start\":\"2023-07-11T13:16:13.589Z\",\"end\":\"2023-07-11T13:19:32.225Z\",\"ok\":true}\n```\n\n## Pipeline details\n\nThe DOI pipeline consists of the following steps:\n\n### 1. Create DOI list\n\nFirst, create list of unique DOIs\n\n- Extract DOIs from [`raw`](raw) text references\n- Find DOIs in Crossref, CRISTIN, and OpenAlex\n- Add these into the NDJSON-formatted DOIs in [`doi`](doi)\n- De-deplicate the DOIs\n\n(The raw text references were augmented with DOIs from Crossref's [SimpleTextQuery](https://apps.crossref.org/SimpleTextQuery), when missing in in the original source.)\n\n### 2. Fetch metadata\n\nFor each DOI\n\n- Fetch \"works\" metadata from [Crossref API](https://api.crossref.org/)\n- Create slim metadata from Crossref works\n- Fetch PDF location from [Unpaywall API](https://unpaywall.org/products/api)\n\n\u003e Notice: The pipeline aggressively caches all HTTP responses, calling the APIs just once per DOI across all invocations. On linux, the cache is located in `$HOME/.cache/deno/https/api.*.org`.\n\n### 3. Create slim metadata product\n\nFinally\n\n- Add PDF URL to slim metadata\n- Partition slim metadata, creating one file per year in `slim`\n- Show summary counts\n- Verify SHA checksums\n\n## Prerequisites\n\n[Deno](https://deno.land) and [newline](https://deno.land/x/newline@v0.1.0) command line tools.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakvaplan-niva%2Fdois","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakvaplan-niva%2Fdois","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakvaplan-niva%2Fdois/lists"}