{"id":51002665,"url":"https://github.com/databio/refgetstore-node-demo","last_synced_at":"2026-06-20T16:32:45.526Z","repository":{"id":351169075,"uuid":"1209757015","full_name":"databio/refgetstore-node-demo","owner":"databio","description":"Lightweight Node.js refget + seqcol API server backed by RefgetStore","archived":false,"fork":false,"pushed_at":"2026-06-19T13:01:26.000Z","size":28,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-19T14:29:38.662Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/databio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-13T18:47:46.000Z","updated_at":"2026-06-19T13:01:39.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/databio/refgetstore-node-demo","commit_stats":null,"previous_names":["databio/refgetstore-node-demo"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/databio/refgetstore-node-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databio%2Frefgetstore-node-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databio%2Frefgetstore-node-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databio%2Frefgetstore-node-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databio%2Frefgetstore-node-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/databio","download_url":"https://codeload.github.com/databio/refgetstore-node-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/databio%2Frefgetstore-node-demo/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34578089,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-20T02:00:06.407Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-20T16:32:42.522Z","updated_at":"2026-06-20T16:32:45.519Z","avatar_url":"https://github.com/databio.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RefgetStore Node Server\n\nA lightweight Node.js **proxy** for GA4GH refget sequences and sequence collections APIs, backed by a [RefgetStore](https://refgenie.org/refget/refgetstore/). The server never materializes sequence bytes in memory — it either redirects raw-store bytes to the backing store or stream-decodes encoded-store bytes directly to the HTTP response.\n\n## Quick Start\n\n```bash\nnpm install\nnpm run build\n\n# Run the demo (builds a store from test FASTAs and starts the server)\nbash demo_up.sh\n```\n\n## Live Demo\n\nA demo server backed by the pangenome jungle RefgetStore runs at:\n\n**http://ecs.databio.org:8150/**\n\nExample links:\n\n- [Service info](http://ecs.databio.org:8150/service-info)\n- [List collections](http://ecs.databio.org:8150/collection)\n- [Get a collection](http://ecs.databio.org:8150/collection/-Ffl-8v7R0Wh53_pRA4WtKoDQL9GmC-v)\n\n## How it works\n\nThe server proxies sequence bytes in one of two ways, depending on how the backing RefgetStore is stored:\n\n- **Redirect (Raw-mode stores).** The server returns `302` with a `Location` header pointing at `\u003cREFGET_STORE_URL\u003e/sequences/\u003cdigest[0:2]\u003e/\u003cdigest\u003e.seq`. Clients follow the redirect and hit the backing store (typically S3) directly. Range headers on the original request flow through to the backing store, which responds with `206 Partial Content`. The server never loads bytes. Query-param partials (`?start=\u0026end=`) are rejected by default — use the `Range` header.\n- **Stream-decode (Encoded-mode stores).** Stored bytes are 2-bit/3-bit packed; they cannot be redirected verbatim. The server calls `RefgetStore.streamSequence(digest, start, end)` which returns a `Readable` of decoded ASCII bases, piped directly to the HTTP response. Memory use is bounded by the stream's internal buffer regardless of sequence size.\n\n### Proxy mode matrix\n\n| Store mode | `REFGET_PROXY_MODE=auto` | `redirect-only` | `stream-only` |\n|---|---|---|---|\n| Raw | redirect (302) | redirect (302) | stream (decode is a no-op) |\n| Encoded | stream | startup error | stream |\n\n## Configuration\n\n| Env var | Default | Description |\n|---|---|---|\n| `REFGET_STORE_URL` | — | URL to a remote RefgetStore (S3 / HTTP). Required for redirect mode. |\n| `REFGET_STORE_PATH` | — | Path to a local RefgetStore dir. Forces `stream-only` mode. |\n| `REFGET_CACHE_PATH` | `/tmp/refgetstore_cache` | Metadata cache for remote stores. |\n| `REFGET_PROXY_MODE` | `auto` | `auto` (redirect Raw, stream Encoded), `redirect-only`, `stream-only`. |\n| `REFGET_ALLOW_QUERY_PARAM_PARTIALS` | `false` | When true, `?start=\u0026end=` in redirect mode fall through to streaming instead of 400. |\n| `PORT` | `3000` | HTTP port. |\n\nExactly one of `REFGET_STORE_URL` or `REFGET_STORE_PATH` must be set.\n\n## API Endpoints\n\n### Service Info\n\n| Endpoint | Description |\n|---|---|\n| `GET /service-info` | GA4GH service-info with store statistics |\n\n### Refget Sequences (GA4GH refget v2)\n\n| Endpoint | Description |\n|---|---|\n| `GET /sequence` | List all sequences (disabled for stores with \u003e 10,000 sequences) |\n| `GET /sequence/:digest` | Retrieve sequence bases (302 redirect or streaming, depending on proxy mode). Supports `Range` header; `?start=\u0026end=` accepted in stream mode. |\n| `GET /sequence/:digest/metadata` | Sequence metadata (length, md5, ga4gh digest) |\n| `GET /sequence/service-info` | Refget service capabilities |\n\n### Sequence Collections (GA4GH seqcol)\n\n| Endpoint | Description |\n|---|---|\n| `GET /collection` | List all collections |\n| `GET /collection/:digest` | Collection metadata |\n| `GET /collection/:digest/metadata` | Collection metadata (explicit) |\n\n## Building a Store from FASTA Files\n\n```bash\nnode scripts/build_store.mjs --fasta path/to/genome.fa --output my_store\nREFGET_STORE_PATH=my_store REFGET_PROXY_MODE=stream-only npm start\n```\n\n## Development (local-linked `@databio/gtars-node`)\n\nUntil `@databio/gtars-node` is published with `streamSequence`, link to a local build:\n\n```bash\n# In the gtars repo\ncd repos/gtars/gtars-node\nnpm run build\nnpm link\n\n# In this repo\ncd repos/refgetstore-node-demo\nnpm link @databio/gtars-node\nnpm run dev\n```\n\n## Docker\n\n```bash\n# Build\ndocker build -f deployment/dockerhub/Dockerfile -t refgetstore-server .\n\n# Run (redirect-mode example)\ndocker run -p 80:80 \\\n  -e REFGET_STORE_URL=https://my-bucket.s3.amazonaws.com/refget/store \\\n  refgetstore-server\n```\n\n## Comparison to seqcolapi\n\n| | seqcolapi | refgetstore-server |\n|---|---|---|\n| Runtime | Python + FastAPI | Node.js + Hono |\n| Storage | PostgreSQL | RefgetStore (flat files, local or S3) |\n| Infrastructure | Database server required | Single binary store on disk / object store |\n| Sequence delivery | Reads DB, builds response in Python | Redirect or stream-decode; no bytes buffered |\n\n## Known Limitations\n\n- No comparison endpoint (`/comparison/:digest1/:digest2`) — pending napi binding support\n- Read-only: store must be pre-built from FASTA files\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabio%2Frefgetstore-node-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdatabio%2Frefgetstore-node-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdatabio%2Frefgetstore-node-demo/lists"}