{"id":51054587,"url":"https://github.com/clidey/connparse","last_synced_at":"2026-06-22T20:01:42.451Z","repository":{"id":358650687,"uuid":"1238773780","full_name":"clidey/connparse","owner":"clidey","description":"Connparse parses database connection strings, DSNs, URLs, file paths, and cloud storage URIs into one safe structured object.","archived":false,"fork":false,"pushed_at":"2026-05-25T16:57:17.000Z","size":309,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-25T18:32:29.601Z","etag":null,"topics":["cloudstorage","data-source","database","dsn","golang","java","javascript","nosql","parse","postgres","python","rust","sql","yaml"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clidey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-05-14T12:46:59.000Z","updated_at":"2026-05-25T16:57:14.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/clidey/connparse","commit_stats":null,"previous_names":["clidey/connparse"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/clidey/connparse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clidey%2Fconnparse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clidey%2Fconnparse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clidey%2Fconnparse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clidey%2Fconnparse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clidey","download_url":"https://codeload.github.com/clidey/connparse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clidey%2Fconnparse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34663524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloudstorage","data-source","database","dsn","golang","java","javascript","nosql","parse","postgres","python","rust","sql","yaml"],"created_at":"2026-06-22T20:01:41.043Z","updated_at":"2026-06-22T20:01:42.422Z","avatar_url":"https://github.com/clidey.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/connparse-icon.svg\" width=\"96\" height=\"96\" alt=\"Connparse icon\"\u003e\n\u003c/p\u003e\n\n# Connparse\n\nConnparse parses database connection strings, DSNs, URLs, file paths, and cloud\nstorage URIs into one safe structured object. It supports PostgreSQL, MySQL,\nMariaDB, SQLite, DuckDB, ClickHouse, Redis, Memcached, Elasticsearch, MongoDB,\nCockroachDB, QuestDB, YugabyteDB, TiDB, Valkey, Dragonfly, OpenSearch,\nFerretDB, ElastiCache, DocumentDB, SQL Server, Oracle, Snowflake, Cassandra,\nBigQuery, Redshift, Aurora, Neo4j, Trino, Databricks, DynamoDB, StarRocks,\nSAP HANA, Athena, Spanner, Google Cloud Storage, Azure Blob, Azure Data Lake\nStorage, Azure Files, Azure Cosmos DB, Business Central, TallyPrime, Amazon S3,\nand local file paths.\n\nThis repository contains JavaScript, Go, Python, Rust, and Java implementations,\nshared Connparse Definition Specification (CPDS) YAML definitions, and shared\ncompatibility fixtures.\n\n## Packages\n\n- npm: `@clidey/connparse`\n- Go: `github.com/clidey/connparse/packages/go`\n- Python: `connparse`\n- Rust: `connparse`\n- Java: `com.clidey:connparse`\n\n## Repository Layout\n\n```text\nspecs/\n  definitions/   Shared Connparse Definition Specification (CPDS) YAML definitions\n  fixtures/      Cross-implementation compatibility fixtures\n  schemas/       JSON Schemas for definitions, fixtures, and outputs\n  docs/          Reference and porting docs\n\npackages/\n  js/            JavaScript/npm implementation\n  go/            Go implementation\n  python/        Python implementation\n  rust/          Rust crate implementation\n  java/          Java implementation\n```\n\n## Install\n\nThe JavaScript package has one runtime dependency: `yaml`, used to load CPDS\ndefinition files. The Go package uses `github.com/goccy/go-yaml` for the same\nCPDS loader API. The Python and Java parsers use the standard library at\nruntime. The Rust crate uses `serde` and `serde_json` for the shared\nJSON-shaped contract.\n\n```bash\nnpm install @clidey/connparse\n```\n\nFor local development in this repo:\n\n```bash\npnpm install\npnpm verify:definitions\npnpm conformance\npnpm generate:definitions\npnpm check:versions\npnpm test\npnpm test:go\npnpm test:python\npnpm test:rust\npnpm test:java\npnpm check:package\n```\n\n## Quick Start\n\n```js\nimport { parse } from '@clidey/connparse';\n\nconst result = parse('postgres://user:pass@localhost:5432/app?sslmode=require');\n\nif (!result.ok) {\n  console.error(result.errors);\n} else {\n  console.log(result.value);\n}\n```\n\nOutput:\n\n```json\n{\n  \"scheme\": \"postgres\",\n  \"type\": \"database\",\n  \"authority\": {\n    \"host\": \"localhost\",\n    \"port\": 5432\n  },\n  \"resource\": {\n    \"type\": \"database\",\n    \"name\": \"app\"\n  },\n  \"path\": \"\",\n  \"query\": {\n    \"sslmode\": \"require\"\n  },\n  \"fragment\": null,\n  \"credentials\": {\n    \"username\": \"user\",\n    \"password\": \"pass\"\n  },\n  \"options\": {},\n  \"raw\": \"postgres://user:pass@localhost:5432/app?sslmode=require\",\n  \"safe\": \"postgres://user:***@localhost:5432/app?sslmode=require\"\n}\n```\n\n## Supported Providers\n\nCurrent built-in providers include:\n\n- Core databases and stores: PostgreSQL, MySQL, MariaDB, SQLite, DuckDB,\n  ClickHouse, Redis, Memcached, Elasticsearch, MongoDB, CockroachDB, QuestDB,\n  YugabyteDB, TiDB, Valkey, Dragonfly, OpenSearch, FerretDB, ElastiCache, and\n  DocumentDB.\n- Enterprise and cloud systems: H2, IBM Db2, Trino, Databricks, Neo4j,\n  Memgraph, Microsoft SQL Server, Dynamics NAV, Oracle, GaussDB, Snowflake,\n  DynamoDB, Cassandra, StarRocks, SAP HANA, BigQuery, Spanner, Athena,\n  Redshift, Aurora PostgreSQL, Aurora MySQL, MemoryDB, CrateDB, SingleStore,\n  OceanBase, RisingWave, Materialize, Apache Doris, Azure Managed Redis,\n  Azure Managed Cassandra, Firestore, Timestream, Neptune, Azure Cosmos DB,\n  Business Central, and TallyPrime.\n- Object/file sources: Amazon S3, Google Cloud Storage, Azure Blob, Azure Data\n  Lake Storage, Azure Files, common HTTPS object-storage URLs, and local or\n  `file:` paths.\n\nThe exact scheme list and parser rules live in `specs/definitions/*.yaml`.\n\n## API\n\n### `parse(input, options?)`\n\nReturns a result object:\n\n```ts\ntype ParseResult = {\n  ok: boolean;\n  value: ConnparseAddress | null;\n  errors: ConnparseDiagnostic[];\n  warnings: ConnparseDiagnostic[];\n};\n```\n\nBy default Connparse is permissive: unknown query parameters become warnings.\nUse strict mode to turn them into errors:\n\n```js\nparse('postgres://localhost/app?unexpected=1', { strict: true });\n```\n\nSome common formats require a provider hint because they do not identify their\nprovider:\n\n```js\nparse('host=db.example.com dbname=app user=alice', { provider: 'postgres' });\nparse('https://es.example.com:9200/logs', { provider: 'elasticsearch' });\nparse('https://clickhouse.example.com:8443/default', { provider: 'clickhouse' });\n```\n\n### `parseOrThrow(input, options?)`\n\nReturns a `ConnparseAddress` or throws an `Error`.\n\n### `parseNormalize(input, options?)`\n\nReturns a `ParseResult`, but `value` is a stable normalized object instead of a\nfaithful raw parse object. Equivalent inputs produce the same normalized JSON.\nIt also includes an optional `semantic` block with provider-normalized field\nvalues that consumers can map onto forms without re-implementing provider\naliases.\n\nUse `parse()` when you need to preserve the exact original input in `raw`. Use\n`parseNormalize()` when you need dedupe keys, config comparison, cache keys, or\nstable UI state.\n\n```js\nparseNormalize('postgresql://user:pass@LOCALHOST:5432/app?sslmode=require\u0026application_name=myapp');\nparseNormalize('postgres://localhost/app?application_name=myapp\u0026sslmode=require');\n```\n\nBoth return the same normalized `value`:\n\n```json\n{\n  \"scheme\": \"postgres\",\n  \"type\": \"database\",\n  \"authority\": {\n    \"host\": \"localhost\",\n    \"port\": null\n  },\n  \"resource\": {\n    \"type\": \"database\",\n    \"name\": \"app\"\n  },\n  \"path\": \"\",\n  \"query\": {\n    \"application_name\": \"myapp\",\n    \"sslmode\": \"require\"\n  },\n  \"fragment\": null,\n  \"credentials\": {},\n  \"options\": {},\n  \"raw\": \"postgres://localhost/app?application_name=myapp\u0026sslmode=require\",\n  \"safe\": \"postgres://localhost/app?application_name=myapp\u0026sslmode=require\",\n  \"canonical\": \"postgres://localhost/app?application_name=myapp\u0026sslmode=require\",\n  \"semantic\": {\n    \"provider\": \"postgres\",\n    \"fields\": {\n      \"ssl_mode\": \"required\"\n    },\n    \"consumed\": {\n      \"query\": [\"sslmode\"]\n    }\n  }\n}\n```\n\n### `canonicalize(input, options?)`\n\nReturns a stable string identity for a connection address. Canonicalization\nnormalizes scheme aliases, removes default ports, sorts query parameters, and\nnormalizes typed query values where the CPDS definition declares the type.\n\nCanonical strings are safe by default: URI credentials are omitted, and\nCPDS-declared sensitive query values are replaced with `***`.\n\n```js\ncanonicalize('postgresql://user:pass@LOCALHOST:5432/app?sslmode=require\u0026application_name=myapp');\n// postgres://localhost/app?application_name=myapp\u0026sslmode=require\n\ncanonicalize('postgres://user:pass@localhost/app?sslkey=/tmp/client.key\u0026sslmode=require');\n// postgres://localhost/app?sslkey=***\u0026sslmode=require\n```\n\nUse explicit options only when the caller intentionally needs secret-inclusive\nidentity strings:\n\n```js\ncanonicalize('postgres://user:pass@localhost/app?sslkey=/tmp/client.key', {\n  includeCredentials: true,\n  includeSensitive: true\n});\n// postgres://user:pass@localhost/app?sslkey=%2Ftmp%2Fclient.key\n```\n\n### `equivalent(left, right, options?)`\n\nCompares two inputs by canonical identity:\n\n```js\nequivalent(\n  'postgresql://localhost:5432/app?sslmode=require\u0026application_name=myapp',\n  'postgres://localhost/app?application_name=myapp\u0026sslmode=require'\n);\n// true\n```\n\n### `mask(input, definition?)`\n\nRedacts URI userinfo passwords from a raw connection string. Query parameters\nand key/value fields are redacted only when the matched CPDS definition declares\nthem in `redaction.sensitive_keys`.\n\n```js\nconst mysql = defaultRegistry.getById('mysql');\n\nmask('mysql://root:secret@localhost/shop', mysql);\n// mysql://root:***@localhost/shop\n\nmask('mysql://root:secret@localhost/shop?ssl-key=/tmp/client.key', mysql);\n// mysql://root:***@localhost/shop?ssl-key=***\n```\n\nThe CLI uses this same spec-driven rule for `safe` output. It does not guess\nthat arbitrary query keys are secret unless the provider definition says so.\n\n### `parseDefinition(input, format?)`\n\nLoads a CPDS definition from JSON or YAML:\n\n```js\nimport { parse, parseDefinition } from '@clidey/connparse';\n\nconst definition = parseDefinition(`\nid: warehouse\nname: Warehouse\ntype: analytics\nschemes:\n  - warehouse\nadapter: generic-uri\nauthority:\n  host: true\nresource:\n  type: database\n  required: true\npath:\n  type: object_path\n  required: false\nquery_parameters: {}\nvalidation:\n  require_host: true\n`);\n\nconst result = parse('warehouse://example.com/main/schema', {\n  definitions: [definition]\n});\n```\n\nCustom definitions are merged with the built-ins. If a custom definition uses an\nexisting scheme, it overrides that scheme for the parse call.\n\n## Connparse Address Shape\n\nEvery successful parse returns this top-level shape:\n\n```ts\ntype ConnparseAddress = {\n  scheme: string;\n  type:\n    | 'database'\n    | 'object_storage'\n    | 'file'\n    | 'stream'\n    | 'cache'\n    | 'analytics'\n    | 'api'\n    | 'unknown';\n  authority: Record\u003cstring, unknown\u003e;\n  resource: {\n    type: string;\n    name: string | null;\n  };\n  path: string;\n  query: Record\u003cstring, string | string[]\u003e;\n  fragment: string | null;\n  credentials: Record\u003cstring, string\u003e;\n  options: Record\u003cstring, unknown\u003e;\n  raw: string;\n  safe: string;\n};\n```\n\nCredentials are intentionally separated from `authority` and the `safe` field is\nintended for logs and UI. Do not log `credentials` by default.\n\nFor the full list of keys, provider-specific fields, diagnostics, CPDS keys,\nadapter names, and fixture format, see [specs/docs/reference.md](specs/docs/reference.md).\n\n## Connparse Definition Specification (CPDS)\n\nCPDS files describe source-specific behavior without putting all provider rules\ndirectly in the parser.\n\nExample:\n\n```yaml\nid: postgres\nname: PostgreSQL\ntype: database\nschemes:\n  - postgres\n  - postgresql\nadapter: generic-uri\ndefaults:\n  port: 5432\nauthority:\n  host: true\n  port: true\n  multi_host: true\nresource:\n  type: database\n  required: true\npath:\n  type: object_path\n  required: false\ncredentials:\n  username: true\n  password: true\nquery_parameters:\n  sslmode:\n    type: string\n    allowed: [disable, allow, prefer, require, verify-ca, verify-full]\nvalidation:\n  require_host: true\n  port_range:\n    min: 1\n    max: 65535\nredaction:\n  safe_credentials: [username]\n  sensitive_keys: [password, sslkey, sslcert, sslrootcert]\n```\n\nThe definition language is intentionally small. It handles schemes, provider\ntype, adapter selection, defaults, resource/path rules, query parameter typing,\nallowed values, validation, and provider-specific redaction keys.\n\nProvider-specific structural parsing still lives in adapters where real-world\nformats need it, such as MongoDB SRV URLs, PostgreSQL-compatible conninfo,\nQuestDB ILP config strings, JDBC URLs, and SQLite/DuckDB memory databases.\n\nBuilt-in definitions are generated from `specs/definitions/*.yaml`:\n\n```bash\npnpm verify:definitions\npnpm verify:definitions:strict\npnpm generate:definitions\npnpm check:generated\n```\n\n`verify:definitions` parses every CPDS YAML file and fails on hard schema\nerrors such as missing required keys, invalid ports, duplicate schemes, or bad\nquery/redaction shapes. It also prints suggestions, including missing\n`redaction` for definitions that declare credentials. `verify:definitions:strict`\ntreats those suggestions as failures and is part of `pnpm run check`.\n\nDo not edit generated files directly:\n\n- `packages/js/src/builtin-definitions.js`\n- `packages/go/builtin_definitions.go`\n- `packages/python/src/connparse/builtin_definitions.py`\n- `packages/rust/src/builtin_definitions.rs`\n\n## Fixtures\n\nThe shared compatibility contract lives in `specs/fixtures/compatibility.json`.\n\nThe fixtures are used by the test suite, but they are not only test data. They\nare the portable behavior contract across implementations. A port should be able\nto run the same fixture file and produce the same observable fields.\n\nFixture example:\n\n```json\n{\n  \"name\": \"postgres basic auth\",\n  \"input\": \"postgres://user:pass@localhost:5432/app?sslmode=require\",\n  \"expected\": {\n    \"authority.host\": \"localhost\",\n    \"resource.name\": \"app\",\n    \"safe\": \"postgres://user:***@localhost:5432/app?sslmode=require\"\n  }\n}\n```\n\nRun the fixture suite:\n\n```bash\npnpm test\npnpm conformance\n```\n\nThe Go, Python, and Rust packages also read this same fixture file. See\n[specs/docs/porting.md](specs/docs/porting.md) for the porting contract and the\ngenerator boundary used to keep language implementations aligned.\n\n## Schemas\n\nJSON Schemas live in `specs/schemas/`:\n\n- `cpds.schema.json`\n- `address.schema.json`\n- `normalized-address.schema.json`\n- `parse-result.schema.json`\n- `fixture.schema.json`\n\nCheck them with:\n\n```bash\npnpm check:schemas\n```\n\n## Package Consumption Checks\n\nThe package consumption check packs the npm package, imports it from a temporary\nproject, runs the packed CLI, verifies a temporary Go module with a local\n`replace` directive, verifies a Python import via `PYTHONPATH`, and verifies a\ntemporary Rust crate with a local path dependency.\n\n```bash\npnpm check:package\n```\n\nRelease instructions live in [RELEASING.md](RELEASING.md).\n\n## CLI\n\n```bash\nconnparse 'postgres://user:pass@localhost/app'\nconnparse --safe 'postgres://user:pass@localhost/app'\nconnparse --include-secrets 'postgres://user:pass@localhost/app'\nconnparse --strict 'postgres://localhost/app?unknown=1'\nconnparse --provider postgres 'host=db.example.com dbname=app user=alice'\n```\n\nThe CLI redacts JSON output by default: credential presence is preserved, but\nsensitive credential/query/option values declared by the provider CPDS file are\nreplaced with `***`, and `raw` is replaced with `safe`. Use `--include-secrets`\nonly when you intentionally need the full parse result.\n\n## Boundaries\n\nConnparse parses and normalizes address strings. It does not perform network\nchecks, open sockets, authenticate credentials, infer table schemas, or generate\nUI forms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclidey%2Fconnparse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclidey%2Fconnparse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclidey%2Fconnparse/lists"}