{"id":51191357,"url":"https://github.com/heliosdatabase/any2heliosdb","last_synced_at":"2026-06-29T17:00:58.573Z","repository":{"id":367475448,"uuid":"1278374129","full_name":"HeliosDatabase/Any2HeliosDB","owner":"HeliosDatabase","description":"Migration toolkit for moving Oracle, MySQL, PostgreSQL, and SQL Server into HeliosDB Nano/Lite/Full or stock PostgreSQL.","archived":false,"fork":false,"pushed_at":"2026-06-28T16:13:59.000Z","size":491,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-28T16:23:41.656Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HeliosDatabase.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"docs/roadmap/README.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-23T18:21:32.000Z","updated_at":"2026-06-28T16:13:28.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/HeliosDatabase/Any2HeliosDB","commit_stats":null,"previous_names":["heliosdatabase/any2heliosdb"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/HeliosDatabase/Any2HeliosDB","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HeliosDatabase%2FAny2HeliosDB","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HeliosDatabase%2FAny2HeliosDB/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HeliosDatabase%2FAny2HeliosDB/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HeliosDatabase%2FAny2HeliosDB/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HeliosDatabase","download_url":"https://codeload.github.com/HeliosDatabase/Any2HeliosDB/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HeliosDatabase%2FAny2HeliosDB/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34935530,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-29T02:00:05.398Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-27T15:30:40.684Z","updated_at":"2026-06-29T17:00:58.265Z","avatar_url":"https://github.com/HeliosDatabase.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Any2HeliosDB\n\n**Migrate Oracle, MySQL, PostgreSQL, and SQL Server into HeliosDB — Nano, Lite, or\nFull — or into stock PostgreSQL.** A modern, Apache-2.0, clean-room Python\nsuccessor to [Ora2Pg](https://github.com/darold/ora2pg): an interactive setup\nwizard, a parallel + resumable data load, a live full-screen migration monitor,\nstructural and row-level validation, a GoldenGate-style change-data-capture (CDC)\nengine, and an MCP server — all driving\nthe target over the PostgreSQL wire protocol (with an experimental Oracle-wire\npath).\n\nThe CLI is **`a2h`**. Sources: **Oracle, MySQL, PostgreSQL, and SQL Server** —\nall validated end-to-end. Targets: **HeliosDB** (Nano/Lite/Full) **or stock\nPostgreSQL** through the one `psycopg` PostgreSQL-wire driver — so a2h is also a\nstraight Oracle/MySQL/SQL-Server → **PostgreSQL** migrator (validated Oracle →\nPostgreSQL 16 incl. sequences) — plus a **MySQL** target for heterogeneous /\nmigrate-back flows.\n\n```bash\npip install any2heliosdb[oracle]   # add [mysql] / [mssql] / [mcp] / [all] as needed\na2h wizard            # connect, smoke-test, write config.toml\na2h migrate -c config.toml\na2h monitor -c config.toml          # (optional, in another terminal) watch it live\na2h test-count -c config.toml \u0026\u0026 a2h test-data -c config.toml\n```\n\n---\n\n## Design principle: a thin translation layer over a runtime probe\n\n\u003e a2h keeps the translation layer thin by adapting to what the **target actually\n\u003e supports**, discovered at runtime, instead of hard-coding per-edition behavior.\n\nThis is operational, not aspirational. At connect time a **capability probe**\n([`target/capability.py`](src/any2heliosdb/target/capability.py)) asks the live\nserver what it actually accepts — COPY, `ON CONFLICT`, `RETURNING`, `MERGE`,\nPL/pgSQL, CHECK/FK enforcement — and the emitters translate *only* what this\ntarget cannot take, degrading gracefully for the rest. Editions are never assumed\nfrom a version string. See\n[docs/heliosdb-compatibility.md](docs/heliosdb-compatibility.md) for the supported\neditions, minimum versions, and how graceful degradation works.\n\n---\n\n## Install\n\n```bash\npip install any2heliosdb            # core: psycopg (PG-wire) target + CLI\npip install any2heliosdb[oracle]    # + Oracle source (oracledb)\n# extras: [mysql]  [mssql]  [mcp]  [all]   (combine, e.g. [oracle,mcp])\n```\n\nThe `psycopg` PG-wire target is in core, so a source driver is the only extra you\nusually need: `[oracle]` (oracledb), `[mysql]` (PyMySQL; `[mysql-cdc]` adds binlog\nCDC), `[mssql]` (pyodbc + an ODBC driver), `[mcp]` (the MCP server SDK), or\n`[all]`. Python 3.9+; core deps are light (`psycopg[binary]`, `typer`, `rich`,\n`jinja2`, `tomli`/`tomli-w`). Verify your environment with `a2h doctor`.\n\n\u003e Developing on a checkout? Use an editable install instead:\n\u003e `pip install -e \".[all,dev]\"`.\n\n---\n\n## 60-second quick start\n\n```bash\n# 1. Point a password at an env var (never store it in the config).\nexport ORACLE_PW='hr'\n\n# 2. Interactive setup: connects both ends, detects the edition, probes\n#    capabilities, round-trips a tiny COPY to prove NULL-vs-empty-string fidelity,\n#    and writes config.toml.\na2h wizard\n\n# 3. (optional) Inventory, type mapping and a cost estimate before you commit.\na2h assess -c config.toml\n\n# 4. Schema + data, parallel and resumable.\na2h migrate -c config.toml\n#   migrated 2 tables, 8 rows (load_mode=copy)\n\n# 5. Validate. Non-zero exit on any mismatch, so this gates CI.\na2h test-count -c config.toml      # row-count parity\na2h test-data  -c config.toml      # PK-ordered, per-row checksum compare\n\n# 6. If a load was interrupted, continue it with no duplicates.\na2h status -c config.toml\na2h resume -c config.toml\n```\n\n### Change data capture (v1)\n\n```bash\na2h extract  cdc1 -c config.toml   # capture source changes into a durable trail\na2h replicat cdc1 -c config.toml   # apply the trail to the target (idempotent)\na2h extracts      -c config.toml   # list extracts + capture/apply positions\n```\n\nOracle capture is **SCN-watermark** — a full snapshot on the first cycle, then\nincremental (`ORA_ROWSCN`) after. **MySQL** capture is **log-based binlog** (real\nI/U/D, including deletes). Apply is an idempotent upsert\n(`INSERT … ON CONFLICT … DO UPDATE`), so re-running a trail slice never\nduplicates. Oracle LogMiner and SQL Server CDC are on the\n[roadmap](docs/cdc.md), built on the same Extract → trail → Replicat spine, and\nheterogeneous \"migrate-back\" sinks (HeliosDB → MySQL) are already supported.\n\n---\n\n## Compatibility matrix\n\n**Legend**\n\n| Mark | Meaning |\n|---|---|\n| ✅ | Validated end-to-end (full integration suite passes against a live server). |\n| ⚠️ | Experimental — code-complete and unit-tested, but its live parity test is blocked. |\n\n### Source dialect × target edition (driver + status)\n\n| Source | HeliosDB-Nano | HeliosDB-Lite | HeliosDB-Full | Stock PostgreSQL |\n|---|---|---|---|---|\n| **Oracle 21c** | `psycopg` ✅ | `psycopg` ✅ · `native` ⚠️ | `psycopg` ✅ · `native` ⚠️ | `psycopg` ✅ |\n| **MySQL 8** | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ |\n| **PostgreSQL 14–16** | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ |\n| **SQL Server 2022** | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ | `psycopg` ✅ |\n\nThe right-most column is **stock PostgreSQL as a target** — the same `psycopg`\ndriver, so a2h doubles as an `Oracle/MySQL/SQL-Server → PostgreSQL` migrator (see\n[migration/to-postgresql.md](docs/migration/to-postgresql.md); Oracle → PostgreSQL\n16 validated end-to-end incl. sequences). **PostgreSQL** as a *source* is\nvalidated against real PostgreSQL (Pagila — 65k rows, `timestamptz`, arrays,\ncomposite PKs, declarative partitions).\n\nSQL Server → HeliosDB is validated end-to-end (migrate + `test`/`test-count`/\n`test-data`): `sys.*` introspection, `BIT`→BOOLEAN, `DATETIME2`→TIMESTAMP,\n`NVARCHAR(MAX)`/`VARBINARY(MAX)` byte-perfect, FK referenced columns resolved.\n(Battle-tested against SQL Server 2022 → HeliosDB-Full; the psycopg path is\nedition-agnostic, as for the other sources.) Needs `pip install -e \".[mssql]\"`\n(pyodbc + an ODBC driver).\n\nMySQL→HeliosDB is validated end-to-end (migrate + `test`/`test-count`/`test-data`)\non all three editions: `information_schema` introspection, `TINYINT(1)`→`BOOLEAN`,\n`''` preserved (MySQL doesn't fold it to NULL), BLOB/unicode intact. CDC capture\nis Oracle-only (SCN-watermark) plus **MySQL binlog** (log-based, real I/U/D); other\nsources' log-based CDC is [roadmap](docs/cdc.md).\n\nMinimum HeliosDB build per edition (full details in\n[docs/heliosdb-compatibility.md](docs/heliosdb-compatibility.md)):\n\n- **Nano** — `heliosdb-nano` **3.58.5** or newer. The tool gates **CDC apply to\n  Nano ≥ 3.58.5** and refuses older builds with a clear error.\n- **Lite** — **2.0** or newer. Migrate, validate, **and CDC apply** are green on a\n  current build; the loader's serial-retry pass handles Lite's lack of concurrent\n  transactions.\n- **Full** — a current `main` build. The full suite — migrate, validate, and CDC —\n  is green. (a2h emits standard `CREATE SEQUENCE`/`nextval` DDL and degrades to a\n  warning on a build that does not yet implement it; table data still migrates.)\n\na2h does not assume capabilities from a version string — the runtime probe\ndiscovers them per connection and the tool degrades gracefully on an older build.\n\nThe `native` (Oracle-wire) driver is **⚠️ experimental** on Lite and Full: it is\ncode-complete and unit-tested, but its live parity test is blocked on a HeliosDB\nOracle-listener TNS-version handshake; use `psycopg` for production work. `native`\ndoes not apply to Nano.\n\n### Feature support per edition (via the `psycopg` driver)\n\n| Capability | HeliosDB-Nano | HeliosDB-Lite | HeliosDB-Full |\n|---|---|---|---|\n| Schema DDL (tables, indexes, FKs, views, sequences³) | ✅ | ✅ | ✅ |\n| Bulk load | ✅ INSERT (no COPY) | ✅ COPY | ✅ COPY |\n| Parallel + resumable load | ✅ | ✅ (serial-retry converges¹) | ✅ |\n| Validation (`test` / `test-count` / `test-data`) | ✅ | ✅ | ✅ |\n| CDC apply (`replicat`) | ✅ (≥ 3.58.5)² | ✅ | ✅ |\n\n¹ Lite rejects concurrent transactions today, so a chunk can fail under parallel\ncontention yet succeed on a serial retry; the loader runs a serial mop-up pass and\nchunks are idempotent, so the load still converges with no duplicates.\n\n² The tool gates CDC apply to **Nano ≥ 3.58.5** and refuses older builds with a\nclear error.\n\n³ Sequences are emitted as standard PostgreSQL DDL. Stock PostgreSQL and\nsequence-supporting HeliosDB builds create them natively; on a build that does not\nyet implement `CREATE SEQUENCE`/`nextval`, a2h emits the DDL and degrades to a\nwarning — table data still migrates.\n\n### Heterogeneous targets \u0026 migrate-back (v2)\n\nBeyond migrating *into* HeliosDB, the same engine runs **any-to-any**: a\nPostgreSQL-wire **source** adapter reads a HeliosDB (or PostgreSQL) server *out*,\nand a **MySQL target** driver writes to MySQL — so data can flow **back out of\nHeliosDB** (the GoldenGate-reverse direction).\n\n| Direction | Driver path | Status |\n|---|---|---|\n| Oracle → MySQL | `oracledb` → `mysql` | ✅ validated (migrate + TEST_COUNT + TEST_DATA) |\n| **HeliosDB → MySQL** (migrate-back) | `postgres` source → `mysql` | ✅ validated (migrate + TEST_COUNT) |\n| MySQL → HeliosDB | `pymysql` → `psycopg` | ✅ validated (all editions) |\n\nWhen **HeliosDB is the source**, its PG-wire catalog exposes no PK/FK/index\nmetadata and no column precision/scale today, so chunking falls back to a single\nchunk and `test-data` self-handles the PK-less case — **`test-count` (row parity)\nis the gate** for migrate-back; the tool works around the missing catalog\nmetadata. SQL Server remains scaffolded.\n\n---\n\n## The target drivers\n\n- **`psycopg`** (default) — PostgreSQL wire via psycopg v3. Portable across\n  **Nano / Lite / Full**. The tool performs the Oracle→PG dialect translation;\n  the capability probe decides per connection what must be rewritten vs. passed\n  through. This is the validated, production path into HeliosDB.\n- **`mysql`** — a MySQL **target** (PyMySQL) for heterogeneous / migrate-back\n  flows (`INSERT` + `ON DUPLICATE KEY UPDATE`); validated as the sink for\n  Oracle→MySQL and HeliosDB→MySQL.\n- **`native`** *(experimental)* — connects through the **same wire protocol as the\n  source** (Oracle TNS → Lite/Full via `oracledb`), so HeliosDB's in-database\n  compatibility absorbs the dialect and the tool transforms almost nothing. Bulk\n  load is array `INSERT` (no COPY). Code-complete and unit-tested; live\n  validation pending a HeliosDB TNS-version fix.\n\nPick the driver in `[target].driver` (`\"psycopg\"` or `\"native\"`); see\n[configuration](docs/guides/configuration.md#driver-selection).\n\n---\n\n## Documentation\n\n| Guide | What's inside |\n|---|---|\n| [docs/](docs/README.md) | The full documentation index. |\n| [Getting started](docs/guides/getting-started.md) | Install, prerequisites, `doctor`, wizard, the end-to-end workflow with real transcripts. |\n| [Configuration](docs/guides/configuration.md) | Every `config.toml` field, env-var passwords, driver selection, tuning. |\n| [Oracle → Lite](docs/migration/oracle-to-heliosdb-lite.md) · [→ Full](docs/migration/oracle-to-heliosdb-full.md) · [→ Nano](docs/migration/oracle-to-heliosdb-nano.md) | Per-target migration guides. |\n| [MySQL \u0026 SQL Server](docs/migration/mysql-and-mssql.md) | The MySQL and SQL Server source guides. |\n| [→ PostgreSQL](docs/migration/to-postgresql.md) | a2h as an Oracle/MySQL/SQL-Server → stock PostgreSQL migrator. |\n| [CDC](docs/cdc.md) | Extract → trail → Replicat, the verbs, v1 limits, v2 roadmap. |\n| [MCP server](docs/mcp.md) | Expose the toolkit as MCP tools (Bearer auth + RBAC) for AI agents. |\n| [CLI reference](docs/reference/cli.md) | Every `a2h` command and option. |\n| [Type mapping](docs/reference/type-mapping.md) | The full Oracle→HeliosDB table + overrides. |\n| [HeliosDB compatibility](docs/heliosdb-compatibility.md) | Supported editions, minimum versions, and the runtime capability probe. |\n| [Troubleshooting](docs/troubleshooting.md) | Common issues + the per-edition minimum builds. |\n\n---\n\n## Development\n\n```bash\npip install -e \".[all,dev]\"\npytest -q                       # hermetic unit tests\n\n# Integration tests need a live source + target:\nA2H_TEST_TARGET_PORT=\u003cport\u003e A2H_TEST_ORACLE_DSN=host:1521/SVC \\\nA2H_TEST_ORACLE_USER=hr A2H_TEST_ORACLE_PW=hr pytest tests/integration -q\n```\n\n## License\n\nApache-2.0. Clean-room implementation — no Ora2Pg source is copied; only public\ncatalog-query knowledge and the Apache-licensed HeliosDB migration scaffold are\nreferenced.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheliosdatabase%2Fany2heliosdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fheliosdatabase%2Fany2heliosdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheliosdatabase%2Fany2heliosdb/lists"}