{"id":26367444,"url":"https://github.com/gustapinto/from-to","last_synced_at":"2026-05-07T01:05:16.017Z","repository":{"id":281927466,"uuid":"946903057","full_name":"gustapinto/from-to","owner":"gustapinto","description":"Moving CDC data from PostgreSQL to Apache Kafka with optional mapping using Lua functions","archived":false,"fork":false,"pushed_at":"2025-04-08T12:48:23.000Z","size":90,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-08T13:29:39.563Z","etag":null,"topics":["cdc","go","kafka","lua","postgres"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gustapinto.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-11T21:20:24.000Z","updated_at":"2025-04-08T12:48:27.000Z","dependencies_parsed_at":"2025-04-08T13:37:46.017Z","dependency_job_id":null,"html_url":"https://github.com/gustapinto/from-to","commit_stats":null,"previous_names":["gustapinto/from-to"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/gustapinto/from-to","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustapinto%2Ffrom-to","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustapinto%2Ffrom-to/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustapinto%2Ffrom-to/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustapinto%2Ffrom-to/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gustapinto","download_url":"https://codeload.github.com/gustapinto/from-to/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gustapinto%2Ffrom-to/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262834786,"owners_count":23371849,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdc","go","kafka","lua","postgres"],"created_at":"2025-03-16T21:17:01.987Z","updated_at":"2026-05-07T01:05:15.967Z","avatar_url":"https://github.com/gustapinto.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# From To\n\n\u003cpicture\u003e\n  \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://raw.githubusercontent.com/gustapinto/from-to/main/docs/images/diagram-dark.jpg\"\u003e\n  \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://raw.githubusercontent.com/gustapinto/from-to/main/docs/images/diagram-light.jpg\"\u003e\n  \u003cimg src=\"https://raw.githubusercontent.com/gustapinto/from-to/main/docs/images/diagram-light.jpg\"\u003e\n\u003c/picture\u003e\n\n## Installation\n\nThe pre-built binaries for **FromTo** are available in the [releases page](https://github.com/gustapinto/from-to/releases), they are the recommended installation option, but you can also compile the code yourself\n\n### Compiling as binaries\n\n1. To compile **FromTo** first clone this repository in your machine\n2. Then open the cloned folder on a shell and run `make build` to build the binaries for all supported platforms, or run `make build/\u003cplatform\u003e` to build for an specific platform. The currently supported platforms are:\n   - linux\n   - windows\n   - macos\n\n#### Building Pre Requisites\n\n1. [Go 1.24+](https://go.dev/)\n2. [GNU Make](https://www.gnu.org/software/make/) (Optional)\n\n## Example usage\n\n```bash\n./from_to_linux_amd64 -manifest=./from_to.yaml\n```\n\n## Example config\n\n```yaml\ninput:\n  connector: \"postgres\"\n  postgresConfig:\n    dsn: \"postgres://from-to-user:from-to-passw@localhost:5432/from-to-db?sslmode=disable\"\n    tables:\n      - \"sales\"\n\noutputs:\n  salesKafkaOutput:\n    connector: \"kafka\"\n    kafkaConfig:\n      bootstrapServers:\n        - \"localhost:9094\"\n      topics:\n        - name: \"publicSales\"\n\nmappers:\n  salesMapper:\n    type: \"lua\"\n    luaConfig:\n      filePath: \"./example/mappers.lua\"\n      function: \"map_sales_event_with_http\"\n\nchannels:\n  salesChannel:\n    from: \"sales\"\n    to: \"publicSales\"\n    output: \"salesKafkaOutput\"\n    mapper: \"salesMapper\"\n```\n\nAdditional options and specific documentation on inputs, outputs and mappers can be found at the [examples folder](https://github.com/gustapinto/from-to/blob/main/example)\n\n## Connector support\n\nThe currently supported connectors and mappers are:\n\n- **PostgreSQL (postgres):** Input connector\n- **Kafka (kafka):** Output connector\n- **Webhook (webhook):** Output connector\n- **Lua (lua):** Mapper\n\n## Lua support\n\n**FromTo** supports [Lua](https://www.lua.org/) scripting to create row mappers, an example mapper can be found at [example/mappers.lua](https://github.com/gustapinto/from-to/blob/main/example/mappers.lua). It uses the [yuin/gopher-lua](https://github.com/yuin/gopher-lua) VM and preloads some of its libraries for improved DX.\n\n### Preloaded libraries\n\n- [cjoudrey/gluahttp](https://github.com/cjoudrey/gluahttp)\n- [layeh.com/gopher-json](https://github.com/layeh/gopher-json)\n\n## Development\n\n**FormTo** development follows the [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)\n\n### Development Pre Requisites\n\n1. [Go 1.24+](https://go.dev/)\n2. [GNU Make](https://www.gnu.org/software/make/) (Optional)\n3. [Docker](https://www.docker.com/) or an compatible alternative\n4. [Docker Compose](https://docs.docker.com/compose/) or an compatible alternative\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgustapinto%2Ffrom-to","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgustapinto%2Ffrom-to","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgustapinto%2Ffrom-to/lists"}