{"id":37201050,"url":"https://github.com/rainforestapp/teleport","last_synced_at":"2026-01-14T23:10:53.595Z","repository":{"id":57658024,"uuid":"107473467","full_name":"rainforestapp/teleport","owner":"rainforestapp","description":"🍆 Trigger-based Postgres replicator in Go that works without superuser permissions","archived":true,"fork":true,"pushed_at":"2018-07-03T22:16:36.000Z","size":1161,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-06-20T15:44:42.519Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"pagarme/teleport","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rainforestapp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-10-18T23:16:36.000Z","updated_at":"2023-02-15T12:04:15.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/rainforestapp/teleport","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/rainforestapp/teleport","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rainforestapp%2Fteleport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rainforestapp%2Fteleport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rainforestapp%2Fteleport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rainforestapp%2Fteleport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rainforestapp","download_url":"https://codeload.github.com/rainforestapp/teleport/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rainforestapp%2Fteleport/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28437933,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T22:37:52.437Z","status":"ssl_error","status_checked_at":"2026-01-14T22:37:31.496Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T23:10:52.861Z","updated_at":"2026-01-14T23:10:53.552Z","avatar_url":"https://github.com/rainforestapp.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# teleport\n[![Build Status](https://travis-ci.org/pagarme/teleport.svg?branch=master)](https://travis-ci.org/pagarme/teleport)\n\nA trigger-based Postgres replicator that performs real-time data changes based\non DML triggers, and DDL migrations by either DDL event triggers or by diffing\nschema changes when event triggers are not available. In other words, a\ncomplete replicator that works without any special permissions on the database,\njust like the ones you don't have in AWS RDS.\n\nYes, you read it right\n\n## How it works\n\nWhen DDL event triggers are not available, using a configurable time interval,\nTeleport diffs the current schema and replicate new tables, columns, indexes\nand so on from the source to the target. Inserted, updated or deleted rows are\ndetected by triggers on the source, which generate events that teleport\ntransform into batches for the appropriate targets.\n\nIf teleport fails to apply a batch of new/updated rows due to a schema change\nthat is not reflected on target yet, it will queue the batch, apply the schema\nchange and then apply the failed batches again.  This ensures consistency on\nthe data even after running migrations and changing the source schema.\n\nCurrently only source databases with Postgres versions \u003e= 9.2.16 are supported.\nDDL event triggers are only available for Postgres versions \u003e= 9.3. For AWS\nRDS, event triggers are only [available after Postgres versions \u003e= 9.4.9][aws_event_triggers]\nTeleport requires that all replicated tables have a primary key.\n\n## Features\n\nAll the features above are replicatable by teleport:\n\n- INSERT/UPDATE/DELETE rows\n- Tables/columns\n- Composite types\n- Enums\n- Schemas\n- Functions\n- Indexes\n- Extensions\n\n## Install\n\n```\ngo get -u github.com/pagarme/teleport\n```\n\n## Getting started\n\nEach running instance of teleport is responsible for managing a host, exposing\na HTTP API to receive batches from other instances. For a master-slave\nreplication you should run one teleport instance for the source host (master)\nand other for the target host (slave), and set the API of the target as the\ndestination for the data fetched from the source.\n\n### Configuring the source instance\n\nFor the source, create a config file named `source_config.yml`:\n\n```yml\nbatch_size: 10000\nmax_events_per_batch: 10000\nuse_event_triggers: true # Available for Postgres \u003e= 9.3\nprocessing_intervals:\n  batcher: 100\n  transmitter: 100\n  applier: 100\n  vacuum: 500\n  ddlwatcher: 5000\ndatabase:\n  name: \"finops-db\"\n  database: \"postgres\"\n  hostname: \"postgres.mydomain.com\"\n  username: \"teleport\"\n  password: \"root\"\n  port: 5432\nserver:\n  hostname: \"0.0.0.0\"\n  port: 3000\ntargets:\n  my-target:\n    target_expression: \"public.*\"\n    endpoint:\n      hostname: \"target.mydomain.com\"\n      port: 3001\n    apply_schema: \"test\"\n```\n\nFor each `target` under the `targets` section, it's possible to define a\n`target_expression`, which defines what tables will be replicated. The\nexpression should be schema-qualified.\n\nYou should also set a `apply_schema`, which defines in what schema the data\nwill be applied in the target, and a `endpoint` of the target teleport\ninstance.\n\n### Configuring the target instance\n\nFor the target, create a config file named `target_config.yml`:\n\n```yml\nbatch_size: 10000\nmax_events_per_batch: 10000\nprocessing_intervals:\n  batcher: 100\n  transmitter: 100\n  applier: 100\n  vacuum: 500\n  ddlwatcher: 5000\ndatabase:\n  name: \"my-target\"\n  database: \"postgres\"\n  hostname: \"postgres-replica.mydomain.com\"\n  username: \"teleport\"\n  password: \"root\"\n  port: 5432\nserver:\n  hostname: \"target.mydomain.com\"\n  port: 3001\n```\n\nYou may have noted this config file does not include a `targets` section,\nsimply because this instance will not be the source for any host. You can,\nhowever, use a instance as both source and target by simply including a\n`targets` section.\n\n### Initial load\n\nIt's possible to generate initial-load batches on the source that will be\ntransmitted to the target. To do a initial-load, run on source:\n\n```\n$ teleport -config source_config.yml -mode initial-load -load-target my-target\n```\n\nThis will create batches on the source that will be transmitted to `my-target`\nas soon as teleport starts running.\n\n### Starting up\n\nYou may start instances before the end of the initial load.  This will\nreplicate data as it's extracted from the source to the target, and further\nmodifications will be replicated and applied later on.\n\nOn source, teleport will diff, group and batch events and transmit batches to\nthe target. On the target, batches will be applied on the same order as they\nocurred on the source.\n\nOn source, run:\n\n```\n$ teleport -config source_config.yml\n```\n\nOn target, run:\n\n```\n$ teleport -config target_config.yml\n```\n\nTeleport is now up and running! \\o/\n\n## Sentry Support\nTeleport has native [sentry](https://getsentry.com). To enable it, just use the following config with the sentry DSN:\n\n```\nsentry_endpoint: https://user:password@sentry.endpoint.com/8\n```\n\n## Performance\n\nWe've been using teleport to replicate a roughly large production database\n(150GB) with ~50 DML updates per second and performance is pretty satisfying.\nUnder our normal load, each teleport instance uses ~150MB of memory and not\nsignificant CPU usage nor spikes.\n\nAs teleport relies on (very light) triggers for data replication, the source\ndatabase performance may be slightly affected, but impacts were negligible for\nour use cases.\n\nInitial load uses Postgres' `COPY FROM` to load data, which makes it __very__\nfast. The initial load of our entire 150GB database took under ~14 hours using\nthe `db.m4.xlarge` RDS instance for source and target.\n\n## Tests\n\n```\n$ docker-compose run test\n```\n\n## License\n\nThe MIT license.\n\n[aws_event_triggers](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_PostgreSQL.html#PostgreSQL.Concepts.General.FeatureSupport.EventTriggers)\n\n## Troubleshooting\n\n- Since version 0.4.0, the ids of Teleport's internal tables were changed to\n  `bigserial` and `bigint`. The cose is backwards compatible, and works with\n  the previous version, which used `serial` and `int`. If you are having\n  problems with integer overflow, it will be necessary to either drop the\n  current tables or manually alter them. See the [changelog](CHANGELOG.md) for\n  details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frainforestapp%2Fteleport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frainforestapp%2Fteleport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frainforestapp%2Fteleport/lists"}