{"id":13513930,"url":"https://github.com/nucleuscloud/neosync","last_synced_at":"2025-05-12T13:29:06.200Z","repository":{"id":195828344,"uuid":"682760329","full_name":"nucleuscloud/neosync","owner":"nucleuscloud","description":"Open Source Data Security Platform for Developers to Monitor and Detect PII, Anonymize Production Data and Sync it across environments. ","archived":false,"fork":false,"pushed_at":"2025-05-08T23:07:35.000Z","size":176250,"stargazers_count":3853,"open_issues_count":66,"forks_count":155,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-05-09T00:20:03.382Z","etag":null,"topics":["benthos","docker","etl","faker","fine-tuning","golang","kubernetes","mysql","nextjs","open-source","orchestration","postgresql","reactjs","self-hosted","synthetic-data","synthetic-data-generation","test-data-generator","testing","typescript"],"latest_commit_sha":null,"homepage":"https://www.neosync.dev","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nucleuscloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-24T21:41:12.000Z","updated_at":"2025-05-08T17:47:43.000Z","dependencies_parsed_at":"2023-10-13T07:53:17.208Z","dependency_job_id":"70b1aeeb-7a61-4cbe-b793-3c63687bb351","html_url":"https://github.com/nucleuscloud/neosync","commit_stats":{"total_commits":2137,"total_committers":29,"mean_commits":73.6896551724138,"dds":0.6715021057557323,"last_synced_commit":"6242842ccf2c1910fb075dfe76467e26213a5065"},"previous_names":["nucleuscloud/neosync"],"tags_count":263,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nucleuscloud%2Fneosync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nucleuscloud%2Fneosync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nucleuscloud%2Fneosync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nucleuscloud%2Fneosync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nucleuscloud","download_url":"https://codeload.github.com/nucleuscloud/neosync/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253747431,"owners_count":21957756,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benthos","docker","etl","faker","fine-tuning","golang","kubernetes","mysql","nextjs","open-source","orchestration","postgresql","reactjs","self-hosted","synthetic-data","synthetic-data-generation","test-data-generator","testing","typescript"],"created_at":"2024-08-01T05:00:40.496Z","updated_at":"2025-05-12T13:29:06.141Z","avatar_url":"https://github.com/nucleuscloud.png","language":"Go","readme":"\u003cp align=\"center\"\u003e\n  \u003c!-- \u003cimg alt=\"neosyncbanner\" src=\"https://assets.nucleuscloud.com/neosync/docs/neosync-header.svg\" \u003e --\u003e\n  \u003cpicture\u003e\n  \u003csource\n    srcset=\"https://assets.nucleuscloud.com/neosync/docs/neosync-header.svg\"\n    media=\"(prefers-color-scheme: light)\"\n  /\u003e\n  \u003csource\n    srcset=\"https://assets.nucleuscloud.com/neosync/docs/neosync-header-dark.svg\"\n    media=\"(prefers-color-scheme: dark), (prefers-color-scheme: no-preference)\"\n  /\u003e\n  \u003cimg src=\"https://github-readme-stats.vercel.app/api?username=anuraghazra\u0026show_icons=true\" /\u003e\n\u003c/picture\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\" style=\"font-size: 24px;font-weight: 500;\"\u003e\nOpen Source Data Anonymization and Synthetic Data Orchestration\n\u003cp\u003e\n\n\u003cdiv align='center'\u003e\n | \u003ca href=\"https://www.neosync.dev\"\u003eWebsite\u003c/a\u003e\n | \u003ca href=\"https://docs.neosync.dev\"\u003eDocs\u003c/a\u003e\n | \u003ca href=\"https://discord.com/invite/MFAMgnp4HF\"\u003eDiscord\u003c/a\u003e\n | \u003ca href=\"https://www.neosync.dev/blog\"\u003eBlog\u003c/a\u003e\n | \u003ca href=\"https://docs.neosync.dev/changelog\"\u003eChangelog\u003c/a\u003e\n | \u003ca href=\"https://neosync.productlane.com/roadmap\"\u003eRoadmap\u003c/a\u003e\n\u003c/div\u003e\n\n \u003cbr\u003e\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href='https://makeapullrequest.com'\u003e\n    \u003cimg alt='PRs Welcome' src='https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=shields'/\u003e\n  \u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/github/license/lightdash/lightdash\" /\u003e\n  \u003c!-- \u003ca href=\"https://codecov.io/gh/nucleuscloud/neosync\"\u003e\n    \u003cimg alt=\"CodeCov\" src=\"https://codecov.io/gh/nucleuscloud/neosync/graph/badge.svg?token=A35QDLRU04\"/\u003e\n    \u003c/a\u003e --\u003e\n  \u003ca href=\"https://github.com/nucleuscloud/neosync/actions/workflows/go.yml/\"\u003e\n    \u003cimg alt=\"Go Tests\" src=\"https://github.com/nucleuscloud/neosync/actions/workflows/go.yml/badge.svg\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://x.com/neosynccloud\"\u003e\n    \u003cimg alt=\"Follow X\" src=\"https://img.shields.io/twitter/follow/neosynccloud?label=Follow\"/\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://artifacthub.io/packages/search?repo=neosync\"\u003e\n    \u003cimg alt=\"ArtifactHub Neosync\" src=\"https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/neosync\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://gurubase.io/g/neosync\"\u003e\n    \u003cimg alt=\"Gurubase\" src=\"https://img.shields.io/badge/Gurubase-Ask%20Neosync%20Guru-006BFF\" /\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n## Introduction\n\n[Neosync](https://www.neosync.dev) is an open-source, developer-first way to anonymize PII, generate synthetic data and sync environments for better testing, debugging and developer experience.\n\nCompanies use Neosync to:\n\n1. **Safely test code against production data** - Anonymize sensitive production data in order to safely use it locally for a better testing and developer experience\n2. **Easily reproduce production bugs locally** - Anonymize and subset production data to get a safe, representative data set that you can use to locally reproduce production bugs quickly and efficiently\n3. **High quality data for lower-level environments** - Catch bugs before they hit production when you hydrate your staging and QA environments with production-like data\n4. **Solve GDPR, DPDP, FERPA, HIPAA and more** - Use anonymized and synthetic data to reduce your compliance scope and easily comply with laws like HIPAA, GDPR, and DPDP\n5. **Seed development databases** - Easily seed development databases with synthetic data for unit testing, demos and more\n\n## Features\n\n- **Generate synthetic data** based on your schema\n- **Anonymize existing production-data** for a better developer experience\n- **Subset your production database** for local and CI testing using any SQL query\n- **Complete async pipeline** that automatically handles job retries, failures and playback using an event-sourcing model\n- **Referential integrity** for your data automatically\n- **Declarative, GitOps based configs** as a step in your CI pipeline to hydrate your CI DB\n- **Pre-built data transformers** for all major data types\n- **Custom data transformers** using javascript or LLMs\n- **Pre-built integrations** with Postgres, Mysql, S3\n\n## Getting started\n\nNeosync is a fully dockerized setup which makes it easy to get up and running.\n\nA [compose.yml](./compose.yml) file at the root contains production image refs that allow you to get up and running with just a few commands without having to build anything on your system.\n\nNeosync uses the newer `docker compose` command, so be sure to have that installed on your machine.\n\nTo start Neosync, clone the repo into a local directory, be sure to have docker installed and running, and then run:\n\n```sh\nmake compose/up\n```\n\nTo stop, run:\n\n```sh\nmake compose/down\n```\n\nNeosync will now be available on [http://localhost:3000](http://localhost:3000).\n\nThe production compose pre-seeds with connections and jobs to get you started! Simply run the generate and sync job to watch Neosync in action!\n\n## Kubernetes, Auth Mode and more\n\nFor more in-depth details on environment variables, Kubernetes deployments, and a production-ready guide, check out the [Deploy Neosync](https://docs.neosync.dev/deploy/introduction) section of our Docs.\n\n## Resources\n\nSome resources to help you along the way:\n\n- [Docs](https://docs.neosync.dev) for comprehensive documentation and guides\n- [Discord](https://discord.com/invite/MFAMgnp4HF) for discussion with the community and Neosync team\n- [X](https://x.com/neosynccloud) for the latest updates\n\n## Contributing\n\nWe love contributions big and small. Here are just a few ways that you can contribute to Neosync.\n\n- Join our [Discord](https://discord.com/invite/MFAMgnp4HF) channel and ask us any questions there\n- Open a PR (see our instructions on [developing with Neosync locally](https://docs.neosync.dev/guides/neosync-local-dev))\n- Submit a [feature request](https://github.com/nucleuscloud/neosync/issues/new?assignees=\u0026labels=enhancement%2C+feature\u0026template=feature_request.md) or [bug report](https://github.com/nucleuscloud/neosync/issues/new?assignees=\u0026labels=bug\u0026template=bug_report.md)\n\n## Licensing\n\nWe strongly believe in free and open source software and make this repo is available under the [MIT expat license](./LICENSE.md).\n","funding_links":[],"categories":["Go","By Industry","Repos","\u003ca name=\"Go\"\u003e\u003c/a\u003eGo"],"sub_categories":["DevOps"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnucleuscloud%2Fneosync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnucleuscloud%2Fneosync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnucleuscloud%2Fneosync/lists"}