{"id":21825815,"url":"https://github.com/valmi-io/valmi-activation","last_synced_at":"2025-05-01T09:54:27.116Z","repository":{"id":152051152,"uuid":"598595038","full_name":"valmi-io/valmi-activation","owner":"valmi-io","description":"⚡ valmi.io reverse ETL (data activation) is the open source ( OSS ) data activation platform to load data from warehouses into Webhooks and SaaS tools like Klaviyo, Facebook Ads, Salesforce, Braze etc. Valmi.io Customer Data Platform (CDP) helps track and ingest user activity events from websites, shopify, serverside events. https://cloud.valmi.io","archived":false,"fork":false,"pushed_at":"2024-07-10T12:15:39.000Z","size":49347,"stargazers_count":151,"open_issues_count":14,"forks_count":13,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-05-01T09:54:16.761Z","etag":null,"topics":["airbyte","cdp","composable-cdp","dagster","dbt","duckdb","ecommerce","email-marketing","etl","event-ingestion","event-tracking","marketing-automation","modern-data-stack","open-source","push-notifications","reverse-etl","shopify","shopify-app","shopify-events","user-behavior"],"latest_commit_sha":null,"homepage":"https://www.valmi.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/valmi-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-07T12:40:44.000Z","updated_at":"2025-04-25T07:02:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"8d256a1e-a7f5-46df-bce1-70dd3199a4ab","html_url":"https://github.com/valmi-io/valmi-activation","commit_stats":null,"previous_names":[],"tags_count":159,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valmi-io%2Fvalmi-activation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valmi-io%2Fvalmi-activation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valmi-io%2Fvalmi-activation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valmi-io%2Fvalmi-activation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/valmi-io","download_url":"https://codeload.github.com/valmi-io/valmi-activation/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251856993,"owners_count":21655119,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["airbyte","cdp","composable-cdp","dagster","dbt","duckdb","ecommerce","email-marketing","etl","event-ingestion","event-tracking","marketing-automation","modern-data-stack","open-source","push-notifications","reverse-etl","shopify","shopify-app","shopify-events","user-behavior"],"created_at":"2024-11-27T18:02:47.580Z","updated_at":"2025-05-01T09:54:27.096Z","avatar_url":"https://github.com/valmi-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n  - ### Trusted By\n\t\u003cdiv  style=\"display: flex;\n\t\twidth: calc(250px * 14);\"\u003e\n\t\t\u003cspan   style=\"\n\t\theight: 100px;\n\t\twidth: 250px;\"\u003e\n      \u003ca href=\"https://kegmil.com/\"\u003e\n\t\t\t\u003cimg src=\"https://i0.wp.com/kegmil.com/wp-content/uploads/2019/11/KM_Logo_Full_Primary.png?resize=1024%2C152\u0026ssl=1\" height=\"30\" width=\"202\" alt=\"\" /\u003e\n      \u003c/a\u003e\n\t\t\u003c/span\u003e\n\t\u003c/div\u003e\n\n\n  - We have launched EVENT Streams (powered by Jitsu). Try free @ [cloud.valmi.io](https://cloud.valmi.io) both **User Activity Events Tracking** and **Data Activation**\n\n  - ### See it in [Action](https://cloud.valmi.io/spaces/a9195c50-60ca-4692-8f03-5a486ee9f270/syncs/d69cf9f9-0e20-4e2c-a683-2649404f52ed/runs)\n    Quickly witness a One Million (1M) Rows Sync at [Live Sync](https://cloud.valmi.io/spaces/a9195c50-60ca-4692-8f03-5a486ee9f270/syncs/d69cf9f9-0e20-4e2c-a683-2649404f52ed/runs).\n    \n    Events  |  Shopify App\n    :-------------------------:|:-------------------------:\n    ![](http://profile.mywavia.com/im/events-valmi.png)  |  ![](http://profile.mywavia.com/im/shopify-valmi.png)\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://valmi.io\"\u003e\u003cimg width=\"400\" src=\"https://blog.valmi.io/content/images/2023/06/valmilogo-1.png\" alt=\"valmi.io\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003cb\u003e\n    \u003ca href=\"https://www.valmi.io\"\u003eWebsite\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.valmi.io/slack\"\u003eSlack Community\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.valmi.io/docs/\"\u003eDocumentation\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.valmi.io/blog/\"\u003eBlog\u003c/a\u003e\n    .\n    \u003ca href=\"https://gist.github.com/S-Nagendra/0904e9cac9ca016fba58ac63b6e748a4\"\u003eChangelog\u003c/a\u003e\n    .\n    \u003ca href=\"https://gist.github.com/S-Nagendra/ba70a68809a0ddd08ac4f65f5c76f472\"\u003eRoadmap\u003c/a\u003e\n  \u003c/b\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cem\u003e \u003ca href=\"https://valmi.io\"\u003evalmi.io\u003c/a\u003e Customer Data Platform (CDP) provides open-source Reverse ETL (data activation) platform to load data from warehouses into SaaS platforms such as Advertising (Facebook Ads, Google Ads), Marketing (Klaviyo, Braze), CRM (Salesforce, Hubspot), Accounting (Xero, Netsuite), Customer Support (Zendesk), Messaging (Slack), Webhook Apis etc.\u003c/em\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\u003ca href=\"https://github.com/valmi-io/valmi-activation/stargazers/\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/stars/valmi-io/valmi-activation?style=social\u0026label=Star\u0026maxAge=10000\" alt=\"Test\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/valmi-io/valmi-activation/blob/main/LICENSE.md\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/static/v1?label=license\u0026message=MIT\u0026color=white\" alt=\"License\"\u003e\n\u003c/a\u003e\n\u003ca href=\"https://github.com/valmi-io/valmi-activation/blob/main/LICENSE.md\" target=\"_blank\"\u003e\n    \u003cimg src=\"https://img.shields.io/static/v1?label=license\u0026message=ELv2\u0026color=white\" alt=\"License\"\u003e\n\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n[![valmi-activation](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-activation-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-activation-docker-image-action.yml) [![valmi-connectors](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-connectors-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-connectors-docker-image-action.yml) [![valmi-dagster](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-dagster-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-dagster-docker-image-action.yml) \n\u003cbr/\u003e\n[![valmi-repo](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-repo-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-repo-docker-image-action.yml) [![valmi-app-backend](https://github.com/valmi-io/valmi-app-backend/actions/workflows/valmi-app-backend-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-app-backend/actions/workflows/valmi-app-backend-docker-image-action.yml) [![valmi-app](https://github.com/valmi-io/valmi-app/actions/workflows/valmi-app-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-app/actions/workflows/valmi-app-docker-image-action.yml)\n\n\u003c!---\n\u003ca href=\"/../../issues?q=is%3Aopen+is%3Aissue\"\u003e \u003cimg alt=\"GitHub issues\" src=\"https://img.shields.io/github/issues-raw/valmi-io/valmi-activation?color=%23238636\"\u003e\u003c/a\u003e \u003ca href=\"/../../issues?q=is%3Aissue+is%3Aclosed\"\u003e \u003cimg alt=\"GitHub closed issues\" src=\"https://img.shields.io/github/issues-closed-raw/valmi-io/valmi-activation?color=%238957e5\"\u003e \u003c/a\u003e \u003ca href=\"/../../pulls?q=is%3Aopen+is%3Apr\"\u003e \u003cimg alt=\"GitHub pull requests\" src=\"https://img.shields.io/github/issues-pr-raw/valmi-io/valmi-activation\"\u003e \u003c/a\u003e \u003ca href=\"/../../pulls?q=is%3Apr+is%3Aclosed\"\u003e \u003cimg alt=\"GitHub closed pull requests\" src=\"https://img.shields.io/github/issues-pr-closed-raw/valmi-io/valmi-activation?color=%238957e5\"\u003e \u003c/a\u003e\n---\u003e\n\u003c/div\u003e \n\n  \n✨  **Checkout docs at - [https://www.valmi.io/docs/](https://www.valmi.io/docs/)**\n\n✨  **Read the variety of usecases valmi.io can enable for your organization - [https://www.valmi.io/blog/](https://www.valmi.io/blog/)**\n### 3 ways to start with valmi.io\n    \n👉 **1. Use it in the cloud**\n- You can immediately create a sync at [https://cloud.valmi.io.](https://cloud.valmi.io/)\n- Watch a demo video to create a sync.\n      \n    \u003cdiv align=\"center\" \u003e\n  \u003cbr/\u003e    \n\n  [\u003cimg  src=\"https://blog.valmi.io/content/images/size/w1600/2023/06/Screenshot-2023-06-21-at-3.37.00-PM.png\" width=\"70%\" /\u003e](https://youtu.be/UBY0106gOD8 \"Watch the demo video\") \n\n\n    \u003c/div\u003e\n    \n👉 **2.  Run it locally or in your Cloud**\n- **Prerequisites:**\n  valmi.io relies heavily on both Docker and Docker-compose. Install [Docker Desktop](https://docs.docker.com/compose/install/) to get both docker and docker-compose.\n\n- Clone this repo and recursively clone submodules.\n\n```bash\ngit clone git@github.com:valmi-io/valmi-activation.git\ncd valmi-activation\ngit submodule update --init --recursive\n```\n\n- Setup the environment.\n```bash\n\ncp .env-example .env\n\ncd valmi-app-backend\ncp .env-example .env\n\ncd ../valmi-app\n`For macos`\ncp .env-example.macos .env\n`For linux`\ncp .env-example.linux .env\n```\n\n- Intermediate storage, We are adding support for object stores like S3, GCS. Until then, Local storage is used.\n```bash\nsudo mkdir -p /tmp/shared_dir/intermediate_store\nsudo chmod -R 777 /tmp/shared_dir/intermediate_store\n```\n\n- Launch the reverse-etl service.\n```bash\n./valmi prod --with-jitsu\n```\n\n- To stop the service, run the following.\n```bash\n./valmi prod --with-jitsu down\n```\n\n- Please wait for about 2 minutes before you access the service, since valmi-app builds an optimized compiled version of the app UI. To access the service, please check the ['Accessing the service'](https://github.com/valmi-io/valmi-activation#accessing-the-service-for-local-deployments) section for local deployments.\n\n\n👉 **3. Develop a connector locally to customize valmi.io as per your needs. You can just contact us too.**\n\n- Clone, setup environment variables and create intermediate storage (see above section).\n- Create a new connector (Optional).\n```bash\n# Copy code base from any existing connectors from valmi-integrations folder (ex. destination-webhook)\n\ncd valmi-integrations/connectors\ncp -r destination-webhook destination-awesome_connector\n\n# Make necessary changes and build the connector\ncd destination-awesome_connector\nmake build_docker version=latest\n\n# Add the new connector information to \"valmi-app-backend/init_db/connector_def.json\"\n```\n\n- Run the service.\n```bash\n./valmi dev --with-jitsu\n```\n\n- To access the service, please check the ['Accessing the service'](https://github.com/valmi-io/valmi-activation#accessing-the-service-for-local-deployments) section for local deployments.\n\n- To Stop the service, run the following.\n```bash\n./valmi dev --with-jitsu down\n```\n\n- ### Accessing the service for local deployments\n  \n    Syncs \u003cbr\u003e http://localhost:3000  |  Sync Runs \u003cbr\u003e http://localhost:3000\n    :-------------------------:|:-------------------------:\n    ![](https://blog.valmi.io/content/images/size/w1000/2023/06/syncs_page.png)  |  ![](https://blog.valmi.io/content/images/size/w1000/2023/06/sync_runs_page.png)\n\n    UI Backend Server API \u003cbr\u003e http://localhost:4000/api/docs       |  Activation Server API \u003cbr\u003e http://localhost:8000/docs\n    :-------------------------:|:-------------------------:\n    ![](https://blog.valmi.io/content/images/size/w1000/2023/06/app_backend_api.png)  |  ![](https://blog.valmi.io/content/images/size/w1000/2023/06/activation_server_api.png)\n\n  \n    \n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalmi-io%2Fvalmi-activation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvalmi-io%2Fvalmi-activation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalmi-io%2Fvalmi-activation/lists"}