Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/valmi-io/valmi-activation

⚡ valmi.io reverse ETL (data activation) is the open source ( OSS ) data activation platform to load data from warehouses into Webhooks and SaaS tools like Klaviyo, Facebook Ads, Salesforce, Braze etc. Valmi.io Customer Data Platform (CDP) helps track and ingest user activity events from websites, shopify, serverside events. https://cloud.valmi.io
https://github.com/valmi-io/valmi-activation

airbyte cdp composable-cdp dagster dbt duckdb ecommerce email-marketing etl event-ingestion event-tracking marketing-automation modern-data-stack open-source push-notifications reverse-etl shopify shopify-app shopify-events user-behavior

Last synced: 5 months ago
JSON representation

⚡ valmi.io reverse ETL (data activation) is the open source ( OSS ) data activation platform to load data from warehouses into Webhooks and SaaS tools like Klaviyo, Facebook Ads, Salesforce, Braze etc. Valmi.io Customer Data Platform (CDP) helps track and ingest user activity events from websites, shopify, serverside events. https://cloud.valmi.io

Awesome Lists containing this project

README 

          

  - ### Trusted By

	


		

      

			

      

		

	



  - We have launched EVENT Streams (powered by Jitsu). Try free @ [cloud.valmi.io](https://cloud.valmi.io) both **User Activity Events Tracking** and **Data Activation**



  - ### See it in [Action](https://cloud.valmi.io/spaces/a9195c50-60ca-4692-8f03-5a486ee9f270/syncs/d69cf9f9-0e20-4e2c-a683-2649404f52ed/runs)

    Quickly witness a One Million (1M) Rows Sync at [Live Sync](https://cloud.valmi.io/spaces/a9195c50-60ca-4692-8f03-5a486ee9f270/syncs/d69cf9f9-0e20-4e2c-a683-2649404f52ed/runs).

    

    Events  |  Shopify App

    :-------------------------:|:-------------------------:

    ![](http://profile.mywavia.com/im/events-valmi.png)  |  ![](http://profile.mywavia.com/im/shopify-valmi.png)





  valmi.io






  

    Website

    ·

    Slack Community

    ·

    Documentation

    ·

    Blog

    .

    Changelog

    .

    Roadmap

  






     valmi.io Customer Data Platform (CDP) provides open-source Reverse ETL (data activation) platform to load data from warehouses into SaaS platforms such as Advertising (Facebook Ads, Google Ads), Marketing (Klaviyo, Braze), CRM (Salesforce, Hubspot), Accounting (Xero, Netsuite), Customer Support (Zendesk), Messaging (Slack), Webhook Apis etc.







    Test





    License





    License









[![valmi-activation](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-activation-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-activation-docker-image-action.yml) [![valmi-connectors](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-connectors-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-connectors-docker-image-action.yml) [![valmi-dagster](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-dagster-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-dagster-docker-image-action.yml) 




[![valmi-repo](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-repo-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-activation/actions/workflows/valmi-repo-docker-image-action.yml) [![valmi-app-backend](https://github.com/valmi-io/valmi-app-backend/actions/workflows/valmi-app-backend-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-app-backend/actions/workflows/valmi-app-backend-docker-image-action.yml) [![valmi-app](https://github.com/valmi-io/valmi-app/actions/workflows/valmi-app-docker-image-action.yml/badge.svg)](https://github.com/valmi-io/valmi-app/actions/workflows/valmi-app-docker-image-action.yml)



 



  

✨  **Checkout docs at - [https://www.valmi.io/docs/](https://www.valmi.io/docs/)**



✨  **Read the variety of usecases valmi.io can enable for your organization - [https://www.valmi.io/blog/](https://www.valmi.io/blog/)**

### 3 ways to start with valmi.io

    

👉 **1. Use it in the cloud**

- You can immediately create a sync at [https://cloud.valmi.io.](https://cloud.valmi.io/)

- Watch a demo video to create a sync.

      

    


  
    



  [](https://youtu.be/UBY0106gOD8 "Watch the demo video") 



    


    

👉 **2.  Run it locally or in your Cloud**

- **Prerequisites:**

  valmi.io relies heavily on both Docker and Docker-compose. Install [Docker Desktop](https://docs.docker.com/compose/install/) to get both docker and docker-compose.



- Clone this repo and recursively clone submodules.



```bash

git clone git@github.com:valmi-io/valmi-activation.git

cd valmi-activation

git submodule update --init --recursive

```



- Setup the environment.

```bash



cp .env-example .env



cd valmi-app-backend

cp .env-example .env



cd ../valmi-app

`For macos`

cp .env-example.macos .env

`For linux`

cp .env-example.linux .env

```



- Intermediate storage, We are adding support for object stores like S3, GCS. Until then, Local storage is used.

```bash

sudo mkdir -p /tmp/shared_dir/intermediate_store

sudo chmod -R 777 /tmp/shared_dir/intermediate_store

```



- Launch the reverse-etl service.

```bash

./valmi prod --with-jitsu

```



- To stop the service, run the following.

```bash

./valmi prod --with-jitsu down

```



- Please wait for about 2 minutes before you access the service, since valmi-app builds an optimized compiled version of the app UI. To access the service, please check the ['Accessing the service'](https://github.com/valmi-io/valmi-activation#accessing-the-service-for-local-deployments) section for local deployments.



👉 **3. Develop a connector locally to customize valmi.io as per your needs. You can just contact us too.**



- Clone, setup environment variables and create intermediate storage (see above section).

- Create a new connector (Optional).

```bash

# Copy code base from any existing connectors from valmi-integrations folder (ex. destination-webhook)



cd valmi-integrations/connectors

cp -r destination-webhook destination-awesome_connector



# Make necessary changes and build the connector

cd destination-awesome_connector

make build_docker version=latest



# Add the new connector information to "valmi-app-backend/init_db/connector_def.json"

```



- Run the service.

```bash

./valmi dev --with-jitsu

```



- To access the service, please check the ['Accessing the service'](https://github.com/valmi-io/valmi-activation#accessing-the-service-for-local-deployments) section for local deployments.



- To Stop the service, run the following.

```bash

./valmi dev --with-jitsu down

```



- ### Accessing the service for local deployments

  

    Syncs 
 http://localhost:3000  |  Sync Runs 
 http://localhost:3000

    :-------------------------:|:-------------------------:

    ![](https://blog.valmi.io/content/images/size/w1000/2023/06/syncs_page.png)  |  ![](https://blog.valmi.io/content/images/size/w1000/2023/06/sync_runs_page.png)



    UI Backend Server API 
 http://localhost:4000/api/docs       |  Activation Server API 
 http://localhost:8000/docs

    :-------------------------:|:-------------------------:

    ![](https://blog.valmi.io/content/images/size/w1000/2023/06/app_backend_api.png)  |  ![](https://blog.valmi.io/content/images/size/w1000/2023/06/activation_server_api.png)