https://github.com/official-krish/zapier

This project is a custom automation processor inspired by Zapier, designed to handle complex workflows efficiently using Kafka, Prisma, and TypeScript.
https://github.com/official-krish/zapier

kafka nodejs prisma typescript

Last synced: about 2 months ago
JSON representation

This project is a custom automation processor inspired by Zapier, designed to handle complex workflows efficiently using Kafka, Prisma, and TypeScript.

• Host: GitHub
• URL: https://github.com/official-krish/zapier
• Owner: Official-Krish
• Created: 2024-08-06T11:26:05.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2024-08-28T21:44:21.000Z (4 months ago)
• Last Synced: 2024-08-28T22:57:13.416Z (4 months ago)
• Topics: kafka, nodejs, prisma, typescript
• Language: TypeScript
• Homepage:
• Size: 1.34 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: Readme.md

Awesome Lists containing this project

README

        
# Zapier Automation Processor

This project is a custom automation processor inspired by Zapier, designed to handle complex workflows efficiently using Kafka, Prisma, and TypeScript.

## 📁 Project Structure

  - **primary-backend**: Contains the main backend code of the project.

  - **frontend**: Contains the frontend code of the project.

  - **hooks**: Logic for handling various hooks in the automation process.

  - **processor**: Contains the processing logic for different tasks.

  - **worker**: Background tasks and job runners.

## 🛠️ Technologies Used

- **KafkaJS**: A modern Kafka client for Node.js.

- **Prisma**: An ORM for database management and migrations.

- **TypeScript**: Strongly typed programming language that builds on JavaScript.

- **Node.js**: JavaScript runtime built on Chrome's V8 engine.

## ⚙️ System Architecture

1. **zapier.com FE**: The frontend where users create and manage zaps (automation workflows).

2. **zapier.com BE**: The backend service that interacts with the frontend and handles API requests.

3. **Database (DB)**: Stores triggers and outbox data that need to be processed.

4. **Kafka**: Acts as a message broker to decouple the processing logic.

5. **Worker**: Consumes messages from Kafka and processes them, updating the DB as needed.

6. **Processor**: Handles the actual processing of each zap.

## 🚀 Getting Started

### Prerequisites

- Node.js (v14.x or later)

- Kafka (locally or through Docker)

- Prisma (installed globally or through npm)

Installation



    
Clone the repository:

        
git clone https://github.com/Official-Krish/Zapier



    

    
Install dependencies:

        
npm install

    

    
Set up your environment variables:

        
Create a .env file in the root directory with the following content:

        
DATABASE_URL=your_database_url

KAFKA_BROKERS=localhost:9092

TOPIC_NAME=zapier-events



    

    
Run Prisma migrations to set up the database schema:

        
npx prisma migrate dev

    

    
Start the Kafka consumer:

        
npm run start

    



Usage



    


Create a Zap: Use the frontend to create automation workflows.

    


Process Events: The worker will consume and process events as they are published to Kafka.



## Contributing

We welcome contributions from the community! To contribute, follow these steps:

1. Fork the repository.

2. Create a new branch (`git checkout -b feature/[feature-title]`).

3. Make your changes and commit them (`git commit -am 'Add brief meaningful commit message'`).

4. Push to the branch (`git push origin feature/[feature-title]`).

5. Create a new Pull Request.

For major changes, please open an issue first to discuss what you would like to change.