https://github.com/erickwendel/parallelizing-nodejs-ops

Example of how to Migrate 1M items from MongoDB to Postgres in just a few minutes using Node.js child process
https://github.com/erickwendel/parallelizing-nodejs-ops

child-process docker docker-compose mongo mongodb nodejs performance pg postgres postgresql

Last synced: about 2 hours ago
JSON representation

Example of how to Migrate 1M items from MongoDB to Postgres in just a few minutes using Node.js child process

• Host: GitHub
• URL: https://github.com/erickwendel/parallelizing-nodejs-ops
• Owner: ErickWendel
• Created: 2024-04-17T19:11:15.000Z (8 months ago)
• Default Branch: main
• Last Pushed: 2024-05-29T20:35:21.000Z (7 months ago)
• Last Synced: 2024-12-17T05:51:48.494Z (8 days ago)
• Topics: child-process, docker, docker-compose, mongo, mongodb, nodejs, performance, pg, postgres, postgresql
• Language: JavaScript
• Homepage: https://youtu.be/EnK8-x8L9TY
• Size: 27.3 KB
• Stars: 159
• Watchers: 2
• Forks: 80
• Open Issues: 3
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Parallelizing Node.js operations with child process

Example of [how to Migrate 1M items from MongoDB to Postgres in just a few minutes](https://youtu.be/EnK8-x8L9TY) using Node.js child process

**First leave your star in the repo 🌟**

![Aumentando em 999x a velocidade de processamento de dados com Node](https://github.com/ErickWendel/parallelizing-nodejs-ops/assets/8060102/6974de93-7848-477a-9198-9d99dedc18f3)

## Running

You'll need to install Docker and Docker compose to be able to spin up the DBs instances, after that run:

- docker-compose up -d

- npm ci

- npm run seed

- npm start

## Errors?

In case you got an error of too many processes open, try decreasing the const [CLUSTER_SIZE](https://github.com/ErickWendel/parallelizing-nodejs-ops/blob/main/src/index.js#L8C1-L8C24) variable