Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/biggaji/pipeline-drill
A workflow pipeline queue engine
https://github.com/biggaji/pipeline-drill
Last synced: about 20 hours ago
JSON representation
A workflow pipeline queue engine
- Host: GitHub
- URL: https://github.com/biggaji/pipeline-drill
- Owner: biggaji
- License: mit
- Created: 2024-05-19T09:45:59.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2024-07-11T17:04:35.000Z (4 months ago)
- Last Synced: 2024-07-12T18:55:19.590Z (4 months ago)
- Language: TypeScript
- Size: 51.8 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# pipeline-drill
The project is a simple pipeline workflow engine. It uses a chained step list flow.
Below is a sample image of what I mean, just for you to get an idea. I plan to implement a simple UI to show
pipeline workflow in real-time, in the near future.
NOTE: Image was picked off the net.
Learn me explain the process:
There is a pipeline service class that manages the lifecycle of a pipeline, (i.e creating, starting,
restarting e.t.c), then there is also a steps registry which is an object that is used to map step names to
their corresponding executing function.
I will update this docs again soon.
## Prerequisites
Make sure you have Node.js and npm (or yarn) installed.
### Get started
1. Clone the repository:
```sh
git clone https://github.com/biggaji/pipeline-drill.git
cd pipeline-drill
```
2. Install the dependencies:
```sh
# Using npm:
npm install
# Using yarn
yarn install
```
3. Compile the Typescript code to Javascript:
```sh
# Using npm:
npm run compile
# Using yarn
yarn compile
```
4. Run the application:
```sh
# Using npm:
npm start
# Using yarn
yarn start
```
Technologies used:
- BullMQ for queue.
- MongoDB for database.
- Redis inconjuction with BullMQ
Thank you for sticking around.