https://github.com/bruin-data/bruin

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
https://github.com/bruin-data/bruin

analytics bigquery data-analysis data-modeling data-pipelines data-transformation python snowflake sql

Last synced: 1 day ago
JSON representation

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

• Host: GitHub
• URL: https://github.com/bruin-data/bruin
• Owner: bruin-data
• License: apache-2.0
• Created: 2023-08-03T12:32:10.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-10-28T16:00:17.000Z (4 months ago)
• Last Synced: 2024-10-28T16:10:45.209Z (4 months ago)
• Topics: analytics, bigquery, data-analysis, data-modeling, data-pipelines, data-transformation, python, snowflake, sql
• Language: Python
• Homepage: https://bruin-data.github.io/bruin/
• Size: 27 MB
• Stars: 66
• Watchers: 5
• Forks: 6
• Open Issues: 10
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.md
  • Security: SECURITY.md

Awesome Lists containing this project

README

        


  



Bruin is a data pipeline tool that brings together data ingestion, data transformation with SQL & Python, and data quality into a single framework. It works with all the major data platforms and runs on your local machine, an EC2 instance, or GitHub Actions.



Bruin is packed with features:

- 📥 ingest data with [ingestr](https://github.com/bruin-data/ingestr) / Python

- ✨ run SQL & Python transformations on [many platforms](https://bruin-data.github.io/bruin/#supported-platforms)

- 📐 table/view [materializations](https://bruin-data.github.io/bruin/assets/materialization.html), incremental tables

- 🐍 run Python in isolated environments using [uv](https://github.com/astral-sh/uv)

- 💅 built-in data quality checks

- 🚀 Jinja templating to avoid repetition

- ✅ validate pipelines end-to-end via dry-run

- 👷 run on your local machine, an EC2 instance, or [GitHub Actions](https://bruin-data.github.io/bruin/cicd/github-action.html)

- 🔒 secrets injection via environment variables

- [VS Code extension](https://bruin-data.github.io/bruin/vscode-extension/overview.html) for a better developer experience

- ⚡ written in Golang

- 📦 [easy to install](https://bruin-data.github.io/bruin/getting-started/introduction/installation.html) and use

## Installation

Please see the installation instructions [here](https://bruin-data.github.io/bruin/getting-started/introduction/installation.html).

## Community

Join our Slack community [here](https://join.slack.com/t/bruindatacommunity/shared_invite/zt-2dl2i8foy-bVsuMUauHeN9M2laVm3ZVg).



  

    

  



## Quickstart

Take a look at our quickstart guide [here](https://bruin-data.github.io/bruin/getting-started/introduction/quickstart.html).