https://github.com/mage-ai/mage-ai
🧙 Build, run, and manage data pipelines for integrating and transforming data.
https://github.com/mage-ai/mage-ai
artificial-intelligence data data-engineering data-integration data-pipelines data-science dbt elt etl machine-learning orchestration pipeline pipelines python reverse-etl spark sql transformation
Last synced: 19 days ago
JSON representation
🧙 Build, run, and manage data pipelines for integrating and transforming data.
- Host: GitHub
- URL: https://github.com/mage-ai/mage-ai
- Owner: mage-ai
- License: apache-2.0
- Created: 2022-05-16T22:11:39.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2025-05-05T17:49:38.000Z (9 months ago)
- Last Synced: 2025-05-05T22:17:23.362Z (9 months ago)
- Topics: artificial-intelligence, data, data-engineering, data-integration, data-pipelines, data-science, dbt, elt, etl, machine-learning, orchestration, pipeline, pipelines, python, reverse-etl, spark, sql, transformation
- Language: Python
- Homepage: https://www.mage.ai
- Size: 233 MB
- Stars: 8,300
- Watchers: 63
- Forks: 841
- Open Issues: 522
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing/backend/io/adding-a-class.mdx
- License: LICENSE
Awesome Lists containing this project
- awesome-starred - mage-ai/mage-ai - 🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data. (data-science)
- awesome - mage-ai/mage-ai - 🧙 Build, run, and manage data pipelines for integrating and transforming data. (Python)
- awesome - mage-ai/mage-ai - 🧙 Build, run, and manage data pipelines for integrating and transforming data. (<a name="Python"></a>Python)
README
# Mage OSS
### Build modern data pipelines locally — fast, visual, and production-ready.
Mage OSS is a self-hosted development environment designed to help teams create production-grade data pipelines with confidence.
Ideal for automating ETL tasks, architecting data flow, or orchestrating transformations — all in a fast, notebook-style interface powered by modular code.
When it’s time to scale, [Mage Pro](https://mage.ai) — our core platform — unlocks enterprise orchestration, collaboration, and AI-powered workflows.
## What you can do with Mage OSS
- Build pipelines locally with Python, SQL, or R in a modular notebook-style UI
- Run jobs manually or on a schedule (cron supported)
- Connect to databases, APIs, and cloud storage with prebuilt connectors
- Debug visually with logs, live previews, and step-by-step execution
- Set up quickly with Docker, pip, or conda — no cloud account required
- Your go-to workspace for local pipeline development — fully in your control.
## Start local. Scale when you're ready.
Use Mage OSS to build and run pipelines on your machine. When you're ready for advanced tooling, performance, and AI-assisted productivity, Mage Pro is just one click away.
[**Try Mage Pro free →**](https://mage.ai)
### Quickstart
Install using Docker (recommended):
```bash
docker pull mageai/mageai:latest
```
Or with pip:
```bash
pip install mage-ai
```
Or with conda:
```bash
conda install -c conda-forge mage-ai
```
Full setup guide and docs: [docs.mage.ai](https://docs.mage.ai/getting-started/setup#%E2%9B%B5%EF%B8%8F-mage-oss-overview)
## Core Features
| Feature | Description |
| :- | :- |
| Modular pipelines | Build pipelines block-by-block using Python, SQL, or R |
| Notebook UI | Interactive editor for writing and documenting logic |
| Data integrations | Prebuilt connectors to databases, APIs, and cloud storage |
| Scheduling | Trigger pipelines manually or on a schedule |
| Visual debugging | Step-by-step logs, data previews, and error handling |
| dbt support | Build and run dbt models directly inside Mage |
## Example Use Cases
- Move data from Google Sheets to Snowflake with a Python transform
- Schedule a daily SQL pipeline to clean and aggregate product data
- Develop dbt models in a visual notebook-style interface
- Run simple ETL/ELT jobs locally with full transparency
## Documentation
Looking for how-to guides, examples, or advanced configuration?
Explore our full documentation at [docs.mage.ai](https://docs.mage.ai).
## Contributing
We welcome contributions of all kinds — bug fixes, docs, new features, or community examples.
Start with our [contributing guide](https://docs.mage.ai/contributing/overview), check out open issues, or suggest improvements.
## Ready to scale? Mage Pro has you covered.
Mage Pro is a powered-up platform built for teams.
It adds everything you need for production pipelines, at scale.
- Magical AI-assisted development and debugging
- Multi-environment orchestration
- Role-based access control
- Real-time monitoring & alerts
- Powerful CI/CD & version control
- Powerful enterprise features
- Available fully managed, hybrid, or on-premises
[**Try Mage Pro free →**](https://mage.ai)