https://github.com/dagster-io/dagster-open-platform

Dagster Labs' open-source data platform, built with Dagster.
https://github.com/dagster-io/dagster-open-platform

dagster data-engineering python

Last synced: about 1 month ago
JSON representation

Dagster Labs' open-source data platform, built with Dagster.

• Host: GitHub
• URL: https://github.com/dagster-io/dagster-open-platform
• Owner: dagster-io
• Created: 2023-09-05T21:03:49.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-04-12T21:01:48.000Z (8 months ago)
• Last Synced: 2024-04-13T21:44:55.569Z (8 months ago)
• Topics: dagster, data-engineering, python
• Language: Python
• Homepage: https://www.dagster.io
• Size: 361 KB
• Stars: 218
• Watchers: 8
• Forks: 17
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• jimsghstars - dagster-io/dagster-open-platform - Dagster Labs' open-source data platform, built with Dagster. (Python)

README

        
# Dagster Open Platform

This repository contains the Dagster Open Platform (DOP) project.

## What is this?

Dagster Open Platform is Dagster Lab's open-source data platform.

This is a full-sized Dagster project that contains **real** assets that are used by the Dagster Labs team. These are assets used for our own analytics and operations. Therefore, if you're a high-growth startup or a budding data team, this should also serve as an amazing reference for what it means to run a data platform at a SaaS business.

We are open-sourcing these pipelines as an educational reference for how we do data engineering, build software, and run our analytics.

## How to use this project

Learning Dagster is not a journey that ends once your first assets are made. There are multiple resources available for you to get started with using Dagster, such as our documentation, quickstarts, and Dagster University. These efforts are amazing ways to get started using Dagster properly and flatten the early learning curve of using Dagster at your organization.

It follows our best practices for how to structure a Dagster project, how to connect with external systems, and how to use Dagster in a production setting.

*Note:* This project **does not** include any confidential information about Dagster, environment variables, or configurations for our pipelines. We also excluded assets that contain sensitive business logic. Therefore, this project won't be able to run on your machine without some additional changes.